Docs: added more detailed information about UTF-8 encoding.

+ Revert mistakenly committed Win32+DX11 main.cpp from last commit.

docs/FAQ.md

@@ -628,10 +628,11 @@ builder.BuildRanges(&ranges);                          // Build the final result
 io.Fonts->AddFontFromFileTTF("myfontfile.ttf", 16.0f, nullptr, ranges.Data);
 ```
 
-All your strings need to use UTF-8 encoding. In C++11 you can encode a string literal in UTF-8
-by using the u8"hello" syntax. Specifying literal in your source code using a local code page
-(such as CP-923 for Japanese or CP-1251 for Cyrillic) will NOT work!
-Otherwise, you can convert yourself to UTF-8 or load text data from a file already saved as UTF-8.
+All your strings need to use UTF-8 encoding.
+You need to tell your compiler to use UTF-8, or in C++11 you can encode a string literal in UTF-8 by using the u8"hello" syntax.
+Specifying literal in your source code using a local code page (such as CP-923 for Japanese or CP-1251 for Cyrillic) will NOT work!
+See [About UTF-8 Encoding](https://github.com/ocornut/imgui/blob/master/docs/FONTS.md#about-utf-8-encoding) section
+of [FONTS.md](https://github.com/ocornut/imgui/blob/master/docs/FONTS.md) for details about UTF-8 Encoding.
 
 Text input: it is up to your application to pass the right character code by calling `io.AddInputCharacter()`.
 The applications in examples/ are doing that.

docs/FONTS.md

@@ -12,41 +12,118 @@ In the [misc/fonts/](https://github.com/ocornut/imgui/tree/master/misc/fonts) fo
 
 ## Index
 - [Readme First](#readme-first)
+- [About Filenames](#about-filenames)
+- [About UTF-8 Encoding](#about-utf-8-encoding)
+- [Debug Tools](#debug-tools)
 - [How should I handle DPI in my application?](#how-should-i-handle-dpi-in-my-application)
-- [Fonts Loading Instructions](#font-loading-instructions)
+- [Fonts Loading Instructions](#fonts-loading-instructions)
 - [Using Icon Fonts](#using-icon-fonts)
 - [Using FreeType Rasterizer (imgui_freetype)](#using-freetype-rasterizer-imgui_freetype)
 - [Using Colorful Glyphs/Emojis](#using-colorful-glyphsemojis)
 - [Using Custom Glyph Ranges](#using-custom-glyph-ranges)
 - [Using Custom Colorful Icons](#using-custom-colorful-icons)
 - [Using Font Data Embedded In Source Code](#using-font-data-embedded-in-source-code)
-- [About filenames](#about-filenames)
 - [Credits/Licenses For Fonts Included In Repository](#creditslicenses-for-fonts-included-in-repository)
 - [Font Links](#font-links)
 
 ---------------------------------------
- ## Readme First
 
-- You can use the `Metrics/Debugger` window (available in `Demo>Tools`) to browse your fonts and understand what's going on if you have an issue. You can also reach it in `Demo->Tools->Style Editor->Fonts`. The same information are also available in the Style Editor under Fonts.
+## Readme First
 
-![Fonts debugging](https://user-images.githubusercontent.com/8225057/135429892-0e41ef8d-33c5-4991-bcf6-f997a0bcfd6b.png)
+**A vast majority of font and text related issues encountered comes from 3 things:**
+- Invalid filename due to use of `\` or unexpected working directory. See [About Filenames](#about-filenames). AddFontXXX functions should assert if the filename is incorrect.
+- Invalid UTF-8 encoding of your non-ASCII strings. See [About UTF-8 Encoding](#about-utf-8-encoding). Use the encoding viewer to confirm yours is correct.
+- You need to load a font with explicit glyph ranges if you want to use non-ASCII characters. See [Fonts Loading Instructions](#fonts-loading-instructions). Use Metrics/Debugger->Fonts to confirm loaded fonts and loaded glyph ranges.
 
-- You can use the `UTF-8 Encoding viewer` in `Metrics/Debugger` to verify the content of your UTF-8 strings. From C/C++ code, you can call `ImGui::DebugTextEncoding("my string");` function to verify that your UTF-8 encoding is correct.
+The third point is a current constraint of Dear ImGui (which we will lift in the future): when loading a font you need to specify which characters glyphs to load.
+All loaded fonts glyphs are rendered into a single texture atlas ahead of time. Calling either of `io.Fonts->GetTexDataAsAlpha8()`, `io.Fonts->GetTexDataAsRGBA32()` or `io.Fonts->Build()` will build the atlas. This is generally called by the Renderer backend, e.g. `ImGui_ImplDX11_NewFrame()` calls it.
 
-![UTF-8 Encoding viewer](https://user-images.githubusercontent.com/8225057/166505963-8a0d7899-8ee8-4558-abb2-1ae523dc02f9.png)
+**If you use custom glyphs ranges, make sure the array is persistent** and available during the calls to `GetTexDataAsAlpha8()/GetTexDataAsRGBA32()/Build()`.
+
+##### [Return to Index](#index)
 
-- All loaded fonts glyphs are rendered into a single texture atlas ahead of time. Calling either of `io.Fonts->GetTexDataAsAlpha8()`, `io.Fonts->GetTexDataAsRGBA32()` or `io.Fonts->Build()` will build the atlas.
+## About Filenames
 
-- Make sure your font ranges data are persistent (available during the calls to `GetTexDataAsAlpha8()`/`GetTexDataAsRGBA32()/`Build()`.
+**Please note that many new C/C++ users have issues loading their files _because the filename they provide is wrong_ due to incorrect assumption of what is the current directory.**
 
-- Use C++11 u8"my text" syntax to encode literal strings as UTF-8. e.g.:
+Two things to watch for:
+
+(1) In C/C++ and most programming languages if you want to use a backslash `\` within a string literal, you need to write it double backslash `\\`. At it happens, Windows uses backslashes as a path separator, so be mindful.
 ```cpp
-u8"hello"
-u8"こんにちは"   // this will be encoded as UTF-8
+io.Fonts->AddFontFromFileTTF("MyFiles\MyImage01.jpg", ...);   // This is INCORRECT!!
+io.Fonts->AddFontFromFileTTF("MyFiles\\MyImage01.jpg", ...);  // This is CORRECT
 ```
+In some situations, you may also use `/` path separator under Windows.
 
+(2) Make sure your IDE/debugger settings starts your executable from the right working (current) directory. In Visual Studio you can change your working directory in project `Properties > General > Debugging > Working Directory`. People assume that their execution will start from the root folder of the project, where by default it often starts from the folder where object or executable files are stored.
+```cpp
+io.Fonts->AddFontFromFileTTF("MyImage01.jpg", ...);       // Relative filename depends on your Working Directory when running your program!
+io.Fonts->AddFontFromFileTTF("../MyImage01.jpg", ...);    // Load from the parent folder of your Working Directory
+```
 ##### [Return to Index](#index)
 
+
+## About UTF-8 Encoding
+
+**For non-ASCII characters display, a common user issue is not passing correctly UTF-8 encoded strings.**
+
+(1) We provide a function `ImGui::DebugTextEncoding(const char* text)` which you can call to verify the content of your UTF-8 strings.
+This is a convenient way to confirm that your encoding is correct.
+
+```cpp
+ImGui::SeparatorText("CORRECT");
+ImGui::DebugTextEncoding(u8"こんにちは");
+
+ImGui::SeparatorText("INCORRECT");
+ImGui::DebugTextEncoding("こんにちは");
+```
+![UTF-8 Encoding viewer](https://github.com/ocornut/imgui/assets/8225057/61c1696a-9a94-46c5-9627-cf91211111f0)
+
+You can also find this tool under `Metrics/Debuggers->Tools->UTF-8 Encoding viewer` if you want to paste from clipboard, but this won't validate the UTF-8 encoding done by your compiler.
+
+(2) To encode in UTF-8:
+
+There are also compiler-specific ways to enforce UTF-8 encoding by default:
+
+- Visual Studio compiler: `/utf-8` command-line flag.
+- Visual Studio compiler: `#pragma execution_character_set("utf-8")` inside your code.
+- Since May 2023 we have changed the Visual Studio projects of all our examples to use `/utf-8` ([see commit](https://github.com/ocornut/imgui/commit/513af1efc9080857bbd10000d98f98f2a0c96803)).
+
+Or, since C++11, you can use the `u8"my text"` syntax to encode literal strings as UTF-8. e.g.:
+```cpp
+ImGui::Text(u8"hello");
+ImGui::Text(u8"こんにちは");   // this will always be encoded as UTF-8
+ImGui::Text("こんにちは");     // the encoding of this is depending on compiler settings/flags and may be incorrect.
+```
+
+Since C++20, because the C++ committee hate its users, they decided to change the `u8""` syntax to not return `const char*` but a new type `const char_t*` which doesn't cast to `const char*`.
+Because of type usage of `u8""` in C++20 is a little more tedious:
+```cpp
+ImGui::Text((const char*)u8"こんにちは");
+```
+We suggest using a macro in your codebase:
+```cpp
+#define U8(_S)    (const char*)u8##_S
+ImGui::Text(U8("こんにちは"));
+```
+##### [Return to Index](#index)
+
+
+## Debug Tools
+
+#### Metrics/Debugger->Fonts
+You can use the `Metrics/Debugger` window (available in `Demo>Tools`) to browse your fonts and understand what's going on if you have an issue. You can also reach it in `Demo->Tools->Style Editor->Fonts`. The same information are also available in the Style Editor under Fonts.
+
+![Fonts debugging](https://user-images.githubusercontent.com/8225057/135429892-0e41ef8d-33c5-4991-bcf6-f997a0bcfd6b.png)
+
+#### UTF-8 Encoding Viewer**
+You can use the `UTF-8 Encoding viewer` in `Metrics/Debugger` to verify the content of your UTF-8 strings. From C/C++ code, you can call `ImGui::DebugTextEncoding("my string");` function to verify that your UTF-8 encoding is correct.
+
+![UTF-8 Encoding viewer](https://user-images.githubusercontent.com/8225057/166505963-8a0d7899-8ee8-4558-abb2-1ae523dc02f9.png)
+
+##### [Return to Index](#index)
+
+
 ## How should I handle DPI in my application?
 
 See [FAQ entry](https://github.com/ocornut/imgui/blob/master/docs/FAQ.md#q-how-should-i-handle-dpi-in-my-application).
@@ -54,7 +131,7 @@ See [FAQ entry](https://github.com/ocornut/imgui/blob/master/docs/FAQ.md#q-how-s
 ##### [Return to Index](#index)
 
 
-## Font Loading Instructions
+## Fonts Loading Instructions
 
 **Load default font:**
 ```cpp
@@ -62,7 +139,6 @@ ImGuiIO& io = ImGui::GetIO();
 io.Fonts->AddFontDefault();
 ```
 
-
 **Load .TTF/.OTF file with:**
 ```cpp
 ImGuiIO& io = ImGui::GetIO();
@@ -70,7 +146,6 @@ io.Fonts->AddFontFromFileTTF("font.ttf", size_pixels);
 ```
 If you get an assert stating "Could not load font file!", your font filename is likely incorrect. Read "[About filenames](#about-filenames)" carefully.
 
-
 **Load multiple fonts:**
 ```cpp
 // Init
@@ -86,7 +161,6 @@ ImGui::Text("Hello with another font");
 ImGui::PopFont();
 ```
 
-
 **For advanced options create a ImFontConfig structure and pass it to the AddFont() function (it will be copied internally):**
 ```cpp
 ImFontConfig config;
@@ -96,7 +170,6 @@ config.GlyphExtraSpacing.x = 1.0f;
 ImFont* font = io.Fonts->AddFontFromFileTTF("font.ttf", size_pixels, &config);
 ```
 
-
 **Combine multiple fonts into one:**
 ```cpp
 // Load a first font
@@ -311,28 +384,6 @@ ImFont* font = io.Fonts->AddFontFromMemoryCompressedBase85TTF(compressed_data_ba
 
 ##### [Return to Index](#index)
 
-## About filenames
-
-**Please note that many new C/C++ users have issues loading their files _because the filename they provide is wrong_.**
-
-Two things to watch for:
-- Make sure your IDE/debugger settings starts your executable from the right working directory. In Visual Studio you can change your working directory in project `Properties > General > Debugging > Working Directory`. People assume that their execution will start from the root folder of the project, where by default it often starts from the folder where object or executable files are stored.
-```cpp
-// Relative filename depends on your Working Directory when running your program!
-io.Fonts->AddFontFromFileTTF("MyImage01.jpg", ...);
-
-// Load from the parent folder of your Working Directory
-io.Fonts->AddFontFromFileTTF("../MyImage01.jpg", ...);
-```
-- In C/C++ and most programming languages if you want to use a backslash `\` within a string literal, you need to write it double backslash `\\`. At it happens, Windows uses backslashes as a path separator, so be mindful.
-```cpp
-io.Fonts->AddFontFromFileTTF("MyFiles\MyImage01.jpg", ...);   // This is INCORRECT!!
-io.Fonts->AddFontFromFileTTF("MyFiles\\MyImage01.jpg", ...);  // This is CORRECT
-```
-In some situations, you may also use `/` path separator under Windows.
-
-##### [Return to Index](#index)
-
 ## Credits/Licenses For Fonts Included In Repository
 
 Some fonts files are available in the `misc/fonts/` folder:

examples/example_win32_directx11/main.cpp

@@ -1,4 +1,4 @@
-// Dear ImGui: standalone example application for DirectX 11
+// Dear ImGui: standalone example application for DirectX 11
 // If you are new to Dear ImGui, read documentation from the docs/ folder + read the top of imgui.cpp.
 // Read online: https://github.com/ocornut/imgui/tree/master/docs
 
@@ -26,7 +26,7 @@ LRESULT WINAPI WndProc(HWND hWnd, UINT msg, WPARAM wParam, LPARAM lParam);
 int main(int, char**)
 {
     // Create application window
-    ImGui_ImplWin32_EnableDpiAwareness();
+    //ImGui_ImplWin32_EnableDpiAwareness();
     WNDCLASSEXW wc = { sizeof(wc), CS_CLASSDC, WndProc, 0L, 0L, GetModuleHandle(nullptr), nullptr, nullptr, nullptr, nullptr, L"ImGui Example", nullptr };
     ::RegisterClassExW(&wc);
     HWND hwnd = ::CreateWindowW(wc.lpszClassName, L"Dear ImGui DirectX11 Example", WS_OVERLAPPEDWINDOW, 100, 100, 1280, 800, nullptr, nullptr, wc.hInstance, nullptr);
@@ -71,7 +71,7 @@ int main(int, char**)
     //io.Fonts->AddFontFromFileTTF("../../misc/fonts/DroidSans.ttf", 16.0f);
     //io.Fonts->AddFontFromFileTTF("../../misc/fonts/Roboto-Medium.ttf", 16.0f);
     //io.Fonts->AddFontFromFileTTF("../../misc/fonts/Cousine-Regular.ttf", 15.0f);
-    ImFont* font = io.Fonts->AddFontFromFileTTF("c:\\Windows\\Fonts\\ArialUni.ttf", 18.0f, nullptr, io.Fonts->GetGlyphRangesJapanese());
+    //ImFont* font = io.Fonts->AddFontFromFileTTF("c:\\Windows\\Fonts\\ArialUni.ttf", 18.0f, nullptr, io.Fonts->GetGlyphRangesJapanese());
     //IM_ASSERT(font != nullptr);
 
     // Our state
@@ -114,12 +114,6 @@ int main(int, char**)
         if (show_demo_window)
             ImGui::ShowDemoWindow(&show_demo_window);
 
-        ImGui::SeparatorText("CORRECT");
-        ImGui::DebugTextEncoding(u8"こんにちは");
-
-        ImGui::SeparatorText("INCORRECT");
-        ImGui::DebugTextEncoding("こんにちは");
-
         // 2. Show a simple window that we create ourselves. We use a Begin/End pair to create a named window.
         {
             static float f = 0.0f;
@@ -127,13 +121,6 @@ int main(int, char**)
 
             ImGui::Begin("Hello, world!");                          // Create a window called "Hello, world!" and append into it.
 
-#define U8(_S)    (const char*)u8##_S
-            ImGui::Text(U8("こんにちは"));
-
-//#pragma execution_character_set("utf-8")
-
-            ImGui::Text("日本語");
-
             ImGui::Text("This is some useful text.");               // Display some text (you can use a format strings too)
             ImGui::Checkbox("Demo Window", &show_demo_window);      // Edit bools storing our window open/close state
             ImGui::Checkbox("Another Window", &show_another_window);

imgui.cpp

@@ -13394,7 +13394,7 @@ void ImGui::DebugRenderKeyboardPreview(ImDrawList* draw_list)
 void ImGui::DebugTextEncoding(const char* str)
 {
     Text("Text: \"%s\"", str);
-    if (!BeginTable("list", 4, ImGuiTableFlags_Borders | ImGuiTableFlags_RowBg | ImGuiTableFlags_SizingFixedFit))
+    if (!BeginTable("##DebugTextEncoding", 4, ImGuiTableFlags_Borders | ImGuiTableFlags_RowBg | ImGuiTableFlags_SizingFixedFit | ImGuiTableFlags_Resizable))
         return;
     TableSetupColumn("Offset");
     TableSetupColumn("UTF-8");

imgui.h

@@ -2832,7 +2832,8 @@ struct ImFontAtlas
     //-------------------------------------------
 
     // Helpers to retrieve list of common Unicode ranges (2 value per range, values are inclusive, zero-terminated list)
-    // NB: Make sure that your string are UTF-8 and NOT in your local code page. In C++11, you can create UTF-8 string literal using the u8"Hello world" syntax. See FAQ for details.
+    // NB: Make sure that your string are UTF-8 and NOT in your local code page.
+    // Read https://github.com/ocornut/imgui/blob/master/docs/FONTS.md/#about-utf-8-encoding for details.
     // NB: Consider using ImFontGlyphRangesBuilder to build glyph ranges from textual data.
     IMGUI_API const ImWchar*    GetGlyphRangesDefault();                // Basic Latin, Extended Latin
     IMGUI_API const ImWchar*    GetGlyphRangesGreek();                  // Default + Greek and Coptic