GNOME Bugzilla – Bug 734886
g_locale_from_utf8 on Windows fails to handle code points above U+0100
Last modified: 2018-05-24 16:55:30 UTC
Test program: https://gist.github.com/devinacker/cd09eb2ab4608b3d90f1 (compiled as UTF-8 using MinGW) In the above test program (as well as Windows builds of HexChat, which is how I originally discovered the issue), attempting to call g_locale_from_utf8 with a string containing only Unicode code points below U+0100 returns an appropriately translated string, while higher code points (such as the Japanese text in the test program) result in the error "Invalid byte sequence in conversion input", even when the actual UTF-8 byte sequence appears to be completely valid. I have tested this repeatedly with various combinations of non-ASCII characters both above and below U+0100 and gotten the same results, on both Windows 7 and 8, with the system locale set to English (United States).
Hello Devin, I didn't get the "Invalid byte sequence in conversion input" when I built the test program with Visual Studio 2008 Express. Did you save the file in UTF-8 format, without BOM? With blessings.
I can reproduce this as well (Glib 2.45.2 on Win7 using MinGW), although not all code points above U+100 cause the error. For example š (U+161) converts just fine.
Created attachment 308638 [details] Testcase for characters U+00C8 - U+01FF Attached program tries to convert each character from U+00C8 to U+01FF and shows either the result (which will probably display incorrectly, but that's unimportant) or an error message. From U+0100 onwards, most of the conversions end up with 'Invalid byte sequence in conversion input', with exception of a few: U+0152, U+0153, U+0160, U+0161, U+0178, U+017D, U+017E, U+0192. Again, this is on Win7 with English locale settings, running Glib 2.45.2 compiled with MinGW.
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/glib/issues/916.