GNOME Bugzilla – Bug 710932
g_str_tokenize_and_fold() does not play well with characters that can't be ascii-ified
Last modified: 2018-05-24 15:45:57 UTC
When I do g_str_tokenize_and_fold("礼思朱", NULL, &alternates); then alternates is ["", NULL]. The empty string is because it can't ascii-ify any of the string characters. I think if at least one char of a word can't be ascii-ified then the whole word should be omitted from alternates.
Reading the implementation, I think it deserve a bit more useful comments: 1) /* TODO: Invent time machine. Converse with Mustafa Ataturk... */ --> that does not help understanding what the code does... Why are those 2 unicodes special? If they are really special cases, I think it deserve a specific unit test. 2) g_new0 (gchar *, 0 + 1); and g_new (gchar *, n + 1); --> It is gchar ** actually. It know the sizeof is the same in any case, but that would help understanding that we are building strv IMO. 3) ~decomposed[k] & 0x80 --> where does that magical number comes from? I guess it means "decomposed[k] is an ascii" but I think it should be told in a comment.
Created attachment 258191 [details] [review] Change implementation and API of g_str_tokenize_and_fold() The new code is what Nokia N900 and Empathy are using. It has been wildly tested. It avoid iterating over the string multiple times. It change the assumption that searching for "fré" does not want to match "fre", which is I think better. If I search for "fré" in my contacts I want "frederic@example.com" to match.
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/glib/issues/768.