GNOME Bugzilla – Bug 696624
Japanese (Korean/Chinese) XIM input mode indicator has not been working under linux. (with known solution)
Last modified: 2017-10-21 18:11:44 UTC
Created attachment 239854 [details] [review] Reverting on_status_window_expose_event() to an old working code in gtkimcontextxim.c (I posted a slightly edited article to mozilla's bugzilla, too, since I noticed the problem with Thunderbird and Firefox, and verified the fix. https://bugzilla.mozilla.org/show_bug.cgi?id=854825 Bug 854825 - Japanese (Korean/Chinese) XIM input mode indicator has not been working for TB and FF under linux. (with known solution) ) I hope someone in the know can fix the issue in 2.2x.y series of gtk+ library. Here is the bug and a suggested fix. I am using Thunderbird (TB) 17.0.4 under Debian GNU/Linux. For like 8 months, I have noticed that the input mode indicator for Japanese input that uses XIM protocol (when I use kinput2 for example) is not working right when I use TB to write Japanese messages and such. (And for that matter, this is the same with mozilla's Firefox (FF) browser, but I don't write lengthy message in Japanese using FF so often). The symptom is like this. Before, when all was well, a small rectangle at the lower-left corner of TB (or FF) window, attached as an extra small rectangle [not inside], showed Japanese input mode that uses XIM protocol by using some Japanese characters to indicate the mode (and I assume Korean or Chinese characters for people who use XIM input methods to type in these characters). The character was in black and the background rectangle was white. But for like 8 months, the programs, both TB and FF, have shown only black rectangle and no characters are visible at all there. The lack of visible input mode is sometimes very irritating because we can't tell which characters Hiragana, Katakana, we will type in. (Or if I make a typing mistake and that is why unintended character showed up instead of using the incorrect mode, etc.) This problem began happening after an upgrade of Debian packages last summer and so I suspect that it may have something to do with the X-related libraries, but I could not figure out exactly why until yesterday. (I suppose ordinary users can't figure out the cause of such a problem even though the symptom is easy to describe.) In a post to a Japanese BBS, I found a mention by an anonymous poster that explains this as follows: There seems to have been a problem in GTK2 library. Starting in gtk2 2.21.6, function on_status_window_expose_event() in modules/input/gtkimcontextxim.c has been changed (to the worse. I think it means buggy?) If we revert the function, on_status_window_expose_event(), to a previous code found in 2.21.5 (or earlier), all is well. We can run make below modules/input directory only, and use im-xim.so thus produced to fix the problem, etc. This was posted July 5th, 2012, last year. I wish I had known this earlier. I have checked this out and this indeed fixes the problem for me (!). I believe this behavior affects the usability of XIM input users very much. Some users can do the local fix, or ask an admin-like help to do this for them. I am attaching a simple patch that should work for many 2.2x.y series. [ looking at the code, I realize that maybe through GNOME desktop setting if I change the base default color to white or something, I should be able to regain the white rectangle where the XIM input mode is indicated by characters (?) without library swap. (But gnome desktop setting change by gconf editor and the like is so confusing to an ordinary user. I experimented for a while and gave up.)] This problem should not have happened without such gnome setting IMHO. The anonymous poster didn't mention the name of his linux distribution, but I suspect this problem may have been observed by many XIM input users of East Asian languages, under various linux distributions, for some time. I believe he/she felt the same and posted the solution that worked to a large BBS here in Japan. I have no idea how he/she figured out the cause of the problem. I am attaching a suggested patch here. It may not go well with the current maintainer because I am using old code there. But then please feel free to make it work using the newer API functions. TIA WHAT I DID exactly: My Debian seems to use GTK 2.24.10 from unstable release (from reading /usr/share/doc/libgtk2.0-dev/changelog.Debian.gz and so I downloaded 2.24.10 source from gtk.org repository, and ran configure. Then I patched modules/input/gtkimcontextxim.c as was explained by the anonymous poster. [Reverted the code for the particular function to an earlier one.] I ran make at the top-most directory. Just trying to run make below modules/input doesn't work. The library compilation seems to need an upper-level library or something in the first place. You need to run this whole make once anyway. After a while the compilation finishes. The library seems to be created here in gtk2+ tree. ./modules/input/.libs/im-xim.so So I saved it to easy place to pick up later. cp ./modules/input/.libs/im-xim.so /tmp Now, where is the system-supplied im-xim.so located? Under Debian GNU/Linux this is where: (Other linux distributions will use different locations.) $ mlocate im-xim.so /usr/lib/i386-linux-gnu/gtk-2.0/2.10.0/immodules/im-xim.so /usr/lib/i386-linux-gnu/gtk-3.0/3.0.0/immodules/im-xim.so Oh my.? Do i use gtk-3.0 ? and not even 2.24.10? Anyway I tried replacing the gtk-2.0 version of im-xim.so. (I think TB is using gtk-2.0 version right now : at least TB 17.0.2 seems to do so.) Before I did the following, I quit TB and a few other programs just in case. # cd /usr/lib/i386-linux-gnu/gtk-2.0/2.10.0/immodules I replaced im-xim.so with the one I created using the downloaded and patched gtk2 source. # cp im-xim.so im-xim.so.BACKUP (just in case) # mv /tmp/im-xim.so ./ (replaced from the newly created one.) I restarted TB, and voila, now I can get the XIM indicator (Japanese character) in white background as before!!! BTW, trying to link in only im-xim.so using LD_PRELOAD=the-path-to-new-im-xim.so didn't work. During loading it complained something about missing module. I see the following message on the console, and Japanese input did not work at all. Gtk-Message: Failed to load module "canberra-gtk-module" Hope this helps XIM input users who are missing XIM input mode status indicator. PS: the code analysis. I looked at the code repository at http://www.gtk.org/. The previous code for gtk2 (2.21.5 or older) paints the whole rectangle area for the XIM mode indicator with a couple of different colors (one in full size, the other in a size which is one pixel smaller to draw the border, I think.). https://git.gnome.org/browse/gtk+/tree/modules/input/gtkimcontextxim.c?id=2.21.5 /* Draw the background (normally white) and border for the status window */ static gboolean on_status_window_expose_event (GtkWidget *widget, GdkEventExpose *event) { gdk_draw_rectangle (widget->window, widget->style->base_gc [GTK_STATE_NORMAL], TRUE, 0, 0, widget->allocation.width, widget->allocation.height); gdk_draw_rectangle (widget->window, widget->style->text_gc [GTK_STATE_NORMAL], FALSE, 0, 0, widget->allocation.width - 1, widget->allocation.height - 1); return FALSE; } Please note the difference of the third argument to gdk_draw_rectangle(): TRUE for the first invocation and FALSE for the second. Third argument controlls the "filling" of graphics operation. I think the first one fills the rectangle with the base color (i.e., the default background, which is white on my desktop.) The new code quoted below uses a different graphics library API, but paints the two areas (the sizes being different by one pixel) in a pair of colors, which I think unfortunately corresponds to BLACK, in my desktop setting and others (as on the desktop of the anonymous poster). https://git.gnome.org/browse/gtk+/tree/modules/input/gtkimcontextxim.c?id=2.24.17 /* Draw the background (normally white) and border for the status window */ static gboolean on_status_window_expose_event (GtkWidget *widget, GdkEventExpose *event) { cairo_t *cr; cr = gdk_cairo_create (widget->window); gdk_cairo_set_source_color (cr, &widget->style->base[GTK_STATE_NORMAL]); cairo_rectangle (cr, 0, 0, widget->allocation.width, widget->allocation.height); cairo_fill (cr); gdk_cairo_set_source_color (cr, &widget->style->text[GTK_STATE_NORMAL]); cairo_rectangle (cr, 0, 0, widget->allocation.width - 1, widget->allocation.height - 1); cairo_fill (cr); return FALSE; } One of the two instances of the color in the new routine ought to be changed to white or something, but I am not sure how that can be done. (In the old code, the color is not directly specified, but the two are complementary colors (base_gc[GTK_STYLE_NORMAL] vs text_gc[GTK_STYLE_NORMAL]) and is probably tied to the proper background color of default character drawing. It is not entirely clear how the color in new API should be set. It is now base[GTK_STYLE_NORMAL] vs text[GTK_STYLE_NORMAL] I believe the use of same color parameter won't draw the border properly, and if the color happens to be color of the character drawing (black), the characters for indicating mode won't be readable. As I was writing the above paragraph I realize that maybe there is a way to change the esoteric GNOME setting to make sure the colors thus painted (base[GTK_STYLE_NORMAL] and text[GTK_STYLE_NORMAL]) are made different and preferably the base color be white as it should be. But again, it was not clear to an ordinary user how to do this, and to make sure such setting doesn't interfere with the operation of desktop as a whole, maand so I simply tried the modification of the function by reverting to the old code and it worked. Thank you again for making this great gtk libraies and gnome desktop to the world community
Created attachment 239950 [details] The screen capture of the buggy XIM mode indicator Since the description may not be clear to those who don't use XIM input method, I am attaching two screen captures, one that show what the problematic behavior is, and the next one that shows the fixed expected correct behavior. TIA
Created attachment 239951 [details] Fixed (correct, expected) XIM mode indicator. The screen that shows the corrected behavior. I need to explain a few things. - I am using VMPlayer to catpure the screen image. - I am using IceWeasel, a version of Firefox compiled by Debian. The symptom are the same with FF.
The following patch was responsible for the undesirable change. I am not sure why the new code doesn't work as expected. It seems to me that the desktop environmental setting ought to follow the change so that base[] and text[] would have desirable color(s), but never was committed on Debian GNU/Linux release (?) However, Since reverting the patch (basically that is what I did ) makes the code work again as expected, there is no argueing in my mind from the viewpoint of users. I am afraid that XIM-related code has not seen much scrutiny because of the relative lack of user feedback (the user population may not be that large). https://git.gnome.org/browse/gtk+/commit/modules/input/gtkimcontextxim.c?id=7f1801eae10b67fb8878eb562ed3614514b96097 commit 3cbdc7ac89e58432c8d92570d58463747c9cd946 Author: Benjamin Otte <otte@redhat.com> Date: Sat Jul 17 04:41:58 2010 +0200 imcontextxim: Draw with Cairo modules/input/gtkimcontextxim.c | 25 +++++++++++++++---------- 1 files changed, 15 insertions(+), 10 deletions(-)
Created attachment 240020 [details] [review] Possible fix using the new APIs After looking at the new code (which was buggy), I realize that if I swap the colors of two drawings and shrink the size of the second rectangle, I get the desired effect more or less. Here is the patch to do this, and the next captured image shows the XIM mode box after this fix. Since this fix uses the newer cairo drawing primitives and I merely changed the applied colors (swapped the order), and shrunk the size of one of the rectangles, it is probably easier to accept and merge into 2.2x.y code. TIA
Created attachment 240021 [details] working XIM mode indicator with border line. This is how the new patch shows the XIM mode indicator.
Just wanted to report that the latest gtk+-2.24.18 in GTK2+ series still has the same problem. I noticed this when I upgraded packages on my Debian GNU/Linux a few days ago. Debian packages gtk-2,24,18 in testing repository. The same patch I submitted before fixed the this issue again. (except that compiling gtk+-2.24.18 under Debian is complicated. We need to use Debian patches and use newer packaging tools, etc. Basically, it is best to invoke debian/build in the top-level directory after obtaining gtk+2.0_2.24.18-1.debian.tar.gz from Debian package repository and expand it over the existing gtk+2.24.18 directory. Run debian/build THEN IGNORE the warnings and possible error(s) from an indirectly invoked program (you may need to stop (^C) debian/build whenr it seems to stop for no reason after printing error messages), and FINALLY manually run ./genmake.sh to create Makefiles. THEN type make to create a binary.) TIA
I have no idea why this bug remained unconfirmed because I uploaded screen dumps, etc. (I have heard the original fix from someone else (anonymous in a BBS) and so the problem is wide-spread at least among the XIM input method users in the East Asia. Anyway, the problem still persists in gtk 3 series, and the code in question seems really broken somehow. I have no idea how the code is broken that way, and am investigating. The code in question in gtk 3 is the following. https://git.gnome.org/browse/gtk+/tree/modules/input/gtkimcontextxim.c#n1707 /* Draw the background (normally white) and border for the status window */ static gboolean on_status_window_draw (GtkWidget *widget, cairo_t *cr) { GtkStyleContext *style; GdkRGBA color; style = gtk_widget_get_style_context (widget); gtk_style_context_get_background_color (style, 0, &color); gdk_cairo_set_source_rgba (cr, &color); cairo_paint (cr); gtk_style_context_get_color (style, 0, &color); gdk_cairo_set_source_rgba (cr, &color); cairo_paint (cr); cairo_rectangle (cr, 0, 0, gtk_widget_get_allocated_width (widget) - 1, gtk_widget_get_allocated_height (widget) - 1); cairo_fill (cr); return FALSE; } Please notice there is ONE cairo_rectanble() call missing somehow. (Otherwise the setting of cairo_paint(cr) in the first 4 lines of main body does not make sense. Also, the choice of colors needs to be reversed as in here.) I will file a separate bug on this issue in gtk 3. TIA
I have submitted a bugzilla entry for very similar bug in gtk+ 3 series: Bug 731190 - XIM input mode indicator does not work. (It shows black box, e.g.) I also edited the title to remove TB (for thunderbird) and FF (for firefox) from the title since any program that is linked with gtk+ 2 library (of later version) suffered from the bug. TIA
(In reply to ishikawa from comment #3) > The following patch was responsible for the undesirable change. I > am not sure why the new code doesn't work as expected. It > seems to me that the desktop environmental setting ought to follow the > change so that base[] and text[] would have desirable color(s), but never > was committed on Debian GNU/Linux release (?) > [...] > https://git.gnome.org/browse/gtk+/commit/modules/input/gtkimcontextxim. > c?id=7f1801eae10b67fb8878eb562ed3614514b96097 It's not clear to me why the old code worked, either; I would've expected it to fill with the text colour first, to get a border, then the base/bg colour.
Yeah, that old code seemingly was known *not* to work, and was fixed in GTK+ 3 with this: https://git.gnome.org/browse/gtk+/commit/?h=gtk-3-22&id=c750cea4e57ed3bcb5ba7eada95d0be380aa2fe5 It might be as simple as just applying that patch to GTK+ 2.24 - and I would appreciate if you could test that - otherwise it might take a little more work but probably not much at all.
Created attachment 358974 [details] [review] imcontextxim: Fix showing glyph in status window This custom drawing code was broken, which meant the current mode indicator/character did not show, severely hindering XIM users. Fix by simply removing the custom drawing and just being a GtkWindow, by cherry-picking these two from GTK+ 3: commit 211ccb5c73a444683d493588c3eb29b457a5586b commit c750cea4e57ed3bcb5ba7eada95d0be380aa2fe5t -- I am an input-method novice, so I haven’t yet figured out how I can test this… So, I’d really appreciate if you or any other reader can apply it & let me know. I don’t see why it wouldn’t work, but you never know.
(In reply to Daniel Boles from comment #11) > Created attachment 358974 [details] [review] [review] > imcontextxim: Fix showing glyph in status window > > This custom drawing code was broken, which meant the current mode > indicator/character did not show, severely hindering XIM users. > > Fix by simply removing the custom drawing and just being a GtkWindow, by > cherry-picking these two from GTK+ 3: > commit 211ccb5c73a444683d493588c3eb29b457a5586b > commit c750cea4e57ed3bcb5ba7eada95d0be380aa2fe5t > > > -- > > I am an input-method novice, so I haven’t yet figured out how I can test > this… > So, I’d really appreciate if you or any other reader can apply it & let me > know. > I don’t see why it wouldn’t work, but you never know. Thank you for the patch. gtk 3 code now works. I checked. See https://bugzilla.gnome.org/show_bug.cgi?id=731190#c6 I have noticed that mozilla firefox and TB today seems to be linked against gtk-3. So now the issue for me to test this is to find a program that was linked with gtk-2.x. The latest Debian GNU/Linux seems to have moved to gtk-3.0 and so I have tough time to figure out where I can find programs linked against gtk-2.x. (Wait, there may be an old version of TB still around that works with gtk-2.x). Let me put this task in my LONG work queue. TIA
My plan of testing this is as follows. - download thunderbird 17.x (which was known to be linked to gtk-2.x) and then try Japanese input and check the status indicator. - Then I would download the gtk source for 2.x series that is used by Debian GNU/Linux and apply the same patch that was applied to v3 gtk in bug 731190. - build gtk 2 and extract im-xim.so and copy it to system's gtk 2 library tree. - Now check if status indicator is usable when I attempt Japanese input into thunderbird 17.x If anybody else beats me to it, that will be fine. TIA
Could you not just clone GTK+ 2.24 from git, apply this patch, build it, and try Japanese input on e.g. builddir/tests/testtext ? The reason I haven't done this is because I simply don't know how to make XIM do anything. :)
(In reply to Daniel Boles from comment #14) > Could you not just clone GTK+ 2.24 from git, apply this patch, build it, and > try Japanese input on e.g. builddir/tests/testtext ? > I am not sure where the gtk+ 2.24 git repository is. gtk+ repository pulls all the source including 3.x series, no? Can you tell me the exact URL for cloning 2.24 only? I did not realize builddir/tests/testtext is the input test program, I think this will do. > The reason I haven't done this is because I simply don't know how to make > XIM do anything. :) Well, it is indeed a bit of black magic :-) I need to run a XIM-aware input engine. In my case, I use a version of kinput2. There are other input frontends such as ibus and others (they may not be XIM-compatible). In the Japanese input scheme, usually phonetic rendition is then converted into combination of kanas (Japanese phonetic characters) and kanjis (Chinese characters adopted in Japan). This conversion must be handled by an external program that runs locally, or on LAN, or even WAN (google offers a service) I run so called (free) jserver. In my setup, I need to run programs that require Japanese input using kinput2 front end wit the following environment variable setup: XMODIFIERS=@im=kinput2 (For ibus, it would be XMODIFIERS=@im=ibus, but I am not familiar with the very popular ibus framework.) So we need to - run jserver at system startup, - run kinput2 (a version called kinput2-wnn) when I log into X11 desktop. - run mozilla thunderbird, for example, with XMODIFIER=@im=kinput2, and I configure kinput2-wnn so that when I hit SHIFT-SPACE, I enter a mode where kana (phonetic character) input strings get converted to a combinations of kanas and kanjis at the next hitting of SPACE bar. To tell you the truth, I have no idea how Korean and Chinese input is done using XIM protocol. Similar setup is done, I am sure, but the details of programs and conversion engines all differ. TIA
(In reply to ishikawa from comment #15) > (In reply to Daniel Boles from comment #14) > > Could you not just clone GTK+ 2.24 from git, apply this patch, build it, and > > try Japanese input on e.g. builddir/tests/testtext ? > > > I am not sure where the gtk+ 2.24 git repository is. gtk+ repository pulls > all the source including 3.x series, no? > Can you tell me the exact URL for cloning 2.24 only? That was bad wording. Sure, you just clone gtk+ then check out the gtk-2-24 branch within that. > I did not realize builddir/tests/testtext is the input test program, I think > this will do. It's just a test of a GtkTextView with some extra features, but hopefully enough to let you test this, without having to install any really old program, etc. > > The reason I haven't done this is because I simply don't know how to make > > XIM do anything. :) > > Well, it is indeed a bit of black magic :-) [...] Thanks for all the other info; I might find that useful someday! :)
Hello, did you get a chance to test out the patch?
(In reply to Daniel Boles from comment #17) > Hello, did you get a chance to test out the patch? Hi, I tried but I found "testtext" does not work as expected from the viewpoint of I18N (Internationalization). It does not invoke the XIM input even if I tried to run it in the same environment as thunderbird and other software are run when I invoked key sequence to allow me to input Japanese characters. What it means, I think, "testext" fails to invoke the necessary function call at the beginning of main(): I am sorry I can't recall the name of the function right away, but I probably can look them up in a couple of days. And then I got busy with a preparation of a week-long meeting at the beginning of October, and could not follow up. So is the following the way to go? --- begin quote > > Could you not just clone GTK+ 2.24 from git, apply this patch, build it, and > > try Japanese input on e.g. builddir/tests/testtext ? > > > I am not sure where the gtk+ 2.24 git repository is. gtk+ repository pulls > all the source including 3.x series, no? > Can you tell me the exact URL for cloning 2.24 only? That was bad wording. Sure, you just clone gtk+ then check out the gtk-2-24 branch within that. --- end quote Do you know the exact label name for gtk-2-24 branch? TIA
(In reply to ishikawa from comment #18) > Do you know the exact label name for gtk-2-24 branch? It is precisely that, gtk-2-24 https://git.gnome.org/browse/gtk+/log/?h=gtk-2-24
I think it is setlocale() that needs to be called for enabling the program to pick the locale information and all that, but I am not sure if calling setlocale() in main() is the right thing to do when we use gtk toolkit/library. Come to think of it, we may need some other call(s) for enabing XIM (?) This I am not sure, but let me check further.
I *think* I need to ad gtk_set_locale() in the code below: main() in https://git.gnome.org/browse/gtk+/tree/examples/text/text.c?h=gtk-2-24 int main( int argc, char *argv[] ) { GtkWidget *window; GtkWidget *box1; GtkWidget *box2; GtkWidget *hbox; GtkWidget *button; GtkWidget *check; GtkWidget *separator; GtkWidget *table; GtkWidget *vscrollbar; GtkWidget *text; GdkColormap *cmap; GdkColor color; GdkFont *fixed_font; FILE *infile; <=== HERE I need to call gtk_set_locale(); gtk_init (&argc, &argv); window = gtk_window_new (GTK_WINDOW_TOPLEVEL); gtk_widget_set_size_request (window, 600, 500); gtk_window_set_policy (GTK_WINDOW (window), TRUE, TRUE, FALSE); g_signal_connect (G_OBJECT (window), "destroy", G_CALLBACK (close_application), NULL); *HOWEVER*, I read that gtk_set_locale() has been deprecated since 2.24 and should not be used (?) in the following web page. https://developer.gnome.org/gtk2/stable/gtk2-General.html Maybe I should call setlocale (LC_ALL, "") instead according to the web page.
For whatever readon, I can't make tests/testtext to enable Japanese input using XIM. However, I found that tests/print-editor does enable Japanese input using the special keysequence defined for kinput2. So I am going to test the patch and then use print-editor to see if it works.
OK, bingo. After applying the patch, I tried to see if tests/print-editor works. (Actually, it was already applied when I tested test/testtext.) Then initially the indicator was still a bit buggy. However, after I installed ("make install") into /usr/local/lib, then the indicator works fine. I looked for the copy of im-xim.so used on my system and just in case, I removed the one from GTK-3, and tested again. The modification works. So I think the modification in https://bugzilla.gnome.org/show_bug.cgi?id=696624#c11 https://bugzilla.gnome.org/attachment.cgi?id=358974 is good! TIA
Attachment 358974 [details] pushed as bc555cd - imcontextxim: Fix showing glyph in status window Thanks for testing! Now let's just wait and see if there's ever a 2.24.32...