GNOME Bugzilla – Bug 106081
Inconsistent the returned keycode between java and gtk applications
Last modified: 2004-12-22 21:47:04 UTC
1. Tested gnopernicus with a java and a gtk application,I noticed that the at-spi return different key structure (keycodes , keysyms and keystring ) for a same pressed key on java and on gtk application. 2. At-spi did not return the modifiers status under java application. 3. It has registerd a keyset whit specified keystring (for example ControlLeft). Navigating in java application the callback was called by the at-spi for a keys which was not registered for this listener (in our case the callback for ControlLeft). The returned AccessibleKeystroke structure has keysms with value 0, and the kestring variable contain the keystring of the pressed key. It is not good!! Which key has a keysym value 0? 4. The returned AccessibleKeystroke structure it is same under java application for Return key from keypad and from Enter (return). How it could be made a difference between these two keys? 5.The returned keystring for Return from gtk application is different as the returned keystring returned by the java application("Kp_Return" or "Return" in gtk application and "\n" in java application). All tested on linux machine. Could be make consisten the returned keys struct in the at-spi? It is urgency to have consistent keycodes it. It is very bed idee to check in different way the keys for different applications.
I am concerned that you are listening for these keys at all, I think you should not! It seems to me that gnopernicus is listening to keystrokes and using them to figure out the application's context, i.e. what the user is doing. This approach may make sense in Windows but it is not the right approach in GNOME/AT-SPI. You can get correct context information from the AT-SPI about what is happenning in the UI in response to user keystrokes, you should not be relying on the keystrokes themselves for this. Please explain *in detail* the exact use cases where you are using these events and why they are necessary, and then we may be able to suggest alternatives. Thanks Bill
*** Bug 106004 has been marked as a duplicate of this bug. ***
By the way: I do agree that these are probably bugs that you are seeing - I am just concerned about why you are listening for these events at all.
In our case, we construct a keyset for every user define keys. This used in command mapping options. If the user define a key combination it is need to catch the combination everywhere on the desktop and to execute the associated command. In this case only the keys which defined by the user are registered not for ALL keys. In this case the listener catch the event keys from java application which is not registered. I think it is not good behavior to catched by the listener keys which we are not interesting.
paul: could you point us to the gnopernicus code that registers this keylistener? That will help us diagnose what is happenning.
bill: the file where are the code part is : /gnopernicus/kbd_mouse/libke/libke.c The function ke_register_keylist() the function which register the keycombinations which get after splitting from list of parameters. Called in ke_init(). The deregister function is: ke_unregister_key_list(). Called in ke_terminate(). The callback is: ke_report_user_key()
Darren and I have discussed this at length, and looked at the current "toolkit key event" code in ATK and also what's available in Java. Java handles/defines key events quite differently from GDK or X. Darren and I think we can make the 'id' field of the libspi DeviceEvent match the keysyms from GDK; they already match for 'printable' characters (they return the UNICODE code point), we think we can make them consistent for "control" characters too. Likewise the fact that the 'string' field for Java-access-bridge key events is blank is also a bug which we think we can fix (we are optimistic that we can make the strings consistent with those for other apps and X). However we don't think we can make the 'keycode' values consistent between Java and other toolkits, since Java virtualizes the keycodes and there's no way to get the mapping from the underlying (X) server to the JVM's concept of keycodes. Our thought at this time is that the keycode values will be consistent within a toolkit but won't be guaranteed to be the same across toolkits. Since keycode values are already highly hardware and configuration dependent, we don't think this is a big loss.
Bill, can you assign this to me...
sure Darren: I thought you could reassign bugs in at-spi ?
Created attachment 14966 [details] Test Binary
Paul, I've attached a binary with a possible fix in it, could you give it a go and see if it's what you want - if you're still seeing problems please let me know. BTW - I've discovered a problem with Java and the way it determines if NumLock is on.. : On Linux: If XKB is enabled (which it is by default), the code Java is using is not working - I mentioned this to the XFree86 guys, and they agree, in theory, but at the moment a solution isn't imminent... On Solaris, it doesn't seem to matter and works according to spec... The test being done is a call to get the state via XQueryKeymap which should (in theory) have the bit set corresponding to the numlock key for as long as the toggle is on, but in the case I describe it's only set if, and only if, you're physically holding the key down, as would be the case for most keys on the keyboard... I think I'm going to have to come up with another solution for NumLock, but I'd appreciate if you could verify the general case...
Created attachment 14973 [details] led.c - program demonstrating the bug regarding numlock key
Darren: I made a workaround to catch the key event with numlock active for java application. Thanks for code examples. Changes on CVS
So Paul, did the fixes I made to Java to map certain keys work for you, if so can I commit this and close this bug? BTW, as for the NumLock, I'm not happy with the solution of making the fix in gnopernicus, but there's no easy way for me to fix this in Java - I've logged a but against Java but it could take some time - so I guess I've little option in accepting it - for now :)
Hi Darren: Please apply the commit, but please to not close this bug , because it needed that from at-spi the keyevent from other all keys not only for keypad to be consistente. When I saw at-spi and java-acces-bridge has consistence issues. Not problem if still has the bug open.:-) I want, I need to have a good base for keyboard api for all type of applications. I hope you understand my opinion.
Paul, I'm not sure I do understand your meaning. Either I have fixed the bug, or not... If the fix is imcomplete, can you please tell me what it is that is not correct so that I can fix it? Thanks, Darren.
marking NEEDINFO until either paul indicates what is still broken/inconsistent, or Darren can do confirmation tests that satisfy us we've fixed the problem fully. The NUMLOCK issue is a separate one. One thing we could do, Darren, is to register the JABG as an at-spi listener (!) and thus we could receive "keyboard:modifier" events from at-spi-registryd. We could then use these to determine the NumLock state.
Updating status_whiteboard field to reflect A11Y team's assessment of accessibility impact.
Woops, don't need to track this one with the status_whiteboard field.
Hi Darren: I would to check again your patch and close it if it is work, for this bug (inconsistenc returned values). The attached binary I can't download correct. Could you attach again the binary? On my system the attachment is saved in cgi extension format. I looked it in, and I saw that it is a jar iniatialy. Please attach again the binary.
Paul: Darren is out this week. The test binary is a jarfile. If you save it as gnome-java-bridge.jar then you should be able to install it.
Bill: Thank you very much for help.
Darren: I've checked again your patch for key inconsistency. I've tested it on my Linux RedHat8.0 system with the gnome-2.2 stack from 10th of April, and java-access-bridge.jar the attached to this bug. I've checked it with SwingSet and Gnopernicus application. Using two different combinations of flags to catch the key event (first time I've only used SPI_KELISTENER_ALL_WINDOW, and the second time SPI_KEYLISTENER_CANCONSUME | SPI_KEYLISTENER_NOSYNC), I've noticed the followings: 1. Using SPI_KEYLISTENER_CANCONSUME | SPI_KEYLISTENER_NOSYNC flags the returned keysyms are different between 2 types of application (SwingSet and gnopernicus). For example. (java) KP_1: sym: 49 keycode:97 (gtk) KP_1: sym:65457 keycode:87 In this bug you mentionated that the keycode field could not be made consistent. Let's say that I understand this, but I waited that the keysym field to return consistent value for the same key. 2. Using SPI_KEYLISTENER_ALL_WINDOWS flag, the two applications return the same keysym and keycode values. This flag uses global listener. I don't think that the java-access-bridge has effect at the returned values in keysym fields. About the NUM_LOCK state issue: The NUM_LOCK issue (with an example) the recomandation it's working correctly both with Java Toolkit applications and Gtk Toolkit applications. I consider that this bug is not fixed entirely. The NUM_LOCK part is ok but, the others are NOT.
keysym 49 is defined by X as XK_1, whereas 65457 is XK_KP_1. This is possibly worrying, since it may imply that Java has discarded the information that this key event was from the number pad rather than the primary keyboard. Darren, could you check your logic to confirm that this problem is in the JVM rather than in the java-access-bridge?
Created attachment 16000 [details] [review] Patch for keypad_test only for test
Attached the patch (keypad_test_patch) what I applied on keypad_test.c. I used SPI_KEYSET_ALLKEYS flags to catch all keys, not only the registered keyset. I made this that because when I use other flag that ALL_WINDOW the key event not comes for numpad keys. This changes I made only for this test. I used SPI_KEYLISTENER_CANCONCUME and SPI_KEYLISTENER_NOSYNC flags, instead of SPI_KEYLISTENER_ALL_WINDOWS. The keys registered whitout a keymodifier. (Use SPI_KEYMASK_UNMODIFIED) I leave only one key registration. I tested on SwingSet (java toolkit) and Gedit (gtk toolkit) applications. Use java-access-bridge.jar proposed by Darren. I tested with these two application now to made a difference between a returned structure. The initialization. And the returned keycode values from Xserver for numpad keys. [csongor@CSONGOR test]$ ./keypad-test keycode[0] = 90 keycode[1] = 87 keycode[2] = 88 keycode[3] = 89 keycode[4] = 83 keycode[5] = 84 keycode[6] = 85 keycode[7] = 79 keycode[8] = 80 keycode[9] = 81 keycode[10] = 112 keycode[11] = 63 keycode[12] = 86 keycode[13] = 82 keycode[14] = 91 keycode[15] = 108 Keypad key registry: result succeeded NumLock is not active. Returned messages for NumberPad 1 key: SwingSet: "keypad KeyEvent [keysym 0x0] (keycode 35); string=End; time=bfda72a9" Gedit: "keypad KeyEvent ? [keysym 0x65436] (keycode 87); string=KP_End; time=bfdc7209 keypad KeyEvent ? [keysym 0x65436] (keycode 87); string=; time=bfdc7268" You can observ that the returned values differs at the 2 applications. This happen only when NOT use SPI_KEYLISTENER_ALL_WINDOW.
Paul, I've given the mapping another go and I think I've it got it sorted out this time... (says he hopfully :) I'll attach a copy of the jar file again... Can you see if it does the job?
Created attachment 16006 [details] gnome-java-bridge.jar - 2nd attempt at fix.
Darren: I tested the new attached jar file, but I noticed the same issues. :-( It possible that I need to make any changes in config or in my enviroment? I saved the attachment, I renamed , and I moved in /opt/gnome-2.2/share/jar directory. It's OK?
Paul, Putting it in /opt/gnome-2.2/share/jar is fine, if and only if, you've a symbolic link in the $JAVA_HOME/jre/lib/ext directory to that location, ie: ls -l $JAVA_HOME/jre/lib/ext/gnome-java-bridge.jar: .... gnome-java-bridge.jar -> /opt/gnome-2.2/share/jar/gnome-java-bridge.jar I'm definately seeing the correct keysym - though the key string is "End" not "KP_End" but it's the keysym you're refering to... My output is: Java: keypad KeyEvent ? [keysym 0x65436] (keycode 35); string=End; time=d426c790 GNOME: keypad KeyEvent ? [keysym 0x65436] (keycode 87); string=KP_End; time=d426e168 keypad KeyEvent ? [keysym 0x65436] (keycode 87); string=; time=d426e1b4 I'm a little confused why there are two events in GNOME though (anyone able to explain this to me?), but the essential element is correct - the keysyms are all the same....
Darren: Thanks for solution I had no link in /usr/java/j..../jre/. I checked again it and now it return the same keysym values at GNOME and Java toolkits, excepts only for one key from keypad, which is the key 5. Did not arrive any key event for numpad key 5 in java application, when is the numlock is not active. When numlock is active it is comming. About your question: The listener registered for 2 event press/release. I thing under GNOME both events are comming.
Darren: Could you check a couple of things? (1) make sure this works for both "End" (i.e not from the keypad) and "KP_End" (2) we ought to agree w.r.t the strings as well, I think.
Darren/Paul: yes, it looks as though perhaps we're not getting the "release" events from Java. That's item #3 to check I suppose ...
Phew, at least I'm making progress now :) 1. Numpad 5 This is a problem with Java - a bug if you will - in that it doesn't actually generate ANY event for someone pressing KP_5 with NUMLOCK off! 2. End should be generating GK_End and KP_End should be genrating GDK_KP_End - is that not the case? 3. Release events, that's an interesing one... Seems the code just doesn't handle it... Will add that one now... Are relese events consumable too? 4. Strings - What is expected here, just the name of the key, or should it contain modifiers - I'm assuming name of key... This looks like it's going to also require a mapping table as I've set up for keys - not pretty :(
I can't tell if we still have problems with 105963 as a result of this bug; seems as though we should be able to at least support the arrow keys for non-consuming listeners, for both Java and StarOffice apps.
I am marking this bug as fixed as Darren has done a commit which fixes all issues except for the case of the Numpad-5 key not generating anything when numlock is off, but that's a Java issue for which Java bug 4856167 has been logged.