Thanks. I wonder if there is a programmatic way to easily find out which dead accent combinations should be supported for a keyboard layout?

At least on a keyboard that *does* have separate real apostrophe and dead acute keys (like the Finnish and German ones, for instance), I think there is no reason not to let GTK+ continue to work as it does, i.e. try to combine the dead acute with whatever character is pressed next if such a precombined character exists in Unicode, even if "native" apps do it only for a restricted set of characters? (And in the future, we might even make it produce the next character followed by a Unicode combining accent, so that you could produce things like an accened digit...)

How do people that use the US International keyboard manage this strange behaviour of the "apostrophe" key? Apostrophes are after all pretty common in English. How do they know when they have to press space after hitting the "apostrophe" key to actually get an apostrophe? I.e., if somebody wants to type:

        'arrow'

does he instinctively hit the spacebar after the "apostrophe" because otherwise he would get á ? But then if he wants to type:

        it's

does he equally instinctively knows to just hit "apostrophe" and then "s"?

(Sure, in the latter case hitting space after the apostrophe and then "s" doesn't hurt either, but if people would generally do that when they want an apostrophe, then there would be no reason for this bug report;)

http://support.microsoft.com/kb/97738/EN-US/ describes the expected behaviour of the US International keyboard. Note that the "right-most Alt key" on many (at least European) physical keyboards is labelled "AltGr".

I wonder if it is only users of the US International keyboard that expect dead accents to combine only with a predefined set of following letters? Is it fine for other keyboards if dead accents continue to combine if at all possible with a following letter?

So far we have had user input only for the "Apostrophe" key (which actually is a dead acute accent key) on the US International keyboard. What about the other dead accent keys on it? Do people definitely want also them to behave exactly like in Microsft software? For instance, the dead grave accent key to the left of the "1" key. In GTK+, typing that key followed by "y" produces "ỳ" while in Wordpad it produces "`y". Is this also important to get exactly as in Microsoft software? I don't think so. There is an important difference here, in that spacing grave accents should be very rarely used, while the apostrophe is of course very common in English.

Note that it is likely many end-users confuse the spacing grave and acute accents, "`" and "´" with the left and right single quotation marks, "‘" and "’". These users just need to be educated... The left and right single quotation marks are accessible on the US International keyboard with Right Alt (i.e., AltGr) and "9" and "0" respectively.

Daniel, could you try to get some opinion from Pidgin users on the above issues? Especially from users who know what they are doing and not just blindly doing wrong things, like using spacing grave and acute instead of quotation marks...

I'm inclined to agree with you about other layouts that do have dedicated apostrophe keys - although I'm sure that people misuse them, there is a better argument to be made for maintaining the current behavior.

To address the situation where people are typing 'arrow' - I think think it is a question of common use-case.  When typing English, the double-quote is by far the most common when used for quoting - IIRC the only time that using a apostrophe as a single quote is grammatically correct is when nesting a quote within a quote.  

Of course, context is important - when writing code, using single quotes is far more likely to be problematic.  I suspect that this is just something that people who use the US-Intl layout just get used to and do what they need to do to get the character they want without thinking too hard.

I haven't heard any other complaints except the ones regarding the apostrophe (which I've heard a number of times).

The ideal situation, IMO, would be to make it work the same as the Windows keyboard layout, if there is a way to detect what should be done programmatically.

If it is going to have to be a special case, then I think that just covering the apostrophe behavior will prevent the most common confusion.  If we want to go a step farther, I think that the second most likely deadkey to cause problems is the "double quote" (umlaut) (shift+apostrophe), (think about the case where you are starting a quote).

Hello. I originally posted a bugreport to the people developing Pidgin (the IM client), and I was asked to provide my input here.

I am impacted by this issue too. I use the US-International keyboard layout under Windows, and I find gtk+ applications (like Pidgin) behave differently from all other applications, which causes frustration with the software.

The answers to comment #1 are all "yes". If I want to type 'arrow', I really do instinctively press Space after the apostrophe because I know that '+a would otherwise produce á. But I am equally used to typing words like "it's" and "I'm" without a spacebar, which means that gtk+ causes me to stumble all the time.

To be honest, I find it strange that this is even an issue. All Windows applications handle this the same way, probably by simply capturing a WM_KEYPRESS event which contains the character. I haven't tested this, but I would bet that '+s actually produces two separate WM_KEYPRESS messages. I really don't think there is anything special that gtk+ would need to do to get it to work just the way it works everywhere else. It rather seems that gtk+ contains extraneous code that does this unwanted conversion from '+s.

There is no WM_KEYPRESS message. There are WM_KEYDOWN and WM_KEYUP, and then WM_CHAR which is on a higher level. Windows syntesizes WM_CHAR messages based on the lower level ones.

GDK wants to generate events separately for key press and release, so it's the lower level key down and up messages that have to be handled. WM_CHAR is *not* used at all.

GTK+ handles things like dead accents or compose key (the "Compose" key is a concept not used on Windows, but still) sequences itself. This explains why it behaves differently than "most" Windows apps which use Microsoft's libraries. GTK+ handles a much larger set of compose sequences than Windows does. This is in general a *good* thing. I can produce for instance a ẃ character, which is a legal precombined Unicode character, apparently used in Welsh, on my Finnish keyboard layout in GTK+ by just typing the dead acute accent followed by w. But not in WordPad for instance.

Note that on most Windows keyboard layouts, there are *separate* apostrophe and dead acute accent keys, and separate dieareses and double-quote keys. It's just the US International that mixes them up.

I am fairly sure that there *are* other applications and/or toolkits for Windows than GTK+ that also handle things differently than "all" other Windows apps.

> I would bet that '+s actually produces two separate WM_KEYPRESS messages

Of course, but that is no surprise. Two separate WM_KEYDOWN and WM_KEYUP, yes it does. And two pairs of GDK_KEY_PRESS and GDK_KEY_RELEASE events are generated. It's then the upper levels in GTK+ that (depending on the GTK+ input method used) combines key events into actual Unicode characters.

If you run a GTK+ program with the environment variable GDK_DEBUG set to "events", you will get a printout on standard output of what messages are received, and what GDK events they are translated to. For the keys ' s on the US-Intl keyboard, the messages and GDK events are:

WM_KEYDOWN 001910BA ACUTE/CEDILLA ch:de sc:40 rep:1
  ===> GDK_KEY_PRESS 001910BA 0xde group:0 dead_acute 0:""  => 0
WM_DEADCHAR 001910BA DefWindowProcW => 0
WM_KEYUP 001910BA ACUTE/CEDILLA ch:de KF_UP KF_REPEAT sc:40 rep:1
  ===> GDK_KEY_RELEASE 001910BA 0xde group:0 dead_acute 0:""  => 0
WM_KEYDOWN 001910BA S ch:53 sc:31 rep:1
  ===> GDK_KEY_PRESS 001910BA 0x53 group:0 s 1:"s"  => 0
WM_CHAR 001910BA DefWindowProcW => 0
WM_CHAR 001910BA DefWindowProcW => 0
WM_KEYUP 001910BA S ch:53 KF_UP KF_REPEAT sc:31 rep:1
  ===> GDK_KEY_RELEASE 001910BA 0x53 group:0 s 1:"s"  => 0

As you see Windows itself says that the "apostrophe" key is a dead key for either acute or cedilla. It's just the upper level processing in Windows that then turns it into an apostrophe if the following key isn't one of the small set that they have chosen to combine with the acute or cedilla. Once this bug gets properly fixed (during the weekend?), GTK+ will also work in the same way for the US Intl keyboard.

Thanks for your message. Given the last paragraph, I assume you are suggesting that the current plan is to fix this so that it uses WM_CHAR as it should. Thanks for that.

Of course I meant WM_CHAR when I said WM_KEYPRESS. The main point was that WM_KEYPRESS gives a scancode of a key that is pressed, which is uninteresting in a textbox/textarea where you type characters, while WM_CHAR gives a character codepoint of a character that has been typed (this is what you want for a textarea).

At the risk of stating things you already know, I'd like to point out a few things:

1) Given what you say about how GTK+ interprets keypresses, it implies that it
   cannot possibly work with custom-defined keyboard layouts. So I tested it,
   and indeed, it fails horribly. I have a layout where Shift+6 is a deadkey,
   but it is not recognised as one. The ' key is a deadkey similar in function
   to US International, but GTK+ does not produce the correct characters
   wherever it differs from GTK+'s own idea of what the character should be.
   Therefore, I assume that if I were to define the ' key as a deadkey for the
   grave accent, GTK+ would still (incorrectly) produce acute accents.
   Needless to say, it works just fine in all Windows applications that use
   WM_CHAR as they should.

2) Obviously Windows lets users create new keyboard layouts and switch between
   them at will. Therefore, I don't need GTK+ (which doesn't apply to all of
   Windows) to give me the ability to type a ś or ẃ; I can already type those
   by switching to a custom-defined keyboard layout that has such a character
   (and I do have one installed). This does not change the fact that under the
   US International keyboard layout, '+s => ś is unexpected.

3) In this debug output line:
       WM_KEYDOWN 001910BA ACUTE/CEDILLA ch:de sc:40 rep:1
   I assume the string "ACUTE/CEDILLA" comes from the keyboard layout
   definition. This string is not a meaningful identifier. It is a name for
   the deadkey that can be arbitrarily chosen by the keyboard layout designer.
   It could be called "ACUTE" but produce an umlaut or anything else.

Again, apologies if this information is redundant because it was already known, but given how little I know about GTK+, for all I know it might be useful.

> I assume you are suggesting that the current plan is to
> fix this so that it uses WM_CHAR as it should.

No, not at all. That would break the event sequence the upper GTK+ levels expect from GDK. If only WM_CHAR messages were handled, GDK couldn't generate the correct key press and release events in the correct order. And yes, it is important to get both press and release events in a toolkit like GTK+.

I bet some GTK+ applications also need to know exactly when keys are pressed and when released. Just generating one key press event immediately followed by a key release event from a WM_CHAR would not then correspond to how the user actually has pressed keys.

For instance many GIMP tools behave differently when using the mouse dependeing on whether Shift, Control, Alt, or a combination of them are pressed. I am not sure if GIMP actually tracks the press and release events of these keys, or if it uses the modifier key state field in mouse events though. But it *could* track the key events, and we shouldn't on purpose break GDK so that this is no longer reliable. After all, some app might use "normal" keys as "modifier keys". I might imagine some app could well use the space bar like this, for instance. I.e., some app could have a GUI where doing some kind of drawing or whatnot with the mouse works differently depending on whether the space key is down or not.

(Yeah, I realize that it could be hard to get the code reliable for such an app; it would need to ask the status of them whenever the app receives focus in case a such "app-specific modifier" is down already when the app receives focus.)

> all Windows applications that use WM_CHAR as they should

You have any documentation to back that claim up, that WM_KEYDOWN and WM_KEYUP would somehow be deprecated or recommended not to be used?

> The main point was that WM_KEYPRESS gives a scancode of a key
> that is pressed,

You still keep writing WM_KEYPRESS when you mean WM_KEYDOWN.

It actually also gives a so-called virtual key code in the wParam, which is what GDK looks at. When first needed after an application starts (or after the user has changed the keyboard layout of a running application), GDK builds a keymap that maps the virtual key codes in four modifers states (none, Shift, AltGr, Shift+AltGr) to GDK keysyms. Check update_keymap() in gdkkeys-win32.c

> it implies that it cannot possibly work with custom-defined keyboard layouts

Sure, there are a couple of bug reports about the misbehaviour of GTK+ on Windows with 3rd-party keyboard layouts, macro recorders and the like. So sue me. Or submit a patch. To me it is more important that it works for the Microsoft keyboard layouts. (It doesn't currently for some of the more complex keyboard layouts like the Czech or Canadian Aboriginal one. Plus of course this US International case.)

> This does not change the fact that under the US International
> keyboard layout, '+s => ś is unexpected

Yes, I know (now) that for most users it is unexpected. But by the way, are you sure there aren't people who use the US International keyboard to write "international" text, perhaps even in languages that actually use ś or ŕ? For such people GTK+ as it currently is, would would work *better* than "all other Windows apps", as they can get the dead accent to "stick" to more letters than just the handful Microsoft has chosen.

(But let's not argue about that, my current opinion is indeed that GTK+ on Windows should try as well as possible to behave like other apps on Windows. Even if that actually means less functionality.)

> I assume the string "ACUTE/CEDILLA" comes from 
> the keyboard layout definition. This string is not a 
> meaningful identifier.

That string comes from the GetKeyNameText() Win32 function. But it is used just in GDK for debugging output after all, so I don't see your point? Whether it is meaningful or not is not interesting, but it sure is informative to see the key name spelled out in debugging output.

Created attachment 127725 [details] [review]
Suggested patch

> If only WM_CHAR messages were handled,

That is not what I suggested. The correct solution is to use WM_CHAR when you are looking for characters, and WM_KEYDOWN/UP when you are looking for keypress/release events. That is what they are there for, and that is what all other Windows applications do. WM_KEYDOWN/UP are not deprecated, they are just the wrong tool for the job of listening for text input.

> GDK builds a keymap that maps the virtual key codes to GDK keysyms.

In other words, it tries to reconstruct the keyboard layout using some home-made mechanism, and it gets it wrong. Just looking at the attached patch: it contains a C-code description of the US International layout! This is redundant; you won't find such a reimplementation of every keyboard layout in every single application. Also, surely you must also realise that this solution can never possibly work with custom-made keyboard layouts.

> There are a couple of bug reports about the misbehaviour of GTK+ on Windows
> with 3rd-party keyboard layouts. [...] To me it is more important that it
> works for the Microsoft keyboard layouts.

The two are precisely the same issue, and the correct solution (using WM_CHAR) solves both.

> But by the way, are you sure there aren't people who use the US International
> keyboard to write "international" text, perhaps even in languages that
> actually use ś or ŕ? For such people GTK+ as it currently is, would would
> work *better* than "all other Windows apps"

There *is* no ś or ŕ on US International. So yes, I'm pretty sure there aren't people who do the impossible :). Someone who requires ś and ŕ won't be using US International, but another keyboard layout that actually has those characters.

> That string comes from the GetKeyNameText() Win32 function. But it is used
> just in GDK for debugging output after all

OK, so this piece of information was redundant. Sorry :)

> In other words, it tries to reconstruct the keyboard layout
> using some home-made mechanism, and it gets it wrong.

Exactly. I know that. But the upper levels of GTK+ (*and* GTK+ applications to some extent) expect to get key press and release events, with each event corresponding to a physical *key*, not a character. Note that many keys don't correspond to any character at all, like "Insert", arrow keys, function keys etc.

> you won't find such a reimplementation of every keyboard
> layout in every single application.

No, of course not. Not every application is based on code that was initially written for the X window system without thoughts of potential portability to a quite different windowing system API.

> surely you must also realise that this solution can
> never possibly work with custom-made keyboard layouts

Of course I do realise that; that's why I ponder in the comment whether we actually instead should figure out the exact intended dead key behaviour of the keyboard programmatically and use that information.

But on the other hand, as far as I know, it's only the US International keyboard where the same key is used both for dead diacritic functionality and also for very common plain characters, i.e. apostrophe and double quote. (Are there other similar?)

Surely you understand that I think this is actually a design error in the US Intl keyboard layout? (And yes, a design error that is set in stone now.)

Surely you can't claim with a straight face that it is a good design choice that the user needs to know instinctively that if he is writing some word that happens to start with a vocal within double quotes:

        "exotic"

he needs to press a space after the "double quote" key, but if he is writing another word:

        "wild"

he doesn't need to.

> There *is* no ś or ŕ on US International.

No, but if you look at the actual keyboard, I bet there isn't any key with an á engraving on it either.

So what you mean is that the "standard" applications and/or libraries don't have any trivial way to generate those perfectly existing and legal Unicode precombined characters on the US International keyboard.

(Instead one needs to use some hard to remember Alt+numpad digits sequence which has three different variations (!) if you believe http://www.microsoft.com/globaldev/reference/glossary.mspx: Alt+<xxx> that uses an OEM code page specific code point (yecch), Alt+<0xxx> that uses a Unicode BMP code point in decimal (decimal, ewww!), and Alt+<+>+<xxxx>) that uses a Unicode BMP code point but in hex (finally right, except that it is restricted to the BMP). Experimentation shows that this is not really universal in even Microsoft's own apps, though. Often one has to use an application-specific "Insert:Special" dialog. GTK+ has its own variation on this theme: Shift+Control+U followed by a Unicode code point in hex, followed by for instance space. I can't say that is easy to remember either, but at least it uses full Unicode code points in hex, which is how they are universally referred to in documentation etc. Never in decimal.)

If you think with an open mind, you understand that one could well say that GTK+ is an *improvement* when it easily allows the user to generate more accented letters than the Microsoft "normal" toolkits do. (But as I have indicated, I do realize that we should follow user expectation here at least for the badly designed US Intl keyboard.)

> the correct solution (using WM_CHAR) solves both.

Have you BTW ever heard the phrase "patches welcome"? As you know a lot about these matters, are you actually willing to give concrete help, i.e. write code? Show us how easy it is to use WM_CHAR *and* not break anything.

gdk/win32 used to handle both WM_CHAR and WM_KEYDOWN/UP until the 2.2.3 version. See what the ChangeLog.pre-2-4 in current GTK+ says about how much complexity could be removed when WM_CHAR handling was removed. I don't recall much of the details any more (it was back in 2003), but check the code in gdkevents-win32.c from GTK+ 2.2.0 for instance.)

Note that I would not accept a patch that would decrease non-disputed functionality, for instance on keyboards with dedicated (dead) accent keys. It is also for instance essential that it is possible in GTK+ apps to bind functionality to control-digits, on all keyboards. If you handle only WM_CHAR, that is not possible. So you have to handle WM_KEYDOWN and WM_KEYUP to some extent and keep some state. All the details make it very messy.

I do think it is a good thing that in GTK+, on a Swedish or German keyboaard for instance, you can generate for instance ŕ  by simply pressing the (dedicated) dead acute key and then r. Even if you can't do this in apps that use Microsoft toolkits, where you then get a freestanding (spacing) acute and r instead. Surely freestanding accents are very rarely needed by normal users on such keyboards, and if people actually produce them intentionally, it is because they are confusing them with single quotes or single left or right quotes. (And even in GTK+ you *can* produce a freestanding accent if you want by typing a space after the dead accent key.)

Btw, I am finding it hard to programmatically determine the exact behaviour of a keyboard layout. It presumably is possible to "reverse engineer" the keyboard layout as MSKLC can do it. But does MSKLC use some undocumented API, or some documented API in some undocumented way...? Or does MSKC "cheat" and "decompile" the binary file (kbd*.dll) for a keyoard layout?

Try running the below attached test program and see if you can tweak the code so that in the case of the US Intl, for instance this part of the output is different:

...
 36,   7         Shift: -1 asciicircum
...
   39: 2 asciicircum 9
   41: 1 a               <<--- 
   42: 2 asciicircum b

(And then ditto for the other dead virtual keys, those that have a "-1" in the output line followed by a sublist of combining behaviour.)

The marked line says that the code has determined that typing the keys ^ (i.e. Shift-6) followed by a produces a plain a, not â. Cleary that is wrong. But what tweaks are necessary to make the code find out that correctly? Note that for ^ e it does work...

If you can fix that sample program so that it programmatically can determine the exact intended behaviour of a keyboard layout, then it would be possible to have GTK+ do that without hardcoding this knowledge as in my patch above.

Created attachment 127794 [details]
Test program

This blog post by Michael Kaplan (who else... this guy is probably the most knowledgeable person there is in Windows i18n matters, or at least the most helpful, as his blog is full of useful information) explains a bit: http://blogs.msdn.com/michkap/archive/2004/12/22/329657.aspx . The kbd.h file included with MSKLC apparently describes the structure of the data inside a keyboard layout. But how to get at this data for a HKL? http://www.docdroppers.org/wiki/index.php?title=Writing_Keyloggers has some more useful information. The pathname of the DLL for each keyboard layout is in the Registry (of course...).

Created attachment 127992 [details]
kbddump.c

To understand the structure of keyboard layout DLLs (or actually, the structure of the data pointed to by the KbdLayerDescriptor() function in such DLLs), I wrote this small program. (Partially inspired by the code in the second link in the previous comment). It seems to be pretty clear, except that for some reason the output for the CharModifiers seems bogus, clearly some bug still in the program. You will need the kbd.h header included with MSKLC to compile it.

I realised that as far as I know, no Windows keyboard layout have any key that would be labelled "Compose" or "Multi-key". There is no way to generate the GDK_Multi_key keysym on Windows. Thus much of the prebuilt compose sequence table in gtkimcontextsimpleseqs.h is wasted on Windows. This is probably not anything worth losing sleep over, though.

(In reply to comment #17)
> Thus much of the prebuilt compose sequence
> table in gtkimcontextsimpleseqs.h is wasted on Windows.

I can only hope that, regardless of platform, those tables are written in object files that are then linked in as read-only segments and sharable.

AfC

(In reply to comment #18)
> (In reply to comment #17)
> > Thus much of the prebuilt compose sequence
> > table in gtkimcontextsimpleseqs.h is wasted on Windows.
> 
> I can only hope that, regardless of platform, those tables are written in
> object files that are then linked in as read-only segments and sharable.

Yes, they are read-only segments and sharable.

*** Bug 586557 has been marked as a duplicate of this bug. ***

*** Bug 600432 has been marked as a duplicate of this bug. ***

Maybe a stuppid question, i am just a user, i want to use Geany for Windows, it has this bug, and that makes the use of Geany complete useless on a windows machine, is there a fix/patch or will there one coming? I really hope so! keep up the good work!

Regards, Evert

"Completely useless"? Ha. Workarounds are mentioned above.

Yes, completely useless as i need " and ' for programming, i don't see concrete solutions, i see posts that confirms the bug and discussions about the bug, but i haven't seen a patch or something? I don't want to  sound rude or something, i just really want to use Geany for windows :-)

But why do you use the US-Intl keyboard layout for programming?

Anyway, GTK+ on Windows is maintained by volunteers like me on our spare time, it doesn't really help whining how especially important it is for you to get some specific bug fixed. We work on what we find interesting, and personally, right now, I am not especially interested in this bug. So you will have to wait.

> I don't see concrete solutions

No, but you should see workarounds if you just actually read the comments above.

Press the spacebar after pressing the key that looks like ' or ". That's what I say right in comment #1.

(Note that these keys actually are dead accent keys from Windows internal point of view, but the Microsoft "widgets" that handle input in most programs manage to mix this up royally and make them behave weirdly.)

What is your opinion about what I say there in comment #1, BTW? Do you really find it natural that if you use the US-Intl keyboard, and you use software that works as you expect (for instance Notepad), and want to type in these two lines:

 char foo = 'a';
 /* It's hard to do this */

you must know that in the first line you have to press space after the first ' key (or otherwise you get char foo = á' ), but you don't have to in the second line (but it doesn't hurt).

To repeat: the workaround I suggest is that you simply learn to *always* press space after the ' or " keys on the US-Intl keyboard in those cases where you want them to generate the ASCII single quote or doublequote characters (and not depend on the following letter).

But yeah, I do expect that your answer will be that you are not prepared to change your typing habits a single bit... 

> i haven't seen a patch 

What's that in comment #10 then?

Its the default keyboard layout, i am not whining, i just posted a message, and after that i only replied.

The geany guys sended me over here as they can't solve the problem in geany itself,btw, your workaround doesn't work :-(

I didn't read "all" the replies, only the beginning and the end, because there was still discussion at the end about keyboards and layouts i assumed it wasn't solved.

Sorry, i don't see a solution for me in #10, unless you mean the first part, i have no idea how i should apply that.

> i am not whining

OK, good;)

> Its the default keyboard layout

Where? Anyway, it's trivial to add new keyboard layouts.

> your workaround doesn't work 

Hmm, what happens then if you press the ' key, then a space, then the A or S key, in a GTK+ entry widget? What GTK+ version are you using?

> But why do you use the US-Intl keyboard layout for programming?

Because he wants to. We don't need to justify it or apologise for it to you.

> What is your opinion about what I say there in comment #1, BTW?

I answered these questions in comment #6. The answer is "yes" for anyone who is
used to using US-Intl as their main/default keyboard layout.

> I do expect that your answer will be that you are not prepared to
> change your typing habits a single bit... 

Of course not. Why should we?

> > Its the default keyboard layout
> Where?

On his computer, clearly. Incidentally, on mine too.

Look, i am really not whining, i am just hoping for a solution :-)

I am using the by windows chosen default keyboard layout, why? I don't know, its the 'standard' here (Holland), it always is, and everything else always works fine, except Geany (GTK) ;-)

The problem only occurs in Geany, that is on linux, a perfectly swt IDE, but i can't just install Linux on this computer, that is running Windows 7.

The output in Geany: space + ' = ´ and space + shift + " = ¨

http://www.geany.org/Download/Releases#windowsBinaries

It uses GTK 2.16

I was already afraid this would happen, some people getting a bit annoyed when they see this noob with his questions :-) alot of people are subscribed.

btw, the space + '/" was already a habit, i do that automaticly when a program requires that method of displaying a ' or " ... i wish that was the problem, then i wouldn't be here ;-)

Evert

> > Where?

> On his computer, clearly.

Yeah, but I meant where geographically, of course.

> its the 'standard' here (Holland)

Oh, I didn't know that.

> The output in Geany: space + ' = ´ and space + shift + " = ¨

Ah OK, i.e. the spacing variants of the diacritics in question, not the ASCII ones.

Anyway, I guess I should apply that patch in comment #10. It is not perfect, not generic enough, but probably good enough for now, and better than the current situation.

I originally subscribed to this thread to watch for a fix.  The recent conversation lead me to decide to post my thoughts as well.  On that note, I do naturally hit space before a e i o u and y after ' or ".  I also automatically hit space after ~ before n.  That having been said, I think all dead accent combinations should be available, and Microsoft's custom picking and choosing is just wrong here (how international is that).  However, I do like to get an ' when I type 'n  instead of getting just the n, and I don't know if there is any valid reason the US International Keyboard layout shouldn't do that in GTK.  I think it does in X11, but my linux box is at work, so I can't confirm this.  Anyway, as I said, I don't necessarily think this needs to match Windows identically, though that would be convenient.  I have some additional information, though.  I recently noticed that LeftCTRL+LeftAlt combinations that work in Windows (and again, I'm pretty sure [though not positive since I can't test] that they work in X11 sa well) don't work in GTK+ either.  This could be a bug in my version, but, for instance, LeftCTRL+LeftALT+7 gives me ½ here and doesn't in the GTK app I was using yesterday (Pidgin, obviously really important, lol).  So if applying the aforementioned patch is the best way to go, then do it by all means, but this issue could be more interesting than you realize and might merit some looking into.  Incidentally, not that this is necessarily of any help, but once upon a time (perhaps Win98, perhaps 2000), I believe the character map in Windows would give you the US International Layout shortcut for symbols supported when you were using that keyboard layout; it now gives you the unicode and the Alt+NumPad combination regardless of selected layout.

*** Bug 587531 has been marked as a duplicate of this bug. ***

*** Bug 610928 has been marked as a duplicate of this bug. ***

*** Bug 611447 has been marked as a duplicate of this bug. ***

(In reply to comment #28)
> > But why do you use the US-Intl keyboard layout for programming?
> 
> Because he wants to. We don't need to justify it or apologise for it to you.
> 
> > What is your opinion about what I say there in comment #1, BTW?
> 
> I answered these questions in comment #6. The answer is "yes" for anyone who is
> used to using US-Intl as their main/default keyboard layout.
> 
> > I do expect that your answer will be that you are not prepared to
> > change your typing habits a single bit... 
> 
> Of course not. Why should we?
> 
> > > Its the default keyboard layout
> > Where?
> 
> On his computer, clearly. Incidentally, on mine too.

I would like to share some of my experience here.

- 1 - I am French and I do programming too. As a result, I input code lines using standard languages (PHP, Java, C# etc...) so all synthax is English-based. However I have to write, in the mean time, French, like objects labels, tooltips, doc etc.... therefore I use US Intl keyboard layout because I can input French accents easily... As I don't live and work in France but in Asia, you'll figure out that it is almost impossible to find a French layout keyboard around. I am definitely not alone in this situation. Millions other people are in the same situation as mine.

- 2 - I love Gnome environment and Gnome applications in general. I have been using Linux at home for several years. But since I have no choice but to use Windows in my office, I installed some Gnome appz, like gedit (2.29.4) for Windows....
I have been disappointed when I realized I couldn't input double-quotes (") using US Intl layout. This looks a funny little detail, but in fact, in my eyes, the matter is more serious: 
- Gnome proudly (and definitely rightly) claims to respect international standards (de facto or setup by regulation bodies) to make FOSS a credible, reliable and practicle alternative to proprietary software makers.... but here we are in the situation where a very basic feature is simply not working.
- standard users from Windows environment who give a try to Gnome appz like gedit on Windows, will walk away having the feeling to have played a while with a toy software, not even capable of writing a tiny little double-quotes in US Intl keyboard layout... while in fact the issue does not come from gedit but from GTK+....
- this (very basic) issue: keyboard layout problem, gives credit to all buddies around who hurl FUD on Linux...
- Tor Lillqvist   [gtk+ developer]: I do appreciate and aknowledge the tremendous work and effort you and all the community put into Gnome. But seriously, you just cannot say things like "We work on what we find interesting, and personally,
right now, I am not especially interested in this bug. So you will have to
wait."... this sounds too much like the typical caricature of the Linux buddy (I do what I want, na!, let me jerk off in peace!!), which is used by all FUD makers out there to turn the community into a bunch of pumply childish geeks....

I repeat once again, I do believe in FOSS, I do hope and believe it'll become more and more important and wide spread in coming years... some bad habbits have to be surrended though.... focus has to be put on users. In my eyes: a software without users is not a software, it's just a failure.

You know, people like you make it even less fun to work on FOSS.

I don't care if you belive in FOSS or in the Flying Spaghetti Monster, if all this belief means is that you believe you have the right to insult people.

I work on GTK+ (the little I do, nowadays) for fun in my spare time. That's something people like you seem to have hard to understand. I want more fellow hackers to work on it, sure, but to be frank, the less whining users, the better.

FUD comments I quoted in my previous message are not mine at all. There are comments you can read in some IT reviews and IT forums.

I find those comments obviously outraging and above all very unfair.
I could experience already how many FOSS software do work better than their proprietary counterparts. Great pieces of software such as Eclipse, OpenOffice or PHP work so well out of the box and on multi platforms.

I know how valuable people like you are for the whole community.

I am just trying to stress out that some issues, bugs could, should be fixed before others, even if there is no fun in doing so, because those bugs have an immediate and strong impact on basic users who experience gnome for the first time and leave them having a bad feeling about it.
My job is to develop/maintain an inhouse ERP for a school in Asia. I am confronted on daily basis to very basic users who don't have a clue about how works IT and how difficult/long some features are to be implemented. But I try my best to put myself in their position so I can understand which bug is more troublesome to them. Figure out one second that to one of those users in the school, who complains about a bug that makes in daily work more complicated I answer something like you did: "We work on what we find
interesting, and personally,right now, I am not especially interested in this bug."...

bon courage et bonne continuation

Raphael, this is your *job*. If you get paid to do so, you can't say "I work for what I find interesting". But if someone works voluntarily, you can't *expect* him to fix what bothers you. You can say that you find this bug really annoying and you think it should get a high priority, but you can't complain about a volunteer developer not fixing it. You can fix it by yourself, or complain that no one funds fixing bugs like this, or pay an amount to Tor for fixing this so this would be your donation.

Can't you see that your previous comment just annoyed him? What good does that do?

Noam

(In reply to comment #38)
> Raphael, this is your *job*. If you get paid to do so, you can't say "I work
> for what I find interesting". But if someone works voluntarily, you can't
> *expect* him to fix what bothers you. You can say that you find this bug really
> annoying and you think it should get a high priority, but you can't complain
> about a volunteer developer not fixing it. You can fix it by yourself, or
> complain that no one funds fixing bugs like this, or pay an amount to Tor for
> fixing this so this would be your donation.
> 
> Can't you see that your previous comment just annoyed him? What good does that
> do?
> 
> Noam

Hi Noam,

At no one moment I intended to hurt or annoy anyone.
At no one moment either I complaint Tor for not fixing the bug.


I just wanted to put in perspective the following:

- 1 - users using US Intl keyboard layout are many. For the most part because they have no other alternative that makes their use of computer easier.
As far as I can remember, this bug concerns only GTK+ on Windows isn't it? US Intl keyboard layout works fine in Linux env, doesn't it?

- 2 - some bugs have bigger impact on people who give a try to Linux because they can rely only on what they see or experience. As a result, they may end up their experience with a bitter taste... Those people are not IT pros or dev, they're just basic users who come, for the most part, from Windows environment and were told about Linux through articles, friends, distro popularity (Ubuntu, Mint, Opensuse...).

- 3 - I stressed out that Tor reply I quoted in my post would sound counterproductive to users, especially those who come from Windows env and get into this forum for help.
Companies and individuals who want to turn down Linux env., most of them for business reasons & strategies, rely on comments like Tor's to give credence to their FUD. I repeat once again that FUD arguments are outraging and unfair for the most part.
In the mean time, efforts made by many Linux actors including distro makers (Ubuntu, Mint, Mandriva ...), kernel dev, corporations (Intel. Google, Red Hat, HP etc..) aim precisely at puting Linux system into mainstream and accessible to basic users using Windows....

- 4 - I would like to state again something I said in my previous posts: I do aknowledge and praise the quality work of valuable people like Tor who spend their spare time on working on Gnome in particular and FOSS in general.

If this bug can be solved ASAP, that's great. If it cannot be fixed for now for technical issues reported by Tor, so let's be patient.

regards
rl

I just wanted to share my story:

I'm a developer and I use the US International Keyboard because I speak Spanish and whenever I send an e-mail or IM in Spanish, it is easy for me to compose the special accented vowels or ñ.

Well... actually, regarding IM: I love Pidgin, but I found this problem quite annoying.

As a few other people mentioned, every other program in Windows seems to honor the usage of the apostrophe key, except for it.

I came across this entry, where say that it actually is a problem with GTK+ code:

Ticket #8244
http://developer.pidgin.im/ticket/8244

I understand that this is open-source and I have no rights to demand anything, so I'm not :)

All I'm saying is that I wish this got fixed and that I'll be very happy when that happens.  If people wonder whether we have a valid point to "complain", well... some of us feel that we do.

*** Bug 619003 has been marked as a duplicate of this bug. ***

I've tried to use Pidgin too, but it doesn't work for me too. If I wan't to type "don't", I usually do not use the spacebar. When I try the same in Pidgin, it comes out as "don ". When I DO use a spacebar, it comes out as "don’t". "I'm" and "it's" come out as "Iḿ" and "itś", or with spacebar: "i´m" and "it´s".

It would be great if this could be fixed, but I was actually more interested in what triggers this problem. I have tons of friends who use pidgin without this problem. Yet if I try to install it on my Windows 7 machine, I get this problem. I've tried to install it on my XP desktop too, but then I get the same problem. Yet I only know one other person who has this problem, all the others don't. So does anyone know why I do encouter the problem and other people don't?

Do the other people use the US-intl keyboard layout?

Noam

Not being able to type "don't" etc as expected (yes, Dutch, US-international KB, like above posters) can be become quite annoying.

What steps would I have to take to implement the "suggested patch" in Comment 10?

Thanks in advance,
Ivo

*** Bug 655357 has been marked as a duplicate of this bug. ***

After years of using Pidgin for Linux, I had to install it in a Windows 7 machine, which led me to this bug.

As a user of US International, the obvious problem is the handing of the apostrophe, described here: https://bugzilla.gnome.org/show_bug.cgi?id=610928

This is inconsistent with Windows MFC and with GTK+ on Linux, since one can only obtain a floating accent (´) instead of an apostrophe ('), and a floating accent is almost always the wrong choice.

As a result, I can't use Pidgin. While it is the best IM client out there, I don't want to press CTRL+V every time I need an apostrophe.

Since GTK+ is free software, I'm not in a position to make any demands. However, I was surprised to see that such a significant usability bug has been open for so long.

So it turns out I can't use Pidgin or Geany on Windows due to a years old GTK+ bug… Thanks a bundle!

Feel free to ask for your money back.

For the record, GTK+ is the only case I've seen where i18n is broken by having an non-standard "improved" input method instead of encoding issues. I don't use the US International keyboard, but the most popular keyboard layout for my native language has the same dead key for adding diacritics. I've never used a different layout in my life. I wanted to have Geany and Pidgin in Windows since I use them in Linux. Using a different layout would allow me to type apostrophes correctly, but it wouldn't allow me to type diacritics for my native language, and it's also a burden to manage different layouts for different software. The patch with a hardcoded US Internationa layout doesn't help. The snide comments about whining users from a developer who doesn't understand the issue doesn't help. I'm leaving feedback here to better reflect the scale of the issue.

For the record, dead accent keys work fine for instance on the Swedish/Finnish keyboard layout I use all the time. (Or at least worked fine back when I still was doing GTK+ stuff.) The problem with the US-Intl layout is its weird behaviour; the same key is supposed to work *either* as a dead accent, *or* as a non-accent quote character, depending on what the key typed next is. Or something like that. (I don't recall exactly any more; read my verbose comments earlier.)

If you, Reinis, don't use the US-Intl keyboard layout, it is not at all certain that you are even seeing the problem *this* bug report is about.

I just tested Finnish, Spanish, Latvian and US International layouts in Windows and Linux, and the dead keys for these layouts work the same in Windows. Dead key followed by a space generates the default character for that key. Dead key key followed by a diacritic-enabled character generates a character with a diacritic. Dead key followed by a character that can't be combined with a diacritic generates the dead key default character plus the other character. The difference between these layouts is that in the Finnish layout the default dead key character is the same as the diacritic, while in Spanish and Latvian layouts the default dead key character is the apostrophe. Saying that this difference makes non-Finnish layouts without a dedicated apostrophe key "weird" or "strange" is just boorish. If this bug had a more clear description, there would be more "whining users". It's senseless for GTK+ to try to do the OS's job in the first place, but in this case GTK+ also breaks the input for many users. I get backticks instead of apostrophes in Pidgin and Geany, and I can't use the alphabet of my native language, since GTK+ inserts characters with grave accents instead of Latvian diacritics.

OK, so I am the first to admit that it is badly broken then. So feel free to not use it. Or work on fixing it, or fund somebody else to do it, etc. That's how volunteer-maintained open source works.

Can someone explain to me WHY Gtk+ assumes something about meaning of dead keys? Why it does interpret these dead keys unlike the keyboard?

The keyboard layout must be predictable. If user types "don't" as DON'T, and "it's" as IT'S, a single GTK application should not produce "don" and "itś".

GTK+ provides key press and release events to the application. See the problem?

You mentioned "fund somebody else to do it", how much money are we talking about?

I'm interested in donating in order to have this bug fixed.

Hello, here's another Pidgin user that is experiencing the same thing. Windows 7, both Dutch and United States-International have the problem. I'm going to have to adopt a different input language in order to use Pidgin normally..? 

I'm really happy with all the efforts put into open source software so selflessly, but I cannot understand the annoyance amongst (some?) developers (?) resulting from all the people reporting this. Why even allow people to submit bugs and the like if you're going to tell them to "just don't use the software" then? We're not talking about some obscure feature people would like added, we're talking about inputting apostrophes... 

I'd be willing to put in efforts but I am not much of an developer. I'm just another user of the software reporting the bug. I do happen to be desperate enough to give someone monies if that means they'll fix it. I also think it would be important not to underestimate the scale of this issue - I reckon not many people, if they can even find this page, will go through the trouble of registering an account and posting about this issue.

Also, for everyone looking for a workaround: change your input language. Yes, this will probably also mean some keys will produce unexpected results for you. Pros vs cons. 


Windows 7: 
- 1 - Navigate: Start > Control Panel > Clock, Language and Region > Change keyboards or other input methods
- 2 - Click the fancy "Change Keyboards..." button
- 3 - In the new window with the pretty list of installed languages, press "Add..."
- 4 - Choose a fun language you don't hate. 

Personally, I went with "United Kingdom". Swapping @'s with "'s is not half as annoying as not being able to type normal apostrophes, since - by the way - they were all interpreted as really obscure characters (´ omg) in some of my friends' IRC clients, making my sentences a pain to read. 

Hurray, I can now use Pidgin semi-normally!!!

Personally my workaround has been to use Pidgin in Windows through a Linux virtual  machine, since changing languages would mean I would break a lifetime of typing habits.

I'm not so sure if this is related, but some of us are having a hard time trying to type ć __at all__.

are the rules for <compose> <accent> <character> the same as <dead_accent> <character>? and where are they implemented in the code, or where are they configured? and what about exceptions?

I'm on Linux and I have a <compose> key and I observe the following:

<compose> <'> <a> : á
<compose> <'> <e> : é
<compose> <'> <r> : ŕ
<compose> <'> <s> : ś
<compose> <'> <c> : ç

this seems to be highly desirable for Portuguese speaking programmers using the US International keyboard with dead keys, where the sequence "<compose> <'>" is replaced by "<dead_acute>".

in my opinion if you are using a layout with dead keys, it's more than obvious that <dead_acute>/<dead_diaeresis>/<dead_grave> will put an acute/diaeresis/grave on top of whatever you type after it.

I consider it highly surprising if this does not happen or, worse, if a different effect is achieved (like the acute becoming a cedilla).

in languages like French and Italian where we need apostrophe and both acute and grave accents, we know we must pay some attention when typing "l'encyclopédie" or "ce n'è" if we configured our keyboard with dead keys.

In the recent comments it has become clear that it's not a matter of there being an issue, but of what the ideal functioning of dead keys would be. Would it not also be important to consider that for a cross-platform enviroment like GTK+, functionality should be uniform on all platforms. Why is the Linux functionality not the norm for these events?

It IS issue because it hardcodes dead keys and doesn't work with all keyboard layouts.

The "ideal functioning" would be to not break standard OS input methods. GTK+ shouldn't be making any assumptions about the keyboard layout. The current hardcoded dead key behavior is a big issue since a lare number of users have layouts that behave like US-Intl.

It wouldn't be making assumptions when someone actually set their keyboard to US international. It's not that lay-outs behave like US intl. but that US intl. itself is meant to make it easy to type both words like "Don't" and "Français" with minimal keystrokes. The fact that almost every Windows application, including the input forms in browsers for for instance this website work that way, demonstrates the intention of this design.

GTK+ has a hardcoded behaviour for dead keys that only partially matches the US-Intl keyboard layout and matches other layouts similar to US-Intl even less. For instance, the standard keyboard layout for my native language uses the same dead key as US-Intl, but pressing <dead key>+a is supposed to produce the letter "ā" and not "á" as in US-Intl. The result is that in GTK+ applications I can't use my native language properly. It's all because GTK+ assumes something about the meaning of keys that is not defined in the keyboard layout. Please read the comments here more carefully.

The importance of this bug should be raised. The title is misleading, since the problem is not just US-Intl layout related, but it prevents proper input with *any* layout that uses the same dead-key as in the hardcoded behaviour. This problem renders GTK+ software like Pidgin unusable on Windows for a huge number of people.

Other bug-report threads have been made in the past with more fitting descriptions and concise explanations, but since this particular thread predates all of them the discussion eventually always moved here. Indeed the name of issue is misleading and I also feel the severity "minor" is quite the understatement, given the amount of users affected the detrimental effect it has in applications such as GIMP, Pidgin and HexChat on Windows.

Also noteworthy that this issue will be 5 years old in January and is still marked as NEW.

Similar problem in Bluefish on Win8x64 where, when using US International, typing ' gives you a completely different character (which then causes problems in the code).

' turns into ´
" is typed as ¨

Changing keyboard settings to something else (for example UK English) fixes this, but causes other problems (in all other programs, like the @ being in the wrong place on UK English, and the inability to type é á accented letters quickly ).

I am using US International because I type in 2 languages daily and indeed like being able to type don't as well as reëel or hé or hè .
(I think in the Netherlands US International is the standard keyboard setting?)

Adding a message here in the hopes there has been a fix (I got here via a facebook message) in the 4 years since this was pointed out?

*** Bug 743376 has been marked as a duplicate of this bug. ***

Please, someone, at least mark this as Confirmed so I can sleep better.
I don't see that much into the internals, is this something requiring a lot of work to fix? That may justify the 6 year wait. If not, is here anyone willing to fix this for a small fee?

The fix for me has been not using any software based on gtk+.

(In reply to Mundano from comment #70)
> Please, someone, at least mark this as Confirmed so I can sleep better.

There is no "confirmed" state. All bugs are confirmed until closed.

(In reply to Reinis Ivanovs from comment #71)
> The fix for me has been not using any software based on gtk+.

I can recommend this fix, been working well for me for several months now.

Please take it elsewhere. This is a bug tracker, not a forum

I'm more than willing to try to fix it (in my spare time, which is not much) and for free. But i'll need some directions as i'm not a Gnome developer. If there's anyone kind enough to tell me where to look at, i'll apreciate it. I'll obviously start to look after writting this, but having a hand in that would be nice.

Now, for the bug record, my two cents: 

* US-INTL is used everywhere in the world, as it combines the ASCII keys needed for every programming language and server maintenance tasks, with the rest of the common keys needed for natural languages (we speak Spanish in my country). 

* Every tool i use is multiplatform (except when that's impossible). Having to constantly change tools is costly for my day to day basis. I use Geany as my text/code editor, partly because it's gtk+ based (and not java, like many others). This bug is kicking me out from using gtk+ apps on Windows, and so i will stop using them in any other system as i adapt to new tools. And please note that if i use Windows is because i'm forced to, not because of personal choice; i'm a GNU/Linux user and developer, and i abhor Windows.

Hi Daniel, I'm not a Gnome developer, but I have worked with WINAPI and I can give you some advice:

GTK is reading the raw keyboard buffer using WINAPI's [`GetKeyboardState`][1] in the `build_key_event_state` function in the `gdkevents-win32.c` file, since it's reading the raw keyboard buffer, it doesn't know anything about keyboard layouts, dead keys, etc.

You have to get the Keyboard layout with WINAPI's [`GetKeyboardLayout`][2], and then you have to get the character encoded in UCS-2 with WINAPI's [`ToUnicodeEx`][3], you'll probably need to convert the character to UTF-8, since only Windows use the UCS-2 encoding.

Be aware that `ToUnicodeEx` may return zero or more than 1 character, ex. when you press dead keys.

Since you're a Linux guy you'll need to research how GTK implements dead keys, and translate the events properly.

[1]: https://msdn.microsoft.com/en-us/library/windows/desktop/ms646299(v=vs.85).aspx
[2]: https://msdn.microsoft.com/en-us/library/windows/desktop/ms646296(v=vs.85).aspx
[3]: https://msdn.microsoft.com/en-us/library/windows/desktop/ms646322(v=vs.85).aspx

From the discussion earlier i take it that GTK+ just looks to see if both keys, when combined, give meaningful result. If they do, then that's what GTK+ uses. This is very generic, but obviously runs afoul of the table-based approach that Windows uses.

As mentioned (much) earlier, the correct fix for this bug is to manually load keyboard layout DLL (for US-International it's KBDUSX.DLL) and get the deadkey combination table from it.

This approach has 2 problems:
1) Keyboard layout structures documentation can't be reliably googled, the only official documentation is in kbd.h from MSKLC.
2) Keyboard layout structure alignment depends on native OS bitness, so even if you write a 32-bit program and load KDB*.DLL from SysWoW64 subdirectory when running on 64-bit OS, you *still* need to use 64-bit pointers. This is a problem, because MinGW does not have __ptr64 for pointer coercion. The workaround[1] is to manually declare a different set of structs with manual padding after some fields. That seems to be working, but given (1) and the need to experiment to see what various fields mean, having to deal with (2) at the same time is very inconvenient. Thus, this should be done with MSVC, which does have __ptr64, and then ported to be MinGW-w64-friendly.
3) kbd.h documentation is said to be somewhat misleading, see [2]

P.S. Do source the headers (GTK+ can't rely on kbd.h from MS SDKs, because it's missing from MinGW-w64) responsibly. This[3] seems like a good starting point, it's public domain. Source code from CodeProject is notoriously non-free (even though it's open).

[1] https://stackoverflow.com/questions/11747532
[2] https://stackoverflow.com/questions/22138910
[3] https://github.com/awakecoding/Win32Keyboard

Alternatively, there's this gem:
>		      /* It is a dead key, and it has been stored in
>		       * the keyboard layout's state by
>		       * ToAsciiEx()/ToUnicodeEx(). Yes, this is an
>		       * incredibly silly API! Make the keyboard
>		       * layout forget it by calling
>		       * ToAsciiEx()/ToUnicodeEx() once more, with the
>		       * virtual key code and scancode for the
>		       * spacebar, without shift or AltGr. Otherwise
>		       * the next call to ToAsciiEx() with a different
>		       * key would try to combine with the dead key.
>		       */
>		      reset_after_dead (key_state);

Which implies that you can get away with just calling ToUnicodeEx() on all possible combination of DeadKey+NonDeadKey and record the outcome.

Okay, a correction:
You can get away with just calling ToUnicodeEx() on all possible combinations of DeadKey+DeadKey+NonDeadKey and record the outcome.
Because dead key chaining[1] turns out to be a thing.
Will probably need to come up with a more efficient way to store that, because dead key chains seem to be order-independent, so DK1+DK2+X is the same as DK2+DK1+X, and storing these as separate records will waste space (and i'm not even sure about having more than 2 (dead) keys in a chain...There's a proprietary piece of software[2] that claims that you can even have DeadKey+DeadKey+NormallyNonDeadKey+NonDeadKey sequences, such as COMPOSE(dead)+^(dead)+T(notdead)+M(notdead) = ™).

[1] https://en.wikipedia.org/wiki/Dead_key#Chained_dead_keys
[2] http://www.kbdedit.com/manual/ex17_compose_key_chained_dead.html

Created attachment 331622 [details] [review]
W32: Prefer the deadkey combinations that the OS uses

This patch depends on attachment 331620 [details] [review] from bug 768722.

It makes GDK and GTK pay more attention to deadkey combinations
that the OS supports.

I've tested it with US-International keyboard layout, and it seems
to be working as expected (that said, i never use US-Intl or dead
keys in my day-to-day life, so i might have missed something).

Created attachment 332413 [details] [review]
W32: Prefer the deadkey combinations that the OS uses (gtk2)

Comment on attachment 331622 [details] [review]
W32: Prefer the deadkey combinations that the OS uses

Attachment 331622 [details] pushed as 5e6c192 - W32: Prefer the deadkey combinations that the OS uses

LRN, the patch in #81 needs to be updated after the gdk.symbols change in 768722 (now it should add gdk_win32_keymap_check_compose to the *existing* #if block that that patch added instead of a new block).

Created attachment 335034 [details] [review]
W32: Prefer the deadkey combinations that the OS uses (gtk2)

v2:
* Rebased on top of attachment 335030 [details] [review] from bug 768722

LRN, does your patch makes dead key sequences which aren't predefined by GTK work?

For example, Comment #64 mentions <`> + <a> outputting "ā" and not "á"
Another example: there is a Lithuanian layout called Programmers, where a single <`> dead key is being used to produce Lithuanian letters with four different types of accents: http://www.registrucentras.lt/litwin/kbdltp.gif . Does your patch fix GTK with these layouts as well?

(In reply to Rimas Kudelis from comment #85)
> LRN, does your patch makes dead key sequences which aren't predefined by GTK
> work?

AFAIK, my patch removes *all* predefined GTK dead key sequences and uses dead key sequences built into Windows keyboard layouts instead. It can *miss* more complex dead key sequences, but it won't invent any new ones by itself, or let GTK guess about dead key sequences.

> 
> For example, Comment #64 mentions <`> + <a> outputting "ā" and not "á"

It doesn't mention the layout in which that happens.

> Another example: there is a Lithuanian layout called Programmers

I don't see it in Windows keyboard layouts that can be added for Lithuanian input method.

> , where a
> single <`> dead key is being used to produce Lithuanian letters with four
> different types of accents: http://www.registrucentras.lt/litwin/kbdltp.gif
> . Does your patch fix GTK with these layouts as well?

No idea. I can only test when i know which keys i should press, and in which layout (that is available in Windows), and what character that is supposed to produce.

(In reply to LRN from comment #86)
> AFAIK, my patch removes *all* predefined GTK dead key sequences and uses
> dead key sequences built into Windows keyboard layouts instead. It can
> *miss* more complex dead key sequences, but it won't invent any new ones by
> itself, or let GTK guess about dead key sequences.

That's good to know. I'm not concerned about the "complex" sequences so much.

> > For example, Comment #64 mentions <`> + <a> outputting "ā" and not "á"
> 
> It doesn't mention the layout in which that happens.

It's Latvian.

> > Another example: there is a Lithuanian layout called Programmers
> 
> I don't see it in Windows keyboard layouts that can be added for Lithuanian
> input method.
> 
> > , where a
> > single <`> dead key is being used to produce Lithuanian letters with four
> > different types of accents: http://www.registrucentras.lt/litwin/kbdltp.gif
> > . Does your patch fix GTK with these layouts as well?
> 
> No idea. I can only test when i know which keys i should press, and in which
> layout (that is available in Windows), and what character that is supposed
> to produce.

The keymap in my example is installable, it's not being shipped with Windows. But nevermind that.

What about the GTK2 backport? Will that ever get integrated?

GTK2 backport is stuck on bug 769550.

With Latvian keyboard layout pressing <'>, then <a> produces <ā>, while in US Intl the same sequence produces <á>. This is in Windows and GTK3 (where this patch is already applied). In somewhat recent GIMP nightly (gimp-dev-x86_64-2016-08-25, which uses GTK2 without this patch) the same sequence produces <á> in both US Intl and Latvian layouts.

I think you can duplicate Bug 453068, Bug 559115, and potentially Bug 572369 against this one. Although the latter might still be relevant if it uses those complex dead key sequences you mentioned.

Regarding bug 769550: my impression is that it's stuck in "waiting for somebody to make a debug build" phase. I don't think the build has to be of GIMP specifically. Any GTK+ text editor (e.g. Gedit or Mousepad) or other app should be enough just for testing.

Another possible duplicate: Bug 165723.

@LNR: Can I maybe test the GTK2 patch for you with something other than GIMP? Assuming GIMP is not special in that regard, and Bug 165723 is talking about keyboard shortcuts, not dead keys, does it make sense to depend on that bug with a GTK2 fix for this one?

Bug 165723 relation is...complicated. From the cursory inspection it seems that bug 165723 was, back in 2006, the source of some of the code that i found in GTK this year. But it was reopened, because dead keys still didn't work quite right. So, the reopened version of that bug is indeed a duplicate of this bug.

You can test with anything you want. GIMP is just a high-profile target that a lot of people use and that we wouldn't want to break by pushing this code into GTK2.

Or i can always just push this stuff into GTK2 branch and hope for the best :)

If you could just point me to instructions on building GTk+2 on Windows, I could build it and test with something less fancy. I just spent some time googling, but all I could find was pre-build binary packages. :)

Can I build GTK2 in MSYS2, for example?

> Or i can always just push this stuff into GTK2 branch and hope for the best :)

Would that result in builds I could test?

(In reply to Rimas Kudelis from comment #92)
> @LNR: Can I maybe test the GTK2 patch for you with something other than
> GIMP? Assuming GIMP is not special in that regard, and Bug 165723 is talking
> about keyboard shortcuts, not dead keys, does it make sense to depend on
> that bug with a GTK2 fix for this one?

Oops, I actually meant Bug 769550 here.

(In reply to Rimas Kudelis from comment #94)
> If you could just point me to instructions on building GTk+2 on Windows, I
> could build it and test with something less fancy. I just spent some time
> googling, but all I could find was pre-build binary packages. :)
> 
> Can I build GTK2 in MSYS2, for example?

https://blogs.gnome.org/nacho/2014/08/01/how-to-build-your-gtk-application-on-windows/
would be a good place to start. Also
https://bugzilla.gnome.org/show_bug.cgi?id=768722#c12

> 
> > Or i can always just push this stuff into GTK2 branch and hope for the best :)
> 
> Would that result in builds I could test?

Eventually, but not immediately.

Your patch doesn't apply: there's no file called gdk/win32/gdkwin32keys.h in gtk+ v.2.24.31 source. Would you care to update the patch?

(In reply to Rimas Kudelis from comment #97)
> Your patch doesn't apply: there's no file called gdk/win32/gdkwin32keys.h in
> gtk+ v.2.24.31 source. Would you care to update the patch?

You probably need to apply patches from bug 768722, specifically - attachment 335030 [details] [review] and likely attachment 335031 [details] [review]

Created attachment 340326 [details]
Output of the dead keys in Latvian keyboard layout before/after your patches

@LRN: thanks! I build GTK+ using MSYS2 and ran the demo application. Then I downloaded all three of your patches, added them to PKGBUILD, rebuilt GTK+ and ran the demo application again.

Your patches work as expected: I was able to enter Latvian characters, and I was able to trigger the quit handler by pressing Ctrl+Q while using a Russian keyboard layout.

I think you should really go on and commit them, if nobody is objecting. Thanks a lot!

Created attachment 340327 [details]
Triggering Ctrl+Q while using Russian keyboard layout

Thank you everybody for working on this! It would be great if this can go into GTK2 as well. I just had another bug report a few days ago about the US (international) layout not working correctly in my application.

Thank you again!

LRN: could you check if Bug 453068, Bug 559115, Bug 165723 and Bug 572369 are indeed duplicates as I think and if so, dupe them properly? I don't have permissions in this bugzilla to do that. :)

Thanks for committing by the way! Do you happen to know when the next GTK2 release might happen?

Also, is there any hope to have files in http://ftp.gnome.org/pub/GNOME/binaries/win32/gtk+/2.24/ updated when that release comes out? After all, these patches are Windows-specific, so I believe Windows users deserve an update with them.

*** Bug 453068 has been marked as a duplicate of this bug. ***

*** Bug 559115 has been marked as a duplicate of this bug. ***

(In reply to Rimas Kudelis from comment #102)
> LRN: could you check if Bug 453068, Bug 559115, Bug 165723 and Bug 572369
> are indeed duplicates as I think and if so, dupe them properly? I don't have
> permissions in this bugzilla to do that. :)

Went there, did what i could.

> Thanks for committing by the way! Do you happen to know when the next GTK2
> release might happen?

No idea. If memory serves, gtk2 releases happen on a need-to-release basis (when some fixes accumulate in gtk-2-24 branch).

> Also, is there any hope to have files in
> http://ftp.gnome.org/pub/GNOME/binaries/win32/gtk+/2.24/ updated when that
> release comes out? After all, these patches are Windows-specific, so I
> believe Windows users deserve an update with them.

Whoa, these are old. No, i don't want to update these. Because that would mean an update of all the dependencies, and then there's the question of gobject-introspection, and which patches i should or should not include, and... no, too much trouble. Most Windows gtk2 applications ship their own gtk2 anyway, and a few times i've tried to replace it with my own, self-compiled gtk2, the applications did not quite work, if they started at all.

(In reply to LRN from comment #105)
> > Also, is there any hope to have files in
> > http://ftp.gnome.org/pub/GNOME/binaries/win32/gtk+/2.24/ updated when that
> > release comes out? After all, these patches are Windows-specific, so I
> > believe Windows users deserve an update with them.
> 
> Whoa, these are old. No, i don't want to update these. Because that would
> mean an update of all the dependencies, and then there's the question of
> gobject-introspection, and which patches i should or should not include,
> and... no, too much trouble. Most Windows gtk2 applications ship their own
> gtk2 anyway, and a few times i've tried to replace it with my own,
> self-compiled gtk2, the applications did not quite work, if they started at
> all.

I know at least one app that uses these (PyGTK all-in-one in particular) for its Windows releases: Zim (http://zim-wiki.org/). At least that's what it says in its "Building for Windows" file (http://bazaar.launchpad.net/~jaap.karssenberg/zim/pyzim/view/head:/windows/README-BUILD-win32.txt) and what it's official Windows binaries seem to use. I don't know if it's the only application out there which does this though.

I tried building it (along with its dependencies) myself last weekend. I tried the MSYS2 pacman way first (by editing the Arch PKGBUILD file for it), but that didn't work because apparently MSYS2 pacman doesn't have all required packages (I think I got stuck at pywin32, or something). Then I also tried building full GTK stack according to https://wiki.gnome.org/Projects/GTK+/Win32/MSVCCompilationOfGTKStack, but that turned out to be quite a drag due to numerous dependencies (on both libraries and various build tools), problems compiling these dependencies, choices I had to make without knowing how they affect things etc. In the end, I spend a whole day on this project, but got stuck/fed up up somewhere among the second half of dependencies, and decided that there are probably people who know and can do that better than me.

I mean, I thought that perhaps the person who used to maintain these packages (or the GTK project itself) had some sort of automation at hand to build thes packages and didn't have to do all that manually as I did. Were my expectations wrong? :)

(In reply to Rimas Kudelis from comment #106)
> I mean, I thought that perhaps the person who used to maintain these
> packages (or the GTK project itself) had some sort of automation at hand to
> build thes packages and didn't have to do all that manually as I did. Were
> my expectations wrong? :)

I don't think there's anyone that fits this description. As you might have noticed, some people who used to be "in charge" of W32 GTK+ left the project (tml, for example).

MSYS2 people tend to do decent package maintainership, but they don't develop GTK+, and they mostly don't ship applications (certainly not bundled applications!), except for the ones that are available as packages. GTK+ developers do develop, but don't do much packaging or shipping. Application developers ship, but don't develop GTK+ or maintain packages.

Okay, that's probably an oversimplification.

Anyway, your best bet is to talk to pygtk maintainers and try to make them either maintain their own binaries, instead of using old GTK+ binaries, or cooperate more with MSYS2 or any other packaging venue. That said, there might be some Python-related friction (MSYS2 tends to use its own Python builds, while pygtk will probably want to focus on upstream python.org Windows builds...).