I've updated to new versions.
gnome-settings-daemon 2.30.1
x11-server-xorg 1.7.6.902
setxkbmap 1.1.0

The crash is still happening, however it happens even without setxkbmap. Just after the plugging in the usb keyboard, it crashes. Here is the backtrace:
Program received signal SIGSEGV, Segmentation fault.
0xb77549a5 in __libc_free (mem=0x4c9ffbf4) at malloc.c:3724
3724	  ar_ptr = arena_for_chunk(p);
(gdb) bt

+ Trace 221615

The atom properties are:
_XKB_RULES_NAMES(STRING) = "evdev", "evdev", "fr", "", "compose:rwin"
XKLAVIER_STATE(INTEGER) = 0, 2

Any idea whether it comes from g-s-d or libxklavier, or libgnomekbdui?

Also, as a note I cannot reproduce it on a computer with x86-64, it happens only on my 32 bits laptop.

More about it: it requires to have at least two layout activated in the keyboard preference.

Running g-s-d with --debug --nodaemon didn't bring much info. XKL_DEBUG seemed to have no effect on what was printed on the console. On my x86-64, it doesn't happen, but I see that there is only one group listed in the applet any way (although I've got 4 in the preferences).

The bug is apparently when gkbd_status_render_cairo() destroys ln2cnt_map, the second layout_name cannot be freed. And indeed, ln2cnt_map looks a bit suspicious:
$5 = 0x8437d20 = {
  [0x842cec0 "Fra"] = 0x1,
  [
    0x48deebf4 "\211\370\213\\$\020\213t$\024\213|$\030\203\304\034\303f\220\215\203\063\243\376\377\061\377\211D$\b\215\203\025\350\376\377\211D$\004\215\203BP\376\377\211\004$\350\214_\376\377\353\306f\220\203\354,\211\\$\034\350\033n\376\377\201øC\002"] = 0x1
}

While the first layout name "Fra" looks fine, with a normal address, the second one is very strange... and that's were glibc fails to free.

I eventually got some debug messages from XKL, with export XKL_DEBUG=1000:
:
[1272626886,400,xklavier_evt.c:xkl_engine_process_property_evt/] 	The property '_XKB_RULES_NAMES' changed for 66
[1272626886,150,xklavier.c:xkl_engine_reset_all_info/] 	Resetting all the cached info, reason: [New value of *_NAMES_PROP_ATOM on root window]
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	found 2 groups
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Group 1 has name [France]
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Group 0 has name [Afghanistan]
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[31] is Caps Lock
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[30] is Num Lock
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[29] is Scroll Lock
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[28] is Compose
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[27] is Kana
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[26] is Sleep
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[25] is Suspend
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[24] is Mute
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[23] is Misc
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[22] is Mail
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[21] is Charging
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[20] is Shift Lock
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[19] is Group 2
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[18] is Mouse Keys
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[17] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[16] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[15] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[14] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[13] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[12] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[11] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[10] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[9] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[8] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[7] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[6] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[5] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[4] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[3] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[2] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[1] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Indicator[0] is 
[1272626886,200,xklavier_xkb.c:xkl_xkb_load_all_info/] 	Real indicators are 7FF
[1272626886,100,gkbd-status.c:gkbd_status_kbd_cfg_callback/] 	XKB configuration changed on X Server - reiniting...
[1272626886,150,gkbd-keyboard-config.c:gkbd_keyboard_config_load_from_x_current/] 	Copying config from X(current)
[1272626886,150,gkbd-keyboard-config.c:gkbd_keyboard_config_copy_from_xkl_config/] 	Loaded Kbd model: [evdev]
[1272626886,150,gkbd-keyboard-config.c:gkbd_keyboard_config_copy_from_xkl_config/] 	Loaded Kbd layout (with variant): [fr]
[1272626886,150,gkbd-keyboard-config.c:gkbd_keyboard_config_copy_from_xkl_config/] 	Loaded Kbd option: [compose][compose:rwin]
[1272626886,100,gkbd-desktop-config.c:gkbd_desktop_config_get_lv_descriptions/] 	ids: [fr][(null)]
[1272626886,100,gkbd-desktop-config.c:gkbd_desktop_config_get_lv_descriptions/] 	description: [Fra][Frankrijk][][]
[1272626886,150,gkbd-status.c:gkbd_status_prepare_drawing/] 	Image 0 created -> 0x9611d70[23x25], alpha: 1
*** glibc detected *** /usr/lib/gnome-settings-daemon: free(): invalid pointer: 0x48deebf4 ***

I could also reproduce the bug on x86-64: first you must remove all the layouts, and readd one via the keyboard preferences, then plug a keyboard in.

Also, the bug triggers as well in gkbd-indicator-test, and gkbd-status-test. You just need to have: multiple layouts, and have plugged in a keyboard before hand. The backtrace of the former is:

+ Trace 221646

Ok, so apparently the problem is that xklavier, when using the xkb engine, returns that there are 2 groups, via xkl_engine_get_num_groups(), however it uses the _XKB_RULES_NAMES atom to list all the groups, and it contains only one group after a keyboard is plugged in:
$ xprop -root | grep XKB
_XKB_RULES_NAMES_BACKUP(STRING) = "evdev", "evdev", "fr", "", "compose:rwin"
_XKB_RULES_NAMES(STRING) = "evdev", "evdev", "fr", "", "compose:rwin"

I'm not sure whether this is a bug in xklavier, xkb, or g-s-d which wrongly communicates with xkb for resetting the groups. Nevertheless, I already have a one line patch for libgnomekdb to avoid crashing in this situation.

Created attachment 160096 [details] [review]
Do not try to read outside of the group array

Tossing to Sergey...

One more remark, this happens in g-s-d only if the keyboard engine does not support device discovery (eg: xklavier/xbd without xinput). In case of device discovery, gsd_keyboard_new_device() is called and everything is fine again.

Operations without xinput should work fine, I expected. But it seems you do not have properly functioning XKB as such! Is that so? There is condition before that "else":

xkl_engine_get_features (engine) & XKLF_MULTIPLE_LAYOUTS_SUPPORTED

How happened it returned false? Would you mind running "test_config -d 150 -g" in libxklavier/tests?

> returns that there are 2 groups, via xkl_engine_get_num_groups(), however it
uses the _XKB_RULES_NAMES atom to list all the groups, and it contains only one
group after a keyboard is plugged in:
This leads me to conclusion that your Xorg code is broken. Your X provide inconsistent information - that is the root cause of the issue.

I am not sure if we should code for broken X behaviour.

Yes, it looks like it could be X which is broken. However, I haven't much change in the 1.8 server for xkb, so if there is a bug, it's likely still laying around (and so it's worth fixing).

I'm now looking more precisely at libgnomekdb's gkbd_status_kbd_cfg_callback() which is actually the function triggering this strange state in xkb.

Here is the output of testconfig when this bug happens. Note that as the backup config is applied, the number of groups is synchronized again (and therefore the effects of the bug disappear).

$ ./test_config -d 150 -g
Get the config
[1272758971,150,xklavier.c:xkl_engine_one_switch_to_secondary_group_performed/] 	Resetting allow_one_switch_to_secondary_group flag
[1272758971,150,xklavier.c:xkl_engine_constructor/] 	Trying all backends:
[1272758971,150,xklavier.c:xkl_engine_constructor/] 	Trying XKB backend
[1272758971,100,xklavier_config_xkb.c:xkl_xkb_multiple_layouts_supported/] 	!!! Checking multiple layouts support
[1272758971,100,xklavier_config.c:xkl_engine_get_ruleset_name/] 	Rules set: [evdev]
[1272758971,100,xklavier_config_xkb.c:xkl_xkb_multiple_layouts_supported/] 	!!! Multiple layouts ARE supported
[1272758971,150,xklavier.c:xkl_engine_constructor/] 	Actual backend: XKB
[1272758971,000,test_config.c:main/] 	Xklavier initialized
[1272758971,100,xklavier_config.c:xkl_engine_get_ruleset_name/] 	Rules set: [evdev]
[1272758971,100,xklavier_config.c:xkl_config_registry_load_from_file/] 	Loading XML registry from file /usr/share/X11/xkb/rules/evdev.xml
[1272758971,100,xklavier_config.c:xkl_config_registry_load_from_file/] 	Loading XML registry from file /usr/share/X11/xkb/rules/evdev.extras.xml
[1272758971,000,test_config.c:main/] 	Xklavier registry loaded
[1272758971,000,test_config.c:main/] 	Backend: [XKB]
[1272758971,000,test_config.c:main/] 	Supported features: 0x0F
[1272758971,000,test_config.c:main/] 	Max number of groups: 4
[1272758971,000,test_config.c:main/] 	Number of groups: 3
[1272758971,000,test_config.c:main/] 	Got config from the server
  model: [evdev]
  layouts:
  layouts:
  0: [fr]
  variants:
  options:
  0: [compose:rwin]
[1272758971,000,test_config.c:main/] 	Number of groups: 3
[1272758971,000,test_config.c:main/] 	Got config from the backup
  model: [evdev]
  layouts:
  layouts:
  0: [fr]
  variants:
  options:
  0: [compose:rwin]
[1272758971,100,xklavier_config.c:xkl_engine_get_ruleset_name/] 	Rules set: [evdev]
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	tmp XKB/XKM file names: [/tmp/filedq0JTi]/[/tmp/filexT9901]
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	xkb_keymap {
        xkb_keycodes  { include "evdev+aliases(azerty)" };
        xkb_types     { include "complete" };
        xkb_compat    { include "complete" };
        xkb_symbols   { include "pc+fr+inet(evdev)+terminate(ctrl_alt_bksp)+compose(rwin)" };
        xkb_geometry  { include "pc(pc104)" };
};
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	Return status of 8542 (well, started 8542): 0
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	Hacked the kbddesc - set the display...
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	Loaded /usr/bin/xkbcomp output as XKM file, got 0 (comparing to 127)
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	Loaded legal keymap
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	Activating it...
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	Updating the keyboard...
[1272758971,100,xklavier_config.c:xkl_engine_get_ruleset_name/] 	Rules set: [evdev]
[1272758971,000,test_config.c:main/] 	The backup configuration restored
[1272758971,100,xklavier_config.c:xkl_engine_get_ruleset_name/] 	Rules set: [evdev]
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	tmp XKB/XKM file names: [/tmp/file1XyzfR]/[/tmp/filenkoE4z]
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	xkb_keymap {
        xkb_keycodes  { include "evdev+aliases(azerty)" };
        xkb_types     { include "complete" };
        xkb_compat    { include "complete" };
        xkb_symbols   { include "pc+fr+inet(evdev)+terminate(ctrl_alt_bksp)+compose(rwin)" };
        xkb_geometry  { include "pc(pc104)" };
};
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	Return status of 8543 (well, started 8543): 0
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	Hacked the kbddesc - set the display...
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	Loaded /usr/bin/xkbcomp output as XKM file, got 0 (comparing to 127)
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	Loaded legal keymap
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	Activating it...
[1272758971,150,xklavier_config_xkb.c:xkl_config_get_keyboard/] 	Updating the keyboard...
[1272758971,100,xklavier_config.c:xkl_engine_get_ruleset_name/] 	Rules set: [evdev]
[1272758971,000,test_config.c:main/] 	Reverting the configuration change
[1272758971,000,test_config.c:main/] 	Xklavier registry freed
[1272758971,000,test_config.c:main/] 	Xklavier terminating
closing display: 0x9be75d8

I am confused. Look:

[1272758971,100,xklavier_config_xkb.c:xkl_xkb_multiple_layouts_supported/]    
!!! Multiple layouts ARE supported

And your patch only affects the code that works when multiple layouts are not supported (that code is only for VERY old XFree servers - and, perhaps, some proprietary crap). How can it fix anything??

Actually, what is your XKB configuration (in xorg.conf or whatever is used these days)? In XKL_RULES_NAMES you have one group (fr) while XKB reports you have 3 groups. This is outrageous;)

Could you please attach xkbcomp :0 -xkb out.xkb

Duh... you are totally right, this patch is useless. Strangely, my own compiled version of libgnomekbd handles the discrepancy better than the version provided in my distro (Mandriva cooker): it displays "??" for the groups after the first one, instead of crashing while trying to free a wrong pointer. Maybe it's due to the cflags being "-O0 -g".

Concerning xkb, the default config is defined in xorg.conf as:
    Driver "kbd"
    Option "XkbModel" "pc105"
    Option "XkbLayout" "fr"
    Option "XkbOptions" "compose:rwin"
That's the defaults put by my distrib. I guess I could try to remove the input section completely and see whether it changes something. (In particular, the usage of the old kbd driver looks suspicious, but I believe I've already managed to reproduce this bug on a computer with only evdev.)

However, as soon as gnome starts, the xkb config follows my preferences (defined in .gconf/desktop/gnome/peripherals/keyboard/kbd), so as soon as I have access to a terminal, I get this values:
_XKB_RULES_NAMES(STRING) = "evdev", "dellusbmm", "fr,af,af", ",", "compose:rwin,grp:alts_toggle"
_XKB_RULES_NAMES_BACKUP(STRING) = "evdev", "evdev", "fr", "", "compose:rwin"

Everything is fine, with xkb reporting 3 groups and the atom list three groups, until I plug in a new keyboard, then there is a change in configuration reported by the server, libgnomekdb's gkbd_status_kbd_cfg_callback() is called, and everything goes wrong: 3 groups reported, but the xkb atom is back to the origin.

At any moment, the output of xkbcomp stays the same. I'll attach it.

Created attachment 160149 [details]
xkbcomp :0 -xkb out.xkb

Review of attachment 160096 [details] [review]:

This is wrong as it doesn't change anything to the behaviour (it touches a code path never reached).

Ok, here is the answer. 

1. Xorg initializes XKB correctly. 1 group
2. Gnome configures XKB correctly. 3 groups
3. You plug new kbd.
4. Xorg reconfigures it - with 1 group (same initial config as in #1)
5. X does NOT change XKB_RULES_NAMES. Bug in Xorg. Please file it.
6. Xorg notifies g-s-d about changed configuration. g-s-d crashes because of mismatch between actually available groups - and number of groups in XKB_RULES_NAMES. Invalid environment.

So: 1. Please enable XInput - that's the best choice. It will enforce XKB configuration update for plugged-in kbd.
2. Please ask Xorg guys to fix the XKB_RULES_NAMES handling

Oh.... I'm starting to understand. And it is not so clear anymore that it is a bug in Xorg. Basically, the difficulty is that layouts are not necessarily system wide... but can be per-device. 

So a device can be assigned two layouts (eg: "fr,af"), while another one gets only one (eg: "nl"). The problem is that _XKB_RULES_NAMES reflects only the _last_ assignment. So it is updated, but in the example case it is necessarily wrong, either with the first device or the second device. However, XkbGetControls() returns info from the core keyboard. Therefore the discrepancy happens.

In my distro, when a new keyboard is inserted, hal detects it, and a script from my distro is executed to assign the default system layout. It changes only this specific device. So all the devices have the two layouts as specified by gnome, excepted the new one which only has one layout.

This can also happens if the user changes manually the layout of a device, as I described in the very first comment of this bug report (and it makes sense now why these 2 different events lead to the same crash). So it should be fairly reproducible by selecting multiple layouts in gnome, and then doing on a specific keyboard device this:
setxkbmap -device 3 nl

I think it would be fine if g-s-d was showing only the layouts of the core pointer (it's not going to show a different icon for every single keyboard device!). However, it should not get confused by the fact that some devices have less layouts.

Where do you think this can be solved? In xklavier?

There is a problem here. libxklavier is using XKB_RULES_NAMES in order to find the currently configured layouts. Unfortunately, for a moment there is no way to find layouts on per-device basis (ok, I do not know it).

Perhaps, one day libxklavier has to be tought to use different layouts per-device (once xkb allows it). But for a moment, it should handle same set of layouts for all plugged in devices. 

And in any case, it is somewhat unfair to require support for hotplug in strange environments where xinput is disabled;)

PS Formally, libxklavier is not a part of gnome. Do not confuse with libgnomekbd;)

For the info, I found this bug report in xorg to have one  XKB_RULES_NAMES per device... hopefully the patch will be applied.

https://bugs.freedesktop.org/show_bug.cgi?id=21669

(In reply to comment #18)
> There is a problem here. libxklavier is using XKB_RULES_NAMES in order to find
> the currently configured layouts. Unfortunately, for a moment there is no way
> to find layouts on per-device basis (ok, I do not know it).

this is a rather optimistic approach anyway, since the property is controlled by the clients. except for the initial device add event, the other updates are just made by clients only. So you could have a client that changes the property without changing the actual layout and vice versa.

This wouldn't change, even if we have per-device properties. The whole lot should be integrated into the protocol (the mystical XKB2), but we're not even there to get started yet :(
 
> Perhaps, one day libxklavier has to be tought to use different layouts
> per-device (once xkb allows it). But for a moment, it should handle same set of
> layouts for all plugged in devices. 

fwiw, xkb does support different maps per device, all the functions take a device ID. it's really just the property that is the problem here.

> it's really just the property that is the problem here.
I like the "just" word in your statement ;)

I'm now confused: as of the last xserver release (1.8), is there any way to retrieve all the information found in XKB_RULES_NAMES (assuming there is only one keyboard and it is synchronised) by a call to the server? For example does XkbGetKeyboardByName() permit this?

Nope. That's all about "mystical XKB2"