After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 782530 - Race condition in property querying
Race condition in property querying
Status: RESOLVED FIXED
Product: gnome-bluetooth
Classification: Core
Component: general
unspecified
Other Linux
: Normal normal
: ---
Assigned To: gnome-bluetooth-general-maint@gnome.bugs
gnome-bluetooth-general-maint@gnome.bugs
: 775376 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2017-05-11 18:58 UTC by Benjamin Berg
Modified: 2018-02-21 23:42 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
lib: Rework adapter and device proxy registrations. (33.46 KB, patch)
2017-05-11 19:03 UTC, Benjamin Berg
none Details | Review
lib: Rework adapter and device proxy registrations. (33.45 KB, patch)
2017-05-16 11:48 UTC, Benjamin Berg
none Details | Review
lib: Always send "default-adapter-powered" (1.19 KB, patch)
2017-05-16 11:48 UTC, Benjamin Berg
none Details | Review
lib: Force custom icon code override to run (943 bytes, patch)
2017-05-16 11:48 UTC, Benjamin Berg
none Details | Review
lib: Remove unneeded include (720 bytes, patch)
2017-11-14 16:55 UTC, Bastien Nocera
committed Details | Review
lib: Remove never used FDO_PROPERTIES_INTERFACE constant (907 bytes, patch)
2017-11-14 16:55 UTC, Bastien Nocera
committed Details | Review
lib: Indent and document gtk_tree_store_new() call (2.18 KB, patch)
2017-11-14 16:55 UTC, Bastien Nocera
committed Details | Review
lib: Use GDBusObjectManager (16.63 KB, patch)
2017-11-14 16:55 UTC, Bastien Nocera
committed Details | Review
lib: Remove creation of intermediate GDBusProxies (4.30 KB, patch)
2017-11-14 16:55 UTC, Bastien Nocera
committed Details | Review
lib: Simplify Properties setting (5.83 KB, patch)
2017-11-14 16:55 UTC, Bastien Nocera
committed Details | Review
lib: Remove last users of "Properties" D-Bus wrapper (4.19 KB, patch)
2017-11-14 16:55 UTC, Bastien Nocera
committed Details | Review
lib: Remove fd.o Properties API glue (2.36 KB, patch)
2017-11-14 16:55 UTC, Bastien Nocera
committed Details | Review
lib: Fix possible race when creating adapters or devices (2.35 KB, patch)
2017-11-14 16:55 UTC, Bastien Nocera
committed Details | Review
lib: Always send "default-adapter-powered" (1.29 KB, patch)
2017-11-14 16:55 UTC, Bastien Nocera
committed Details | Review

Description Benjamin Berg 2017-05-11 18:58:52 UTC
Because of the method that the dbus proxies are created there is currently a race condition. When an adapter is added it can have an initial "Powered" state of off which is turned on immediately afterwards. However, the signal connect to get the update only happens later.

This can be easily reproduced on thinkpads by running "rfkill block 0" and "rfkill unblock 0" while looking at the gnome-control-center bluetooth panel.

Attaching a patch that should fix the issue. I hope I didn't totally misunderstand how the whole DBus proxy stuff is supposed to work.
Comment 1 Benjamin Berg 2017-05-11 19:03:39 UTC
Created attachment 351663 [details] [review]
lib: Rework adapter and device proxy registrations.

This fixes a race condition when a property was updated right after the
object appeared on the bus. Due to the old implementation the property
change would not be registered in that case.

This moves the code to use GDBusObjectManagerClient and using the
provided GObject properties instead of a separate proxy to avoid the
race condition.
Comment 2 Bastien Nocera 2017-05-16 09:15:35 UTC
Review of attachment 351663 [details] [review]:

The patch is unreviewable. Make the hunks bigger and try to split up the changes if at all possible.
Comment 3 Benjamin Berg 2017-05-16 11:48:04 UTC
Created attachment 351966 [details] [review]
lib: Rework adapter and device proxy registrations.

This fixes a race condition when a property was updated right after the
object appeared on the bus. Due to the old implementation the property
change would not be registered in that case.

This moves the code to use GDBusObjectManagerClient and using the
provided GObject properties instead of a separate proxy to avoid the
race condition.
Comment 4 Benjamin Berg 2017-05-16 11:48:15 UTC
Created attachment 351967 [details] [review]
lib: Always send "default-adapter-powered"

Fix sending of default-adapter-powered in the case that the default
adapter is being disabled.
Comment 5 Benjamin Berg 2017-05-16 11:48:38 UTC
Created attachment 351968 [details] [review]
lib: Force custom icon code override to run

The property setter currently contains some magic to modify the icon for
certain device types. Send a property change notification to force this
code to run after a device has been added.
Comment 6 Benjamin Berg 2017-05-16 11:49:01 UTC
There, that is all logical changes split out.
Comment 7 Bastien Nocera 2017-05-16 12:27:56 UTC
I won't have time to review this in the short-term, but a dbusmock regression test which shows the problem before, and shows it fixed afterwards would be really useful.

See those in gnome-settings-daemon (though they've been broken for a while for different reasons...):
https://git.gnome.org//browse/gnome-settings-daemon/tree/tests/gsdtestcase.py
https://git.gnome.org//browse/gnome-settings-daemon/tree/plugins/power/test.py
Comment 8 Benjamin Berg 2017-05-16 17:23:35 UTC
The patch might be hard to read as it changes all the property accessors, but really it all boils down to the two questions:
 * Is it correct to use GDBusObjectManagerClient to keep track of the objects?
 * Is it good to use GObject properties on the Adapter1/Device1 proxies (instead of manually driving a DBus properties interface proxy)?

I agree that dbusmock test would be nice in here. But I personally think adding that testing infrastructure is out of the scope for me here. In this particular case I would even say that it would be an over specific test to catch a case of synchronous API abuse.

The race is actually surprisingly obvious. The calls to *_proxy_new_for_bus_sync cause the mainloop to run which means DBus messages are processed too early:
 * InterfacesAdded is fired by bluez
 * Properties change notification for "Powered" is fired by bluez
 * adapter_added runs mainloop for adapter1_proxy_new_for_bus_sync
   - at least dbus calls for name resolving happen
 * adapter_added runs mainloop for properties_proxy_new_for_bus_sync
   - at least dbus calls for name resolving happen
 * Properties notification has been lost while running those mainloops
 * adapter_added uses cached property values from InterfacesAdded method call
 * adapter_added connects to "g-properties-changed" on the proxy

The improvements from bug #701399 made this race condition worse.

With the patch, the code becomes:
 * InterfaceAdded is fired on bluez
 * Properties change notification for "Powered" is fired by bluez
 * GLib creates the Adapter1 proxy object
 * glib sends notification about the new object
 * connect to Adapter1 happens
 * Properties are queried from Adapter1 object through GObject

There simply is no scope for a race condition as connecting/querying happens in an atomic fashion.

--

Note that there is quite a lot of cleanup potential. In particular BLUETOOTH_COLUMN_PROPERTIES and creation of the properties proxy can be removed if this doesn't constitute an API/ABI break.
Comment 9 Bastien Nocera 2017-11-14 16:55:01 UTC
Created attachment 363618 [details] [review]
lib: Remove unneeded include

Nothing is using the code in the generated bluetooth-fdo-glue.h
header.
Comment 10 Bastien Nocera 2017-11-14 16:55:07 UTC
Created attachment 363619 [details] [review]
lib: Remove never used FDO_PROPERTIES_INTERFACE constant

It was added as a constant during the BlueZ 5 port (commit 0b05349) but
was never used.
Comment 11 Bastien Nocera 2017-11-14 16:55:13 UTC
Created attachment 363620 [details] [review]
lib: Indent and document gtk_tree_store_new() call

Makes it easier to see which column has which type.
Comment 12 Bastien Nocera 2017-11-14 16:55:18 UTC
Created attachment 363621 [details] [review]
lib: Use GDBusObjectManager

Instead of our home-made ObjectManager proxy. Note that this is a
minimal port and will require more cleaning up to be up to standards.

Loosely based on patch by Benjamin Berg <bberg@redhat.com>
Comment 13 Bastien Nocera 2017-11-14 16:55:24 UTC
Created attachment 363622 [details] [review]
lib: Remove creation of intermediate GDBusProxies

We already have what we need created by the GDBusObjectManager.
Comment 14 Bastien Nocera 2017-11-14 16:55:29 UTC
Created attachment 363623 [details] [review]
lib: Simplify Properties setting

Stop setting properties on adapters or devices by making D-Bus calls
ourselves, and rely on wrapped GObject properties instead.

For the calls we're making, the only change will be the error message
coming from the generated wrapper, rather than being our own custom ones.
We weren't really able to recover from errors in the past anyway, so
this shouldn't make the overall experience any worse.

This also fixes a possible hang on exit in the Bluetooth panel.

See https://bugzilla.gnome.org/show_bug.cgi?id=789654
Comment 15 Bastien Nocera 2017-11-14 16:55:35 UTC
Created attachment 363624 [details] [review]
lib: Remove last users of "Properties" D-Bus wrapper

As those property objects were not used anywhere anymore.
Comment 16 Bastien Nocera 2017-11-14 16:55:41 UTC
Created attachment 363625 [details] [review]
lib: Remove fd.o Properties API glue

As we removed the last users of that code.
Comment 17 Bastien Nocera 2017-11-14 16:55:47 UTC
Created attachment 363626 [details] [review]
lib: Fix possible race when creating adapters or devices

When new D-Bus objects appear, listen to changes to their properties as
soon as possible to avoid missing out on events. This can happen if a
synchronous D-Bus call is made between the device appearing, and we
start listening to changes.

The window for that race is smaller now that we avoid creating
GDBusProxies and fetching properties synchronously when new adapters or
devices appear, but this close that window altogether.
Comment 18 Bastien Nocera 2017-11-14 16:55:52 UTC
Created attachment 363627 [details] [review]
lib: Always send "default-adapter-powered"

Fix sending of default-adapter-powered in the case that the default
adapter is being disabled.
Comment 19 Bastien Nocera 2017-11-14 16:58:35 UTC
Attachment 363618 [details] pushed as b5a6455 - lib: Remove unneeded include
Attachment 363619 [details] pushed as 899eb7d - lib: Remove never used FDO_PROPERTIES_INTERFACE constant
Attachment 363620 [details] pushed as 215acf6 - lib: Indent and document gtk_tree_store_new() call
Attachment 363621 [details] pushed as 17983ac - lib: Use GDBusObjectManager
Attachment 363622 [details] pushed as 9319001 - lib: Remove creation of intermediate GDBusProxies
Attachment 363623 [details] pushed as 27e88fc - lib: Simplify Properties setting
Attachment 363624 [details] pushed as 61f5ad5 - lib: Remove last users of "Properties" D-Bus wrapper
Attachment 363625 [details] pushed as afccb57 - lib: Remove fd.o Properties API glue
Attachment 363626 [details] pushed as 9c6d93a - lib: Fix possible race when creating adapters or devices
Attachment 363627 [details] pushed as dbe815a - lib: Always send "default-adapter-powered"
Comment 20 Bastien Nocera 2017-11-14 17:01:50 UTC
Which leaves us with attachment 351968 [details] [review]. I have trouble understanding in which case this could happen now. Can you file a new bug if this is still needed?
Comment 21 Benjamin Berg 2017-11-15 10:45:24 UTC
Oh, right. Attachment 351968 [details] was just my way of to not duplicate the icon override logic.
Comment 22 Benjamin Berg 2018-02-15 12:52:02 UTC
*** Bug 775376 has been marked as a duplicate of this bug. ***