After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 624546 - Modification of GDBusMessage in filter function
Modification of GDBusMessage in filter function
Status: RESOLVED FIXED
Product: glib
Classification: Platform
Component: gdbus
unspecified
Other Linux
: Normal normal
: ---
Assigned To: David Zeuthen (not reading bugmail)
gtkdev
Depends on:
Blocks:
 
 
Reported: 2010-07-16 15:13 UTC by David Zeuthen (not reading bugmail)
Modified: 2010-09-09 17:24 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description David Zeuthen (not reading bugmail) 2010-07-16 15:13:07 UTC
<davidz> desrt: so one thing I've been wondering, actually one request that came in, is whether filter functions are allowed to change the GDBusMessage they process
 desrt: clearly we already allow dropping the message
 and for incoming messages modifying it actually works
 for outgoing messages, modifying it does nothing because we've already encoded it to a blob (need to do that early - in the caller's thread)
 maybe we should change the signature of the filter function so instead of returning a gboolean we return values from an enum - then we could have DO_NOTHING, PLEASE_DROP_MESSAGE, MESSAGE_HAS_BEEN_ALTERED etc.
 it might be handy to allow changing the message

<davidz> too handy to disallow it actually
 then again allowing such things invites total hacky hacks

<desrt> davidz: interesting.

<desrt> davidz: you're suggesting the GDBusMessage be pass-by-reference?
<desrt> it's called a filter, not a map...

<davidz> desrt: well, right now we are just passing a pointer

<davidz> desrt: I'll open a bug for it

<davidz> at least the docs should state whether modification is allowed or not
Comment 1 David Zeuthen (not reading bugmail) 2010-07-16 15:17:23 UTC
See bug 621945 comment 7 for a request for this.
Comment 2 Peng Huang 2010-07-17 08:30:33 UTC
Is it possible to call the filter in default main loop for both incoming and outgoing messages?
Comment 3 David Zeuthen (not reading bugmail) 2010-07-19 14:54:28 UTC
(In reply to comment #2)
> Is it possible to call the filter in default main loop for both incoming and
> outgoing messages?

What exactly do you mean by "default main loop"? Note that gdbus needs to work in situations where the user doesn't create a mainloop _at all_; of course asynchronous methods won't work in that case - but synchronous ones definitely will. So the only mainloop you can really assume is the one in the private worker thread.

Btw, Ryan pointed out that libdbus "locks" the message [1] to prevent modifications. We could do the same.

[1]: in the dbus_message_lock() sense of the word - from the docs:

  "Allows checking that applications don't keep a reference to a message
   in the outgoing queue and change it underneath us. Messages are locked
   when they enter the outgoing queue (dbus_connection_send_message()),
   and the library complains if the message is modified while locked."
Comment 4 Peng Huang 2010-07-19 21:35:28 UTC
(In reply to comment #3)
> (In reply to comment #2)
> > Is it possible to call the filter in default main loop for both incoming and
> > outgoing messages?
> 
> What exactly do you mean by "default main loop"? Note that gdbus needs to work
> in situations where the user doesn't create a mainloop _at all_; of course
> asynchronous methods won't work in that case - but synchronous ones definitely
> will. So the only mainloop you can really assume is the one in the private
> worker thread.

Sorry. I mean call filters in default main thread instead of background worker thread for incoming and outgoing messages. I think it could fix this issue, by calling filters in g_dbus_connection_send_message_xxx functions for outgoing messages (call filters before serializing message to data). And it is more connivence for gdbus users, they don't need some thread lock anymore.
Comment 5 Peng Huang 2010-07-22 23:06:45 UTC
Currently on_worker_message_received and on_worker_message_about_to_be_sent are called in worker thread. And in on_worker_message_received will dispatch method call and signal messages to main thread and handle them in main thread. 

I am thinking if gdbus could call on_worker_message_received and on_worker_message_about_to_be_send in main thread directly. It could resolve this bug and make things easier. Do you think it is reasonable?
Comment 6 David Zeuthen (not reading bugmail) 2010-08-22 15:24:22 UTC
OK, I've now fixed this, see

 http://git.gnome.org/browse/glib/commit/?id=3ff9894826215790fdd6c8b53584f94a7172c39f

because I think it is too handy to not allow this. It's also nicer to have filter functions return symbolic constants such as NO_EFFECT, MESSAGE_ALTERED and MESSAGE_CONSUMED - avoids having the programmer remember whether TRUE meant "message unaltered" or "message consumed". So there!

NOTE: There's an API break here but the ABI should be preserved (at least dconf's GSettings backend will keep working).
Comment 7 David Zeuthen (not reading bugmail) 2010-08-22 15:30:08 UTC
(In reply to comment #5)
> Currently on_worker_message_received and on_worker_message_about_to_be_sent are
> called in worker thread. And in on_worker_message_received will dispatch method
> call and signal messages to main thread and handle them in main thread. 
> 
> I am thinking if gdbus could call on_worker_message_received and
> on_worker_message_about_to_be_send in main thread directly. It could resolve
> this bug and make things easier. Do you think it is reasonable?

This would be difficult to do because of how locking works and the fact that we can't hold the lock when running the filter functions (because the filter functions are allowed to call methods on the GDBusConnection object).

The other nice side-effect of how things currently work, that this approach would break, is that the filter function won't run until the message is actually written to the I/Otransport (which could be seconds later). In fact, one of our test cases, see test_overflow()

 http://git.gnome.org/browse/glib/tree/gio/tests/gdbus-peer.c#n1245

actually relies on this behavior.
Comment 8 David Zeuthen (not reading bugmail) 2010-09-08 22:19:43 UTC
Hi,

Ryan complained about how filters currently work and we had a chat on IRC and I think here is what we came up with

First, GDBusMessage should have a concept of being "locked", e.g. we want the following API

  GDBusMessage:locked read-only property (for GObject::notify)
  g_dbus_message_get_is_locked() C binding
  g_dbus_message_lock()

Once locked the GDBusMessage may not be modified and any attempt to set/change stuff will fail and result in warnings on stderr etc.

The idea is that we lock a GDBusMessage in functions for sending it. Typically g_dbus_connection_send_message() (including convenience like g_dbus_connection_call() and g_dbus_proxy_call())

Additionally, we provide a copy function

 GDBusMessage *g_dbus_message_copy (GDBusMessage *message);

that returns a *new object*. It's actually deep copy but it can be cheap insofar that we can just ref the GVariant instances. This works because GVariant instances are more-or-less (e.g. modulo ref-count) immutable. Should also reset the serial

This is pretty much *exactly* how libdbus' DBusMessage works.

--

With g_dbus_message_copy() we can now change the filter function from

  GDBusMessageFilterResult
  filter_function (GDBusConnection *connection,
                   GDBusMessage *message,
                   gboolean incoming,
                   gpointer user_data);

to

  GDBusMessage *
  filter_function (GDBusConnection *connection,
                   GDBusMessage *message,
                   gboolean incoming,
                   gpointer user_data);

where the trivial (non-modifying) filter just returns @message and filters that wants to modify @message do

  GDBusMessage *
  filter_function (GDBusConnection *connection,
                   GDBusMessage *message,
                   gboolean incoming,
                   gpointer user_data)
  {
    GDBusMessage *copy;

    copy = g_dbus_message_copy (message);
    g_object_unref (message);

    /* modify copy */    

    return copy;
  }
 
or if you want to drop a message you just return NULL.

Ryan, is this close to what you had in mind?
Comment 9 Allison Karlitskaya (desrt) 2010-09-08 23:08:35 UTC
it's exactly what i had in mind, assuming the following is also true:


  1) incoming messages are locked as they are taken off the wire

  2) messages are locked as soon as (ie: in the calling thread) 
     g_dbus_connection_send_message() is called
Comment 10 David Zeuthen (not reading bugmail) 2010-09-09 16:02:10 UTC
OK, committed the first part here

 http://git.gnome.org/browse/glib/commit/?id=67a00658eadfd99ffd1be8cb5a7387e3d77e63a7

Now to actually use this.
Comment 11 David Zeuthen (not reading bugmail) 2010-09-09 17:24:21 UTC
Second part

 http://git.gnome.org/browse/glib/commit/?id=c3371efcaa47b03941c6c8148687b0a21d18dfbe

Done.