GNOME Bugzilla – Bug 719362
RFC: add coroutines
Last modified: 2018-05-24 16:07:50 UTC
Coroutines are used in various glib projects in C to simplify asynchronous and concurrent programming without requiring threads and avoiding synchronization issues. This implementation is based on the gtk-vnc & qemu implementation written by Anthony Liguori <anthony@codemonkey.ws> and rewritten by Kevin Wolf <kwolf@redhat.com> to use setjmp()/longjmp() instead of the more expensive swapcontext(). I adopted coroutines in spice-gtk too. I made similar improvements independently there, so I think it's time that those 3 similar implementations become a single supported glib API that can be shared by those projects and others. Furthermore, sharing an implementation could allow sharing future gdb debugging helpers, and tools. For example, some static analysis and CPS conversion tools are being worked on for qemu with interesting results (http://gabriel.kerneis.info/research/files/qemu-cpc.pdf) The propsoed GLib implementation differs slightly from qemu. It aims to be safer (do not free the coroutine automatically on exit for example) and more generic (allows to send and return value between contexts). Contrary to qemu, there is no coroutine "recycle pool", since it can easily be implemented on top and is not considered useful enough in general. TODO (before merge): - add coroutine locks support (from qemu) and/or spice-gtk condition - add thread and win fibers support - configure switch to choose implementation - propose other gobject/gio helpers from spice-gtk (later)
Created attachment 262863 [details] [review] RFC: add coroutines
Created attachment 262864 [details] [review] RFC: add coroutines (and tests) Coroutines are used in various glib projects in C to simplify asynchronous and concurrent programming without requiring threads and avoiding synchronization issues. This implementation is based on the gtk-vnc & qemu implementation written by Anthony Liguori <anthony@codemonkey.ws> and rewritten by Kevin Wolf <kwolf@redhat.com> to use setjmp()/longjmp() instead of the more expensive swapcontext(). I adopted coroutines in spice-gtk too. I made similar improvements independently there, so I think it's time that those 3 similar implementations become a single supported glib API that can be shared by those projects and others. Furthermore, sharing an implementation could allow sharing future gdb debugging helpers, and tools. For example, some static analysis and CPS conversion tools are being worked on for qemu with interesting results (http://gabriel.kerneis.info/research/files/qemu-cpc.pdf) The propsoed GLib implementation differs slightly from qemu. It aims to be safer (do not free the coroutine automatically on exit for example) and more generic (allows to send and return value between contexts). Contrary to qemu, there is no coroutine "recycle pool", since it can easily be implemented on top and is not considered useful enough in general. TODO (before merge): - add coroutine locks support (from qemu) and/or spice-gtk condition - add thread and win fibers support - configure switch to choose implementation - propose other gobject/gio helpers from spice-gtk (later)
see also bug 565501 which was eventually WONTFIXed.
(In reply to comment #3) > see also bug 565501 which was eventually WONTFIXed. Thanks, I forgot that bug. The proposed API was higher level, requiring gobject/gio (that would not be acceptable for qemu), but it could eventually be written on top, same for GTask/iris. From all the arguments in bug 565501, I think the main one is portability concerns (for the rest, if you don't want it, don't use it). But since there is a fallback gthread implementation anyway, and most platform support makecontext or winfibers, I don't see it as a big issue, even if the thread implementation is slower. People using coroutine should be aware of the limitations and issues. Just like programming with threads (or allocating memory etc...). It should be enough to show that 3 important projects (using glib) are succesfully using coroutine to at least consider this a second time.
(In reply to comment #4) > > From all the arguments in bug 565501, I think the main one is portability > concerns (for the rest, if you don't want it, don't use it). No, the argument is rather: if you don't want it, don't put yourself in the position to maintain it. ie why should we accept the maintainance burden for yet another approach to parallelism when we already have many perfectly good offerings in glib ?!
(In reply to comment #2) > Furthermore, sharing an implementation could allow sharing future gdb > debugging helpers, and tools. You could unify them into a "copylib" first though. Try porting qemu to your new implementation?
(In reply to comment #5) > (In reply to comment #4) > > > > > From all the arguments in bug 565501, I think the main one is portability > > concerns (for the rest, if you don't want it, don't use it). > > No, the argument is rather: if you don't want it, don't put yourself in the > position to maintain it. > > ie why should we accept the maintainance burden for yet another approach to > parallelism when we already have many perfectly good offerings in glib ?! The main point is to share this common burden from those glib projects who rely on it, avoiding duplication of effort and sharing further tools and abstractions. Coroutine is an interesting alternative to threads, avoiding a lot of issues due to their cooperative nature, and allowing easier communication. They are much more convenient than async/callback. They can also be viewed as complementary to threads (you can have several coroutines per thread). Furthermore, they are arguably simpler, lighter and faster.
(In reply to comment #6) > (In reply to comment #2) > > > Furthermore, sharing an implementation could allow sharing future gdb > > debugging helpers, and tools. > > You could unify them into a "copylib" first though. Try porting qemu to your > new implementation? That is the plan: http://lists.nongnu.org/archive/html/qemu-devel/2013-11/msg03366.html I wanted some initial feedback before going further.
Created attachment 263260 [details] [review] RFC: add coroutines
Created attachment 263261 [details] [review] RFC: add coroutines locks Add mutex and rw lock based on Qemu implementation.
Created attachment 263262 [details] [review] docs: add coroutines section
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/glib/issues/797.