GNOME Bugzilla – Bug 322935
Threads are left idling
Last modified: 2018-08-17 13:49:49 UTC
Please describe the problem: If you do a LOT of async operations (a good example is using the gnome_vfs_async_load_directory()) then 'n' threads are created depending on priority and the number of operations. If you create 10 threads and all operations have finished, those 10 threads are left in the async thread pool waiting for more work. They should really be cleaned up if they are not needed after a period of time. Steps to reproduce: Actual results: Expected results: Does this happen every time? Other information:
Created attachment 55476 [details] [review] Cleans up idling threads after timeout OK, so this patch adds a timeout to every thread when it enters the thread job pool (which is there so that when new jobs come along they can be picked out of the pool instead of creating a new thread). This timeout is 15 seconds. After those 15 seconds, the thread flag 'exit_requested' is set to TRUE, and the thread is woken up to process the state change to exit normally (as it would if calling gnomevfs-shutdown()). Note: There is a programs/gnomevfs-async test application with this patch to demonstrate this working. To find out what threads are running, I use 'ps -eLf | grep gnomevfs-async'. Note: Debugging statements are turned on for the job queue module.
Created attachment 55491 [details] [review] My first take on that Make gnome-vfs use g_thread_pool (). It obsoletes gnome-vfs-thread-pool.[ch]. Make check works as expected and we get rid of a whole bunch of code that was hard to maintain.
Created attachment 55528 [details] [review] Next try, this time we killed a bunch of complicated code in job-queue as well
Created attachment 55530 [details] The whole gnome-vfs-job-queue.c file for easier patch review.
Created attachment 55533 [details] gnome-vfs-job-queue.c again. a few cleanups
I committed the GThreadPool patch to gnome-vfs HEAD.
Christian: Can't we fix this bug just by limiting the idle time using g_thread_pool_set_max_idle_time ? One or two minutes should be enough. Updating version.
That was the idea I believe. I did the work in gthreadpool.c specifically for this.
The thing is that g_thread_pool_set_max_idle_time (guint interval) will set the idle time not for a specific thread pool but for all threadpools (I am not sure if this was on purpose or was it the only choice to do it?). I am not sure it is a good idea to set the general max_idle_time for the application's threadpools in a library. Maybe I am paranoid here, but I guess nautilus and other gnome-vfs uses could as well use that function and also control gnome-vfs' threadpool. Opinions? Either way we should document it somewhere (where?).
gnome-vfs got deprecated in 2008. gnome-vfs is not under active development anymore and had its last code changes in 2011. Its codebase has been archived: https://gitlab.gnome.org/Archive/gnome-vfs/commits/master gio (in glib) and gvfs are its successors. See https://developer.gnome.org/gio/stable/ch33.html and https://people.gnome.org/~gicmo/gio-migration-guide/ for porting info. Closing this report as WONTFIX as part of Bugzilla Housekeeping to reflect reality. Feel free to open a task in GNOME Gitlab if the issue described in this task still applies to a recent + supported version of glib/gio/gvfs. Thanks!