Bug 787915 – watchdog: race between trigger and stop causes deadlock

After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.

Bug 787915 - watchdog: race between trigger and stop causes deadlock


Summary:	watchdog: race between trigger and stop causes deadlock


Status:	RESOLVED OBSOLETE

Product:	GStreamer
Classification:	Platform
Component:	gst-plugins-bad
Version:	1.12.2
Hardware:	Other Linux

Importance:	Normal normal
Target Milestone:	git master
Assigned To:	GStreamer Maintainers
QA Contact:	GStreamer Maintainers

URL:
Whiteboard:

Depends on:
Blocks:

Reported:	2017-09-19 18:37 UTC by Simon Descarries
Modified:	2018-11-03 14:13 UTC

See Also:
GNOME target:	---
GNOME version:	---

Attachments
Proposed path in git format (1.24 KB, patch) 2017-09-19 19:17 UTC, Simon Descarries	none	Details \| Review

Description Simon Descarries 2017-09-19 18:37:06 UTC

There's a potential deadlock when a state change occurs while a timeout is triggered, the main thread holds the element lock and waits for the worker thread to finish, but the worker is stuck waiting on the element lock to be able to issue its error message on the bus.

Backtraces:

+ Trace 237988

Thread 4 (Thread 0x7fa4c37fd700 (LWP 54848))

#0 pthread_join
from /lib64/libpthread.so.0
#1 g_system_thread_wait
from /usr/lib/libglib-2.0.so.0
#2 g_thread_join
from /usr/lib/libglib-2.0.so.0
#3 gst_watchdog_stop
at gstwatchdog.c line 301
#4 gst_base_transform_activate
at gstbasetransform.c line 2427
#5 gst_base_transform_sink_activate_mode
at gstbasetransform.c line 2447
#6 activate_mode_internal
at gstpad.c line 1179
#7 gst_pad_set_active
at gstpad.c line 1080
#8 activate_pads
at gstelement.c line 2831
#9 gst_iterator_fold
at gstiterator.c line 617
#10 iterator_activate_fold_with_resync
at gstelement.c line 2855
#11 gst_element_pads_activate
at gstelement.c line 2899
#12 gst_element_change_state_func
at gstelement.c line 2965
#13 gst_watchdog_change_state
at gstwatchdog.c line 394
#14 gst_element_change_state
at gstelement.c line 2743
#15 gst_element_continue_state
at gstelement.c line 2451
#16 gst_element_change_state
at gstelement.c line 2782
#17 gst_element_set_state_func
at gstelement.c line 2697
#18 gst_element_set_state
at gstelement.c line 2598
#0 __lll_lock_wait
from /lib64/libpthread.so.0
#1 _L_lock_975
from /lib64/libpthread.so.0
#2 pthread_mutex_lock
from /lib64/libpthread.so.0
#3 g_mutex_lock
from /usr/lib/libglib-2.0.so.0
#4 gst_element_post_message_default
at gstelement.c line 1772
#5 gst_element_post_message
at gstelement.c line 1826
#6 gst_element_message_full_with_details
at gstelement.c line 1962
#7 gst_element_message_full
at gstelement.c line 2001
#8 gst_watchdog_trigger
at gstwatchdog.c line 192
#9 g_timeout_dispatch
from /usr/lib/libglib-2.0.so.0
#10 g_main_dispatch
from /usr/lib/libglib-2.0.so.0
#11 g_main_context_dispatch
from /usr/lib/libglib-2.0.so.0
#12 g_main_context_iterate
from /usr/lib/libglib-2.0.so.0
#13 g_main_loop_run
from /usr/lib/libglib-2.0.so.0
#14 gst_watchdog_thread
at gstwatchdog.c line 178




Proposed Fix:

Simply unlock the object before joining the worker thread



--- gst-plugins-bad/gst/debugutils/gstwatchdog.c
+++ gst-plugins-bad/gst/debugutils/gstwatchdog.c
@@ -297,6 +297,7 @@
       NULL);
   g_source_attach (quit_source, watchdog->main_context);
   g_source_unref (quit_source);
+  GST_OBJECT_UNLOCK (watchdog);
 
   g_thread_join (watchdog->thread);
   watchdog->thread = NULL;
@@ -307,7 +308,6 @@
   g_main_context_unref (watchdog->main_context);
   watchdog->main_context = NULL;
 
-  GST_OBJECT_UNLOCK (watchdog);
   return TRUE;
 }

Comment 1 Tim-Philipp Müller 2017-09-19 18:43:41 UTC

Thanks for the bug report and the patch!

Could you attach the patch in "git format-patch" format please?

https://gstreamer.freedesktop.org/documentation/contribute/index.html#how-to-submit-patches

Comment 2 Simon Descarries 2017-09-19 19:17:03 UTC

Created attachment 360079 [details] [review]
Proposed path in git format

Comment 3 GStreamer system administrator 2018-11-03 14:13:41 UTC

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/612.