Bug 656751 – deadlock in decodebin2

After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.

Bug 656751 - deadlock in decodebin2


Summary:	deadlock in decodebin2


Status:	RESOLVED INVALID

Product:	GStreamer
Classification:	Platform
Component:	gst-plugins-base
Version:	git master
Hardware:	Other Linux

Importance:	Normal normal
Target Milestone:	git master
Assigned To:	GStreamer Maintainers
QA Contact:	GStreamer Maintainers

URL:
Whiteboard:

Depends on:
Blocks:

Reported:	2011-08-17 14:59 UTC by Youness Alaoui
Modified:	2011-08-19 08:43 UTC

See Also:
GNOME target:	---
GNOME version:	---

Description Youness Alaoui 2011-08-17 14:59:39 UTC

I'm working on HLS streaming and when I do a seek in an HLS stream, I can trigger a deadlock in decodebin2. It's not easy to reproduce with GST_DEBUG enabled, but I can reproduce it with enough tries (race condition).
Basically what happens is that tsdemux will switch streams by adding pads and sending an EOS on the previous pads, this results in the deadlock where the eos handlers (for audio and video) will get triggered.. one of them takes the chain/group/expose lock and tries to remove the group/chain but blocks because it tries to get the stream pad lock which has already been taken by the gst_pad_send_event that is sending the EOS on the other thread, which itself is blocked waiting for the chain lock taken by the first thread...

Have a look at the backtrace for the two threads involved :

+ Trace 228117

Thread 8 (Thread 0x9e7feb70 (LWP 31178))

#0 _dl_sysinfo_int80
from /lib/ld-linux.so.2
#1 __lll_lock_wait
from /lib/libpthread.so.0
#2 _L_lock_600
from /lib/libpthread.so.0
#3 pthread_mutex_lock
from /lib/libpthread.so.0
#4 drain_and_switch_chains
at gstdecodebin2.c line 2973
#5 gst_decode_pad_handle_eos
at gstdecodebin2.c line 3042
#6 source_pad_event_probe
at gstdecodebin2.c line 3570
#7 source_pad_event_probe
at gstdecodebin2.c line 3554
#8 gst_marshal_BOOLEAN__POINTER
at gstmarshal.c line 586
#9 g_closure_invoke
from /lib/libgobject-2.0.so.0
#10 ??
from /lib/libgobject-2.0.so.0
#11 g_signal_emitv
from /lib/libgobject-2.0.so.0
#12 gst_pad_emit_have_data_signal
at gstpad.c line 4144
#13 gst_pad_push_event
at gstpad.c line 5237
#14 gst_proxy_pad_event_default
at gstghostpad.c line 143
#15 gst_pad_send_event
at gstpad.c line 5399
#16 gst_pad_push_event
at gstpad.c line 5251
#17 gst_ffmpegdec_sink_event
at gstffmpegdec.c line 2444
#18 gst_pad_send_event
at gstpad.c line 5399
#19 gst_pad_push_event
at gstpad.c line 5251
#20 gst_single_queue_push_one
at gstmultiqueue.c line 1106
#21 gst_multi_queue_loop
at gstmultiqueue.c line 1314
#22 gst_task_func
at gsttask.c line 243
#23 default_func
at gsttaskpool.c line 70
#24 ??
from /lib/libglib-2.0.so.0
#25 ??
from /lib/libglib-2.0.so.0
#26 start_thread
from /lib/libpthread.so.0
#27 clone
from /lib/libc.so.6

Thread 7 (Thread 0x9dffdb70 (LWP 31179))

#0 _dl_sysinfo_int80
from /lib/ld-linux.so.2
#1 __lll_lock_wait
from /lib/libpthread.so.0
#2 _L_lock_600
from /lib/libpthread.so.0
#3 pthread_mutex_lock
from /lib/libpthread.so.0
#4 g_static_rec_mutex_lock
from /lib/libglib-2.0.so.0
#5 post_activate
at gstpad.c line 657
#6 post_activate
at gstpad.c line 650
#7 gst_pad_activate_push
at gstpad.c line 960
#8 gst_ghost_pad_activate_push_default
at gstghostpad.c line 893
#9 gst_pad_activate_push
at gstpad.c line 953
#10 gst_pad_set_active
at gstpad.c line 715
#11 pad_removed_cb
at gsturidecodebin.c line 1123
#12 g_cclosure_marshal_VOID__OBJECT
from /lib/libgobject-2.0.so.0
#13 g_closure_invoke
from /lib/libgobject-2.0.so.0
#14 ??
from /lib/libgobject-2.0.so.0
#15 g_signal_emit_valist
from /lib/libgobject-2.0.so.0
#16 g_signal_emit
from /lib/libgobject-2.0.so.0
#17 gst_element_remove_pad
at gstelement.c line 881
#18 gst_decode_chain_free_internal
at gstdecodebin2.c line 2498
#19 gst_decode_group_free_internal
at gstdecodebin2.c line 2614
#20 gst_decode_chain_free_internal
at gstdecodebin2.c line 2432
#21 gst_decode_group_free_internal
at gstdecodebin2.c line 2614
#22 gst_decode_group_hide
at gstdecodebin2.c line 2687
#23 drain_and_switch_chains
at gstdecodebin2.c line 3002
#24 gst_decode_pad_handle_eos
at gstdecodebin2.c line 3042
#25 source_pad_event_probe
at gstdecodebin2.c line 3570
#26 source_pad_event_probe
at gstdecodebin2.c line 3554
#27 gst_marshal_BOOLEAN__POINTER
at gstmarshal.c line 586
#28 g_closure_invoke
from /lib/libgobject-2.0.so.0
#29 ??
from /lib/libgobject-2.0.so.0
#30 g_signal_emitv
from /lib/libgobject-2.0.so.0
#31 gst_pad_emit_have_data_signal
at gstpad.c line 4144
#32 gst_pad_push_event
at gstpad.c line 5237
#33 gst_proxy_pad_event_default
at gstghostpad.c line 143
#34 gst_pad_send_event
at gstpad.c line 5399
#35 gst_pad_push_event
at gstpad.c line 5251
#36 gst_faad_sink_event
at gstfaad.c line 682
#37 gst_pad_send_event
at gstpad.c line 5399
#38 gst_pad_push_event
at gstpad.c line 5251
#39 gst_pad_event_default_dispatch
at gstpad.c line 3556
#40 gst_pad_event_default
at gstpad.c line 3640
#41 gst_base_parse_sink_event
at gstbaseparse.c line 889
#42 gst_pad_send_event
at gstpad.c line 5399
#43 gst_pad_push_event
at gstpad.c line 5251
#44 gst_single_queue_push_one
at gstmultiqueue.c line 1106
#45 gst_multi_queue_loop
at gstmultiqueue.c line 1314
#46 gst_task_func
at gsttask.c line 243
#47 default_func
at gsttaskpool.c line 70
#48 ??
from /lib/libglib-2.0.so.0
#49 ??
from /lib/libglib-2.0.so.0
#50 start_thread
from /lib/libpthread.so.0
#51 clone
from /lib/libc.so.6

Comment 1 Youness Alaoui 2011-08-17 22:25:04 UTC

Found the issue. The deadlock situation was caused by decodebin2 draining the decode group before it was completely drained. This is why it locked trying to remove the pad which itself was sending an EOS.
This is caused by this commit :
http://cgit.freedesktop.org/~bilboed/gst-plugins-base/commit/?id=f65fede141f85924071dacccac92e0ee8a628a1e
which, apparently, is not in the master git repository yet, so I'm marking this bug as invalid.
I'll contact bilboed with the fix to this bug, which is actually quite simple :
diff --git a/gst/playback/gstdecodebin2.c b/gst/playback/gstdecodebin2.c
index 180c170..6d0c7e6 100644
--- a/gst/playback/gstdecodebin2.c
+++ b/gst/playback/gstdecodebin2.c
@@ -2944,7 +2944,7 @@ drain_and_switch_group (GstDecodeGroup * group, GstDecodePad * drainpad,
    * new information */
   for (tmp = group->children; tmp; tmp = tmp->next) {
     GstDecodeChain *chain = (GstDecodeChain *) tmp->data;
-    gboolean subdrained;
+    gboolean subdrained = FALSE;
 
     handled |=
         drain_and_switch_chains (chain, drainpad, last_group, &subdrained,

Comment 2 Sebastian Dröge (slomo) 2011-08-19 08:43:15 UTC

drain_and_switch_chains() should always set the three out-parameters I guess. Currently there are some paths that don't set some