Bug 575068 – [mp3parse] deadlock with accurate seeking

After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.

Bug 575068 - [mp3parse] deadlock with accurate seeking


Summary:	[mp3parse] deadlock with accurate seeking


Status:	RESOLVED FIXED

Product:	GStreamer
Classification:	Platform
Component:	gst-plugins-ugly
Version:	git master
Hardware:	Other Linux

Importance:	Normal blocker
Target Milestone:	0.10.11
Assigned To:	GStreamer Maintainers
QA Contact:	GStreamer Maintainers

URL:
Whiteboard:

Depends on:
Blocks:

Reported:	2009-03-12 11:46 UTC by Alessandro Decina
Modified:	2009-03-12 15:23 UTC

See Also:
GNOME target:	---
GNOME version:	---

Attachments
fix (1.23 KB, patch) 2009-03-12 11:47 UTC, Alessandro Decina	none	Details \| Review
fix 2 (1.46 KB, patch) 2009-03-12 14:18 UTC, Alessandro Decina	none	Details \| Review
also change the seek variable while the lock is held (1.54 KB, patch) 2009-03-12 14:31 UTC, Alessandro Decina	committed	Details \| Review

Description Alessandro Decina 2009-03-12 11:46:34 UTC

I'm seeing this deadlock seeking in mp3 files with pitivi git and -ugly git

(gdb) info threads
  7 Thread 0x42537950 (LWP 26984)  0x00007f5d62ca52d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  6 Thread 0x41672950 (LWP 26994)  0x00007f5d62ca555d in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  5 Thread 0x40a04950 (LWP 26995)  0x00007f5d62ca52d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  4 Thread 0x43539950 (LWP 26997)  0x00007f5d62ca52d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  3 Thread 0x42d38950 (LWP 26998)  0x00007f5d62ca52d9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  2 Thread 0x43d3a950 (LWP 26999)  0x00007f5d62ca7b04 in __lll_lock_wait ()
   from /lib/libpthread.so.0
  1 Thread 0x7f5d630b66e0 (LWP 26977)  0x00007f5d62ca7b04 in __lll_lock_wait ()
   from /lib/libpthread.so.0

(gdb) thr a a bt

+ Trace 213409

Thread 1 (Thread 0x7f5d630b66e0 (LWP 26977))

#0 __lll_lock_wait
from /lib/libpthread.so.0
#1 _L_lock_102
from /lib/libpthread.so.0
#2 pthread_mutex_lock
from /lib/libpthread.so.0
#3 IA__g_static_rec_mutex_lock
at /build/buildd/glib2.0-2.18.2/glib/gthread.c line 313
#4 gst_base_src_perform_seek
at gstbasesrc.c line 1210
#5 gst_base_src_default_event
at gstbasesrc.c line 1542
#6 gst_base_src_event_handler
at gstbasesrc.c line 1577
#7 gst_pad_send_event
at gstpad.c line 4701
#8 gst_pad_push_event
at gstpad.c line 4557
#9 gst_type_find_element_src_event
at gsttypefindelement.c line 425
#10 gst_pad_send_event
at gstpad.c line 4701
#11 gst_pad_push_event
#12 gst_tag_demux_srcpad_event
at gsttagdemux.c line 834
#13 gst_pad_send_event
at gstpad.c line 4701
#14 gst_pad_push_event
at gstpad.c line 4557
#15 gst_tag_demux_srcpad_event
at gsttagdemux.c line 834
#16 gst_pad_send_event
at gstpad.c line 4701
#17 gst_pad_push_event
at gstpad.c line 4557
#18 mp3parse_handle_seek
at gstmpegaudioparse.c line 1787
#19 mp3parse_src_event
at gstmpegaudioparse.c line 1836
#20 gst_pad_send_event
at gstpad.c line 4701
#21 gst_pad_push_event
at gstpad.c line 4557
#22 gst_mad_src_event
at gstmad.c line 817
#23 gst_pad_send_event
at gstpad.c line 4701
#24 gst_pad_push_event
at gstpad.c line 4557
#25 gst_proxy_pad_do_event
at gstghostpad.c line 128
#26 gst_pad_send_event
at gstpad.c line 4701
#27 gst_pad_push_event
at gstpad.c line 4557
#28 gst_proxy_pad_do_event
at gstghostpad.c line 128
#29 ghostpad_event_function
at gnlobject.c line 711
#30 gst_pad_send_event
at gstpad.c line 4701
#31 gst_pad_push_event
at gstpad.c line 4557
#32 gst_proxy_pad_do_event
at gstghostpad.c line 128
#33 ghostpad_event_function
at gnlobject.c line 711
#34 gnl_composition_event_handler
#35 gst_pad_send_event
at gstpad.c line 4701
#36 gst_pad_push_event
at gstpad.c line 4557
#37 gst_base_transform_src_eventfunc
at gstbasetransform.c line 1773
#38 _wrap_GstBaseTransform__do_src_event
at gstbase.override line 831
#39 PyEval_EvalFrameEx
at ../Python/ceval.c line 3595
#40 PyEval_EvalCodeEx
at ../Python/ceval.c line 2858
#41 function_call
at ../Objects/funcobject.c line 517
#42 PyObject_Call
at ../Objects/abstract.c line 1861
#43 instancemethod_call
at ../Objects/classobject.c line 2519
#44 PyObject_Call
at ../Objects/abstract.c line 1861
#45 PyEval_CallObjectWithKeywords
at ../Python/ceval.c line 3464
#46 _wrap_GstBaseTransform__proxy_do_src_event
at gst.c line 19032
#47 gst_base_transform_src_event
at gstbasetransform.c line 1742
#48 gst_pad_send_event
at gstpad.c line 4701
#49 gst_pad_push_event
at gstpad.c line 4557
#50 gst_proxy_pad_do_event
at gstghostpad.c line 128
#51 gst_pad_send_event
at gstpad.c line 4701
#52 gst_pad_push_event
at gstpad.c line 4557
#53 gst_pad_event_default_dispatch
at gstpad.c line 3327
#54 gst_pad_event_default
at gstpad.c line 3401
#55 gst_pad_send_event
at gstpad.c line 4701
#56 gst_pad_push_event
#57 gst_queue_handle_src_event
at gstqueue.c line 1176
#58 gst_pad_send_event
at gstpad.c line 4701
#59 gst_pad_push_event
at gstpad.c line 4557
#60 gst_proxy_pad_do_event
at gstghostpad.c line 128
#61 gst_pad_send_event
at gstpad.c line 4701
#62 gst_pad_push_event
at gstpad.c line 4557
#63 gst_base_transform_src_eventfunc
at gstbasetransform.c line 1773
#64 gst_base_transform_src_event
at gstbasetransform.c line 1742
#65 gst_pad_send_event
at gstpad.c line 4701
#66 gst_pad_push_event
at gstpad.c line 4557
#67 gst_base_transform_src_eventfunc
at gstbasetransform.c line 1773
#68 gst_base_transform_src_event
at gstbasetransform.c line 1742
#69 gst_pad_send_event
at gstpad.c line 4701
#70 gst_pad_push_event
at gstpad.c line 4557
#71 gst_base_transform_src_eventfunc
at gstbasetransform.c line 1773
#72 gst_base_transform_src_event
at gstbasetransform.c line 1742
#73 gst_pad_send_event
at gstpad.c line 4701
#74 gst_pad_push_event
at gstpad.c line 4557
#75 gst_proxy_pad_do_event
at gstghostpad.c line 128
#76 gst_pad_send_event
at gstpad.c line 4701
#77 gst_pad_push_event
at gstpad.c line 4557
#78 gst_base_sink_send_event
at gstbasesink.c line 3603
#79 gst_element_send_event
#80 gst_bin_send_event
at gstbin.c line 2502
#81 gst_element_send_event
at gstelement.c line 1383
#82 gst_bin_send_event
at gstbin.c line 2502
#83 gst_element_send_event
at gstelement.c line 1383
#84 gst_bin_send_event
at gstbin.c line 2502
#85 gst_element_send_event
at gstelement.c line 1383
#86 gst_element_seek
at gstelement.c line 1423
#87 _wrap_gst_element_seek
at gst.c line 4774
#88 PyEval_EvalFrameEx
at ../Python/ceval.c line 3595
#89 PyEval_EvalCodeEx
#90 PyEval_EvalFrameEx
at ../Python/ceval.c line 3691
#91 PyEval_EvalCodeEx
at ../Python/ceval.c line 2858
#92 function_call
at ../Objects/funcobject.c line 517
#93 PyObject_Call
at ../Objects/abstract.c line 1861
#94 instancemethod_call
at ../Objects/classobject.c line 2519
#95 PyObject_Call
at ../Objects/abstract.c line 1861
#96 PyEval_CallObjectWithKeywords
at ../Python/ceval.c line 3464
#97 pyg_closure_marshal
at pygtype.c line 1109
#98 IA__g_closure_invoke
#99 signal_emit_unlocked_R
at /build/buildd/glib2.0-2.18.2/gobject/gsignal.c line 3244
#100 pygobject_emit
at pygobject.c line 1734
#101 PyEval_EvalFrameEx
at ../Python/ceval.c line 3595
#102 PyEval_EvalCodeEx
at ../Python/ceval.c line 2858
#103 function_call
at ../Objects/funcobject.c line 517
#104 PyObject_Call
at ../Objects/abstract.c line 1861
#105 PyEval_EvalFrameEx
at ../Python/ceval.c line 3875
#106 PyEval_EvalCodeEx
#107 function_call
at ../Objects/funcobject.c line 517
#108 PyObject_Call
at ../Objects/abstract.c line 1861
#109 PyEval_EvalFrameEx
at ../Python/ceval.c line 3875
#110 PyEval_EvalCodeEx
at ../Python/ceval.c line 2858
#111 PyEval_EvalFrameEx
at ../Python/ceval.c line 3691
#112 PyEval_EvalCodeEx
at ../Python/ceval.c line 2858
#113 function_call
at ../Objects/funcobject.c line 517
#114 PyObject_Call
at ../Objects/abstract.c line 1861
#115 instancemethod_call
#116 PyObject_Call
at ../Objects/abstract.c line 1861
#117 PyEval_CallObjectWithKeywords
at ../Python/ceval.c line 3464
#118 _pyglib_handler_marshal
at pyglib.c line 562
#119 g_timeout_dispatch
at /build/buildd/glib2.0-2.18.2/glib/gmain.c line 3589
#120 IA__g_main_context_dispatch
at /build/buildd/glib2.0-2.18.2/glib/gmain.c line 2144
#121 g_main_context_iterate
at /build/buildd/glib2.0-2.18.2/glib/gmain.c line 2778
#122 IA__g_main_loop_run
at /build/buildd/glib2.0-2.18.2/glib/gmain.c line 2986
#123 _wrap_g_main_loop_run
at pygmainloop.c line 319
#124 PyEval_EvalFrameEx
at ../Python/ceval.c line 3579
#125 PyEval_EvalFrameEx
at ../Python/ceval.c line 3681
#126 PyEval_EvalFrameEx
at ../Python/ceval.c line 3681
#127 PyEval_EvalFrameEx
at ../Python/ceval.c line 3681
#128 PyEval_EvalCodeEx
at ../Python/ceval.c line 2858
#129 PyEval_EvalCode
at ../Python/ceval.c line 514
#130 PyRun_FileExFlags
at ../Python/pythonrun.c line 1273
#131 PyRun_SimpleFileExFlags
at ../Python/pythonrun.c line 879
#132 Py_Main
at ../Modules/main.c line 532
#133 __libc_start_main
from /lib/libc.so.6
#134 _start
#0 __lll_lock_wait
from /lib/libpthread.so.0


there's a race acquiring mp3parse->pending_accurate_seeks_lock. It's acquired from both the seeking thread and the streaming thread. The deadlock happens when a seek happens; the lock is taken; in the meantime the streaming thread tries to acquire it too but it's taken so the streaming thread blocks; basesrc flushes and waits for the streaming thread to stop -> BOOM

Comment 1 Alessandro Decina 2009-03-12 11:47:44 UTC

Created attachment 130507 [details] [review]
fix

Comment 2 Edward Hervey 2009-03-12 12:10:03 UTC

Marking this as a blocker, since it's a regression since last -ugly release.

Comment 3 Jan Schmidt 2009-03-12 14:04:36 UTC

This looks wrong to me:
1) The 'seek' structure is added to the accurate seek list within the lock, and then modified in the if (send_event() == TRUE) clause without the lock held.
2) If the send_event fails, the seek structure is freed, and a pointer to the freed memory is left in the pending_accurate_seeks list, waiting to make things go kaboom later.

Comment 4 Alessandro Decina 2009-03-12 14:18:43 UTC

Created attachment 130522 [details] [review]
fix 2

oops. Now i remove seek from the list. It should be safe afaics.

Comment 5 Alessandro Decina 2009-03-12 14:31:58 UTC

Created attachment 130523 [details] [review]
also change the seek variable while the lock is held

Comment 6 Jan Schmidt 2009-03-12 14:53:13 UTC

Looks healthier to me, yip. Please commit.

Comment 7 Edward Hervey 2009-03-12 15:07:52 UTC

same here. commit !