After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 730989 - omxdec: Get stuck while doing ctrl+c during preroll
omxdec: Get stuck while doing ctrl+c during preroll
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-omx
1.3.1
Other Linux
: Normal normal
: 1.2.0
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2014-05-30 10:23 UTC by Aurélien Zanelli
Modified: 2014-07-23 08:14 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Trace omx*:6 getting stuck in preroll (61.97 KB, text/x-log)
2014-05-30 10:23 UTC, Aurélien Zanelli
  Details
Trace omx*:6 getting stuck in preroll (61.97 KB, text/x-log)
2014-05-30 11:21 UTC, Aurélien Zanelli
  Details
Threads backtrace (14.52 KB, text/x-log)
2014-05-30 11:21 UTC, Aurélien Zanelli
  Details
omx: Don't handle disabling/enabling ports exactly like flushing (2.41 KB, patch)
2014-05-31 13:13 UTC, Sebastian Dröge (slomo)
committed Details | Review
omxvideodec: add missing stream unlock in error path (747 bytes, patch)
2014-06-02 13:42 UTC, Aurélien Zanelli
committed Details | Review

Description Aurélien Zanelli 2014-05-30 10:23:07 UTC
Created attachment 277529 [details]
Trace omx*:6 getting stuck in preroll

I can get the omxdecoder stuck in acquire_buffer() by doing ctrl+c during the preroll state with the following pipeline:

$ gst-launch-1.0 filesrc location=Sintel.2010.720p.mkv ! matroskademux ! h264parse ! omxh264dec ! fakesink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
handling interrupt.
Interrupt: Stopping pipeline ...
Setting pipeline to NULL ...
Caught SIGQUIT


I can reproduce it on my OMX platform and on RPi. Howver the issue does not always happened.
I attached a log of the above pipeline on RPi with --gst-debug=*:3,omx*:6

I think it's a race condition between port flushing, pipeline stop and component reconfiguration.
Comment 1 Tim-Philipp Müller 2014-05-30 10:41:25 UTC
A stack trace from gdb (thread apply all bt) would also be helpful.
Comment 2 Aurélien Zanelli 2014-05-30 11:21:18 UTC
Created attachment 277537 [details]
Trace omx*:6 getting stuck in preroll

Update gstlog for following backtrace.
Comment 3 Aurélien Zanelli 2014-05-30 11:21:59 UTC
Created attachment 277538 [details]
Threads backtrace

All threads backtrace
Comment 4 Nicolas Dufresne (ndufresne) 2014-05-30 14:00:20 UTC
Just a note, I've had to fix the same pattern in v4l2videodec (didn't check OMX). The loop thread is a pad task, hence it's holding on the stream lock. We cannot rely on GstVideoDecoder::stop() as it's called after trying to deactivate the pad. When we try to deactivate the pad, the loop is still holding on the stream lock, so deactivate will block on that:

gstpad.c:957
      GST_PAD_STREAM_LOCK (pad); <-- ** Here **
      GST_DEBUG_OBJECT (pad, "stopped streaming");
      GST_OBJECT_LOCK (pad);
      remove_events (pad);
      GST_OBJECT_UNLOCK (pad);
      GST_PAD_STREAM_UNLOCK (pad);

So what I did, is to override the state change virtual, and before chaining to the base class, I do what's needed to make by loop exit. The base class could be improved, but this fixed it.
Comment 5 Aurélien Zanelli 2014-05-30 14:09:08 UTC
(In reply to comment #4)
> Just a note, I've had to fix the same pattern in v4l2videodec (didn't check
> OMX). The loop thread is a pad task, hence it's holding on the stream lock. We
> cannot rely on GstVideoDecoder::stop() as it's called after trying to
> deactivate the pad. When we try to deactivate the pad, the loop is still
> holding on the stream lock, so deactivate will block on that:
> 
> gstpad.c:957
>       GST_PAD_STREAM_LOCK (pad); <-- ** Here **
>       GST_DEBUG_OBJECT (pad, "stopped streaming");
>       GST_OBJECT_LOCK (pad);
>       remove_events (pad);
>       GST_OBJECT_UNLOCK (pad);
>       GST_PAD_STREAM_UNLOCK (pad);
> 
> So what I did, is to override the state change virtual, and before chaining to
> the base class, I do what's needed to make by loop exit. The base class could
> be improved, but this fixed it.

I take a look and try the same solution that you did in v4l2videodec.
Comment 6 Sebastian Dröge (slomo) 2014-05-30 14:39:36 UTC
Note that the same change will be necessary in the audio encoders/decoders and the video encoders. And in the androidmedia plugin :)
Comment 7 Sebastian Dröge (slomo) 2014-05-31 08:54:39 UTC
Note this code here
http://cgit.freedesktop.org/gstreamer/gst-omx/tree/omx/gstomxvideodec.c#n362
which in theory should unblock this here and stop the loop
http://cgit.freedesktop.org/gstreamer/gst-omx/tree/omx/gstomxvideodec.c#n1186
(the FLUSHING case). That's exactly what you describe Nicolas, right?


The problem according to the debug output here is that the flushing setting happens while in this code here
http://cgit.freedesktop.org/gstreamer/gst-omx/tree/omx/gstomxvideodec.c#n1206

I wonder why the acquire_buffer() afterwards does not immediately return flushing... I think it's because of
http://cgit.freedesktop.org/gstreamer/gst-omx/tree/omx/gstomx.c#n2148
which is called in the output port reconfiguration.

The enabled-flushing and flushing-flushing probably needs to be tracked separately, but handled the same in many other places. I'll try to give you a patch for testing in a bit.
Comment 8 Sebastian Dröge (slomo) 2014-05-31 13:13:17 UTC
Created attachment 277621 [details] [review]
omx: Don't handle disabling/enabling ports exactly like flushing

Otherwise we might abort a flush operation in another thread when
enabling/disabling ports, leading to deadlocks sometimes.
Comment 9 Sebastian Dröge (slomo) 2014-05-31 13:14:07 UTC
Aurélien, does this fix it for you? This definitely needs some further testing
Comment 10 Aurélien Zanelli 2014-06-02 13:41:50 UTC
(In reply to comment #9)
> Aurélien, does this fix it for you? This definitely needs some further testing

I tested it on both RPi and on my OMX platform.

On RPi after applying the patch, I didn't succeed to reproduce the issue.
However on my OMX platform, I ended up with a deadlock involving gstvideodecoder stream_lock due to a missing unlock in an error path, I will add a patch to this bug report to add it.

Finally, with your patch, I can't reproduce the bug so I think it fix it.
Comment 11 Aurélien Zanelli 2014-06-02 13:42:29 UTC
Created attachment 277723 [details] [review]
omxvideodec: add missing stream unlock in error path
Comment 12 Sebastian Dröge (slomo) 2014-06-02 17:30:48 UTC
Thanks for testing and for your patch, I pushed both patches now.

Which platforms is your OMX platform btw?
Comment 13 Aurélien Zanelli 2014-06-03 07:20:08 UTC
(In reply to comment #12)
> Thanks for testing and for your patch, 
No problem :)

> Which platforms is your OMX platform btw?
It's a SoC made by my company with HW decoders.
Comment 14 Sebastian Dröge (slomo) 2014-06-03 07:23:03 UTC
(In reply to comment #13)

> > Which platforms is your OMX platform btw?
> It's a SoC made by my company with HW decoders.

If it's not all secret and you're interested, we could add a config file for that platform to gst-omx to make it easier for people to use your hardware
Comment 15 Aurélien Zanelli 2014-06-03 16:03:32 UTC
(In reply to comment #14)
> (In reply to comment #13)
> If it's not all secret and you're interested, we could add a config file for
> that platform to gst-omx to make it easier for people to use your hardware
Unfortunately, hardware is not intended for end-user :(, but thanks for the proposal.