GNOME Bugzilla – Bug 767499
Spatial Resolution Change Causes Playback Freeze
Last modified: 2018-05-07 15:48:24 UTC
Created attachment 329578 [details] xz compressed log ftp://ftp.bbc.co.uk/incoming/bbb-avc3-rep2-rep4.mp4 I've got an issue with decoding DASH content on a Raspberry Pi using git latest GStreamer. When something in a running stream causes a spatial resolution change (e.g. changing a DASH representation), if the spatial resolution change is the only change then the stream will freeze. If the change involves a spatial resolution change along with another change in the stream such as H.264 level or profile, then the video doesn't freeze and the change works. The stream decodes correctly on two other platforms that I have which are both built from the same source code but do not use gst-omx. Below is the exact versions of what I'm running, in case it helps: * Buildroot 2016.05, building Linux kernel 4.4.8, Raspberry Pi firmware version cc6d7bf8 * GStreamer latest (git 8c7da1d4) * gst-plugins-base (git 8b8708f9) * gst-plugins-good (git b4a4fa19) * gst-plugins-bad (git 3107f5df) * gst-libav (git 1a4c54a6) * gst-omx (git 94789f19) When the failure happens, no errors or warnings are printed to the console by any element. As far as I've been able to deduce so far, when a representation change happens which changes more than just the spatial resolution it triggers gst_omx_video_dec_set_format() with the new caps for the stream. However, if the change is only the spatial resolution, then this call never happens and the stream freezes. Looking at the caps, the spatial resolution on the input caps of the omx decoder seems to be set to the maximum resolution of all representations in a given adaptation set all the way through and never changes. The output caps then differ according to what it's supposed to be playing. I've managed to create a short clip which recreates this issue outside of DASH, taken from a Big Buck Bunny DASH stream. The stream is effectively one file containing an initialisation segment, two 4 second segments from the first representation and a two 4 second segments from the second representation, concatenated together. The spatial resolution changes from 704x396p50 to 960x540p50. Both representations use an avc3 sample entry format, have a H.264 level of 3.1 and both use high profile. You can download the clip from here: ftp://ftp.bbc.co.uk/incoming/bbb-avc3-rep2-rep4.mp4 I've also attached a log made with GST_DEBUG="videodecoder:5,omx*:5".
Can you also get a backtrace of all threads when it freezes?
Hi Sebastian, It seems that when I recompile with debugging symbols included, the stream doesn't pause indefinitely, instead it pauses for a short period of time (roughly a second or so) and then continues. I'll attach a full thread dump from when it does the short pause anyway, in case it is helpful.
Created attachment 329787 [details] Backtrace of all threads
Nothing problematic in there. A short pause (how long?) would be normal, as the decoder has to be reinitialized and that involves reallocating all buffers (and as such waiting for them to be returned first).
Hi Sam, I remember that the pause was maximum 1 sec. This only happens when using OMX_UseBuffer(or OMX_UseEGLImage) because the gstomxvideodec wraps the downstream pool in its own buffer pool see https://cgit.freedesktop.org/gstreamer/gst-omx/tree/omx/gstomxbufferpool.c#n353 . So before the decoder can reset its resolution it needs all the buffer to return to its pool. The decoder cannot directly call 'gst_buffer_pool_acquire_buffer' on the video sink 's buffer pool, otherwise it might acquire a buffer that does not match what the decoder has last used/filled. So it affects zero copy (because OMX_UseEGLImage) and OMX resize/convert component (because OMX_UseBuffer https://bugzilla.gnome.org/show_bug.cgi?id=722686). (This problem does not affect cases with OMX_AllocateBuffer, but this is much slower so not practical in most of the cases) There is potentially a solution which is to use the new 'gst_video_decoder_allocate_output_frame_with_params' I added recenlty which allows to pass some params (instead of https://cgit.freedesktop.org/gstreamer/gst-omx/tree/omx/gstomxvideodec.c#n1475 ). This will allow to pass an ID by suclassing GstBufferPoolAcquireParams. So that the downstream pool can return the matching buffer for the given ID instead of returning the next buffer from its queue. At least it should save the time it takes currently to reclaim the buffers which are still in use downstream from the decoder point of view. With this solution the decoder does not need to wait since it does not own them. Btw, were you testing in rpi2 or rpi3 ?
Closing this bug report as no further information has been provided. Please feel free to reopen this bug report if you can provide the information that was asked for in a previous comment. Thanks!