GNOME Bugzilla – Bug 785745
[gl] glcolorscale passthrough mode causing playback failure on video resolution change
Last modified: 2018-11-03 11:58:44 UTC
Created attachment 356793 [details] Excerpt of log for the issue described I've been using GStreamer on a Raspberry Pi 2 to display fullscreen DASH. There is no X session involved, it's using OpenGL ES directly on the framebuffer. As gst-launch-1.0 starts with a window size equal to the video resolution of the first DASH representation received and not fullscreen, I use a custom pipeline to resize the video window to match the output screen size. My pipeline is below: gst-launch-1.0 souphttpsrc location=$MPD_URL ! dashdemux name=d \ d. ! video/quicktime ! queue ! qtdemux ! h264parse ! omxh264dec ! queue \ ! glupload ! glcolorconvert ! glcolorscale \ ! "video/x-raw(memory:GLMemory),width=$X_RES,height=$Y_RES" ! queue \ ! glimagesink d. ! audio/x-m4a ! queue ! qtdemux ! aacparse ! avdec_aac ! audioconvert \ ! omxhdmiaudiosink Where $MPD_URL is the MPD URL of a given stream, and $X_RES and $Y_RES representing the screen size to display on. This works fine, unless the MPD contains a video representation with the same resolution as the screen output. Then, GStreamer has issues when the resolution changes away from that specified in the capsfilter downstream of the glcolorscale element. For example: * Player starts, output resolution is 1280x720 * dashdemux selects first representation of a much smaller video resolution than the output resolution * Playback begins * Good network conditions mean dashdemux picks a representation with video resolution equal to that of the output resolution of 1280x720 * Playback continues as normal * Network conditions dictate that dashdemux changes representation again to a representation with a different video resolution to the output resolution * The pipeline ends with a "not-negotiated" error I've attached an excerpt of a debug log made with GST_DEBUG="*:3,glfilter:9,basetransform:9,GST_PADS:6" which shows the error (glcolorscale-passthrough.log.xz). Of particular note seem to be the lines which state: 0:00:19.704991143 317 0x25d830 WARN basetransform gstbasetransform.c:1346:gst_base_transform_setcaps:<gluploadelement0> transform could not transform video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)960, height=(int)540, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)50/1 in anything we support After a bit of investigation, I discovered that it seemed to be that the element was stuck in passthrough mode, and that this was causing the playback to fail because once in passthrough mode the filter could not be brought out of it should the caps change upstream of the element. By removing the check for gst_base_transform_is_passthrough() in gst_gl_filter_transform_caps() in gst-plugins-bad/gst-libs/gst/gl/gstglfilter.c, it now forces it to do a full evaluation of the caps when it changes, instead of just taking a reference to the sink pad caps as they arrive. By removing the aforementioned check for ::is_passthrough(), this has fixed the issue for me. I will also attach a patch that I'm now using internally. This has fixed the issue for me, but as I'm not an expert in either OpenGL nor the GstBaseTransform side of things I would like to ask what the GStreamer community's take on this is. I can't find any other GstBaseTransform element doing what GstGlFilter is doing with that ::is_passthrough() call, so I thought it a tad suspect. If you need a test stream, I have created a standalone file which replicates the behaviour of dashdemux which I can provide. Unfortunately, the video is too large to upload here (9.1MB and the file size limit is 3.6MB), but I'm looking into other ways of getting it to you. For reference, this is running from Git Master as of about three weeks ago with a few patches pulled in for gl and omx improvements ontop of that, and was also reproducible on the 1.12.1 release.
Created attachment 356794 [details] [review] Patch to remove check for passthrough in gst_gl_filter_transform_caps Attaching patch that fixes the issue for me.
Review of attachment 356794 [details] [review]: Looks like just a hack, not a solution isn't it ?
Yes, it was a kludge to make it work for me, not a lasting solution. It's been working fine for my purposes for the last four months or so, but I fully realise it's not ideal. Hopefully it might help someone who knows more about GL/Caps transformation to find the underlying issue and fix it.
Though, looking a bit more, it seems that it's incorrect to change the transform_caps behaviour base on the passthrough mode. It should be the other way around, we should decide to passthrough base on the resulting caps. To make it worst, it's ignoring the in-place mode.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/371.