After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 785745 - [gl] glcolorscale passthrough mode causing playback failure on video resolution change
[gl] glcolorscale passthrough mode causing playback failure on video resoluti...
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-base
git master
Other Linux
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2017-08-02 17:15 UTC by Sam Hurst
Modified: 2018-11-03 11:58 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Excerpt of log for the issue described (23.13 KB, application/x-xz)
2017-08-02 17:15 UTC, Sam Hurst
  Details
Patch to remove check for passthrough in gst_gl_filter_transform_caps (1.54 KB, patch)
2017-08-02 17:17 UTC, Sam Hurst
reviewed Details | Review

Description Sam Hurst 2017-08-02 17:15:53 UTC
Created attachment 356793 [details]
Excerpt of log for the issue described

I've been using GStreamer on a Raspberry Pi 2 to display fullscreen DASH. There is no X session involved, it's using OpenGL ES directly on the framebuffer. As gst-launch-1.0 starts with a window size equal to the video resolution of the first DASH representation received and not fullscreen, I use a custom pipeline to resize the video window to match the output screen size. My pipeline is below:

gst-launch-1.0 souphttpsrc location=$MPD_URL ! dashdemux name=d \
    d. ! video/quicktime ! queue ! qtdemux ! h264parse ! omxh264dec ! queue \
        ! glupload ! glcolorconvert ! glcolorscale \
        ! "video/x-raw(memory:GLMemory),width=$X_RES,height=$Y_RES" ! queue \
        ! glimagesink
    d. ! audio/x-m4a ! queue ! qtdemux ! aacparse ! avdec_aac ! audioconvert \
        ! omxhdmiaudiosink

Where $MPD_URL is the MPD URL of a given stream, and $X_RES and $Y_RES representing the screen size to display on.

This works fine, unless the MPD contains a video representation with the same resolution as the screen output. Then, GStreamer has issues when the resolution changes away from that specified in the capsfilter downstream of the glcolorscale element.

For example:

* Player starts, output resolution is 1280x720
* dashdemux selects first representation of a much smaller video resolution than the output resolution
* Playback begins
* Good network conditions mean dashdemux picks a representation with video resolution equal to that of the output resolution of 1280x720
* Playback continues as normal
* Network conditions dictate that dashdemux changes representation again to a representation with a different video resolution to the output resolution
* The pipeline ends with a "not-negotiated" error

I've attached an excerpt of a debug log made with GST_DEBUG="*:3,glfilter:9,basetransform:9,GST_PADS:6" which shows the error (glcolorscale-passthrough.log.xz). Of particular note seem to be the lines which state:

0:00:19.704991143   317   0x25d830 WARN           basetransform gstbasetransform.c:1346:gst_base_transform_setcaps:<gluploadelement0> transform could not transform video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)960, height=(int)540, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)50/1 in anything we support

After a bit of investigation, I discovered that it seemed to be that the element was stuck in passthrough mode, and that this was causing the playback to fail because once in passthrough mode the filter could not be brought out of it should the caps change upstream of the element. By removing the check for gst_base_transform_is_passthrough() in gst_gl_filter_transform_caps() in gst-plugins-bad/gst-libs/gst/gl/gstglfilter.c, it now forces it to do a full evaluation of the caps when it changes, instead of just taking a reference to the sink pad caps as they arrive.

By removing the aforementioned check for ::is_passthrough(), this has fixed the issue for me. I will also attach a patch that I'm now using internally. This has fixed the issue for me, but as I'm not an expert in either OpenGL nor the GstBaseTransform side of things I would like to ask what the GStreamer community's take on this is. I can't find any other GstBaseTransform element doing what GstGlFilter is doing with that ::is_passthrough() call, so I thought it a tad suspect.

If you need a test stream, I have created a standalone file which replicates the behaviour of dashdemux which I can provide. Unfortunately, the video is too large to upload here (9.1MB and the file size limit is 3.6MB), but I'm looking into other ways of getting it to you.

For reference, this is running from Git Master as of about three weeks ago with a few patches pulled in for gl and omx improvements ontop of that, and was also reproducible on the 1.12.1 release.
Comment 1 Sam Hurst 2017-08-02 17:17:09 UTC
Created attachment 356794 [details] [review]
Patch to remove check for passthrough in gst_gl_filter_transform_caps

Attaching patch that fixes the issue for me.
Comment 2 Nicolas Dufresne (ndufresne) 2017-12-06 22:00:13 UTC
Review of attachment 356794 [details] [review]:

Looks like just a hack, not a solution isn't it ?
Comment 3 Sam Hurst 2017-12-07 09:56:05 UTC
Yes, it was a kludge to make it work for me, not a lasting solution. It's been working fine for my purposes for the last four months or so, but I fully realise it's not ideal. Hopefully it might help someone who knows more about GL/Caps transformation to find the underlying issue and fix it.
Comment 4 Nicolas Dufresne (ndufresne) 2017-12-17 21:04:44 UTC
Though, looking a bit more, it seems that it's incorrect to change the transform_caps behaviour base on the passthrough mode. It should be the other way around, we should decide to passthrough base on the resulting caps. To make it worst, it's ignoring the in-place mode.
Comment 5 GStreamer system administrator 2018-11-03 11:58:44 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/371.