GNOME Bugzilla – Bug 769580
Fail to decode media on OSX because it's not supported by vtdec_hw but vtdec can play it
Last modified: 2016-08-26 14:31:10 UTC
Hardware decoders usually have constraints on the frame sizes they can handle. Ideally we should expose such constraints in the sink pad template caps but I couldn't find a way to query for such constraints in VideoToolbox. [gst-master] bash-3.2$ GST_DEBUG=*vtdec*:6 gst-launch-1.0 --gst-debug-no-color filesrc location= ~/media/test.mov ! decodebin ! fakesink Setting pipeline to PAUSED ... Pipeline is PREROLLING ... 0:00:00.147749000 98341 0x7fb5aa0171e0 DEBUG vtdec vtdec.c:179:gst_vtdec_start:<vtdechw0> start 0:00:00.148137000 98341 0x7fb5a88c9b70 DEBUG vtdec vtdec.c:355:gst_vtdec_set_format:<vtdechw0> set_format 0:00:00.148154000 98341 0x7fb5a88c9b70 INFO vtdec vtdec.c:1005:gst_vtdec_set_latency:<vtdechw0> setting latency frames:0 time:0:00:00.000000000 Redistribute latency... 0:00:00.148288000 98341 0x7fb5a88c9b70 INFO vtdec vtdec.c:294:gst_vtdec_negotiate:<vtdechw0> negotiated output format video/x-raw, format=(string)NV12, width=(int)2592, height=(int)3248, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt2020, framerate=(fraction)30/1 previous (NULL) 0:00:00.151937000 98341 0x7fb5a88c9b70 WARN vtdec vtdec.c:309:gst_vtdec_negotiate:<vtdechw0> error: VTDecompressionSessionCreate returned -12906 ERROR: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVtdecHw:vtdechw0: GStreamer encountered a general resource error. Additional debug info: vtdec.c(309): gst_vtdec_negotiate (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstVtdecHw:vtdechw0: VTDecompressionSessionCreate returned -12906 ERROR: pipeline doesn't want to preroll. Setting pipeline to NULL ... 0:00:00.152226000 98341 0x7fb5a88c9b70 DEBUG vtdec vtdec.c:355:gst_vtdec_set_format:<vtdechw0> set_format 0:00:00.152238000 98341 0x7fb5a88c9b70 INFO vtdec vtdec.c:1005:gst_vtdec_set_latency:<vtdechw0> setting latency frames:0 time:0:00:00.000000000 0:00:00.152276000 98341 0x7fb5a88c9b70 INFO vtdec vtdec.c:250:gst_vtdec_negotiate:<vtdechw0> current and peer caps are compatible, keeping current caps 0:00:00.152344000 98341 0x7fb5a86aa130 DEBUG vtdec vtdec.c:200:gst_vtdec_stop:<vtdechw0> stop Freeing pipeline ...
Created attachment 332850 [details] [review] vtdec: fallback to software decoding on unsupported frame sizes Proposed patch to fix the issue.
Comment on attachment 332850 [details] [review] vtdec: fallback to software decoding on unsupported frame sizes The goal here is that vtdec_hw really fails if it can't support the stream in hardware. If a fallback to another decoder is required, that should be implemented around this. E.g. decodebin will switch from vtdec_hw to avdec_h264 or vtdec in such situations. The reason for this is that avdec_h264 is usually the better choice than vtdec for software decoding. It supports more formats and is usually faster.
The media here is mjpeg, not sure which is the best fallback on such cases. I agree that would be better let decodebin fallback, how I can achieve it? Right now the pipeline just stops with a not negotiated and doesn't fallback to vtdec.
A failing clip can be created with the following pipeline. gst-launch-1.0 videotestsrc num-buffers=100 ! video/x-raw, width=2592, height=3248 ! avenc_mjpeg ! qtmux ! filesink location=test.mov
The issue is a bit different that I'd thought in the beginning. This is an early 2011 macbook pro with a SandyBridge Intel cpu. It seems that hw doesn't support mjpeg but vtdec_hw has caps for it. But in any case no other decoder is negotiated so something must be wrong somewhere else. In decodebin? Same behaviour with decodebin3.
Just forgot to say that vtdec_hw plays h264 clips on this hardware.
It turns out that issue is because the sticky events aren't propagated to the multiqueue src pad in decode bin and the vtdec_hw isn't properly tested if it is functional when there's chances to fallback to another decoder.
Created attachment 332990 [details] [review] decodebin: forward sticky events on multiqueue When connecting a demuxer through a multiqueue ensure to copy sticky events in order to allow the following factory being properly checked that it is functional.
Comment on attachment 332990 [details] [review] decodebin: forward sticky events on multiqueue commit 970ea49d30d28a3f03e67b36a07d91a6c4649550 Author: Josep Torra <n770galaxy@gmail.com> Date: Tue Aug 9 11:39:53 2016 +0200 decodebin: forward sticky events on multiqueue When connecting a demuxer through a multiqueue ensure to copy sticky events in order to allow the following factory being properly checked that it is functional. https://bugzilla.gnome.org/show_bug.cgi?id=769580
I'm keeping the issue open because I see the same issue with decodebin3.
Closing because the issue with decodebin3 is already tracked in https://bugzilla.gnome.org/show_bug.cgi?id=769079
*** This bug has been marked as a duplicate of bug 769079 ***
It's not a duplicate :) For decodebin2 it was fixed by the above commit.
Oh right, sorry!