GNOME Bugzilla – Bug 794306
vaapih264enc caps negotiation is broken by default
Last modified: 2018-06-06 09:23:20 UTC
It also fails if one puts h264parse or rtph264pay afterwards without forcing a profile. $ gst-launch-1.0 -v videotestsrc ! vaapih264enc ! video/x-h264 ! fakesink failed to open /usr/lib64/dri/hybrid_drv_video.so Failed to wrapper hybrid_drv_video.so failed to open /usr/lib64/dri/hybrid_drv_video.so Failed to wrapper hybrid_drv_video.so Setting pipeline to PAUSED ... failed to open /usr/lib64/dri/hybrid_drv_video.so Failed to wrapper hybrid_drv_video.so Pipeline is PREROLLING ... Got context from element 'vaapiencodeh264-0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayWayland\)\ vaapidisplaywayland0"; /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(string)NV12, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Internal data stream error. Additional debug info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: streaming stopped, reason not-negotiated (-4) ERROR: pipeline doesn't want to preroll. Setting pipeline to NULL ... Freeing pipeline ...
It works if you specify the profile after the encoder, so it's clearly a caps negotiation bug.
That's odd. It works for me (intel vaapi driver with skylake): gst-launch-1.0 -v videotestsrc ! vaapih264enc ! video/x-h264 ! fakesink Setting pipeline to PAUSED ... Pipeline is PREROLLING ... Got context from element 'vaapiencodeh264-0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayGLX\)\ vaapidisplayglx0"; /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(string)NV12, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstVaapiEncodeH264:vaapiencodeh264-0.GstPad:sink: caps = video/x-raw, width=(int)320, height=(int)240, framerate=(fraction)30/1, format=(string)NV12, multiview-mode=(string)mono, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstVaapiEncodeH264:vaapiencodeh264-0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)high, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, chroma-site=(string)jpeg, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)high, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, chroma-site=(string)jpeg, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono /GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)high, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, chroma-site=(string)jpeg, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)high, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, chroma-site=(string)jpeg, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono Pipeline is PREROLLED ... Setting pipeline to PLAYING ... New clock: GstSystemClock ^Chandling interrupt. Interrupt: Stopping pipeline ... Execution ended after 0:00:32.928997166 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ... if I recall correctly, by default it is chosen the high profile. Does your driver support high profile?
*** Bug 794812 has been marked as a duplicate of this bug. ***
The other bug has some stereo profile selected by default, forcing it to high makes it work.
The debug log from bug #794812 contains lots of bits that say using stereo-high profile as target decoder constraints so maybe it picks a stereo profile as output profile and then wants multiview input because of that and rejects the mono input? Unsetting NEEDINFO because in the other bug it was confirmed that the driver supports high profile, because it works when high profile is forced explicitly with a capsfilter after the encoder.
Created attachment 370354 [details] [review] vaapiencode: h264: find best profile in those available Instead to look for the best profile in the allowed profiles by downstream, the encoder should look for the base profile in the available profile in VA-API.
I don't have a baytrail at hande. Can you please test the posted patch?
Review of attachment 370354 [details] [review]: Seems to work for me.
Let's get that in then?
(In reply to Víctor Manuel Jáquez Leal from comment #7) > I don't have a baytrail at hande. Can you please test the posted patch? Do you still need that test on a baytrail system?
(In reply to Sebastian Dröge (slomo) from comment #9) > Let's get that in then? Yes, please. I has been out of my office this week. I you can push it @slomo/@olivier, please go ahead :)
Nevermind. I have a slot of time to push it (and for 1.14) right now. Sorry for the noise.
Attachment 370354 [details] pushed as fa77b2b - vaapiencode: h264: find best profile in those available
(In reply to Víctor Manuel Jáquez Leal from comment #11) > (In reply to Sebastian Dröge (slomo) from comment #9) > > Let's get that in then? > > Yes, please. I has been out of my office this week. I you can push it > @slomo/@olivier, please go ahead :) I just tested the patch on my Baytrail system, and its working well. https://pastebin.com/YVeLXHAA
On branch 1.14 Attachment 370354 [details] pushed as 67e33d3d - vaapiencode: h264: find best profile in those available
This commit breaks negotiation for me with this pipeline: gst-launch-1.0 videotestsrc ! vaapih264enc ! h264parse ! \ video/x-h264,profile=constrained-baseline ! fakesink Using the intersection like this works for me: GstCaps *tmp_caps = gst_caps_intersect(allowed_caps, available_caps); profile = find_best_profile (tmp_caps); I'm not sure if this is the correct solution.
Created attachment 372375 [details] [review] vaapiencode: h264: find best profile available and allowed if possible The commit 67e33d3de225d0e006d7bf606e7abb20d4544eab ("vaapiencode: h264: find best profile in those available") changed the code to pick a profile that is actually supported by the hardware. Unfortunately it dropped the downstream constraints. This can cause negotiation failures under certain circumstances. Fix this by picking a profile that is supported and allowed by the downstream constraints if possible. How about this?
Review of attachment 372375 [details] [review]: overall it looks good ::: gst/vaapi/gstvaapiencode_h264.c @@ +326,3 @@ } /* Check whether "stream-format" is avcC mode */ I wonder if we should use profile_caps to look for the avcC mode
Created attachment 372563 [details] [review] vaapiencode: h264: find profile in available and allowed caps The commit 67e33d3de225d0e006d7bf606e7abb20d4544eab ("vaapiencode: h264: find best profile in those available") changed the code to pick a profile that is actually supported by the hardware. Unfortunately it dropped the downstream constraints. This can cause negotiation failures under certain circumstances. The fix is split in two cases: 1\ the available VA-API caps doesn't intersect with pipeline's allowed caps: * The best allowed profile (pipeline's caps) is set as the encoding target profile (it will be adjusted later by the available profiles and properties) 2\ the available VA-API caps does intersect with pipeline's allowed caps: * The intersected caps are fixed, and its profile is set as the encoding target profile. In this case the is not the best profile, but the minimal one (if VA-API reports the profiles in order). Setting the minimal profile of the intersected caps is better for compatibility. This patch fixes other tests related with caps negotiation, for example, it handles baseline profile, even when VA only supports constrained-baseline. Original-patch-by: Michael Olbrich <m.olbrich@pengutronix.de>
*** Bug 795340 has been marked as a duplicate of this bug. ***
Attachment 372563 [details] pushed as af3e6a8 - vaapiencode: h264: find profile in available and allowed caps
branch 1.14 * 1f1e9eef vaapiencode: h264: find profile in available and allowed caps