GNOME Bugzilla – Bug 766978
vaapidecode: mpeg-2 don't render with ximagesink
Last modified: 2018-11-03 15:47:49 UTC
$ gst-play-1.0 https://people.freedesktop.org/~tpm/samples/bbcnews.m2t Doesn't work at all for me with gstreamer-vaapi (i.e. if vaapisink is picked as sink which it is by default). Lots of criticals and a pink image. Works with glimagesink and xvimagesink.
Hi Tim, Which hardware are you using? Because I tested it, as is, in my Haswell and I don't see those artifacts neither the criticals.
$ vainfo libva info: VA-API version 0.39.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_0_39 libva info: va_openDriver() returns 0 vainfo: VA-API version: 0.39 (libva 1.7.0) vainfo: Driver version: Intel i965 driver for Intel(R) Haswell Mobile - 1.7.0 vainfo: Supported profile and entrypoints VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Simple : VAEntrypointEncSlice VAProfileMPEG2Main : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice VAProfileH264High : VAEntrypointVLD VAProfileH264High : VAEntrypointEncSlice VAProfileH264MultiviewHigh : VAEntrypointVLD VAProfileH264MultiviewHigh : VAEntrypointEncSlice VAProfileH264StereoHigh : VAEntrypointVLD VAProfileH264StereoHigh : VAEntrypointEncSlice VAProfileVC1Simple : VAEntrypointVLD VAProfileVC1Main : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD VAProfileNone : VAEntrypointVideoProc VAProfileJPEGBaseline : VAEntrypointVLD
Created attachment 328722 [details] debug log (xz) 0:00:01.949885021 6898 0x7f5094002b20 ERROR vaapivideomemory gstvaapivideomemory.c:259:gst_video_meta_map_vaapi_memory: failed to make image current 0:00:01.949913223 6898 0x7f5094002b20 ERROR default video-frame.c:161:gst_video_frame_map_id: failed to map video frame plane 0 WARNING Internal GStreamer error: code not implemented. Please file a bug at http://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer. WARNING debug information: gstvideofilter.c(292): gst_video_filter_transform (): /GstPlayBin:playbin/GstPlaySink:playsink/GstBin:vdbin/GstVideoConvert:vdconv: invalid video buffer received Full log attached, compressed with xz.
Just a quick guess. I too can't reproduce the issue. But my gstreamer-environment in not the latest. So could be something related with recent changes in upstream ???
Works with gstreamer-vaapi 1.8.0 (and gst master)
git bisect yields: 7baacda91c1e2f66d4618669862b7b6f7dfc4f6c is the first bad commit commit 7baacda91c1e2f66d4618669862b7b6f7dfc4f6c Author: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com> Date: Thu May 5 18:23:10 2016 +0200 vaapipostproc: negotiate frame size fixation Refactor _fixate_frame_size(). Now, instead of fixating the frame size only using the sink caps, also it use the next capsfilter. This code is a shameless copy of gst_video_scale_fixate_caps() from https://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/gst/videoscale/gstvideoscale.c?id=1.8.1#n634 https://bugzilla.gnome.org/show_bug.cgi?id=758548
:( I don't understand. That commit only get the frame resize in downstream caps. I have all gstreamer repos in master and I can't reproduce the issue. Perhaps I should make a clean setup of everything.
If I revert just that one commit, it works again, fwiw.
Great! Though I don't get those errors, what I found in this particular sample is resized, from 720x576 to 1024x576, because of the display-aspect-ratio. But also, glimagesink and ximagesink handle well that resize, but vaapisink doesn't.
(not directly related, but isn't vaapidecode/postproc supposed to do some deinterlacing here?)
(In reply to Tim-Philipp Müller from comment #10) > (not directly related, but isn't vaapidecode/postproc supposed to do some > deinterlacing here?) vaapipostproc should do deinterlacing.
Seems to work again now for me with git master and vaapisink or glimagesink. I have problems with xvimagesink now though. The bccnews clip just flashes green frames after a few seconds, and with another H264-HD-in-TS clip I get green macroblock-height rows but the pic is also distorted and wrong colours.
(In reply to Tim-Philipp Müller from comment #12) > I have problems with xvimagesink now though. The bccnews clip just flashes > green frames after a few seconds, and with another H264-HD-in-TS clip I get > green macroblock-height rows but the pic is also distorted and wrong colours. Just as a side comment, with bbcnews clip, the green frames start to show up when the CC is rendered. By the way, I had send a patch to libva-intel-driver to fix the crash of bbcnews with ximagesink: https://lists.freedesktop.org/archives/libva/2016-June/004026.html
I also noticed that the subtitles are stretched horizontally with vaapi compared to software rendering, perhaps that's related? The other clip I tested was MPEG-2 after all actually, not H264, but without subs.
(In reply to Tim-Philipp Müller from comment #14) > I also noticed that the subtitles are stretched horizontally with vaapi > compared to software rendering, perhaps that's related? > > The other clip I tested was MPEG-2 after all actually, not H264, but without > subs. The vaapidecodebin within the decodebin scales the video for 1/1 p-a-r and deinterlaces the video. I think the scaling is interfering with the later subtitle overlay in the playbin. I'm not yet familiar enough with dvb subtitles to know what the right thing to do is, but I'll look into it.
(In reply to Tim-Philipp Müller from comment #12) > Seems to work again now for me with git master and vaapisink or glimagesink. > > I have problems with xvimagesink now though. The bccnews clip just flashes > green frames after a few seconds, and with another H264-HD-in-TS clip I get > green macroblock-height rows but the pic is also distorted and wrong colours. Yes, confirmed. Somehow vaapipostproc pushes empty buffers or xvimagesink renders empty buffers **@Tim, just wondering, is this still a blocker for 1.10?** (In reply to Scott D Phillips from comment #15) > (In reply to Tim-Philipp Müller from comment #14) > > The vaapidecodebin within the decodebin scales the video for 1/1 p-a-r and > deinterlaces the video. I think the scaling is interfering with the later > subtitle overlay in the playbin. I'm not yet familiar enough with dvb > subtitles to know what the right thing to do is, but I'll look into it. Setting vaapidecodebin as GST_RANK_NONE, vaapimpeg2dec is used (haswell) and works fine with xvimagesink (deinterlacing by software). Also, running gst-play-1.0 ~/patterns/bug766978.m2t --videosink=xvimagesink --flags=0x613 (disabling subtitles) the play -with vaapidecodebin- is OK. This empty buffers are result of scaling + deinterlacing + subtitle rendering, as you found.
> > I have problems with xvimagesink now though. The bccnews clip just flashes > > green frames after a few seconds, and with another H264-HD-in-TS clip I get > > green macroblock-height rows but the pic is also distorted and wrong colours. > > Yes, confirmed. Somehow vaapipostproc pushes empty buffers or xvimagesink > renders empty buffers > > **@Tim, just wondering, is this still a blocker for 1.10?** Well, it's less of a blocker than it was before, but the overall situation with gstreamer-vaapi or this decoder/postproc combination is far from great: - with xvimagesink: green frames/flashes, unusuable - with ximagesink: just aborts (driver assert afaict) - with glimagesink: works, even subs rendered right - with vaapisink: works, subs not scaled right We're having a decoder with primary rank that's pretty much unusable unless the application picks vaapisink or glimagesink, which is not great.
Just to confirm, is this something only reproducible in HSW or older hw? Can't see any issue in SKL..
(In reply to sreerenj from comment #18) > Just to confirm, is this something only reproducible in HSW or older hw? > Can't see any issue in SKL.. I had to update my setups for SKL and KBL and in both the green frames appear when using xvimagesink. It is something related with mapping deinterlaced frames and composite them with subtitles.
(In reply to Tim-Philipp Müller from comment #17) > Well, it's less of a blocker than it was before, but the overall situation > with gstreamer-vaapi or this decoder/postproc combination is far from great: > > - with xvimagesink: green frames/flashes, unusuable Yep. The user needs to disable the subtitles :( The same behavior happens with kmssink. > - with ximagesink: just aborts (driver assert afaict) Using the HEAD of intel-driver it doesn't abort, but only black frames are shown. > - with glimagesink: works, even subs rendered right > - with vaapisink: works, subs not scaled right > > We're having a decoder with primary rank that's pretty much unusable unless > the application picks vaapisink or glimagesink, which is not great. Agree.
Created attachment 334971 [details] [review] videomemory: load also VA image from surface in case of GST_MAP_WRITE flag The problem with xvimagesink is caused by not loading valid image in vaapi video map function. I also tested with ximagesink based on the latest intel driver, but it's still showing black frames even without dvbsuboverlay. Probably, this is driver's issue.
Comment on attachment 334971 [details] [review] videomemory: load also VA image from surface in case of GST_MAP_WRITE flag This patch fixes the issue with xvimagesink for me as well, thanks!
Comment on attachment 334971 [details] [review] videomemory: load also VA image from surface in case of GST_MAP_WRITE flag This a long time issue. Look at bug 704078 for the debate. Perhaps we should check for GST_MAP_READ or GST_MAP_WRITE, but no GST_MAP_READWRITE
Created attachment 335005 [details] [review] libs: surface: ensure overlay is not bigger Ensure that the composition overlay rectangle (subtitles) is not bigger than the surface where it is going to be composite and render.
(In reply to Víctor Manuel Jáquez Leal from comment #23) > Comment on attachment 334971 [details] [review] [review] > videomemory: load also VA image from surface in case of GST_MAP_WRITE flag > > This a long time issue. Look at bug 704078 for the debate. > > Perhaps we should check for GST_MAP_READ or GST_MAP_WRITE, but no > GST_MAP_READWRITE Thanks for letting me know about history of this code. I was wondering why :) Discussion on bug 704083 helps me understanding a lot. IIUC, dvbsuboverlay should change the map flag.
Created attachment 335056 [details] [review] dvbsuboverlay: use GST_MAP_READWRITE instead of only GST_MAP_WRITE when calling gst_video_frame_map To work with some HW decoders correctly, it needs to specify video map flag clearly.
Review of attachment 335056 [details] [review]: Nope. I didn't mean that. The patch for vaapi mapping is correct, the issue is that we have to forbid the GST_MAP_READWRITE flag, but we can handel GST_MAP_READ or GST_MAP_WRITE... not both at the same time.
Review of attachment 334971 [details] [review]: I would use this if (((flags & GST_MAP_READWRITE) != GST_MAP_READWRITE) && !ensure_image_is_current (mem))
I have committed both patches with the asked changes. I won't close the bug because we still need to figure out what's happening with ximagesink.
Aaaggrr!!! The patch for mapping the memory for writing breaks the use case of H264 decoding + external subtitles + xvimagesink I won't revert it now. I'll try to find a proper fix tomorrow.
I have reverted my version of the patch in attachment 334971 [details] [review], and created the bug 771382. Hyunjun was right, but I didn't realize why until I checked that gst_video_overlay_composition_blend() reads and writes the frame. So, in order to fix this bug for xvimagesink, we need to merge that patch.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/39.