GNOME Bugzilla – Bug 772151
vaapi: gst_vaapi_video_memory_map() should ignore unknown mapping flags
Last modified: 2016-11-11 10:49:41 UTC
Created attachment 336479 [details] Output with GST_DEBUG=5 This command... GST_DEBUG=2 gst-launch-1.0 videotestsrc num-buffers=1 ! video/x-raw,format=I420,width=1277,height=719 ! vaapipostproc ! video/x-raw,format=NV12 ! gamma gamma=2 ! fakesink ...causes these errors to be emitted every frame: 0:00:00.163031112 1669 0x8f0660 WARN structure gststructure.c:1935:priv_gst_structure_append_to_gstring: No value transform to serialize field 'gst.vaapi.Display' of type 'GstVaapiDisplay' Got context from element 'vaapipostproc0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)NULL; 0:00:00.196789933 1669 0xa21320 ERROR vaapivideomemory gstvaapivideomemory.c:437:gst_vaapi_video_memory_map: unsupported map flags (0x10003) 0:00:00.196852361 1669 0xa21320 ERROR GST_MEMORY gstmemory.c:324:gst_memory_map: mem 0x7f687001b0e0: subclass map failed 0:00:00.196881515 1669 0xa21320 ERROR vaapivideomemory gstvaapivideomemory.c:437:gst_vaapi_video_memory_map: unsupported map flags (0x10003) 0:00:00.196891154 1669 0xa21320 ERROR GST_MEMORY gstmemory.c:324:gst_memory_map: mem 0x7f687001b190: subclass map failed 0:00:00.196904182 1669 0xa21320 ERROR default video-frame.c:169:gst_video_frame_map_id: failed to map buffer 0:00:00.196920849 1669 0xa21320 WARN videofilter gstvideofilter.c:340:gst_video_filter_transform_ip:<gamma0> warning: invalid video buffer received WARNING: from element /GstPipeline:pipeline0/GstGamma:gamma0: Internal GStreamer error: code not implemented. Please file a bug at http://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer. Additional debug info: ../../.././../gst-libs/gst/video/gstvideofilter.c(340): gst_video_filter_transform_ip (): /GstPipeline:pipeline0/GstGamma:gamma0: The issue seems to be caused by vaapipostproc negotiating buffers that are not writable.
Created attachment 336480 [details] vainfo output
GstVidoFilter base class maps the buffers with flags that are only managed by gst_video_frame_map (GST_VIDEO_FRAME_MAP_FLAG_NO_REF). First observation: somehow gst_video_frame_map() doesn't get the vaapi meta and uses gst_memory_map() instead of gst_video_meta_map(). Why? Second observation: gst_video_frame_map() doesn't remove the GST_VIDEO_FRAME_MAP_FLAG* bits from flag and passes it as is to the mapping callbacks. Should gst_video_frame_map() remove them? Finally, gst_vaapi_video_memory_map() is very strict with the mapping flags, if there is a flag that doesn't manages, bails out with an error. The main problem is the extra flag that is not managed by gst_vaapi_video_memory_map(). Thus, the question is shall gst_video_frame_map() remove its extra flags or shall gst_vaapi_video_memory_map() relax it flags handling? As comment, normally is very expensive to upload a frame to the video processing unit (the GPU in this case) and download it again to main memory. Perhaps it would be less expensive, in terms of time, to do the color conversion with software too.
More info. This is interesting. Neither GstVideoFilter nor GstGamma propose bufferpool allocation with GST_VIDEO_META_API_TYPE. And vaapi elements won't add GstVideMeta to its buffers if the buffer pool config doesn't have GST_VIDEO_META_API_TYPE in the allocation query. *But*, the vaapi base class adds the GstVideoMeta if the allocations caps are different from the negotiation caps. That is why, under certain resolutions, the mapping works. Again, two options: a) GstVideoFill shall always add in propose allocation's query the GST_VIDEO_META_API_TYPE or b) Vaapi base class shall always add the GstVideoMeta on its downstream frames.
Created attachment 337108 [details] [review] Fix for GstVideoFilter in gst-plugins-base This patch makes the error messages go away, without ill effects to the pipeline as far as I can tell, but I'm not sure if the approach is exactly correct. Can someone review this?
@slomo can you shed some light here? :) The patch proposed by Joel is the way to go, or vaapi elements should stamp GstVideoMeta regardless it is requested by downstream.
Yeah - except on further testing, my patch causes 2 weird problems in our pipeline which encodes to H.264 using vaapiencode. 1. It seems to cause frame dropping 2. It adds about 5-seconds of latency. I'm struggling to understand why this would be.
Comment on attachment 337108 [details] [review] Fix for GstVideoFilter in gst-plugins-base Most of the pipelines fail with this patch.
Created attachment 339390 [details] [review] vaapivideomemory: refactor vaapi memory mapping There were duplicated code in gst_video_meta_map_vaapi_memory() and gst_vaapi_video_memory_map() when doing the READ and WRITE mapping. This patch refactors both methods adding the common function map_vaapi_memory(). Additionally, only when flag has the READ bit it calls ensure_images_is_current(), which was done in gst_video_meta_map_vaapi_memory() but no in gst_vaapi_video_memory_map().
Created attachment 339411 [details] [review] vaapivideomemory: refactor vaapi memory mapping There were duplicated code in gst_video_meta_map_vaapi_memory() and gst_vaapi_video_memory_map() when doing the READ and WRITE mapping. This patch refactors both methods adding the common function map_vaapi_memory(). Additionally, only when flag has the READ bit it calls ensure_images_is_current(), which was done in gst_video_meta_map_vaapi_memory() but no in gst_vaapi_video_memory_map().
Attachment 339411 [details] pushed as eda0323 - vaapivideomemory: refactor vaapi memory mapping