GNOME Bugzilla – Bug 776084
vaapipostproc: segfault when direct-uploading an unsupported color-space
Last modified: 2016-12-21 10:32:21 UTC
The following segfaults: gst-launch-1.0 filesrc location=/dev/zero ! rawvideoparse width=3840 height=2160 framerate=30 format=rgb ! vaapipostproc ! 'video/x-raw, width=(int)1920, height=(int)1080' ! fakesink
This is caused by a failure in gst_vaapi_video_format_get_best_native to get chroma_type in case that the format is "RGB" If set rgba/rgbx/bgra/bgrx to the sink caps, it's working. But we should fix this crash at least.
Created attachment 342297 [details] [review] vaapivideomemory: change return type of allocator_configure_surface_info To detect and handle errors, which is happening during allocator_configure_surface_info.
Comment on attachment 342297 [details] [review] vaapivideomemory: change return type of allocator_configure_surface_info would be nice to do the same in allocator_configure_image_info()
Created attachment 342310 [details] [review] vaapivideomemory: change return type of allocator_configure_surface/image_info To detect and handle errors, which is happening during execution of allocator_configure_surface/image_info.
Thanks! But this solution doesn't fix the real problem: in this pipeline the negotiation caps occur when the VA display hasn't been created, so all the "possible" formats are negotiated, even though the hardware hasn't been queried for its supported formats. In this case, RGB format is not supported by the hardware, but the caps are still negotiated. The error should have occurred in the caps negotiation. One possible solution is to expose in the caps template the possible raw formats when registering the vaapipostproc element.
committed with minor modifications.