Bug 759893 – vaapipostproc crashes when negotiates ANY caps

After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.

Bug 759893 - vaapipostproc crashes when negotiates ANY caps


Summary:	vaapipostproc crashes when negotiates ANY caps


Status:	RESOLVED FIXED

Product:	gstreamer-vaapi
Classification:	Other
Component:	general
Version:	git master
Hardware:	Other Linux

Importance:	Normal normal
Target Milestone:	---
Assigned To:	gstreamer-vaapi maintainer(s)
QA Contact:	gstreamer-vaapi maintainer(s)

URL:
Whiteboard:

Depends on:
Blocks:

Reported:	2015-12-26 23:25 UTC by Florent Thiéry
Modified:	2016-01-19 11:48 UTC

See Also:
GNOME target:	---
GNOME version:	---

Attachments
vaapipostproc: check ANY caps at transform_caps() (1.27 KB, patch) 2016-01-14 16:41 UTC, Víctor Manuel Jáquez Leal	committed	Details \| Review

Description Florent Thiéry 2015-12-26 23:25:33 UTC

Running the following pipeline crashes

gst-launch-1.0 videotestsrc num-buffers=10000 ! video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)320\,\ height\=\(int\)240\,\ framerate\=\(fraction\)30/1\,\ interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1 ! vaapipostproc ! vaapiencode_h265 ! fakesink -v

libva info: VA-API version 0.38.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_38
libva info: va_openDriver() returns 0

(gst-launch-1.0:24941): GStreamer-CRITICAL **: gst_caps_get_structure: assertion 'index < GST_CAPS_LEN (caps)' failed

(gst-launch-1.0:24941): GStreamer-CRITICAL **: gst_structure_fixate: assertion 'GST_IS_STRUCTURE (structure)' failed

(gdb) bt

+ Trace 235845

#0 __nanosleep_nocancel
from /usr/lib/libpthread.so.0
#1 g_usleep
from /usr/lib/libglib-2.0.so.0
#2 fault_spin
at gst-launch.c line 112
#3 fault_handler_sighandler
at gst-launch.c line 93
#4 <signal handler called>
#5 gst_caps_fixate
at gstcaps.c line 2147
#6 gst_vaapipostproc_transform_caps_impl
at gstvaapipostproc.c line 1106
#7 gst_vaapipostproc_transform_caps
at gstvaapipostproc.c line 1144
#8 gst_base_transform_transform_caps
at gstbasetransform.c line 536
#9 gst_base_transform_query_caps
at gstbasetransform.c line 757
#10 gst_base_transform_default_query
at gstbasetransform.c line 1596
#11 gst_pad_query
at gstpad.c line 3898
#12 gst_pad_query_caps
at gstutils.c line 2841
#13 gst_element_get_compatible_pad
#14 gst_element_link_pads_full
at gstutils.c line 1691
#15 gst_element_link_pads
at gstutils.c line 1874
#16 gst_element_link_pads_filtered
at gstutils.c line 1956
#17 gst_parse_perform_link
at ./grammar.y line 574
#18 priv_gst_parse_launch
#19 gst_parse_launch_full
#20 gst_parse_launchv_full
at gstparse.c line 262
#21 gst_parse_launchv
at gstparse.c line 215
#22 main
at gst-launch.c line 998

Comment 1 Florent Thiéry 2015-12-26 23:27:36 UTC

NB: Hardware is Intel(R) Core(TM) i5-3230M CPU @ 2.60GHz

Comment 2 Víctor Manuel Jáquez Leal 2016-01-14 16:31:50 UTC

mmmhh... it looks like we should handle the ANY caps when negotiating the caps in vaapipostroc.

This shorten version of the pipeline also reproduce the error:

gst-launch-1.0 videotestsrc num-buffers=10000 ! video/x-raw,format=I420,width=320,height=240,framerate=30/1,interlace-mode=progressive ! vaapipostproc ! vaapisink

What surprise me is that removing the interlace-mode property (or any other else) the pipeline is negotiated correctly.

Comment 3 Víctor Manuel Jáquez Leal 2016-01-14 16:41:24 UTC

Created attachment 319026 [details] [review]
vaapipostproc: check ANY caps at transform_caps()

When transforming downstream caps we should check for ANY caps from peer pad,
otherwise we get a segmentation fault.

Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>

Comment 4 sreerenj 2016-01-19 06:44:01 UTC

Review of attachment 319026 [details] [review]:

lgtm

Comment 5 Víctor Manuel Jáquez Leal 2016-01-19 11:48:15 UTC

Attachment 319026 [details] pushed as b8da44b - vaapipostproc: check ANY caps at transform_caps()