After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 775490 - vaapi: encode: jpeg should release configuration after each frame (multi file support)
vaapi: encode: jpeg should release configuration after each frame (multi file...
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gstreamer-vaapi
1.10.1
Other Linux
: Normal minor
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2016-12-01 20:05 UTC by Reza Razavi
Modified: 2017-02-22 14:48 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
debug output for vaapi*:6 (20.58 KB, text/plain)
2016-12-01 20:06 UTC, Reza Razavi
  Details
Added a stop function to allow operation with multifilesrc. (1.28 KB, patch)
2016-12-13 21:55 UTC, Stirling Westrup
none Details | Review
libs: utils: return NULL if failed to get surface formats (946 bytes, patch)
2016-12-15 07:00 UTC, Hyunjun Ko
committed Details | Review
libs: utils: return NULL if failed to get surface formats (946 bytes, patch)
2016-12-15 07:02 UTC, Hyunjun Ko
none Details | Review
Combined Patch for mulitfiles and in-stream renegotiation (5.27 KB, patch)
2017-01-19 17:34 UTC, Stirling Westrup
needs-work Details | Review
libs: encoder: caps can change at any time (1.57 KB, patch)
2017-02-17 01:07 UTC, Víctor Manuel Jáquez Leal
committed Details | Review

Description Reza Razavi 2016-12-01 20:05:02 UTC
I want to use vaapijpeganc as a replacement for jpegenc element, I can successfully run this command :
gst-launch-1.0 multifilesrc  location="/home/reza/Pictures/%01d.jpg" index=1  ! decodebin ! jpegenc ! fakesink sync=true

but same command fails with vaapijpegenc fails :
gst-launch-1.0 --gst-debug=*:3 multifilesrc  location="/home/reza/Pictures/%01d.jpg" index=1  ! decodebin ! vaapijpegenc ! fakesink sync=true

I'll attach full debug output of vaapi components for this pipeline to this bug.
Comment 1 Reza Razavi 2016-12-01 20:06:22 UTC
Created attachment 341176 [details]
debug output for vaapi*:6
Comment 2 Víctor Manuel Jáquez Leal 2016-12-02 12:09:49 UTC
Confirmed.

The problem is that vaapijpegenc doesn't support multifilesrc currently. The fix should be trivial: release caps, pools, etc. after finishing every image.
Comment 3 Reza Razavi 2016-12-02 14:26:24 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #2)
> Confirmed.
> 
> The problem is that vaapijpegenc doesn't support multifilesrc currently. The
> fix should be trivial: release caps, pools, etc. after finishing every image.

This clean up should happen after finishing every image or whenever pipeline state changes from PLAY state, like adding stop function to vaapi:encode ?
my original usecase is encoding video play back frames into jpegs and this problem happens when the video changes, I thought maybe doing clean up after every image will impact the performance.
Comment 4 Stirling Westrup 2016-12-13 21:55:15 UTC
Created attachment 341919 [details] [review]
Added a stop function to allow operation with multifilesrc.

Here's a proposed patch. It appears to work with our test frameworks.
Comment 5 Hyunjun Ko 2016-12-14 01:05:22 UTC
(In reply to Stirling Westrup from comment #4)
> Created attachment 341919 [details] [review] [review]
> Added a stop function to allow operation with multifilesrc.
> 
> Here's a proposed patch. It appears to work with our test frameworks.

Lately, there was similar commit in https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/commit/?id=55aa83c5bf7b4408db2b23c0a88c6e1f4e408896

Could you test on the master?
Comment 6 Víctor Manuel Jáquez Leal 2016-12-14 08:47:17 UTC
Review of attachment 341919 [details] [review]:

::: gst/vaapi/gstvaapiencode.c
@@ +671,3 @@
+{
+  GstVaapiEncode *const encode = GST_VAAPIENCODE_CAST (venc);
+  gst_vaapiencode_flush (venc);

This is a good idea. We should add it. Could you rebase your patch with master?
Comment 7 Stirling Westrup 2016-12-14 18:29:52 UTC
The gstreamer-vaapi in master appears quite broken. We can't test our patch on it as there seem to be failures in base functionality.
Comment 8 Reza Razavi 2016-12-14 19:00:18 UTC
gstreamer and all the plugins of it are current master, vaapi is also build from master vaapijpegdec and vaapijpegenc are completely broken, while video decoders and vaapisink are working fine. 


gst-launch-1.0 --gst-debug=vaapi*:6 filesrc location=/home/reza/Pictures/1.jpg ! vaapijpegdec ! fakesink 
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
0:00:00.132153495  8842       0x92e8f0 INFO                   vaapi gstvaapidisplay.c:131:libgstvaapi_init_once: gstreamer-vaapi version 1.11.0-79-gdaed911
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
0:00:00.133034302  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:813:gst_vaapi_display_calculate_pixel_aspect_ratio: calculated pixel aspect ratio: 1.000000
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
0:00:00.133224200  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:953:gst_vaapi_display_create_unlocked: VA-API version 0.39
0:00:00.133237720  8842       0x92e8f0 INFO            vaapidisplay gstvaapidisplay.c:961:gst_vaapi_display_create_unlocked:<vaapidisplaydrm0> new display addr=0xb080b0
0:00:00.133246217  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:549:ensure_profiles: 14 profiles
0:00:00.133251026  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileMPEG2Simple
0:00:00.133254216  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileMPEG2Main
0:00:00.133257057  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264ConstrainedBaseline
0:00:00.133260293  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264Main
0:00:00.133263200  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264High
0:00:00.133266106  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264MultiviewHigh
0:00:00.133271415  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264StereoHigh
0:00:00.133275300  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileVC1Simple
0:00:00.133278538  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileVC1Main
0:00:00.133284339  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileVC1Advanced
0:00:00.133288155  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileJPEGBaseline
0:00:00.133290926  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileVP8Version0_3
0:00:00.133294059  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   <unknown>
0:00:00.133606205  8842       0x92e8f0 DEBUG            vaapidecode gstvaapidecode.c:1195:gst_vaapidecode_sink_getcaps:<vaapidecode_jpeg0> Returning sink caps image/jpeg, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:00.133628914  8842       0x92e8f0 DEBUG            vaapidecode gstvaapidecode.c:1195:gst_vaapidecode_sink_getcaps:<vaapidecode_jpeg0> Returning sink caps image/jpeg, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
Setting pipeline to PAUSED ...
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
0:00:00.134216220  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:813:gst_vaapi_display_calculate_pixel_aspect_ratio: calculated pixel aspect ratio: 1.000000
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
0:00:00.134400066  8842       0x92e8f0 DEBUG           vaapidisplay gstvaapidisplay.c:953:gst_vaapi_display_create_unlocked: VA-API version 0.39
0:00:00.134406461  8842       0x92e8f0 INFO            vaapidisplay gstvaapidisplay.c:961:gst_vaapi_display_create_unlocked:<vaapidisplaydrm1> new display addr=0xb08410
0:00:00.134413139  8842       0x92e8f0 INFO             vaapidecode gstvaapipluginbase.c:49:plugin_set_display:<vaapidecode_jpeg0> set display <vaapidisplaydrm1>
Pipeline is PREROLLING ...
Got context from element 'vaapidecode_jpeg0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayDRM\)\ vaapidisplaydrm1";

** (gst-launch-1.0:8842): CRITICAL **: gst_vaapi_decoder_parse: assertion 'decoder != NULL' failed
0:00:00.134573871  8842       0xb0c4a0 ERROR            vaapidecode gstvaapidecode.c:1094:gst_vaapidecode_parse_frame: parse error 11

** (gst-launch-1.0:8842): CRITICAL **: gst_vaapi_decoder_parse: assertion 'decoder != NULL' failed
0:00:00.134590001  8842       0xb0c4a0 ERROR            vaapidecode gstvaapidecode.c:1094:gst_vaapidecode_parse_frame: parse error 11
ERROR: from element /GstPipeline:pipeline0/GstVaapiDecode_jpeg:vaapidecode_jpeg0: No valid frames decoded before end of stream
Additional debug info:
gstvideodecoder.c(1179): gst_video_decoder_sink_event_default (): /GstPipeline:pipeline0/GstVaapiDecode_jpeg:vaapidecode_jpeg0:
no valid frames found
ERROR: pipeline doesn't want to preroll.
Comment 9 Reza Razavi 2016-12-14 19:10:19 UTC
and this is how vaapijpegenc fails :
gst-launch-1.0 --gst-debug=vaapi*:6 filesrc location=/home/reza/Pictures/1.jpg ! jpegdec ! vaapijpegenc ! fakesink sync=true
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
0:00:00.129384948  9327       0xbb2b80 INFO                   vaapi gstvaapidisplay.c:131:libgstvaapi_init_once: gstreamer-vaapi version 1.11.0-79-gdaed911
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
0:00:00.132173347  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:813:gst_vaapi_display_calculate_pixel_aspect_ratio: calculated pixel aspect ratio: 1.000000
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
0:00:00.132882380  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:953:gst_vaapi_display_create_unlocked: VA-API version 0.39
0:00:00.132940361  9327       0xbb2b80 INFO            vaapidisplay gstvaapidisplay.c:961:gst_vaapi_display_create_unlocked:<vaapidisplaydrm0> new display addr=0xc7c2f0
0:00:00.132970596  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:549:ensure_profiles: 14 profiles
0:00:00.132984939  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileMPEG2Simple
0:00:00.132997110  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileMPEG2Main
0:00:00.133014085  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264ConstrainedBaseline
0:00:00.133029049  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264Main
0:00:00.133042609  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264High
0:00:00.133055640  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264MultiviewHigh
0:00:00.133067668  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264StereoHigh
0:00:00.133081702  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileVC1Simple
0:00:00.133094460  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileVC1Main
0:00:00.133104880  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileVC1Advanced
0:00:00.133119097  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileJPEGBaseline
0:00:00.133132144  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileVP8Version0_3
0:00:00.133145790  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   <unknown>
0:00:00.134539264  9327       0xbb2b80 DEBUG            vaapiencode gstvaapiencode.c:410:gst_vaapiencode_get_caps:<vaapiencodejpeg0> Returning sink caps video/x-raw(memory:VASurface), format=(string){ ENCODED, NV12, I420, YV12, P010_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string)progressive; video/x-raw, format=(string){ I420, YV12, YUY2, UYVY, VYUY, AYUV, RGBx, BGRx, xRGB, xBGR, RGBA, BGRA, ARGB, ABGR, RGB, BGR, Y41B, Y42B, YVYU, Y444, v210, v216, NV12, NV21, NV16, NV61, NV24, GRAY8, GRAY16_BE, GRAY16_LE, v308, IYU2, RGB16, BGR16, RGB15, BGR15, UYVP, A420, RGB8P, YUV9, YVU9, IYU1, ARGB64, AYUV64, r210, I420_10LE, I420_10BE, I422_10LE, I422_10BE, Y444_10LE, Y444_10BE, GBR, GBR_10LE, GBR_10BE, NV12_64Z32, A420_10LE, A420_10BE, A422_10LE, A422_10BE, A444_10LE, A444_10BE, P010_10LE, P010_10BE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string)progressive
0:00:00.134750175  9327       0xbb2b80 DEBUG            vaapiencode gstvaapiencode.c:410:gst_vaapiencode_get_caps:<vaapiencodejpeg0> Returning sink caps video/x-raw(memory:VASurface), format=(string){ ENCODED, NV12, I420, YV12, P010_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string)progressive; video/x-raw, format=(string){ I420, YV12, YUY2, UYVY, VYUY, AYUV, RGBx, BGRx, xRGB, xBGR, RGBA, BGRA, ARGB, ABGR, RGB, BGR, Y41B, Y42B, YVYU, Y444, v210, v216, NV12, NV21, NV16, NV61, NV24, GRAY8, GRAY16_BE, GRAY16_LE, v308, IYU2, RGB16, BGR16, RGB15, BGR15, UYVP, A420, RGB8P, YUV9, YVU9, IYU1, ARGB64, AYUV64, r210, I420_10LE, I420_10BE, I422_10LE, I422_10BE, Y444_10LE, Y444_10BE, GBR, GBR_10LE, GBR_10BE, NV12_64Z32, A420_10LE, A420_10BE, A422_10LE, A422_10BE, A444_10LE, A444_10BE, P010_10LE, P010_10BE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string)progressive
Setting pipeline to PAUSED ...
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
0:00:00.136884976  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:813:gst_vaapi_display_calculate_pixel_aspect_ratio: calculated pixel aspect ratio: 1.000000
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
0:00:00.137622475  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:953:gst_vaapi_display_create_unlocked: VA-API version 0.39
0:00:00.137650673  9327       0xbb2b80 INFO            vaapidisplay gstvaapidisplay.c:961:gst_vaapi_display_create_unlocked:<vaapidisplaydrm1> new display addr=0xc7c410
0:00:00.137683322  9327       0xbb2b80 INFO             vaapiencode gstvaapipluginbase.c:49:plugin_set_display:<vaapiencodejpeg0> set display <vaapidisplaydrm1>
0:00:00.137863354  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:549:ensure_profiles: 14 profiles
0:00:00.137885785  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileMPEG2Simple
0:00:00.137898368  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileMPEG2Main
0:00:00.137909191  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264ConstrainedBaseline
0:00:00.137921159  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264Main
0:00:00.137933301  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264High
0:00:00.137947360  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264MultiviewHigh
0:00:00.137961440  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileH264StereoHigh
0:00:00.137975158  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileVC1Simple
0:00:00.137988824  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileVC1Main
0:00:00.138001558  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileVC1Advanced
0:00:00.138014518  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileJPEGBaseline
0:00:00.138028248  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   VAProfileVP8Version0_3
0:00:00.138042167  9327       0xbb2b80 DEBUG           vaapidisplay gstvaapidisplay.c:556:ensure_profiles:   <unknown>
0:00:00.138076484  9327       0xbb2b80 DEBUG                  vaapi gstvaapiutils.c:53:vaapi_check_status: vaGetConfigAttributes(): the requested VAEntryPoint is not supported
0:00:00.138095408  9327       0xbb2b80 INFO                   vaapi gstvaapiencoder.c:869:get_rate_control_mask: supported rate controls: 0x00000000
Pipeline is PREROLLING ...
Got context from element 'vaapiencodejpeg0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayDRM\)\ vaapidisplaydrm1";
0:00:00.139224414  9327       0xa9bf20 DEBUG            vaapiencode gstvaapiencode.c:410:gst_vaapiencode_get_caps:<vaapiencodejpeg0> Returning sink caps EMPTY
0:00:00.139359209  9327       0xa9bf20 DEBUG            vaapiencode gstvaapiencode.c:410:gst_vaapiencode_get_caps:<vaapiencodejpeg0> Returning sink caps EMPTY
0:00:00.144823675  9327       0xa9bf20 DEBUG            vaapiencode gstvaapiencode.c:410:gst_vaapiencode_get_caps:<vaapiencodejpeg0> Returning sink caps EMPTY
0:00:00.144846629  9327       0xa9bf20 DEBUG            vaapiencode gstvaapiencode.c:410:gst_vaapiencode_get_caps:<vaapiencodejpeg0> Returning sink caps EMPTY
0:00:00.144899193  9327       0xa9bf20 DEBUG            vaapiencode gstvaapiencode.c:410:gst_vaapiencode_get_caps:<vaapiencodejpeg0> Returning sink caps EMPTY
0:00:00.144938397  9327       0xa9bf20 DEBUG            vaapiencode gstvaapiencode.c:410:gst_vaapiencode_get_caps:<vaapiencodejpeg0> Returning sink caps EMPTY
ERROR: from element /GstPipeline:pipeline0/GstFileSrc:filesrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(2951): gst_base_src_loop (): /GstPipeline:pipeline0/GstFileSrc:filesrc0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
0:00:00.145032996  9327       0xbb2b80 DEBUG                  vaapi gstvaapicontext.c:100:context_destroy: context 0xffffffff
Comment 10 Hyunjun Ko 2016-12-15 06:50:10 UTC
(In reply to Reza Razavi from comment #8)
> 
> gst-launch-1.0 --gst-debug=vaapi*:6 filesrc
> location=/home/reza/Pictures/1.jpg ! vaapijpegdec ! fakesink 

Maybe you should use jpegparse in front of vaapijpegdec.
Comment 11 Hyunjun Ko 2016-12-15 06:56:28 UTC
(In reply to Reza Razavi from comment #9)
> and this is how vaapijpegenc fails :
> gst-launch-1.0 --gst-debug=vaapi*:6 filesrc
> location=/home/reza/Pictures/1.jpg ! jpegdec ! vaapijpegenc ! fakesink
> sync=true

This is regression due to commits in #bug 769266, definetely.

> 0:00:00.139224414  9327       0xa9bf20 DEBUG            vaapiencode
> gstvaapiencode.c:410:gst_vaapiencode_get_caps:<vaapiencodejpeg0> Returning
> sink caps EMPTY
> 0:00:00.139359209  9327       0xa9bf20 DEBUG            vaapiencode
> gstvaapiencode.c:410:gst_vaapiencode_get_caps:<vaapiencodejpeg0> Returning
> sink caps EMPTY
> 0:00:00.144823675  9327       0xa9bf20 DEBUG            vaapiencode
> gstvaapiencode.c:410:gst_vaapiencode_get_caps:<vaapiencodejpeg0> Returning
> sink caps EMPTY
> 0:00:00.144846629  9327       0xa9bf20 DEBUG            vaapiencode
> gstvaapiencode.c:410:gst_vaapiencode_get_caps:<vaapiencodejpeg0> Returning
> sink caps EMPTY
> 0:00:00.144899193  9327       0xa9bf20 DEBUG            vaapiencode
> gstvaapiencode.c:410:gst_vaapiencode_get_caps:<vaapiencodejpeg0> Returning
> sink caps EMPTY
> 0:00:00.144938397  9327       0xa9bf20 DEBUG            vaapiencode
> gstvaapiencode.c:410:gst_vaapiencode_get_caps:<vaapiencodejpeg0> Returning
> sink caps EMPTY

In case of jpegdecoder, it couldn't get surface formats via gst_vaapi_get_surface_formats.
So returned formats is empty, which leads to raw sink caps including "format=  ".
This is why it failed to make proper sink caps.

> ERROR: from element /GstPipeline:pipeline0/GstFileSrc:filesrc0: Internal
> data stream error.
> Additional debug info:
> gstbasesrc.c(2951): gst_base_src_loop ():
> /GstPipeline:pipeline0/GstFileSrc:filesrc0:
> streaming stopped, reason not-negotiated (-4)
> ERROR: pipeline doesn't want to preroll.
> Setting pipeline to NULL ...
> 0:00:00.145032996  9327       0xbb2b80 DEBUG                  vaapi
> gstvaapicontext.c:100:context_destroy: context 0xffffffff
Comment 12 Hyunjun Ko 2016-12-15 07:00:53 UTC
Created attachment 341999 [details] [review]
libs: utils: return NULL if failed to get surface formats

@Victor, does this patch make sense?
Comment 13 Hyunjun Ko 2016-12-15 07:02:50 UTC
Created attachment 342001 [details] [review]
libs: utils: return NULL if failed to get surface formats

@Victor, does this patch make sense?
Comment 14 Víctor Manuel Jáquez Leal 2016-12-15 10:36:54 UTC
I wonder if the problem is in the intel vaapi driver. The jpegencoder context doesn't return any color formats, that's strange. We should look at the driver.
Comment 15 Reza Razavi 2016-12-15 13:29:23 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #14)
> I wonder if the problem is in the intel vaapi driver. The jpegencoder
> context doesn't return any color formats, that's strange. We should look at
> the driver.

If there is any extra information that I can provide to help please let me know.
This is my system vainfo output :
vainfo 
libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
vainfo: VA-API version: 0.39 (libva 1.7.3)
vainfo: Driver version: Intel i965 driver for Intel(R) Skylake - 1.7.2
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Simple            :	VAEntrypointEncSlice
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSliceLP
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointEncSliceLP
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointEncSliceLP
      VAProfileH264MultiviewHigh      :	VAEntrypointVLD
      VAProfileH264MultiviewHigh      :	VAEntrypointEncSlice
      VAProfileH264StereoHigh         :	VAEntrypointVLD
      VAProfileH264StereoHigh         :	VAEntrypointEncSlice
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileNone                   :	VAEntrypointVideoProc
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointEncPicture
      VAProfileVP8Version0_3          :	VAEntrypointVLD
      VAProfileVP8Version0_3          :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
Comment 16 Stirling Westrup 2017-01-19 17:34:54 UTC
Created attachment 343828 [details] [review]
Combined Patch for mulitfiles and in-stream renegotiation

Okay, we've now gotten vaapijpegenc working with both multifilesrc and with mid-stream renegotiation of caps. I had to restrict the template caps as it turns out the current jpeg encoder simply can't handle RGB formats and rejects them outright.

We backported the 1.11 version of gstreamer-vaapi to 1.10 as our base for this work, and once we had a working bugfix, I unbackported our code and built a patch set. As I don't have a working 1.11 setup, I didn't test the unbackported code, but its small enough I'm pretty confidant it works.
Comment 17 Víctor Manuel Jáquez Leal 2017-01-19 17:57:05 UTC
Review of attachment 343828 [details] [review]:

Thanks. It is better to keep atomic commits, isolating the changes as much as possible.

::: gst/vaapi/gstvaapidecode.c
@@ +218,2 @@
 #if (USE_GLX || USE_EGL)
+  if (gst_vaapi_display_has_opengl (GST_VAAPI_PLUGIN_BASE_DISPLAY (decode))) {

Good catch! could you split this in a different patch?

@@ +225,3 @@
     return FALSE;
   }
+#endif

This should be above, where it was before. It purpose is for possible memory exhaustion.

::: gst/vaapi/gstvaapipostproc.c
@@ +1068,3 @@
+  if (!gst_vaapi_display_has_opengl (GST_VAAPI_PLUGIN_BASE_DISPLAY (postproc))
+      && gl_upload_meta_idx > -1) {
+    gst_caps_remove_structure (caps, gl_upload_meta_idx);

Add this in the same patch.
Comment 18 Stirling Westrup 2017-01-19 22:33:55 UTC
The two patches I combined were combined because the first cannot be tested without the second. So I don't think they're really independent.
Comment 19 Stirling Westrup 2017-01-19 22:35:28 UTC
If you want me to split out the opengl test I can, but would I submit that here, or to the other bug I worked on for hybrid card detection, which it is perhaps more relevant to (although I that bug has now been closed...)
Comment 20 Víctor Manuel Jáquez Leal 2017-02-01 19:06:14 UTC
Comment on attachment 341999 [details] [review]
libs: utils: return NULL if failed to get surface formats

Attachment 341999 [details] pushed as a5650e8 - libs: utils: return NULL if failed to get surface formats
Comment 21 Víctor Manuel Jáquez Leal 2017-02-01 19:15:39 UTC
Now the jpegencoder "works". But need to fix the intel driver since it doesn't report the input color formats. Modifying the caps template it just a workaround, since it breaks the compatibility with other possible drivers.

In the case of multiple files, it only works if the sink caps don't change (image size and color format).
Comment 22 Víctor Manuel Jáquez Leal 2017-02-08 19:03:59 UTC
Fix proposal for intel-vaapi-driver

https://github.com/01org/intel-vaapi-driver/pull/34
Comment 23 Víctor Manuel Jáquez Leal 2017-02-15 15:42:26 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #22)
> Fix proposal for intel-vaapi-driver
> 
> https://github.com/01org/intel-vaapi-driver/pull/34

It is already merged in master of intel-vaapi-driver.
Comment 24 Víctor Manuel Jáquez Leal 2017-02-15 15:52:55 UTC
Summarizing, right now in master is possible to encode multiple images of the same caps. But, not for different ones, is that case still the matter of this bug? Or do we close it now?
Comment 25 Hyunjun Ko 2017-02-16 02:40:53 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #24)
> Summarizing, right now in master is possible to encode multiple images of
> the same caps. But, not for different ones, is that case still the matter of
> this bug? Or do we close it now?

In Stirling's patch, the following is worth merge, I guess.
Looks it has potential bugs without this, even though it doesn't affect at the moment.

encoder->num_codedbuf_queued--;


And what about this?
call gst_vaapiencode_flush (venc); in stop method.
Comment 26 Víctor Manuel Jáquez Leal 2017-02-17 00:25:17 UTC
(In reply to Hyunjun Ko from comment #25)
> (In reply to Víctor Manuel Jáquez Leal from comment #24)
> > Summarizing, right now in master is possible to encode multiple images of
> > the same caps. But, not for different ones, is that case still the matter of
> > this bug? Or do we close it now?
> 
> In Stirling's patch, the following is worth merge, I guess.
> Looks it has potential bugs without this, even though it doesn't affect at
> the moment.
> 
> encoder->num_codedbuf_queued--;

That cannot be merged because that is not the intention of that variable AFAIK. It is a global counter of processed buffers, not the size of the queue.

Though, it seems the fix should be in gst_vaapi_encoder_set_codec_state()

> 
> 
> And what about this?
> call gst_vaapiencode_flush (venc); in stop method.
Comment 27 Víctor Manuel Jáquez Leal 2017-02-17 01:07:54 UTC
Created attachment 346031 [details] [review]
libs: encoder: caps can change at any time

The encoder should be able to change its caps even it is already
processing a stream.

This is suppose to happen after a flush so the codedbuf_queue should
be empty.
Comment 28 Stirling Westrup 2017-02-17 05:31:16 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #26)
> (In reply to Hyunjun Ko from comment #25)
> > (In reply to Víctor Manuel Jáquez Leal from comment #24)
> > > Summarizing, right now in master is possible to encode multiple images of
> > > the same caps. But, not for different ones, is that case still the matter of
> > > this bug? Or do we close it now?
> > 
> > In Stirling's patch, the following is worth merge, I guess.
> > Looks it has potential bugs without this, even though it doesn't affect at
> > the moment.
> > 
> > encoder->num_codedbuf_queued--;
> 
> That cannot be merged because that is not the intention of that variable
> AFAIK. It is a global counter of processed buffers, not the size of the
> queue.
> 

It certainly wasn't clear from context what that counter was for but there is an explicit test of it that prevents caps renegotiation when it is non-zero, so I assumed it was a count of buffers in progress.

Since defeating that test had no negative consequences on functionality and allowed for mid-stream caps renegotiation, I recommend the removal of the test of encoder->num_codedbuf_queued being non-zero.
Comment 29 Víctor Manuel Jáquez Leal 2017-02-22 04:30:46 UTC
Comment on attachment 346031 [details] [review]
libs: encoder: caps can change at any time

Attachment 346031 [details] pushed as a0a2f7b - libs: encoder: caps can change at any time
Comment 30 Víctor Manuel Jáquez Leal 2017-02-22 04:36:50 UTC
(In reply to Hyunjun Ko from comment #25)

> And what about this?
> call gst_vaapiencode_flush (venc); in stop method.

The only missing part is this. But I don't think it is necessary to do this. Can you confirm me it?

Otherwise, I guess we can close this bug report.
Comment 31 Hyunjun Ko 2017-02-22 06:55:10 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #30)
> (In reply to Hyunjun Ko from comment #25)
> 
> > And what about this?
> > call gst_vaapiencode_flush (venc); in stop method.
> 
> The only missing part is this. But I don't think it is necessary to do this.
> Can you confirm me it?
> 
> Otherwise, I guess we can close this bug report.

Confirmed. It's not necessary I think too.
Comment 32 Víctor Manuel Jáquez Leal 2017-02-22 14:48:40 UTC
As everything has landed, closing as fixed.