GNOME Bugzilla – Bug 789739
vaapijpegdec: get a dark image because of wrong quantization matrix
Last modified: 2018-11-03 15:51:48 UTC
Hello! I am check it on Gstreamer 1.12.3 and 1.13.0 and have same situatuin. I have two camera, one HP HD 4310, and second camera ELP with IMX Sony 179 sensor. MJPEG images from HP HD 4310, decoded and look fine. Fom ELP cam, it look dark. I am add to this post sample images. https://drive.google.com/open?id=0B3_jMty60ScdRHAwZ3R2VDdXUkU Here i use two pipeline: 1. to create images from camera as is gst-launch-1.0 -v v4l2src num-buffers=100 device=/dev/video1 ! image/jpeg,width=3264,height=2448,framerate=15/1 ! jpegparse ! multifilesink location="frame%d.jpg" 2. to transcode jpeg images in vaapijpegdec: gst-launch-1.0 -v multifilesrc location="frame%d.jpg" ! image/jpeg,width=3264,height=2448,framerate=15/1 ! jpegparse ! vaapijpegdec ! vaapipostproc ! pngenc ! multifilesink location="frame%d.png" 3. to transcode jpeg images in jpegdec: gst-launch-1.0 -v multifilesrc location="frame%d.jpg" ! image/jpeg,width=3264,height=2448,framerate=15/1 ! jpegparse ! jpegdec ! videoconvert ! pngenc ! multifilesink location="frame%d.png" Decoded images from vaapijpegdec look dark. I also test this cam with Intel Media SDK and gst-mfx plugin, and it look fine. Why? How to solve it? Another problem i have, it is not working contrast/brighness in vaapipostproc. If i set here not default value, i get black screen... Why? Thank you!
I test it in two PC 1. Gigabite H110V-S2V motheboard and Intel core i3-6098P 2. Notebook Lenovo with Intel Core i5-4210H And booth have same effect. It used i965 driver. OS: Ubuntu 17.10. and use 2 version va: $ vainfo libva info: VA-API version 0.40.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_0_40 libva info: va_openDriver() returns 0 vainfo: VA-API version: 0.40 (libva ) vainfo: Driver version: Intel i965 driver for Intel(R) Skylake - 1.8.3 vainfo: Supported profile and entrypoints VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Simple : VAEntrypointEncSlice VAProfileMPEG2Main : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice VAProfileH264Main : VAEntrypointEncSliceLP VAProfileH264High : VAEntrypointVLD VAProfileH264High : VAEntrypointEncSlice VAProfileH264High : VAEntrypointEncSliceLP VAProfileH264MultiviewHigh : VAEntrypointVLD VAProfileH264MultiviewHigh : VAEntrypointEncSlice VAProfileH264StereoHigh : VAEntrypointVLD VAProfileH264StereoHigh : VAEntrypointEncSlice VAProfileVC1Simple : VAEntrypointVLD VAProfileVC1Main : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD VAProfileNone : VAEntrypointVideoProc VAProfileJPEGBaseline : VAEntrypointVLD VAProfileJPEGBaseline : VAEntrypointEncPicture VAProfileVP8Version0_3 : VAEntrypointVLD VAProfileVP8Version0_3 : VAEntrypointEncSlice VAProfileHEVCMain : VAEntrypointVLD VAProfileHEVCMain : VAEntrypointEncSlice and, also using vaapi 1.0: $ gst-launch-1.0 -v filesrc location=./frame5.jpg ! image/jpeg,width=3264,height=2448,framerate=15/1 ! jpegparse ! vaapijpegdec ! vaapipostproc ! pngenc ! filesink location="frameCC1.png" 0:00:00.013741645 21206 0x557174586d50 INFO vaapi gstvaapiutils.c:94:gst_vaapi_log: VA-API version 1.0.0 0:00:00.013821853 21206 0x557174586d50 INFO vaapi gstvaapiutils.c:94:gst_vaapi_log: va_getDriverName() returns 0 0:00:00.013828983 21206 0x557174586d50 INFO vaapi gstvaapiutils.c:94:gst_vaapi_log: Trying to open /usr/local/lib/dri/i965_drv_video.so 0:00:00.014036746 21206 0x557174586d50 INFO vaapi gstvaapiutils.c:94:gst_vaapi_log: Found init function __vaDriverInit_1_0 0:00:00.014652703 21206 0x557174586d50 INFO vaapi gstvaapiutils.c:94:gst_vaapi_log: va_openDriver() returns 0 0:00:00.014675435 21206 0x557174586d50 INFO vaapi gstvaapiutils.c:121:vaapi_initialize: VA-API version 1.0 0:00:00.014719525 21206 0x557174586d50 INFO vaapidisplay gstvaapidisplay.c:899:gst_vaapi_display_create_unlocked:<vaapidisplayx11-0> new display addr=0x55717459b1d0 Установка конвейера в состояние PAUSED… 0:00:00.019280788 21206 0x557174586d50 INFO vaapipostproc gstvaapipluginbase.c:1226:gst_vaapi_plugin_base_create_gl_context:<vaapipostproc0> creating a new GstGL context 0:00:00.026057530 21206 0x557174586d50 INFO vaapipostproc gstvaapipluginbase.c:1194:gst_vaapi_plugin_base_set_gl_context:<vaapipostproc0> GL context: <glcontextglx0> 0:00:00.026164752 21206 0x557174586d50 INFO vaapi gstvaapiutils.c:94:gst_vaapi_log: VA-API version 1.0.0 0:00:00.028013687 21206 0x557174586d50 INFO vaapi gstvaapiutils.c:94:gst_vaapi_log: va_getDriverName() returns 0 0:00:00.028034100 21206 0x557174586d50 INFO vaapi gstvaapiutils.c:94:gst_vaapi_log: Trying to open /usr/local/lib/dri/i965_drv_video.so 0:00:00.028066314 21206 0x557174586d50 INFO vaapi gstvaapiutils.c:94:gst_vaapi_log: Found init function __vaDriverInit_1_0 0:00:00.028414133 21206 0x557174586d50 INFO vaapi gstvaapiutils.c:94:gst_vaapi_log: va_openDriver() returns 0 0:00:00.028433124 21206 0x557174586d50 INFO vaapi gstvaapiutils.c:121:vaapi_initialize: VA-API version 1.0 0:00:00.028444797 21206 0x557174586d50 INFO vaapidisplay gstvaapidisplay.c:899:gst_vaapi_display_create_unlocked:<vaapidisplayglx0> new display addr=0x55717459b530 0:00:00.028464483 21206 0x557174586d50 INFO vaapipostproc gstvaapipluginbase.c:54:plugin_set_display:<vaapipostproc0> set display <vaapidisplayglx0> 0:00:00.028548646 21206 0x557174586d50 INFO vaapidecode gstvaapipluginbase.c:54:plugin_set_display:<vaapidecode_jpeg0> set display <vaapidisplayglx0> Подготовка конвейера (PREROLL)… Получен контекст из элемента «vaapipostproc0»: gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0"; Получен контекст из элемента «vaapipostproc0»: gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayGLX\)\ vaapidisplayglx0"; /GstPipeline:pipeline0/GstJpegParse:jpegparse0.GstPad:src: caps = image/jpeg, parsed=(boolean)true, format=(string)UYVY, width=(int)3264, height=(int)2448, framerate=(fraction)1/1 0:00:00.035850802 21206 0x5571745c89e0 INFO vaapidecode gstvaapidecode.c:208:gst_vaapidecode_update_sink_caps:<vaapidecode_jpeg0> new sink caps = image/jpeg, parsed=(boolean)true, format=(string)UYVY, width=(int)3264, height=(int)2448, framerate=(fraction)1/1 /GstPipeline:pipeline0/GstVaapiDecode_jpeg:vaapidecode_jpeg0.GstPad:sink: caps = image/jpeg, parsed=(boolean)true, format=(string)UYVY, width=(int)3264, height=(int)2448, framerate=(fraction)1/1 0:00:00.042630913 21206 0x5571745c89e0 INFO vaapidecode gstvaapidecode.c:254:gst_vaapidecode_ensure_allowed_srcpad_caps:<vaapidecode_jpeg0> allowed srcpad caps: video/x-raw(memory:VASurface), format=(string){ ENCODED, NV12, I420, YV12, P010_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw(meta:GstVideoGLTextureUploadMeta), format=(string){ RGBA, BGRA }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw(memory:DMABuf), format=(string){ I420, YV12, RGBA }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw, format=(string){ NV12, YV12, I420, YUY2, UYVY, BGRx, RGBx }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ] 0:00:00.042919392 21206 0x5571745c89e0 INFO vaapidecode gstvaapidecode.c:373:gst_vaapidecode_update_src_caps:<vaapidecode_jpeg0> new src caps = video/x-raw(memory:VASurface), format=(string)NV12, width=(int)3264, height=(int)2448, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)1/1 Перераспределение латентности… /GstPipeline:pipeline0/GstVaapiDecode_jpeg:vaapidecode_jpeg0.GstPad:src: caps = video/x-raw(memory:VASurface), format=(string)NV12, width=(int)3264, height=(int)2448, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)1/1 0:00:00.043225003 21206 0x5571745c89e0 INFO vaapipostproc gstvaapipostproc.c:960:gst_vaapipostproc_update_sink_caps:<vaapipostproc0> new sink caps = video/x-raw(memory:VASurface), format=(string)NV12, width=(int)3264, height=(int)2448, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)1/1 0:00:00.043251644 21206 0x5571745c89e0 INFO vaapipostproc gstvaapipostproc.c:981:gst_vaapipostproc_update_src_caps:<vaapipostproc0> new src caps = video/x-raw, width=(int)3264, height=(int)2448, framerate=(fraction)1/1, pixel-aspect-ratio=(fraction)1/1, format=(string)RGBA, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, colorimetry=(string)sRGB 0:00:00.061211872 21206 0x5571745c89e0 INFO vaapipostproc gstvaapipluginbase.c:54:plugin_set_display:<vaapipostproc0> set display <vaapidisplayglx0> /GstPipeline:pipeline0/GstVaapiPostproc:vaapipostproc0.GstPad:src: caps = video/x-raw, width=(int)3264, height=(int)2448, framerate=(fraction)1/1, pixel-aspect-ratio=(fraction)1/1, format=(string)RGBA, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, colorimetry=(string)sRGB /GstPipeline:pipeline0/GstPngEnc:pngenc0.GstPad:sink: caps = video/x-raw, width=(int)3264, height=(int)2448, framerate=(fraction)1/1, pixel-aspect-ratio=(fraction)1/1, format=(string)RGBA, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, colorimetry=(string)sRGB 0:00:00.063217151 21206 0x5571745c89e0 INFO vaapipostproc gstvaapipluginbase.c:970:gst_vaapi_plugin_base_decide_allocation:<vaapipostproc0> ignoring non-VAAPI pool: <videobufferpool0> /GstPipeline:pipeline0/GstVaapiPostproc:vaapipostproc0.GstPad:src: caps = video/x-raw, width=(int)3264, height=(int)2448, framerate=(fraction)1/1, pixel-aspect-ratio=(fraction)1/1, format=(string)RGBA, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, colorimetry=(string)sRGB /GstPipeline:pipeline0/GstPngEnc:pngenc0.GstPad:sink: caps = video/x-raw, width=(int)3264, height=(int)2448, framerate=(fraction)1/1, pixel-aspect-ratio=(fraction)1/1, format=(string)RGBA, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, colorimetry=(string)sRGB 0:00:00.068045817 21206 0x5571745c89e0 INFO vaapipostproc gstvaapipluginbase.c:590:create_dmabuf_srcpad_allocator:<vaapipostproc0> dmabuf allocator generates unmappable buffers /GstPipeline:pipeline0/GstVaapiPostproc:vaapipostproc0.GstPad:sink: caps = video/x-raw(memory:VASurface), format=(string)NV12, width=(int)3264, height=(int)2448, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)1/1 0:00:00.073100819 21206 0x5571745c89e0 INFO vaapipostproc gstvaapipluginbase.c:970:gst_vaapi_plugin_base_decide_allocation:<vaapipostproc0> ignoring non-VAAPI pool: <videobufferpool1> /GstPipeline:pipeline0/GstPngEnc:pngenc0.GstPad:src: caps = image/png, width=(int)3264, height=(int)2448, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)sRGB, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = image/png, width=(int)3264, height=(int)2448, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)sRGB, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono Конвейер подготовлен (PREROLLED)… Установка конвейера в состояние PLAYING… New clock: GstSystemClock Получен маркер EOS («конец потока») от элемента «pipeline0». Execution ended after 0:00:00.007418042
I am solve this bug self, but i think it is not good solution, in any case. As it turned out, the quantization matrix is filled, even before the parsing of the JPEG format, and reading the quantification table. In this case, the default quantization matrix is used. But it is for this camera that it does not work properly. I really want to see good solution, in fact a bug very serious and unpleasant. diff --git a/gst-libs/gst/vaapi/gstvaapidecoder_jpeg.c b/gst-libs/gst/vaapi/gstvaapidecoder_jpeg.c index bef1eb71..ccba6a45 100644 --- a/gst-libs/gst/vaapi/gstvaapidecoder_jpeg.c +++ b/gst-libs/gst/vaapi/gstvaapidecoder_jpeg.c @@ -830,9 +830,9 @@ gst_vaapi_decoder_jpeg_start_frame (GstVaapiDecoder * base_decoder, if (!fill_picture (decoder, picture, &priv->frame_hdr)) return GST_VAAPI_DECODER_STATUS_ERROR_UNKNOWN; - status = fill_quantization_table (decoder, picture); - if (status != GST_VAAPI_DECODER_STATUS_SUCCESS) - return status; +// status = fill_quantization_table (decoder, picture); +// if (status != GST_VAAPI_DECODER_STATUS_SUCCESS) +// return status; /* Update presentation time */ picture->pts = GST_VAAPI_DECODER_CODEC_FRAME (decoder)->pts; @@ -845,6 +845,11 @@ gst_vaapi_decoder_jpeg_end_frame (GstVaapiDecoder * base_decoder) GstVaapiDecoderJpeg *const decoder = GST_VAAPI_DECODER_JPEG_CAST (base_decoder); + GstVaapiDecoderJpegPrivate *const priv = &decoder->priv; + VAStatus status = fill_quantization_table (decoder, priv->current_picture); + if (status != GST_VAAPI_DECODER_STATUS_SUCCESS) + return status; + return decode_current_picture (decoder); }
Hi Nikolai, Thanks for reporting this issue and proposing a possible solution. It seems like the ELP camera is sending the DQT marker in a non expected segment. Still I don't see why it is a major issue, since it is, as far as we know, this specific camera that is failing.
In the tarball, which is the original image generated by the ELP camera?
Yes, frame 5 jpeg it is original image. I just start compare it with /decode/loadjpeg samples from va_urils package, and found this difference. Did not check, exactly, but the image from the HP camera also looks a bit different. Although it is not dark.
This effect I see only on this decoder. libjpeg-turbo, GST-MFX pubin, and this example from va, fine decode and it look fine.
https://drive.google.com/open?id=1wE6yoRIaiC93LGXzwReUuovchHGNzqZm Also, 10 images from camera as is.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/72.