GNOME Bugzilla – Bug 763321
Failed to play VAAPI HW JPEG decode for video clip with codec ID: MJPG
Last modified: 2018-05-04 13:42:23 UTC
Intel VA driver: upstream master Libva: upstream master gstreamer framework: 1.6.3 gstreamer-vaapi: 1.6.0 command: gst-launch-1.0 -v filesrc location=/home/root/MJPEG.avi ! avidemux ! vaapidecode ! vaapisink hit the assert checking in Intel VA driver side. Check the HW spec, HW side does not support horizontal & vertical sampling factors in H1=H2=H3=1 & V1=V2=V3=2. Trace it until the decode_picture function in gstvaapidecoder_jpeg.c in gstreamer-vaapi plugin to gst_jpeg_segment_parse_frame_header function and print out get the value as this frame_hdr->components[0].horizontal_factor = 1 frame_hdr->components[1].horizontal_factor = 1 frame_hdr->components[2].horizontal_factor = 1 frame_hdr->components[0].vertical_factor = 2 frame_hdr->components[1].vertical_factor = 2 frame_hdr->components[2].vertical_factor = 2 Not sure it this codecparse side the issue due new codec ID=MJPG? or gstreamer-vaapi should be add this new checking on decode_picture function in gstvaapidecoder_jpeg.c. Because this vertical and horizontal sampling factors is not supported in HW. I'm able to play this video clip using the AVICodec from YUV FOURCC web site. Play this video clip from codec ID=MJPG not that smooth using jpegdec plugins but able to play the video. command: gst-launch-1.0 -v filesrc location=/home/root/MJPEG.avi ! avidemux ! jpegdec ! vaapisink
Created attachment 323394 [details] gst_debug.log
dump video clip info from mediainfo: General Complete name : MPJEG.avi Format : AVI Format/Info : Audio Video Interleave File size : 40.6 MiB Duration : 13s 433ms Overall bit rate : 25.4 Mbps Writing application : Lavf56.15.102 Video ID : 0 Format : JPEG Codec ID : MJPG Duration : 13s 433ms Bit rate : 25.4 Mbps Width : 1 280 pixels Height : 720 pixels Display aspect ratio : 16:9 Frame rate : 60.000 fps Color space : YUV Bit depth : 8 bits Compression mode : Lossy Bits/(Pixel*Frame) : 0.459 Stream size : 40.6 MiB (100%)
Created attachment 323395 [details] [review] JPEG Decode: check values of horizontal and vertical sampling factors
using ffmpeg to generate the mjpeg video clip. command: ffmpeg.exe -i h264_800x600_Main_at_L3.mp4 -c:v mjpeg -q:v 3 -an -pix_fmt yuvj444p output_444.avi Trace until in ffmpeg: libavcodec/mjpegenc_common.c (gst-libav: gst-libs/ext/libav/libavcodec/mjpegenc_common.c) , the ff_mjpeg_init_hvsample function is the one set the vertical sampling factor to 2 if it is pix_fmt yuvj444p. Or can suggest any tools will be help to create jpeg video clip for YUV444 for vertical sampling factor=1? void ff_mjpeg_init_hvsample(AVCodecContext *avctx, int hsample[4], int vsample[4]) { int chroma_h_shift, chroma_v_shift; av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &chroma_h_shift, &chroma_v_shift); if (avctx->codec->id == AV_CODEC_ID_LJPEG && ( avctx->pix_fmt == AV_PIX_FMT_BGR0 || avctx->pix_fmt == AV_PIX_FMT_BGRA || avctx->pix_fmt == AV_PIX_FMT_BGR24)) { vsample[0] = hsample[0] = vsample[1] = hsample[1] = vsample[2] = hsample[2] = vsample[3] = hsample[3] = 1; } else if (avctx->pix_fmt == AV_PIX_FMT_YUV444P || avctx->pix_fmt == AV_PIX_FMT_YUVJ444P) { vsample[0] = vsample[1] = vsample[2] = 2; hsample[0] = hsample[1] = hsample[2] = 1; hsample=1\n"); } else { vsample[0] = 2; vsample[1] = 2 >> chroma_v_shift; vsample[2] = 2 >> chroma_v_shift; hsample[0] = 2; hsample[1] = 2 >> chroma_h_shift; hsample[2] = 2 >> chroma_h_shift; } }
Tested this issue also happen in git master branch.
(In reply to Lim Siew Hoon from comment #5) > Tested this issue also happen in git master branch. For gstreamer-vaapi.
Similar one: https://bugzilla.gnome.org/show_bug.cgi?id=762500
In theory, Hi and Vi can have values from 1 to 4 as per Jpeg Spec. And there could be other samples like this (with different h_factor/v_facotr) which are not decodable with the driver. How about a more generalized approach: -- write utility function to get the chroma_format (SUBSAMPLE_YUV420,SUBSAMPLE_YUV422H etc) from h_factor/v_factor -- if chromat_format != any of the GST_VAAPI_CHROMA_TYPE_* then return GST_VAAPI_DECODER_STATUS_ERROR_UNSUPPORTED_CHROMA_FORMAT
Created attachment 324589 [details] [review] JPEG Decode: check values of horizontal and vertical sampling factors (v2) Sreerenj, I'm not sure how to make it become generic. Is it really all the video format (h264, h265, MPEG2, VC1...JPEG) decode is using generic horizontal sampling factor and vertical sampling factor? Maybe can you explain more a bit? Or send me the link I can go read it. And come out properly fix. Intel VA driver: VA_FOURCC_NV12 VA_FOURCC_P010 VA_FOURCC_YV12 VA_FOURCC_IMC1 VA_FOURCC_I420 VA_FOURCC_IYUV VA_FOURCC_IMC3 SUBSAMPLE_YUV420 VA_FOURCC_YUY2 VA_FOURCC_UYVY VA_FOURCC_422H VA_FOURCC_YV16 SUBSAMPLE_YUV422H VA_FOURCC_422V SUBSAMPLE_YUV422V VA_FOURCC_RGBA VA_FOURCC_RGBX VA_FOURCC_BGRA VA_FOURCC_BGRX SUBSAMPLE_RGBX VA_FOURCC_Y800 SUBSAMPLE_YUV400 VA_FOURCC_411P SUBSAMPLE_YUV411 VA_FOURCC_444P SUBSAMPLE_YUV444 #define SUBSAMPLE_YUV400 0 -> VA_RT_FORMAT_YUV400 (libva) #define SUBSAMPLE_YUV420 1 -> VA_RT_FORMAT_YUV420 #define SUBSAMPLE_YUV422H 2 -> VA_RT_FORMAT_YUV422 #define SUBSAMPLE_YUV422V 3 -> VA_RT_FORMAT_YUV422 #define SUBSAMPLE_YUV444 4 -> VA_RT_FORMAT_YUV444 #define SUBSAMPLE_YUV411 5 -> VA_RT_FORMAT_YUV411 #define SUBSAMPLE_RGBX 6
Marking this as duplicate. IMO gstreamer-vaapi should not do the work of the driver. A different provider could process this type of jpegs. *** This bug has been marked as a duplicate of bug 762500 ***