GNOME Bugzilla – Bug 764607
Motion JPEG (MJPEG) Fails to Play
Last modified: 2016-10-31 14:28:28 UTC
Created attachment 325368 [details] gst-launch output Errors generated (see attached output) when trying to play MJPEG video file (attached) using the following pipeline: gst-launch-1.0 -vm filesrc location=~/Videos/sintel.mjpg ! jpegparse ! vaapijpegdec ! vaapisink I tried this on both HSW and SKL. Both systems exhibit the same behavior. ::System:: Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz Fedora 22 x86_64 ::SW Stack:: drm (master) heads/master-0-gea78c17854ad mesa (master) heads/master-0-g1eeec7ec41dc libva (master) heads/master-0-g2339d10b9a39 cmrt (master) heads/master-0-gf60b0a6996e5 intel-driver (master) heads/master-0-g759e44d4cbe7 intel-hybrid-driver (master) heads/master-0-g0375a7287e55 gstreamer (master) heads/master-0-g4503232f838c gst-plugins-base (master) heads/master-0-g052fe119492c gst-plugins-good (master) heads/master-0-gdf247f091cba gst-plugins-bad (master) heads/master-0-g0fbbbd042a11 gst-ffmpeg (master) heads/master-0-g1af733a6366d gstreamer-vaapi (master) heads/master-0-g506c9e2b5bbc
Created attachment 325373 [details] Motion JPEG clip - truncated to 150 frames due to upload size constraints
This pipeline works, though error messages are shown gst-launch-1.0 -v filesrc location= ~/patterns/sintel.mjpg ! \ image/jpeg, width=1280,height=720,framerate=0/1 ! vaapijpegdec ! vaapisink On one side, it seems that jpegparse doesn't work correctly for mjpegs, and, on the other, looks to me that vaapijpegdec doesn't handle correctly streams of images.
The log 0:00:00.332357890 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder_objects.c:262:gst_vaapi_picture_decode: decode picture 0x04000001 0:00:00.333226046 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder.c:376:push_frame: push frame 1 (surface 0x04000001) 0:00:00.333263175 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder.c:397:pop_frame: pop frame 1 (surface 0x04000001) 0:00:00.338373722 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder_objects.c:262:gst_vaapi_picture_decode: decode picture 0x04000002 0:00:00.339032661 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder.c:376:push_frame: push frame 2 (surface 0x04000002) 0:00:00.339090740 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder.c:397:pop_frame: pop frame 2 (surface 0x04000002) 0:00:00.355280718 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder_objects.c:262:gst_vaapi_picture_decode: decode picture 0x04000003 0:00:00.355989164 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder.c:376:push_frame: push frame 3 (surface 0x04000003) 0:00:00.356017813 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder.c:397:pop_frame: pop frame 3 (surface 0x04000003) 0:00:00.371832375 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder_objects.c:262:gst_vaapi_picture_decode: decode picture 0x04000004 0:00:00.372428818 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder.c:376:push_frame: push frame 4 (surface 0x04000004) 0:00:00.372450280 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder.c:397:pop_frame: pop frame 4 (surface 0x04000004) 0:00:00.390403754 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder_objects.c:262:gst_vaapi_picture_decode: decode picture 0x04000005 0:00:00.390989591 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder.c:376:push_frame: push frame 5 (surface 0x04000005) 0:00:00.391016480 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder.c:397:pop_frame: pop frame 5 (surface 0x04000005) 0:00:00.405135523 21753 0x12de8a0 DEBUG vaapi gstvaapidecoder_objects.c:262:gst_vaapi_picture_decode: decode picture 0x04000000 0:00:00.405164086 21753 0x12de8a0 DEBUG vaapi gstvaapiutils.c:53:vaapi_check_status: vaBeginPicture(): surface is in use 0:00:00.405178651 21753 0x12de8a0 ERROR vaapidecode gstvaapidecode.c:612:gst_vaapidecode_handle_frame: decode error -1
I found that in release 1.8 it works using playbin (deleting libgstjpeg.so to force the use of vaapijpegdec) Then I bisected and the result blames commit 6b17ed90 vaapidecode: Derive and save the decoded surface format.
Created attachment 325550 [details] [review] surface: destroy derived image If gst_vaapi_image_new_with_image() fails, the created derived image should be destroyed, otherwise the surface cannot be processed because is being used.
Created attachment 325551 [details] [review] image: hack: handle IMC3 format as YV12 Intel driver internally uses IMC3 format for JPEG decoding. This patch add a hackish support to IMC3 as a YV12 to avoid color conversions if it can be negotiated. http://www.fourcc.org/yuv.php#IMC3wq
(In reply to Víctor Manuel Jáquez Leal from comment #6) > Created attachment 325551 [details] [review] [review] > image: hack: handle IMC3 format as YV12 > > Intel driver internally uses IMC3 format for JPEG decoding. This patch add a > hackish support to IMC3 as a YV12 to avoid color conversions if it can be > negotiated. > > http://www.fourcc.org/yuv.php#IMC3wq I'm not sure of this patch. As I said, it is a hack. But I posted for discussion.
I'm going to push attachment 325550 [details] [review] since it is a required fix. @Artie let me know if that fixes your use-case and then close this bug.
(In reply to Víctor Manuel Jáquez Leal from comment #7) > (In reply to Víctor Manuel Jáquez Leal from comment #6) > > Created attachment 325551 [details] [review] [review] [review] > > image: hack: handle IMC3 format as YV12 > > > > Intel driver internally uses IMC3 format for JPEG decoding. This patch add a > > hackish support to IMC3 as a YV12 to avoid color conversions if it can be > > negotiated. > > > > http://www.fourcc.org/yuv.php#IMC3wq > > I'm not sure of this patch. As I said, it is a hack. But I posted for > discussion. I didn't know that driver is selecting any format other than nv12/yv12/p010 for decoding! Yup, we should think about this. Driver set the flags in such a way that, IMC3 can be used in vaCreateSurface() but not for vaCreateImage, interesting :) May be we can stick on with "hidden" color-space conversion for all formats other than NV12/YV12/I420/P010 from vaapidecode for better backward compatibility .
Comment on attachment 325550 [details] [review] surface: destroy derived image Attachment 325550 [details] pushed as bb0b8ce - surface: destroy derived image
(In reply to sreerenj from comment #9) > (In reply to Víctor Manuel Jáquez Leal from comment #7) > > (In reply to Víctor Manuel Jáquez Leal from comment #6) > > > Created attachment 325551 [details] [review] [review] [review] [review] > > > image: hack: handle IMC3 format as YV12 > > > > > > Intel driver internally uses IMC3 format for JPEG decoding. This patch add a > > > hackish support to IMC3 as a YV12 to avoid color conversions if it can be > > > negotiated. > > > > > > http://www.fourcc.org/yuv.php#IMC3wq > > > > I'm not sure of this patch. As I said, it is a hack. But I posted for > > discussion. > > I didn't know that driver is selecting any format other than nv12/yv12/p010 > for decoding! > > Yup, we should think about this. Driver set the flags in such a way that, > IMC3 can be used in vaCreateSurface() but not for vaCreateImage, interesting > :) > > May be we can stick on with "hidden" color-space conversion for all formats > other than NV12/YV12/I420/P010 from vaapidecode for better backward > compatibility . Here is another related one. https://bugzilla.gnome.org/show_bug.cgi?id=765039
I have retested the pipeline with current master, and it works: gst-launch-1.0 filesrc location=~/patterns/bug764607.mjpg ! jpegparse ! vaapijpegdec ! vaapisink sync=false sync=false is because jpegparse set a framerate of 1/1 and the rendering is sloooow... with sync=false the sink ignores the framerate As it is working as expected, I'm closing the bug.