GNOME Bugzilla – Bug 748676
vaapidecode: MPEG2 playback has macro blocking with some streams
Last modified: 2015-05-27 14:58:23 UTC
Created attachment 302620 [details] Picture of the resulting decode There is an issue with the MPEG2 decoder which results in residual images and macroblocking around scene changes. It appears that there is an issue with the way the decoder is referencing the pictures that are sitting inside the decoded picture buffer. A patch that I made to flush the buffer out on I-Frames prevents the blocking, but seems to delete the frames inside the buffer instead of outputting them. Example Pipeline: filesrc location=/home/owner/test_streams/Meredith_Macroblocks.ts ! tsdemux name=dmx_1 program-number=1 dmx_1. ! mpegvideoparse ! video/mpeg,stream-format=avc,alignment=au ! vaapidecode name=vaapidecode_1 ! vaapisink display=x11 max-lateness=-1 owner@pc2182:~/gst/master/gstreamer-vaapi$ libva info: VA-API version 0.37.1 libva info: va_getDriverName() returns 0 libva info: Trying to open /home/owner/gst/master/prefix/lib/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_0_37 libva info: va_openDriver() returns 0 Setting pipeline to PAUSED ... Pipeline is PREROLLING ... Got context from element 'vaapidecode_1': gst.vaapi.Display=context, display=(GstVaapiDisplay)NULL; Pipeline is PREROLLED ... Setting pipeline to PLAYING ... New clock: GstSystemClock XIO: fatal IO error 11 (Resource temporarily unavailable) on X server ":0" after 1021 requests (1021 known processed) with 2 events remaining.
Created attachment 302621 [details] [review] Flush the decoded picture buffer after decoding an I-Frame This change flushes the decoded picture buffer after an I-Frame is decoded. It does prevent the blocking from happening, but it seems like some of the frames are being deleted instead of output. The resulting video can be jerky.
Created attachment 302622 [details] Another picture of the broken decode showing the residual image
I can provide the source stream that reproduces this issue upon request. It is very short, ~10 seconds and has the problem occur three times pretty close together.
Here is a short 10 second clip which has the issue. https://drive.google.com/file/d/0B1YF842NJ8L8cXNkZllYN0QwSmc/view?usp=sharing Here is another clip which also has the problem. https://drive.google.com/file/d/0B1YF842NJ8L8VFdXZjZ6OV9VN0E/view?usp=sharing
Yup, we need a proper fix other than skipping frames.. What I can see is that the software decoder is able to decode all the frames correctly.
It appears like the poc value is occasionally computed incorrectly when a new gop sequence starts and the previous sequence ends with a P frame. The poc value for the first I frame is computed to be the same as the poc value for the last frame in the previous gop. This appears to cause the frame from the previous gop to remain in the dpb and be used as reference for the new gop, which results in macroblocking.
Modifying this patch to only flush the DPB when the I-frame POC matches an existing frame in the buffer would rectify the stuttery video, but you'd still be losing 1-2 frames every time this occurred. It seems that the "correct" fix for this would be one of two options: - correct the POC calculation such that the I-frame always has a unique one - modify the POC values of the content in the DPB such that they don't overlap with the incoming I-frame. Both of the above (in theory) would prevent dropped frames which I think would be preferable.
Created attachment 304001 [details] [review] decoder: mpeg2: fix PTS cache for GOP start Please try this patch and tell me if you see any missing frame. Another patch is possible, assuming PTS from parser are correct, but this one needs to go in first.
Update: - For the "Meredith" clip, 402 frames are decoded and displayed correctly with the attached patch, whereas the SW decoder would only produce 399 frames. - For the "Disney" clip, 2086 frames are decoded and displayed correctly with the attached patch, whereas the SW decoder would only produce 2083 frames. That off-by-three frames is an unrelated error in the SW decoder maybe. By "SW decoder", I mean the GStreazmer mpeg2dec decoder, based off libmpeg2.
commit efc07f6bb1873704458d3a5f6432dfc1871ec661 Author: Gwenole Beauchesne <gwenole.beauchesne@intel.com> Date: Tue May 26 13:28:32 2015 +0200 mpeg2: fix PTS cache for GOP start. If the GOP temporal sequence number (TSN) is interpolated from a valid PTS, then we need to compensate that PTS corresponding to the start of GOP with the next picture to be decoded, which shall be an I-frame, based on its sequence number. https://bugzilla.gnome.org/show_bug.cgi?id=748676
(In reply to Gwenole Beauchesne from comment #9) > Update: > - For the "Meredith" clip, 402 frames are decoded and displayed correctly > with the attached patch, whereas the SW decoder would only produce 399 > frames. > - For the "Disney" clip, 2086 frames are decoded and displayed correctly > with the attached patch, whereas the SW decoder would only produce 2083 > frames. > > That off-by-three frames is an unrelated error in the SW decoder maybe. By > "SW decoder", I mean the GStreazmer mpeg2dec decoder, based off libmpeg2. I usually use libav decoder as reference... :)