GNOME Bugzilla – Bug 796786
v4l2videodec: trickmode-key-frame hangs on second frame with imx6
Last modified: 2018-11-03 15:31:51 UTC
Video decoding in trickmode-key-frame fails with GStreamer 1.14.1 on imx6, because the v4l2videodec sends a STREAMOFF-STREAMON sequence to the V4L2 driver when draining the decoder. I can reproduce the issue using the following pipeline and scenario: Pipeline: gst-validate-1.0 --set-scenario trickmode-seek-fast-forward \ filesrc location=${FILENAME} ! \ h264parse ! v4l2h264dec ! \ waylandsink Scenario: description, duration=25.0, seek=true, need-clock-sync=true seek, playback-time=0.0, rate=8.0, start=0.0, flags=trickmode-key-units+key-unit+flush wait, duration=10.0 stop The first frame after the seek event is decoded fine, but CODA decoder hangs on the second frame and the CODA driver prints a message that SEQ_INIT failed. The decoder expects an SPS/PPS NAL unit for the initialization, but the decoder starts with the IDR without sending the SPS/PPS first. If I set config-interval=-1 on the h264parse to send the SPS/PPS for every keyframe the decoding works. This only works for lower decoding rates (e.g. 2.0). For higher rates the sink starts to drop frames because the buffers are too late. In trickmode-key-frame, the videodecoder drains the decoder after every frame. The problem is that the v4l2videodec flushes while draining the decoder in addition to run CMD_STOP, which runs a STREAMOFF-STREAMON sequence. The STREAMOFF-STREAMON causes the driver to reset/reinitialize the CODA and thus requiring the SPS/PPS NAL units for a proper initialization. My understanding is that the STREAMOFF-STREAMON during the drain implies a seek which requires a resume point for continuing and violates the assumption that sub-classes should be prepared to handle new data after the drain. Instead the v4l2videodec should not flush the decoder while draining but use the CMD_START to restart the decoder once all buffers are drained. I have not found many users or implementers of CMD_START and I am not sure if this is the right solution.
This is related to: https://bugzilla.gnome.org/show_bug.cgi?id=796771 https://bugzilla.gnome.org/show_bug.cgi?id=791091 Your proposition requires kernel spec and code work. Also, requires to prove that firmware can support this.
One of Olivier's argument was that the following I frame should force the previous one out, so with 1 frame latency, this mode would kind of work. But according to Edward, this long delay was not acceptable: https://bugzilla.gnome.org/show_bug.cgi?id=767232#c4
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/491.