After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 796786 - v4l2videodec: trickmode-key-frame hangs on second frame with imx6
v4l2videodec: trickmode-key-frame hangs on second frame with imx6
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
1.14.1
Other Linux
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2018-07-11 09:37 UTC by Michael Tretter
Modified: 2018-11-03 15:31 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Michael Tretter 2018-07-11 09:37:13 UTC
Video decoding in trickmode-key-frame fails with GStreamer 1.14.1 on imx6,
because the v4l2videodec sends a STREAMOFF-STREAMON sequence to the V4L2 
driver when draining the decoder.

I can reproduce the issue using the following pipeline and scenario:

Pipeline:

gst-validate-1.0 --set-scenario trickmode-seek-fast-forward \
        filesrc location=${FILENAME} ! \
        h264parse ! v4l2h264dec ! \
        waylandsink

Scenario:

description, duration=25.0, seek=true, need-clock-sync=true
seek, playback-time=0.0, rate=8.0, start=0.0, flags=trickmode-key-units+key-unit+flush
wait, duration=10.0
stop

The first frame after the seek event is decoded fine, but CODA decoder hangs
on the second frame and the CODA driver prints a message that SEQ_INIT failed.
The decoder expects an SPS/PPS NAL unit for the initialization, but the
decoder starts with the IDR without sending the SPS/PPS first.

If I set config-interval=-1 on the h264parse to send the SPS/PPS for every
keyframe the decoding works. This only works for lower decoding rates (e.g.
2.0). For higher rates the sink starts to drop frames because the buffers are
too late.

In trickmode-key-frame, the videodecoder drains the decoder after every frame.
The problem is that the v4l2videodec flushes while draining the decoder in
addition to run CMD_STOP, which runs a STREAMOFF-STREAMON sequence. The
STREAMOFF-STREAMON causes the driver to reset/reinitialize the CODA and thus
requiring the SPS/PPS NAL units for a proper initialization.

My understanding is that the STREAMOFF-STREAMON during the drain implies a 
seek which requires a resume point for continuing and violates the assumption 
that sub-classes should be prepared to handle new data after the drain.
Instead the v4l2videodec should not flush the decoder while draining but use
the CMD_START to restart the decoder once all buffers are drained. I have not
found many users or implementers of CMD_START and I am not sure if this is the
right solution.
Comment 1 Nicolas Dufresne (ndufresne) 2018-07-11 11:56:23 UTC
This is related to:

https://bugzilla.gnome.org/show_bug.cgi?id=796771
https://bugzilla.gnome.org/show_bug.cgi?id=791091

Your proposition requires kernel spec and code work. Also, requires to prove that firmware can support this.
Comment 2 Nicolas Dufresne (ndufresne) 2018-07-11 11:59:54 UTC
One of Olivier's argument was that the following I frame should force the previous one out, so with 1 frame latency, this mode would kind of work. But according to Edward, this long delay was not acceptable:

https://bugzilla.gnome.org/show_bug.cgi?id=767232#c4
Comment 3 GStreamer system administrator 2018-11-03 15:31:51 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/491.