After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 734386 - mpegvideoparse: generate proper PTS for outgoing frames
mpegvideoparse: generate proper PTS for outgoing frames
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
git master
Other Linux
: Normal enhancement
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks: 734135 734547
 
 
Reported: 2014-08-06 21:59 UTC by sreerenj
Modified: 2018-11-03 13:25 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description sreerenj 2014-08-06 21:59:08 UTC
The parser elements are not generating right PTS for the outgoing frames in current implementation. This is a feature request to add PTS generation in mpegvideoparse element.The same has been implemented in gstreamer-vaapi https://gitorious.org/vaapi/gstreamer-vaapi/source/7ac501d026036887c114b28bc9fb4aabc557e91a:gst-libs/gst/vaapi/gstvaapidecoder_mpeg2.c
Comment 1 Gwenole Beauchesne 2014-08-07 04:10:31 UTC
IIRC, Jan had mentioned that this is taken care of already, that's why I applied his patch to properly propagate PTS generated from the parser elements. Assumed it was right from there. :)

Do you have sample clips showing off this incorrect behaviour? The original reason why I always recomputed PTS within libgstvaapi was because the information from the demuxer was not always correct a few years ago.
Comment 2 sreerenj 2014-08-07 08:26:02 UTC
Does any of the videoparser elements generating PTS if there are B frames? I am getting PTS=none for all the clips i have tested.!
Comment 3 sreerenj 2014-08-07 08:44:23 UTC
IIUC, the PTS calculation should be based on the POC, right? None of the parsers have implementation to find poc/pts based on spec. Take any elementary bitstream (tested h264 & mpeg2) , the pts seems to be none. It may be more complicated to add the support in h264parser :)
Comment 4 Gwenole Beauchesne 2014-08-12 08:22:49 UTC
Hi, here are some clips that generate bad PTS through vaapidecode, unless I recalculate them right from vaapidecode, as before:

https://dl.dropboxusercontent.com/u/55728161/1032_20130801154100.ts
https://dl.dropboxusercontent.com/u/55728161/tennis.ts

Causes are multiple:
- tsdemux | mpegvideoparse does generate invalid timestamps?
- I incorrectly propagate the PTS? e.g. not from the right GstAdapter point?

WDYT?
Comment 5 Gwenole Beauchesne 2014-08-12 08:24:57 UTC
(In reply to comment #4)
> Hi, here are some clips that generate bad PTS through vaapidecode, unless I
> recalculate them right from vaapidecode, as before:

In particular, revert git commit 22dc8c4 from libgstvaapi.
Comment 6 Gwenole Beauchesne 2014-08-12 08:51:19 UTC
Update: the decoded timestamps for tennis.ts for mpeg2dec and vaapidecode are rigorously identical (as videodecode exposes them in logs).
Comment 7 Gwenole Beauchesne 2014-08-12 08:54:18 UTC
(In reply to comment #3)
> IIUC, the PTS calculation should be based on the POC, right? None of the
> parsers have implementation to find poc/pts based on spec. Take any elementary
> bitstream (tested h264 & mpeg2) , the pts seems to be none. It may be more
> complicated to add the support in h264parser :)

I believe the videoparser element should be able to generate PTS if the upper element (demuxer) did not do this. Though, for h264 that's indeed more complicated as additional parsing might be needed. :)
Comment 8 Jan Schmidt 2014-08-12 10:35:25 UTC
Not that specifically, sreerenj seems to be saying that raw elementary streams don't get PTS calculated by the parser - which has always been true. In that situation, the existing vaapi code will generate PTS, and still propagate upstream PTS when they do exist.
Comment 9 sreerenj 2014-08-12 11:14:46 UTC
(In reply to comment #8)
> Not that specifically, sreerenj seems to be saying that raw elementary streams
> don't get PTS calculated by the parser - which has always been true. In that
> situation, the existing vaapi code will generate PTS, and still propagate
> upstream PTS when they do exist.

Exactly :)

Parsers should generate PTS if and only if upstream didn't provide those. 
And that should be based on display-order, basically poc .

Copying the code from vaapideocde_mpeg2 to mpegvideoparse seems to be straight forward. If I understood correctly the poc calculation based on h264 spec is complicated but that is the only way to find out the display order with in parser, not like decoder where you can utilize the GstAdapter and other heuristics to find out PTS with out following the specification exactly.
Comment 10 Jan Schmidt 2014-08-12 11:31:41 UTC
tennis.ts seems to generate perfect output timestamps with both vaapidecode and mpeg2dec - except that mpeg2dec fails to output the last 2 frames in the sequence. For tennis.ts all the buffers out of mpegvideoparse have useful and correct PTS and get passed through for me.

1032_20130801154100.ts is more interesting - only the keyframes have PTS and the rest of them have to be interpolated. None of mpeg2dec, gstreamer-vaapi or gstreamer-vaapi-without-22dc8c4 generate a complete valid set of output timestamps for that file. mpeg2dec output looks closest to correct, except for a weird 40ms offset for the first group of frames.
Comment 11 Jan Schmidt 2014-08-12 11:40:30 UTC
It would be nice if parsers interpolated output PTS, but for MPEG-2 specifically, it's not clear a direct copy of the vaapi parsing code unless it's fixed to generate correct output timestamps first.
Comment 12 Jan Schmidt 2014-08-12 11:43:36 UTC
For comparison, here's some output from each decode variation for 1032_20130801154100.ts 

Correct		VAAPI		VAAPI-old	mpeg2dec
0.434326593	0.434326593	0.514326593	0.474326593
0.474326593	0.554326593	0.554326593	0.514326593
0.514326593	0.594326593	0.594326593	0.554326593
0.554326593	0.634326593	0.634326593	0.594326593
0.594326593	0.674326593	0.674326593	0.634326593
0.634326593	0.714326593	0.714326593	0.674326593
0.674326593	0.754326593	0.754326593	0.714326593
0.714326593	0.794326593	0.794326593	0.754326593
0.754326593	0.834326593	0.834326593	0.794326593
0.794326593	0.874326593	0.874326593	0.834326593
0.834326593	0.914326593	0.914326593	0.874326593
0.874326593	0.954326593	0.954326593	0.914326593
0.914326593	0.954326593	0.994326593	0.914326593
0.954326593	1.034326593	1.034326593	0.954326593
0.994326593	1.074326593	1.074326593	0.994326593
1.034326593	1.114326593	1.114326593	1.034326593
1.074326593	1.154326593	1.154326593	1.074326593
1.114326593	1.194326593	1.194326593	1.114326593
1.154326593	1.234326593	1.234326593	1.154326593
1.194326593	1.274326593	1.274326593	1.194326593
1.234326593	1.314326593	1.314326593	1.234326593
Comment 13 sreerenj 2014-08-12 13:08:01 UTC
(In reply to comment #10)
> tennis.ts seems to generate perfect output timestamps with both vaapidecode and
> mpeg2dec - except that mpeg2dec fails to output the last 2 frames in the
> sequence. 

This reminds me another bug somewhere is gstreamer. The avdec_h264 decoder was also failing to decode the last two frames while I was testing something a couple of weeks ago.
Comment 14 Sebastian Dröge (slomo) 2014-08-12 13:20:10 UTC
(In reply to comment #13)
> (In reply to comment #10)
> > tennis.ts seems to generate perfect output timestamps with both vaapidecode and
> > mpeg2dec - except that mpeg2dec fails to output the last 2 frames in the
> > sequence. 
> 
> This reminds me another bug somewhere is gstreamer. The avdec_h264 decoder was
> also failing to decode the last two frames while I was testing something a
> couple of weeks ago.

This bug #734608 ?
Comment 15 sreerenj 2014-08-12 13:39:15 UTC
(In reply to comment #14)
> (In reply to comment #13)
> > (In reply to comment #10)
> > > tennis.ts seems to generate perfect output timestamps with both vaapidecode and
> > > mpeg2dec - except that mpeg2dec fails to output the last 2 frames in the
> > > sequence. 
> > 
> > This reminds me another bug somewhere is gstreamer. The avdec_h264 decoder was
> > also failing to decode the last two frames while I was testing something a
> > couple of weeks ago.
> 
> This bug #734608 ?

yes, most probably...
Comment 16 GStreamer system administrator 2018-11-03 13:25:47 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/167.