After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 668995 - playbin: about-to-finish track changes don't work properly yet
playbin: about-to-finish track changes don't work properly yet
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-base
git master
Other Linux
: Normal normal
: 1.13.90
Assigned To: GStreamer Maintainers
GStreamer Maintainers
playback
: 792913 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2012-01-30 07:45 UTC by Jonathan Matthew
Modified: 2018-03-01 18:46 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
use playbin rather than playbin2 for tests (3.79 KB, patch)
2012-01-30 07:45 UTC, Jonathan Matthew
none Details | Review
fileSequence0.m4v (bip) (293.47 KB, application/octet-stream)
2012-11-20 22:54 UTC, Tim-Philipp Müller
  Details
fileSequence1.m4v (bop) (289.88 KB, application/octet-stream)
2012-11-20 22:55 UTC, Tim-Philipp Müller
  Details
videodecoder: Reset QoS time after pushing segment (1.07 KB, patch)
2018-03-01 04:21 UTC, Nicolas Dufresne (ndufresne)
committed Details | Review

Description Jonathan Matthew 2012-01-30 07:45:10 UTC
Created attachment 206398 [details] [review]
use playbin rather than playbin2 for tests

After changing the URI in the about-to-finish signal handler, audio buffers are dropped up to the length of the previous track.  Using a sink with max-lateness set to 1s produces this debug output:

GST_PERFORMANCE gstbasesink.c:2688:gst_base_sink_is_too_late:<fakesink0> buffer is too late 0:00:38.046552918 > 0:00:01.979591837

(the previous track was around 38 seconds long)

which makes it look like the sink's segment isn't configured correctly.  In 0.10 it accumulated the running time from previous segments, but this has been removed in 0.11.

If the new track is longer than the previous one, playback returns to normal after that.  Otherwise, the new track reaches EOS quickly and then another broken track change happens.

This can be reproduced easily with tests/icles/playback/test7 with the trivial patch I'm attaching.
Comment 1 Sebastian Dröge (slomo) 2012-01-30 08:56:59 UTC
Yes this is a known problem. We need to re-enable streamsynchronizer to update the base of the segments and to align all streams of a group at the end (e.g. when audio finishes earlier than the video the stream group has to be aligned to the video duration).
Comment 2 Tim-Philipp Müller 2012-11-20 22:54:46 UTC
Created attachment 229514 [details]
fileSequence0.m4v (bip)
Comment 3 Tim-Philipp Müller 2012-11-20 22:55:49 UTC
Created attachment 229515 [details]
fileSequence1.m4v (bop)
Comment 4 Tim-Philipp Müller 2012-11-20 23:01:06 UTC
Sorry, I'm going to hijack this bug, hope you don't mind.

Streamsynchronizer is enabled now, but track changes and about-to-finish still doesn't work properly yet.

Try:

 $ -base/tests/icles/playback/test7 file:///fileSequence0.m4v file:///fileSequence1.m4v

Should bip-bop. Doesn't. (Funnily, the original mpeg-ts files work much better, with only a 1-2 second video freeze before the transition).

  $ gst-launch-1.0 playbin uri=file:///fileSequence0.m4v

is instructive as well, wait for the end, it basically cuts out short, you don't see the video move for the last 0.2 seconds or so.
Comment 5 Sebastian Dröge (slomo) 2013-08-16 10:44:37 UTC
Almost works now, both files are played for 10 seconds with a/v sync. There's just something going wrong at about 8 seconds when the transition to the second file happens, there is some freezing.
Comment 6 Tim-Philipp Müller 2016-08-14 10:11:35 UTC
Still broken in master:

gst-play-1.0 --gapless /tmp/fileSequence0.m4v /tmp/fileSequence1.m4v
Comment 7 Nicolas Dufresne (ndufresne) 2018-03-01 04:21:03 UTC
Created attachment 369132 [details] [review]
videodecoder: Reset QoS time after pushing segment

This fixes playbin gapless playback. An ancient QoS time was used and
would lead to all frames being dropped.

I'm not sure it's really gapless with the two files in this bug, but I
believe this patch improves a lot the situation.
Comment 8 Nicolas Dufresne (ndufresne) 2018-03-01 04:33:55 UTC
In fact, it looks gapless if I add a queue before the sink:

gst-play-1.0 --gapless  one.m4v two.m4v --videosink="queue ! glimagesink"

And no buffer are being dropped by the decoder anymore.
Comment 9 Sebastian Dröge (slomo) 2018-03-01 08:37:39 UTC
Comment on attachment 369132 [details] [review]
videodecoder: Reset QoS time after pushing segment

This does not seem enough unfortunately, but it's nonetheless correct. What could happen is that the QOS event is received after this reset and before downstream ever saw the SEGMENT event.

The same problem probably exists in all elements that currently handle QOS events and they should also all be checked at least for this part of a solution.
Comment 10 Olivier Crête 2018-03-01 17:16:25 UTC
This fixes the about-to-finish functionality. The gaplessness is broken by the allocation query always being serialized, but that's another problem.

Attachment 369132 [details] pushed as 604c1bf - videodecoder: Reset QoS time after pushing segment
Comment 11 Olivier Crête 2018-03-01 17:19:22 UTC
Fixed for the next 1.13 release!
Comment 12 Sebastian Dröge (slomo) 2018-03-01 17:24:56 UTC
(In reply to Olivier Crête from comment #10)
> This fixes the about-to-finish functionality.

No, see in comment 9. It makes the race condition smaller but does not fix it :) Can you open a new bug about the remaining parts, e.g. at least what I wrote in comment 9 and the allocation query?
Comment 13 Sebastian Dröge (slomo) 2018-03-01 18:00:08 UTC
Also e.g. basetransform and all other elements doing something with QOS events should be looked at.
Comment 14 Olivier Crête 2018-03-01 18:26:18 UTC
(In reply to Sebastian Dröge (slomo) from comment #12)
> No, see in comment 9. It makes the race condition smaller but does not fix
> it :) Can you open a new bug about the remaining parts, e.g. at least what I
> wrote in comment 9 and the allocation query?

Yes, I'm sadly also aware of this. My current thoughts are that we need to send a downstream serialized event with the segment, and when it reaches the sink, send an upstream event, and only start modifying the qos events after this upstream event comes back.. But I'm not sure how this would work with multiple sinks (with a tee for example), where you could get a mix of transformed and untransformed qos events during the interval. But I think it's something we need to give more thoughts to. I'll file a new bug.
Comment 15 Olivier Crête 2018-03-01 18:26:48 UTC
*** Bug 792913 has been marked as a duplicate of this bug. ***
Comment 16 Nicolas Dufresne (ndufresne) 2018-03-01 18:46:36 UTC
Backported to 1.12