GNOME Bugzilla – Bug 657794
Playback does not start or is poor if pipeline is started before packets start arriving
Last modified: 2012-10-17 15:07:17 UTC
When playing a MPEG transport stream being either unicast or multicast over UDP GStreamer does not handle well pipeline being started before packets start arriving. Player: gst-launch-0.10 playbin2 uri=udp://127.0.0.1:65501 Content provider: gst-launch-0.10 dvbbasebin frequency=505833330 code-rate-hp=3/4 modulation="QAM 16" bandwidth=8 trans-mode=2k guard=32 hierarchy=0 code-rate-lp=3/4 program-numbers=4164 name=src .program_4164 src.program_4164 ! udpsink host=127.0.0.1 port=65501 (Example above streams BBC ONE from DVB-T Crystal Palace transmitter - for different channels/locations values will need to be different.)
Appears to be two different issues. The main one is that the first buffer received will be timestamped with 0 from the source, when the sink will already have reached time T. I have fixed this, and will post the patch. The second issue is that, when the source restarts, there is no indication of this, so the demuxer does not know to reset the offsetting of incoming packets. I will create a new bug for this, and add a dependency to this one, as it's related.
Created attachment 195716 [details] [review] mpegdemux: take into account offset of first buffer This fixes playback when the source starts later than the playback pipeline.
It should take into account the offsets on all buffers, probably doing some smoothing or filtering on the offsets to make it work smoothly. This will also fix your 'reset time' bug.
Wouldn't that only be doable for live use ? Otherwise we should heed input timestamps...
It should work for all use cases, for non-live sources, the first timestamp should be 0 and subsequent timestamps -1.
Created attachment 195723 [details] [review] mpegdemux: take into account offset of first buffer This fixes playback when the source starts later than the playback pipeline. Updated patch, avoids offseting invalid timestamps
Hah. I forgot buffer timestamps are ignored by the demuxer in favor of PTS-generated timestamps. Makes sense now. I intended to resync on discontinuities, but I may do something like that running resync on any input timestamp jump.
Created attachment 195761 [details] [review] mpegdemux: take into account offset of first buffer This fixes playback when the source starts later than the playback pipeline. Seems I mixed up the updated patch upload, here is the latest patch.
Created attachment 195774 [details] [review] mpegdemux: take into account offset of first buffer This fixes playback when the source starts later than the playback pipeline, and when a source drops for a while then restarts.
The restart issue wasn't what I thought at first: Timestamps come out OK, accounting for the gap. However, there was data left in the adapter from before the gap, and, when more data comes in, buffers with the old PTS could be sent, causing then a jump in output timestamps which would cause the sink to then wait. The solution is to flush the adapter when we detect a gap in input timestamps. Stopping and restarting a source now works fine. There is now (AFAICT) one single remaining issue, which is starting a new source after stopping the current source, as the PTS then do not match (eg, when stopping stream of some station, then starting streaming of a new station). I'll look at this now, and I'll close the other bug since on second thoughts they are pretty much the same core issue.
*** Bug 658275 has been marked as a duplicate of this bug. ***
And that issue is not only about timestamps being unrelated, but streams from two sources having colliding PIDs. It seems possible to reset the demuxer when a gap in input timestamps happens, but that starts being really hacky. When I try this (and resetting some more bits in _reset), I'm left with only the unrelated timestmap issue. As far as I can tell, I don't get a notification from the udpsrc, it's oblivious to the other side of the network having changed source. It does seem like a bad place to "fix" it, but I don't see another possible place. That said, is this (switching sources real time while the player continues streaming without being aware a switch happens) a use case for you ?
(In reply to comment #12) > That said, is this (switching sources real time while the player continues > streaming without being aware a switch happens) a use case for you ? We need to be able to handle this, but it doesn't have to be clean - timeshared channels (e.g. BBC THREE/BBC FOUR and CBBC Channel/CBeebies) is the use-case. We can thus cope if GStreamer simply throws an event at us to say that things have changed and we need to restart. For background: the BBC (and many other broadcasters) share an encoder between two stations. At 7:00pm, it stops encoding one station and starts on the new one, resulting in a PTS discontinuity, as the new station will switch from playing out a static loop to playing the encoder's output, and the old station will switch from the encoder's output to a static loop. We don't need to follow the switch, but we do need to stop playing and not follow the encoder to its new service.
Those timestamp discontinuities should be handled. I'll check at 7pm (British time, not GMT, right ?) to see if it works with that. The issue I've seen with the switch that I mentioned is: - the previous station has a number of stream PIDs - one being a PMT referencing the other ones. - the next station has a number of (likely different) stream PIDs, one being a PMT referencing the other ones. - the PID of the PMT is the same in both cases This causes GStreamer to not realize the PMT is different, since its PID is the same, and thus not realize the streams have changed, and thus ignore the new streams. If something hacky's acceptable, then it should be alright, I can err on the side of caution, and reset everything when an input ts discont happens. Or always decode the PMT and inspect the data it contains to decide if it's changed. I'll get on that then.
I have seen PMTs where the only clue that there's a PID change is the version number shifting - if we deliberately change station on you, we will always stop and restart, but broadcasters do occasionally renumber the PIDs in the stream, and rely on PMT versioning to ensure you get the right PIDs.
I had the NTB streaming from BBC 4 (or 3 ? One of the first pair you mentioned, I think it was 4) from one minute before to a couple minutes after 7pm yesterday, I saw no obvious change, so this seems to work, unless the switch did not happen. I assume when you mean 7:00pm, it does mean roughly 7:00pm and not 5 minutes earlier or later, in which case I could have missed the switch. Could you explain what you said last: "We don't need to follow the switch, but we do need to stop playing and not follow the encoder to its new service." Do I understand correctly that, when that switch happens, you need to become aware of it in the Python layer, so you can give new commands to the pipeline, such as stopping it ? Presumably in that case there's no input timestamp gap, and I have to determine that a switch happens by constantly inspecting the innards of the PAT/PMT. Would you happen to have a stream that includes such a gap, so I'm sure I'm testing with the kind of switch you're talking about ?
Created attachment 196004 [details] [review] mpegtsdemux: send an element message after an input stall When streaming off a UDP source, if the backend switches channels, we won't know about it. No new segment, no flushes, etc. We will get a PAT from the new channel, but it will have the same PID and probably version 0 as well, so we'll think it's all fine. However, we won't be getting buffers for the old PIDs (or only some of them), and will be getting buffers for new PIDs, which will be ignored as not part of the current program. Attempts at righting this wrong led to Big Cans of Worms, especially the playbin2 one. However, we can detect this is happening by observing gaps gaps in input timestamps (since the UDP source is live, it will timestamp incoming buffers with current pipeline time). If we don't receive anything for half a second, we will send an element message with name 'input-stall', which can then be interpreted by the player as a request to restart the pipeline, and playbin2 will then properly pick up the stream.
So in the end I've had to go the way of telling the application about it. When a input stall is detected, the demuxer sends an element message about it. This will do nothing by default, and python code to restart the pipeline when receiving it is: def on_message(self, bus, message): t = message.type src = message.src.get_name() if t == gst.MESSAGE_ELEMENT: src = message.src.get_name() name = message.structure.get_name() if name == "input-stall": print 'Element %s reported input stall' % (src) self.player.set_state(gst.STATE_READY) self.player.set_state(gst.STATE_PLAYING) else: print 'Received unknown element specific message %s from %s' % (name, src) This seems to work very well switching channels on the NTB. That now leaves us with just the BBC same-encoder-with-two-stations possible issue, which I might or might not have missed yesterday. I'll stream from 6:50 to 7:10 tonight to make really sure I can't have missed it.
(In reply to comment #16) > I had the NTB streaming from BBC 4 (or 3 ? One of the first pair you mentioned, > I think it was 4) from one minute before to a couple minutes after 7pm > yesterday, I saw no obvious change, so this seems to work, unless the switch > did not happen. I assume when you mean 7:00pm, it does mean roughly 7:00pm and > not 5 minutes earlier or later, in which case I could have missed the switch. > > Could you explain what you said last: > "We don't need to follow > the switch, but we do need to stop playing and not follow the encoder to its > new service." > > Do I understand correctly that, when that switch happens, you need to become > aware of it in the Python layer, so you can give new commands to the pipeline, > such as stopping it ? Presumably in that case there's no input timestamp gap, > and I have to determine that a switch happens by constantly inspecting the > innards of the PAT/PMT. Would you happen to have a stream that includes such a > gap, so I'm sure I'm testing with the kind of switch you're talking about ? When the switch happens, the PAT is unlikely to change version, but the PMT will change version number (staying on the same PID). We need GStreamer to at least identify that this has happened, so that we can restart; if it can follow the changed PMT, that's a better fix.
Between last two patches there is a mystery of this line: - gst_mpegts_demux_reset (demux); Which input discontinuity patch references, but previous patch does not add. What is the solution here?
Created attachment 196932 [details] [review] mpegtsdemux: send an element message after an input stall When streaming off a UDP source, if the backend switches channels, we won't know about it. No new segment, no flushes, etc. We will get a PAT from the new channel, but it will have the same PID and probably version 0 as well, so we'll think it's all fine. However, we won't be getting buffers for the old PIDs (or only some of them), and will be getting buffers for new PIDs, which will be ignored as not part of the current program. Attempts at righting this wrong led to Big Cans of Worms, especially the playbin2 one. However, we can detect this is happening by observing gaps gaps in input timestamps (since the UDP source is live, it will timestamp incoming buffers with current pipeline time). If we don't receive anything for half a second, we will send an element message with name 'input-stall', which can then be interpreted by the player as a request to restart the pipeline, and playbin2 will then properly pick up the stream.
Ah, indeed, this was part of one of my earlier attempts to fix it without having to restart (by resetting the demuxer when we might need to force re-parsing of PAT/PMT/etc), and that did go anywhere, but I was still keeping it in my tree. New updated patch attached.
First patch pushed. The second (stall message) one is a bit too specific to push without feedback first. commit 9715fc318980763a3665afb977c8fc52aaf794a8 Author: Vincent Penquerc'h <vincent.penquerch@collabora.co.uk> Date: Mon Sep 5 17:40:07 2011 +0100 mpegdemux: take into account offset of first buffer This fixes playback when the source starts later than the playback pipeline, and when a source drops for a while then restarts. https://bugzilla.gnome.org/show_bug.cgi?id=657794
Is this still an issue with current git, which uses tsdemux by default ?
Which branch, 0.10 or 0.11? If TS demuxer got rewritten it will be a huge task to validate it given the number of edge cases that were found and worked around in the old one.
in both branches (0.10 and master).
When can we expect a 0.10 release with all this new stuff in? It turns out it is not that easy to built from git of the day and if 0.10 release will come soon I would rather wait a bit than waste a lot of time.
I would not count on "soon". Perhaps sooner if there was confidence that people have tested things though :)
What are the chances to get a buildable source tarballs snapshot for gstreamer, gst-plugins-base, good, bad, ugly and ffmpeg, of the 0.10 git branch?
Current 0.10 git should build. I've worked on it for the last couple weeks, and the only trouble I had was some git branch merge issue locally (I did not try to use -ffmpeg though). It should "just work" if you update all at the same time, then build/install in order core, -base, -good-, -bad, -ugly. If a tarball is preferred for further distribution, each revision can be made into a separate tarball (eg, the "download" link from http://cgit.freedesktop.org/gstreamer/gst-plugins-base/commit/?id=e729ad1c9c58987984e1220d6323f7bb24a9cae5).
-base did not build for me today (nor could it ran "make dist" successfully), I don't know.. I can try building RPMs from git tarballs by running autogen.sh instead of configure if you think that should work.
Are you sure you're building that -base againts a matching core ? Just now, -base builds fine (using core and -base from 5 minutes ago, installed in my $HOME). There could be dependencies I don't have and you do, or you could be building -base against an older core installed previously (especially if you have both a system gstreamer install and one you're building manually).
Could we please move all this unrelated tarball/build discussion out of bugzilla, at least out of this bug? > If a tarball is preferred for further distribution, each revision can be made > into a separate tarball (eg, the "download" link from [cgit]) Keep in mind however that those tarballs have little to do with 'make dist' tarballs, they are just a snapshot of the git repository, they don't contain all the usual autotools stuff found in release tarballs.
Closing this bug report as no further information has been provided. Please feel free to reopen this bug if you can provide the information asked for. Thanks!