GNOME Bugzilla – Bug 746479
rtsp: Only two second of playback with rtpsrc and test-mp4 (rtsp-server)
Last modified: 2015-04-02 22:53:05 UTC
I'm having issue at the moment with RTSP playback on latest git (kind of worked two weeks ago). Right now if I run test-mp4 and gst-play-1.0 rtps://... only two second is played then it EOS. 2s is the size of the jitter buffer. So far I know that EOS comes from the jitter buffer. I notice that npt time are going backward. As these is no protection in the code for that, we get large value which I suspect are responsible for the eos timeout to trigger early. It's notice that Farstream unit test started failing too.
Marking as blocker.
Created attachment 299969 [details] [review] Patch adding warning and protection around substractions I don't think this patch is entirely correct, at least it uncovers some of the issues going on, allowing to trap overflow in places we did assume there would not be.
Created attachment 299970 [details] [review] rtpjitter: Protect substractions This patch protects all the substraction that where not before. It also warn as clearly these where not expected to overflow.
With the fixed version of this patch, we see that base_time and last_rtptime becomes invalid only after a clock skew. Maybe something is not reset properly ?
Created attachment 299972 [details] [review] rtpjitter: Protect substractions This patch protects all the substraction that where not before. It also warn as clearly these where not expected to overflow.
With the right check (was checking > instead of >=) it does not warn anymore. Which mean there is no overflow, my bad, the patch is still worth it probably.
Created attachment 299975 [details] [review] rtpjitter: Protect substractions This patch protects all the substraction that where not before. It also warn as clearly these where not expected to overflow.
Ok, new investigation path. First time we clock_id_wait(id, &clock_jitter), the resulting jitter is: 5124095:34:33.689709342 This cause all timeout to be reached at once. This does not seem like a reasonable jitter.
Will be my week of false positive, jitter is all right, it's negative (but only slightly). Well, sometimes it's 2s ahead of time, but in general it's fine.
Created attachment 299998 [details] Capture showing missing reply to play request So it seems the the server does not reply to PLAY request.
More information, the server do send the reply (we can see it in wireshark with vlc as a client), but it looks like the client stop reading from the TCP socket. One other thing I notice though is that RTPInfo is broken. The initial rtptime information is not correct. When running with VLC: RTP-Info: url=rtsp://localhost:8554/test/stream=0;seq=24747;rtptime=0, url=rtsp://localhost:8554/test/stream=1;seq=17776;rtptime=0\r\n And first RTP packet for each streams have: PT 96 Seq:24747 TS:931696044 PT 97 Seq:17776 TS:2784754487
Ok, no idea why I didn't get a reply in previous record, I always got it now. I think one should focus on the wrong ntptime in the RTP Info.
Created attachment 300023 [details] [review] rtsp-media: Properly return first rtptime Instead we where returning first GstBuffer timestamp. This would result in clock skew and unwanted behaviour in RTSP playback.
This fixes the skew at start, which would confuse the RTP jitter buffer. Though a skew and an overflow still occurs on seeks. (gstrtpjitterbuffer.c:2104):calculate_jitter: runtime check failed: (priv->last_rtptime == -1 || rtptime >= priv->last_rtptime) From the spec, after a seek, some buffers of the old segment might be travelling over the network and we are supposed to drop it base on the initial rtptime and seq we got from the RTPInfo. I'm not sure yet if we have code for that, e.g. we have trusty start seq + rtptime, please trust and drop mode. Any input about how this is supposed to work would be appreciated. I'm not sure how this could have worked in the past.
Additional note, this last patch playback position reporting when using VLC as a client. Though, seeks are followed by long freeze before resume on this client (which may be related to the seek issue).
Interesting, the method that sends start_segment is simply never called. Unless I'm mistaken again, gst_rtspsrc_handle_data() is never called. We endup with the default segment all the time, which could explain the clock skew after seek.
Learning the hard way, but in interleaved mode (streaming over the TCP socket) this method is called, and segment are correct. In UDP mode, this method is not called, and there is no code to actually fix the segment. The jitterbuffer receives a generic segment and fails.
Comment on attachment 300023 [details] [review] rtsp-media: Properly return first rtptime Shouldn't both values be the same? Or are they not because of the segment?
(In reply to Nicolas Dufresne (stormer) from comment #14) > From the spec, after a seek, some buffers of the old segment might be > travelling over the network and we are supposed to drop it base on the > initial rtptime and seq we got from the RTPInfo. I'm not sure yet if we have > code for that That's the seqnum-base (and clock-base) from the caps. I added something to jitterbuffer last week to drop every packet with a seqnum before that. (In reply to Nicolas Dufresne (stormer) from comment #17) > In UDP mode, this method is not called, and there is no code to actually > fix the segment. The jitterbuffer receives a generic segment and fails. How did that ever work at all? :) Maybe people only tested seeks in TCP mode?! I guess there should be a pad probe for modifying the segment after the udpsrcs then. Does that fix the remaining problem?
(In reply to Sebastian Dröge (slomo) from comment #19) > How did that ever work at all? :) Maybe people only tested seeks in TCP > mode?! > I guess there should be a pad probe for modifying the segment after the > udpsrcs then. Does that fix the remaining problem? This is exactly what I started writing, we'll see I think if it worked before it would be because the skew code may have succeed where it actually fails badly now. Or that we were doing like Wowza in the past (setting rtptime=0,seq=0 after every seek) and we only changed that to be compliant "recently" without testing the seeks. Second case is not that likelyly since seeks are also broken with Wowza server as I can observe: totem rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov
(In reply to Sebastian Dröge (slomo) from comment #18) > Comment on attachment 300023 [details] [review] [review] > rtsp-media: Properly return first rtptime > > Shouldn't both values be the same? Or are they not because of the segment? No, the rtp timestamp has a different scale. The buffer timestamp matches segment start in this case (0), while the first rtp timestamp is a random number.
Good point, forgot about that :)
For the segment event, you can just put a pad probe on the udpsrc srcpad... and replace all segment events with a new segment event that contains the rtspsrc segment.
(In reply to Sebastian Dröge (slomo) from comment #23) > For the segment event, you can just put a pad probe on the udpsrc srcpad... > and replace all segment events with a new segment event that contains the > rtspsrc segment. I tried, but it didn't work. It's probably not enough, or out of sync. I need to learn more again.
commit dfb053add33e6636ef473b78826e4a720cc1ed24 Author: Nicolas Dufresne <nicolas.dufresne@collabora.com> Date: Sat Mar 21 11:04:05 2015 -0400 rtsp-media: Properly return first rtptime Instead we where returning first GstBuffer timestamp. This would result in clock skew and unwanted behaviour in RTSP playback. https://bugzilla.gnome.org/show_bug.cgi?id=746479
Ok, was wondering, and then I just decided to test seeks with TCP streaming, to find out it does not work. calculate_jitter: cannot get current running_time gst_rtp_jitter_buffer_chain:<rtpjitterbuffer1> error: Received packet without DTS after a gap This is the error I get if I try to replace the segment in UDP mode using a probe.
For TCP we should always set a timestamp on the first buffer (after a flush or in the very beginning). Otherwise rtpjitterbuffer can't really compensate discontinuities or anything. With UDP we should get a DTS/PTS on every single buffer. The capture time.
(In reply to Sebastian Dröge (slomo) from comment #27) > Otherwise rtpjitterbuffer can't really compensate discontinuities or anything jitter I mean, it can't compensate for discontinuities/gaps at all in TCP mode. But in TCP mode we should not get them as TCP is a reliable transport.
Ok, actually it's depayloaders (handled in baseclass) that deals with segments. Position reporting is wrong after seeks though, so there must be some bug there. Meanwhile, this is a totally different bug (not clearly a regression, it might have never worked) and this one is fixed now.
Just realise overnight that this is not fully right yet :-( I need to check if the first buffer matches first expected. Otherwise the missing part may get clipped in one stream but not the other causing de-synch. In the case I don't know which one should be first, seeking should not be possible, hence the position should simply be the running time as before. Will work on that today.
This is fixed now.