(In reply to codebythepound from comment #0)
> Created attachment 330919 [details]
> MPD
> 
> Using source from 1.9.0.1.
> 
> Youtube uses mpeg-dash as one method to distribute its live streams, and I
> am trying to use gstreamer to play a stream directly from youtube servers. 
> Attached is part of the MPD for one of these streams.  I am assuming the MPD
> is valid.  It is from the live stream here:
> https://www.youtube.com/watch?v=njCDZWTI-xg.  Here is a link to an MPD for
> this file:
> https://manifest.googlevideo.com/api/manifest/dash/playlist_type/DVR/key/yt6/
> upn/OlXrWwK-70c/source/yt_live_broadcast/sparams/as,gcr,hfr,id,ip,ipbits,
> itag,playlist_type,requiressl,source,expire/gcr/us/signature/
> 80C14B2024BE29F75C1DBF68CDBDEF411A632D5E.
> 893DDA5444737864A0068A89270AB08F26ACF9D3/fexp/9407060,9416126,9416891,
> 9422596,9428398,9431012,9433096,9433223,9433946,9435526,9435876,9437066,
> 9437553,9437742,9439652,9440376/ip/216.228.112.21/requiressl/yes/itag/0/as/
> fmp4_audio_clear,webm_audio_clear,webm2_audio_clear,fmp4_sd_hd_clear,
> webm2_sd_hd_clear/expire/1467351997/id/njCDZWTI-xg.54/sver/3/ipbits/0/hfr/1
> 

How exactly to do you run this? I get a forbidden when trying. Is it region restricted?

> I was hoping to get your help with several issues relating to dashdemux when
> dealing with these streams:
> 
> 1) Youtube puts the <SegmentTimeline> in the <Period> but outside of any
> <AdaptdationSet>.  I assume this means that this timeline should be valid
> for all future <SegmentList>'s.  The current parser doesn't seem to like
> this as after the SegmentList is parsed, I get the error "segment has
> neither duration nor timeline" printed from
> gst_mpdparser_parse_mult_seg_base_type_ext().
> 
> To get around this for the time being I put a huge hack in
> gst_mpdparser_parse_segment_timeline_node that will store any timelines that
> have been parsed (in these files - only one) and add them to any node that
> did not contain a timeline in the method
> gst_mpdparser_parse_mult_seg_base_type_ext().  This gets me past this
> problem for the time being, but then I ran into the next issue (which may be
> caused by doing this, I'm not sure).

This is close to the real solution. Stuff that is under the <Period> should be considered default for their children that don't have that same node as a child. I'd prefer here to just reference the parent's node when needed instead of doing a copy.

> 
> 2) gst_mpd_client_get_period_index_at_time () iterates through the periods
> to try to find one that is valid for the current TOD.  The 1 period in the
> file is checked and has a correct start time, but for some reason has a
> duration of -1.  This causes the calculation to fail and return G_MAXUINT
> causing dashdemux to think there are no streams valid for the current time. 
> I'm not sure if -1 is an error or means "plays forever" - but I worked
> around this issue by checking if the duration was -1, and if so calling it a
> valid period.

This is a live stream, is it detected as such? The period duration of -1 might just mean that it is infinite/unknown as it is live.

Would you mind sharing a GST_DEBUG=*adaptive*:6,dashdemux:6,uridownloader:6 log of this issue? It would be easier to understand why it fails after you apply your hack.

I guess the dashmpd urls timeout after a while.  Here is the current one for that live stream:

http://manifest.googlevideo.com/api/manifest/dash/ip/216.228.112.21/fexp/9405988%2C9410705%2C9416126%2C9416891%2C9419452%2C9422596%2C9426198%2C9428398%2C9431012%2C9431881%2C9432055%2C9433096%2C9433221%2C9433946%2C9435526%2C9435693%2C9435876%2C9436607%2C9436636%2C9436804%2C9436824%2C9437066%2C9437226%2C9437282%2C9437552%2C9437742%2C9438662%2C9439160%2C9439260%2C9439482%2C9439652%2C9439944%2C9440009%2C9440376%2C9440738/sparams/as%2Cgcr%2Chfr%2Cid%2Cip%2Cipbits%2Citag%2Cplaylist_type%2Csource%2Cexpire/gcr/us/upn/seYXhS8AOv4/source/yt_live_broadcast/key/yt6/playlist_type/DVR/signature/0A26DCD10BCFFF4D82EBA7F13DB1D5159552FF98.C16E6E0673DBDCB6001016554FC1940AD9A48792/hfr/1/ipbits/0/sver/3/id/njCDZWTI-xg.54/as/fmp4_audio_clear%2Cfmp4_sd_hd_clear/expire/1467848496/itag/0
dbp = ChwKFmNTNGdVa3RheWptczE1OURVR3RhbXcQASABGAE

If that no longer works when you try it, you can obtain the current data for that video by examining the output of:

$ curl www.youtube.com/get_video_info?video_id=njCDZWTI-xg

extract the dashmpd argument, and url decode the string.  Then, just use gst-play-1.0 <that URL> and it should attempt to play.

I would be happy to add the debug output you asked for, but it would probably be easier if you just tried the above to see if it works for you.

I attached my hacks to gstmpdparser.c in case they are useful. 

> This is a live stream, is it detected as such? The period duration of -1 
> might just mean that it is infinite/unknown as it is live.

Yes, it is a live stream.  It seems to have been detected as live (dynamic).

Thanks.

Created attachment 330966 [details]
my hacks to this file

Is there anything further I can do to help here?

Thanks.

You could provide a patch with your changes instead of the whole source code, ideally as a commit that can be considered for merging or at least reviewing :)

My changes are a hack, meant to help show the problem and get further along parsing these manifests.  I seriously doubt they would be considered for merging, but I can submit as a review if that is what you would like.  Even with my changes there are still several problems that I wasn't able to solve.

It would make it easier to understand which hacks exactly helped in your situation at least

Created attachment 331637 [details] [review]
dash: properly inherit segmentlist from period

Representation can inherit SegmentList from Period if none is available
from the AdaptationSet

Created attachment 331638 [details] [review]
dash: do not use invalid stream duration

If it is is unknown, consider it infinite

These 2 patches bring it closer to playing. The problem now is that the current time is ahead of the last segment by a few minutes so dashdemux keeps waiting and waiting forever.

Looking at the MPD attached here, does anyone see how can we sync 'now' with the fragments or is indeed a few minutes behind the present?

I guess not. What do you think on having a 'present-offset' that is the difference between the time of the 'present' and the beginning of the last fragment available?

For example, 'now' is 22:52 on some day, and the last fragment of this live MPD is actually at 22:45. So we detect that and set an offset of ~7 minutes to care for this difference between 'now' and the tip of the MPD fragment list.

Will work on that unless someone sees something in this MPD to properly sync and account for this time difference. So far all my efforts failed on finding a field to properly align 'now' and the MPD fragments, my calculations always end up ahead of the available fragments.

There is already a 'presentation-delay' property of dashdemux.  Can we just supply a negative number here to correct for this?

Seems reasonable, however that means that it will never work out of the box with playbin. There must be a way to somehow guesstimate a "now". How are other players handling this?

Created attachment 332414 [details] [review]
this is not a proper patch

These are the remaining things to be changed to have this working.
This is not a proper solution just hacks I applied locally to identify what was missing.
Going to turn those into proper fixes soon.

Note for future debuggers:

Occasionally this stream sends a different availabilityStartTime (and then goes back to normal), which might cause dashdemux's logic to break.

availabilityStartTime: 2016/09/29 06:44:10.000000 (normal)
availabilityStartTime: 2016/10/01 22:20:06.000000 (bad)

When the second value is received it causes our client to wait for a fragment to be available in the server, as the jump if rather large, it believes it has to wait for 65+ hours. Oops.

Not sure if we have any option other than check for such large waits and ignore them.

What should happen with the other patches here?

I've been (slowly) working on having a 'global timeline' to allow attaching periods to it to keep them aligned. I was hoping to merge it all at once in case I have to change something but I guess those 2 bugfixes won't change anymore.

Should I push those 2 before the release?

Yes, they seem safe enough and especially the second seems like a very good idea... :)

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/406.