Apparently, EXT-X-TARGETDURATION is only required if we're not linking to *another* playlist.

I've update shared-mime-info to not try and handle those files.

Contents of http://qthttp.apple.com.edgesuite.net/1006ad9g4hjk/all_w.m3u8 (linked to from http://events.apple.com.edgesuite.net/1006ad9g4hjk/event/index.html)

#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=246221
0240/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=38872
0064/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=153059
0150/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=455394
0440/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=650356
0640/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=861738
0840/prog_index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1258987
1240/prog_index.m3u8

This is a simple redirection, which would select a particular stream based on the available bandwidth.

Hi, an other example is this one:

http://iphone.envivio.com/iphone/downloads/ch6/index.m3u8

which contains:

#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=257184
01.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=359456
02.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=565504
03.m3u8

then this one http://iphone.envivio.com/iphone/downloads/ch6/01.m3u8

which contains 'ts' file which have to be played successively:

#EXTM3U
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:264169
#EXTINF:5,
20100711T120659-01-264169.ts
#EXTINF:5,
20100711T120659-01-264170.ts
#EXTINF:5,
20100711T120659-01-264171.ts

Finally, http://iphone.envivio.com/iphone/downloads/ch6/20100711T120659-01-264169.ts

is playable using:

gst-launch-0.10 souphttpsrc location='the final uri.ts' ! mpegtsparse ! mpegtsdemux ! ...

Note that whithout mpegtsparse then the pipeline (especially mpegtsdemux) is waiting around 5 sec before demuxing. 


If you first download all the 'ts' files then you can play the full stream:

gst-launch-0.10 multifilesrc location=20100711T120659-01-%06d.ts index=264169 ! mpegtsparse ! decodebin2 name=d ! queue2 ! glimagesink d. ! queue2 ! audioconvert ! autoaudiosink

Gstreamer should automatically handle this kind of streaming.

You can find a very experimental, early (and ugly?) implementation of a source element I started to work on here:

http://gitorious.org/gstreamer/gst-fragmented

It currently plays only vod streams (although live should come soon), ex:

gst-launch applehlssrc location=http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8  ! decodebin2 ! xvimagesink (this takes a long time to bufferise because of decodebin multiqueue0 max-size-bytes, iirc.


It's starting faster when putting mpegtsdemux before decodebin.

early feedback from developers would be greatly appreciated!

Hi,

It's says "no new fragments" after some seconds, with the source exposed in comment 2.
It should increment the name of ".ts" by 1.

(In reply to comment #4)
> Hi,
> 
> It's says "no new fragments" after some seconds, with the source exposed in
> comment 2.
> It should increment the name of ".ts" by 1.

Yeah, it's kinda broken, especially with live sources :) It's really an experiment.

It does increment media sequence (the name of .ts), the part that doesn't work is the playlist refresh.

Feel free to provide patches!

I am more worried by the fact that it doesn't play smoothly, when pushing new fragments it hangs a little, perhaps because of the way it uses appsrc and httpsrc. I am sure it's not thread-safe etc...

(In reply to comment #5)

> I am more worried by the fact that it doesn't play smoothly, when pushing new
> fragments it hangs a little,

I am not sure this is the same problem but adding the 'mpegtsparse' filter resolves those hangs. (see pipelines from comment #2)

I retook Marc-Andre's work and the plugin is now working now pretty well. I still need to clean-up the code a little bit, handle some corner cases and improve the bitrate switch decision algorithm, but I hope it will be in a good shape to be in -bad by the end of the week.
I still don't know how to integrate it properly with uridecodebin and I was wondering if it's a good idea to create a new 'hls' protocol to have something like:
uridecodebin2 uri=hls://my-stream.net/main.m3u8

The code is hosted in gitorius:
http://gitorious.org/~ylatuya/gstreamer/ylatuya-gst-fragmented 

You can test it with the following pipeline:
gst-launch hlssrc location=http://devimages.apple.com/iphone/samples/bipbop/gear1/prog_index.m3u8 ! mpegtsdemux ! ffdec_h264 ! xvimagesink

The bitrate switching doesn't work with apple's demo, because they encode each bitrate at a different resolution and the decoder doesn't seem to like it.

(In reply to comment #7)
> I retook Marc-Andre's work and the plugin is now working now pretty well. I
> still need to clean-up the code a little bit, handle some corner cases and
> improve the bitrate switch decision algorithm, but I hope it will be in a good
> shape to be in -bad by the end of the week.
> I still don't know how to integrate it properly with uridecodebin and I was
> wondering if it's a good idea to create a new 'hls' protocol to have something
> like:
> uridecodebin2 uri=hls://my-stream.net/main.m3u8

I don't like that. I'd much rather the content of the m3u8 list was detected properly as an HLS playlist (some keywords must be in the playlist [1]), and ignored as just a text file otherwise.

[1]: This is how shared-mime-info does it:
http://cgit.freedesktop.org/xdg/shared-mime-info/tree/freedesktop.org.xml.in#n3358

That would be easier, some tags like #EXT-X-TARGETDURATION or #EXT-X-MEDIA-SEQUENCE are specific to these playlists and could be used to identify them.

And the mime-type too 'application/vnd.apple.mpegurl'

> I don't like that. I'd much rather the content of the m3u8 list was detected
> properly as an HLS playlist (some keywords must be in the playlist [1]), and
> ignored as just a text file otherwise.
> 
> [1]: This is how shared-mime-info does it:
> http://cgit.freedesktop.org/xdg/shared-mime-info/tree/freedesktop.org.xml.in#n3358

To be honest I don't know how to do that. The souphttp element implements the URIHandle interface and it will be selected for any uri with the 'html' protocol. My knowledge in autoselection of source elements is very limited and I don't know where I should be parsing the playlist to decide which source element should be used. Could you point me to an example?

souphttpsrc will be selected for http:// URIs, yes. But your plugin will step in one step later. You have to write a typefinder for the m3u8 playlists (see gst-plugins-base/gst/typefindfunctions for example), which detects them as (for example) "playlist/m3u8" and then your "demuxer" element for it will be selected because it accepts "playlist/m3u8" and has the highest rank.

Ok, instead of being a source element, I'll make it a normal element with a sink pad accepting 'playlist/m3u8' and a source pad with pushing 'video/mpegts" buffers.

I have been thinking again on how to implemented it, and it looks impossible to do the way you propose. 
If the m3u8 is a variant playlist (http://tools.ietf.org/html/draft-pantos-http-live-streaming-04#section-8.5), it might be doable this way because this playlist is only downloaded once and contains the uri's for the fragments playlists.
Instead, if the m3u8 is a sliding window playlist (http://tools.ietf.org/html/draft-pantos-http-live-streaming-04#section-8.3) it won't work, because the client needs to refresh the playlist periodically and the element won't know its uri.

What I have done at the moment is to add the 'hls' protocol in the UriInterface implementation of the source element, where 'hls' is then translated to 'http' to fetch the resource from the good location.

The following commands works:
gst-launch hlsrc location=http://my-server/main.m3u8
gst-launch playbin2 uri=hls://my-server/main.m3u8

Yes if you need to refetch the playlist over and over again, you have to use a different URI scheme and implement a source element...

Or do it the demuxer-like way and additionally refetch the playlist inside your demuxer.

problem with demuxer way is you need to know the source url of the original playlist...

(In reply to comment #16)
> problem with demuxer way is you need to know the source url of the original
> playlist...

gst_query_new_uri() can be used for that.

Created attachment 174029 [details] [review]
Add HTTP live streaming source

Patch against gst-plugins-bad to inlude the HLS source

Review of attachment 174029 [details] [review]:

Thank you very much Andoni for taking the time to improve greatly my initial work!

Just to highlight some of the changes that you made that might be controvorsial:

- make it a source, instead of a bin/"demuxer", implementing a "hls" URI handler
- use Soup instead of gst_element_make_from_uri() for the "fetching" part.
- drop "appsrc" usage, replaced by a local fragment cache
- and many many things that I am sure improved a lot the element, and make it actually work :)

I am glad you are pushing it forward, and I'll let someone more experienced with GStreamer comment the patch.
Let's hope it's going to be merged soon! finger crossed!

(In reply to comment #19)
> Review of attachment 174029 [details] [review]:
> 
> Thank you very much Andoni for taking the time to improve greatly my initial
> work!
I must say that It was an excellent starting point :D
> 
> Just to highlight some of the changes that you made that might be
> controvorsial:
> 
> - make it a source, instead of a bin/"demuxer", implementing a "hls" URI
> handler
I though the element should be a source rather than a "demuxer" because it only pushes fragments, like if it was reading from a file. I don't think gstreamer supports playlists at the moment, and I haven't found any other element that takes an uri, "demuxes" it and pushes the packets to a real demuxer. I don't think it's very intuitive. And this could be implement with with a playlists demuxer, whith a redirect.
 > - use Soup instead of gst_element_make_from_uri() for the "fetching" part.
The fetching part is very abstracted and could be easily replaced, but anyway I think that's the right way of doing it, since it's pure HTTP (it's like making souphttpsrc use inernally gst_element_make_from_uri() to support reading from files) 
> - drop "appsrc" usage, replaced by a local fragment cache
It now uses an adapter and the element subclasses GstBaseSrc, which makes it more or less the same. It could use a queue and push the whole fragment downstream, but I don't if it's a good idea.
> - and many many things that I am sure improved a lot the element, and make it
> actually work :)
Yeiii!
> 
> I am glad you are pushing it forward, and I'll let someone more experienced
> with GStreamer comment the patch.
> Let's hope it's going to be merged soon! finger crossed!
Let see if it's approved and we have it in bad before the freeze :D

I don't really like these changes. If you make it a real source it will be impossible to autoplug this unless you also have your own URI scheme (which you don't, you have http). And if you use libsoup directly instead of using "any" http source for fetching the file, you make it harder to reuse the element in other use cases where you don't have libsoup but some other http library (think of webkit, which also has its own http source element).

IMHO this should really be a demuxer, the first "playlist"-like demuxer *shrug*

Ok, I'll restart with the "demuxer"-like design.

+1 for having it as a demuxer

Hello, I have 2 more sources for tests:

http://12.139.66.61/ftproot/BBCA/playlist.m3u8

http://12.139.66.61/ftproot/MC/playlist.m3u8

user : hlitdemo
passwd : C0nverge

ping ?

I have restarted this weekend and I'll probably have something for tomorrow.

The element is almost ready but I'm having some issues make it work with uridecodebin. You can find it here[0]

It now uses internally gst_make_element_from_uri to fetch the fragments and has a sink pad with the "palylist/m3u8" caps.
I have also patched typefind[1] to detect the m3u8 playlists and the element is now  autoplugged, but anything gets plugged after it, very likely because it doesn't push any buffer until it has queued the first fragments.

This pipeline works:
gst-launch souphttpsrc location=http://devimages.apple.com/iphone/samples/bipbop/gear1/prog_index.m3u8 ! hlssrc ! decodebin !  xvimagesink -v

But with uridecodebin, the demuxer is not linked:
gst-launch  uridecodebin uri=http://devimages.apple.com/iphone/samples/bipbop/gear1/prog_index.m3u8  ! xvimagesink -v

I have also patched souphttpsrc[1] to handle URI queries.
Another issue that I have found is that filesrc respond the uri query with an invalid uri, and gst_element_make_from_uri() just fails with the query's uri :/


[0] http://gitorious.org/ylatuya-gstreamer/gst-plugins-bad/commits/hls
[1] http://gitorious.org/ylatuya-gstreamer/gst-plugins-base/commit/b826823798814b9f8f1e8f9c88aac46d1bcad191
[2] http://gitorious.org/ylatuya-gstreamer/gst-plugins-good/commit/c1c297ad6c70c86650fa2c6e69dd1638f76c155b

I have fixed the the typefind patch, which was adding a '\0' for the string comparison.

http://gitorious.org/ylatuya-gstreamer/gst-plugins-base/commit/ff9de9e4d41f544f93df193d05e5de5bb39e5b25

I have also some patches for webkit, but I still haven't managed to use the webkit source properly as it hangs from some reason randomnly in:
webkitwebsrc WebCore/platform/graphics/gstreamer/WebKitWebSourceGStreamer.cpp:595:webKitWebSrcNeedDataCb:<webkitwebsrc1> Need more data: 4096
webkitwebsrc  WebCore/platform/graphics/gstreamer/WebKitWebSourceGStreamer.cpp:443:webKitWebSrcStart:<webkitwebsrc1> Started request


http://gitorious.org/~ylatuya/webkitgtk/ylatuyas-webkit

Created attachment 181032 [details]
webkit playing an http live stream using the video tag

very cool :)

I have been playing around with the hls element and it seems like there might be a problem with swapping bitrate. The element starts by fetching (and caches) three clip fragments from the stream with the lowest bitrate and they play just fine, during the caching the element notice that it should increase bitrate and fetches a few seconds later a version of clip4 that has higher bandwith. 

So far so good, BUT everything freeze when clip4 is about to start playing!

I have been using this stream to test with:
http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8

I also have a small (and most likely very very wrong) theory as to what might be the problem:

http://devimages.apple.com/iphone/samples/bipbop/gear1/fileSequence2.ts :

Program 1 
Stream #0.0[0x101]: Audio: aac, 22050 Hz, mono, s16, 35 kb/s
Stream #0.1[0x102]: Video: h264, yuv420p, 192x144, 14.99 tbr, 90k tbn, 180k tbc

http://devimages.apple.com/iphone/samples/bipbop/gear2/fileSequence3.ts :

Program 1 
Stream #0.0[0x101]: Video: h264, yuv420p, 320x240, 29.97 tbr, 90k tbn, 180k tbc
Stream #0.1[0x102]: Audio: aac, 22050 Hz, mono, s16, 1 kb/s

That is the PIDs change "mid stream"!

Am I the only one seeing this (that is that if playing the HLS stream above it freezes after 30 seconds) or have I found an actual problem?

The problem with apple's multibitrate example is that they use a different picture size for the different qualities and the ffdec_h264 decoder doesn't support it, so it's rather a bug in the decoder than in the element itself.

For the first quality the video stream is encoded at 192x144 and the second one at 320x240.

Hi, 

More sources for testing:

http://etf1-apple-live.adaptive.level3.net/apple/etf1/etf1live/stc_5_0.m3u8

http://etf1-apple-live.adaptive.level3.net/apple/etf1/etf1live/stc_4_0.m3u8

http://etf1-apple-live.adaptive.level3.net/apple/etf1/etf1live/stc_3_0.m3u8

http://etf1-apple-live.adaptive.level3.net/apple/etf1/etf1live/stc_2_0.m3u8

http://etf1-apple-live.adaptive.level3.net/apple/etf1/etf1live/stc_1_0.m3u8

I tested a little bit the new design. It takes some times when entering the playing state. Around 1 min for gst-launch playbin2 uri=http://etf1-apple-live.adaptive.level3.net/apple/etf1/etf1live/stc_1_0.m3u8

Would it be possible to force hlssrc to play the stream which has its bitrate the closest than a given bitrate through a property ?
(for example, even if a user can play the high quality stream, he could also decide to play the low one) (Or also if in a certain context, the user knows that he cannot play high quality stream, then it's not necessary to try to play it) (There is a "connection-speed" property in playbin2)

Also would it be possible to report the decoded url for the fragment, through a tag event or something else useable (not only in debug) ?

Is there a way to retrieve the downloaded fragments on the local hard disk ? (download property of uridecodebin ?)

Julien

(In reply to comment #34)
> Hi, 
> 
> I tested a little bit the new design. It takes some times when entering the
> playing state. Around 1 min for gst-launch playbin2

I have also experienced the same problem but I didn't have time to debug it further. It also happens whith stream with a low bitrate, so I believe somewhere a queue is waiting to be filled to start playing.

> Would it be possible to force hlssrc to play the stream which has its bitrate
> the closest than a given bitrate through a property ?
> (for example, even if a user can play the high quality stream, he could also
> decide to play the low one) (Or also if in a certain context, the user knows
> that he cannot play high quality stream, then it's not necessary to try to play
> it) (There is a "connection-speed" property in playbin2)

I think we should have a property to select the bitrate switch algorithm. The current one is very basic and it only takes in account the download speed but it should also take in account QOS from downstream elements. It would be straightforward having the following algorithms:
 * Always lowest
 * Always highest
 * Connection speed
> 
> Also would it be possible to report the decoded url for the fragment, through a
> tag event or something else useable (not only in debug) ?
> 
We could post a message in the bus with the decoded url if you think it's useful.

> Is there a way to retrieve the downloaded fragments on the local hard disk ?
> (download property of uridecodebin ?)
You can use multifilesink element for that, because the element pushes one buffer for each fragment
souphttpsrc ! hlssrc ! multifdsink

(In reply to comment #35)
> (In reply to comment #34)
> > Hi, 
> > 
> > I tested a little bit the new design. It takes some times when entering the
> > playing state. Around 1 min for gst-launch playbin2
> 
> I have also experienced the same problem but I didn't have time to debug it
> further. It also happens whith stream with a low bitrate, so I believe
> somewhere a queue is waiting to be filled to start playing.
I meant: "It only happens with"
For instance using "souphttpsrc ! hlssrc ! mpegtsdemux ! ffdec_h264 ! xvimagesink" plays right after downloading the first 2 fragments.

Created attachment 182822 [details]
zipped log1.txt and log2.txt that contain output of 2 pipelines

log1.txt:

GST_DEBUG=decodebin2:5,hlssrc:5,GST_EVENT:5 gst-launch-0.10 souphttpsrc location=http://etf1-apple-live.adaptive.level3.net/apple/etf1/etf1live/stc_1_0.m3u8 ! hlssrc ! decodebin2 ! xvimagesink -v


log2.txt:

GST_DEBUG=decodebin2:5,hlssrc:5,GST_EVENT:5 gst-launch-0.10 souphttpsrc location=http://etf1-apple-live.adaptive.level3.net/apple/etf1/etf1live/stc_1_0.m3u8 ! hlssrc ! mpegtsdemux  ! decodebin2 ! xvimagesink -v


In the log1.txt it takes about 45 sec to reach the playing state whereas it takes 14 sec in log2.txt

I noticed that in log1.txt decodebin2 does not appear between the 21.407182007 sec and the 45.868456809 sec (gstdecodebin2.c:2538:multi_queue_overrun_cb:<decodebin20>[00m multiqueue 'multiqueue0' (0x88b4ec0) is full).
It seems the pad was blocked and reaching the full condition woke it up. 

So it seems we are lucky here and a normal behavior would be to reach the playing state before reaching the full condition.

> > Also would it be possible to report the decoded url for the fragment, through a
> > tag event or something else useable (not only in debug) ?
> > 
> We could post a message in the bus with the decoded url if you think it's
> useful.
> 

Also it would be useful to post a message on the bus when hlssrc selects an other bitrate

(In reply to comment #37)

> In the log1.txt it takes about 45 sec to reach the playing state whereas it
> takes 14 sec in log2.txt
> 
> I noticed that in log1.txt decodebin2 does not appear between the 21.407182007
> sec and the 45.868456809 sec
> (gstdecodebin2.c:2538:multi_queue_overrun_cb:<decodebin20>[00m multiqueue
> 'multiqueue0' (0x88b4ec0) is full).
> It seems the pad was blocked and reaching the full condition woke it up. 
> 
> So it seems we are lucky here and a normal behavior would be to reach the
> playing state before reaching the full condition.

That's what I was saying in the previous comment, for lower bitrates the queue is not filled until we fetch more fragments. We fetch the first 3 fragments and wait 10 seconds to fetch the next one, but the queue isn't filled yet and then we wait another 10 seconds to fetch the 4th fragments, which fills the queue.
The problem might be in the way queues are configured in decodebin2[1]

Try with fragments-cache=5, which will start caching 5 fragments and therefore filling the queue.


[1] http://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/gst/playback/gstdecodebin2.c#n2645

(In reply to comment #39)
> The problem might be in the way queues are configured in decodebin2[1]
> 
One of the comments says:
/* update runtime limits. At runtime, we try to keep the amount of buffers
 * in the queues as low as possible (but at least 5 buffers). */
I need to look at it more deeply but  "at least 5 buffers" s not met before de the demuxer and it's met before the decoder (a fragment as more than 5 frames), so that might be the reason why using decodebin2 after the demuxer is working

Just a random guess but is the fragment demuxer emitting no-more-pads (gst_element_no_more_pads())? Otherwise decodebin2 will wait until the queues are filled before doing anything else

(In reply to comment #41)
> Just a random guess but is the fragment demuxer emitting no-more-pads
> (gst_element_no_more_pads())? Otherwise decodebin2 will wait until the queues
> are filled before doing anything else
The source pad is GST_PAD_ALWAYS, should it emit no-more-pads?

No, that's fine then

(In reply to comment #28)
> I have fixed the the typefind patch, which was adding a '\0' for the string
> comparison.
> 
> http://gitorious.org/ylatuya-gstreamer/gst-plugins-base/commit/ff9de9e4d41f544f93df193d05e5de5bb39e5b25
@Sebastian: Can you review this one so it gets before the next freeze of gst-plugins-base?

Created attachment 183047 [details]
log when using fragments-cache=5

(In reply to comment #39)
> Try with fragments-cache=5, which will start caching 5 fragments and therefore
> filling the queue.

Attached log_cache5.tar.gz generated with:

GST_DEBUG=hlssrc:5,decodebin2:5 gst-launch-0.10 souphttpsrc location=http://etf1-apple-live.adaptive.level3.net/apple/etf1/etf1live/stc_1_0.m3u8 ! hlssrc fragments-cache=5 !  decodebin2  ! xvimagesink -v

But I got an error:

ERROR: from element /GstPipeline:pipeline0/GstHLSSrc:hlssrc0: Could not cache the first fragments

stc_1_0.m3u8 contains only 3 fragments but hlssrc is supposed to wait for the next playlist no ?


(In reply to comment #40)
> (In reply to comment #39)
> > The problem might be in the way queues are configured in decodebin2[1]
> > 
> One of the comments says:
> /* update runtime limits. At runtime, we try to keep the amount of buffers
>  * in the queues as low as possible (but at least 5 buffers). */
> I need to look at it more deeply but  "at least 5 buffers" s not met before de
> the demuxer and it's met before the decoder (a fragment as more than 5 frames),
> so that might be the reason why using decodebin2 after the demuxer is working

Yep, when using decodebin2 after the demuxer, it does not add a multiqueue element.


A random question:
Would it be referred to https://bugzilla.gnome.org/show_bug.cgi?id=600648 ?

(In reply to comment #45)
> Created an attachment (id=183047) [details]
> log when using fragments-cache=5
> 
> (In reply to comment #39)
> GST_DEBUG=hlssrc:5,decodebin2:5 gst-launch-0.10 souphttpsrc
> location=http://etf1-apple-live.adaptive.level3.net/apple/etf1/etf1live/stc_1_0.m3u8
> ! hlssrc fragments-cache=5 !  decodebin2  ! xvimagesink -v
> 
> But I got an error:
> 
> ERROR: from element /GstPipeline:pipeline0/GstHLSSrc:hlssrc0: Could not cache
> the first fragments
> 
> stc_1_0.m3u8 contains only 3 fragments but hlssrc is supposed to wait for the
> next playlist no ?
> 
> 
I'm not sure what should be done here... I think it's well handled in the way it's done because this cache option is only to start playback, so if try to cache before playback more fragments than the ones provided by the playlist it's because there is something wrong. I added this property mostly for debugging and for people wishing the decrease the latency by reducing the number of fragments to cache, not in the other way :)

> A random question:
> Would it be referred to https://bugzilla.gnome.org/show_bug.cgi?id=600648 ?
I saw it yesterday too, we should ask @slomo

(In reply to comment #44)
> (In reply to comment #28)
> > I have fixed the the typefind patch, which was adding a '\0' for the string
> > comparison.
> > 
> > http://gitorious.org/ylatuya-gstreamer/gst-plugins-base/commit/ff9de9e4d41f544f93df193d05e5de5bb39e5b25
> @Sebastian: Can you review this one so it gets before the next freeze of
> gst-plugins-base?

Ok, that's not good. Is there nothing else to distinguish a m3u8 live streaming playlist from a normal m3u playlist? #EXTM3U is also used in normal (extended) m3u playlists that additionally provide the duration and title information and these playlists are very common.

Comment on attachment 174029 [details] [review]
Add HTTP live streaming source

Marking as obsolete because a newer version is on gitorious

(In reply to comment #47)
> (In reply to comment #44)
> > (In reply to comment #28)
> > > I have fixed the the typefind patch, which was adding a '\0' for the string
> > > comparison.
> > > 
> > > http://gitorious.org/ylatuya-gstreamer/gst-plugins-base/commit/ff9de9e4d41f544f93df193d05e5de5bb39e5b25
> > @Sebastian: Can you review this one so it gets before the next freeze of
> > gst-plugins-base?
> 
> Ok, that's not good. Is there nothing else to distinguish a m3u8 live streaming
> playlist from a normal m3u playlist? #EXTM3U is also used in normal (extended)
> m3u playlists that additionally provide the duration and title information and
> these playlists are very common.

Right, the draft defines new tags[1] so these ones should be checked instead. I would first check that the playlist starts with #EXTM3U as it's done now and than check that #EXT-X-TARGETDURATION (normal playlist) or if #EXT-X-STREAM-INF (for variant playlists) are in the playlist

[1]http://tools.ietf.org/html/draft-pantos-http-live-streaming-01#section-3.1

(In reply to comment #49) 
> Right, the draft defines new tags[1] so these ones should be checked instead. I
> would first check that the playlist starts with #EXTM3U as it's done now and
> than check that #EXT-X-TARGETDURATION (normal playlist) or if #EXT-X-STREAM-INF
> (for variant playlists) are in the playlist
> 
> [1]http://tools.ietf.org/html/draft-pantos-http-live-streaming-01#section-3.1

From shared-mime-info:
    <magic priority="70">
      <match type="string" value="#EXTM3U" offset="0">
        <match type="string" value="#EXT-X-TARGETDURATION" offset="0:128"/>
        <match type="string" value="#EXT-X-STREAM-INF" offset="0:128"/>
      </match>
    </magic>

Only that these new tags are not mandatory and you could have streaming playlists without them. Or did I miss anything?

If there's no reliable way to distinguish between real playlists and this that's going to be a real problem

(In reply to comment #51)
> Only that these new tags are not mandatory and you could have streaming
> playlists without them. Or did I miss anything?
> 
> If there's no reliable way to distinguish between real playlists and this
> that's going to be a real problem

EXT-X-TARGETDURATION is a MUST:
   The EXT-X-TARGETDURATION tag indicates the approximate duration of
   the next media file that will be added to the main presentation.  It
   MUST appear in the Playlist file.

Ah nice, who wants to update the patch? I'll push it then

Some comments about the hls element.

- Call it hlsdemux to be consistent with other elements
- The hls branch contains unrelated changes to vp8enc and the video encoder base class
- It currently crashes for me in gsthlssrc.c:374 because hlssrc is NULL (pipeline is souphttpsrc ! hlssrc ! fakesink)
- The query in line 323 is leaked
- The URI query should IMHO return the URI of the currently played fragment because that's what is visible upstream
- gst_element_make_from_uri() already sets the URI, no need to do it yourself (and if you do you should use the GstURIHandler interface)


Other than that this looks good at a first look. If you fix the name of the element and the crash I'll merge it into gst-plugins-bad :)

Created attachment 184629 [details] [review]
typefind patch for m3u8 playlists

Updated patch

I pushed some untested changes that I'm testing right now, that's probably why it's crashing for you :P

Created attachment 184633 [details] [review]
typefind patch for m3u8 playlists

Fixed typo in previous patch

(In reply to comment #54)
> Some comments about the hls element.
> 
> - Call it hlsdemux to be consistent with other elements
Right
> - The hls branch contains unrelated changes to vp8enc and the video encoder
> base class
I will remove all the unrelated commits from the branch
> - It currently crashes for me in gsthlssrc.c:374 because hlssrc is NULL
> (pipeline is souphttpsrc ! hlssrc ! fakesink)
Fixed
> - The query in line 323 is leaked
Fixed too
> - The URI query should IMHO return the URI of the currently played fragment
> because that's what is visible upstream
> - gst_element_make_from_uri() already sets the URI, no need to do it yourself
> (and if you do you should use the GstURIHandler interface)
Do you mean here that the element shouldn't handle the URI queries?

I have updated my branch in gitorius which is now rebased on top of gst-plugins-bad/master and do not contain unrelated commits anymore.  I have renamed the element to hlsdemux and fixed both the crash and the leak. Still pending the last two points commented by Sebastian regarding the URI.

commit d5e4f8caf88fd60ae99992d4466a217eb73ae896
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Wed Mar 30 03:34:39 2011 +0200

    hlsdemux: fix indentation and docs sections

commit 91196418eaf0cdfe7e07f954aac44ecc47e67bcc
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Tue Mar 29 23:18:24 2011 +0200

    hlsdemux: don't leek the query

commit 77c0971b62c7d691ab7c65583bfed2b8b457fc8e
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Tue Mar 29 23:06:14 2011 +0200

    hlsdemux: check if the task's cond was signaled because it's the end of playlist

commit 42470cd31316528c6a89d6ac33be33c908e91fde
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Sat Mar 12 13:32:57 2011 +0100

    hlsdemux: post a message in the bus when the playlist changes

commit daeed217e53289fe30a02eb6ce8dddac5d85de76
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Sat Mar 12 13:15:52 2011 +0100

    hlsdemux: don't update the playlist if we stay in the same bitrate

commit cd0aeb2403378b93c7051495e9de1613c182af56
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Sat Mar 12 13:00:06 2011 +0100

    hlsdemux: Add support for URI queries

commit 5daae4bb18bb910668958b4bbbab0488fe6c24ef
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Sat Mar 12 12:50:25 2011 +0100

    hlsdemux: Add support for duration queries

commit 18cd90e73f82d94c15767c3ffac2de5b1529c96e
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Sat Mar 12 12:28:42 2011 +0100

    hlsdemux: m3u8: protect public methods properly

commit 09b2b31422d0276b776d26b7dde066f85c3964d8
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Sat Mar 12 12:20:32 2011 +0100

    hlsdemux: m3u8: add support to get the duration from a playlist

commit c599e4a9a13ee354dcd13b2c7f2fe447f42ddc95
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Wed Feb 16 03:51:08 2011 +0100

    hlsdemux: don't print an error if the download was cancelled

commit 4602b42dee03a78b65f00b2321ac6be750d9477d
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Wed Feb 16 03:49:49 2011 +0100

    hlsdemux: make sure the fetcher state change is complete before continuing

commit c580ff26a4d968c9738784e16f6af19307ce1ac1
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Wed Feb 16 01:19:45 2011 +0100

    hlsdemux: don't leak the first buffer

commit ec1d03e1b4e7ee103c9577adda3d8f87d0935039
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Wed Feb 16 00:55:30 2011 +0100

    hlsdemux: clean up code a little bit

commit 1450233a21f62133ace4b8b5c919100c28758458
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Wed Feb 16 00:53:48 2011 +0100

    hlsdemux: only check for the end of playlist when the queue is empty

commit b76526009849f1529254acb1b9307e796af6862f
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Tue Feb 15 22:40:21 2011 +0100

    hlsdemux: make sure to stop fragments cache if something cancelled it

commit fe883740c5819b079cd0877e7d6ad5e107757fa4
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Tue Feb 15 21:55:26 2011 +0100

    hlsdemux: reuse the code in reset() to free resources in dispose()

commit 5a40a62ff4f402f8a15cda1ae1817b26d730c6d0
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Tue Feb 15 21:49:20 2011 +0100

    hlsdemux: use a typefinder to set the caps in the source pad

commit 3fa4b22149ca99fdcce358a876caab10d9051552
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Tue Feb 15 04:39:34 2011 +0100

    hlsdemux: add more comments and document better all the threads involved

commit 8a683fc035ef0d96bc021ccee36b1ac0f98bf147
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Tue Feb 15 03:42:29 2011 +0100

    hlsdemux: handle 404 from the source element

commit cbba7c8bc5a3881048d7af73b73d9fca80fedcf2
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Tue Feb 15 03:41:43 2011 +0100

    hlsdemux: stop the fetcher in the PAUSED_TO_READY transition, not when disposing()

commit b9a0b4ddd420812bac15fad2093c6fc2c6933835
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Tue Feb 15 03:41:01 2011 +0100

    hlsdemux: make sure we don't stop the fetcher twice from different threads

commit 6a0aec0639d7c2382741af9b27743f6d7acfb91c
Author: Andoni Morales Alastruey <ylatuya@gmail.com>
Date:   Tue Feb 15 02:13:56 2011 +0100

    hlsdemux: query the uri upstream before updating the playlist

commit 9aff2de6253419909226416133f24bd05abcf12d
Author: Andoni Morales Alastruey <amorales@flumotion.com>
Date:   Mon Feb 14 18:51:32 2011 +0100

    hlsdemux: Add HTTP live streaming demuxer element
    
    Based on previous work by Marc-André Lureau