After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 752858 - hlsdemux: Double http request of first fragment in m3u8
hlsdemux: Double http request of first fragment in m3u8
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
Other Linux
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on: 753484
Reported: 2015-07-25 10:13 UTC by Stefano
Modified: 2018-11-03 13:38 UTC
See Also:
GNOME target: ---
GNOME version: ---

gstereamer doulbe http request log (30.75 KB, text/x-log)
2015-07-27 18:00 UTC, Stefano
Playlist retrieved with http before pipleline launch (645 bytes, audio/x-mpegurl)
2015-07-27 18:02 UTC, Stefano
Playlist retrieved with http after pipleline ended with error (645 bytes, audio/x-mpegurl)
2015-07-27 18:03 UTC, Stefano
gstreamer git master log (40.65 KB, text/x-log)
2015-08-03 11:06 UTC, Stefano
GST_DEBUG=6,hlsdemux:9,adaptivedemux:9 log (1.07 MB, application/gzip)
2015-08-06 10:09 UTC, Stefano
mp4_fragments (393.50 KB, application/x-tar)
2015-08-06 13:31 UTC, Stefano

Description Stefano 2015-07-25 10:13:17 UTC
I opened this new ticket from

In the camcorder server streaming implementation I can try to consuming cannot require a fragment in the m3u8 twice because after the first consumption it will be invalidate and it is unavailable to a second one.

Analyzing network traffic and gstreamer log seems that the first fragment it is required twice (the first for negotiation in prerolling and one for consumption in Playing phase).

m3u8 live playlist and gstreamer log are attached in the cited ticket.
Comment 1 Thiago Sousa Santos 2015-07-27 13:27:17 UTC
I tried reproducing it with git master and some online non-live playlists but I couldn't.

Perhaps it is something specific with live playlists or with your particular stream. Did you try git master to check if it wasn't solved already?

The log on the other bug isn't very helpful. Could you attach a new one with GST_DEBUG=hlsdemux:9,adaptivedemux:9 here, please?
Comment 2 Stefano 2015-07-27 18:00:25 UTC
I've attached the new log of:

GST_DEBUG_FILE=gstreamer_m3u8.log GST_DEBUG=hlsdemux:9,adaptivedemux:9  gst-launch-1.0 -v souphttpsrc location="" keep-alive=TRUE ! hlsdemux ! decodebin ! autovideosink

And the two m3u8 states. One before pipeline launch and the other after pipeline stopped.
Comment 3 Stefano 2015-07-27 18:00:59 UTC
Created attachment 308238 [details]
gstereamer doulbe http request log
Comment 4 Stefano 2015-07-27 18:02:29 UTC
Created attachment 308239 [details]
Playlist retrieved with http before pipleline launch
Comment 5 Stefano 2015-07-27 18:03:05 UTC
Created attachment 308240 [details]
Playlist retrieved with http after pipleline ended with error
Comment 6 Stefano 2015-07-31 13:05:44 UTC
Thiago do you need more test or logs?
Comment 7 Thiago Sousa Santos 2015-07-31 18:18:35 UTC
Could you try getting a log with git master, a few things have changed in HLS stream repositioning and it might be fixed already. If not, please submit new logs with master.

Comment 8 Stefano 2015-08-03 11:05:41 UTC
Ok. I've attached the git master log.
Comment 9 Stefano 2015-08-03 11:06:46 UTC
Created attachment 308660 [details]
gstreamer git master log
Comment 10 Stefano 2015-08-05 17:59:36 UTC
Is this enough? I can make some more log and test tomorrow if you need.
Comment 11 Thiago Sousa Santos 2015-08-05 20:49:57 UTC
I found one issue with adaptivedemux with this log but it seems weird that some element downstream returns EOS from a pad push. Is this using playbin or some custom pipeline?

Can you get a full GST_DEBUG=6 log for further investigation?
Comment 12 Stefano 2015-08-05 20:53:12 UTC
The pipeline it is exactly in the comment 2. If you want i can use this GST_DEBUG level to log this pipeline.
Comment 13 Thiago Sousa Santos 2015-08-05 20:54:11 UTC
(In reply to Stefano from comment #12)
> The pipeline it is exactly in the comment 2. If you want i can use this
> GST_DEBUG level to log this pipeline.

Yes, please.
Comment 14 Stefano 2015-08-06 10:09:39 UTC
Created attachment 308843 [details]
GST_DEBUG=6,hlsdemux:9,adaptivedemux:9  log

gstreamer master branch log of pipeline:
GST_DEBUG_FILE=gstreamer_master_m3u8_debug6.log GST_DEBUG=6,hlsdemux:9,adaptivedemux:9  ./gst-launch-1.0 -v souphttpsrc location="" keep-alive=true ! hlsdemux keep-alive=true ! decodebin ! autovideosink
Comment 15 Stefano 2015-08-06 10:11:43 UTC
Ok i've attached with the log level you requirded. I can send other log today or execute other test then I will start a vacation period so if you are available today send me other requests.
Comment 16 Thiago Sousa Santos 2015-08-06 12:45:43 UTC
It seems like the issue is related to the fact that this is HLS serving mp4 fragments but without moof atoms. It is playback of a sequence of moov|mdat chunks which is a new case that hasn't been implemented yet. So qtdemux is not ready to play fragmented stream that is not using the fragmented atoms.

That being said, I'd believe it is not that far from working. The best way for us to implement it would be to get access to a short sequence of those fragments so we could simulate the playback of those without needing access to your hardware. Would it be possible to get a sequence of those fragments and host them somewhere for analysis?

Worst case you could just try:

gst-launch-1.0 -v souphttpsrc location="" keep-alive=true ! hlsdemux keep-alive=true ! filesink location=streamdump.mp4

but the fragments split as the original stream would be better.

Comment 17 Stefano 2015-08-06 12:51:53 UTC
I can download mp4 fragment trough a custom python script if this is enough for you.
Comment 18 Thiago Sousa Santos 2015-08-06 12:54:26 UTC
That would be great, getting a short sequence of fragments would be enough (at least 3).

Thanks again!
Comment 19 Stefano 2015-08-06 13:31:05 UTC
Created attachment 308851 [details]

This are 4 mp4 fragments downloaded from playlist with a custom python script.
Comment 20 Thiago Sousa Santos 2015-08-10 22:13:44 UTC
It helps to have proper timestamping support to play those samples properly, so adding the other bug as a dependency. It is not directly related to the double http requests, though.
Comment 21 Stefano 2015-09-02 14:53:08 UTC
Thiago do you need more tests, logs or fragments? What is causing the double http reqeust? When a fragment was downloaded for negotiation cannot be pushed in the pipeline instead of downloading again?
Comment 22 Thiago Sousa Santos 2015-09-04 19:24:18 UTC
I'm already able to reproduce it, was busy what other stuff in the past weeks but I'm back on fixing those issues. The problem is not fully in hlsdemux, it also needs some fixes in qtdemux.
Comment 23 Stefano 2015-10-09 13:33:25 UTC
Thank you for the update. Now that 1.6 was released do you plan to push something in master that I can test with the camcoder?
Comment 24 Thiago Sousa Santos 2015-11-09 21:29:27 UTC
One additional trickery here is that when starting a fragment, should audio and video start aligned? Because usually the previous fragment won't end with both audio and video having the same end-timestamp.
Comment 25 Stefano 2015-11-11 09:32:43 UTC
Thiago what about your last comment? Do you need some test?
Comment 26 A Ashley 2015-11-20 11:46:29 UTC
The playlists lack a #EXT-X-MEDIA-SEQUENCE tag. Without this, hlsdemux cannot know how to deal with a playlist update. That might explain why hlsdemux is downloading a fragment multiple times.

According to the HLS draft:

6.2.2.  Live Playlists

   The server MAY limit the availability of Media Segments by removing
   Media Segments from the Playlist file (Section 6.2.1).  If Media
   Segments are to be removed, the Playlist file MUST contain an EXT-X
   -MEDIA-SEQUENCE tag.  Its value MUST be incremented by 1 for every
   Media Segment that is removed from the Playlist file
Comment 27 Edward Hervey 2018-05-05 15:49:04 UTC
Is this still an issue with current master ?
Comment 28 GStreamer system administrator 2018-11-03 13:38:48 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: