And the missing branch where the work is beeing done

https://github.com/ylatuya/gst-plugins-bad/tree/base_fragmented

Can you attach patch(es) for this to the bug for easier reviewing, once the other patches have landed and this can be merged? :)

Gathered the required patches in bugs and public repositories and I have dashsink working (for the basic tests I did) with those branches:

http://cgit.freedesktop.org/~thiagoss/gst-plugins-good/log/?h=dashsink
http://cgit.freedesktop.org/~thiagoss/gst-plugins-bad/log/?h=dashsink

The bugs involved are:
good:
https://bugzilla.gnome.org/show_bug.cgi?id=706509
https://bugzilla.gnome.org/show_bug.cgi?id=708221
https://bugzilla.gnome.org/show_bug.cgi?id=660260
https://bugzilla.gnome.org/show_bug.cgi?id=668091
bad:
This one

I'd recommend reviewing in those branches as minor updates have been made to the patches. If you prefer I can go over on the bugs and upload new versions for the patches if needed.

@@ -2144,6 +2181,17 @@ flush:
GstBuffer *buffer;
guint i, total_size;
+ if (GST_BUFFER_FLAG_IS_SET (buf, GST_BUFFER_FLAG_DELTA_UNIT)) {
+ GST_WARNING ("Next fragment will not start with a keyframe");
+ }

Maybe we would post a warning in this case (I mean GST_ELEMENT_WARNING), since it means multiple issues could arrived around this area of the file.

+++ b/gst/isomp4/gstqtmux.c
@@ -104,6 +104,7 @@
#include <gst/audio/audio.h>
#include <gst/video/video.h>
#include <gst/tag/tag.h>
+#include <gst/video/video.h>

you are including video.h twice.

+ {FRAGMENT_METHOD_EVENT, "GstForceKeyUnit events", "event"},

Arguably, the GstForceKeyUnit was really designed to ask a decoder to produce a keyframe. Overriding it for this task have an interesting side effect. Let's say you have a decoder set to produce keyframe only on request. Then there would be no way to have more then 1 keyframe per fragment. By adding a specific event to force a Fragment, we could send GstForceKeyUnit to insert keyframes in the current fragment and later cut the fragment. This may or may not be very useful, but I think it's good to understand the limitation.

One thing I notice is that with time method, we use a simplistic approach which makes the fragment always slightly bigger then then target duration. I don't know if that is a problem or not.

p.s. I'll be posting multiple comment while going through

+Android.mk: Makefile.am

Androgenizer stuff isn't maintain anymore, I would just remove it.

+VOID:STRING,STRING

We use the generic marshaller now in newly introduce code.

Just a suspiciously C++ commented unref in gst-libs/gst/baseadaptive/gstbaseadaptivesink.c:

+ if (pad_data->pad != NULL) {
+ //gst_object_unref (pad_data->pad);
+ pad_data->pad = NULL;
+ }

+ case GST_STATE_CHANGE_PAUSED_TO_READY:
+ gst_base_adaptive_sink_unlock (sink);
+ gst_base_adaptive_sink_stop (sink);

Hard for me to tell, but we you chain and then unlock, does it mean you may deadlock here ?

(In reply to Thiago Sousa Santos from comment #3)
> Gathered the required patches in bugs and public repositories and I have
> dashsink working (for the basic tests I did) with those branches:
> 
> http://cgit.freedesktop.org/~thiagoss/gst-plugins-good/log/?h=dashsink
> http://cgit.freedesktop.org/~thiagoss/gst-plugins-bad/log/?h=dashsink
> 
> The bugs involved are:
> good:
> https://bugzilla.gnome.org/show_bug.cgi?id=706509
> https://bugzilla.gnome.org/show_bug.cgi?id=708221
> https://bugzilla.gnome.org/show_bug.cgi?id=660260
> https://bugzilla.gnome.org/show_bug.cgi?id=668091
> bad:
> This one
> 
> I'd recommend reviewing in those branches as minor updates have been made to
> the patches. If you prefer I can go over on the bugs and upload new versions
> for the patches if needed.

I am tring to reproduce your test buf I failed to use chrome to play mpd file. Any thing I missed? 

My steps are:
==================== Server side ==============================
1. Build and install these 2 packages
http://cgit.freedesktop.org/~thiagoss/gst-plugins-good/log/?h=dashsink
http://cgit.freedesktop.org/~thiagoss/gst-plugins-bad/log/?h=dashsink

2. gst-launch-1.0 videotestsrc is-live=true ! video/x-raw, format=NV12, framerate=30/1 ! queue ! x264enc bitrate=200 ! mp4dashmux fragment-method=2 ! queue ! dashsink name=sink base-url=http://localhost/dash output-directory=/home/wzw/public_html/dash title=sample

3. 
cd /home/wzw/public_html/dash
python -m SimpleHTTPServer 8111 

========= Client(chrome browser + dash.js) side ===============
4. 
open URL: http://127.0.0.1:8111/samples/dash-if-reference-player/
In the "Stream" entry, key in "http://127.0.0.1:8111/sample.mpd"
Click "Load"

5. Nothing displayed, but got these logs from dash.js:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Parsing complete: ( xml2json: 3ms, objectiron: 2ms, total: 0.005s) 
Debug.js:115 Manifest has loaded. 
Debug.js:115 Manifest has been refreshed. 
Debug.js:115 MediaSource is open! 
Debug.js:115 [object Event] 
Debug.js:115 Added 0 inline events 
Debug.js:115 video codec: video/mp4;codecs="undefined" 
Debug.js:115 videoCodec (video/mp4;codecs="undefined") is not supported. 
Debug.js:115 No buffer was created, skipping video data. 
Debug.js:115 No audio data. 
Debug.js:115 No text data. 
Debug.js:115 No fragmentedText data. 
Debug.js:115 No streams to play.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

6. The gstreamer pipline log is so large, so I put it at:
https://drive.google.com/file/d/0BzhQfLFttwrNVHhDWUZpcHNvRVU/view?usp=sharing

12 month ping: We should probably commit this and the patches in the other days one of these days.

(In reply to Jan Schmidt from comment #9)
> 12 month ping: We should probably commit this and the patches in the other
> days one of these days.

IIRC, one remaining issue here is how to properly inform dashsink of bitrates and alternative content.

Should we consider all video/audio inputs as being bitrate alternatives to each other or should we design some mechanism to properly signal that?

Rebased everything again:

http://cgit.freedesktop.org/~thiagoss/gst-plugins-good/log/?h=dashsink
http://cgit.freedesktop.org/~thiagoss/gst-plugins-bad/log/?h=dashsink

Tried to just make things work. Will start doing a proper review of how stuff was implemented. Hopefully we can start merging this soon unless someone has further comments or concerns.

One issue is that dashsink uses discoverer to inspect the fragments it gets from mp4dashmux. It would be easier to just forward the container information to it. Can the new streams API help here? dashsink somehow needs to know the formats inside the container to properly create the mpd file.

Please, merge this plugin to the gst-plugins-bad. It's very helpful to support fragmented MP4 for HLS.
As described in https://developer.apple.com/services-account/download?path=/Documentation/HTTP_Live_Streaming_Internet__Draft_Preview/HLSdraftpantospreview.pdf HLS in macOS v10.12 or later, iOS 10 or later, and tvOS 10 or later supports fMP4 segments, as they are used by MPEG-DASH as well.

Dashsink looks awesome! I'm running into problems using it though, it generates an MPD and segments but the MPD doesn't play in the DashIF reference player (version 2.4.1), but gives the error: 'Cannot play media. No decoders for requested formats: video/mp4;codecs="undefined"'

Here's my Python streaming script: https://gist.github.com/hunternet93/d3dc2ed304c559f51836e06c77620f3f It creates three pipelines like "videotestsrc ! x264enc ! h264parse ! mp4dashmux" that are each linked to a single dashsink. Here's the generated MPD: https://video.crosscommunity.com/dashtest/GStreamer%20Adaptive%20Streaming.mpd

It'd be awesome to see this merged into GStreamer, DASH is a cool streaming tech but there's not much FOSS that can generate it.

I have placed a $200 USD bounty on this issue: https://www.bountysource.com/issues/5503093-dashsink-new-sink-for-dash

I think I've found the source of the issue, dashsink doesn't set the required codecs property of the <Representation> tag in the MPD. Here's the RFC on the codec property: https://tools.ietf.org/html/rfc6381 And a more human-readable description: https://stackoverflow.com/questions/16363167/html5-video-tag-codecs-attribute

Generating the correct codec string for incoming media seems to be a fairly simple job for someone familiar with the inner workings of GStreamer, if someone could code it up I'd be very grateful.

Created attachment 351575 [details] [review]
Dashsink patch adding partial, untested support for the codecs property

This needs a lot of work, I am not a C programmer so I probably missed a lot. Also, still needs proper audio codec detection.

After using the above patch, the dash.js reference player attempts to play the stream but dies with a media player error. It appears that the MP4 segments are getting written incorrectly, they don't play in any player I've tested.

Here's a sample MPD: https://video.crosscommunity.com/dashtest/GStreamer%20Adaptive%20Streaming.mpd 

Produced with the pipeline:

    videotestsrc ! x264enc ! h264parse ! mp4dashmux fragment-method=1 fragment-duration=2 ! dashsink fragment-duration=2

When played in dash.js on Firefox, resulting in the error "Video Element Error: MEDIA_ERR_DECODE". When played using playbin, it fails to play after the following warnings:

    0:00:00.698391401 15039 0x7f6474005540 WARN                 qtdemux qtdemux.c:6512:gst_qtdemux_process_adapter:<qtdemux0> Unknown fourcc while parsing header : styp
    0:00:00.828871236 15039 0x7f6474005540 WARN                 qtdemux qtdemux.c:6315:gst_qtdemux_process_adapter:<qtdemux0> error: This file is invalid and cannot be played.
    0:00:00.828909916 15039 0x7f6474005540 WARN                 qtdemux qtdemux.c:6315:gst_qtdemux_process_adapter:<qtdemux0> error: atom ...Q has bogus size 904110311

My best guess is that mp4dashmux isn't working correctly, but I'm way out of my depth here so I have no idea what's wrong.

(In reply to Isaac Smith from comment #18)
> After using the above patch, the dash.js reference player attempts to play
> the stream but dies with a media player error. It appears that the MP4
> segments are getting written incorrectly, they don't play in any player I've
> tested.
> 
> Here's a sample MPD:
> https://video.crosscommunity.com/dashtest/GStreamer%20Adaptive%20Streaming.
> mpd 
> 
> Produced with the pipeline:
> 
>     videotestsrc ! x264enc ! h264parse ! mp4dashmux fragment-method=1
> fragment-duration=2 ! dashsink fragment-duration=2
> 
> When played in dash.js on Firefox, resulting in the error "Video Element
> Error: MEDIA_ERR_DECODE". When played using playbin, it fails to play after
> the following warnings:
> 
>     0:00:00.698391401 15039 0x7f6474005540 WARN                 qtdemux
> qtdemux.c:6512:gst_qtdemux_process_adapter:<qtdemux0> Unknown fourcc while
> parsing header : styp
>     0:00:00.828871236 15039 0x7f6474005540 WARN                 qtdemux
> qtdemux.c:6315:gst_qtdemux_process_adapter:<qtdemux0> error: This file is
> invalid and cannot be played.
>     0:00:00.828909916 15039 0x7f6474005540 WARN                 qtdemux
> qtdemux.c:6315:gst_qtdemux_process_adapter:<qtdemux0> error: atom ...Q has
> bogus size 904110311
> 
> My best guess is that mp4dashmux isn't working correctly, but I'm way out of
> my depth here so I have no idea what's wrong.

I also have a problem corresponding to the absence of 'codecs' property. I have tested similar pipeline (but without 'h264parse' before mp4dashdemux) in Chrome, Firefox and Epiphany, and only Epiphany was able to play my stream (with dash.js library) without 'codecs' property in manifest file.

But I don't have a problem (MEDIA_ERR_DECODE), described in your last message. My guess is that 'h264parse' is not necessary in your pipeline, but I am not sure.

(In reply to Alex from comment #19)
> (In reply to Isaac Smith from comment #18)
> > After using the above patch, the dash.js reference player attempts to play
> > the stream but dies with a media player error. It appears that the MP4
> > segments are getting written incorrectly, they don't play in any player I've
> > tested.
> > 
> > Here's a sample MPD:
> > https://video.crosscommunity.com/dashtest/GStreamer%20Adaptive%20Streaming.
> > mpd 
> > 
> > Produced with the pipeline:
> > 
> >     videotestsrc ! x264enc ! h264parse ! mp4dashmux fragment-method=1
> > fragment-duration=2 ! dashsink fragment-duration=2
> > 
> > When played in dash.js on Firefox, resulting in the error "Video Element
> > Error: MEDIA_ERR_DECODE". When played using playbin, it fails to play after
> > the following warnings:
> > 
> >     0:00:00.698391401 15039 0x7f6474005540 WARN                 qtdemux
> > qtdemux.c:6512:gst_qtdemux_process_adapter:<qtdemux0> Unknown fourcc while
> > parsing header : styp
> >     0:00:00.828871236 15039 0x7f6474005540 WARN                 qtdemux
> > qtdemux.c:6315:gst_qtdemux_process_adapter:<qtdemux0> error: This file is
> > invalid and cannot be played.
> >     0:00:00.828909916 15039 0x7f6474005540 WARN                 qtdemux
> > qtdemux.c:6315:gst_qtdemux_process_adapter:<qtdemux0> error: atom ...Q has
> > bogus size 904110311
> > 
> > My best guess is that mp4dashmux isn't working correctly, but I'm way out of
> > my depth here so I have no idea what's wrong.
> 
> I also have a problem corresponding to the absence of 'codecs' property. I
> have tested similar pipeline (but without 'h264parse' before mp4dashdemux)
> in Chrome, Firefox and Epiphany, and only Epiphany was able to play my
> stream (with dash.js library) without 'codecs' property in manifest file.
> 
> But I don't have a problem (MEDIA_ERR_DECODE), described in your last
> message. My guess is that 'h264parse' is not necessary in your pipeline, but
> I am not sure.

I only get the MEDIA_ERR_DECODE message after fixing the codecs property issue using the patch I posted earlier.

Anyone rebased this on top of master recently?

What's the status here? Is anybody going to take it forward or should we close it?

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/59.