Created attachment 122850 [details] [review]
Patch that creates the resettime element

This patch creates the resettime plugin and element in the gst subdirectory.
Note: it is based on gst-plugins-bad 0.10.5, so the patch to configure.ac will probably not apply cleanly to CVS head.
Note 2: apply with patch -p3

The usual way for making a sink lock on to the current running_time is to set it to playing with a custom base_time.

(In reply to comment #2)
> The usual way for making a sink lock on to the current running_time is to set
> it to playing with a custom base_time.

I may be mistaken, but doesn't that only work for sources?  The use case is something like:

Initial pipeline:

videotestsrc ! tee name=t ! queue ! matroskamux ! filesink location=...

After dynamic reconfig:

videotestsrc ! tee name=t ! queue ! matroskamux ! filesink location=... \
                        t ! queue ! matroskamux ! filesink location=...

Both files start at a different absolute time, but both should start at 0 running time.  So setting the source's base_time is not an option.

In this situation, you need something that changes timestamps in the middle of the pipeline.  identity actually does that, but you have to supply it with the new base_time - and for that you need to probe pad buffers to find out the first buffer's timestamp.  Writing a new plugin was much simpler.  I could have added it to identity, of course, but my feeling is that identity is already way overloaded.

I must admit, this element probably fits more in gentrans (which already contains most of identity's functionalities as separate elements).  If you prefer I can add it there.

what's the situation with this problem?
it's also a problem for us. what's the recommended solution?
thanks.

comment #3, you are right, we don't have a way for elements to figure out when they were added to the stream and to adjust things. Not sure how that would work or if the muxer should simply be able to cope with that (taking the first ts it sees as the offset or something)

Doing it in the muxer sounds like the right thing to do... That's more or less what happens in sinks, isn't it?  I seem to remember seeing something like that in basesink - or is that just for the clock provider?

Also, for muxers, I'm not sure how this affects AV synchronisation. For instance, you could get a video buffer, reset the timestamp, and later on receive an audio buffer with an earlier timestamp...  But perhaps that is solved with collectpads.  Anyway, the resettime plugin is not going to solve this, unless it becomes something like multiqueue.

Just subtracting the base_time is not enough if you have a running pipeline like:
videosrc ! tee ! xvimagesink
               ! videoenc ! muxer ! filesink
an dyou allow to pause and continue the recording part. One needs to accumulate timestamps when its paused and subtract this value. Maybe muxers need a property wheter they should mux sparse streams or produce continous streams.
It mostly matters for container wheter the data streams are timestamped, e.g. mp4.

Is it still useful to add this plugin or should another bug for the real problem be opened?

Slomo, I don't think it can be 'fixed'. The core and the plugins don't really know what the application want to do. We would need an interface on muxers that offers a 'sparse-stream' property. Setting it to FALSE would cause the muxer to rewrite timestamps to compensate for disconts.

So what do you suggest how to continue with this bug? Make a new one for the sparse-stream interface for muxers?

I vote for making a new one with the sparse-stream interface.  I ended up discarding the resettime plugin for my own application anyway, and instead combining it with other timestamping hacks at the application level.

So is anybody creating the new bug report as per comment #10?

Tobias, lets leave this for now as it is. Eventualy someone will need the feature enough to file the bug :)

Stefan, fair enough :) All I'm caring about is the status of *this* bug though ;) I just wonder why this bug is still open and why it's set to NEEDINFO :-)

Reopening since we want the bug to be left for now. There's still a unrejected/unaccepted patch in this bug :-)

Since I've seen questions about something similar several times on the list, and since the fundamental solution (fixing the muxers) seems a long way off, I vote for including this plugin.  This also gives more time to experiment with the solution before fixing the muxers.

If somebody reviews the patch I'm willing to improve the code.

Is there a reason why there is no base GstMuxer class ?

Based on the numbers of related email threads, its seems like a very common use case. Cheese, SLTV, Artista and other streaming/recording needing projects would greatly benefit from this.

Extending the resettime plugin in a similar way of multiqueue (for multiple streams handling) seems a good intermediate option to me...

> Is there a reason why there is no base GstMuxer class ?

Yes, because it's not an easy thing to get right. There's a bug for it somewhere though ;)

I would like to see this element in -bad, it is useful for every recording system,

thanks
Nicola

Nicola, and I still think someone needs to make a small testapp and try to achieve the same thing with a newsegment event.

If that works we need to figure how to tell tee/output-selector elements that they should send such (right now tee/output-selector don't send newsegments at all, but they probably should). Don't know yet wheter we should add a flag to the elements to select wheter newsegments should start from zero or should be insync with the other pads. Restart from zero might not make sense for tee.

I cannot post full code but i do something similar to what is requested in post number 3.

I have this function that inserts a new segment when it is not the first request pad in a tee

            def insert_new_segment_cb(pad, buffer, data):
                # Wait for keyframe
                if buffer.flag_is_set(gst.BUFFER_FLAG_DELTA_UNIT):
                    return False
                
                pad.remove_buffer_probe(data['id'])
                
                try:
                    position, _ = pipeline.query_position(gst.FORMAT_TIME)
                except gst.QueryError:
                    return True
                
                if position != -1:
                    event = gst.event_new_new_segment(False, 1.0, gst.FORMAT_TIME, position, -1, 0)
                    pad.get_peer().send_event(event)
                
                return True

And before I have a custom element that resets timestamps to 0 and inserts metadata in the stream if it is necessary. The metadata is stored in a custom source.

Very hackish, but it works.

Would be great to have this implemented officially in GStreamer.

FWIW, I don't see how segments would (presently) help in the typical recording/muxing scenario given above, since (afaik) there is practically no muxer that considers segments.  If they were fixed/modified to consider running time (or so), and maybe there is also some element(s) in the recording pipeline that will drop out-of-segment stuff (comparable to basesink behaviour), then newsegment event would also serve to "reset" time, otherwise ...

Also, until then I don't mind having this in -bad, although any particular application/context may well need its own minor adaptation of it to serve just right.

(In reply to comment #22)
> Also, until then I don't mind having this in -bad, although any particular
> application/context may well need its own minor adaptation of it to serve just
> right.

I can confirm that, since I ended up not using the element myself :-)

Over private mail Nicola and I decided that what is really needed is something that facilitates refreshing a pipeline tail, i.e. a muxer-sink combination.  Such a scheme would have to combine the timestamp manipulation with pad blocking, reopening files or connections, state changes, and being able to safely terminate the pipeline in the middle of a modification.  It's something that is very error-prone and that many applications have to re-invent.

Perhaps this is something that could be discussed in a BoF at the GStreamer Conference?

Is this still needed? In 0.11 this will probably be solved by the new segment mechanisms and a way to specify offsets anyway

I'm not familiar enough with the new segment mechanisms to decide if that is sufficient...

For me, it's no problem if this feature gets dropped.  If there really is a need for it someone will surely reinvent it.

Well, you will be able to set an offset on pads to adjust the running time of buffers :) I'll close this then