GNOME Bugzilla – Bug 559250
segments need identifiers.
Last modified: 2008-11-10 17:08:29 UTC
Let's imagine you have a pipeline on which you have performed a segment seek. This pipeline has a queue in it after the demuxer, so the segment-done message will be normally posted two seconds before the time the last video frame hits the sinks. Now, in your mind, start scrubbing this pipeline, performing flushing seeks. If you seek to within the last two seconds of the segment, corresponding to the length of the queue, a flushing seek will cause a segment-done message to be emitted, asynchronously, as the demuxer fills up the queue but reaches the end of the segment. The problem comes in on the application side: it gets segment-done messages, but does not know if they correspond to the *current* segment set by the last flushing seek, or if it is just receiving stale segment-done messages from the previous segment. This is especially pernicious in the normal case in which you want to react to segment-done messages, for example by performing a nonflushing rewind on the pipeline. You might rewind in response to a stale segment-done message, when in reality that message was already "invalidated" by your subsequent flushing seek. The solution to this is simple: have identifiers associated with segments, and pass them along through the events and messages that deal with segments: the SEEK and EOS events, and the SEGMENT_START, SEGMENT_DONE, and EOS messages. The application can store the last segment identifier associated with the last seek that it performed, and react only to segment-done messages corresponding to the current segment. I am not sure if GstBin can handle this for us directly; the synchronization between the seeking thread, the streaming thread, and the thread handling the messages (normally the seeking thread) is not particularly clear to me. It would be ideal if the application only got non-stale segment-done messages, but given the asynchronous queue in the top-level bus, it seems that a seek would have to filter the queue of unpopped messages -- sounds a bit nasty. For that reason my current proposal is to handle this in application logic instead, although suggestions for GStreamer handling this would be ideal. I'm filing the rationale first, and now will work on a patch. Comments most appreciated.
I have previously conceived something that sounds really close to what you've just described, but never gotten to actually doing anything about it. I foresee some complexity in dealing with the reality of GstSegments in any useful pipeline - there's likely to be moments where multiple segments are active, and in some pipelines there'll *always* be multiple segments active, in different formats (BYTES -> TIME through a demuxer or parser for e.g.). Creating a unified segment identification across those sorts of boundaries probably requires the cooperation of the elements involved. gnonlin is another use case to examine, where there are bins sending their own seeks and changing segments about. I'm not sure about the "It would be ideal if the application only got non-stale segment-done messages" statement. I think uniformity and predictability of the expected messages is valuable, without worry about races caused by thread interactions - one guy sending a seek just after an element queues segment-done onto the bus for example. If there's a unique way to identify which segment-done message you're receiving, then it doesn't matter if you get 'stale' ones.
2008-11-04 Andy Wingo <wingo@pobox.com> Add sequence numbers to events and messages. See #559250. * gst/gstutils.c (gst_util_seqnum_next, gst_util_seqnum_compare): New functions. * gst/gstevent.h: * gst/gstevent.c (_gst_event_copy, gst_event_new): Initialize new events with a new sequence number, and copy it when copying. (gst_event_get_seqnum, gst_event_set_seqnum): Accessors for an event's sequence number. * gst/gstmessage.h: * gst/gstmessage.c (_gst_message_copy, gst_message_new_custom): (gst_event_get_seqnum, gst_event_set_seqnum): As with events, so with messages. * docs/gst/gstreamer-sections.txt: Add new functions to the docs. Now we just have to make elements Do The Right Thing.