GNOME Bugzilla – Bug 108268
[PATCH] -Add min threshold to queue for network streaming apps
Last modified: 2004-12-22 21:47:04 UTC
I'm building an MPEG-2 transport stream viewer process, which receives input from a UDP source and displays the resulting video on an Xv-capable X server. I have a plugin which extracts a PID from the MPEG-2 TS and decodes TS and PES headers, and am creating a pipeline equivalent to: gst-launch udpsrc port=<port> ! queue max-level=<M> leaky=2 min-threshold-bytes=<N> ! { mytsdecoder pid=<pid> ! mpeg2dec ! xvideosink xid=<window_id> } The "min-threshold-bytes" property is an enhancement to the standard GStreamer queue. I've added this so that, when the queue is emptied, the reader thread (the "downstream" side of the queue) is not awakened by a "not_empty" signal until the queue has refilled to at least this level. Without this property, the queue level bounces along near the bottom, rather than filling to a reasonable level, causing freeze frames and/or a "jerky" display: Before: After: +--------- block reader thread | +-- signal reader thread | | / v v /| / min_threshold_bytes - - - - -/- /|- - - -/- | - - - - - - /| / | / / | / __ / |/ | /| / |/| / etc. / |/ | / |/ / / ^ / / |/ __/|__/|___/|__ _/|_/ A similar threshold could be added easily to block the "upstream" writer thread until a full queue drops below a threshold, rather than waking it as soon as anything is removed, as at present. However, in most network streaming applications it is probably sufficient to allow the queue to leak on the downstream side, since blocking the incoming network traffic is not possible. Since max_level is specified in buffers, but min_threshold_bytes is in bytes, the queue must also signal the reader thread if the maximum number of (presumably smaller than expected) buffers has been enqueued, but the threshold has not yet been reached. The attached patch adds a min_threshold_bytes property to gstqueue.{h,c}. Note that it already includes the change from Bugzilla #108236 (a fix to a rather unlikely race condition in gst_queue_chain()), and updates a number of debugging statements to show the queue level in both buffers and bytes. Please review, and feel free to add to the next GStreamer release! Cheers...
Created attachment 14985 [details] [review] Adds min-threshold-bytes property to gstqueue
Created attachment 15175 [details] [review] Updated patch to add min-threshold-bytes property to gstqueue
The updated patch does not include the (incorrect) patch mentioned in #108263. Please pardon the mixup!
Assigning to Ronald for review
This is ABI breakage since gstqueue.h is a public header file. Re-targetting to 0.7.x.
I've been running default apps with this for a while now, and I'm not really noticing any difference. This is good, because it means nothing breaks, but can you give some examples of where this has clear advantages over the old approach? Network streams? Something else? I'm basically in for applying this, unless someone objects.
It's for streaming video/audio applications. If your input source is a file, DVD, etc., then it's true that the minimum threshold doesn't make a bit of difference, since the input source can fill the queue much faster than the data is needed. However, it's very useful in situations in which the input is being received at a more or less constant bit rate, eg. over a network via UDP or RTP. In such cases it is necessary to allow the queue to fill to a minimum level before beginning playout, for two reasons: First, data will never be added to the queue at a perfectly constant rate, due to blocking (eg. multiple MPEG-2 transport stream packets in one UDP or RTP PDU), network jitter, scheduling latency on multitasking OSs, and so on. Second, data will not be removed from the queue at a constant rate either. For example, when a video pipeline first begins to play, the gstmpeg2dec decoder (based on libmpeg2) will remove a great deal of data very quickly, throwing away everything until it finds the next MPEG-2 sequence header. Even after that, the rate at which data is removed will still be quite bursty, since I-frames tend to be large while P- and B-frames are much smaller. Therefore, the application must try to keep the queue filled to a reasonable level. If the queue ever empties completely, there will be unexpected delays while it refills, resulting in jerky video playback or in audio pops or dropouts. (See the flashy ASCII art in the initial Bugzilla report.) The min-threshold-bytes attribute helps with this, since it allows the queue to block the reader thread until a reasonable amount of data has been received. It's only one part of the solution: the queue can still empty if the input rate and playout rate are different, for instance, as will happen if there are differences in clock speed between the encoder and the (GStreamer) decoder. This needs to be addressed by the application, eg. using watermark-crossing signals from the queue to "tune" the mapping between MPEG-2 PCR values and the system time. But that's another story...
OK, applied to CVS. Thanks for all this!