GNOME Bugzilla – Bug 613827
queue: add burst/batch feature to reduce writer wakeups and context switches
Last modified: 2018-11-03 12:13:44 UTC
When I use max-size-buffers and min-threshold-buffers values as parameters to QUEUE in gstreamer pipe line, Its not waiting it is not waiting until it reach min-threshold-buffers. Currently queue fills the buffer as the buffer empty. It is not able to perform the requirement of waiting until it all buffers empty. The changes needed found in gstqueue.c at gstreamer/plugins/elements/ function:gst_queue_chain (GstPad * pad, GstBuffer * buffer) /* don't leak. Instead, wait for space to be available */ do { /* for as long as the queue is filled, ait until min-threshold-buffers reaches. */ GST_QUEUE_WAIT_DEL_CHECK (queue, out_flushing); } while (!gst_queue_is_empty(queue)); and Need changes in gst_queue_is_empty function to satisfy all the conditions. I am not sure all the conditions (need some evaluation in this) static gboolean gst_queue_is_empty (GstQueue * queue) { if (queue->queue->length == 0) return TRUE; if ((queue->cur_level.buffers == 0) || (queue->cur_level.bytes == 0) || (queue->cur_level.time == 0) ) return TRUE; if (( queue->min_threshold.buffers == 0 && queue->cur_level.buffers < queue->max_size.buffers) || (queue->min_threshold.bytes == 0 && queue->cur_level.bytes < queue->max_size.bytes) || (queue->min_threshold.time == 0 && queue->cur_level.time < queue->max_size.time)) && !gst_queue_is_filled (queue) return TRUE; /* It is possible that a max size is reached before all min thresholds are. * Therefore, only consider it empty if it is not filled. */ return ((queue->min_threshold.buffers > 0 && queue->cur_level.buffers < queue->min_threshold.buffers) || (queue->min_threshold.bytes > 0 && queue->cur_level.bytes < queue->min_threshold.bytes) || (queue->min_threshold.time > 0 && queue->cur_level.time < queue->min_threshold.time)) && !gst_queue_is_filled (queue); }
What is the problem? The min-threshold is only used to decide when buffering should restart. It sounds like you want to block the queue from outputting anything when it is below min-threshold. Why would you want to do that, it'll just cause unneeded latency and slowness?
Currently we are optimizing the video and audio stack for po. The current queueing policy is, it fills buffers with data as it empty's. with this we are frequently waking up the hardware video decoder for every frame. because of this it consuming more power with frequent wake ups. Hardware have the ability to decode more frames. and also current processors have the high processing capability to fill queue quickly. once it fills max-size-buffers, we want to wait <or put the hardware in sleep> until all the buffers become empty in queue. I tried different ways to reach this requirement. Then I did some research on the queue. Found queue need some enhancements to satisfy the requirements. What is the problem? The min-threshold is only used to decide when buffering should restart. -- It is true. But this is not performing like that. Buffering should start when the current buffer level goes below < min-threshold-buffers. --once the queue is filled with Max-size-buffers, queue shouldn't buffer until the current buffer level goes below < min-threshold-buffers. -- I am ok. if I dont pass any parameters to queue. no issue with that default behavior. the problem starts when I use max-size-buffers and min-threshold-buffers. Thanks Venkat
Reopening as I think the requested information has been provided.
I see now what this patch tries to achieve. The idea is to not wake up the reader/writer as often by batching input and output before signaling the other end to do a context switch. I would like to see this behaviour implemented with a new boolean property on the object so that we don't break the existing buffering case. I'll see if I can do that.
(In reply to comment #4 Current implementation of queue is ineffective with min-threshold-buffers parameter. This parameter is very helpful to reduce CPU wake ups at various levels i.e. user, kernel driver and platform level. I am requesting you to change the following line of code. This is a bug in the code. no one see this unless they use full features of queue with max-size-buffers and min-threshold-buffers parameters. The changes needed in gstqueue.c at gstreamer/plugins/elements/ function:gst_queue_chain (GstPad * pad, GstBuffer * buffer) /* don't leak. Instead, wait for space to be available */ do { /* for as long as the queue is filled, ait until min-threshold-buffers reaches. */ GST_QUEUE_WAIT_DEL_CHECK (queue, out_flushing); } while (!gst_queue_is_empty(queue)); /*bug in the code*/
This sounds really like a cool feature. I looked on my laptop at powertop and iotop. When I run totem (Ubuntu 10.04) I see lots of interrupts (70 with compared to 20 without running totem) on USB (due to my external USB harddisk) in powertop while iotop shows only 170kb/s reads for a H.264 720x576, 128kb/48khz/stereo stream. Thus this bug could be a starting point for reducing interrupts for IO (for me on USB).
A new threshold could be interesting indeed but fairly useless without more support. For most pipelines you will get more results when you enable large buffers in the audiosink. pulseaudio is capable of taking 2 seconds of audio if we let it. If this enabled, GStreamer would read 2 seconds of data in one burst and then sleep for 2 seconds. This should automatically reduce disk access. The only problem with that now is that we need to know if low latency is needed or not before we can enable the 2 second buffers. Likely the fact that the pipeline is live can give an indication that larger buffers are not wanted (but not always if you have large buffers in the source too). Also most local pipelines don't have a queue after the source.
The buffering in the audiosink would be handled by bug #641072 but a bursty queue would nonetheless be a good idea. Anybody want to make a patch?
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gstreamer/issues/12.