GNOME Bugzilla – Bug 722900
multiqueue: Add signal to avoid deadlocks
Last modified: 2014-11-24 18:49:14 UTC
When multiqueue is used in a pipeline with an element with collect pads after the queue, there is a risk for deadlocks with badly muxed content, or content with missing frames in one stream. This happens when one queue is filled to the byte or time limit so that it can't be grown and therefor blocking a write, and at the same time an other queue is empty. The collect pads are then waiting for for data from the empty queue before reading anymore from the filled queue, but that will never arrive since the writing thread is blocking on the full queue. The result is a deadlock. I have added a new signal hard_overrun that is sent in the case where one queue is filled to the hard byte or time limit and another is empty.
Created attachment 267125 [details] [review] Suggested change
*** Bug 722901 has been marked as a duplicate of this bug. ***
This is related to bug #719893 btw, maybe that one can be closed?
Note that this can not (reliably) be used to prevent deadlocks. It's as useless as the overrun signal for that, both can be emitted although a deadlock never occurs.
The new one is emitted when time/byte limit is reached but completely ignores the buffer limit, right? So it's basically the same as the existing signal except for the buffer limit? (But nonetheless, even if the time/byte limit is reached, the queue can still work properly and not deadlock. It can also be reached multiple times without problems. It's all a matter of timing of the different threads, and what is happening upstream of multiqueue)
Yes, bug 719893 can be closed.
The difference is that the new signal is sent when it is no longer possible to increase the size of one of the single queues, which would otherwise be done when one other single queue is empty. You are completely right that if this situation would cause a deadlock or not depends on what is happening both upstream and downstream of the queue. However there are cases where we know that this state generates a deadlock, and we need a way to detect this state.
(In reply to comment #7) > The difference is that the new signal is sent when it is no longer possible to > increase the size of one of the single queues, which would otherwise be done > when one other single queue is empty. I don't think that this is generally true. I can imagine a (rather artificial and unlikely) situation where you get the new signal, but afterwards a singlequeue could grow.
What's left to be done here? After the discussion on IRC with Wim and others it should be clear that this is not solving anything except for a very specific use case and does not make much sense with the concepts of multiqueue... and multiqueue has to work in the generic case.