GNOME Bugzilla – Bug 764635
splitmuxsink: deadlock when reference stream has low framerate
Last modified: 2018-11-03 15:08:34 UTC
I'm using splitmuxsink with matroskamux and filesink. I've got 3 input streams: - video: 8 jpeg frames per sec, so all buffers are keyframes - audio: normal mp4a - subtitle: 1 buffer every second 1) During the 1s it waits for a subtitle buffer, check_queue_length() allows audio/video queues to grow because the subtitle queued_bufs is empty, see the /* If another stream is starving, grow */ case. The 7th GOP arrives at t=875ms. 2) At t=900ms (so between 2 GOPs), the subtitle buf arrives, it's blocked in handle_mq_input() waiting for the next GOP to complete, because max_in_running_time=875ms. 3) At t=1000ms the 8th GOP arrives, max_in_running_time is set to 1000ms. 4) The subtitle buffer enters the queue but cannot go out yet because max_out_running_time is still 875ms because audio/subtitle streams didn't catchup to 1000ms yet. 5) Since the subtitle queued_bufs is not empty anymore, the audio queue is not allowed to grow anymore. So next audio buffer needed to catchup to 1000ms are blocked because its queue is full.
I guess a "normal" subtitle stream knows in advance when is the next subtitle, so just after a subtitle buffer you immediately get a GAP event with a duration up to the next one. In my case I cannot know in advance when the next subtitle buffer will arrive, so if after 1s I had nothing, I generate a GAP event for that period. That means that there are periods of 1s where matroskamux can legitimately wait for something on the subtitle stream and thus audio/video must be queued until then. In the "normal" usage it knows when to expect next subtitle and will consume audio/video during that period, so queues won't grow that much.
Created attachment 325438 [details] [review] splitmuxsink: Fix deadlock when queues get full too early
Review of attachment 325438 [details] [review]: Works for me.
Wait, no, that's not doing what I though. Didn't realize it sets "max-size-buffers" on the whole mq and not on individual sq. I don't think we can set different size limit on individual queues, can we? In my case the video (reference) stream have much lower buffer rate than the audio one. That means it will allow all queues to grow to make room for audio buffers, effectively allowing infinite expand.
Created attachment 325525 [details] [review] splitmuxsink: Remove mq buffer limits Limiting buffers in the mq is leading to deadlocks because queues could become full before some streams gets ready. It can happen because events are not counted into the queue size, or when it needs many video GOPs before reaching the first subtitle buffer. It was not working anyway in the case some streams have more frequent buffers than the reference stream. This happens for example with low rate mjpeg video stream (e.g. 8jpegs/s) where every buffer is a keyframe. It's going to have much more audio buffers and the queue will grow infinitely. TODO: Implement a limit logic in handle_mq_input() to avoid growing the queue infinitely when the src is faster than the sink.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/265.