GNOME Bugzilla – Bug 778273
AVFVideoSource must post a latency message after it renegotiates caps
Last modified: 2018-11-03 14:04:25 UTC
Background: The AVFVideoSource represents the iphone front- and rear-facing cameras. The cameras can output at various frame-rates and putting a capsfilter downstream will trigger the frame-rate selection during caps negotiation. The frame-rate affects the latency, because for example 30fps leads to a latency of ~33ms. The bug: Adding an avfvideosrc to a pipeline in the NULL state works fine because PLAYING the pipeline will calculate latency after all the caps have been negotiated. However, if you add an avfvideosrc to an already running pipeline, it will renegotiate caps but it will not trigger a latency event after deciding its frame-rate. If the pipeline already had a latency less than 1/frame-rate frames will be dropped. The fix: Post latency message after caps have been set.
Created attachment 345095 [details] [review] Post latency message after setting caps This patch was tested on top of bug 777847 , however it should be independent.
Looks good to me, though, all source will have the same issue. Maybe we could add gst_base_src_set_latency() that does this for us, and implement the LATENCY query generically ?
Yes, having such API in basesrc would definitely be useful. So many sources replicate that already... Let's go with that
One minor wrinkle in this code is that it's essential that 'gst_caps_replace' be called before latency message is posted. That's easy to deal with in this code (call set_latency after gst_caps_replace rather than as its currently done https://github.com/GStreamer/gst-plugins-bad/blob/master/sys/applemedia/avfvideosrc.m#L708) but if your goal is to make this all abstracted away, I don't think your proposal will work.
(In reply to Nick Kallen from comment #4) > One minor wrinkle in this code is that it's essential that > 'gst_caps_replace' be called before latency message is posted. Why is that order important? The latency message is handled asynchronously anyway The gst_base_src_set_latency() API should basically work the same as in e.g. GstVideoDecoder or GstAudioDecoder.
Honestly I have no idea why but I tested the code and I'm pretty certain it didn't fix my bug in any other order.
It all depends if you use a sync bus handler or a async bus handler in your application to handle the latency message. Though, it no timportant, you can just call gst_base_src_set_latency() after your call to gst_caps_replace().
That caps replace thing, this is a potential race between update latency bin function and the caps being replace I guess. We should add some locking to be extra safe. We've been delaying this one a lot, shall we merge it (with maybe some locking around the caps) and revisit later when we have helpers on the base class ?
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/517.