GNOME Bugzilla – Bug 675761
videoencoder: Support latency adjustment with dynamic framerate
Last modified: 2018-11-03 11:21:28 UTC
Currently, live src like v4l2src (many uvc cameras) dynamically change (reduce) the framerate to compensate for lower ambiant light. In a live pipeline, this has the effect of increasing the effective latency. This is because most encoders have a latency in frames, but exposes that latency in time. When the framerate dynamically changes, the latency is not increased. This can be solve in the base class by monitoring the effective latency time (time duration of the buffered frames). This will not prevent some of the buffer to be late and dropped, but will at least allow the pipeline to exit the recovery (1 fps) mode. Some people have thought that this could be solved using videorate, but videorate does not operate in live mode. It will only produce frames when the next one arrive. videorate could expose it's latency to compensate, but duplicating frames is not very efficient.
Created attachment 213743 [details] [review] videoencoder: Remove done ToDo
Created attachment 213744 [details] [review] videoencoder: Remove trailing white space
Created attachment 213745 [details] [review] videoencoder: Port to GQueue
Created attachment 213746 [details] [review] videoencoder: Documentation fix
Created attachment 213747 [details] [review] videoencoder: Add support for dynamic framerate and latency Most encoder know their latency in frames, but set it in time. This time will vary with the framerate. This code will increase the latency if the framerate is detected to be slower then expected, allowing the sink to exit recovery mode.
Created attachment 213748 [details] x264enc: Don't set latency while holding object lock This reverts commit 30a0b50e9ca0d625e61f994d4f8acd022dcddf38.
Comment on attachment 213748 [details] x264enc: Don't set latency while holding object lock Sorry, wrong bug.
Can also be seen at: http://cgit.collabora.com/git/user/nicolas/gst-plugins-base.git/log/?h=675761-latency-and-dynamic-framerate
Any comments about this patch ?
Definitely sounds like something we'd want for 1.x also. Note that videorate *should* also have a live mode, by which it uses timeouts/deadline to push again the current frame if it still hasn't got a new one before the configured latency (which could default to be 1 or 2 output frame durations).
Nicolas, is this still valid ? Can you rebase your patches against master if so.
I never forgot about this one, but also never got the time to revisit. I will try and rebase. The idea works, but I'm not sure it scales very well (e.g. if there is multiple encoders). A second opinion would be nice. It would also solve the fact the with 0/1 framerate none of the element reports latency, so live dynamic framerate does not work.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/66.