GNOME Bugzilla – Bug 739996
videomixer: Drops a lot of frames, if one of the sources is live
Last modified: 2014-11-20 10:23:09 UTC
sample pipelines (from videomixer help): GST_DEBUG=videomixer:6 \ gst-launch-1.0 \ videotestsrc pattern=1 is-live=true ! \ video/x-raw,format=AYUV,framerate=\(fraction\)10/1,width=100,height=100 ! \ videobox border-alpha=0 top=-70 bottom=-70 right=-220 ! \ videomixer name=mix sink_0::alpha=0.7 sink_1::alpha=0.5 ! \ videoconvert ! xvimagesink \ videotestsrc ! \ video/x-raw,format=AYUV,framerate=\(fraction\)5/1,width=320,height=240 ! mix. Note that the property is-live=true part of log: videomixer2.c:987:gst_videomixer2_fill_queues:<mix:sink_0> Taking new buffer with start time 0:00:07.402963486 videomixer2.c:987:gst_videomixer2_fill_queues:<mix:sink_1> Taking new buffer with start time 0:00:07.400000000 videomixer2.c:1195:gst_videomixer2_do_qos:<mix> qostime 0:00:07.400000000, earliest 0:00:07.408122570 videomixer2.c:1199:gst_videomixer2_do_qos:<mix> we are late, drop frame STOP PIPELINE (show stats) videomixer2.c:2031:gst_videomixer2_change_state:<mix> stopping collectpads videomixer2.c:2044:gst_videomixer2_change_state:<mix> qos: rendered 39, dropped 37 If simply remove the check jitter from videomixer2.c (gst_videomixer2_do_qos), then it will work, and frames will not be dropped by the xvimagesink, because default max lateness value for videosink is 20ms
It can be even easier to leave only one living source: GST_DEBUG=videomixer:6 \ gst-launch-1.0 videotestsrc pattern=1 is-live=true ! \ video/x-raw,format=AYUV ! \ videomixer ! videoconvert ! xvimagesink Ratio are rendered to dropped = 1/1
This is also still a problem with compositor from git master.
Why can not weaken the condition check jitter or remove it?
always jitter > 0 for live sources and each second frame is drawn from the fact that the previous frame was dropped
I used to DirectShow on Windows, and decided to move to open source software, so cool to see the source code ;). But how many bugs here that do not give fulfill my task :( Excuse me for my complaints.
Created attachment 290508 [details] pipeline schemas Short term correction: for normal source: earliest_time = qos_timestamp + 2 * qos_diff + frame_duration; For live source you need to make a correction. Shift by one frame (see schema, element inter) earliest_time = qos_timestamp + 2 * qos_diff + 2*frame_duration; qos_diff always > 0 I can a little bit wrong, but this formula is more accurate for live sources
Or perhaps it's just bug #640610 ?
comment 6: earliest_time_live_source = qos_timestamp + 2 * qos_diff
Created attachment 290578 [details] [review] videoaggregator: fix up qos handling for live sources The other option is to put the frame dropping under a !live flag.
comment 9. Agreed. Why is this not in the repository?
Review of attachment 290578 [details] [review]: Maybe we should instead just disable QoS for live pipelines. Or only use this formula with the 2*diff if the pipeline is not live.
Created attachment 290828 [details] [review] aggregator: agg _get_latency() for subclass usage
Created attachment 290829 [details] [review] videoaggregator: fix up qos handling for live sources
Review of attachment 290828 [details] [review]: That looks good to me.
Review of attachment 290829 [details] [review]: Sounds right.
Typo in commit message: "adative" :)
commit 051955e5fac2b124bc50655c811bb13794b6d278 Author: Matthew Waters <matthew@centricular.com> Date: Mon Nov 17 14:00:10 2014 +1100 aggregator: add _get_latency() for subclass usage API: gst_aggregator_get_latency https://bugzilla.gnome.org/show_bug.cgi?id=739996 commit f4a10c64c872dbe1d4ed2d5d3b730704dda43f87 Author: Matthew Waters <matthew@centricular.com> Date: Mon Nov 17 14:05:01 2014 +1100 videoaggregator: fix up QoS handling for live sources Only attempt adaptive drop when we are not live https://bugzilla.gnome.org/show_bug.cgi?id=739996
commit 0053ad08471e14bed592fe125f1b2999d23c00cd Author: Matthew Waters <matthew@centricular.com> Date: Mon Nov 17 23:16:03 2014 +1100 videomixer: fix up QoS handling for live sources Only attempt adaptive drop when we are not live https://bugzilla.gnome.org/show_bug.cgi?id=739996