GNOME Bugzilla – Bug 108301
source element synchronization (clock)
Last modified: 2004-12-22 21:47:04 UTC
Video and audio (realtime) source elements should syncornize themselves to the GstClock that's leading the element. Let's assume a situation where we want the audio to be master and the video slave (i.e. audio sets the clock, video synchronizes according to it), osssrc (or alsasrc, or ...) should set the clock. Besides that (in case this fails!), it should also test whether the clock was actually set and if not, it should resample internally to make sure that it reaches the requested audio playback rate. v4l*src should drop/insert frames to reach the requested fps (25/PAL or 29.97/NTSC) and should also correct for lost frames. QoS might also be used here (FIXME: how?). Billy (vektor) disagrees here, btw, he thinks video should be the master. Another option here is to just set a different framerate (tell the following elements that we're actually 25.02 fps or so) and not drop/insert frames at all... this might get messy, though, most files 'just' set 25fps and you want to try to reach that framerate yourself too... All of this together means, though, that the output framerate/samplerate of v4l*src/audio*src, the timestamps used on the buffers and the one given in the caps/_convert() functions should match, which currently is not the case.
Created attachment 15312 [details] [review] this simple patch adds clocking support to osssrc. It doesn't check whether it's the master yet.
Created attachment 15316 [details] [review] try #2 - also redoes state changing a bit
Now that audio is done, here's what's to be done for video: <BBB> I make a property use_fixed_fps <BBB> (boolean) <BBB> if TRUE, we try to drop/insert frames to keep as close to the framerate that we're supposed to give out (25fps/29.97fps) and report that framerate on _convert() and _query() <BBB> if FALSE, we just insert frames for lost ones (to prevent sudden glitches) and don't do any further processing (so no framedrops) and report the framerate at which we're actually giving out frames <BBB> in the first case, the timestamp is just num_frames * GST_SECOND / fps <BBB> the fps is fps <BBB> in the second case, this is tricky <BBB> we do gst_clock_get_time() and substract the time when we went from PAUSED->PLAYING (this gets even more tricky when we go from PLAYING->PAUSED and back, but that should be fixable too) <BBB> the fps is num_frames * GST_SECOND / that
Created attachment 15712 [details] [review] Example patch to add clock synchronization to v4lsrc
The above patch doesn't actually look at the need_writes for requeueing the buffer. This means that: *) if the buffer is not used, the buffer is never requeued *) if the buffer is used more than once, it's requeued multiple times To fix this, we need to keep a log of which buffers are used how many times (we can probably us the is_queued[]-array for that).
Created attachment 15864 [details] [review] Fix the above bugs...
Created attachment 15865 [details] [review] same changes for v4lmjpegsrc
Created attachment 15867 [details] [review] same change for v4l2src