GNOME Bugzilla – Bug 748337
ksvideosrc: dropping frames
Last modified: 2015-11-01 09:17:33 UTC
Created attachment 302185 [details] Patch provided by Sebastian Dröge Apparently ksvideosrc is dropping 2 frames out of 3 Sebastian provided a patch (attached) that removes some complexity in ksvideosrc but the issue still seems to be there. Please note that I am building ksvideosrc myself to benefit from all the fixes available in master. To buil build it, I just took the ksvideosrc into my Qt 5.4.0 dev environment and compiled them with my toolchain. With my build ksvideosrc is not getting a clock... while the version from the official 1.4.5 windows distribution seems to get a clock. So all this is to be taken with a pinch of salt as the issue might be on my side. I'll try to investigate further and in the meantime I am trying to build gstreamer myself.
One minor issue I got when building ksvideosrc is that it assumes to be built in ANSI mode and not UNICODE. If building with UNICODE all the CreateFile calls fail because they are being passed regular ansi strings. There are a few places were explicit Wide calls are made and the result are converveted from utf16 to utf8. The resulting utf8 string (i.e. device path) is then used in CreateFile calls. Those calls fail if CreateFile is defined as CreateFileW (which is the case if UNICODE is defined) Fixing this is straightforward : simply replace all three occurrences of CreateFile with calls to CreateFileA.
Created attachment 302310 [details] Log showing various issues with ksvideosrc
Found a minor comment issue. The pipeline examples given gstksvidesrc.c use the now removed ffmpegcolorspace element.
I have been able to build gstreamer using cerbero! I am building 1.4 with mingw on windows 7. Build had only two minor hiccups: 1- problem with certificates which I worked around with this change: diff --git a/cerbero/bootstrap/windows.py b/cerbero/bootstrap/windows.py index 71010f9..9d11456 100644 --- a/cerbero/bootstrap/windows.py +++ b/cerbero/bootstrap/windows.py @@ -136,7 +136,7 @@ class WindowsBootstrapper(BootstrapperBase): else: inst_path = os.path.join(self.prefix, 'x86_64-w64-mingw32/include/GL/wglext.h') gl_header = 'http://www.opengl.org/registry/api/GL/wglext.h' - shell.download(gl_header, inst_path) + shell.download(gl_header, inst_path, False, False) def install_bin_deps(self): # FIXME: build intltool as part of the build tools bootstrap 2- failure to build libtag which I simply skipped. I then did a number of tests around ksvideosrc. See next comment...
Using gst-launch I ran this simple pipeline : ksvideosrc ! fpsdisplaysink ---------------------------------------- ksvideosrc 1.4 branch ---------------------------------------- A lot of buffers are being dropped. (gstbasesink.c(2794): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstAutoVideoSink:fps-display-video_sink/GstD3DVideoSink:fps-display-video_sink-actual-sink-d3dvideo: There may be a timestamping problem, or this computer is too slow.) has a clock fpsdisplaysink reports: rendered: 164, dropped: 136, fps 13.13, drop rate: 7.50 ---------------------------------------- ksvideosrc master branch ---------------------------------------- Fails to acquire a clock: WARN : 0:00:03.072565684 5372 08FEAA40 WARN ksvideosrc gstksvideosrc.c:456:gst_ks_video_src_open_device:<ksvideosrc0> failed to create/open KsClock fpsdisplaysink reports: rendered: 328, dropped: 92, fps 3.38, drop rate: 6.76 ---------------------------------------- ksvideosrc master branch + patch ---------------------------------------- fpsdisplaysink reports: rendered: 0, dropped: 192, fps 0.00, drop rate: 28.27 Same clock warning as master. ---------------------------------------- clock issue ---------------------------------------- The clock issue that appears in master comes from this addition: + /* Enumerate only capture devices */ + if_alias_data.cbSize = sizeof (SP_DEVICE_INTERFACE_DATA); + success = SetupDiGetDeviceInterfaceAlias (devinfo, &if_data, &KSCATEGORY_CAPTURE, + &if_alias_data); + if (!success) + continue; ---------------------------------------- other ---------------------------------------- In all test scenario above, I always receive one qos event: qos event: live: 1; running time: 0; stream time: 0; timestamp: 1741382649; duration: 33333333 jitter: 1756806371; proportion: -1; quality: 1000000; format: ; processed: 0; dropped: 1; I need to test one more scenario which is the 1.4 branch + patch. Doing it now...
---------------------------------------- ksvideosrc 1.4 branch + patch ---------------------------------------- A lot of buffers are being dropped. (gstbasesink.c(2794): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstAutoVideoSink:fps-display-video_sink/GstD3DVideoSink:fps-display-video_sink-actual-sink-d3dvideo: There may be a timestamping problem, or this computer is too slow.) has a clock fpsdisplaysink reports: rendered: 0, dropped: 129, fps 0.00, drop rate: 15.83 What is strange is that in all test cases, the video visually seems to run at ~30fps. Forcing the frame rate to 10fps makes a big difference (i.e. you can see it runs at 10fps). Pipeline used for 10fps: ksvideosrc ! video/x-raw, framerate=10/1 ! fpsdisplaysink
Created attachment 314557 [details] [review] Adapted Sebastian's patch to 1.7.0 Found time to adapt and retest Sebastian's fix on master. Works fine. No more dropped frames. The fix simplifies frame time-stamping by not trying to align the current timestamp on a multiple of the duration since the previous frame.
What did you change compared to my patch?
Here is a diff of the two patches: $ diff 0001-ksvideosrc-don-t-try-to-align-current-timestamp-on-p.patch ../../../ksvideo.patch 1,13d0 < From 1943601a0601da915279e02817c72e7d15543746 Mon Sep 17 00:00:00 2001 < From: Philippe Renon <philippe_renon@yahoo.fr> < Date: Sat, 31 Oct 2015 18:47:39 +0100 < Subject: [PATCH 1/1] ksvideosrc: don't try to align current timestamp on < previous timestamp < < this causes frames to be dropped < < https://bugzilla.gnome.org/show_bug.cgi?id=748337 < --- < sys/winks/gstksvideosrc.c | 76 ++--------------------------------------------- < 1 file changed, 3 insertions(+), 73 deletions(-) < 15c2 < index 07cdf8b..12c7eed 100644 --- > index 07cdf8b..b981e8d 100644 38c25 < @@ -813,12 +807,11 @@ gst_ks_video_src_timestamp_buffer (GstKsVideoSrc * self, GstBuffer * buf, --- > @@ -813,7 +807,7 @@ gst_ks_video_src_timestamp_buffer (GstKsVideoSrc * self, GstBuffer * buf, 47,52c34 < duration = timestamp = GST_CLOCK_TIME_NONE; < - priv->offset++; < goto timestamp; < } < < @@ -828,88 +821,25 @@ gst_ks_video_src_timestamp_buffer (GstKsVideoSrc * self, GstBuffer * buf, --- > @@ -828,88 +822,25 @@ gst_ks_video_src_timestamp_buffer (GstKsVideoSrc * self, GstBuffer * buf, 143,145d124 < -- < 1.9.2.msysgit.0 <
My change was to remove related code added after your fix.
commit c4657f427651beefefe3ee3aff08dc2f7eb1d6af Author: Philippe Renon <philippe_renon@yahoo.fr> Date: Sat Oct 31 18:47:39 2015 +0100 ksvideosrc: don't try to align current timestamp on previous timestamp this causes frames to be dropped. Based on a patch by Sebastian Dröge https://bugzilla.gnome.org/show_bug.cgi?id=748337