After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 748337 - ksvideosrc: dropping frames
ksvideosrc: dropping frames
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
1.4.5
Other Windows
: Normal normal
: 1.7.1
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2015-04-22 21:09 UTC by Philippe Renon
Modified: 2015-11-01 09:17 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Patch provided by Sebastian Dröge (4.12 KB, text/plain)
2015-04-22 21:09 UTC, Philippe Renon
  Details
Log showing various issues with ksvideosrc (291.73 KB, text/plain)
2015-04-24 16:53 UTC, Philippe Renon
  Details
Adapted Sebastian's patch to 1.7.0 (4.55 KB, patch)
2015-10-31 18:00 UTC, Philippe Renon
committed Details | Review

Description Philippe Renon 2015-04-22 21:09:36 UTC
Created attachment 302185 [details]
Patch provided by Sebastian Dröge

Apparently ksvideosrc is dropping 2 frames out of 3

Sebastian provided a patch (attached) that removes some complexity in ksvideosrc but the issue still seems to be there.

Please note that I am building ksvideosrc myself to benefit from all the fixes available in master. To buil build it, I just took the ksvideosrc into my Qt 5.4.0 dev environment and compiled them with my toolchain.
With my build ksvideosrc is not getting a clock... while the version from the official 1.4.5 windows distribution seems to get a clock.
So all this is to be taken with a pinch of salt as the issue might be on my side.

I'll try to investigate further and in the meantime I am trying to build gstreamer myself.
Comment 1 Philippe Renon 2015-04-22 21:18:08 UTC
One minor issue I got when building ksvideosrc is that it assumes to be built in ANSI mode and not UNICODE.

If building with UNICODE all the CreateFile calls fail because they are being passed regular ansi strings. 
There are a few places were explicit Wide calls are made and the result are converveted from utf16 to utf8. The resulting utf8 string (i.e. device path) is then used in CreateFile calls. Those calls fail if CreateFile is defined as CreateFileW (which is the case if UNICODE is defined)

Fixing this is straightforward : simply replace all three occurrences of CreateFile with calls to CreateFileA.
Comment 2 Philippe Renon 2015-04-24 16:53:04 UTC
Created attachment 302310 [details]
Log showing various issues with ksvideosrc
Comment 3 Philippe Renon 2015-04-25 13:08:49 UTC
Found a minor comment issue. The pipeline examples given gstksvidesrc.c use the now removed ffmpegcolorspace element.
Comment 4 Philippe Renon 2015-04-25 13:12:12 UTC
I have been able to build gstreamer using cerbero! I am building 1.4 with mingw on windows 7. Build had only two minor hiccups:

1- problem with certificates which I worked around with this change:

diff --git a/cerbero/bootstrap/windows.py b/cerbero/bootstrap/windows.py
index 71010f9..9d11456 100644
--- a/cerbero/bootstrap/windows.py
+++ b/cerbero/bootstrap/windows.py
@@ -136,7 +136,7 @@ class WindowsBootstrapper(BootstrapperBase):
         else:
             inst_path = os.path.join(self.prefix, 'x86_64-w64-mingw32/include/GL/wglext.h')
         gl_header = 'http://www.opengl.org/registry/api/GL/wglext.h'
-        shell.download(gl_header, inst_path)
+        shell.download(gl_header, inst_path, False, False)

     def install_bin_deps(self):
         # FIXME: build intltool as part of the build tools bootstrap

2- failure to build libtag which I simply skipped.

I then did a number of tests around ksvideosrc. See next comment...
Comment 5 Philippe Renon 2015-04-25 13:45:41 UTC
Using gst-launch I ran this simple pipeline : ksvideosrc ! fpsdisplaysink

----------------------------------------
ksvideosrc 1.4 branch
----------------------------------------

A lot of buffers are being dropped. (gstbasesink.c(2794): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstAutoVideoSink:fps-display-video_sink/GstD3DVideoSink:fps-display-video_sink-actual-sink-d3dvideo:
There may be a timestamping problem, or this computer is too slow.)

has a clock

fpsdisplaysink reports:
rendered: 164, dropped: 136, fps 13.13, drop rate: 7.50

----------------------------------------
ksvideosrc master branch
----------------------------------------

Fails to acquire a clock:
WARN : 0:00:03.072565684  5372   08FEAA40 WARN              ksvideosrc gstksvideosrc.c:456:gst_ks_video_src_open_device:<ksvideosrc0> failed to create/open KsClock

fpsdisplaysink reports:
rendered: 328, dropped: 92, fps 3.38, drop rate: 6.76

----------------------------------------
ksvideosrc master branch + patch
----------------------------------------

fpsdisplaysink reports:
rendered: 0, dropped: 192, fps 0.00, drop rate: 28.27

Same clock warning as master.

----------------------------------------
clock issue
----------------------------------------

The clock issue that appears in master comes from this addition:

+    /* Enumerate only capture devices */
+    if_alias_data.cbSize = sizeof (SP_DEVICE_INTERFACE_DATA);
+    success = SetupDiGetDeviceInterfaceAlias (devinfo, &if_data, &KSCATEGORY_CAPTURE,
+        &if_alias_data);
+    if (!success)
+      continue;

----------------------------------------
other
----------------------------------------

In all test scenario above, I always receive one qos event:
qos event: live: 1; running time: 0; stream time: 0; timestamp: 1741382649; duration: 33333333 jitter: 1756806371; proportion: -1; quality: 1000000; format: ; processed: 0; dropped: 1;

I need to test one more scenario which is the 1.4 branch + patch. Doing it now...
Comment 6 Philippe Renon 2015-04-25 14:13:48 UTC
----------------------------------------
ksvideosrc 1.4 branch + patch
----------------------------------------

A lot of buffers are being dropped. (gstbasesink.c(2794): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstAutoVideoSink:fps-display-video_sink/GstD3DVideoSink:fps-display-video_sink-actual-sink-d3dvideo:
There may be a timestamping problem, or this computer is too slow.)

has a clock

fpsdisplaysink reports:
rendered: 0, dropped: 129, fps 0.00, drop rate: 15.83



What is strange is that in all test cases, the video visually seems to run at ~30fps.
Forcing the frame rate to 10fps makes a big difference (i.e. you can see it runs at 10fps).
Pipeline used for 10fps: ksvideosrc ! video/x-raw, framerate=10/1 ! fpsdisplaysink
Comment 7 Philippe Renon 2015-10-31 18:00:26 UTC
Created attachment 314557 [details] [review]
Adapted Sebastian's patch to 1.7.0

Found time to adapt and retest Sebastian's fix on master.
Works fine. No more dropped frames.

The fix simplifies frame time-stamping by not trying to align the current timestamp on a multiple of the duration since the previous frame.
Comment 8 Sebastian Dröge (slomo) 2015-10-31 18:10:59 UTC
What did you change compared to my patch?
Comment 9 Philippe Renon 2015-10-31 18:36:45 UTC
Here is a diff of the two patches:

$ diff 0001-ksvideosrc-don-t-try-to-align-current-timestamp-on-p.patch ../../../ksvideo.patch
1,13d0
< From 1943601a0601da915279e02817c72e7d15543746 Mon Sep 17 00:00:00 2001
< From: Philippe Renon <philippe_renon@yahoo.fr>
< Date: Sat, 31 Oct 2015 18:47:39 +0100
< Subject: [PATCH 1/1] ksvideosrc: don't try to align current timestamp on
<  previous timestamp
<
< this causes frames to be dropped
<
< https://bugzilla.gnome.org/show_bug.cgi?id=748337
< ---
<  sys/winks/gstksvideosrc.c | 76 ++---------------------------------------------
<  1 file changed, 3 insertions(+), 73 deletions(-)
<
15c2
< index 07cdf8b..12c7eed 100644
---
> index 07cdf8b..b981e8d 100644
38c25
< @@ -813,12 +807,11 @@ gst_ks_video_src_timestamp_buffer (GstKsVideoSrc * self, GstBuffer * buf,
---
> @@ -813,7 +807,7 @@ gst_ks_video_src_timestamp_buffer (GstKsVideoSrc * self, GstBuffer * buf,
47,52c34
<      duration = timestamp = GST_CLOCK_TIME_NONE;
< -    priv->offset++;
<      goto timestamp;
<    }
<
< @@ -828,88 +821,25 @@ gst_ks_video_src_timestamp_buffer (GstKsVideoSrc * self, GstBuffer * buf,
---
> @@ -828,88 +822,25 @@ gst_ks_video_src_timestamp_buffer (GstKsVideoSrc * self, GstBuffer * buf,
143,145d124
< --
< 1.9.2.msysgit.0
<
Comment 10 Philippe Renon 2015-10-31 23:13:43 UTC
My change was to remove related code added after your fix.
Comment 11 Sebastian Dröge (slomo) 2015-11-01 09:17:33 UTC
commit c4657f427651beefefe3ee3aff08dc2f7eb1d6af
Author: Philippe Renon <philippe_renon@yahoo.fr>
Date:   Sat Oct 31 18:47:39 2015 +0100

    ksvideosrc: don't try to align current timestamp on previous timestamp
    
    this causes frames to be dropped.
    
    Based on a patch by Sebastian Dröge
    
    https://bugzilla.gnome.org/show_bug.cgi?id=748337