After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 774850 - decklinksrc: Stop using the "hardware" timestamps and directly use the pipeline clock
decklinksrc: Stop using the "hardware" timestamps and directly use the pipeli...
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
unspecified
Other All
: Normal normal
: 1.11.1
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on: 774916
Blocks:
 
 
Reported: 2016-11-22 17:33 UTC by Sebastian Dröge (slomo)
Modified: 2017-08-02 08:10 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
decklinksrc: Stop using the "hardware" timestamps and directly use the pipeline clock (28.94 KB, patch)
2016-11-22 17:33 UTC, Sebastian Dröge (slomo)
none Details | Review
decklinksrc: Stop using the "hardware" timestamps and directly use the pipeline clock (28.94 KB, patch)
2016-11-23 16:12 UTC, Sebastian Dröge (slomo)
none Details | Review
decklinksrc: Keep track of a window of capture/stream time mappings and calculate relationship (12.41 KB, patch)
2016-11-23 16:12 UTC, Sebastian Dröge (slomo)
none Details | Review
decklinksrc: Keep track of a window of capture/stream time mappings and calculate relationship (12.52 KB, patch)
2016-11-23 21:10 UTC, Sebastian Dröge (slomo)
none Details | Review
decklinksrc: Stop using the "hardware" timestamps and directly use the pipeline clock (28.99 KB, patch)
2016-11-24 13:15 UTC, Sebastian Dröge (slomo)
committed Details | Review
decklinksrc: Keep track of a window of capture/stream time mappings and calculate relationship (12.52 KB, patch)
2016-11-24 13:15 UTC, Sebastian Dröge (slomo)
none Details | Review
decklinksrc: Keep track of a window of capture/stream time mappings and calculate relationship (15.73 KB, patch)
2016-11-24 17:21 UTC, Sebastian Dröge (slomo)
none Details | Review
decklinksrc: Keep track of a window of capture/stream time mappings and calculate relationship (15.73 KB, patch)
2016-11-25 13:04 UTC, Sebastian Dröge (slomo)
committed Details | Review
decklinksrc: Set GAP flag on buffers that are captured without signal (7.35 KB, patch)
2016-11-25 13:04 UTC, Sebastian Dröge (slomo)
committed Details | Review
decklinksrc: Add property to skip the first N seconds of capture (5.89 KB, patch)
2016-11-25 13:04 UTC, Sebastian Dröge (slomo)
committed Details | Review
decklinksrc: Add property to drop frames that are marked as having no input signal (5.14 KB, patch)
2016-11-25 13:04 UTC, Sebastian Dröge (slomo)
committed Details | Review

Description Sebastian Dröge (slomo) 2016-11-22 17:33:34 UTC
See commit message
Comment 1 Sebastian Dröge (slomo) 2016-11-22 17:33:38 UTC
Created attachment 340541 [details] [review]
decklinksrc: Stop using the "hardware" timestamps and directly use the pipeline clock

The hardware timestamps have no relation to when frames were produced,
only when frames arrived somewhere in the hardware. Especially there is
no guarantee that audio and video will have the same hardware timestamps
although they belong together, and even more important: the rate with
which the hardware timestamps increase is completely unrelated to the
rate with which the frames are captured!

As such we can as well use the pipeline clock directly and stop doing
complicated calculations. Also as a side effect this allows now running
without any pipeline clock, by directly making use of the stream times
as reported by the driver.
Comment 2 Sebastian Dröge (slomo) 2016-11-23 07:32:10 UTC
There will be more patches on top of this, not mergeable as is. We can do better
Comment 3 Sebastian Dröge (slomo) 2016-11-23 16:12:52 UTC
Created attachment 340614 [details] [review]
decklinksrc: Stop using the "hardware" timestamps and directly use the pipeline clock

The hardware timestamps have no relation to when frames were produced,
only when frames arrived somewhere in the hardware. Especially there is
no guarantee that audio and video will have the same hardware timestamps
although they belong together, and even more important: the rate with
which the hardware timestamps increase is completely unrelated to the
rate with which the frames are captured!

As such we can as well use the pipeline clock directly and stop doing
complicated calculations. Also as a side effect this allows now running
without any pipeline clock, by directly making use of the stream times
as reported by the driver.
Comment 4 Sebastian Dröge (slomo) 2016-11-23 16:12:58 UTC
Created attachment 340615 [details] [review]
decklinksrc: Keep track of a window of capture/stream time mappings and calculate relationship

Based on this we calculate the actual capture time, which should get us
rid of any capturing jitter by averaging it out.

Also add a output-stream-time property which forces the elements to
output the stream time directly instead of doing any conversion to the
pipeline clock. Use with care.
Comment 5 Sebastian Dröge (slomo) 2016-11-23 21:10:32 UTC
Created attachment 340647 [details] [review]
decklinksrc: Keep track of a window of capture/stream time mappings and calculate relationship

Based on this we calculate the actual capture time, which should get us
rid of any capturing jitter by averaging it out.

Also add a output-stream-time property which forces the elements to
output the stream time directly instead of doing any conversion to the
pipeline clock. Use with care.
Comment 6 Sebastian Dröge (slomo) 2016-11-24 13:15:52 UTC
Created attachment 340680 [details] [review]
decklinksrc: Stop using the "hardware" timestamps and directly use the pipeline clock

The hardware timestamps have no relation to when frames were produced,
only when frames arrived somewhere in the hardware. Especially there is
no guarantee that audio and video will have the same hardware timestamps
although they belong together, and even more important: the rate with
which the hardware timestamps increase is completely unrelated to the
rate with which the frames are captured!

As such we can as well use the pipeline clock directly and stop doing
complicated calculations. Also as a side effect this allows now running
without any pipeline clock, by directly making use of the stream times
as reported by the driver.
Comment 7 Sebastian Dröge (slomo) 2016-11-24 13:15:58 UTC
Created attachment 340681 [details] [review]
decklinksrc: Keep track of a window of capture/stream time mappings and calculate relationship

Based on this we calculate the actual capture time, which should get us
rid of any capturing jitter by averaging it out.

Also add a output-stream-time property which forces the elements to
output the stream time directly instead of doing any conversion to the
pipeline clock. Use with care.
Comment 8 Sebastian Dröge (slomo) 2016-11-24 17:21:41 UTC
Created attachment 340701 [details] [review]
decklinksrc: Keep track of a window of capture/stream time mappings and calculate relationship

Based on this we calculate the actual capture time, which should get us
rid of any capturing jitter by averaging it out.

Also add a output-stream-time property which forces the elements to
output the stream time directly instead of doing any conversion to the
pipeline clock. Use with care.
Comment 9 Sebastian Dröge (slomo) 2016-11-25 13:04:12 UTC
Created attachment 340753 [details] [review]
decklinksrc: Keep track of a window of capture/stream time mappings and calculate relationship

Based on this we calculate the actual capture time, which should get us
rid of any capturing jitter by averaging it out.

Also add a output-stream-time property which forces the elements to
output the stream time directly instead of doing any conversion to the
pipeline clock. Use with care.
Comment 10 Sebastian Dröge (slomo) 2016-11-25 13:04:18 UTC
Created attachment 340754 [details] [review]
decklinksrc: Set GAP flag on buffers that are captured without signal
Comment 11 Sebastian Dröge (slomo) 2016-11-25 13:04:23 UTC
Created attachment 340755 [details] [review]
decklinksrc: Add property to skip the first N seconds of capture

On some hardware the first few frames are bogus and not very useful.
Their timestamps are off, they have no timecodes, or there are spurious
black frames / no-signal frames. After a few frames this stabilizes
though.
Comment 12 Sebastian Dröge (slomo) 2016-11-25 13:04:28 UTC
Created attachment 340756 [details] [review]
decklinksrc: Add property to drop frames that are marked as having no input signal
Comment 13 Sebastian Dröge (slomo) 2016-11-28 12:28:05 UTC
Attachment 340680 [details] pushed as 881a086 - decklinksrc: Stop using the "hardware" timestamps and directly use the pipeline clock
Attachment 340753 [details] pushed as 09dd64b - decklinksrc: Keep track of a window of capture/stream time mappings and calculate relationship
Attachment 340754 [details] pushed as cf224a9 - decklinksrc: Set GAP flag on buffers that are captured without signal
Attachment 340755 [details] pushed as 331b2af - decklinksrc: Add property to skip the first N seconds of capture
Attachment 340756 [details] pushed as aec4614 - decklinksrc: Add property to drop frames that are marked as having no input signal
Comment 14 Baby octopus 2017-08-02 07:52:58 UTC
Could this have been configurable through a property, just like 'provide-clock' in alsasrc?
Comment 15 Sebastian Dröge (slomo) 2017-08-02 08:10:17 UTC
Yes but it would not make any sense. The timestamps have no real meaning.