GNOME Bugzilla – Bug 729887
decklinksrc requires output sinks to run with sync=FALSE.
Last modified: 2014-12-19 17:07:47 UTC
The documentation for the decklink capture card we recently purchased states (without explanation) that when using decklinksrc in a pipeline, the output sink needs to have clock sync disabled, (ie property sync=FALSE). Some simple tests have proven this to be true. With sync=FALSE we are able to capture and display HDMI content at (or at least at what appears to be) a full 60fps. When we leave sync at the default of TRUE, we start to get errors about dropped frames if we go over 9 fps. What causes this requirement is not clear. Below is a simple pipeline with sync=TRUE, and the timestamps all appear correct. I'm not sure what else would cause this kind of issue though. Knowing that most of the developers won't have access to one of these expensive capture cards, I'll be most willing to run any additional tests and report any results that may aid in tracking down the issue. gst-launch-1.0 -v decklinksrc connection=HDMI mode=18 name=d d.videosrc ! fakesink silent=false Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstSystemClock /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: stream-start (10254), GstEventStreamStart, stream-id=(string)74ffd1fa, flags=(GstStreamFlags)GST_STREAM_FLAG_NONE, group-id=(uint)0;) 0x20399e0 /GstPipeline:pipeline0/GstDecklinkSrc:d.GstPad:audiosrc: caps = audio/x-raw, format=(string)S16LE, channels=(int)2, rate=(int)48000, layout=(string)interleaved /GstPipeline:pipeline0/GstDecklinkSrc:d.GstPad:videosrc: caps = video/x-raw, format=(string)UYVY, width=(int)1280, height=(int)720, framerate=(fraction)60/1, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, chroma-site=(string)mpeg2 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: caps (12814), GstEventCaps, caps=(GstCaps)video/x-raw, format=(string)UYVY, width=(int)1280, height=(int)720, framerate=(fraction)60/1, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, chroma-site=(string)mpeg2;) 0x2039aa0 /GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)1280, height=(int)720, framerate=(fraction)60/1, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)bt709, chroma-site=(string)mpeg2 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: segment (17934), GstEventSegment, segment=(GstSegment)"GstSegment, flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE, rate=(double)1, applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME, base=(guint64)0, offset=(guint64)0, start=(guint64)0, stop=(guint64)18446744073709551615, time=(guint64)0, position=(guint64)0, duration=(guint64)18446744073709551615;";) 0x2039b00 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = preroll ******* /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1843200 bytes, dts: none, pts: 0:00:00.000000000, duration: 0:00:00.016666666, offset: 0, offset_end: 0, flags: 00000040 discont ) 0x7f6450005010 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1843200 bytes, dts: none, pts: 0:00:00.016666666, duration: 0:00:00.016666667, offset: 1, offset_end: 1, flags: 00000000 ) 0x7f6450005120 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1843200 bytes, dts: none, pts: 0:00:00.033333333, duration: 0:00:00.016666667, offset: 2, offset_end: 2, flags: 00000000 ) 0x7f6450005230 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1843200 bytes, dts: none, pts: 0:00:00.050000000, duration: 0:00:00.016666666, offset: 3, offset_end: 3, flags: 00000000 ) 0x7f6450005340 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1843200 bytes, dts: none, pts: 0:00:00.066666666, duration: 0:00:00.016666667, offset: 4, offset_end: 4, flags: 00000000 ) 0x7f6450005450 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1843200 bytes, dts: none, pts: 0:00:00.083333333, duration: 0:00:00.016666667, offset: 5, offset_end: 5, flags: 00000000 ) 0x7f6450005560 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1843200 bytes, dts: none, pts: 0:00:00.100000000, duration: 0:00:00.016666666, offset: 6, offset_end: 6, flags: 00000000 ) 0x7f6450005450 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1843200 bytes, dts: none, pts: 0:00:00.116666666, duration: 0:00:00.016666667, offset: 7, offset_end: 7, flags: 00000000 ) 0x7f6450005340 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1843200 bytes, dts: none, pts: 0:00:00.133333333, duration: 0:00:00.016666667, offset: 8, offset_end: 8, flags: 00000000 ) 0x7f6450005230 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1843200 bytes, dts: none, pts: 0:00:00.150000000, duration: 0:00:00.016666666, offset: 9, offset_end: 9, flags: 00000000 ) 0x7f6450005120 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1843200 bytes, dts: none, pts: 0:00:00.166666666, duration: 0:00:00.016666667, offset: 10, offset_end: 10, flags: 00000000 ) 0x7f6450005010 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1843200 bytes, dts: none, pts: 0:00:00.183333333, duration: 0:00:00.016666667, offset: 11, offset_end: 11, flags: 00000000 ) 0x7f6450005120 /GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1843200 bytes, dts: none, pts: 0:00:00.200000000, duration: 0:00:00.016666666, offset: 12, offset_end: 12, flags: 00000000 ) 0x7f6450005230
The PTS seems correct, it might not be handling the latency query/event correctly, though.
It has some handling for LATENCY on the video pad though, but it looks a bit too simple. Anyway, can you get a debug log with... hmm. GST_DEBUG=6? Of a pipeline with a sink that has sync=TRUE. That would tell us why frames are dropped.
Sorry, I didn't manage to get access to the card before the end of the day. I'll have to get you your debug log on Monday.
I experience the same bug in 1.2 HEAD. Just to let you know that this problem is present in 0.10 HEAD as well. I can contribute some logs by Monday if need be.
Sorry for the delay. The fellow with the decklinksrc card at work was not in on Monday, and no one knew what he had done with it. Today that fellow obligingly made me the requested debug log, but its too big to attach or upload to pastebin, so I've put it up on google drive. Here's the URL: https://drive.google.com/file/d/0BwCOnjTaGreOVlZieDU0czFNZm8/edit Let me know if there is anything else I can do to help.
Created attachment 280160 [details] Logs for different runs of gst-launch
Comment on attachment 280160 [details] Logs for different runs of gst-launch I've also found a similar behaviour while trying to capture from a DeckLink Quad pci card; in this case, "fakesink" works fine but "xvimagesink" or "autovideosink" drop lots of frames if sync is not fine tuned by hand. Following your advice, I created some logs which show the output of gst-inspect. In my case, I can fix the problem by manually specifying a "max-lateness" of at least 250 milliseconds; any value below that seems to have a high probability of dropping frames, but it's not absolutely 100% sure that this will happen: sometimes (the least of them) it has worked properly with max-lateness values of 200 or 190. I'm using Ubuntu 14.04 with kernel 3.13.0-29 and GStreamer version 1.2.4. Attached come the logs for different command lines: $ gst-launch-1.0 -v -m decklinksrc device-number=2 connection=sdi mode=pal ! fakesink $ gst-launch-1.0 -v -m decklinksrc device-number=2 connection=sdi mode=pal ! fakesink silent=false $ gst-launch-1.0 -v -m decklinksrc device-number=2 connection=sdi mode=pal ! xvimagesink $ gst-launch-1.0 -v -m decklinksrc device-number=2 connection=sdi mode=pal ! xvimagesink max-lateness=150000000 $ gst-launch-1.0 -v -m decklinksrc device-number=2 connection=sdi mode=pal ! xvimagesink max-lateness=250000000 $ GST_DEBUG=6 GST_DEBUG_NO_COLOR=1 gst-launch-1.0 -v -m decklinksrc device-number=2 connection=sdi mode=pal ! xvimagesink max-lateness=150000000 $ GST_DEBUG=6 GST_DEBUG_NO_COLOR=1 gst-launch-1.0 -v -m decklinksrc device-number=2 connection=sdi mode=pal ! xvimagesink max-lateness=250000000
This looks like decklink latency not being exposed handled. If you guys want to have a look, there is a LATENCY query that would ideally be replied with sensible value.
In the latest log, the drift is pretty stable, between 215 and 230 ms. Do decklink specify the latency in it's specification or within it's API ?
Interesting, we completely ignore provided timestamp, and craft our own: GST_BUFFER_TIMESTAMP (buffer) = gst_util_uint64_scale_int (decklinksrc->frame_num * GST_SECOND, mode->fps_d, mode->fps_n); GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int ((decklinksrc->frame_num + 1) * GST_SECOND, mode->fps_d, mode->fps_n) - GST_BUFFER_TIMESTAMP (buffer); If there is a drift between the decklink and the pipeline clock it won't work well. This need fixing.
Is there anything I can do to help with this Bug? Progress seems to have stalled.
If you could trace the source timestamp and the pipeline clock value, it would be nice to compare each other on a graphic.
(In reply to comment #12) > If you could trace the source timestamp and the pipeline clock value, it would > be nice to compare each other on a graphic. I'm not sure how to go about gathering that data. Give me some idea what I would need to do, and I'll try to do it.
Please test again with latest GIT master. The decklink elements were rewritten, and especially should handle synchronization properly now.
Confirmed. We no longer have any issues with the sync property of decklinksrc since the rewrite.