GNOME Bugzilla – Bug 666524
Pipeline using decklinksink requires a particular ordering of conversion elements
Last modified: 2015-02-12 15:26:29 UTC
HI all, I've been trying to use the decklinksink GStreamer element lately, and with little success... The following gst-launch command-line works flawlessly: " gst-launch-0.10 videotestsrc ! decklinksink " And shows the SMTPE bars on my TV. I could even use textoverlay to create a nice soviet screen :) The following ALSO works (ALSO using videotestsrc): " gst-launch-0.10 videotestsrc pattern=red ! textoverlay text="Сою́з Сове́тских Социалисти́ческих Респу́блик" valignment=center font-desc="Sans Bold 20" color=16776960 ! decklinksink " BUT, when I try to use filesrc in a very simple pipeline, it all goes bad. The following gst-launch command gives no error, and the pipeline state is PLAYING, but my screen remains black: " gst-launch-0.10 filesrc location=/home/joaopizani/btest.mp4 ! decodebin2 ! autoconvert ! decklinksink " I think I have isolated any problem with my card and/or driver, because: 1) It works with videotestsrc very well! 2) The failing command, when I change "decklinksink" to "xvimagesink", also works OK. Do any of you guys have any experience using decklinksink? If if you don't, does any of you have any idea of what's happening to me? :)
A log would help here. Prepend GST_DEBUG=5 to the gst-launch-0,10 command, then ^C after the pipeline went to playing. Also, I notice decklinksink has both a video and audio sink pad. It *might* be that, since autoconvert can do either video or audio, you get the audio out of decodebin2. Unlikely unless you don't have speakers on though.
Created attachment 203898 [details] Log of running the command mentioned in the report with options -v and -m
Created attachment 203900 [details] Report of running the command mentioned in the report with the GST_DEBUG=3 variable set I though this was still A LOT verbose, even though I decreased the level from 5 to 3. That's why I sent the other attachment, with the verbose (-v) log.
OK, so now I have added two attachments to the bug. There are both the running of the command-line with GST_DEBUG=3 and a log generated by adding the -v and -m options to the command-line. Regarding speakers, NO, I don't have them. Anyways, I tried substituting autovideoconvert for autoconvert, then the pipeline couldn't end negotiation. Do you know how can I redirect the video src of decodebin2 to the video sink of decklinksink specifically? I am a big newbie when it comes to named pads, caps negotiation and so forth, I still have lots to learn...
Created attachment 203903 [details] Same gst-launch command-line, but selecting specifically src0 pad of decodebin2 I ran the same gst-launch command-line, but this time I managed to understand how to link specific pads. I knew that decodebin2 was generating 2 sink pads, one for audio and one for video, so in this attachtment I post the log of running the command linking specifically decodebin20.src0 to autoconvert0.sink. The result is that the pipeline will not even start playing, it will just abort with "reason not-linked".
Created attachment 203904 [details] Same gst-launch command-line, but this time selecting src1 pad of decodebin2 This time I selected the generated src1 pad from decodebin2, which is the one that apparently plays. I cannot test, however, because I have no speakers. My question is then "Why on earth is video from videotestsrc playing OK, but video from a file being not-linked?"
Thanks. The -v flag is unnecessary here. These logs are not generated with GST_DEBUG=3, you probably typoed or set it incorrectly. Even with GST_DEBUG=3, it is very likely it will be insufficient, so a 5 log would be best.
OK, so now I have uploaded a complete GST_DEBUG=5 report of running the aforementioned gst-launch command line " GST_DEBUG=5 gst-launch-0.10 filesrc location=btest.mp4 ! decodebin2 decodebin20.src0 ! autoconvert0.sink autoconvert ! decklinksink -m 2> debug_decklink.txt 1> debug_decklink.txt " The resulting log file has 26MB, and since it was too big for posting directly here, I uploaded ir to 4shared. Here is the link: http://www.4shared.com/get/9cWb0n6n/debug_decklink.html I hope I have been helpful :)
Thanks. The file can be only 1.1 MB when compressed with, eg, bzip2 :) The logs seem to show the failure happening with the audio pad, but this log was made with autoconvert, not autovideoconvert as per your later attempt. In any case, I thought using autovideoconvert should ensure that the video pads are used, but I checked and it has ANY caps, not only video caps, so you could use ! videoscale ! videorate ! colorspace ! instead of autovideoconvert, so we're sure the video pads are used. Can you upload a compressed log with that please ?
Created attachment 203966 [details] Log of running the pipeline with !videoscale!videorate!colorspace! instead of autovideoconvert Wow how dumb I was sending the raw file :) Logs are the "dream case" for compressors... Well, this time I tried as you said, replacing autovideoconvert with "! videoscale ! videorate ! colorspace !". The gst-launch command line was like this: GST_DEBUG=5 gst-launch-0.10 filesrc location=flyby-xvid.avi ! decodebin2 ! videoscale ! videorate ! colorspace ! decklinksink -m Even though, the pipeline could not play and gst complained something like "reason not-negotiated" (I guess that refers to caps). I think we are in the right direction now, I just need to find an appropriate sequence of elements in this pipeline, which I though would be much easier! :)
More news: Using the following command-line: GST_DEBUG=4 gst-launch-0.10 filesrc location=test/akiyo_cif.mpg ! decodebin2 decodebin20.src0 ! autovideoconvert0.sink autovideoconvert autovideoconvert0.src ! decklinksink0.sink decklinksink I got a debug (level 4) file, and then looking at this file I found the critical piece of information that make caps negotiation fail. Here it is the debug message (a little formatted by me): " autoconvert gstautoconvert.c:742:factory_can_intersect:<autoconvertchild> Factories <elementfactory326> static caps video/x-raw-rgb, bpp=(int)32, depth=(int)32, endianness=(int)4321, red_mask=(int)-16777216, green_mask=(int)16711680, blue_mask=(int)65280, alpha_mask=(int)255, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw-rgb, bpp=(int)32, depth=(int)32, endianness=(int)4321, red_mask=(int)65280, green_mask=(int)16711680, blue_mask=(int)-16777216, alpha_mask=(int)255, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw-rgb, bpp=(int)32, depth=(int)32, endianness=(int)4321, red_mask=(int)16711680, green_mask=(int)65280, blue_mask=(int)255, alpha_mask=(int)-16777216, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw-rgb, bpp=(int)32, depth=(int)32, endianness=(int)4321, red_mask=(int)255, green_mask=(int)65280, blue_mask=(int)16711680, alpha_mask=(int)-16777216, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw-yuv, format=(fourcc)AYUV, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ] and caps video/x-raw-yuv, format=(fourcc)I420, width=(int)352, height=(int)288, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30000/1001, interlaced=(boolean)false can not intersect " I don't know what happens (in terms of matching) when a property is present in one cap set and absent in another, but it seems we have a problem of format here. The only format in the output of autovideoconvert for video/x-raw-yuv is AYUV, whereas decklinksink requires I420. Is there some way (an element) to change the bit format of the stream?
That's pretty odd. A colorspace converter ought to convert to (almost) any YUV variant, including I420. What does happen if you use videorate ! colorspace ! videoscale instead of autovideoconvert ?
Well, I FINALLY managed to get my simpsons trailer :) to play through the decklinksink. That's the gst-launch line that worked fine: gst-launch-0.10 filesrc location=test/simpsons.mp4 ! decodebin2 ! videoscale ! ffmpegcolorspace ! interlace ! videorate ! decklinksink My procedure to "find out" this pipeline involved adding the elements, one by one, and seeing what decklinksink was complaining about in a GST_DEBUG=4 log. In my last "rational" attempt, I tried to use: ! videorate ! videoscale ! ffmpegcolorspace ! interlace ! to no avail. Then I decided to put videorate at the end of the pipeline "just for fun", and it worked! But, until now, I couldn't understand how changing the order of elements helped in this case? Can please some enlightened soul explain this to me? I'm planning to develop applications using GStreamer and decklinksink, so... Is there some more "pragmatic" way to find out about where to insert elements rather than trial and error? Last but not least, perhaps you guys can consider this bug as CLOSED now :)
It seems odd that the ordering should matter here. It looks like a bug. Out of curiosity, why is interlace needed ? Does decklinksink really need interlaced video ?
My "rationalization" as for why the ordering matters is that the videorate element works by frame dropping and duplication. So, when videorate is inserted too "early" in a complex pipeline, the further processing by other elements cause "deviations" in the timestamps of the frames that break the very strict framerate requirement of decklinksink... Anyways, I also think it shouldn't matter, and it would be nice to hear the opinion of a more experienced GStreamer developer about this :) Regarding interlace, YES, decklinksink does require interlaced video... the property "interlaced=true" is there, in the FIXED caps set of decklinksink's sink pad.
Why is this a blocker bug? Changing to normal.
More than three years without anything happening, let's close this. In the mean time, GStreamer 1.x was released, and the decklink elements have been pretty much rewritten from scratch and should generally work nicely now. Please re-open or file new bugs if there's still an issue with the decklink elements in git master, thanks!