GNOME Bugzilla – Bug 704946
playsink: Forces custom sinks to have sync=true
Last modified: 2013-07-30 19:08:06 UTC
Created attachment 250222 [details] Example of playbin+funnel+appsink This is causing problems for me in WebKit: https://bugs.webkit.org/show_bug.cgi?id=103771 I'm trying to use a funnel in playbin to mix all of the text streams, then get them out with an appsink. The problem is that the order of the buffer is fairly random if they show up at the same time (the funnel could send multiple buffers from one stream before the first buffer in another stream), and I don't get buffers at my appsink until it's time to present them (meaning any buffer that was out of order may show up too late to be displayed, because it's waiting for another later buffer to be displayed). An example is attached, run it on this file (download it, this issue happens less often when streaming): https://github.com/WebKit/webkit/raw/master/LayoutTests/media/content/counting-subtitled.m4v Then using the attached file: gst-git python3 playbin-funnel.py file://$PWD/counting-subtitled.m4v I get output like this: Got sample at 00.33: 00.33 --> 00.67 Two Got sample at 00.33: 00.33 --> 00.67 Deux Got sample at 00.67: 00.67 --> 01.00 Trois Got sample at 01.00: 01.00 --> 01.33 Quatre Got late sample at 01.00: 00.67 --> 01.00 Three Got sample at 01.33: 01.33 --> 01.67 Cinq Got sample at 01.67: 01.67 --> 02.00 Six Got sample at 02.00: 02.00 --> 02.33 Seven Got late sample at 01.33: 01.00 --> 01.33 Four Got late sample at 01.67: 01.33 --> 01.67 Five Got late sample at 02.00: 01.67 --> 02.00 Six Got sample at 02.00: 02.00 --> 02.33 Seven Got sample at 02.33: 02.33 --> 02.67 Eight Got sample at 02.67: 02.67 --> 03.00 Nine Got sample at 03.00: 03.00 --> 03.33 Ten Got sample at 03.33: 03.33 --> 03.67 Eleven Got sample at 03.67: 03.67 --> 04.00 Twelve Got sample at 04.00: 04.00 --> 04.33 Thirteen Got sample at 04.33: 04.33 --> 04.67 Fourteen Got sample at 04.67: 04.67 --> 05.00 Fifteen Got sample at 05.00: 05.00 --> 05.33 Sixteen Got sample at 05.33: 05.33 --> 05.67 Seventeen Got sample at 05.67: 05.67 --> 06.00 Eighteen Got late sample at 02.67: 02.33 --> 02.67 Huit Got late sample at 03.00: 02.67 --> 03.00 Nine Got late sample at 03.33: 03.00 --> 03.33 Ten Got late sample at 03.67: 03.33 --> 03.67 Onze Got late sample at 04.00: 03.67 --> 04.00 Douze Got late sample at 04.33: 04.00 --> 04.33 Thirteen Got late sample at 04.67: 04.33 --> 04.67 Quatorze Got late sample at 05.00: 04.67 --> 05.00 Quinze Got late sample at 05.33: 05.00 --> 05.33 Seize Got late sample at 05.67: 05.33 --> 05.67 Seventeen Got sample at 05.67: 05.67 --> 06.00 Dix-huit Got sample at 06.00: 06.00 --> 06.33 Nineteen Got sample at 06.33: 06.33 --> 06.67 Vingt Got sample at 06.67: 06.67 --> 07.00 Vingt et un Got sample at 07.00: 07.00 --> 07.33 Vingt-deux Got sample at 07.33: 07.33 --> 07.67 Vingt-trois Got late sample at 06.33: 06.00 --> 06.33 Nineteen Got late sample at 06.67: 06.33 --> 06.67 Twenty Got late sample at 07.00: 06.67 --> 07.00 Twenty one Got late sample at 07.33: 07.00 --> 07.33 Twenty two Got sample at 07.34: 07.33 --> 07.67 Twenty three Got sample at 07.67: 07.67 --> 08.00 Twenty four Got sample at 08.00: 08.00 --> 08.33 Twenty five Got sample at 08.33: 08.33 --> 08.67 Twenty six Got sample at 08.67: 08.67 --> 09.00 Twenty seven Got sample at 09.00: 09.00 --> 09.33 Twenty eight Got sample at 09.33: 09.33 --> 09.67 Twenty nine Got late sample at 08.00: 07.67 --> 08.00 Vingt-quatre Got late sample at 08.33: 08.00 --> 08.33 Vingt-cinq Got late sample at 08.67: 08.33 --> 08.67 Vingt-six Got late sample at 09.00: 08.67 --> 09.00 Vingt-sept Got sample at 09.67: 09.67 --> 10.80 Thirty Got late sample at 09.33: 09.00 --> 09.33 Vingt-huit Got late sample at 09.67: 09.33 --> 09.67 Vingt-neuf Got sample at 09.67: 09.67 --> 10.00 Trente EOS "late" in this output means "so late that the cue won't be displayed at all" (current_time >= end_time).
And before you ask: appsink.set_property("sync", False)
In playsink there's code that forces the text-sink to "sync=true" unfortunately. If you change that, does it work better? I'm not sure I understand the output of your script correctly
gstplaysink.c:2287, in gen_text_chain
Might want to have some kind of funnel-like with a slight delay that forces the buffers to be time-ordered ?
You mean a synchronizing funnel? Sure, why not ;)
But you end up with the same probleme as any synchronizing unknown-gap-friendly muxer, you must add latency and sync to the clock..
Removing that section in playsink seems to fix it. Would it cause problems if it was removed? My assumption is that custom sinks should have their properties set by their creators, but changing that could cause problems if existing programs depend on it. I'm having another problem where the position is suddenly wrong, but I'll create a different bug for that (I think it's using the appsink for the position for some reason).
(In reply to comment #7) > Removing that section in playsink seems to fix it. Would it cause problems if > it was removed? My assumption is that custom sinks should have their properties > set by their creators, but changing that could cause problems if existing > programs depend on it. I think removing that is not a good idea, as you say, because applications might depend on that already. As an alternative maybe just put your appsink into a GstBin and use that as a text-sink... Note that you can won't get buffers properly interleaved from the different streams btw... you can get, as you noticed probably, a bunch of buffers for stream 1, then another number of buffers for stream 2 that are for the same positions. > I'm having another problem where the position is suddenly wrong, but I'll > create a different bug for that (I think it's using the appsink for the > position for some reason). Yes, quite possible. You could prevent that by implement a query function on your sink and not answering the position query there.
(In reply to comment #8) > As an alternative maybe just put your appsink > into a GstBin and use that as a text-sink... > You could prevent that by implement a query function on > your sink and not answering the position query there. I tried a couple things, and ended up creating an element to extend appsink, overriding the position and duration queries (to return false), and overriding the "sync" property to do nothing. Putting the element in a bin didn't work because playsink actually looks into the bin to find elements with the properties it wants to set (gst_play_sink_find_property_sinks). I'm marking this as "NOTABUG" because this is by design apparently, and it can't easily be fixed now.