After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 618916 - [playbin2] bad text-sink support (vobsub decoding, caps information)
[playbin2] bad text-sink support (vobsub decoding, caps information)
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-base
unspecified
Other Linux
: Normal normal
: NONE
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2010-05-17 18:48 UTC by Andreas Frisch
Modified: 2013-05-11 17:19 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
testcase, call with file uri as parameter (10.76 KB, text/x-csrc)
2010-05-17 21:58 UTC, Andreas Frisch
Details
GST_DEBUG_NO_COLOR=1 GST_DEBUG=*:2,*basesink*:5,*appsink*:4,*sub*:4 enigma2 >vobsub_latency.log 2>&1 (141.85 KB, application/x-bzip)
2010-07-08 15:07 UTC, Andreas Frisch
Details

Description Andreas Frisch 2010-05-17 18:48:48 UTC
playbin2 doesn't support automatic pluggin of a dvdsubdec element in case the custom text-sink wants video/x-raw-rgb input caps.

furthmore, there is no way of querying subtitle "types" at the beginning of the playback. on GST_MESSAGE_ASYNC_DONE, all subtitle streams are PRESENT but may have no caps set yet. using notify::caps only tells me about the one selected stream but not about the others, text-tags only hold information about the language.
in my application, i want to display a list of all available subtitle streams, including information about their type (whether it's plain-text, ssa/ass/pango or vobsubs etc.)


dvdsubdec is not being plugged by playbin2 even when patching its rank to be marginal instead of none.
Comment 1 Sebastian Dröge (slomo) 2010-05-17 18:56:41 UTC
Could you attach some sample code and maybe a sample file for this? Makes it easier to fix ;)
Comment 3 Andreas Frisch 2010-05-17 21:56:51 UTC
root@dm8000:~# /testProgs/testcase_618916_subtitle_info file:///hdd/movie/mkv/Final.Fantasy.VII.Advent.Children-001.mkv              
gstCBsubtitleCAPS:: signal::caps callback
eServiceMP3::async-done - 1 video, 3 audio, 4 subtitle
eServiceMP3::audio stream=0 codec=AC-3 audio language=de
eServiceMP3::audio stream=1 codec=AC-3 audio language=en
eServiceMP3::audio stream=2 codec=AC-3 audio language=ja
eServiceMP3::subtitle stream=0 language=de, type=text/x-pango-markup
eServiceMP3::subtitle stream=1 language=de, type=und
eServiceMP3::subtitle stream=2 language=en, type=text/x-pango-markup
eServiceMP3::subtitle stream=3 language=en, type=und
** Message: received a subtitle at position 0:00:00.044530000, running_time 0:00:00.044778000
** Message: received a subtitle at position 0:00:02.011678000, running_time 0:00:02.011791000
00000000 (0x4a7a28): 41 6c 6c 65 2c 20 64 69 65 20 64 69 65 73 65 20  Alle, die diese 
00000010 (0x4a7a38): 57 65 6c 74 20 6c 69 65 62 74 65 6e 2e           Welt liebten.   
** Message: received a subtitle at position 0:00:05.250585000, running_time 0:00:05.250665000
00000000 (0x64d980): 41 6c 6c 65 2c 20 64 69 65 20 64 69 65 20 67 65  Alle, die die ge
00000010 (0x64d990): 6d 65 69 6e 73 61 6d 65 20 5a 65 69 74 20 0a 69  meinsame Zeit .i
00000020 (0x64d9a0): 6e 20 69 68 72 65 6e 20 48 65 72 7a 65 6e 20 74  n ihren Herzen t
00000030 (0x64d9b0): 72 61 67 65 6e 3a                                ragen:          
** Message: received a subtitle at position 0:00:08.250605000, running_time 0:00:08.250687000
00000000 (0x651400): 44 69 65 73 65 20 52 65 75 6e 69 6f 6e 20 69 73  Diese Reunion is
00000010 (0x651410): 74 20 65 75 63 68 20 67 65 77 69 64 6d 65 74 2e  t euch gewidmet.

expected: subtitle streams 1 and 3 discovered to be text/plain instead of undefined (no caps)
Comment 4 Andreas Frisch 2010-05-17 21:57:34 UTC
root@dm8000:~# /testProgs/testcase_618916_subtitle_info file:///hdd/movie/mkv/BrokenPlayback2.mkv              
eServiceMP3::async-done - 1 video, 3 audio, 7 subtitle
eServiceMP3::audio stream=0 codec=AC-3 audio language=de
eServiceMP3::audio stream=1 codec=AC-3 audio language=en
eServiceMP3::audio stream=2 codec=AC-3 audio language=fr
eServiceMP3::subtitle stream=0 language=de, type=und
eServiceMP3::subtitle stream=1 language=fr, type=und
eServiceMP3::subtitle stream=2 language=de, type=und
eServiceMP3::subtitle stream=3 language=fr, type=und
eServiceMP3::subtitle stream=4 language=de, type=und
eServiceMP3::subtitle stream=5 language=fr, type=und
eServiceMP3::subtitle stream=6 language=en, type=und
ERROR: from element /GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/GstMatroskaDemux:matroskademux0: GStreamer encountered a general stream error.
Additional debug info:
matroska-demux.c(5241): gst_matroska_demux_loop (): /GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/GstMatroskaDemux:matroskademux0:
stream stopped, reason not-negotiated
** Message: stopping

expected: discover all subtitle streams to be vobsubs and automatically plug dvdsubdec
Comment 5 Andreas Frisch 2010-05-17 21:58:26 UTC
Created attachment 161287 [details]
testcase, call with file uri as parameter
Comment 6 Sebastian Dröge (slomo) 2010-05-18 06:23:05 UTC
Ok, I found the problem... I'll fix this later

Thanks for the tests :)
Comment 7 Sebastian Dröge (slomo) 2010-05-18 06:48:54 UTC
Actually that's not a good idea, after plugging dvdsubdec playbin2 has no way to detect that video/x-raw-rgb stream as subtitle stream and will not link it to the subtitle sink.

Anyway, this "fixes" this:

commit 9d7538247ff1bf9841b53eeb71ddc47f2c662415
Author: Sebastian Dröge <sebastian.droege@collabora.co.uk>
Date:   Tue May 18 08:45:52 2010 +0200

    playbin2: Don't put "raw" subtitle types in the raw caps for decodebin2
    
    We handle them from the autoplug-continue signal, where the caps supported
    by the subtitle sink or overlay are known already.


It still won't work in your example though. No idea how to fix that but I guess you should plug the dvdsubdec inside your sink again (just connect to notify::caps, check the caps and if it's subpicture caps you put a dvdsubdec before your appsink and otherwise link it directly).
Comment 8 Sebastian Dröge (slomo) 2010-05-24 08:02:16 UTC
One thing how to fix this would be, to add a "subpicture=true" field to the video/x-raw-* caps created by dvdsubdec and check this in playbin2 to differentiate between subpictures and real raw video.
Comment 9 Andreas Frisch 2010-07-08 15:06:21 UTC
here's another issue having to do with subtitles:

when i have a suburi with an srt file plus an inline vobsub stream in my mkv then the vobsub buffers are handed over with a latency. the latency varies per each playback from between 2 and 6 seconds but is relatively constant then. if the suburi is omitted, then the buffers show up at expected times.
the latency is processed by reading the actual play pts of the hardware h264 decoder and comparing it to the pipeline running time
delay is manually being done with a timer in my app that renders the subtitles at the expected pts

log to be attached
Comment 10 Andreas Frisch 2010-07-08 15:07:05 UTC
Created attachment 165483 [details]
GST_DEBUG_NO_COLOR=1 GST_DEBUG=*:2,*basesink*:5,*appsink*:4,*sub*:4 enigma2 >vobsub_latency.log 2>&1
Comment 11 Andreas Frisch 2010-07-26 10:20:32 UTC
okay we found out that the latency only occures if there are vob subtitle streams AND AT THE SAME TIME a SRT suburi. this is probably a very rare condition in the field so let's forget about the delay.

another thing that I still desperatly long for is correct subtitle type detection right after preroll and not just ...SOME TIME as soon as the demux starts seeing the first buffers. and preferably ALL sub streams, not just the currently selected one.
Comment 12 Sebastian Dröge (slomo) 2010-07-26 12:38:40 UTC
(In reply to comment #11)
> okay we found out that the latency only occures if there are vob subtitle
> streams AND AT THE SAME TIME a SRT suburi. this is probably a very rare
> condition in the field so let's forget about the delay.

We talked about that on IRC already. You need to expose the hardware clock and use that in the pipeline for everything.

> another thing that I still desperatly long for is correct subtitle type
> detection right after preroll and not just ...SOME TIME as soon as the demux
> starts seeing the first buffers. and preferably ALL sub streams, not just the
> currently selected one.

Well, you could typefind the external subtitle files and the demuxer can only know the type when it's currently pre-rolling. What exactly do you need?
Comment 13 Andreas Frisch 2010-07-26 13:17:25 UTC
unfortunately the clock thing is very tricky with the hardware... we had already tried that before but without any success so far.

typefind is not gonna help because this is about an mkv container. so far all information i have at preroll time is that there are 4 subsomething streams... i can't even determine and display in our gui whether or not the streams will be displayable at all since they might be anything. it would just be great if i knew right from the beginnen whether a subtitle stream is actually utf-8 or SSA/ASS text or vob or pgs subpictures. that way i could setup our renderer accordingly and display useful information in the gui.
Comment 14 Sebastian Dröge (slomo) 2010-07-26 13:44:11 UTC
(In reply to comment #13)
> unfortunately the clock thing is very tricky with the hardware... we had
> already tried that before but without any success so far.

Right, the delay will magically disappear if you get this right ;)

> typefind is not gonna help because this is about an mkv container. so far all
> information i have at preroll time is that there are 4 subsomething streams...
> i can't even determine and display in our gui whether or not the streams will
> be displayable at all since they might be anything. it would just be great if i
> knew right from the beginnen whether a subtitle stream is actually utf-8 or
> SSA/ASS text or vob or pgs subpictures. that way i could setup our renderer
> accordingly and display useful information in the gui.

Well, take a look at the demuxer pads and get their caps. They will have the ASS, SRT, whatever caps set from the beginning for most demuxers from the beginning.
Comment 15 Tim-Philipp Müller 2012-10-03 23:34:09 UTC
This bug seems to have moved from one issue to the next, it's a bit confusing. May I suggesting closing it an filing new bugs for any specific issues that remain?

I believe the CAPS event in 1.0 should take care of at least the original complaint, since it doesn't require buffer flow to announce the stream type to downstream elements.
Comment 16 Tobias Mueller 2013-03-29 13:29:58 UTC
closing as per comment 15.
Comment 17 Tim-Philipp Müller 2013-05-11 17:19:47 UTC
FIXED -> OBSOLETE, since there's no specific commits we know of that fixed it.