After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 657809 - [tsdemux] report audio stream type (for visually impaired audio-description, receiver-mixed supplementary channel, ...)
[tsdemux] report audio stream type (for visually impaired audio-description, ...
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
git master
Other Linux
: Normal enhancement
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2011-08-31 11:51 UTC by Tvrtko Ursulin
Modified: 2018-11-03 13:08 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
mpegtsmux: send audio type as a tag (3.65 KB, patch)
2011-09-01 09:14 UTC, Vincent Penquerc'h
none Details | Review
Patch to send audio type tag from tsdemux (1.79 KB, patch)
2012-08-10 13:39 UTC, Tvrtko Ursulin
none Details | Review
add theUNSELECT stream flag to non default streams (1.46 KB, patch)
2014-11-18 15:12 UTC, Vincent Penquerc'h
none Details | Review

Description Tvrtko Ursulin 2011-08-31 11:51:13 UTC
Playing a MPEG transport stream from disk with a simplest gst-launch invocation like gst-launch-0.10 playbin2=file://... and playback starts without sound in a certain percentage of invocations.

Test stream (~35Mb) available on request.

If sync=false on the output alsasink then sound always work but with a sync issue.

May be a core issue and not specific to MPEG TS since I am seeing similar problem when playing V4L2 video with ALSA sound.
Comment 1 Vincent Penquerc'h 2011-08-31 13:59:18 UTC
As a data point, using this command line, whether sync is set to 1 or 0, I sometimes get sound and sometimes not, so it seems not directly related to sync, though sync might interact with some other timing issue.

gst-launch playbin2 uri=file://.../c5-sometimes-no-sound.ts  audio-sink='alsasink sync=0'
Comment 2 Tvrtko Ursulin 2011-08-31 14:18:04 UTC
You are right - originally I did ten invocations of each test and had 100% incidence of sound where sync was off. But when I do more I see the same thing as you.
Comment 3 Vincent Penquerc'h 2011-08-31 14:24:24 UTC
This stream has two audio tracks. One of them seems silent as far as I can tell, and is tagged "visually impaired commentary". I guess it will have audio on selected parts of the stream only, where it makes sense and commentary of the program is available. 

The random selection of a default stream is a known long standing bug.
See for instance https://bugzilla.gnome.org/show_bug.cgi?id=607466, I'm pretty sure there's a much older bug about it I can't find, but basically it's a race between threads, and the one selected is (IIRC) the first one to expose its source pad.
Comment 4 Tvrtko Ursulin 2011-08-31 14:52:39 UTC
Are there any options to work around this?

Like teaching playbin2 about something, or exposing something from it?

Or only option will be to build the pipeline completely manually? At least I assume streams are then selectable?
Comment 5 Vincent Penquerc'h 2011-08-31 15:00:00 UTC
You can select which streams to use with playbin2.
The issue is that they might get numbered differently at each run.
I'm looking at whether there's a foolproof way to tell at the moment.
Comment 6 Vincent Penquerc'h 2011-09-01 09:14:47 UTC
Created attachment 195369 [details] [review]
mpegtsmux: send audio type as a tag

Audio type may be used to tag streams as "hearing impaired",
"visual impaired commentary", etc.
This tag should help determine which is the main track if
there are several.
Comment 7 Vincent Penquerc'h 2011-09-01 09:21:39 UTC
The above patch exposes the internal type of the stream as a tag on the relevant source pad. This can then be checked by the playbin2 using code, with something like this:

    def select_audio(self):
        idx=0
        n_audio = self.player.get_property("n-audio")
        print "Considering %d audio streams:" % (n_audio)
        selected = None
        while idx<n_audio:
          language = None
          audio_type = None
          taglist=self.player.emit("get-audio-tags", idx)
          if taglist != None and ("language-code" in taglist):
              language = taglist["language-code"]
          if taglist != None and ("mpeg-audio-type" in taglist):
              audio_type = taglist["mpeg-audio-type"]
          print "Stream %d has language %s, type %d" % (idx, language, audio_type)
          if audio_type == 0:
              selected = idx
          idx=idx+1
        if selected != None:
            print "Selecting audio stream %d (from %d)" % (selected, self.player.get_property("current-audio"))
            self.player.set_property("current-audio", selected)


This can then be called when you get tag updates:

    def on_message(self, bus, message):
        t = message.type
        src = message.src.get_name()
        if t == gst.MESSAGE_TAG:
            self.select_audio()

You can pass the tags to that function here too to avoid getting them again, in case you do not need to call it anywhere you do not have the tags.

This python snippets were coded on your sample Python player, and tested with the stream you linked to. Note that the main stream is tagged with 0, which the spec says is "undefined" (found in 2.6.19 of the spec).

For convenience, I've added code to display the language too, you may want to favor selecting the language specified in LANG, if any, etc.
Comment 8 Tvrtko Ursulin 2011-09-01 13:20:40 UTC
I have tested it with the patch and this Python code and can confirm that it indeed works around the issue.

Thanks a lot, especially for this Python example!
Comment 9 Vincent Penquerc'h 2011-09-01 13:40:29 UTC
You're welcome :)

For GStreamer committers: this patch to mpegtsdemux isn't meant to be pushed, it's mpeg specific, and ideally we'd need a common way for demuxers to expose this kind of thing, either priority based or "semantic" based.
Comment 10 Vincent Penquerc'h 2012-01-18 12:52:16 UTC
Setting as new. Would be good to get a generic solution, possibly via default tracks and/or track priority.
Comment 11 Tvrtko Ursulin 2012-06-14 09:01:47 UTC
This is still broken with the new TS demuxer. I am yet to try porting this patch there.
Comment 12 Edward Hervey 2012-06-29 14:24:45 UTC
tsdemux does push out language code tags fwiw
Comment 13 Tvrtko Ursulin 2012-06-29 14:37:13 UTC
So the sample code from comment #7 should work? So far I only tested it with straight gst-launch. Will put this on my TODO list.
Comment 14 Vincent Penquerc'h 2012-06-29 14:39:55 UTC
Note that I also posted the mpeg audio type, which is probably not included in the language code tags Edward is mentioning, but would be needed to distinguish between two (eg) English tracks, a main one, and a commentary one.
Comment 15 Tvrtko Ursulin 2012-06-29 14:57:31 UTC
Vincent is right, which means there is insufficient data to select the right stream.

For the record these are the tags I see:
['language-code', 'audio-codec', 'nominal-bitrate', 'has-crc', 'channel-mode', 'minimum-bitrate', 'bitrate', 'maximum-bitrate']
Comment 16 Tvrtko Ursulin 2012-08-10 13:39:07 UTC
Created attachment 220884 [details] [review]
Patch to send audio type tag from tsdemux

This patch kind of works with the new tsdemuxer, however every time the situation triggers that the wrong audio track is selected, or in other words that the player has to set playbin2's 'current-audio' property to the right track, there is a very long delay (multiple second) where the playback stalls.
Comment 17 Edward Hervey 2013-06-07 11:12:34 UTC
I'm not sure an integer is the way to go for this tag.

There are other ways (for example via the component_descriptor component_type field) to indicate that a stream is audio-description, or for the hard of hearing, or supplemental-stream (to be mixed in with the main audio stream), ...).

Changing the title accordingly also.
Comment 18 Edward Hervey 2013-06-07 11:25:48 UTC
from discussion from slomo/__tim, we should at least set the GstStreamFlags correctly to indicate which streams should be selected/unselected by default

http://cgit.freedesktop.org/gstreamer/gstreamer/tree/gst/gstevent.h#n371

Maybe a flag could also be added to indicate it's a "supplementary" stream
Comment 19 Vincent Penquerc'h 2014-11-18 15:12:24 UTC
Created attachment 290919 [details] [review]
add theUNSELECT stream flag to non default streams

Something like this (using the existing flags) ?
Comment 20 GStreamer system administrator 2018-11-03 13:08:51 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/46.