After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 550613 - [mpegtsmux] Add support for ac3/dts/lpcm
[mpegtsmux] Add support for ac3/dts/lpcm
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
git master
Other Linux
: Normal normal
: 0.10.10
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2008-09-03 06:53 UTC by Sebastian Dröge (slomo)
Modified: 2009-05-10 21:55 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Initial, basic support for extended audio (3.79 KB, patch)
2008-11-20 17:26 UTC, vanista
committed Details | Review
Implementation of ac3_stream_descriptor (5.60 KB, patch)
2008-12-12 22:24 UTC, vanista
none Details | Review
Corrected implementation of ac3_stream_descriptor and related info (6.99 KB, patch)
2008-12-13 19:49 UTC, vanista
committed Details | Review

Description Sebastian Dröge (slomo) 2008-09-03 06:53:46 UTC
Hi,
mpegtsmux should get support for ac3/dts/lpcm audio. libtsmux has some stubs for this already that just need to be implemented ;)
Comment 1 vanista 2008-10-27 14:52:11 UTC
Has anyone started working on this yet?
I will need support for PCM soon, so I might implement it myself.
Comment 2 Sebastian Dröge (slomo) 2008-10-27 16:11:26 UTC
You mean LPCM? This is not the same as normal PCM which is not supported by mpeg-ts.

If you add support for LPCM adding the two other's should be more or less trivial (all three are put the same way into PES packets)
Comment 3 vanista 2008-10-27 17:51:36 UTC
I see... The requirement is for uncompressed audio so I guess it'll be LPCM. I haven't found a PCM to LPCM encoder yet, know of any?
Comment 4 vanista 2008-11-20 17:26:57 UTC
Created attachment 123122 [details] [review]
Initial, basic support for extended audio

This patch simply enables muxing the extended audio formats. I don't have an LPCM encoder yet so I've just tested AC3 using the following pipeline :

gst-launch-0.10 audiotestsrc num-buffers=100 ! ffenc_ac3 ! mpegtsmux ! mpegtsdemux ! ffdec_ac3 ! alsasink

Information about LPCM is very cryptic. I got my hands the AES3-2003 standard and I've analyzed all the code I found related to LPCM handling (mpegpsdemux, mpegtsdemux, dvdlpcmdec...), there are lots of inconsistencies between the spec and the code. I believe LPCM packetized in PES has its own standard, and I can't find anything official about it.

Also the mpegtsdemux (not sure if I should use it as reference), has raw LPCM caps which provide all parameters needed to decode (as opposed to audio/x-private1-lpcm which includes a header to be parsed by the decoder). If this is accurate, then the PES must contain all parameters somehow, but I've checked the code in the demux and there's no such special handling of the stream.

I'll try to find a valid TS stream which contains LPCM to clarify this.
Comment 5 Sebastian Dröge (slomo) 2008-11-24 10:45:59 UTC
2008-11-24  Sebastian Dröge  <sebastian.droege@collabora.co.uk>

        Patch by: vanista <vanista at gmail dot com>

        * gst/mpegtsmux/mpegtsmux.c: (mpegtsmux_create_stream):
        * gst/mpegtsmux/tsmux/tsmuxstream.c: (tsmux_stream_new):
        Add initial support for muxing AC3/DTS/LPCM into MPEG TS.
        Fixes bug #550613.
Comment 6 Christian Fredrik Kalager Schaller 2008-11-24 15:30:46 UTC
Thanks for doing this patch Vanista, wanted AC3 support for MPEG TS for a long while, going to test if it works on my PS3 this evening :)

Btw, do you have a full name we can put into the credits for our next release notes?
Comment 7 Christian Fredrik Kalager Schaller 2008-11-25 10:05:19 UTC
I tested a MPEG TS last night which included h264 video and AC3 audio on my PS3. The PS3 is not able to recognize the audio stream. Totem-GStreamer and Mplayer where both able to play back the file, but VLC also seemed to have issues with the audio. Vanista would you mind looking at what the problem could be?
Comment 8 Christian Fredrik Kalager Schaller 2008-11-25 10:35:38 UTC
Googling a bit it seems the reason the audio track is not visible to the PS3 might be because the PS3 expects it to be in the Blu-Ray format inside the MPEG TS file, more info about Blu-Ray format here:
http://forum.doom9.org/showthread.php?t=125447
Comment 9 vanista 2008-11-25 13:46:48 UTC
| Btw, do you have a full name we can put into the credits for our next release
notes?
> Sorry, must contribute with a pseudonym for now.

I'm a bit on and off this project, but I should get a PS3 or stb some time soon to validate the outputs.
I still haven't found the exact specifications for muxing private streams. I only found the stream ids which need to be set. I wouldn't be surprised if these needed some kind of codec data. I'll have to check if the AC3 enc has some in its caps.

Christian, concerning bluray format, did you enable the m2ts argument for the muxer? Also, was your PS3 able to demux and play video? .. is it only AC3 which doesn't decode?


Comment 10 Christian Fredrik Kalager Schaller 2008-11-25 14:20:01 UTC
Yeah, the video played fine, it was just the audio AC3 track that was not recognized. (I also tested creating a file with mp3 audio to verify, and that the PS3 handled fine).

Comment 11 vanista 2008-12-05 21:41:04 UTC
Still haven't found the specs, but I found a bunch of samples which we could analyze to figure out what's missing.

Christian, I still don't have any hardware to validate, can you test a couples samples which have AC-3 and tell me if they work on your PS3 please?

Samples coming from

  http://x264.nl/h.264.samples/

I know this one has AC-3 and I can play it with mpegtsdemux ! ffdec_ac3

http://x264.nl/h.264.samples/force.php?file=./jan.2008/ProSieben.HD.astra1.19e.h.264.1920.1080.tandberg.03.jan.2008.ts

Comment 12 Christian Fredrik Kalager Schaller 2008-12-06 12:55:02 UTC
I tested some of these files with some interesting results. The BBC file from x264.nl worked perfectly. It was recognized by the PS3 as a valid file right away, in fact it was the only file to be so. The AC3 in that file is stereo only though.
http://x264.nl/h.264.samples/force.php?file=./bbc.hd.ts

The skymovies clip on the other hand was the only clip the PS3 was totally unable to recognize and play.
http://x264.nl/h.264.samples/force.php?file=./sky.movies.9.hd.ts

The ProSieben clip you linked wasn't automatically recognized by the PS3, in fact it was originally listed as Unsupported Data, but once I tried asking the PS3 to try playing it anyway it did play it fine, including full surround sound.

The astra.hd clip had the same behaviour as the ProSieben clip, ie. not recognized as playable at first, but played once I tried 'forcing' the PS3. That file is stereo only. 
http://x264.nl/h.264.samples/force.php?file=./astra.hd.ts

So all in all I guess the bbc.hd.ts file is the one we need to mimick to create files the PS3 really likes, hopefully the fact that it contained stereo and not surround sound is not related to the PS3 instantly recognizing it as supported.
Comment 13 vanista 2008-12-08 15:50:59 UTC
Thanks for the investigation! I'm analyzing the BBC sample to see what we're missing. I haven't found anything conclusive yet, I'm checking for the adaptation field.

We're probably going to rent a PS3 for a couple days to get this going. Can you tell me how you do your tests on the PS3? Do you have to burn a disk every time or is it possible to access files on a network share?

Comment 14 Christian Fredrik Kalager Schaller 2008-12-08 16:08:39 UTC
I have one of the early models so I got a memory card reader in my PS3, but I could also be using a USB penndrive. I just put the files into a directory on the penndrive called 'Videos' and the PS3 detects them automatically and put the files into its playlist.
Comment 15 vanista 2008-12-09 19:54:37 UTC
I finaly have a PS3 to do some testing! However I'm nowhere close to the results you get. Everything I mux in TS or m2ts can't be read by the PS3.

It's a model CECHH01 made in march 2008.

Can you give me a pipeline you use for producing your TS outputs?

I use ffenc_libx264 and ffenc_ac3 into mpegtsmux with or without m2ts-mode=true

Comment 16 vanista 2008-12-09 22:22:03 UTC
I've done some additional tests and here's the compatibility matrix I get

ts container not supported at all
m2ts container supported

MPEG-2 video plays
MPEG-4 video identified as MPEG-1 and doesn't play
H-264 video not recognized

MP3 audio plays
AAC audio not recognized
AC3 audio not recognized


The Kill Bill sample does play on the PS3, the video is id as AVC and audio as Dolby Digital. If I passthrough this stream and remux it with our mpegtsmux, using the same PIDs, the video plays* but the audio isn't identified.

* for some reason the video stutters... there's probably some codec data that gets mixed up


Comment 17 Christian Fredrik Kalager Schaller 2008-12-10 11:31:26 UTC
Hi Vanista,
The pipeline I used was this one:
gst-launch-0.10 filesrc location=/home/cschalle/Videos/starwars.mkv ! decodebin2 name="decode" decode. ! ffmpegcolorspace ! x264enc ! queue ! mpegtsmux name=mux ! gnomevfssink location=file:///tmp/test6.ts decode. ! audioconvert ! faac ! queue ! mux.

The Kill Bill experience you relate seem to be the same I got when using this pipeline, ie working video but no audio. 

You said you got very different results than me, do that include the results I reported for the testclips from x264.nl?

Be aware that Sony has updated their firmware quite often and added more media support etc., so make sure your PS3 is internet connected so it can grab the latest firmware update.
Comment 18 vanista 2008-12-12 22:24:17 UTC
Created attachment 124557 [details] [review]
Implementation of ac3_stream_descriptor

Finally found the specifications! It's all in ATSC A/52-2001, Annex A : AC-3 Elementary Streams in the MPEG-2 Multiplex

I've hardcoded a few values to simplify the implementation based on what FFMPEG uses as default encoding parameters. I set the proper or most compatible values for the given caps. Additionally the bitrate is required in this descriptor, so we need to pass the parameter through filtered caps. Maybe this whole structure should be provided by the encoder as codec-data, but I'm not sure if the structure is standard in other containers (AVI, 3gpp...)

gst-launch-0.10 mpegtsmux name=mux m2ts-mode=true ! filesink location=output.m2ts audiotestsrc num-buffers=10 ! audio/x-raw-int, rate=48000 ! audioconvert ! audio/x-raw-int, channels=2 ! ffenc_ac3 bitrate=256000 ! audio/x-ac3, bitrate=256 ! mux. videotestsrc num-buffers=10 ! x264enc cabac=false ! mux.


http://www.dolby.com.cn/uploadedFiles/Chinese_CN/Consumer/Dolby_Technologies/Dolby_Digital/Dolby_Digital_Links/a_52a.pdf

http://www.scte.org/documents/standards/approved/SCTE542006.pdf
Comment 19 vanista 2008-12-13 19:49:08 UTC
Created attachment 124602 [details] [review]
Corrected implementation of ac3_stream_descriptor and related info

Finally got it!

This version is tested successfully on the PS3. Audio is properly detected as Dolby Digital and played! The video is still not identified correctly, but at least it plays well...

This patch is against HEAD so ignore the previous one.
Comment 20 vanista 2008-12-13 19:58:00 UTC
My time is pretty much exhausted for this feature, someone will have to continue the research for LPCM and DTS, and also investigate why the files are not recognized right away by the PS3.
My guess is, as for AC-3, most of it comes from the descriptors and the Program Map Table.
Good luck and thanks Christian for the help with the tests :-)
Comment 21 Christian Fredrik Kalager Schaller 2008-12-14 00:09:11 UTC
Re-oppening this bug since we have a new patch attach. 

Vanista I did some testing here too and was able to reproduce your results more or less. I noticed that if the file was stereo only I didn't need to set the bitrate for the AC3 and it still worked (although also with the file not being recognized by the PS3 as supported). To get a file with 5+1 channels working I needed to specify the bitrate.

Interestingly enough it also turns out VLC is unable to play the generated files, but both GStreamer and mplayer are able to play them.

So we are closer. I will leave it up to others to decide if this patch should be merged or not, but to me it seems like it takes MPEG TS muxing forward a bit, with things keeping on working in GStreamer and mplayer, working better in the PS3, and still not working at all under VLC :)

Thanks for your hard work and effort so far Vanista, even if you don't have time to take it all the way over the finish line it does at least take us a good part of the way.
Comment 22 vanista 2008-12-15 15:46:27 UTC
Good news, I have an extension for polishing this feature!

Since the AC-3 part is pretty much complete and functional, I've opened a new bug for the PS3 specific issue : Bug 564622
Comment 23 Sebastian Dröge (slomo) 2009-01-05 11:28:37 UTC
2009-01-05  Sebastian Dröge  <sebastian.droege@collabora.co.uk>

        Patch by: vanista <vanista at gmail dot com>

        * gst/mpegtsmux/mpegtsmux.c: (mpegtsmux_create_stream):
        * gst/mpegtsmux/tsmux/tsmux.c: (tsmux_write_pmt):
        * gst/mpegtsmux/tsmux/tsmuxstream.c: (tsmux_stream_new),
        (tsmux_stream_write_pes_header), (tsmux_stream_get_es_descrs):
        * gst/mpegtsmux/tsmux/tsmuxstream.h:
        Improve muxing of AC3/h264 streams a bit. Fixes bug #550613.
Comment 24 Christian Fredrik Kalager Schaller 2009-05-10 21:55:35 UTC
Just a comment to this bug for posterity. In response to my own comment about having to set bitrate in #21. Seems the PS3 AC3 decoder simply refuses low bitrate files. I am assuming it is very targeted towards the Blu-Ray spec, so once I tested with bitrate values 'relevant' to a Blu-Ray they all worked fine. Don't know where the bottom is, but so far I verified that files with a bitrate between 256 and 640 (640 is Blu-Ray official highest bitrate for AC3) works fine.