GNOME Bugzilla – Bug 615740
aacparse: Add conversion support from/to ADIF/ADTS/RAW/LOAS stream-formats
Last modified: 2018-11-03 14:41:32 UTC
The FAAD plugin do not searches for LATM/LOAS format in the AAC stream. The chain function searches for the sync word (line 1061), but only for the ADTS syncword (gst_faad_sync method). My suggestion is: Adding a new parameter in the src caps of the plugin: transport{adts,latm} 1) if the transport parameted is set, the plugin must looking for the correct syncword (latm or adts syncword). 2) if the transport parameter is not set, the plugin look for: a) First the adts syncword for 17000 Bytes (thats a guess :) b) If no ADTS syncword was found, looking for LATM syncword. *) This approach is suggested in cases of continuous streams, such one that come from a live source, like a dvbsrc->mpegdemux for example.
Created attachment 158717 [details] LATM AAC sample
A little help, the VLC player can play the attached sample.
Hi, I'm working with André Dieb in the fix of this element. When we have a stable version we let you know.
Created attachment 160744 [details] [review] Proposed patch This patch adds LOAS/LATM support into gstfaad.
Hi guys, I've also been working on AAC code for GStreamer lately, and it involves LOAS. I've even posted to a mailing list a patch to extend aac typefind for LOAS streams. I would love to coordinate with you. My task at hand is a little different: I have LOAS stream, written by commercial encoder, that I have to mux to FLV using gstreamer with video that comes from another gstreamer pipeline. The hard part is demuxing LATM in LOAS into raw AAC packets on one hand and reading and storing in "codec_data" field in caps AudioSpecificConfig that comes with each AAC stream. The first task was to recognize LOAS stream as it comes from FIFO through simple filesrc. That's what my typefind patch is doing. Next is to frame the LOAS packets for further processing, that's what I suppose should be done by aacparse plugin which I'm currently working on. But I see that in your patch you're doing exacly the same thing: parsing LOAS and getting LATM out of it, along with AudioSpecificConfig. How about getting the parsing part out so both aacparse and gstfaad could use that without code duplication? Both plugins are in gst-plugins-bad module so it might be a little easier, but I'm relatively new to GStreamer development process, so I'd welcome suggestions from more experienced developers.
(In reply to comment #5) > Hi guys, > > I've also been working on AAC code for GStreamer lately, and it involves LOAS. > I've even posted to a mailing list a patch to extend aac typefind for LOAS > streams. I would love to coordinate with you. > > My task at hand is a little different: I have LOAS stream, written by > commercial encoder, that I have to mux to FLV using gstreamer with video that > comes from another gstreamer pipeline. The hard part is demuxing LATM in LOAS > into raw AAC packets on one hand and reading and storing in "codec_data" field > in caps AudioSpecificConfig that comes with each AAC stream. > > The first task was to recognize LOAS stream as it comes from FIFO through > simple filesrc. That's what my typefind patch is doing. > Next is to frame the LOAS packets for further processing, that's what I suppose > should be done by aacparse plugin which I'm currently working on. But I see > that in your patch you're doing exacly the same thing: parsing LOAS and getting > LATM out of it, along with AudioSpecificConfig. > > How about getting the parsing part out so both aacparse and gstfaad could use > that without code duplication? Both plugins are in gst-plugins-bad module so it > might be a little easier, but I'm relatively new to GStreamer development > process, so I'd welcome suggestions from more experienced developers. Sorry, I might have misunderstood: getting the LAOS/LATM parsing out of gstfaad into aacparse? Then one would use gstfaad only for decoding what faad supports (ADTS, raw). In case of different AAC profiles, user would use aacparse+faad? I'll gladly test any patch and help you if it is decided to go on with this.
(In reply to comment #6) > Sorry, I might have misunderstood: getting the LAOS/LATM parsing out of gstfaad > into aacparse? > > Then one would use gstfaad only for decoding what faad supports (ADTS, raw). In > case of different AAC profiles, user would use aacparse+faad? > > I'll gladly test any patch and help you if it is decided to go on with this. Yes, more or less that's what I'd like to have in the end: parsing of different AAC streams, so that other elements could use that, not only faad decoder. I'm working now on finalizing the aacparse patch so it gets LOAS/LATM right, but I'm not sure how it should handle multiple streams in LATM. If the output from aacparse is to be LOAS/LATM then there's no problem, but for raw AAC output that is needed by FLV should I create more src pads? Also, if an element like flvmux requires raw AAC stream should it be able to negotiate it with aacparse? That's also a case for faad, which might want to get raw AAC data from LATM: should it negotiate or should it be a property in aacparse ("raw-output" in example)?
The aacparse caps has as its output caps the following format: stream-format: { raw, adts, adif } ADTS and ADIF are mux/container formats such as LATM/LOAS. Instead of make a new property, I think we should use: 1) Adding a new output format in aacparse stream-format: { raw, adts, adif, latm } Then Creating a separate parse for LATM/LOAS with as input format stream-format: { latm } and as output: stream-format: { raw } That way we do not need to implement something to extract raw aac from ADTS/ADIF (which most of aac decoders already make). I think that with this approach we can handle most of cases. For example, if I implement an aac decoder that handles LATM/LOAS format, I can use it directly with aacparse (as I prefer to use my decoder LATM/LOAS parse instead of gstreamer one). If not, I can use the loasparse with my decoder (which handles raw data). []s
Adding another parser element to handle conversion from LATM/LOAS to raw aac makes sense. It might actually be a demuxer (latmdemux?), since each AAC stream muxed in LATM have separate AudioSpecificConfig so it could provide pad for each stream separately, so each stream needs to expose separate codec_data (== AudioSpecificConfig) in pad caps for raw AAC stream.
(In reply to comment #9) > Adding another parser element to handle conversion from LATM/LOAS to raw aac > makes sense. It might actually be a demuxer (latmdemux?), since each AAC stream > muxed in LATM have separate AudioSpecificConfig so it could provide pad for > each stream separately, so each stream needs to expose separate codec_data (== > AudioSpecificConfig) in pad caps for raw AAC stream. Why can't faad expose a pad for each stream separately? I'm not sure there's a need a new element. Do you have updates on your end?
I think the best would be to add conversion support between ADTS and LATM/LOAS and any other AAC formats to aacparse and let faad require ADTS input
Looking at other projects, mplayer managed to add LATM/LOAS support for faad2 also. Make faad2 support LATM/LOAS will be cleaner then add a conversion of formats in my opinion. The stream-format for LATM/LOAS was agreed to be "loas".
When added to aacparse the advantage would be, that all AAC decoders could support ADIF/ADTS/LOAS/RAW without any changes and that you could transcode from one container format to another without reencoding the AAC stream. aacparse is autoplugged by decodebin2 before faad and other AAC decoders anyway
FWIW, the same is done for h264. h264parse can convert between all the different stream formats and decoders only need to support at least one.
The aacparse route seems preferable to me as well, since we eventually have to implement conversion there anyway, and having it in there means it immediately works with a whole bunch of other decoders as well (e.g. proprietary ones or dsp codecs and somesuch).
So the idea would be to adapt this code to aacparse?
Yes, and ideally aacparse would support conversion from/to ADIF/ADTS/RAW/LOAS and select the appropiate format by negotiating with downstream.
*** Bug 701588 has been marked as a duplicate of this bug. ***
fwiw, vlc also converts from LOAS/LATM to "raw" AAC
Created attachment 249223 [details] [review] Conversion from ADTS to raw - a first step A patch to convert from ADTS to raw when downstream doesn't like ADTS but likes raw (such as qtmux). It doesn't do any other conversion (because I didn't need any other).
Created attachment 249224 [details] [review] Fix apparent off by one in ADTS object type parsing Accessory to the previous patch. This is from seeing http://wiki.multimedia.cx/index.php?title=ADTS (and confirming ffmpeg also does +1).
Review of attachment 249223 [details] [review]: ::: gst/audioparsers/gstaacparse.c @@ +112,3 @@ +static inline gint +gst_aac_parse_get_index_from_sample_rate (guint rate) Maybe this should be in codec-utils too?
> +static inline gint > +gst_aac_parse_get_index_from_sample_rate (guint rate) > > Maybe this should be in codec-utils too? Hrm, higher-level functions would be better (as in: make me an aac codec data from this and that) IMHO
Vincent, please push
Thanks. I modified the patch to move the sample rate index mapping to pbutils (I'm not sure how the other one came up to be diuplicated, the code's the same...), and added a check that the rate is in the list. I'll push that new one soon if no more comments. I've not made a higher level "make me a codec data" because the actual codec data it way more complicated, this two byte one is (thankfully) a corner case, but the spec for is is scary.
Created attachment 250135 [details] [review] ADTS->raw conversion
Created attachment 250136 [details] [review] index from AAC sample rate
Pushed, consider the bug 5% fixed :)
Created attachment 251341 [details] [review] Raw AAC -> ADTS conversion A patch to convert from raw AAC to ADTS.
Review of attachment 251341 [details] [review]: Looks almost good, just some nitpicks :) ::: gst/audioparsers/gstaacparse.c @@ +967,3 @@ + srccaps = gst_pad_get_current_caps (GST_BASE_PARSE_SRC_PAD (aacparse)); + srcstruct = gst_caps_get_structure (srccaps, 0); + profile = gst_structure_get_value (srcstruct, "profile"); gst_structure_get_string(), and if it's not successful or NULL return @@ +1079,3 @@ + gsize frame_size; + guint8 id, profile, channel_configuration, sampling_frequency_index; + const gsize adts_headers_length = 7UL; /* Total byte-length of fixed Maybe make this a #define
Created attachment 251486 [details] [review] Raw AAC -> ADTS conversion Thanks. Patch modified as per your comments.
commit b40bf67526189db009bf768c0541aefeb0d697ca Author: Chris Bass <floobleflam@gmail.com> Date: Tue Aug 13 14:09:20 2013 +0100 aacparse: allow conversion from raw AAC to ADTS This patch will prepend ADTS headers to raw AAC audio frames, allowing upstream elements to link to decoders that only support AAC in ADTS format. Note that no error correction bits are added to ADTS frames in this code. https://bugzilla.gnome.org/show_bug.cgi?id=615740
Hi! So what is remaining here? LOAS/LATM to raw? Seems that bug #678078 would be a duplicate of this then?
Conversion from/to LOAS/LATM and ADIF is missing. Only raw<->ADTS is implemented right now bug #678078 is related but IMHO not 100% the same, independent of aacparse decodebin should not fail like that in any case.
*** Bug 770699 has been marked as a duplicate of this bug. ***
*** Bug 783060 has been marked as a duplicate of this bug. ***
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/25.