turned out that bugzilla limits the file size so I have to create other sample cases smaller than what I tested.

I just attached the first parts of each file.

Created attachment 280226 [details]
header created by wavenc that makes the whole file noise.

Created attachment 280227 [details]
working header

Created attachment 280259 [details]
filesrc ! aacparse ! omxaacdec ! audioconvert ! wavenc ! filesink

Created attachment 280260 [details]
filesrc ! aacparse ! omxaacdec ! audioconvert ! wavenc ! filesink

the log generated by filesrc ! aacparse ! omxaacdec ! audioconvert ! wavenc ! filesink, currently creating not-working wav header.

One thing we notice, is that the input caps have 6 channels, the decoder sets the caps to 6 channels, and then reset it to 2 channels. wavenc receives header with two channels. Something here is not going right. Does the duration of the broken wav file differ ? (e.g. 3 times longer)

the file with a broken wav header sounds that human can't even distinguish. it's like a tv noise you hear when the tv channel isn't aired..

Created attachment 280269 [details]
filesrc ! aacparse ! avdec_aac ! audioconvert ! wavenc ! filesink

not logged in the attached file but it also says,


/temp # gst-launch-1.0 filesrc location=/temp/test_qc.aac ! aacparse ! avdec_aac
  ! audioconvert ! wavenc ! filesink location=/temp/test2.wav -v >> ./avdec_aac_
audioconvert_wavenc.log
0:00:00.282700469  1513   0x155800 ERROR                  libav :0:: channel element 0.0 is not allocated

(gst-launch-1.0:1513): GStreamer-WARNING **: gstpad.c:4652:store_sticky_event:<avdec_aac0:src> Sticky event misordering, got 'segment' before 'caps'

(gst-launch-1.0:1513): GStreamer-WARNING **: gstpad.c:4652:store_sticky_event:<audioconvert0:src> Sticky event misordering, got 'segment' before 'caps'

(gst-launch-1.0:1513): GStreamer-WARNING **: gstpad.c:4652:store_sticky_event:<wavenc0:sink> Sticky event misordering, got 'segment' before 'caps'

(gst-launch-1.0:1513): GStreamer-WARNING **: gstpad.c:4652:store_sticky_event:<audioconvert0:sink> Sticky event misordering, got 'segment' before 'caps'

In this "working" scenario, audioconvert will convert from 6 channels to 1 channels, and pass that to waveenc. This does not seem optimal, but wavenc only support 1 or 2 channels. Note that playing a 6 channel audio stream as if it was 2 channels will sound like noise, and nothing else.

So the problem is in gst-omx, or patches you have applied to it.

Created attachment 280272 [details]
filesrc ! aacparse ! omxaacdec ! audioconvert ! wavenc ! filesink

this gstomxaacdec.c is what I made based on gstomxmp3dec.c and possibly wrong.


static void
gst_omx_aac_dec_class_init (GstOMXAACDecClass * klass)
{
  GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
  GstOMXAudioDecClass *audiodec_class = GST_OMX_AUDIO_DEC_CLASS (klass);

  audiodec_class->set_format = GST_DEBUG_FUNCPTR (gst_omx_aac_dec_set_format);
  audiodec_class->is_format_change =
      GST_DEBUG_FUNCPTR (gst_omx_aac_dec_is_format_change);
  audiodec_class->get_samples_per_frame =
      GST_DEBUG_FUNCPTR (gst_omx_aac_dec_get_samples_per_frame);

  audiodec_class->cdata.default_sink_template_caps = "audio/mpeg, "
      "mpegversion=(int)[2,4],"
      "rate=(int)[8000,48000], "
      "channels=(int)[1,2], " "parsed=(boolean) true";


  gst_element_class_set_static_metadata (element_class,
      "OpenMAX AAC Audio Decoder",
      "Codec/Decoder/Audio",
      "Decode AAC audio streams",
      "Sebastian Dröge <sebastian@centricular.com>");

  gst_omx_set_default_role (&audiodec_class->cdata, "audio_decoder.aac");
}

(In reply to comment #11)
> Created an attachment (id=280272) [details]
> filesrc ! aacparse ! omxaacdec ! audioconvert ! wavenc ! filesink
> 
> this gstomxaacdec.c is what I made based on gstomxmp3dec.c and possibly wrong.
> 
> 
> static void
> gst_omx_aac_dec_class_init (GstOMXAACDecClass * klass)
> {
>   GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
>   GstOMXAudioDecClass *audiodec_class = GST_OMX_AUDIO_DEC_CLASS (klass);
> 
>   audiodec_class->set_format = GST_DEBUG_FUNCPTR (gst_omx_aac_dec_set_format);
>   audiodec_class->is_format_change =
>       GST_DEBUG_FUNCPTR (gst_omx_aac_dec_is_format_change);
>   audiodec_class->get_samples_per_frame =
>       GST_DEBUG_FUNCPTR (gst_omx_aac_dec_get_samples_per_frame);
> 
>   audiodec_class->cdata.default_sink_template_caps = "audio/mpeg, "
>       "mpegversion=(int)[2,4],"
>       "rate=(int)[8000,48000], "
>       "channels=(int)[1,2], " "parsed=(boolean) true";

I don't think a decoder should set parsed=true. Also, in your test file, you have 6 channels, while your class only supports up to 2. Negotiation should simply fail.

OK, I just changed it to have same sinkcap with avdec_aac so no negotiation issue happens now.

What I still don't understand is that avdec_aac generates wav from the same test aac file that has 6 channels but the wav gets an working wav header by wavenc supposed to support only 1,2 channels.

the rest of the pipeline is totally same except for the decoder element.

- filesrc ! aacparse ! omxaacdec ! audioconvert ! wavenc

- filesrc ! aacparse ! avdec_aac ! audioconvert ! wavenc


so does it mean that avdec_aac internally does something more to convert stream so that the stream can work with wavenc?

I don't see related parts for this in avdec_aac but just am suspecting.

I mean even converting the number of channels happens in audioconvert. wav file created by omxaacdec doesn't get an working header from wavenc.. isn't it weird?

stormer, sorry that I didn't understand what you said so far.

---------------------------------------------
One thing we notice, is that the input caps have 6 channels, the decoder sets
the caps to 6 channels, and then reset it to 2 channels. wavenc receives header
with two channels. Something here is not going right. Does the duration of the
broken wav file differ ? (e.g. 3 times longer)
---------------------------------------------

=> this is correct. I'll look into the code and find why it changes channels.
Thanks.

Created attachment 280340 [details]
filesrc ! aacparse ! omxaacdec ! audioconvert ! wavenc ! filesink

This is what happens between the two output settings. Also I confirmed that for this omx audio decoder, I haven't made any change on slomo's original except for gstomxaacdec.c that I made based on gstomxaacmp3.c

This looks like a caps negotiation problem somewhere... the log only shows failed attempts of "audioconvert ! wavenc" to negotiate, no succeeded attempt... but nonetheless data is sent to wavenc.

If you use e.g. matroskamux instead of wavenc, does it work better? wavenc has no multichannel support and there seems to be something going wrong in audioconvert here at this point.

it seems that the second output setting happens because there is a moment that
satisfies the condition port->port_def.eDir == OMX_DirOutput &&
port->settings_cookie != port->configured_settings_cookie in
gst_omx_port_acquire_buffer() so that it returns
GST_OMX_ACQUIRE_BUFFER_RECONFIGURE.

Also the caps set on the decoder srcpad are not exactly ideal:
audio/x-raw, format=(string)S16BE, layout=(string)interleaved, rate=(int)22050, channels=(int)6, channel-mask=(bitmask)0x0000000000000000

A channel mask full of zeroes here means that we have an undefined channel layout. Either the openmax component didn't tell us any channel layout (bad!) or there is a bug in the code to convert openmax layouts to gstreamer ones.

no luck with matroskamux.

filesrc ! aacparse ! omxaacdec ! audioconvert ! matroskamux ! filesink

gst-typedef identifies the output as audio/x-matroska and when I play the file with playbin, the sound is the same noise.

here is a qc code that generates an working header for msm8926.

https://www.codeaurora.org/cgit/external/thundersoft/qrd7225-ds-froyo/plain/vendor/qcom/opensource/omx/mm-audio/adec-aac/qdsp6/test/omx_aac_dec_test.c?id=cf2880e06f7f20cb3f10f8ee386924e4d0fe6fc9

however, the qc code gives channels, samplerate and etc fixed values.. but works.. confusing..

slomo, what should I do first? if omxaacdec doesn't do second output setting, the channel 6 and samplerate 22050 will be passed to the next as avdec_aac does and the problem would be fixed?

This fixes the bug in wavenc, nonetheless there is also something wrong in gst-omx here. It only fails properly now instead of creating broken files.

commit dd5144fd4ef5cfb1ee45d4009ea2bc4775395874
Author: Sebastian Dröge <sebastian@centricular.com>
Date:   Thu Jul 10 12:10:45 2014 +0200

    wavenc: Return not-negotiated if we got no caps or caps negotiation failed
    
    And do it always, not inside a g_return_val_if_fail().
    
    See https://bugzilla.gnome.org/show_bug.cgi?id=732939

omxaacdec should probably hardcode a channel layout (using the standard AAC channel layouts) if the component does not provide any useful information about it. Then it should work.

Ah, I see I see, without layout, audioconvert can't really do the 6 -> 2 conversion.

Thank you for the guide, slomo. I should start from learning channel-mask but can't find how gst-omx handles channel-mask.. nothing is found by grep 'channel-mask' whereas you mentioned that 'there is a bug in the code to convert openmax layouts to gstreamer ones.'

Could you please tell me where the code to convert openmax layouts to gstreamer ones is?

This is msm8926 and I found the below.

1. when it calls OMX_SetParameter( , OMX_IndexParamAudioAac, ) the values set are correctly shown again by OMX_GetParameter(); (nPortIndex is 0);

2. However, for a reason, OMX_EventPortSettingsChanged happens and the port forgets the values and gives me back values that look like default settings. (rate 44100, channel 2).

3. This is a serious problem, right? why OMX_EventPortSettingsChanged is called and is it normal that after OMX_EventPortSettingsChanged, nPortIndex 0 loses setting values??


I'm trying to figure out more..

ah, for the comment #28, the code I looked into was qc's sample code. FYI

(In reply to comment #28)
> This is msm8926 and I found the below.
> 
> 1. when it calls OMX_SetParameter( , OMX_IndexParamAudioAac, ) the values set
> are correctly shown again by OMX_GetParameter(); (nPortIndex is 0);
> 
> 2. However, for a reason, OMX_EventPortSettingsChanged happens and the port
> forgets the values and gives me back values that look like default settings.
> (rate 44100, channel 2).
> 
> 3. This is a serious problem, right? why OMX_EventPortSettingsChanged is called
> and is it normal that after OMX_EventPortSettingsChanged, nPortIndex 0 loses
> setting values??

Yes, this is a bug in QCOM's code... that we should nonetheless try to workaround by providing some sensible default channel layouts.


Did you ever try a stereo file though? Those should give much less problems :)

1. so OMX_EventPortSettingsChanged is not called properly?

2. the default channel layouts you mentioned must be set whenever the event occurred so that the port of the nPortIndex 0 forgot the intended values?

3. it's another problem to encode an wav file to an aac.

gst-launch-1.0 filesrc location=[.wav]
! wavparse ! avenc_aac ! filesink location=[.aac]


gst-launch-1.0 filesrc location=[.wav]
! wavparse ! omxaacenc ! filesink location=[.aac]


The ouputs from the above two commands can't be identified by gst-typedef saying 'FAILED: unknown error', even 'aacparse' is appended.

But, appending 'qtmux' makes them to generate correct audio files in video/quicktime and playbin plays them well.

Am I missing something to create an aac file?

(In reply to comment #31)
> 1. so OMX_EventPortSettingsChanged is not called properly?

It's probably called properly... but the port must not forget all its settings. Instead it should tell us the correct settings it is using now after the settings-changed event.

> 2. the default channel layouts you mentioned must be set whenever the event
> occurred so that the port of the nPortIndex 0 forgot the intended values?

Yes, as a workaround when we get an invalid channel layout.

Also the channel layout should be on the output port, so nPortIndex 1!


> 3. it's another problem to encode an wav file to an aac.
> 
> gst-launch-1.0 filesrc location=[.wav]
> ! wavparse ! avenc_aac ! filesink location=[.aac]
> 
> 
> gst-launch-1.0 filesrc location=[.wav]
> ! wavparse ! omxaacenc ! filesink location=[.aac]
> 
> 
> The ouputs from the above two commands can't be identified by gst-typedef
> saying 'FAILED: unknown error', even 'aacparse' is appended.
> 
> But, appending 'qtmux' makes them to generate correct audio files in
> video/quicktime and playbin plays them well.
> 
> Am I missing something to create an aac file?

Raw AAC files are generally not very well supported. Usually you would put AAC into an MP4 container with mp4mux.

(In reply to comment #32)
> (In reply to comment #31)
> > 1. so OMX_EventPortSettingsChanged is not called properly?
> 
> It's probably called properly... but the port must not forget all its settings.
> Instead it should tell us the correct settings it is using now after the
> settings-changed event.
> 
> > 2. the default channel layouts you mentioned must be set whenever the event
> > occurred so that the port of the nPortIndex 0 forgot the intended values?
> 
> Yes, as a workaround when we get an invalid channel layout.
> 
> Also the channel layout should be on the output port, so nPortIndex 1!
> 
> 
> > 3. it's another problem to encode an wav file to an aac.
> > 
> > gst-launch-1.0 filesrc location=[.wav]
> > ! wavparse ! avenc_aac ! filesink location=[.aac]
> > 
> > 
> > gst-launch-1.0 filesrc location=[.wav]
> > ! wavparse ! omxaacenc ! filesink location=[.aac]
> > 
> > 
> > The ouputs from the above two commands can't be identified by gst-typedef
> > saying 'FAILED: unknown error', even 'aacparse' is appended.
> > 
> > But, appending 'qtmux' makes them to generate correct audio files in
> > video/quicktime and playbin plays them well.
> > 
> > Am I missing something to create an aac file?
> 
> Raw AAC files are generally not very well supported. Usually you would put AAC
> into an MP4 container with mp4mux.

Right.. so it was hard to get aac file to test. That's right that other test sample with different channel would be helpful to finish this work. Let me try searching again.

ah.. qc's sample code doesn't set OMX_IndexParamAudioPcm but only gets values from it. So what I thought first is that when the inport gets some value by OMX_IndexParamAudioAac, then somehow OMX passes its value to OMX_IndexParamAudioPcm automatically. But gst-omx does set OMX_IndexParamAudioPcm separately so it makes me confusing.

Are we talking about the encoder or the decoder now? The decoder only gets OMX_IndexParamAudioPcm in the code I have here (to know what the decoder component will output), while the encoder only sets OMX_IndexParamAudioPcm (to tell the encoder component what data it will soon get).

it's about the decoder :) and you're right. it was my mistake that was confused between gstomxaudioenc.c and gstomxaudiodec.c

I see. Decoder only gets OMX_IndexParamAudioPcm..

if start again from the beginnig,, what's happening on msm8926 is that

1. set OMX_IndexParamAudioAac with attributes of the input wav file, for now, rate 22050 and ch #6.
2. regardless of the values, OMX_IndexParamAudioPcm return 44100 and 2
3. after reconfiguration of the OUTPORT, OMX_IndexParamAudioAac (inport) forgets the initially set value but gives back 44100 and 2. Omx_IndexParamAudioPcm also shows 44100 and 2 as it does from the beginning.


My question, if it normally works, what should be the rate and channels of OMX_IndexParamAudioPcm when the device sets OMX_IndexParamAudioAac with rate 22050 and ch #6 as the current test does??

On my second thought, possibly nothing goes wrong but just I'm misunderstanding things..

(In reply to comment #36)

> 1. set OMX_IndexParamAudioAac with attributes of the input wav file, for now,
> rate 22050 and ch #6.
> 2. regardless of the values, OMX_IndexParamAudioPcm return 44100 and 2
> 3. after reconfiguration of the OUTPORT, OMX_IndexParamAudioAac (inport)
> forgets the initially set value but gives back 44100 and 2.

This is a bug.

> Omx_IndexParamAudioPcm also shows 44100 and 2 as it does from the beginning.
> 
> 
> My question, if it normally works, what should be the rate and channels of
> OMX_IndexParamAudioPcm when the device sets OMX_IndexParamAudioAac with rate
> 22050 and ch #6 as the current test does??

It should probably contain some PCM format with 22050Hz sampling rate, 6 channels and some sensible channel layout. The other values in the PCM format depend on the exact codec.

Can you please try if it works with a stereo, 44.1kHz AAC file? :)

Created attachment 280614 [details]
gst-launch-1.0 filesrc location=/441kHz_stereo.aac ! aacparse ! omxaacdec  ! audioconvert ! wavenc ! wavparse ! alsasink -v --gst-debug=*CAPS*:6,*audiocon vert*:6,*omx*:6

unfortunately 44.1kHz stereo AAC also isn't played well. but the width is much more recognizable than the previous 22050 & 6 channel one. I'll keep working on it.

I found that right after OMX_SetParameter(,OMX_IndexParamAudioAac,), OMX_IndexParamAudioPcm sends back the same rate and ch # that OMX_IndexParamAudioAac was given.

so currently OMX_EventPortSettingsChanged looks like the bad one that makes the both ports reset to have 44100 and 2.

Have no idea why this bug is not recognized yet.. probably this decoder hasn't been used on Android.

Created attachment 280627 [details]
gst-launch-1.0 filesrc location=/temp/22050_6ch.aac ! aacparse ! omxaacd ec ! audioconvert ! wavenc ! wavparse ! alsasink --gst-debug=*:6

with the gst-omx 08385fe94425cb935f22dfa8cb399ac5589cbab0

The second OMX_EventPortSettingsChanged on omxaacdec's output port is fine because
"OMX.qcom.audio.decoder.multiaac" detects 6 which is different from the 2 channels configured though aacparse.
Since aacparse has not detected this change, the "qcom" decoder is probably wrong.

Please try against gstreamer 1.12, if you can still reproduce the problem should you attach 22050_6ch.aac and 441kHz_stereo.aac.

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-omx/issues/3.