GNOME Bugzilla – Bug 740101
amcaudiodec error: 'first buffer should have OMX_BUFFERFLAG_CODECCONFIG set'
Last modified: 2018-11-03 13:28:31 UTC
Created attachment 290679 [details] gst logs Environment: Google Nexus 9, Android 5.0 (L), non custom factory image, official 1.4.4 Android debug binaries App: Unmodified gst sdk tutorial 5 Error description: Tested out the latest Android binaries for kicks, fail to play back lots of streams. 11-13 17:33:16.110 5481 5773 D GStreamer+amcaudiodec: 0:26:00.482850462 0xab9894f0 gstamcaudiodec.c:775:gst_amc_audio_dec_set_format:<amcaudiodec-omxgoogleaacdecoder2> Setting new caps audio/mpeg, framed=(boolean)true, mpegversion=(int)4, level=(string)1, base-profile=(string)lc, profile=(string)lc, rate=(int)24000, channels=(int)2, stream-format=(string)adts 11-13 17:33:16.110 5481 5773 D GStreamer+amcaudiodec: 0:26:00.482993212 0xab9894f0 gstamcaudiodec.c:891:gst_amc_audio_dec_set_format:<amcaudiodec-omxgoogleaacdecoder2> Configuring codec with format: {channel-count=2, mime=audio/mp4a-latm, sample-rate=24000} 11-13 17:33:16.118 5481 5777 E SoftAAC2: first buffer should have OMX_BUFFERFLAG_CODECCONFIG set 11-13 17:33:16.118 5481 5777 W SoftAAC2: aacDecoder_ConfigRaw decoderErr = 0x2003 11-13 17:33:16.118 5481 5776 E ACodec : [OMX.google.aac.decoder] ERROR(0x80001001) 11-13 17:33:16.121 5481 5779 E GStreamer+amcaudiodec: 0:26:00.494109128 0xab7cb000 gstamcaudiodec.c:476:gst_amc_audio_dec_loop:<amcaudiodec-omxgoogleaacdecoder2> Failure dequeueing output buffer 11-13 17:33:16.121 5481 5779 W GStreamer+amcaudiodec: 0:26:00.494204545 0xab7cb000 gstamcaudiodec.c:590:gst_amc_audio_dec_loop:<amcaudiodec-omxgoogleaacdecoder2> error: Failed to dequeue output buffer: java.lang.IllegalStateException More complete log and media sample attached. It appears that amcaudiodec advertises support for AAC in adts format. aacparse isn't producing any "codec_data" or "stream_header" so no "csd-0" parameter is set. Furthermore, the equivalent of mediaFormat.setInteger(MediaFormat.KEY_IS_ADTS, 1); isn't done either. MediaCodec therefore assumes raw AAC and expects the first buffer to contain ESDS and have the flag MediaCodec.BUFFER_FLAG_CODEC_CONFIG set. When that's not the case, this error is thrown. adts AAC streams are quite common so a bug like this is a blocker for broader adoption on Android (as is #731204) A workaround is to compile in faad which works perfectly. (This is based on a one time reading of the code and logs, once I figure out how to actually build gst for android without cerbero failing left and right on my 64 bit system looking for 32 bit libs I *may* be able to offer more insight)
Created attachment 290680 [details] Sample media file experiencing this issue
Any news on this ? All of my streams are encountering this issue on Android.
*** Bug 765050 has been marked as a duplicate of this bug. ***
We're not using BUFFER_FLAG_CODEC_CONFIG anywhere currently, but this here looks like it is needed here. So multiple things 1) we should set KEY_IS_ADTS to 1 if the input is ADTS 2) for raw AAC we have codec_data and need to put that into the "csd-0", and I assume in that case BUFFER_FLAG_CODEC_CONFIG is not needed? For 1), that should be trivial to fix. Taking a short look now
Created attachment 342015 [details] [review] amcaudiodec: Set "is-adts" to 1 for ADTS AAC Otherwise it fails to decode.
Created attachment 342016 [details] [review] amcaudiodec: Set "is-adts" to 1 for ADTS AAC Otherwise it fails to decode.
Created attachment 342017 [details] [review] amcaudiodec: Set "is-adts" to 1 for ADTS AAC Otherwise it fails to decode.
Attachment 342017 [details] pushed as 3357ddb - amcaudiodec: Set "is-adts" to 1 for ADTS AAC
I still have the same problem even with this fix for playing the HLS apple test stream: http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8 12-22 16:41:05.192 14069 14102 I OMXClient: Using client-side OMX mux. 12-22 16:41:05.197 14069 14102 E OMXMaster: A component of name 'OMX.qcom.audio.decoder.aac' already exists, ignoring this one. 12-22 16:41:05.203 14069 14100 W GStreamer+amcaudiodec: 0:00:00.695035781 0xdd092e60 gstamcaudiodec.c:898:gst_amc_audio_dec_set_format skip forcing aac lc 12-22 16:41:05.225 14069 14101 E libEGL : validate_display:255 error 3008 (EGL_BAD_DISPLAY) 12-22 16:41:05.238 14069 14103 E SoftAAC2: first buffer should have OMX_BUFFERFLAG_CODECCONFIG set 12-22 16:41:05.238 14069 14103 W SoftAAC2: aacDecoder_ConfigRaw decoderErr = 0x2003 12-22 16:41:05.238 14069 14102 E ACodec : [OMX.google.aac.decoder] ERROR(0x80001001) 12-22 16:41:05.238 14069 14102 E ACodec : signalError(omxError 0x80001001, internalError -2147483648) 12-22 16:41:05.238 14069 14102 E MediaCodec: Codec reported err 0x80001001, actionCode 0, while in state 6 12-22 16:41:05.239 14069 14100 W GStreamer+amcaudiodec: 0:00:00.730825312 0xdd092e60 gstamcaudiodec.c:1244:gst_amc_audio_dec_handle_frame:<amcaudiodec-omxgoogleaacdecoder0> error : Failed to call Java method: java.lang.IllegalStateException 12-22 16:41:05.239 14069 14100 W GStreamer+amcaudiodec: java.lang.IllegalStateException 12-22 16:41:05.239 14069 14100 W GStreamer+amcaudiodec: at android.media.MediaCodec.native_queueInputBuffer(Native Method) 12-22 16:41:05.239 14069 14100 W GStreamer+amcaudiodec: at android.media.MediaCodec.queueInputBuffer(MediaCodec.java:2165) 12-22 16:41:05.239 14069 14100 W GStreamer+amcaudiodec: 12-22 16:41:05.239 14069 14100 W GStreamer+audiodecoder: 0:00:00.730882812 0xdd092e60 gstaudiodecoder.c:3047:_gst_audio_decoder_error:<amcaudiodec-omxgoogleaacdecoder0> error: Fa iled to call Java method: java.lang.IllegalStateException 12-22 16:41:05.239 14069 14100 W GStreamer+audiodecoder: java.lang.IllegalStateException 12-22 16:41:05.239 14069 14100 W GStreamer+audiodecoder: at android.media.MediaCodec.native_queueInputBuffer(Native Method) 12-22 16:41:05.239 14069 14100 W GStreamer+audiodecoder: at android.media.MediaCodec.queueInputBuffer(MediaCodec.java:2165) 12-22 16:41:05.239 14069 14100 W GStreamer+audiodecoder: 12-22 16:41:05.239 14069 14105 E GStreamer+amcaudiodec: 0:00:00.730943906 0xdc5f2320 gstamcaudiodec.c:477:gst_amc_audio_dec_loop:<amcaudiodec-omxgoogleaacdecoder0> Failure dequeu eing output buffer 12-22 16:41:05.239 14069 14105 W GStreamer+amcaudiodec: 0:00:00.730979583 0xdc5f2320 gstamcaudiodec.c:614:gst_amc_audio_dec_loop:<amcaudiodec-omxgoogleaacdecoder0> error: Failed to call Java method: java.lang.IllegalStateException
Ok, can you confirm that is-adts is actually set in the MediaFormat for you? If you set GST_DEBUG to amcaudiodec:6, you should see a line saying: "Configuring codec with format: ..."
12-23 13:40:33.763 31286 31331 D GStreamer+amcaudiodec: 0:00:01.855872499 0xdc626260 gstamcaudiodec.c:828:gst_amc_audio_dec_set_format:<amcaudiodec-omxgoogleaacdecoder0> Setting new caps audio/mpeg, framed=(boolean)true, mpegversion=(int)4, level=(string)1, base-profile=(string)lc, profile=(string)lc, rate=(int)22050, channels=(int)1, stream-format=(str$ ng)adts 12-23 13:40:33.763 31286 31331 W GStreamer+amcaudiodec: 0:00:01.856183176 0xdc626260 gstamcaudiodec.c:896:gst_amc_audio_dec_set_format setting is-adts 12-23 13:40:33.763 31286 31331 D GStreamer+amcaudiodec: 0:00:01.856344478 0xdc626260 gstamcaudiodec.c:965:gst_amc_audio_dec_set_format:<amcaudiodec-omxgoogleaacdecoder0> Configuring codec with format: {is-adts=1, channel-count=1, mime=audio/mp4a-latm, sample-rate=22050} 12-23 13:40:33.767 31286 31339 I OMXClient: Using client-side OMX mux. 12-23 13:40:33.770 31286 31340 D GStreamer+amcaudiodec: 0:00:01.863462030 0xdc649f20 gstamcaudiodec.c:424:gst_amc_audio_dec_loop:<amcaudiodec-omxgoogleaacdecoder0> Waiting for a$ailable output buffer 12-23 13:40:33.771 26175 26280 I OMX-VDEC-1080P: Video slvp perflock acquired 12-23 13:40:33.771 31286 31331 D GStreamer+amcaudiodec: 0:00:01.864661510 0xdc626260 gstamcaudiodec.c:1070:gst_amc_audio_dec_handle_frame:<amcaudiodec-omxgoogleaacdecoder0> Hand$ ing frame 12-23 13:40:33.774 31286 31331 D GStreamer+amcaudiodec: 0:00:01.867306562 0xdc626260 gstamcaudiodec.c:1190:gst_amc_audio_dec_handle_frame:<amcaudiodec-omxgoogleaacdecoder0> Queu$ ing buffer 0: size 11 time 26666 flags 0x00000001 12-23 13:40:33.775 31286 31334 E SoftAAC2: first buffer should have OMX_BUFFERFLAG_CODECCONFIG set 12-23 13:40:33.775 31286 31334 W SoftAAC2: aacDecoder_ConfigRaw decoderErr = 0x2003 12-23 13:40:33.776 31286 31333 E ACodec : [OMX.google.aac.decoder] ERROR(0x80001001) 12-23 13:40:33.776 31286 31331 D GStreamer+amcaudiodec: 0:00:01.868825572 0xdc626260 gstamcaudiodec.c:1070:gst_amc_audio_dec_handle_frame:<amcaudiodec-omxgoogleaacdecoder0> Hand$ ing frame 12-23 13:40:33.776 31286 31333 E ACodec : signalError(omxError 0x80001001, internalError -2147483648) 12-23 13:40:33.776 31286 31333 E MediaCodec: Codec reported err 0x80001001, actionCode 0, while in state 6
Seems like a bug in the codec then, should check if we can somehow work around that.
I think I traced it back to that function call: The E SoftAAC2: first buffer should have OMX_BUFFERFLAG_CODECCONFIG set is in fact the SoftAAC2 codec setting the flag by itself on first input buffer, the next trace, SoftAAC2: aacDecoder_ConfigRaw decoderErr = 0x2003 is https://github.com/mstorsjo/fdk-aac/blob/72037da0badf4c2ae0293d7b8b350651e4f96a61/libMpegTPDec/src/tpdec_latm.cpp#L211 returning one of the TRANSPORTDEC_UNSUPPORTED_FORMAT
Another clue might be that in https://android.googlesource.com/platform/frameworks/av/+/jb-mr0-release/media/libstagefright/codecs/aacdec/SoftAAC2.cpp With gstreamer, we see the error trace at line 517. When trying to play the same bipbop stream with ExoPlayer, the only trace I see is on line 748. So it seems that when calling SoftAAC2::onQueueFilled from gstreamer, the condition at line 509 is true, inQueue is not empty while for ExoPlayer, the inQueue is empty. I hope this may help...
I got it working a bit better with this: http://stackoverflow.com/questions/12942201/decoding-aac-using-mediacodec-api-on-android What I did is adding this in the set format function: guint8 test[] = { 0x12, 0x12 }; gst_amc_format_set_buffer (format, "csd-0", test, 2, &err); After setting that, the stream starts without audio and it crashes a bit later, but at list that's a start.
The problem here is that you pass in a stack allocated buffer. The buffer passed to the format must be valid as long as the format is. Try creating it in malloc'd memory and just leak it to see if that makes it work. However, for ADTS there is not really a out of band header. It should just parse that from the stream itself. Did you try setting the CODECCONFIG flag on the very first buffer for ADTS? Does that also work?
Met the same problem with 1.11, and led me here. ----- 01-01 08:30:47.003 12045 12077 D [gstplayer.c:665] <gst-player> Set uri=http://10.9.44.131/1080p-hls/a012c3d1-1d3c-4b79-bdd2-0c6dd16aba3b/index.m3u8 ... 01-01 08:30:48.902 12045 12292 D [gstamcaudiodec.c:834] <amcaudiodec-nonpublic> Setting new caps audio/mpeg, framed=(boolean)true, mpegversion=(int)4, level=(string)4, base-profile=(string)lc, profile=(string)lc, rate=(int)48000, channels=(int)6, stream-format=(string)adts ... 01-01 08:30:48.966 12045 12299 E SoftAAC2: first buffer should have OMX_BUFFERFLAG_CODECCONFIG set 01-01 08:30:48.966 12045 12299 W SoftAAC2: aacDecoder_ConfigRaw decoderErr = 0x2003 ... 01-01 08:30:48.967 12045 12298 E ACodec : [OMX.google.aac.decoder] ERROR(0x80001001) 01-01 08:30:48.967 12045 12298 E ACodec : signalError(omxError 0x80001001, internalError -2147483648) ---- After testing, OMX.google.aac.decoder compared to ffmpeg‘s audio decoder, stability and compatibility seems to be worse, so my temporary solution: --- a/sys/androidmedia/gstamc.c +++ b/sys/androidmedia/gstamc.c @@ -3308,7 +3308,7 @@ register_codecs (GstPlugin * plugin) g_str_has_suffix (codec_info->name, ".sw.dec")) { /* For video we prefer hardware codecs, for audio we prefer software * codecs. Hardware codecs don't make much sense for audio */ - rank = is_video ? GST_RANK_SECONDARY : GST_RANK_PRIMARY; + rank = is_video ? GST_RANK_SECONDARY : GST_RANK_NONE; } else if (g_str_has_prefix (codec_info->name, "OMX.Exynos.") && !is_video) { /* OMX.Exynos. audio codecs are existing on some devices like the --
Created attachment 373302 [details] [review] proposed patch git master is still valid. Sorry, please ignore the noise made above. Anyway, how about inserting 2 bytes of harmless codec_data in aacparse? And,fix codec_data_buffer memory leak by the way.
I've pushed the leak fix already, thanks.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/189.