GNOME Bugzilla – Bug 746595
omxaacdec requires pulsesink a large 'discont-wait' value
Last modified: 2018-05-01 07:06:46 UTC
I found that the below pipeline generates several discontinuities. I believe this is why there are moments that playing stops occasionally in a very moment. (but at certain points, I mean the points that it stops are not random but are fixed) gst-launch-1.0 filesrc location=/temp/test6_eAACPlus.3gp2 ! qtdemux ! aacparse ! omxaacdec ! pulsesink Adding discont-wait=18446744073709551614 to the pipeline removes the problem at this moment. gst-launch-1.0 filesrc location=/temp/test6_eAACPlus.3gp2 ! qtdemux ! aacparse ! omxaacdec ! pulsesink discont-wait=18446744073709551614 It seems like a synchronization issue but I'm not sure if it's a defect of omxaacdec as this decoder creates a complete wav file with gst-launch-1.0 filesrc location=/temp/test6_eAACPlus.3gp2 ! aacparse ! omxaacdec ! audio/x-raw,format=S16LE ! audioconvert ! wavenc ! filesink location=/temp/test.wav If I can have any advice, I will try to fix it and share the patch as well. Thanks Jun
Can you check the timestamps of the buffers on the srcpad of aacparse and omxaacdec? And maybe attach them here? In gst-launch you could use: > gst-launch-1.0 -v filesrc location=/temp/test6_eAACPlus.3gp2 ! qtdemux ! \ > aacparse ! identity silent=false name=aacparse_identity ! omxaacdec ! \ > identity silent=false name=omxaacdec_identity ! pulsesink Are the timestamps/durations of aacparse more or less continuous and are the jumps in the ones from omxaacdec?
Created attachment 300106 [details] filesrc ! qtdemux ! aacparse ! aacparse_identity ! omxaacdec ! omxaacdec_identity ! pulsesink I need to figure out what it says about the continuity.
Created attachment 300109 [details] filesrc ! qtdemux ! aacparse ! aacparse_identity ! avdec_aac ! avdec_aac_identity ! pulsesink avdec_aac shows a regular pattern: a pair of aacparse_identity and avdec_aac_identity.
The durations are wrong. omxaacdec outputs one output buffer per two input frames. The duration of each output buffer should be around 90ms, but is only around 45ms (thus one frame). It should go into this code here: http://cgit.freedesktop.org/gstreamer/gst-omx/tree/omx/gstomxaudiodec.c#n527 Where spf should be 1024 (which is not true for all AAC configurations, but should be true for yours here). Your output buffers seem to contain 2042 (not 2048!) samples, so you should get those warnings there... and it should calculate nframes to 2.
Slomo, thanks for the guide. So the problem seems that somehow the outbuf duration becomes only for one frame. 1) For the case that nframes needs to be 1 as the outbuf duration is for one frame : I tried to give it nframes = 1 and found that the discontinuities reduced significantly. However, still few noise happens so it may not the ultimate solution. 2) For the case that duration should be for two frames as currently nframes is calculated as 2 : However, buf->omx_buf->nTickCount looks like coming from the device directly.. Or what else would I need to start with? Thanks Jun
For your case, nframes should always be two. And it should calculate that from the number of samples in the output buffers already. Why doesn't it? :) The nTickCount is useless and is not going to be used for the output buffers. GstAudioDecoder is overwriting the duration of buffers based on the number of frames you put into the finish() call. See also bug #685730 and bug #746017
Slomo, Regarding #685730, I believe the omxaacdec handles nframe and spf correctly now. If it's ok that qcom returns 2042 after processing 1024*2, I don't see any fault on how gstreamer calculates the duration. gst_audio_decoder_finish_frame() looks having a correct formula for duration. 0:00:01.887685104 3691 0x6ef0aa60 LOG audiodecoder gstaudiodecoder.c:1138:gst_audio_decoder_finish_frame:<omxaacdec-omxaacdec0> accepting 8168 bytes == 2042 samples for 2 frames GST_BUFFER_DURATION (buf) = GST_FRAMES_TO_CLOCK_TIME (samples, ctx->info.rate); According to your comment, this duration should have been wrong but I think it works correctly.. --- The durations are wrong. omxaacdec outputs one output buffer per two input frames. The duration of each output buffer should be around 90ms, but is only around 45ms (thus one frame). --- So, could I get a little bit more information about your comment? Thanks Jun
Ah one thing that confused me is that the log first shows 22050Hz sample rate and 1 channel in the srcpad caps of omxaacdec... and then later 44100kHz and 2 channels. I guess the latter is correct? Would be good to also fix omxaacdec to not first output the wrong caps. But in any case, you have buffers with 8168 bytes. That means 2042 samples, which is calculated correctly there. And which is also more or less 2 AAC frames. And should also have a duration of about 45ms (2042/44100 seconds), which also looks correct. The problems I see in the log are the following: 1) Each input buffer has a duration of about 45ms. Each input buffer *is* a single AAC frame, which according to the omxaacdec code is supposed to have 1024 samples. 45ms would be 2048 samples. Which AAC profile is used here? This would already confuse the GstAudioDecoder base class a lot as now it sees the following: input of 1 frame with 45ms and 1024 samples (wrong, 45ms are 2048 samples), output of 2 frames with 45ms and 2042 samples. This completely confuses it's time tracking :) 2) The output frames have the correct duration of 45ms for 2048 samples. But the timestamps are wrong, sometimes there's a spacing of 90ms between two output buffers (see the first two buffers for example) 3) WARNING: from element /GstPipeline:pipeline0/GstOMXAACDec-omxaacdec:omxaacdec-omxaacdec0: received more decoded frames 2 than provided 0 This is just a symptom of 1). We get one frame with 45 ms, and finish two frames with 45 ms. So the base class thinks we produce frames out of thin air. So the first step here would be to check what the AAC frames actually contain, how many samples per frame, which sample rate and number of channels.
Created attachment 300326 [details] -v comparision between omxaacdec and avdec_aac accparse gives both decoders 22050 but decoders read it again as 44100
Created attachment 300331 [details] the result of the code https://www.codeaurora.org/cgit/external/gigabyte/froyo-gb-dsds-7227/plain/vendor/qcom/opensource/omx/mm-audio/adec-aac/qdsp6/test/omx_aac_dec_test.c I attached a log, the result of the code https://www.codeaurora.org/cgit/external/gigabyte/froyo-gb-dsds-7227/plain/vendor/qcom/opensource/omx/mm-audio/adec-aac/qdsp6/test/omx_aac_dec_test.c I found that this qcom test code also gets 8168 as a output buffer which would mean that getting this number of samples is not a tricky bug in gstreamer. And personally, I added aac_param.eAACProfile = OMX_AUDIO_AACObjectHE_PS; to gstomxaacdec.c but I don't think it changes something. I'm still working on it to find more clues.
One difference is that qcom test code manually assigns nFilledLen to 4096 in Read_Buffer(); but gst-omx considers it 2042.
That doesn't really answer any of the questions I asked above, and without access to the hardware in question I can only give you hints at where to look. The debugging you will have to do yourself :) Also for getting the number of samples per frame, look at a proper software decoder and not compare it with other code using the same OMX codec.
No answers in 3 years. Closing. Please re-open if you can reproduce the issue with latest stable GStreamer.