GNOME Bugzilla – Bug 587570
Use libavcodec codec's own downmixing
Last modified: 2018-11-03 12:55:13 UTC
When i proposed to use ffdec_ac3 instead of a52dec as a default ac3 decoder (ffdec_ac3 is licensed under LGPL and is being maintained by FFmpeg developers, while liba52 is in -ugly and looks unmaintained), __tim expressed some concerns about libavcodec's ability to decode and output multichannel audio and to do correct downstream downmixing negotiation (which supposedly means that libavocodec should heed the channel count requirements of downstream elements and use internal codec-specific downmixers to reduce the channel count instead of relying on audioconvert to do so). Multichannel output in libavcodec is fairly stable (at least for ac3 decoder), and it should be, because 5.1 AC3s are very widespread these days. Some of the libavcodec's codecs (ac3, mlp, dca and faad) are capable of performing internal downmixing. OK, we aren't very interested in dca (got dca-based decoder in -bad, with downmixing) and faad (same here, although libgstfaad does not use downmixing, which is available via faadcfg->downMatrix), but mlp and ac3 are desirable (since there is no mlp decoders in GStreamer and since ffdec_ac3 is LGPL), and with downmixing too! So, i've made a patch that adds downmixing support to GstFFmpegDec. What it does: * Exposes request_channels AVCodecContext as "request-channels" property of ffdec_* elements * When request_channels is not set, ffmpegdec sets it up automatically to match the caps of downstream elements The latter part is a bit shaky, but i hope that it works at least as good as the similar code in a52dec. Even if it doesn't, it's a good start. To check the results of my changes, i decided to run a couple of tests. I had to use vorbis to encode various ac3 decoders' output, since i have to real multichannel hardware. Vorbis appears to be the only encoder capable of encoding 6-channel audio, except flac, which encodes it too, but with highly reduced volume level (dunno why it happens and i had no desire to find that out). I've performed tests only on combinations of 2 and 6 channel audio, for simplicity. I've used Audacity to open the resulting ogg-vorbis files and see the contents. The test subject was this file - http://www.tfm.ro/ac3/download/test_ac3.rar (i've used ffmpeg -i "Test AC3 v2.0.avi" -acodec copy surround.ac3 to extract raw .ac3 audio stream) The pipelines i've used: gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_org_e2.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_org_e6.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_org_default.ogg Replaced libgstffmpeg with modified version gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 request-channels=2 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_r2e2.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 request-channels=6 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_r6e2.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 request-channels=2 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_r2e6.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 request-channels=6 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_r6e6.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_default.ogg For reference, the output from a52dec gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 lfe=true ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e6_lfe.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 lfe=true ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e6_lfe.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 lfe=true ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e2_lfe.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 lfe=true ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e2_lfe.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 lfe=false ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e6_nolfe.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 lfe=false ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e6_nolfe.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 lfe=false ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e2_nolfe.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 lfe=false ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e2_nolfe.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e6_defaultlfe.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e6_defaultlfe.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e2_defaultlfe.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e2_defaultlfe.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec lfe=true ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_a52dec_default_lfe.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec lfe=false ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_a52dec_default_nolfe.ogg gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_a52dec_default.ogg Conclusions: * Despite what g_object_class_install_property (G_OBJECT_CLASS (klass), ARG_LFE, g_param_spec_boolean ("lfe", "LFE", "LFE", TRUE, G_PARAM_READWRITE)); says, LFE is disabled by default (or maybe a52dec disables it by itself in such circumstances) * There is no lfe on/off switch in ffdec_ac3 (lfe is always on) * Unpatched (request_channel-unaware) ffdec_ac3 outputs 6-channel audio by default (no downstream downmixing negotiation) * When lfe is off, a52dec outputs mono audio without lfe by default (negotiates 1 channel?) * Patched ffdec_ac3 outputs mono audio without lfe by default, just like a52dec with lfe=false (negotiates 1 channel?) * When lfe is forced on, a52dec outputs 3-channel audio with lfe by default (negotiates 3 channels?) * When 3 channels are requested, patched ffdec_ac3 outputs 3-channel audio with lfe (although the mixing strategy is different from default a52dec with lfe=false) * When lfe is forced on and 2 channels are requested, a52dec outputs stereo with lfe * When 2 channels are requested, patched ffdec_ac3 outputs stereo without lfe * When lfe is forced on and 6 channels are requested, a52dec outputs 6ch audio with lfe * When 6 channels are requested, patched ffdec_ac3 outputs 6ch audio with lfe * When 2 or 3 channels are forced downstream, patched ffdec_ac3 behaves just as if 2 or 3 channels were explicitly requested from it (which means that negotiation works as intended) * It seems that ffmpeg is being "smart" about lfe, discarding it when low-channel-count output is requested/negotiated, and preserving it when high-channel-cout output is requested/negotiated surround.ac3 and the resulting ogg-vorbis files are available here - http://lrn.no-ip.info/other/gstreamer/surround_ffdec_ac3_vs_a52dec.7z
Created attachment 137719 [details] [review] The patch in question
You should use request_channel_layout instead. request_channels is deprecated. You could also try to get the channel layout of the peer of the src pad and use that if present. Other than that this is definitely a good idea :)
$ cd ffmpeg $ git pull Already up-to-date. $ fgrep -r "request_channel_layout" * | egrep -v "\.o|\.exe|\.dll|\.a" libavcodec/avcodec.h: * @deprecated Deprecated in favor of request_channel_layout. libavcodec/avcodec.h: int64_t request_channel_layout; libavcodec/options.c:{"request_channel_layout", NULL, OFFSET(request_channel_layout), FF_OPT_TYPE_INT64, DEFAULT, 0, INT64_MAX, A|D, "request_channel_layout"}, Any more questions about request_channel_layout? As for channel layout of the peer of the src pad, i'll try.
Any news on this?
request_channel_layout is still not supported by anything in ffmpeg. As for channel layout of the peer - i don't really remember. Whatever i did was probably lost during my HDD failure, only my first patch remained.
Reopening as I can't see any open question.
This would still be nice to have. Regarding which field to use ... how about setting both request_channels and request_channel_layout ?
No open questions here from what I can see? Just needs someone to implement it
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-libav/issues/3.