After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 587570 - Use libavcodec codec's own downmixing
Use libavcodec codec's own downmixing
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-libav
git master
Other All
: Normal enhancement
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2009-07-02 03:39 UTC by LRN
Modified: 2018-11-03 12:55 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
The patch in question (4.18 KB, patch)
2009-07-02 03:40 UTC, LRN
needs-work Details | Review

Description LRN 2009-07-02 03:39:00 UTC
When i proposed to use ffdec_ac3 instead of a52dec as a default ac3 decoder (ffdec_ac3 is licensed under LGPL and is being maintained by FFmpeg developers, while liba52 is in -ugly and looks unmaintained), __tim expressed some concerns about libavcodec's ability to decode and output multichannel audio and to do correct downstream downmixing negotiation (which supposedly means that libavocodec should heed the channel count requirements of downstream elements and use internal codec-specific downmixers to reduce the channel count instead of relying on audioconvert to do so).

Multichannel output in libavcodec is fairly stable (at least for ac3 decoder), and it should be, because 5.1 AC3s are very widespread these days.

Some of the libavcodec's codecs (ac3, mlp, dca and faad) are capable of performing internal downmixing. OK, we aren't very interested in dca (got dca-based decoder in -bad, with downmixing) and faad (same here, although libgstfaad does not use downmixing, which is available via faadcfg->downMatrix), but mlp and ac3 are desirable (since there is no mlp decoders in GStreamer and since ffdec_ac3 is LGPL), and with downmixing too!

So, i've made a patch that adds downmixing support to GstFFmpegDec. What it does:
* Exposes request_channels AVCodecContext as "request-channels" property of ffdec_* elements
* When request_channels is not set, ffmpegdec sets it up automatically to match the caps of downstream elements
The latter part is a bit shaky, but i hope that it works at least as good as the similar code in a52dec. Even if it doesn't, it's a good start.

To check the results of my changes, i decided to run a couple of tests. I had to use vorbis to encode various ac3 decoders' output, since i have to real multichannel hardware. Vorbis appears to be the only encoder capable of encoding 6-channel audio, except flac, which encodes it too, but with highly reduced volume level (dunno why it happens and i had no desire to find that out). I've performed tests only on combinations of 2 and 6 channel audio, for simplicity. I've used Audacity to open the resulting ogg-vorbis files and see the contents.

The test subject was this file - http://www.tfm.ro/ac3/download/test_ac3.rar (i've used ffmpeg -i "Test AC3 v2.0.avi" -acodec copy surround.ac3 to extract raw .ac3 audio stream)

The pipelines i've used:

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_org_e2.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_org_e6.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_org_default.ogg

Replaced libgstffmpeg with modified version

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 request-channels=2 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_r2e2.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 request-channels=6 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_r6e2.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 request-channels=2 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_r2e6.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 request-channels=6 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_r6e6.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_default.ogg

For reference, the output from a52dec

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 lfe=true ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e6_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 lfe=true ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e6_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 lfe=true ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e2_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 lfe=true ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e2_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 lfe=false ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e6_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 lfe=false ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e6_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 lfe=false ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e2_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 lfe=false ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e2_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e6_defaultlfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e6_defaultlfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e2_defaultlfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e2_defaultlfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec lfe=true ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_a52dec_default_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec lfe=false ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_a52dec_default_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_a52dec_default.ogg

Conclusions:
* Despite what g_object_class_install_property (G_OBJECT_CLASS (klass), ARG_LFE, g_param_spec_boolean ("lfe", "LFE", "LFE", TRUE, G_PARAM_READWRITE)); says, LFE is disabled by default (or maybe a52dec disables it by itself in such circumstances)
* There is no lfe on/off switch in ffdec_ac3 (lfe is always on)

* Unpatched (request_channel-unaware) ffdec_ac3 outputs 6-channel audio by default (no downstream downmixing negotiation)

* When lfe is off, a52dec outputs mono audio without lfe by default (negotiates 1 channel?)
* Patched ffdec_ac3 outputs mono audio without lfe by default, just like a52dec with lfe=false (negotiates 1 channel?)

* When lfe is forced on, a52dec outputs 3-channel audio with lfe by default (negotiates 3 channels?)
* When 3 channels are requested, patched ffdec_ac3 outputs 3-channel audio with lfe (although the mixing strategy is different from default a52dec with lfe=false)

* When lfe is forced on and 2 channels are requested, a52dec outputs stereo with lfe
* When 2 channels are requested, patched ffdec_ac3 outputs stereo without lfe

* When lfe is forced on and 6 channels are requested, a52dec outputs 6ch audio with lfe
* When 6 channels are requested, patched ffdec_ac3 outputs 6ch audio with lfe

* When 2 or 3 channels are forced downstream, patched ffdec_ac3 behaves just as if 2 or 3 channels were explicitly requested from it (which means that negotiation works as intended)

* It seems that ffmpeg is being "smart" about lfe, discarding it when low-channel-count output is requested/negotiated, and preserving it when high-channel-cout output is requested/negotiated

surround.ac3 and the resulting ogg-vorbis files are available here - http://lrn.no-ip.info/other/gstreamer/surround_ffdec_ac3_vs_a52dec.7z
Comment 1 LRN 2009-07-02 03:40:11 UTC
Created attachment 137719 [details] [review]
The patch in question
Comment 2 Sebastian Dröge (slomo) 2009-07-30 11:21:21 UTC
You should use request_channel_layout instead. request_channels is deprecated.
You could also try to get the channel layout of the peer of the src pad and use that if present.

Other than that this is definitely a good idea :)
Comment 3 LRN 2009-07-30 14:55:22 UTC
$ cd ffmpeg

$ git pull
Already up-to-date.

$ fgrep -r "request_channel_layout" * | egrep -v "\.o|\.exe|\.dll|\.a"
libavcodec/avcodec.h:     * @deprecated Deprecated in favor of request_channel_layout.
libavcodec/avcodec.h:    int64_t request_channel_layout;
libavcodec/options.c:{"request_channel_layout", NULL, OFFSET(request_channel_layout), FF_OPT_TYPE_INT64, DEFAULT, 0, INT64_MAX, A|D, "request_channel_layout"},

Any more questions about request_channel_layout?

As for channel layout of the peer of the src pad, i'll try.
Comment 4 Sebastian Dröge (slomo) 2010-01-14 19:54:07 UTC
Any news on this?
Comment 5 LRN 2010-01-14 20:15:40 UTC
request_channel_layout is still not supported by anything in ffmpeg.
As for channel layout of the peer - i don't really remember. Whatever i did was probably lost during my HDD failure, only my first patch remained.
Comment 6 Tobias Mueller 2010-04-10 14:09:23 UTC
Reopening as I can't see any open question.
Comment 7 Edward Hervey 2013-07-18 05:51:12 UTC
This would still be nice to have.

Regarding which field to use ... how about setting both request_channels and request_channel_layout ?
Comment 8 Sebastian Dröge (slomo) 2018-05-04 09:05:52 UTC
No open questions here from what I can see? Just needs someone to implement it
Comment 9 GStreamer system administrator 2018-11-03 12:55:13 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-libav/issues/3.