After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 340362 - [PATCH] new plugin - hardlimiter
[PATCH] new plugin - hardlimiter
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
git master
Other Linux
: Normal enhancement
: 0.10.6
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2006-05-02 02:51 UTC by Артём Попов
Modified: 2007-02-05 16:27 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
the plugin (209.45 KB, application/x-bzip)
2006-05-02 02:52 UTC, Артём Попов
  Details
patch against gst-plugins-bad (16.99 KB, patch)
2006-05-11 15:14 UTC, Tim-Philipp Müller
none Details | Review
Graphs of different transfer functions (202.60 KB, image/png)
2006-10-01 12:06 UTC, René Stadler
  Details

Description Артём Попов 2006-05-02 02:51:19 UTC
This is a simple brickwall audio limiter, originally used for my pet project. Supporting both integer and floating-point audio samples, it does not requantize incoming signal, so it can be losslessly applied to dithered audio.

I want to propose it for inclusion in GStreamer, because such a limiter is a must-have for any audio player with replaygain support:

http://replaygain.hydrogenaudio.org/player.html
http://amarok.kde.org/amarokwiki/index.php/ReplayGain/VorbisGain_Support_-_What_Would_it_Take%3F#With_respect_to_the_player

A -6dB hardlimiter may be also used to enchance output on some soundcard models (a signal level above -6dB may be unsafe (introduce clipping) on consumer hardware).

The attachement is full autoconfed source package, based on gst-template. The plugin only needs to be updated to use GST_LICENSE, GST_PACKAGE_NAME and GST_PACKAGE_ORIGIN in GST_PLUGIN_DEFINE macro.
Comment 1 Артём Попов 2006-05-02 02:52:55 UTC
Created attachment 64646 [details]
the plugin
Comment 2 Tim-Philipp Müller 2006-05-03 14:01:04 UTC
Nice!

Re-assigning to gst-plugins-bad (that's where new plugins get added first).
Comment 3 Tim-Philipp Müller 2006-05-11 15:14:16 UTC
Created attachment 65249 [details] [review]
patch against gst-plugins-bad

This looks like very nice and clean code. Attached a modified version as a patch against gst-plugins-bad.

Some minor things I came across:

 - if the type is called GstHardLimit, functions should be gst_hard_limit_foo
   and not gst_hardlimit_foo as well

 - you don't need GST_BOILERPLATE_FULL here, the debug init stuff is usually
   done in plugin_init or the base init function

 - in gst_hard_limiter_transform_ip() - why just return GST_FLOW_OK if the
   buffer isn't writable? Is that an expected situation? If not, we should
   probably g_assert() it instead.

 - there's a GST_BUFFER_TIMESTAMP_IS_VALID macro as well

 - the HARDLIMITER_DO_LIMIT_FUNCTION macro in hardlimit.h uses 'static inline'
   for the functions but they won't be inlined (I think) because they're
   refered to by address and assigned to filter->do_limit, so the whole
   transform business is highly inefficient at the moment because it involves
   a function call for every single sample in the buffer.

 - might make sense to use the G_UNLIKELY macros in the hardlimiter_limit
   functions

 - also - minor point: the element doesn't actually seem to do anything at
   all no matter what :) I tracked this down to the buffer writable check
   mentioned above. Adding something like
     gstbasetrans_class->passthrough_on_same_caps = TRUE;
   and later
     gst_base_transform_set_in_place (GST_BASE_TRANSFORM (filter), TRUE);
   seems to fix this and give us a writable buffer.


Lastly, could you explain to me the exact difference between 'hard limiting' and 'clipping'?

Apologies in advance for my ignorance, but it seems to me that all this element does is actually clip the signal at a certain level. I would have thought that 'hard limiting' involves reducing the signal in general once the limit has been reached (with a 0-second attack, thus 'hard') or something like that. Please correct me if I'm wrong :)
Comment 4 Tim-Philipp Müller 2006-05-11 15:26:29 UTC
Oh also, I don't understand the GstController stuff - can you explain to me how that is supposed to be used in this case? (again apologies for my ignorance)
Comment 5 Thomas Vander Stichele 2006-05-11 21:10:33 UTC
I like this as well, excellent work, code looks nice.

Same comments as tim above, please reply to them so we can move this in.

Additional comment - if the element is called "hardlimit", why is it in a dir called audiocompress ? Are you planning to add more compressors in the same plugin ?
Comment 6 Tim-Philipp Müller 2006-05-11 21:25:52 UTC
(In reply to comment #5)

> Additional comment - if the element is called "hardlimit", why is it in a dir
> called audiocompress ? Are you planning to add more compressors in the same
> plugin ?

Actually, that was my doing. I figured if there were more compressors to come they could go in there, no particular reason other than that. I like to keep the plugin name different / more general from element names if it makes sense. Can rename it to 'hardlimit' if you prefer that.
Comment 7 Артём Попов 2006-05-12 01:13:00 UTC
Answering questions:

- GST_BOILERPLATE_FULL was inserted by the template tool from gst-template package. I thought all elements require this one.

- transform_ip() indeed should return GST_FLOW_OK if buffer is unwritable (checking if in passthrough mode).

- strange, that the element did not do anything for you. It shouldn't passthrough on same caps and works for me here.

- yes, hardlimiting sounds like clipping/distortion. And sounds horrible but still better than soundcard clipping. However, I'm currently writing a more general compressor unit that should provide a better sounding solution.

- GstController stuff also was sitting in the template. I thought it's generally a good idea to have controlled elements. Aren't they especially useful for NLE apps? Correct me, if I'm wrong.
Comment 8 Артём Попов 2006-05-12 01:25:08 UTC
By the way, did you guys consider creating a "dsp" directory or even something like a "gst-plugins-dsp" module for all the audio-processing stuff?
Comment 9 Tim-Philipp Müller 2006-05-12 12:05:24 UTC
> - GST_BOILERPLATE_FULL was inserted by the template tool from gst-template
> package. I thought all elements require this one.

Apologies, I missed that that is from template :) (It's ugly nevertheless IMHO)


> - transform_ip() indeed should return GST_FLOW_OK if buffer is unwritable
> (checking if in passthrough mode).
> - strange, that the element did not do anything for you. It shouldn't
> passthrough on same caps and works for me here.

No idea about these, have to investigate.

 
> - yes, hardlimiting sounds like clipping/distortion. And sounds horrible but
> still better than soundcard clipping. 

I was looking at the wave curve in a sound editor, not listening to it :)


> However, I'm currently writing a more general
> compressor unit that should provide a better sounding solution.

Great!


> - GstController stuff also was sitting in the template. I thought it's
> generally a good idea to have controlled elements. Aren't they especially
> useful for NLE apps? Correct me, if I'm wrong.

No, you're absolutely right. I thought the GstController stuff was supposed to serve another purpose and got confused.

Comment 10 René Stadler 2006-09-21 18:21:35 UTC
I have to say that this element is _not_ a hard limiter.  Also, it is _wrong_ to say that hard limiting sounds bad and it _is_ true that this element just does clipping (which can never sound better than "sound card clipping").  Using this element can only mess up the output, I don't see a use case for it.  The example about consumer DACs that the reporter provides is flawed.

-6 dB hard limiting means to apply a transfer function like

   ouput = tanh ((input - 0.5) / 0.5) * 0.5 + 0.5

to all values with input > 0.5 and a respective negative variant for values < 
-0.5.

This will smoothly compress values above 0.5 (ca. -6 dB), which has the following advantages:

  - You can input values >1.0 and still get a good sounding output (unless you really overdo it).  This is of use for ReplayGain if you disable clipping prevention (which you don't have to).

  - The dynamic range gets reduced (input 1.0 -> output  ca. 0.88).  This allows you for example to listen to classical music through loudspeakers in front of a computer (noisy environment), or for playing music on a party.  This is entirely unrelated to ReplayGain.

You really have to watch out when investigating about DSP stuff on the web.  Lots of seemingly useful documents talk about an analog processing background.  I assume this is how the reporter got the wrong idea that clipping is useful.  In the analog world, you need such a thing as protection for equipment or as last resort protection to make sure output is always within certain limits (radio stations).
Comment 11 Stefan Sauer (gstreamer, gtkdoc dev) 2006-09-30 18:28:13 UTC
I can second René's comments. Dynamic compression, expansion, limmiting can also be seen as a lookup-table transformation in->out. A compressor starts to work for a signal above a threshold and then compresses the signal. A hard limmiter clamps the signal above the treshhold to the treshhold. A compressor compresses signals above the threshold, e.g. a compressor with the ration 2:1 would do

out = thresh + ((in-thresh)/2.0)

That would be a compressor with hard knee characteristcs. The other bahaviour is soft-knee which would use a smooth curved transition when going from linear to compressed.

As a sidenote - in gst-plugins-good we now have the audiofx plugin, where such elements could possibly be added (to keep number of plugins low).
Comment 12 René Stadler 2006-10-01 12:06:57 UTC
Created attachment 73746 [details]
Graphs of different transfer functions

A picture says more than thousand words:

 1. Red graph: Clipping (as originally proposed by Артём)
 2. Blue graph: Linear filter (like Stefan mentioned)
 3. Green graph: Smooth hard limiting (like I mentioned)

All filters have a threshold of 0.5 (ca. -6 dB), that is they leave values below 0.5 (above -0.5) intact.  The picture shows clearly why for clipping, the waveform gets irreversibly distorted if it exceeds the threshold.  I mention this because some people seem to have the impression that clipping can somehow be undone by reducing the volume or something.  It is not.  Clipping is also done implicitely by audioconvert when it converts from float to integer, which is most often done to finally send the data to the sound card.  That's the red area on the graphs, it can never be reached in integer formats as it is beyond what is defined as maximum.

Second one is a real signal compressor that gives good results instead of useless distortion.  It's exactly what Stefan said: Basically a selective volume change that only operates on values exceeding a threshold.  It is very efficient (just a multiplication in addition to the switching), but that comes at a price: Trained ears (which I don't claim to have) seem to be able to hear the point of the threshold because of the rather abrupt change that exceeding the threshold introduces.  This depends on the input data and especially the ratio of course.

The third one avoids that by giving smooth values instead.  These come at the price of higher computational comlexity (a tanhf in addition to the switching plus scaling around a bit).
Comment 13 Stefan Sauer (gstreamer, gtkdoc dev) 2006-10-01 12:25:12 UTC
more explanation ->
http://en.wikipedia.org/wiki/Audio_level_compression
Comment 14 Edward Hervey 2006-10-01 16:55:24 UTC
Rene, FWIW, the two last proposal (stefan and yours) both have their advantages (one is fast, the other more professional) and disadvantages (one doesn't avoid clipping, the other one is slow), and considering they would share the exact same code, except for the actual computation, how about making it one element with a method property to choose which method you want ? Setting it by default to the 'best' one (yours) but leaving an option to go to the 'fast' one .
Comment 15 René Stadler 2006-10-01 18:12:49 UTC
I also thought about that, but how to sanely express the parameters as properties?  Both have a threshold, but the linear one has a ratio and the smooth one a limit.  Maybe call the mode enum nicks "smooth" and "linear", and let the element have four properties: mode, threshold, linear-ratio and smooth-limit or something.  This should make it clear that either property is ignored depending on the mode.

I completely ignore advanced features such as attack time or side-chaining for now.  I think these make lots of sense for music production (buzztard, jokosher, etc.).  The use case I focus on is playback.
Comment 16 Stefan Sauer (gstreamer, gtkdoc dev) 2006-10-02 07:05:26 UTC
In pro-audio devices compressors have four parameters:
* attack (lets omit this for now)
* ratio
* threshold (the point where the compression kicks in)
* characteristics = {soft-knee, hard-knee)

I would use threshold (double) and characteristics (enum) and ration (fraction) as gobject properties. I am not sure wheter the ration has to be a fraction or can't simply be a double too.
Comment 17 René Stadler 2006-10-09 18:59:30 UTC
I think it is best to cut back on it for now... signal compression is a very large field, so to focus back on the real topic of the bug: Seeing it that the reporter mistook clipping for "real" compression/limiting, the bug is invalid.  If someone can come up with a use case for an element that just CLAMPS() values, this should go into the audiofx plugin.

In regard to providing the "must-have" limiter for ReplayGain (the tanh/"smooth" variant I presented here): I will provide it as a very simple element alongside the ReplayGain volume element that is coming soon.  This gives applications at least the opportunity to provide the minimum limiting capabilities needed to run playback without clipping prevention.  Also, I can't find any other implementation of this style of limiter other than in a ReplayGain context[1], so having a separate element more or less specific for RG makes sense in the end.

[1]: This is implemented at least in xmms1 and flac (metaflac can apply RG, an undocumented feature as it is lossy).  David Robinson mentions in the proposed RG standard that the filter originates in Cool Edit Pro.
Comment 18 Sebastian Dröge (slomo) 2007-02-05 16:27:37 UTC
An element that just clamps values is now in audiofx: audioamplify with method=clip which is in plugins-good CVS.

As such I guess we can close this bug now as fixed and can open a new one for a compressor, which is now bug #404646.