GNOME Bugzilla – Bug 340362
[PATCH] new plugin - hardlimiter
Last modified: 2007-02-05 16:27:37 UTC
This is a simple brickwall audio limiter, originally used for my pet project. Supporting both integer and floating-point audio samples, it does not requantize incoming signal, so it can be losslessly applied to dithered audio. I want to propose it for inclusion in GStreamer, because such a limiter is a must-have for any audio player with replaygain support: http://replaygain.hydrogenaudio.org/player.html http://amarok.kde.org/amarokwiki/index.php/ReplayGain/VorbisGain_Support_-_What_Would_it_Take%3F#With_respect_to_the_player A -6dB hardlimiter may be also used to enchance output on some soundcard models (a signal level above -6dB may be unsafe (introduce clipping) on consumer hardware). The attachement is full autoconfed source package, based on gst-template. The plugin only needs to be updated to use GST_LICENSE, GST_PACKAGE_NAME and GST_PACKAGE_ORIGIN in GST_PLUGIN_DEFINE macro.
Created attachment 64646 [details] the plugin
Nice! Re-assigning to gst-plugins-bad (that's where new plugins get added first).
Created attachment 65249 [details] [review] patch against gst-plugins-bad This looks like very nice and clean code. Attached a modified version as a patch against gst-plugins-bad. Some minor things I came across: - if the type is called GstHardLimit, functions should be gst_hard_limit_foo and not gst_hardlimit_foo as well - you don't need GST_BOILERPLATE_FULL here, the debug init stuff is usually done in plugin_init or the base init function - in gst_hard_limiter_transform_ip() - why just return GST_FLOW_OK if the buffer isn't writable? Is that an expected situation? If not, we should probably g_assert() it instead. - there's a GST_BUFFER_TIMESTAMP_IS_VALID macro as well - the HARDLIMITER_DO_LIMIT_FUNCTION macro in hardlimit.h uses 'static inline' for the functions but they won't be inlined (I think) because they're refered to by address and assigned to filter->do_limit, so the whole transform business is highly inefficient at the moment because it involves a function call for every single sample in the buffer. - might make sense to use the G_UNLIKELY macros in the hardlimiter_limit functions - also - minor point: the element doesn't actually seem to do anything at all no matter what :) I tracked this down to the buffer writable check mentioned above. Adding something like gstbasetrans_class->passthrough_on_same_caps = TRUE; and later gst_base_transform_set_in_place (GST_BASE_TRANSFORM (filter), TRUE); seems to fix this and give us a writable buffer. Lastly, could you explain to me the exact difference between 'hard limiting' and 'clipping'? Apologies in advance for my ignorance, but it seems to me that all this element does is actually clip the signal at a certain level. I would have thought that 'hard limiting' involves reducing the signal in general once the limit has been reached (with a 0-second attack, thus 'hard') or something like that. Please correct me if I'm wrong :)
Oh also, I don't understand the GstController stuff - can you explain to me how that is supposed to be used in this case? (again apologies for my ignorance)
I like this as well, excellent work, code looks nice. Same comments as tim above, please reply to them so we can move this in. Additional comment - if the element is called "hardlimit", why is it in a dir called audiocompress ? Are you planning to add more compressors in the same plugin ?
(In reply to comment #5) > Additional comment - if the element is called "hardlimit", why is it in a dir > called audiocompress ? Are you planning to add more compressors in the same > plugin ? Actually, that was my doing. I figured if there were more compressors to come they could go in there, no particular reason other than that. I like to keep the plugin name different / more general from element names if it makes sense. Can rename it to 'hardlimit' if you prefer that.
Answering questions: - GST_BOILERPLATE_FULL was inserted by the template tool from gst-template package. I thought all elements require this one. - transform_ip() indeed should return GST_FLOW_OK if buffer is unwritable (checking if in passthrough mode). - strange, that the element did not do anything for you. It shouldn't passthrough on same caps and works for me here. - yes, hardlimiting sounds like clipping/distortion. And sounds horrible but still better than soundcard clipping. However, I'm currently writing a more general compressor unit that should provide a better sounding solution. - GstController stuff also was sitting in the template. I thought it's generally a good idea to have controlled elements. Aren't they especially useful for NLE apps? Correct me, if I'm wrong.
By the way, did you guys consider creating a "dsp" directory or even something like a "gst-plugins-dsp" module for all the audio-processing stuff?
> - GST_BOILERPLATE_FULL was inserted by the template tool from gst-template > package. I thought all elements require this one. Apologies, I missed that that is from template :) (It's ugly nevertheless IMHO) > - transform_ip() indeed should return GST_FLOW_OK if buffer is unwritable > (checking if in passthrough mode). > - strange, that the element did not do anything for you. It shouldn't > passthrough on same caps and works for me here. No idea about these, have to investigate. > - yes, hardlimiting sounds like clipping/distortion. And sounds horrible but > still better than soundcard clipping. I was looking at the wave curve in a sound editor, not listening to it :) > However, I'm currently writing a more general > compressor unit that should provide a better sounding solution. Great! > - GstController stuff also was sitting in the template. I thought it's > generally a good idea to have controlled elements. Aren't they especially > useful for NLE apps? Correct me, if I'm wrong. No, you're absolutely right. I thought the GstController stuff was supposed to serve another purpose and got confused.
I have to say that this element is _not_ a hard limiter. Also, it is _wrong_ to say that hard limiting sounds bad and it _is_ true that this element just does clipping (which can never sound better than "sound card clipping"). Using this element can only mess up the output, I don't see a use case for it. The example about consumer DACs that the reporter provides is flawed. -6 dB hard limiting means to apply a transfer function like ouput = tanh ((input - 0.5) / 0.5) * 0.5 + 0.5 to all values with input > 0.5 and a respective negative variant for values < -0.5. This will smoothly compress values above 0.5 (ca. -6 dB), which has the following advantages: - You can input values >1.0 and still get a good sounding output (unless you really overdo it). This is of use for ReplayGain if you disable clipping prevention (which you don't have to). - The dynamic range gets reduced (input 1.0 -> output ca. 0.88). This allows you for example to listen to classical music through loudspeakers in front of a computer (noisy environment), or for playing music on a party. This is entirely unrelated to ReplayGain. You really have to watch out when investigating about DSP stuff on the web. Lots of seemingly useful documents talk about an analog processing background. I assume this is how the reporter got the wrong idea that clipping is useful. In the analog world, you need such a thing as protection for equipment or as last resort protection to make sure output is always within certain limits (radio stations).
I can second René's comments. Dynamic compression, expansion, limmiting can also be seen as a lookup-table transformation in->out. A compressor starts to work for a signal above a threshold and then compresses the signal. A hard limmiter clamps the signal above the treshhold to the treshhold. A compressor compresses signals above the threshold, e.g. a compressor with the ration 2:1 would do out = thresh + ((in-thresh)/2.0) That would be a compressor with hard knee characteristcs. The other bahaviour is soft-knee which would use a smooth curved transition when going from linear to compressed. As a sidenote - in gst-plugins-good we now have the audiofx plugin, where such elements could possibly be added (to keep number of plugins low).
Created attachment 73746 [details] Graphs of different transfer functions A picture says more than thousand words: 1. Red graph: Clipping (as originally proposed by Артём) 2. Blue graph: Linear filter (like Stefan mentioned) 3. Green graph: Smooth hard limiting (like I mentioned) All filters have a threshold of 0.5 (ca. -6 dB), that is they leave values below 0.5 (above -0.5) intact. The picture shows clearly why for clipping, the waveform gets irreversibly distorted if it exceeds the threshold. I mention this because some people seem to have the impression that clipping can somehow be undone by reducing the volume or something. It is not. Clipping is also done implicitely by audioconvert when it converts from float to integer, which is most often done to finally send the data to the sound card. That's the red area on the graphs, it can never be reached in integer formats as it is beyond what is defined as maximum. Second one is a real signal compressor that gives good results instead of useless distortion. It's exactly what Stefan said: Basically a selective volume change that only operates on values exceeding a threshold. It is very efficient (just a multiplication in addition to the switching), but that comes at a price: Trained ears (which I don't claim to have) seem to be able to hear the point of the threshold because of the rather abrupt change that exceeding the threshold introduces. This depends on the input data and especially the ratio of course. The third one avoids that by giving smooth values instead. These come at the price of higher computational comlexity (a tanhf in addition to the switching plus scaling around a bit).
more explanation -> http://en.wikipedia.org/wiki/Audio_level_compression
Rene, FWIW, the two last proposal (stefan and yours) both have their advantages (one is fast, the other more professional) and disadvantages (one doesn't avoid clipping, the other one is slow), and considering they would share the exact same code, except for the actual computation, how about making it one element with a method property to choose which method you want ? Setting it by default to the 'best' one (yours) but leaving an option to go to the 'fast' one .
I also thought about that, but how to sanely express the parameters as properties? Both have a threshold, but the linear one has a ratio and the smooth one a limit. Maybe call the mode enum nicks "smooth" and "linear", and let the element have four properties: mode, threshold, linear-ratio and smooth-limit or something. This should make it clear that either property is ignored depending on the mode. I completely ignore advanced features such as attack time or side-chaining for now. I think these make lots of sense for music production (buzztard, jokosher, etc.). The use case I focus on is playback.
In pro-audio devices compressors have four parameters: * attack (lets omit this for now) * ratio * threshold (the point where the compression kicks in) * characteristics = {soft-knee, hard-knee) I would use threshold (double) and characteristics (enum) and ration (fraction) as gobject properties. I am not sure wheter the ration has to be a fraction or can't simply be a double too.
I think it is best to cut back on it for now... signal compression is a very large field, so to focus back on the real topic of the bug: Seeing it that the reporter mistook clipping for "real" compression/limiting, the bug is invalid. If someone can come up with a use case for an element that just CLAMPS() values, this should go into the audiofx plugin. In regard to providing the "must-have" limiter for ReplayGain (the tanh/"smooth" variant I presented here): I will provide it as a very simple element alongside the ReplayGain volume element that is coming soon. This gives applications at least the opportunity to provide the minimum limiting capabilities needed to run playback without clipping prevention. Also, I can't find any other implementation of this style of limiter other than in a ReplayGain context[1], so having a separate element more or less specific for RG makes sense in the end. [1]: This is implemented at least in xmms1 and flac (metaflac can apply RG, an undocumented feature as it is lossy). David Robinson mentions in the proposed RG standard that the filter originates in Cool Edit Pro.
An element that just clamps values is now in audiofx: audioamplify with method=clip which is in plugins-good CVS. As such I guess we can close this bug now as fixed and can open a new one for a compressor, which is now bug #404646.