GNOME Bugzilla – Bug 646474
rtpspeexpay should drop empty samples
Last modified: 2011-04-08 11:58:04 UTC
speexenc has dtx (Discontinuous Transmission) option. If it enabled, speex generate dummy samples if it get silence on input. This dummy samples are needed if we write a file, but thay are useless if we stream it over network with rtp. With current speexenc ! rtpspeexpay ... i get about 30 kbit/s with speexenc vbr=1 dtx=1 ! rtpspeexpay ... i get about 25 kbit/s and about 19 kbit/s on complete silence. This can be reduced if rtpspeexpay will drop this dummy samples or speexenc will have option to not generate it. Here is the sample where it can be done. diff --git a/ext/speex/gstspeexenc.c b/ext/speex/gstspeexenc.c index 4a0bba6..d4ded47 100644 --- a/ext/speex/gstspeexenc.c +++ b/ext/speex/gstspeexenc.c @@ -901,7 +901,7 @@ gst_speex_enc_encode (GstSpeexEnc * enc, gboolean flush) while (gst_adapter_available (enc->adapter) >= bytes) { gint16 *data; - gint outsize, written; + gint outsize, written, dtx_ret; GstBuffer *outbuf; data = (gint16 *) gst_adapter_take (enc->adapter, bytes); @@ -913,10 +913,13 @@ gst_speex_enc_encode (GstSpeexEnc * enc, gboolean flush) if (enc->channels == 2) { speex_encode_stereo_int (data, frame_size, &enc->bits); } - speex_encode_int (enc->state, data, &enc->bits); + dtx_ret = speex_encode_int (enc->state, data, &enc->bits); g_free (data); + if (!dtx_ret) + continue; + enc->frameno++; enc->frameno_out++;
my understanding is that the silence bits aren't pure silence, they should contain information for CNG (Comfort noise generation) by the receiver. So you want to transmit them anyway (at least enough of them).
this fames do _not_need_ to be transmitted. Please take a look at this parts: speex-1.2~rc1/include/speex/speex.h: .... /** Uses an existing encoder state to encode one frame of speech pointed to by "in". The encoded bit-stream is saved in "bits". @param state Encoder state @param in Frame that will be encoded with a +-2^15 range. This data MAY be overwritten by the encoder and should be considered uninitialised after the call. @param bits Bit-stream where the data will be written @return 0 if frame needs not be transmitted (DTX only), 1 otherwise */ int speex_encode(void *state, float *in, SpeexBits *bits); /** Uses an existing encoder state to encode one frame of speech pointed to by "in". The encoded bit-stream is saved in "bits". @param state Encoder state @param in Frame that will be encoded with a +-2^15 range @param bits Bit-stream where the data will be written @return 0 if frame needs not be transmitted (DTX only), 1 otherwise */ int speex_encode_int(void *state, spx_int16_t *in, SpeexBits *bits); /** Used like the ioctl function to control the encoder parameters * * @param state Encoder state * @param request ioctl-type request (one of the SPEEX_* macros) * @param ptr Data exchanged to-from function * @return 0 if no error, -1 if request in unknown, -2 for invalid parameter And here: http://www.speex.org/docs/manual/speex-manual/node4.html#SECTION00418000000000000000 Discontinuous Transmission (DTX) Discontinuous transmission is an addition to VAD/VBR operation, that allows to stop transmitting completely when the background noise is stationary. In file-based operation, since we cannot just stop writing to the file, only 5 bits are used for such frames (corresponding to 250 bps).
Sorry, you're right.. It should probably dropped (unconditionally) in the payloader then.
maybe the encoder wants to mark them with GST_BUFFER_FLAG_GAP (although this may be an abuse of the GAP flag)
GAP buffers are defined as neutral buffers and are used for silence in audio. Sounds like a good use of the flag IMHO :)
Created attachment 185432 [details] [review] speex_patch This patch add gap flag on silence (in speexenc) and drop GAP packets in rtpspeexpay. The network traffic usage drops to _1!_kbit/s_ if microphone is muted. To make usage of silence detection use ... ! speexenc vbr=1 dtx=1 ! rtpspeexpay ! ...
Created attachment 185505 [details] [review] 0001-Use-speex-intern-silence-detection
Created attachment 185506 [details] [review] 0002-Do-not-transmitt-samples-with-GAP-flag This two patches are just splitted version of my first patch
commit 9b15f9c6a12a8d20fcce1844e4fba8f3df466404 Author: Alexey Fisher <bug-track@fisher-privat.net> Date: Fri Apr 8 11:13:07 2011 +0200 rtpspeexpay: Do not transmitt samples with GAP flag If we get GAP samples, there is no need to transmitt it. In some situations, microphone is muted, we can drop net traffick usage to ~1 kbit/s. Without patch it will stay ~20 kbit/s commit 0016ceaa2b0c56d7522eb3c412768c9371fd23bd Author: Alexey Fisher <bug-track@fisher-privat.net> Date: Fri Apr 8 11:11:58 2011 +0200 speexenc: Use speex intern silence detection Speex has build in silence detection. If speex_encode_int returns 0, than there is silence and sample do not need to be transmitted. This work only if vbr=1 and dtx=1 optionas are enabled. So if we get 0, we add GAP flag to the sample.