GNOME Bugzilla – Bug 644945
Gstreamer audio synchronization fails at low latency.
Last modified: 2013-07-24 15:07:19 UTC
The following pipeline usually fails to play sound on the ubuntu lucid desktops I have tried: {{{ rosrun gstreamer gst-launch alsasrc latency-time=10000 ! alsasink buffer-time=20000 latency-time=10000 }}} Folks on my team are also getting similar problems with drivers other than alsa, but I don't have hard data yet on what is going on in those cases. As far as I can tell, the problem is that the alsasink is stuffing data into libasound as fast as it can, and libasound is willing to buffer a lot more than 10ms worth of data. Hence, the read pointer in the ring buffer gets ahead of the write pointer, and all you hear is silence. I am basing this interpretation on the following observations: - When playback starts, you see a bunch of reading from the ring buffer. - Before the batch of reads, diff is positive, after the batch of reads diff is negative (so everything gets dropped). - The delay goes way up during the batch of reads. I believe the correct way to fix this is for gst_ring_buffer_prepare_read to check if a segment is available, check the current delay on the audio device, and if the delay on the audio device is sufficient, wait on the ring buffer condition variable. Similarly the commit method should notify the condition variable to wake up the reading thread. I would be interested in any comments on this approach, and hope to be providing a patch to fix this.
Generally it is a very bad idea to have the device wait for anything at all. I have seen BSOD on Windows and complete sound-system failure on OSX as a result of doing this. So don´t :) Another tip is to not interpret, but rather log out all read/write access so that you are able to simulate the failing scenario. My guess is that you are being a bit unfair with that pipeline. Remember that your pipeline latency effectively will end up as headroom in your ringbuffer. If there is no, or only very little headroom, you *will* get under-runs (glitches / silence), this is the case for all ringbuffers. If you simply were to increase the latency of the pipeline with a bit more, you would effectively increase the headroom in the ringbuffer, making underruns more unlikely. I would also be interested to know if this patch will help a bit: Given that your buffers is 10ms, and the resync threshold is default 20ms, it could very well be that you end up in that "grey" zone that this patch addresses: http://cgit.freedesktop.org/gstreamer/gst-plugins-base/commit/?id=588ac0ae6fdfb55c612cc84f5426cbf1ee4aec4e
Havard, Blaise: any updates ?
The patch referenced made a huge difference in audio-glitches for my previous employer, but if this fixed things for Blaise I can't really say?
I'm afraid we switched away from gstreamer a couple of years ago so I don't have any updates. If somebody wanted to work on it, the best bet would be to attempt to reproduce from the pipeline in my original post.
The situation described sounds similar to what I saw elsewhere. There the problem was that the alsa driver and the hardware were completely unreliable about their period-time/buffer-time. In any case this problem should be caused by (as Håvard said) not enough headroom in the sink, or by your sink reading faster than the source can provide data (clock differences?). There's not really point in waiting on the reader side in both cases as you will only be too late providing data by waiting. When the reader reads, it is time to transform the data into sound waves, there's no time left to wait a bit. You would get this waiting however if you set a higher latency, in which case the playback would just happen overall a bit later and there's more time for the source to fill more of the buffer in the sink.
If I believe the initial comment I wrote, the problem was that alsa was willing to buffer a lot more data than gstreamer was anticipating. Alsa wasn't asking for data when it was needed, it was asking for as much data as it could put in its multi-second (if I recall correctly) buffer. The data was being requested long before gstreamer was getting it, gstreamer was assuming that Alsa would only buffer 10ms, so gstreamer was reading from a place in the ring buffer that would only get filled much later, and that was hence full of zeros. Thus, no sound was coming out. You should note that alsasink uses the snd_pcm_writei API for sending data to the device, and snd_pcm_wait to wait for free space. If the audio device's buffer is larger than gstreamer expects, snd_pcm_wait could return long before the device actually needs data, and long before the data is actually available. In that case, I stand by my opinion that waiting is the right thing to do. I'm sure that I investigated why alsa's buffer was so large at the time, but I'm afraid I don't recall the details. The only thing that I'm sure of is that it was very hardware dependent and unpredictable. And if you happen to be using dmix, or alsa going via pulse you're really in for a mess.
That sounds like a bug in the driver then, we explicitly set buffer and period sizes, then afterwards query whatever the driver has set and then work with these values. If the driver is doing something completely different than it says it does, stuff will go horribly wrong
I agree that with non-buggy drivers life would be easier, but try to explain that to a customer who says their sound works find with some other product. In any case, I'm not trying to start an argument here. I no longer use gstreamer so I have vested interest here. Nevertheless, my observation was that in many cases the simple pipeline above fails to produce any sound, and in practice, that is a major limitation for an application based on gstreamer.
@Blaise: Do you remember which device/driver you were using?
I'm afraid not, but I wouldn't be surprised if I was going alsa to pulse. And my recollection is that the latency reporting for alsa to pulse is very sketchy. Honestly, I'm not sure there is enough information there to be successful.
I'm curious, does the pipeline at the top of this email work well for you guys?
(In reply to comment #11) > I'm curious, does the pipeline at the top of this email work well for you guys? Works for me. It's doing alsa over pulse for me like that. This also works. gst-launch-1.0 alsasrc device=hw:0 latency-time=10000 ! alsasink device=hw:0 buffer-time=20000 latency-time=10000 There are small noise clicks because the sink has not much headroom, doing latency-time=5000 makes it perfect without any noise.
Could be that pulse has improved in the past couple of years. We were running Ubuntu Lucid at the time. Frankly, this ticket is so old, and the reporter (that's me) has moved on so much, that if this were me I'd probably resolve the ticket as cannot reproduce.
Let's just do that then :) If someone still has the problem it will be reported sooner or later