GNOME Bugzilla – Bug 759141
Audio/video recording performance regression 1.4.5 -> 1.6.1
Last modified: 2018-01-20 11:37:13 UTC
Hi, While updating GStreamer from 1.4.5 to 1.6.1 in NixOS, I found that video recording performance regressed quite a bit (lots of dropped audio samples). I initially found this in Cheese (webcam app), but it was easily reproducible on the command line. In 1.4.5 I could capture just fine at 1280x720, but with 1.6.1 I have to stay at 640x480 or lower to prevent dropped audio samples: $ gst-launch-1.0 -e webmmux name=mux ! filesink location=test.webm v4l2src ! video/x-raw,width=1280,height=720 ! videoconvert ! vp8enc ! queue ! mux. autoaudiosrc ! audio/x-raw,rate=44100 ! vorbisenc ! queue ! mux. Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstPulseSrcClock Redistribute latency... Redistribute latency... WARNING: from element /GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls: Can't record audio fast enough Additional debug info: gstaudiobasesrc.c(866): gst_audio_base_src_create (): /GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstPulseSrc:autoaudiosrc0-actual-src-puls: Dropped 26019 samples. This is most likely because downstream can't keep up and is consuming samples too slowly. The video has so much dropped audio that it's completely useless. (Maybe ~50 percent of the real time audio gets recorded?) If you have the Nix package manager, you can reproduce by checking out https://github.com/NixOS/nixpkgs/pull/11491 and running $ timeout -s INT 30s nix-shell -p gst_all_1.gstreamer gst_all_1.gst-plugins-base gst_all_1.gst-plugins-good gst_all_1.gst-plugins-bad gst_all_1.gst-plugins-ugly less time -I nixpkgs=. --run "\time gst-launch-1.0 -e webmmux name=mux ! filesink location=test.webm v4l2src ! video/x-raw,width=1280,height=720 ! videoconvert ! vp8enc ! queue ! mux. autoaudiosrc ! audio/x-raw,rate=44100 ! vorbisenc ! queue ! mux." (Then switch back to HEAD~4 to get back to gstreamer 1.4.5 and see the difference.) My system is Intel i7-5500U CPU @ 2.40GHz with 8 GiB RAM (laptop). Best regards, Bjørn Forsman
User thiblahute from pitivi IRC channel tested on his archlinux system with gstreamer 1.4 and 1.6. Version 1.6 dropped audio samples, 1.4 did not. On my system, the recording loads the CPU to 100 % (both gst 1.4 and 1.6), so the question might be not just about performance (maybe _slightly_ degraded in 1.6?), but how graceful gstreamer is at coping with an overloaded system. The above command (previous post) indicates that the error happens before 30 seconds. That is not true, it may happen later, but for my system it happens typically within the first minute.
Try autoaudiosrc is-live=1
It still drops audio samples with "autoaudiosrc is-live=1".
I am confused, I read autovideosrc here. Try setting a deadline to vp8enc. gst-launch-1.0 autoaudiosrc ! vorbisenc ! queue ! mux. v4l2src ! video/x-raw,width=1280,height=720 ! videoconvert ! vp8enc deadline=1 ! queue ! mux. webmmux name=mux ! filesink location=test.webm Otherwise VP8 behaviour is to max out the CPU. It will also introduce a lot of delay which is higher then the audiosrc capacity (hence buffer being dropped). deadline=1 is use for zero-latency pipeline.
That pipeline worked like a charm! Thanks, Nicolas. I recorded a 5 min video without any audio drop-outs. (And thanks for the info about what that deadline=1 parameter does.) So, what is the bug here? Did gstreamer change a default setting between 1.4.5 and 1.6.1? Is Cheese (the webcam app) using a bad pipeline and is being punished for it by (a stricter) gstreamer 1.6.1? And I happened to construct a similarly bad pipeline while trying to find an easy way to reproduce this issue? :-)
That's an excellent question, we need to define what is a bug here, bad default ? Wrong reported latency ? A mix ?
I looked up the gstreamer source code and found that the default value for "deadline" in vp8enc is 0 (by macro expansion), and it comes from https://github.com/webmproject/libvpx/blob/master/vpx/vpx_encoder.h#L898-L903, pasted below: #define VPX_DL_REALTIME (1) /**< deadline parameter analogous to * VPx REALTIME mode. */ #define VPX_DL_GOOD_QUALITY (1000000) /**< deadline parameter analogous to * VPx GOOD QUALITY mode. */ #define VPX_DL_BEST_QUALITY (0) /**< deadline parameter analogous to * VPx BEST QUALITY mode. */ So it selects "best quality" by default. I tried with "good quality" mode, but that too drops audio. No wonder, as it is 1000000x longer deadline than the "realtime" mode :-) I don't know what is worse, dropping audio in recording scenarios or having less than best quality when doing offline / non-realtime transcoding (or whatnot)? Well, I'm a bit biased now because I hit this issue, so I guess I'd prefer the realtime mode by default. But I appreciate that it's not a simple choice to make. And either way, Cheese should really specify "vp8enc deadline=1" in its pipeline. I think I'll suggest that NixOS switches gstreamer from 1.4.5 to 1.6.1 now. Thanks for your help, Nicolas.
We could find at least a default deadline that fits the sources default buffer time (200ms). If you want higher quality, you'll need to increase the source to compensate (see buffer-time properties on source, autoaudiosrc don't show it, but will forward the option to it's child).
Can this be closed then? If not, what's left to do?
Closing this bug report as no further information has been provided. Please feel free to reopen this bug report if you can provide the information that was asked for in a previous comment. Thanks!