GNOME Bugzilla – Bug 762929
simple transcoding pipeline takes over 4GB RAM
Last modified: 2016-05-22 18:07:42 UTC
Created attachment 322785 [details] dot file for the pipeline that causes the issue I have been told on IRC that this kind of simple transcoding should take about 500 MB RAM, but I experience huge RAM usage even above 4GB. I measure RSS. (Resident Set Size) on Ubuntu. I have been trying various types of input streams with about the same results. The memory consumption grows after hours. But will peak at about 4GB and not grow since then. With a few transcoders running, the kernel will start killing them. Most of memory is consumed by x264enc, but the mpegdecoder is also hungry. I can tell this from vagrind --massif. I cannot attach two files here I guess. I never got 4GB consumption under vagrind/massif, I think because it runs slower. Not sure if this is considered a bug, but since I've been told that is should definitely be under 1 GB I think there is a chance it is.
Which version of GStreamer are you using? Can you provide a gst-launch-1.0 line for this, and a public input stream you're using?
Here are some inputs we used for this: http://tva.fastbroad.com/i/kinoprank_1@136809/master.m3u8 http://109.104.129.195:10000/3 http://91.187.113.181:8001/Ekids http://208.49.220.228:8003/16982 Here is the output link to push: (this is just regular wowza) rtmp://4.31.30.153:1935/devapp/ramissue gst-launch-1.0 uridecodebin uri=http://208.49.220.228:8003/16982 ! videoconvert ! x264enc ! flvmux ! rtmpsink location=rtmp://4.31.30.153:1935/devapp/ramissue Mostly we test on 1.6.2, but 1.6.1 has the same issue.
By the way, we were trying the elecard eavcenc in the same pipeline with the same streams, and the memory consumption goes down very much. According to massif, eavcenc only uses 3 MB (compared to 1+ GB with x264enc).
What's eavcenc? And did you try if selecting a different rc-lookahead makes a difference for you with x264enc?
Elecard is paid codecs for GStreamer, they have free version with their logo. http://www.elecard.com/ I tried many options, including the one you mention (you suggested it on the mail list), and almost every other, "threads" for example. But I did not see much difference in memory consumption, (unless you set very low bitrate).
You'll have to run in valgrind's massif then and check *where* exactly all that memory is allocated.
Can't reproduce this here with 1.8.1 and gst-launch-1.0 videotestsrc ! videoconvert ! x264enc ! flvmux ! rtmpsink location=rtmp://4.31.30.153:1935/devapp/ramissue (none of the above input streams work for me). By default x264enc outputs very high quality with large internal buffers. You should be able to tweak that via properties, e.g. by setting tune=zerolatency and key-int-max and other properties. Not sure what we can do here without massif logs and more info.