GNOME Bugzilla – Bug 719506
gst-rtsp-server leaks file descriptors and sometimes service is unavailable
Last modified: 2018-01-13 16:04:14 UTC
When media is set shared, and there are two or more concurrent clients periodically connecting and disconnecting, GstRTSPServer suffers noticable problems. 1. Number of file descriptors open by server process increases until service is denied because of allowed FD limit reached. These FDs are not sockets, because netstat -napt does not show excessive entries, but /proc/$PID/fdinfo shows that there is a lot of them. When all clients disconnect (which should lead to destruction of GstRTSPMedia instance) excessive FDs are not freed, too. 2. Sometimes clients fail getting "503 Service Unavailable" reply for DESCRIBE request. I've noticed such server log messages which may be related to the issue (they go together): ERROR rtspclient rtsp-client.c:599:find_media: client 0x7f3224002960: can't prepare media ERROR rtspclient rtsp-client.c:1819:handle_describe_request: client 0x7f3224002960: no media FIXME rtspmedia rtsp-media.c:2392:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing WARN basesrc gstbasesrc.c:1586:gst_base_src_perform_seek:<videotestsrc13> duplicate event found 5210 3. There may be some issue with timestamps: ffmpeg laments "Application provided invalid, non monotonically increasing dts to muxer in stream 0: 87001 >= 36000". Such messages never appear when there is single client. And when second client connects, a lot of such errors are produced to the first one infinitely. Revisions used: gst-rtsp-server b1e8172ef366458eecccd5f0d3b77d87fbb30d4a (today) gstreamer 73a58cb0a2b861df6c20a8c30747117562fa7523 (today) gst-plugins-base 1d40b1d2bb68963941720c588bfcc94d6fa3f939 (today) gst-plugins-good b8f689a9d97b7a90a3c40d5249c964e35a000b4a (today) gst-plugins-bad 1.2.0 Used application: https://gist.github.com/krieger-od/7618457 , derived from https://github.com/RidgeRun/eval-sdk-imx6/blob/master/myapps/rr-rtsp-server/src/src/rr_rtsp_server.c Command line: GST_DEBUG=3 ./rr "( videotestsrc ! x264enc ! rtph264pay name=pay0 pt=96 )" Output: https://gist.github.com/krieger-od/7695759 Clients command and output: https://gist.github.com/krieger-od/7695806
(In reply to comment #0) > 1. Number of file descriptors open by server process increases until service is > denied because of allowed FD limit reached. These FDs are not sockets, because > netstat -napt does not show excessive entries, but /proc/$PID/fdinfo shows that > there is a lot of them. When all clients disconnect (which should lead to > destruction of GstRTSPMedia instance) excessive FDs are not freed, too. Still investigating this. It seems to all free things correctly here from a quick check with valgrind and 2 ffmpeg sources. Maybe this fix helps: commit 3b4894c4f1ea28cc317f0c53b110cf678d27bf10 Author: Ognyan Tonchev <ognyan@axis.com> Date: Fri Nov 29 15:50:23 2013 +0100 media: also do state change in suspended state > > 2. Sometimes clients fail getting "503 Service Unavailable" reply for DESCRIBE > request. > I've noticed such server log messages which may be related to the issue (they > go together): > ERROR rtspclient rtsp-client.c:599:find_media: client 0x7f3224002960: can't > prepare media > ERROR rtspclient rtsp-client.c:1819:handle_describe_request: client > 0x7f3224002960: no media > FIXME rtspmedia rtsp-media.c:2392:gst_rtsp_media_suspend: suspend for dynamic > pipelines needs fixing > WARN basesrc gstbasesrc.c:1586:gst_base_src_perform_seek:<videotestsrc13> > duplicate event found 5210 The warning is unrelated and the fix for the 503 is here: commit 53859ac34be48bc3b8210e4598e885ea04e3efa8 Author: Wim Taymans <wtaymans@redhat.com> Date: Fri Nov 29 10:53:08 2013 +0100 media: also handle prepare and range in suspended state When we are suspended, we are already prepared. We can get the range in the suspended state. > > 3. There may be some issue with timestamps: ffmpeg laments "Application > provided invalid, non monotonically increasing dts to muxer in stream 0: 87001 > >= 36000". Such messages never appear when there is single client. And when > second client connects, a lot of such errors are produced to the first one > infinitely. This looks like a bug in ffmpeg. I don't see timestamps go backwards.
503 fixed indeed. FD leak is still there (timestamps issue also). Could you please check with 2-3 instances of script: while true; do ffmpeg -i rtsp://localhost:8554/test -t 5 -codec copy -f null /dev/null -y; sleep 1; done Of course you may use another tool instead of ffmpeg, which can be set to disconnect after some time.
There is still such message, maybe it was left by mistake? FIXME rtspmedia rtsp-media.c:2394:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing
Wim?
Does this still happen with recent versions of gst-rtsp-server / GStreamer?
I'm going to close this to reduce bugzilla clutter. Running the test command line on ./test-launch '( videotestsrc ! x264enc tune=zerolatency ! rtph264pay name=pay0 )' and checking /proc/$pid/fdinfo shows a steady number of fds (7 or 19 depending on whether a client is connected or not). Same for multiple clients. Please re-open or file new bugs if there are still issues with recent versions of GStreamer. Please file separate bugs for separate issues in future.