GNOME Bugzilla – Bug 789999
hlsdemux: Fix memory leak caused by double buffer map
Last modified: 2017-11-20 08:44:20 UTC
Created attachment 363120 [details] Valgrind definitely lost HLS playback ubuntu 16.04 + gstreamer sdk 1.12.3 valgrind -v --tool=memcheck --track-fds=yes --leak-check=full --show-reachable=yes --time-stamp=yes --undef-value-errors=no --malloc-fill=0xc --free-fill=0xd --freelist-vol=100000000 --trace-children=yes --num-callers=50 --suppressions=/home/shakin/work/src/ssc/gstreamer/cerbero/build/sources/linux_x86_64/gstreamer-1.0-1.12.2/common/gst.supp --log-file=valgrind_gst_bipbopall gst-launch-1.0 playbin uri=http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8 --- ==00:00:00:23.619 9299== 8,494 bytes in 2 blocks are definitely lost in loss record 11,003 of 11,064 ==00:00:00:23.619 9299== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==00:00:00:23.619 9299== by 0x5413D98: g_malloc (gmem.c:94) ==00:00:00:23.619 9299== by 0x542C5C2: g_slice_alloc (gslice.c:1025) ==00:00:00:23.619 9299== by 0x4E6BC99: _sysmem_new_block (gstallocator.c:417) ==00:00:00:23.619 9299== by 0x4E773D1: gst_buffer_new_allocate (gstbuffer.c:839) ==00:00:00:23.619 9299== by 0x8649194: gst_base_src_default_alloc (gstbasesrc.c:1427) ==00:00:00:23.619 9299== by 0x8D2015B: gst_soup_http_src_alloc_buffer (gstsouphttpsrc.c:1237) ==00:00:00:23.619 9299== by 0x8D2015B: gst_soup_http_src_read_buffer (gstsouphttpsrc.c:1567) ==00:00:00:23.619 9299== by 0x8D2015B: gst_soup_http_src_create (gstsouphttpsrc.c:1688) ==00:00:00:23.619 9299== by 0x864B1F5: gst_base_src_get_range (gstbasesrc.c:2453) ==00:00:00:23.619 9299== by 0x864CDCE: gst_base_src_loop (gstbasesrc.c:2729) ==00:00:00:23.619 9299== by 0x4EE1D80: gst_task_func (gsttask.c:335) ==00:00:00:23.619 9299== by 0x54371ED: g_thread_pool_thread_proxy (gthreadpool.c:307) ==00:00:00:23.619 9299== by 0x5436854: g_thread_proxy (gthread.c:784) ==00:00:00:23.619 9299== by 0x570A6B9: start_thread (pthread_create.c:333) ... ==00:00:00:23.631 9299== LEAK SUMMARY: ==00:00:00:23.631 9299== definitely lost: 8,494 bytes in 2 blocks ==00:00:00:23.631 9299== indirectly lost: 0 bytes in 0 blocks ==00:00:00:23.631 9299== possibly lost: 0 bytes in 0 blocks ==00:00:00:23.631 9299== still reachable: 2,134,231 bytes in 30,992 blocks ==00:00:00:23.631 9299== of which reachable via heuristic: ==00:00:00:23.631 9299== length64 : 10,848 bytes in 252 blocks ==00:00:00:23.631 9299== newarray : 2,000 bytes in 45 blocks ==00:00:00:23.631 9299== suppressed: 2,198,060 bytes in 27,779 blocks ==00:00:00:23.631 9299== ==00:00:00:23.631 9299== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 177 from 177) BTW,playbin3 has three more indirectly lost.
Can anybody else confirm this?
Leak tracer complain "WARNING **: Leaks detected" while HLS playback. shakin@shakin-dev:~/Desktop$ GST_TRACERS="leaks" GST_DEBUG="GST_TRACER:7" gst-launch-1.0 playbin uri=http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8 -------- shakin@shakin-dev:~/Desktop$ GST_TRACERS="leaks" GST_DEBUG="GST_TRACER:7" gst-launch-1.0 playbin uri=http://devimages.appl e.com/iphone/samples/bipbop/bipbopall.m3u8 0:00:00.015422315 14178 0x1ca68f0 DEBUG GST_TRACER gsttracer.c:164:gst_tracer_register:<registry0> update existing feature 0x1ac5a80 (latency) 0:00:00.015465028 14178 0x1ca68f0 DEBUG GST_TRACER gsttracer.c:164:gst_tracer_register:<registry0> update existing feature 0x1ac5b40 (log) 0:00:00.015478961 14178 0x1ca68f0 DEBUG GST_TRACER gsttracer.c:164:gst_tracer_register:<registry0> update existing feature 0x1ac5c00 (rusage) 0:00:00.015491082 14178 0x1ca68f0 DEBUG GST_TRACER gsttracer.c:164:gst_tracer_register:<registry0> update existing feature 0x1ac5cc0 (stats) 0:00:00.015502321 14178 0x1ca68f0 DEBUG GST_TRACER gsttracer.c:164:gst_tracer_register:<registry0> update existing feature 0x1ac5d80 (leaks) 0:00:00.015533201 14178 0x1ca68f0 TRACE GST_TRACER gsttracerrecord.c:111:gst_tracer_record_build_format: object-alive.class, type-name=(structure)"value\,\ type\=\(type\)gchararray\,\ related-to\=\(GstTracerValueScope\)GST_TRAC ER_VALUE_SCOPE_PROCESS\;", address=(structure)"value\,\ type\=\(type\)gpointer\,\ related-to\=\(GstTracerValueScope\)GST_T RACER_VALUE_SCOPE_PROCESS\;", description=(structure)"value\,\ type\=\(type\)gchararray\,\ related-to\=\(GstTracerValueSco pe\)GST_TRACER_VALUE_SCOPE_PROCESS\;", ref-count=(structure)"value\,\ type\=\(type\)guint\,\ related-to\=\(GstTracerValueS cope\)GST_TRACER_VALUE_SCOPE_PROCESS\;", trace=(structure)"value\,\ type\=\(type\)gchararray\,\ related-to\=\(GstTracerVal ueScope\)GST_TRACER_VALUE_SCOPE_PROCESS\;"; 0:00:00.015575953 14178 0x1ca68f0 DEBUG GST_TRACER gsttracerrecord.c:125:gst_tracer_record_build_format: new format string: object-alive, type-name=(string)%s, address=(gpointer)%p, description=(string)%s, ref-count=(uint)%u, trace=(string)%s; 0:00:00.015597259 14178 0x1ca68f0 TRACE GST_TRACER gsttracerrecord.c:111:gst_tracer_record_build_format: object-refings.class, ts=(structure)"value\,\ type\=\(type\)guint64\,\ related-to\=\(GstTracerValueScope\)GST_TRACER_VALUE _SCOPE_PROCESS\;", type-name=(structure)"value\,\ type\=\(type\)gchararray\,\ related-to\=\(GstTracerValueScope\)GST_TRACE R_VALUE_SCOPE_PROCESS\;", address=(structure)"value\,\ type\=\(type\)gpointer\,\ related-to\=\(GstTracerValueScope\)GST_TR ACER_VALUE_SCOPE_PROCESS\;", description=(structure)"value\,\ type\=\(type\)gchararray\,\ related-to\=\(GstTracerValueScop e\)GST_TRACER_VALUE_SCOPE_PROCESS\;", ref-count=(structure)"value\,\ type\=\(type\)guint\,\ related-to\=\(GstTracerValueSc ope\)GST_TRACER_VALUE_SCOPE_PROCESS\;", trace=(structure)"value\,\ type\=\(type\)gchararray\,\ related-to\=\(GstTracerValu eScope\)GST_TRACER_VALUE_SCOPE_PROCESS\;"; 0:00:00.015633610 14178 0x1ca68f0 DEBUG GST_TRACER gsttracerrecord.c:125:gst_tracer_record_build_format: new format string: object-refings, ts=(guint64)%lu, type-name=(string)%s, address=(gpointer)%p, description=(string)%s, re f-count=(uint)%u, trace=(string)%s; Setting pipeline to PAUSED ... Pipeline is PREROLLING ... Redistribute latency... Prerolled, waiting for buffering to finish... Pipeline is PREROLLED ... Setting pipeline to PLAYING ... New clock: GstPulseSinkClock ^Chandling interrupt. Interrupt: Stopping pipeline ... Execution ended after 0:00:02.676396656 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ... 0:00:15.493299195 14178 0x1ca68f0 TRACE GST_TRACER :0:: object-alive, type-name=(string)GstMemory, addres s=(gpointer)0x7fc48c00a9a0, description=(string)0x7fc48c00a9a0, ref-count=(uint)1, trace=(string); 0:00:15.493330891 14178 0x1ca68f0 TRACE GST_TRACER :0:: object-alive, type-name=(string)GstMemory, addres s=(gpointer)0x7fc48c009900, description=(string)0x7fc48c009900, ref-count=(uint)1, trace=(string); ** (gst-launch-1.0:14178): WARNING **: Leaks detected
Non-playbin's pipeline has the same issure. GST_TRACERS="leaks" GST_DEBUG="6,GST_TRACER:7" GST_DEBUG_COLOR_MODE=off GST_DEBUG_FILE='/home/shakin/work/log/gst.log' GST_DEBUG_DUMP_DOT_DIR='/home/shakin/work/log' gst-launch-1.0 souphttpsrc location=http://devimages.apple.com/iphone/samples/bipbop/gear4/prog_index.m3u8 ! hlsdemux ! decodebin ! videoconvert ! videoscale ! autovideosink
Can be reproduced with 1.10.5
Created attachment 363152 [details] git_master_hls_playback_leak_log And reproducible with git master.
yes, I confirm memory leak in git master. I will provide a patch here and please verify with it
Created attachment 363272 [details] [review] fix hls playback memory leak
rolandz, please verify with the patch.
This patch work well, thanks. Valgrind and gstleak tracer are quiet now,no noise while hls playback. In addition,Maybe we still need do more works, There are several leaks of gstcaps and gstbuffer in dashdemux. (Obviously, this is not associated with hlsdemux's patch, which shouldn't block the previous patch --) ) Run this command,then CTRL+c: GST_TRACERS="leaks" GST_DEBUG="GST_TRACER:7" gst-launch-1.0 souphttps rc location=http://www-itec.uni-klu.ac.at/ftp/datasets/mmsys12/RedBullPlayStreets/redbull_4s/RedBullPlayStreets_4s_isoffma in_DIS_23009_1_v_2_1c2_2011_08_30.mpd ! dashdemux ! decodebin ! videoconvert ! videoscale ! autovideosink Setting pipeline to PAUSED ... Pipeline is PREROLLING ... Redistribute latency... Redistribute latency... Redistribute latency... Redistribute latency... Redistribute latency... ^Chandling interrupt. Interrupt: Stopping pipeline ... ERROR: pipeline doesn't want to preroll. Setting pipeline to NULL ... Freeing pipeline ... 0:00:21.510940728 13757 0x1db86f0 TRACE GST_TRACER :0:: object-alive, type-name=(string)GstBuffer, addres s=(gpointer)0x7fcbec5070c0, description=(string)buffer: 0x7fcbec5070c0, pts 99:99:99.999999999, dts 99:99:99.999999999, du r 99:99:99.999999999, size 38, offset none, offset_end none, flags 0x0, ref-count=(uint)1, trace=(string); 0:00:21.510957241 13757 0x1db86f0 TRACE GST_TRACER :0:: object-alive, type-name=(string)GstBuffer, addres s=(gpointer)0x7fcbec1ab5a0, description=(string)buffer: 0x7fcbec1ab5a0, pts 99:99:99.999999999, dts 99:99:99.999999999, du r 99:99:99.999999999, size 39, offset none, offset_end none, flags 0x0, ref-count=(uint)1, trace=(string); 0:00:21.510968411 13757 0x1db86f0 TRACE GST_TRACER :0:: object-alive, type-name=(string)GstCaps, address=(gpointer)0x7fcbd4050050, description=(string)video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string )1.3, profile=(string)constrained-baseline, codec_data=(buffer)0142c00dffe100166742c00dd98141fb0e1000003e90000ea608f142a68001000568c960f2c8, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)119/4, ref-count =(uint)1, trace=(string); 0:00:21.510978648 13757 0x1db86f0 TRACE GST_TRACER :0:: object-alive, type-name=(string)GstCaps, address=(gpointer)0x7fcbd409e540, description=(string)video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string )3, profile=(string)constrained-baseline, codec_data=(buffer)0142c01effe100176742c01ed981e0bfe5c38400000fa40003a9823c58b9a 001000568c960f2c8, width=(int)480, height=(int)360, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)119/4, ref-count=(uint)1, trace=(string); 0:00:21.510987753 13757 0x1db86f0 TRACE GST_TRACER :0:: object-alive, type-name=(string)GstMemory, addres s=(gpointer)0x7fcbe000d300, description=(string)0x7fcbe000d300, ref-count=(uint)1, trace=(string); 0:00:21.510995369 13757 0x1db86f0 TRACE GST_TRACER :0:: object-alive, type-name=(string)GstMemory, addres s=(gpointer)0x7fcbe000d180, description=(string)0x7fcbe000d180, ref-count=(uint)1, trace=(string); ** (gst-launch-1.0:13757): WARNING **: Leaks detected
Ooops,"Mid-air collision detected" --). So, it's better to create a new ticket for dashdemux: https://bugzilla.gnome.org/show_bug.cgi?id=790110
thanks for your confirm, I think this hls patch will be merged after reviewed. regarding dashdemux issue, let's discuss in #790110
Could anyone please review Xie's patch? --).
commit 0ef2a41d84dc12abea7c67c695c6298e60af383b (HEAD -> master) Author: Jun Xie <jun.xie@samsung.com> Date: Thu Nov 9 12:36:44 2017 +0800 hlsdemux: Fix memory leak caused by double buffer map https://bugzilla.gnome.org/show_bug.cgi?id=789999
I also moved the GstMapInfo inside the if block. Thanks for the patch