After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 134839 - buffers leaked by filesrc, typefind, spider or fakesink
buffers leaked by filesrc, typefind, spider or fakesink
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gstreamer (core)
git master
Other Linux
: Normal normal
: 0.8.2
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2004-02-19 14:26 UTC by Tim-Philipp Müller
Modified: 2004-12-22 21:47 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
program to demonstrate the leak (7.68 KB, application/octet-stream)
2004-02-19 14:27 UTC, Tim-Philipp Müller
  Details
proposed patch: free structure->fields GArray in gst_structure_free() (506 bytes, patch)
2004-04-02 06:31 UTC, Tim-Philipp Müller
none Details | Review

Description Tim-Philipp Müller 2004-02-19 14:26:56 UTC
Hi, 
 
The following pipeline: 
 
   filesrc ! typefind ! spider ! fakesink 
 
seems to leak buffers. 
 
Check out the attached program and watch memory usage in top while 
running it over a folder that contains loads of mp3s and other files, 
like this: 
 
  ./scimport /mnt/media/mp3s 
 
 
The leak seems to occur mainly, if not exclusively, with mp3 files that 
have an ID3 tag, at least it doesn't seem to occur with ogg files or with 
mpeg/audio files that have no ID3 tag. Possibly it is related to typefind 
seeking to the end of the file when checking for an ID3v1 tag. 
 
Cheers 
-Tim
Comment 1 Tim-Philipp Müller 2004-02-19 14:27:45 UTC
Created attachment 24557 [details]
program to demonstrate the leak
Comment 2 Tim-Philipp Müller 2004-02-19 14:36:43 UTC
More details: 
 
No leak: 
  * mpeg/audio files without any ID3 tags 
  * mpeg/audio files with only an ID3v1 tag at the end 
 
Leak: 
  * mpeg/audio files with only an ID3v2.3.0 tag at the beginning of 
the file  
 
Cheers 
-Tim 
 
 
Comment 3 Christophe Fergeau 2004-02-29 10:48:13 UTC
rhythmbox consumes huge amount of memory (mine went up to 600
megabytes) when using gstreamer to read metadatas, this is probably
related to that bug.
Comment 4 David Schleef 2004-03-09 20:28:17 UTC
I think this was fixed:

2004-03-06  Benjamin Otte  <otte@gnome.org>

        * ext/mad/gstid3tag.c: (gst_id3_tag_chain):
          fix huge leak: gst_buffer_merge doesn't unref the first argument
          itself.
Comment 5 Tim-Philipp Müller 2004-03-09 20:38:07 UTC
I don't think this is fully fixed yet. 
 
As far as I can see, it leaks less than before, but still leaks 
massively, with CVS HEAD from 2004-March-09. 
 
 
 
Comment 6 Tim-Philipp Müller 2004-03-10 15:50:18 UTC
Some more information: 
 
I think the 'mad' element that is created when running the  
 
        filesrc ! typefind ! spider ! fakesink 
 
pipeline on an mp3 file is never disposed of when the pipeline gets 
destroyed. 
 
Here's what I did to come to that conclusion: 
 
static gpointer maddata;  /* NULL */ 
static gpointer pipedata; /* NULL */ 
 
static void 
ea_cb (GstBin *spider, GstElement *element, gpointer foo) 
{ 
  g_print("ELEMENT ADDED: %s\n", gst_element_get_name(element)); 
  if (strncmp(gst_element_get_name(element), "mad", 3) == 0) 
  { 
    g_print ("hooking up nullifier...\n"); 
    maddata = spider; 
    g_object_set_data_full(G_OBJECT(element), "madfoo", &maddata, 
(GDestroyNotify)g_nullify_pointer); 
  } 
} 
 
..... 
 
g_signal_connect(spider, "element-added", ea_cb, NULL); 
 
gobject_set_data_full(G_OBJECT(pipeline), "pipefoo", &pipedata, 
(GDestroyNotify)g_nullify_pointer); 
 
...... 
 
after the pipeline is unrefed, pipedata will be NULL, but maddata 
will still be some non-NULL value, so the mad object didn't get 
destroyed. 
 
I am not sure if that would cause huge leaks like that, but it's 
worth a shot. 
 
Hope this helps whoever is debugging this further :) 
 
Cheers 
-Tim 
 
Comment 7 David Schleef 2004-03-10 20:01:20 UTC
The pipeline

  filesrc ! typefind ! spider ! fakesink

doesn't create a mad element.
Comment 8 Tim-Philipp Müller 2004-03-10 20:04:53 UTC
I meant: if you feed it an mp3 file as input... 
 
 
Comment 9 David Schleef 2004-03-10 21:30:36 UTC
same thing.  You have to use a pipeline like the following to get a
mad element:

  filesrc ! typefind ! spider ! audio/x-raw-int ! fakesink

Comment 10 Tim-Philipp Müller 2004-03-10 21:38:08 UTC
gah - yes, of course. It wasn't meant as an exact gst-launch 
pipeline but refered to the above attached test program, which in 
fact does exactly that. And _obviously_ it must have created a a mad 
element somewhere, because otherwise my callback wouldn't have 
worked, would it? 
 
Anyway, this discussion doesn't add any useful info at all, so maybe 
we can stop it. 
 
Comment 11 Tim-Philipp Müller 2004-04-02 06:28:50 UTC
Maybe this has something to do with it? (This is after the pipeline has been 
unref'ed) 
 
Patch attached (doesn't fix the problem though, just a small part of it). 
 
 2380428 bytes in 16219 blocks are still reachable in loss record 23 of 23 
    at 0x3C01FD01: realloc (vg_replace_malloc.c:162) 
    by 0x3C31C870: g_realloc (gmem.c:169) 
    by 0x3C2FF4FA: g_array_maybe_expand (garray.c:350) 
    by 0x3C2FED8B: g_array_append_vals (garray.c:138) 
    by 0x3C07F228: gst_structure_set_field (gststructure.c:450) 
    by 0x3C07ED66: gst_structure_id_set_value (gststructure.c:288) 
    by 0x3C054F45: gst_caps_structure_intersect_field (gstcaps.c:713) 
    by 0x3C07F9C9: gst_structure_foreach (gststructure.c:722) 
    by 0x3C054FBF: gst_caps_structure_intersect (gstcaps.c:741) 
    by 0x3C05517D: gst_caps_intersect (gstcaps.c:840) 
    by 0x3C90A028: gst_autoplug_caps_intersect (gstsearchfuncs.c:64) 
    by 0x3C90A13B: gst_autoplug_can_connect_sink (gstsearchfuncs.c:132) 
    by 0x3C90A741: gst_autoplug_sp (gstsearchfuncs.c:349) 
    by 0x3C9080F0: gst_spider_plug_from_srcpad (gstspider.c:690) 
    by 0x3C907E79: gst_spider_plug (gstspider.c:656) 
    by 0x3C907637: gst_spider_identity_plug (gstspider.c:499) 
    by 0x3C909D59: gst_spider_identity_sink_loop_type_finding 
(gstspideridentity.c:569) 
    by 0x3C8DAAAA: loop_group_schedule_function (gstoptimalscheduler.c:1166) 
    by 0x3C8DA4A1: schedule_group (gstoptimalscheduler.c:1009) 
    by 0x3C8DA565: gst_opt_scheduler_schedule_run_queue 
(gstoptimalscheduler.c:1052) 
 
 
Cheers 
 -Tim 
 
Comment 12 Tim-Philipp Müller 2004-04-02 06:31:22 UTC
Created attachment 26234 [details] [review]
proposed patch: free structure->fields GArray in gst_structure_free()
Comment 13 Christophe Fergeau 2004-04-02 08:45:21 UTC
I tested this patch with Rhythmbox library import, and it reduces memory leaks
by 25% ;)
Without the patch, after importing 1500 files (using gstreamer to read
metadata), rhythmbox uses 102MB. With the patch, it "only" eats 78MB
Comment 14 Benjamin Otte (Company) 2004-04-02 23:15:28 UTC
patch applied, leaving open as it isn't fixed completely.
Comment 15 Benjamin Otte (Company) 2004-04-26 22:51:01 UTC
I believe this is fixed after the recent memleak fixage.