Does it work if you play the file locally instead over http? Does it always happen for these large files?

Which versions of GStreamer and the plugin modules are you using?

Any chance you could make such a file available somewhere so we can try to reproduce it?

(In reply to comment #1)
> Does it work if you play the file locally instead over http? Does it always
> happen for these large files?
> Which versions of GStreamer and the plugin modules are you using?
The same file playing locally does not have the problem. I also convert the file to mp3 format and ogg format, both can be played without problem, but the file size in mp3 and ogg is reduced to 1/3 of the original size. 
The version are:
  gstreamer-0.10.35
  gst-plugins-ugly-0.10.18
  gst-plugins-good-0.10.30
  gst-plugins-base-0.10.35
  gst-plugins-bad-0.10.22
  gst-ffmpeg-0.10.13
running on a ARM9 platform with 64M bytes RAM.

(In reply to comment #2)
> Any chance you could make such a file available somewhere so we can try to
> reproduce it?
Due to the copyright issue, I cannot post that song. But I am trying to make a source to create a large flac file and put in a http server. If ok, I will post it. Let you know later.

Created attachment 203442 [details]
oom-killer message during playing flac stream from http server

I have create a flac file with size 582M bytes, duration 2:28:30. I use gst-launch playbin2 to play it from a http server. Since the file and the environment (free memory size) is changed, it trigger the oom-killer at around 1:30. Before I start the playbin2, the free memory is 41M bytes. The flac stream is available at "http://www.aldtech.com/largefile.flac"

Attached is the console message from the oom-killer.

Running playbin2 on the "http://www.aldtech.com/largefile.flac" URL with 0.10 git on x86_64, it all works fine, starting off at 0.3% of memory, and keeping at 0.4% of memory after more than two minutes playback. This is close enough to 0.3% to say that there's no runaway leaks here.

Maybe there's an ARM specific plugin that gets used here, and leaks, or you're just dangerously close to OOM from the start and you get caught by some small fluctuation. Or maybe there was a leak that got fixed since.

Can you try with a more recent version ?

Created attachment 216247 [details]
Log of the console and "top" message

You cannot duplicate the problem by just playing 2 minutes. The time when the problem happened depends on the file size and the system memory available. Attached is the log of the "top" command used to monitor the system free memory, and the log from console running the gst-launch script. The script run "cat /proc/meminfo", "date", "gst-launch ...", "date", then "cat /proc/meminfo". 
I think the problem is related to the flac decoder that build the frame index table in the system memory, it gets large and large during playing and finally it gets beyond the free memory limit and trigger the OOM.

Under my system's config (64Mbyte physical memory, gst-launch starts with 49MByte free memory), the problem happens after around 1 hour and 56 minutes playing. If the system has lower free memory, it happens earlier.

Ah, with the new file, you said "it trigger the oom-killer at around 1:30.", was that a typo then ? Or does that mean 1 hour 30 minutes ? Or am I misunderstanding something here ?

It means 1 hour 30 minutes, sorry for misleading you. The time is mainly depends on how the free memory available. If you have more than 128Mbyte memory, may be it will not happen for this "largefile.flac" file.

Running gst-launch in the background with a generated flac file, I see gradual RSS increase of 800 KB in about 50 min playback (though with a virtual size of half a GB, which can't be right...).

Moving libgstaudioparsers away (which includes flacparse), I see no increase in RSS after about 45 min playback.

Indeed, when in use, the parser base class adds 10 entries per second to the index, so your suspiscion seems correct.

Actually, only 1 a second, there's some extra code going on to not add entries too close to each other in the index addition code.

Still, it's open ended. Maybe we should make a "closeness" threshold that depends on the duration of the stream, so the index would grow to a maximum known size, but that still leaves live streams with no known duration.

Another possibility would be to winnow the index from time to time when it grows too much, but that doesn't seem like a nice solution.

I think we had the same issue with mp3 streams. You could check what we are doing in mpegaudioparse (or used to do in the old mp3parse in -ugly).

For live streams: I think if upstream can't provide a duration (in the format of the upstream newsegment event) or is not seekable as per seeking query, then it's reasonable to assume we're live or pseudo-live and don't need to index at all.

I think reducing the granularity of the index whenever we hit N index items also seems fair for reasonably large N.

The -ugly mpegaudioparse seems to do a rough "large intervals for large streams" with three steps (though a least one entry a second too). The newer one uses baseparse as flacparse.

I guess one could add a 'max-index-entries' property to baseparse with a largeish default, and deduce the time threshold from that. 'max-index-size' would seem better, but the index is possibly a list or hash table, looking at the code, so probably not that useful.

Created attachment 216266 [details] [review]
baseparse: prevent excessively high memory usage with large streams

Large streams would index one frame every second, which can get quite
large with multi-hour streams.

The above patch ensures memory usage from the index is bounded.
Memory will still increase slowly with playback though, but with longer streams, increase will be slower.
The current default is pulled out of a hat, it causes a 3 hours 30 minutes file to add an entry every 3 seconds, which may be still too much (likely to be for the original poster).
If the patch is deemed a good way to fix the issue, then I can add an API to set the entries limit, and a baseparse property for it.

Looks like this was never pushed? Moving to core and changing bug title so we can check if it's still an issue with 1.0

*** Bug 680619 has been marked as a duplicate of this bug. ***

Pushed this with minor changes (no need to use gst_scale_* to determine the min byte distance; some comments; use LOG level for 'already have entries up to offset N').

 commit 37a72b1a82143f47dc78933e209a3cb0e1f358d0
 Author: Vincent Penquerc'h <vincent.penquerch@collabora.co.uk>
 Date:   Wed Jun 13 13:02:48 2012 +0100

    baseparse: prevent excessively high memory usage with long streams
    
    Large streams would index one frame every second, which can get quite
    large with multi-hour streams, so add an additional byte-based
    minimum distance as well, which will kick in for long streams
    and make sure we never have more than a couple of thousand index
    entries.
    
    https://bugzilla.gnome.org/show_bug.cgi?id=666053

Can you tell me how to fix this problem for version 0.10.x? I have to use this version for the old product.

Colman

I believe the patch was created originally against 0.10 and should apply to 0.10 as well.

I apply the patch, but the result is the same. The problem still presents even I change the MAX_INDEX_ENTRIES to 512. Whatever the MAX_INDEX_ENTRIES = 4096 or = 512, it eats up all the memory at around 1 hour and 30 minutes on my 64Mbyte RAM platform.

May be there are another index table built for flac stream.

Colman

I have done a valgrind --tool=massif run with valgrind and a --leak-check=yes run with valgrind, and current git master (~1.0.2), and I'm not seeing any constant or major increases even after 2 hours (but I have tested the entire file to the end as well, over http).

There are a couple of reads of uninitialised memory though, which should be looked into, but I don't think those can cause any leaks.

Please re-test with git master. I don't think anyone is going to investigate any issues with 0.10 though, sorry.