mmap caching

[George Sakkis]

Martin v. Löwis wrote:

> George Sakkis schrieb:
> > I've been trying to track down a memory leak (which I initially
> > attributed erroneously to numpy) and it turns out to be caused by a
> > memory mapped file. It seems that mmap caches without limit the chunks
> > it reads, as the memory usage grows to several hundreds MBs according
> > to the Windows task manager before it dies with a MemoryError.
>
> You must be misinterpreting what you are seeing. It's the operating
> system that decides what part of a memory-mapped file are held in
> memory, and that is certainly not without limits.

Sure; what I meant was that that whatever the limit is, it's high
enough that a MemoryError is raised before the limit is reached.

> Notice that there are several values that can be called "memory
> usage" (such as the size of the committed address space, the working
> set size, etc); you don't mention which of these values grows several
> hundreds MB.

It's the one in the 'Processes' tab of the Windows task manager (XP
proffesional). By the way, I ran the same program on a box with more
physical memory and the mem. usage stops growing at around 430MB, by
which time the whole file is most likely cached. I'd be interested in
any suggestions other than "buy more RAM" :) (these are not my machines
anyway).

Thanks,
George