Python memory handling

[Nick Craig-Wood]

Andrew MacIntyre <andymac at bullseye.apana.org.au> wrote:
>  You should also appreciate something about PyMalloc: it only handles 
>  allocation requests of 256 bytes or smaller, and this limitation is part
>  of PyMalloc's design.
> 
>  If most of your allocations are >256 bytes, you're at the mercy of the 
>  platform malloc

You can tweak this if you are using libc (eg under linux) at least :-

  http://www.gnu.org/software/libc/manual/html_node/Malloc-Tunable-Parameters.html

Setting M_MMAP_THRESHOLD should result in blocks that are perfectly
free()able back to the OS if allocated with malloc(). By default this
is 128k I think so you can set it to 4k and it should help a lot.

Note that a mmap block is a minimum of 4k (under x86 - one OS page
anyway) so set this too small and you program will use a *lot* of
memory, but only temporarily ;-)

If PyMalloc stretched up to 4k and M_MMAP_THRESHOLD was set to 4k then
you'd have the perfect memory allocator...

-- 
Nick Craig-Wood <nick at craig-wood.com> -- http://www.craig-wood.com/nick