[Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes

[Sturla Molden]

Julian Taylor <jtaylor.debian at googlemail.com> wrote:
 
> The problem with this approach is that it is already difficult enough to
> handle memory in numpy.

I would not do this in a way that complicates memory management in NumPy. I
would just replace malloc and free with temporarily cached versions. From
the perspective of NumPy the API should be the same.

> Having a cache that potentially stores gigabytes
> of memory out of the users sight will just make things worse.

Buffer don't need to stay in cache forver, just long enough to allow resue
within an expression. We are probably talking about delaying the call to
free with just a few microseconds.

We could e.g. have a setup like this:

NumPy thread on "malloc":
- tries to grab memory off the internal heap
- calls system malloc on failure

NumPy thread on "free":
- returns a buffer to the internal heap
- signals a condition

Background daemonic GC thread:
- wakes after sleeping on the condition
- sleeps for another N microseconds (N = magic number)
- flushes or shrinks the internal heap with system free
- goes back to sleeping on the condition 
 
It can be implemented with the same API as malloc and free, and plugged
directly into the existing NumPy code. 

We would in total need two mutexes, one condition variable, a pthread, and
a heap.

Sturla