[Numpy-discussion] Should ndarray be a context manager?

[Sturla Molden]

Chris Barker <chris.barker at noaa.gov> wrote:

> my  first thought iust that you can just do:
> 
> x = np.zeros(n)
> [... your code here ]
> del x
> 
> x's ref count will go down, and it will be deleted  if there are no other
> references to it. 

1. This depends on reference counting. PyPy supports numpy too (albeit with
its own code) and does not reference count.

2. del does not delete, it just decrements the refcount. x can still be
kept alive,

3. If x is a part of a reference cycle it is reclaimed later on.


> If there Are other references to it, you really wouldn't
> want to delete the memory buffer anyway, would you?

Same thing for file descriptors. For example consider what happens if you
memory map a file, then close the file, but continue to read and write to
the mapped address. NumPy allows us to construct these circumstances if we
want to.

 
> I suppose you could write a generic context manger that would do the del
> for you, but I'm not sure what the point would be.

A del is very different from a deallocation that actually disposes of the
data buffer, regardless of references to the memory that might still be
alive.


> I guess this comes down to -- why would anyone want/need a numpy array
> object with no underlying data?

I don't. The PyArrayObject struct is so small that I don't care about it.
But it could reference a huge data buffer, and I might want to get rid of
that more deterministically than just waiting for the gc.


> (although I'm still confused as to why it's so important (in cPython) to
> have a file context manager..)

Because we often want to run setup and teardown code deterministically,
rather than e.g. having it happen at random from the gc thread when it runs
the finalizer. If Python raises an exception, a io.file object can be kept
alive by the traceback for decades. If Python raises an exception, a an
acquire/release pair for a threading.Lock can be separated, and the lock
ends up in an undefined state further down in your code.

In what I suggested the setup and teardown code would be malloc() and
free().


Sturla