[Numpy-discussion] Should ndarray be a context manager?

[Robert Kern]

On Tue, Dec 9, 2014 at 5:57 PM, Julian Taylor <jtaylor.debian at googlemail.com>
wrote:
>
> On 09.12.2014 18:55, Sturla Molden wrote:
> > On 09/12/14 18:39, Julian Taylor wrote:
> >
> >> A context manager will also not help you with reference cycles.
> >
> > If will because __exit__ is always executed. Even if the PyArrayObject
> > struct lingers, the data buffer will be released.
>
> a exit function would not delete the buffer, only decrease the reference
> count of the array. If something else still holds a reference it stays
> valid.
> Otherwise you would end up with a crash when the other object holding a
> reference tries to access it.

I believe that Sturla is proposing that the buffer (the data pointer) will
indeed be free()ed and the ndarray object be modified in-place to have an
empty shape. Most references won't matter, because they are opaque; e.g. a
frame being held by a caught traceback somewhere or whatever. These are
frequently the references that keep alive large arrays when we don't them
to, and are hard to track down. The only place where you will get a crasher
is when other ndarray views on the original array are still around because
those are not opaque references.

The main problem I have is that this is much too likely to cause a segfault
to be part of the main API for ndarrays. I perhaps wouldn't mind a
non-public-API function hidden in numpy somewhere (but not in numpy.* or
even numpy.*.*) that did this. The user would put it into a finally:
clause, if that such things matters to them, instead of using it as a
context manager. Like as_strided(), which creates similar potential for
crashes, it should be not be casually available. This kind of action needs
to be an explicit statement
yes_i_dont_need_this_memory_anymore_references_be_damned() rather than
implicitly hidden behind generic syntax.

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20141209/d3724683/attachment.html>