[Numpy-discussion] Bug in pickling an ndarray?

[Robert Kern]

On Sat, Jun 30, 2012 at 11:33 PM, Daniel Hyams <dhyams at gmail.com> wrote:
> Hmmm, I wouldn't think that it is correct behavior; I would think that *any*
> ndarray arising from pickling would have its .base attribute set to None.
>  If not, then who is really the one that owns the data?
>
> It was my understanding that .base should hold a reference to another
> ndarray that the data is really coming from, or it's None.  It certainly
> shouldn't be some random string, should it?

It can be any object that will keep the data memory alive while the
object is kept alive. It does not have to be an ndarray. In this case,
the numpy unpickling constructor takes the string object that the
underlying pickling machinery has just created and views its memory
directly. In order to keep Python from freeing that memory, the string
object needs to be kept alive via a reference, so it gets assigned to
the .base.

> And yes, it is causing a problem for me, which is why I noticed it.  In my
> application, ndarrays can come from various sources, pickling being one of
> them.  Later in the app, I was wanting to resize the array, which you cannot
> do if the data is not really owned by that array...

You also can't resize an array if any *other* array has a view on that
array too, so checking for ownership isn't going to help. .resize()
will raise an exception if it can't do this; it's better to just
attempt it and catch the exception than to look before you leap.

> I had explicit check for
> myarray.base==None, which it is not when I get the ndarray from a pickle.

That is not the way to check if an ndarray owns its data. Instead,
check a.flags['OWNDATA']

-- 
Robert Kern