[Numpy-discussion] .transpose() of memmap array fails to close()

[Glen W. Mabey]

[I keep posting hoping that someone knowledgeable in these things will
take notice ...]

Just a couple of more notes regarding this numpy.memmap issue.

It seems that any slice of a numpy.memmap that is greater than 1-d has 
a similar problem.



In [1]:import numpy

In [2]:amemmap = numpy.memmap( '/tmp/afile', dtype=numpy.float32,
shape=(4,5), mode='w+' )

In [3]:amemmap[1,3:4]
Out[3]:memmap([ 0.], dtype=float32)

In [4]:amemmap[0:1,3:4]
Exception exceptions.AttributeError: "'memmap' object has no attribute '_mmap'" in <bound method memmap.__del__ of memmap([ 0.], dtype=float32)> ignored
Out[4]:memmap([[ 0.]], dtype=float32)



A very naive hack-fix of overloading the __getitem__ method of the
numpy.memmap class such that the result of ndarray.__getitem__ gets the
._mmap attribute added didn't work ...

I tried to follow the program flow into the bowels of
multiarraymodule.c, but that was beyond me ...

This problem started showing up when I changed to python 2.5 and
persists in 2.5.1.  I've considered switching back to 2.4 but I really
need 64-bit array indexing ...

Best Regards,
Glen Mabey


On Fri, Aug 10, 2007 at 11:20:16AM -0500, Glen W. Mabey wrote:
> Hello,
> 
> I posted this a while back and didn't get any replies.  I'm running in
> to this issue again from a different aspect, and today I've been trying
> to figure out which method of ndarray needs to be overloaded for memmap
> so that the the ._mmap attribute gets handled appropriately.
> 
> But, I have not been able to figure out what methods of ndarray are
> getting used in code such as this:
> 
> >>> import numpy
> >>> amemmap = numpy.memmap( '/tmp/afile', dtype=numpy.float32,
> >>> shape=(4,5), mode='w+' )
> >>> b = amemmap[2:3]
> >>> b
> >>> Exception exceptions.AttributeError: "'memmap' object has no attribute '_mmap'" in <bound method memmap.__del__ of memmap([ 0.,  0.,  0.,  0.,  0.], dtype=float32)> ignored memmap([[ 0.,  0.,  0.,  0.,  0.]], dtype=float32)
> 
> 
> Furthermore, can anyone enlighten me as to why an AttributeError
> exception would be ignored?
> 
> Am I using numpy.memmap instances appropriately?
> 
> Thank you,
> Glen Mabey
> 
> 
> 
> 
> On Thu, Jun 07, 2007 at 04:46:20PM -0500, Glen W. Mabey wrote:
> > Hello,
> > 
> > When assigning a variable that is the transpose() of a memmap array, the
> > ._mmap member doesn't get copied, I guess:
> > 
> > In [1]:import numpy
> > 
> > In [2]:amemmap = numpy.memmap( '/tmp/afile', dtype=numpy.float32, shape=(4,5), mode='w+' )
> > 
> > In [3]:bmemmap = amemmap.transpose()
> > 
> > In [4]:bmemmap.close()
> > ---------------------------------------------------------------------------
> > <type 'exceptions.AttributeError'>        Traceback (most recent call last)
> > 
> > /home/gmabey/src/R9619_dev_acqlibweb/Projects/R9619_NChannelDetection/NED/<ipython console> in <module>()
> > 
> > /usr/local/stow/numpy-20070605_svn-py2.5/lib/python2.5/site-packages/numpy/core/memmap.py
> > in close(self)
> >      86 
> >      87     def close(self):
> > ---> 88         self._mmap.close()
> >      89 
> >      90     def __del__(self):
> > 
> > <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'close'
> > > /usr/local/stow/numpy-20070605_svn-py2.5/lib/python2.5/site-packages/numpy/core/memmap.py(88)close()
> >      87     def close(self):
> > ---> 88         self._mmap.close()
> >      89 
> > 
> > 
> > 
> > 
> > This is an issue when the data is accessed in an order that is different
> > from how it is stored on disk, as:
> > 
> > bmemmap = numpy.memmap( '/tmp/afile', dtype=numpy.float32, shape=(4,5), mode='w+' ).transpose()
> > 
> > So the object that was originally produced not accessible.  I imagine
> > there is some better way to indicate order of dimensions, but
> > regardless, doing
> > 
> > In [4]:bmemmap._mmap = amemmap._mmap
> > 
> > is a hack workaround.
> > 
> > Best regards,
> > Glen Mabey
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion at scipy.org
> > http://projects.scipy.org/mailman/listinfo/numpy-discussion
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion