[Numpy-discussion] MemoryError in numpy.test() on AIXinstallation

[Charles R Harris]

On Mon, Apr 27, 2009 at 2:10 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

>
>
> On Mon, Apr 27, 2009 at 1:59 PM, Whitcomb, Mr. Tim <
> tim.whitcomb at nrlmry.navy.mil> wrote:
>
>> >> ----------------------------------
>> >>              import numpy as np
>> >>              from cStringIO import StringIO
>> >>              from numpy.lib import format
>> >>
>> >>              f = StringIO()
>> >>              a = np.array([], dtype='object')
>> >>              format.write_array(f, a)
>> >>              f2 = StringIO(f.getvalue())
>> >>              a2 = format.read_array(f2)
>> >>
>> >>              print('%s %s' % (a, a2))
>> >>              -----------------------------------
>>
>> >>              On the AIX/PPC install, this crashes with:
>> >>              Traceback (most recent call last):
>> >>               File "<stdin>", line 1, in <module>
>> >>               File "crash_numpy.py", line 10, in <module>
>> >>                 a2  = format.read_array(f2)
>> >>               File
>> >>
>> "$HOME/.local/python/lib/python2.5/site-packages/numpy/lib/format.py",
>> >>              line 362, in read_array
>> >>                 array = cPickle.load(fp)
>> >>              MemoryError
>>
>> >       I think this means out of memory, which is probably not correct.
>> What happens if you simply pickle and object array? Something like
>> >
>> >       If that fails, try it using pickle instead of cPickle, then with
>> a non-empty object array like a = np.array([None,None], dtype='object')
>>
>> Just trying to pickle an empty object array:
>> -------
>> pickle_file = 'foo.pkl'
>> a = np.array([], dtype='object')
>> cPickle.dump(a, open(pickle_file, 'w'))
>> b = cPickle.load(open(pickle_file, 'r'))
>> print(b)
>> -------
>> Traceback (most recent call last):
>>   File "c_harris_test.py", line 7, in <module>
>>    b = cPickle.load(open(pickle_file, 'r'))
>> MemoryError
>>
>> Changing 'cPickle' to 'pickle' crashes again, with:
>> Traceback (most recent call last):
>>   File "c_harris_test.py", line 7, in <module>
>>    b = pickle.load(open(pickle_file, 'r'))
>>  File "/site/python-2.5.2/lib/python2.5/pickle.py", line 1370, in load
>>    return Unpickler(file).load()
>>  File "/site/python-2.5.2/lib/python2.5/pickle.py", line 858, in load
>>    dispatch[key](self)
>>  File "/site/python-2.5.2/lib/python2.5/pickle.py", line 1217, in
>> load_build
>>    setstate(state)
>> MemoryError
>>
>> If I try either pickle or cPickle with a non-empty object (e.g.
>> np.array([None])), it works fine.  Using np.array([ [] ]) crashes as
>> well.  If I take Numpy out of the loop, I can successfully pickle an
>> empty list with no issues.  Would platform endian-ness cause a problem?
>>
>
> It could, but I think AIX on the PPC is big endian, as is OSX, and I would
> expect endian problems to show up in many other places, not just for empty
> arrays. Hmm...
>

What compiler are you using? Any optimization flags?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20090427/e5e0817c/attachment.html>