[Matrix-SIG] Limited size arrays with cPickle

[Herbert L. Roitblat]

Yes, I did.  I have made that mistake in the past.  It does work with
smaller arrays, but this is the biggest one I have tried to do.

Thank you for asking.
-----Original Message-----
From: David Ascher <da@ski.org>
To: Herbert L. Roitblat <roitblat@hawaii.edu>; matrix-SIG@python.org
<matrix-SIG@python.org>
Date: Saturday, January 01, 2000 5:50 PM
Subject: Re: [Matrix-SIG] Limited size arrays with cPickle


>Did you make sure to use "binary" pickle to a file opened in 'wb' mode?
>
>--david
>
>----- Original Message -----
>From: Herbert L. Roitblat <roitblat@hawaii.edu>
>To: <matrix-SIG@python.org>
>Sent: Saturday, January 01, 2000 4:41 PM
>Subject: [Matrix-SIG] Limited size arrays with cPickle
>
>
>> Help Please:
>>
>> I have run into this problem before and I have a workaround for it now,
>but
>> it seems like something that would be good to fix.
>> I have a very large array (19741 x 53566) stored as 8-bit unsigned.  I
>want
>> to cPickle it into a file and then cPickle it back when I want to use it.
>I
>> have a 2 GB machine, so memory is not immediately the problem, rather it
>> seems to be the routine that translates the array into a string for
>cPickle.
>> It reports a memory error.  I suppose that it could take up that much
more
>> memory to translate this array into a string, but it seems that there
>ought
>> to be a more efficient way to store arrays that would not take any more
>> memory than the amount taken by the array.  The actual routine that does
>the
>> translation is in multiarray.pyd, so I cannot get to it with my limited
>> skills.
>>
>> I'm running NT on a Pentium II 450MHz machine.  My Numeric is dated July
>13,
>> 1999, but I think that this is the date I downloaded it.  I believe that
>it
>> is the latest release of Numeric.  The pdf manual is copyright 1999.
>>
>> Thank you.
>> Herbert L. Roitblat, Ph.D.
>> Department of Psychology
>> University of Hawaii
>> 2430 Campus Road
>> Honolulu, HI 96822
>>
>>
>> _______________________________________________
>> Matrix-SIG maillist  -  Matrix-SIG@python.org
>> http://www.python.org/mailman/listinfo/matrix-sig
>>
>
>
>_______________________________________________
>Matrix-SIG maillist  -  Matrix-SIG@python.org
>http://www.python.org/mailman/listinfo/matrix-sig
>