[Numpy-discussion] reading large files

[Mike Ressler]

On 9/22/06, Bill Baxter <wbaxter at gmail.com> wrote:
>
> Do you also have a 64-bit processor?  Just checking since you didn't
> mention it.
> --bb
>
> On 9/22/06, Svein-Erik.Hamran at ffi.no <Svein-Erik.Hamran at ffi.no> wrote:
> > I would like to read files >2Gbyte. From earlier posting I believed it
> > should be possible with python 2.5.
> >
> > I am getting MemoryError when trying to read a file larger than 2Gb.
> >
> > I am using python2.5 and numpy1.0rc1.
>

Also, how much memory do you have? You will have to use memory mapping as
well if the file size is greater than your memory size.

But you should be hopeful. I have been working with > 10 GB files for some
time using beta versions of Python 2.5 and numpy-1.0 on a 64-bit AMD machine
with 4 GB of memory.

Mike

-- 
mike.ressler at alum.mit.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20060922/aa6590da/attachment-0001.html>