Python and Memory

[Fernando Perez]

Ben Fairbank wrote:

> I have to pick a language to commit to for general purpose
> scientific/statistical/utility/database programming at my office and
> have pretty much narrowed it down to R or Python.  Problem:  none of
> the various Python books I have looked into  has had much to say about
> memory.  I will be operating on some big arrays, probably with Numpy;
> if I run out of space and upgrade a Win 2000 or Win XP pro machine
> from 256 Meg to 500Meg or even 1G will Python automatically recognize
> and take advantage of the increase?  Where are questionss such as this
> discussed in the documentation?
> 
> Thanks for any suggestions,

For large arrays, look into numarray (next-gen Numeric).  That's where most of
the effort of numarray has gone: being a _lot_ smarter than Numeric in memory
management for large arrays.  It's also cleaner and nicer than numeric in many
ways, and already mature enough that many people are using it for production
work.  Its only big stumbling block right now is that small array performance
is pretty poor, and this one will take a bit of time to fix.

I'd personally stick to python (with the scipy libraries, as was already
suggested), and would add the RPy interface for those cases when you need a
routine which R provides, which you need NOW, and which would be more work to
write in python.

Cheers,

f.