[Numpy-discussion] multiprocessing shared arrays and numpy

[Gael Varoquaux]

On Sun, Mar 07, 2010 at 07:00:03PM +0000, René Dudfield wrote:
> 1. Mmap'd files are useful since you can reuse disk cache as program
> memory.  So large files don't waste ram on the disk cache.  

I second that. mmaping has worked very well for me for large datasets,
especialy in the context of reducing memory pressure.

Gaël