[Numpy-discussion] Saving an array on disk to free memory - Pickling

[Francesc Alted]

A Monday 17 May 2010 13:03:19 Jean-Baptiste Rudant escrigué:
> Hello,
> 
> I tried to create an object :
> - which behave just like a numpy array ;
> - which can be saved on disk in an efficient way (numpy.save in my example
>  but with pytables in my real program) ; - which can be "unloaded" (if it
>  is saved) to free memory : it can exsit has an empty stuff which knows how
>  to retrieve real values ; it will be loaded only when we need to work with
>  it ; - which unloads itself before being pickled (values are already saved
>  and don't have to be pickled).
[clip]

Well, if you are using Linux, you can make use of /dev/shm in order to save 
your files in-memory instead of disk.  There are some considerations to have 
in mind for doing this though:

http://superuser.com/questions/45342/when-should-i-use-dev-shm-and-when-
should-i-use-tmp

However, I don't know the equivalent to this in Win, Mac OSX or other UNICES.

-- 
Francesc Alted