[Numpy-discussion] Openmp support (was numpy's future (1.1 and beyond): which direction(s) ?)

[Robert Kern]

On Mon, Mar 24, 2008 at 12:12 PM, Gnata Xavier <xavier.gnata at gmail.com> wrote:

>  Well it is not that easy. We have several numpy code following like this :
>  1) open an large data file to get a numpy array
>  2) perform computations on this array (I'm only talking of the numpy
>  part here. scipy is something else)
>  3) Write the result is another large file
>
>  It is so simple to write using numpy :)
>  Now, if I want to have several exe, step 3 is often a problem.

If that large file can be accessed by memory-mapping, then step 3 can
actually be quite easy. You have one program make the empty file of
the given size (f.seek(FILE_SIZE); f.write('\0'); f.seek(0,0)) and
then make each of the parallel programs memory map the file and only
write to their respective portions.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco