[SciPy-User] Multiprocessing and shared memory

[Robin]

On Sun, Oct 18, 2009 at 7:18 PM, Felix Schlesinger <schlesin at cshl.edu> wrote:
> I have been working on an application using scipy that solves a highly
> parallel problem. To avoid the GIL in python I used to multiprocessing
> package. The main issue I ran into is shared memory. All workers share
> (read-only) access to a single large numpy array.

I'm probably wrong - but if it is really only read only access you
need to an array can't you just put it as a module variable before the
fork - then all the workers can access it and as long as they don't
touch it it shouldn't make a copy.

ie something like
import mymodule
mymodule.data = data_array
p = Pool(8)  or multiprocessing fork
then each process can access mymodule.data? I don't see why the
reference count would have to change if you leave the reference in a
module.

Cheers

Robin