[SciPy-User] Speeding things up - how to use more than one computer core

Pauli Virtanen pav at iki.fi
Mon Apr 8 13:53:07 EDT 2013


Hi,

08.04.2013 19:55, Gael Varoquaux kirjoitti:
> On Mon, Apr 08, 2013 at 07:44:20AM -0500, J. David Lee wrote:
>> I've used shared memory arrays in the past, and it's actually quite easy. They
>> can be created using the multiprocessing module in a couple of lines,
> 
>> mp_arr = multiprocessing.Array(ctypes.c_double, 100)
>> arr = np.frombuffer(mp_arr.get_obj())
> 
> I believe that this does synchronization by message passing. Look at the
> corresponding multiprocessing code if you want to convince yourself. Thus
> you are not in fact sharing the memory between processes.

I think this uses memory obtained from a mmap, which is shared between
the parent child processes.

-- 
Pauli Virtanen




More information about the SciPy-User mailing list