[Python-ideas] multiprocessing IPC

[Sturla Molden]

Den 12.02.2012 14:52, skrev shibturn:
>
> As Mike says, on Unix you can just create a file in /tmp to back an 
> mmap.  On Linux, posix mmaps created with shm_open() seem to be normal 
> files on a tmpfs file system, usually /dev/shm.  Since /tmp is also 
> usually a tmpfs file system on Linux, I assume this whould be 
> equivalent in terms of overhead.

Mark did not use shm_open, he memory mapped from disk.


> I have a patch (http://bugs.python.org/issue8713) to make 
> multiprocessing on Unix work with fork+exec which has to do this 
> because semaphores cannot be inherited across exec.  Making sure all 
> the named semaphores get removed if the program terminates abnormally 
> is a bit awkward though.  It could be modified to make them picklable 
> in general.
>
> On Windows dealing with "named objects" is easier since they are 
> refcounted by the operating system and deleted when no more processes 
> have handles for them.
>
> If you make a feature request at bugs.python.org I might work on a patch.

Cleaning up SysV ipc semaphores and shared memory is similar (semctl 
instead of shmctl to get refrerence count). And then we need a monkey 
patch for os._exit.

Look at the Cython code here:
http://dl.dropbox.com/u/12464039/sharedmem.zip

Sturla