[SciPy-user] mpi4py - MPI_Comm_join example ?

[Sebastien Binet]

hi there,

I am trying to parallelize an application with mpi4py.
For various reasons[*], I need to dynamically create new processes and get 
inter-communicators via MPI_Comm_join.
Documentation about that function is pretty scarce and I am still a MPI 
rookie.

I came up with that kind of code:
##
import sys,os,socket
from mpi4py import MPI

def msg (fmt, *args):
    map (sys.stdout.write,
         ('[%i] :: '%os.getpid(),
          fmt%args,
          '\n'))
    sys.stdout.flush()

def main(ntasks=5):
    msg ("creating a socket pair...")
    fd_parent, fd_child = socket.socketpair (socket.AF_UNIX)
    msg ("forking...")
    pid = os.fork()
    if pid == 0:
        fd_child.close() # not needed anymore
        msg ("child process...")
        main_child (fd_parent)
    else:
        fd_parent.close() # idem
        msg ("parent process...")
        main_parent (fd_child)
    return

def main_child (fd):
    comm = MPI.COMM_WORLD
    msg ("joining parent communicator...")
    icomm = MPI.Comm.Join (fd.fileno())
    fd.close() # not needed anymore
    msg ("icomm is [%s]", "OK" if icomm != MPI.COMM_NULL else "ERR")
    if icomm == MPI.COMM_NULL:
        return
    ### do something with icomm...
    return

def main_parent (fd):
    comm = MPI.COMM_WORLD
    msg ("mpi: rank=%r nprocs=%r", comm.rank, comm.size)
    msg ("joining child communicator...")
    icomm = MPI.Comm.Join (fd.fileno())
    fd.close() # not needed anymore
    msg ("icomm is [%s]", "OK" if icomm != MPI.COMM_NULL else "ERR")
    if icomm == MPI.COMM_NULL:
        return
    ### do something with icomm
    return
##

I'd like to fork as many as `ntasks` subprocesses and farm out some work...
But I am a bit stuck as to how to proceed...
Any help would be much appreciated.

Cheers,
Sebastien.

[*] main reason being that the application takes ~1-2 minutes to completely 
initialize and eats up to 1-2 Gb of memory. Hence, relying on os.fork to 
efficiently share memory between processes after the initialization completed 
seems to be a good idea.
I don't think I can use MPI_Win_xyz as the pieces of data I want to 
share/reuse among processes are C++ containers, so the usual problem of STL 
allocators and shm-segments kicks in...
Spawning new processes via MPI_Comm_spawn is a no go either b/c of the above 
main reason...
-- 
###################################
# Dr. Sebastien Binet             #
# Lawrence Berkeley National Lab. #
# 1 Cyclotron Road                #
# Berkeley, CA 94720              #
###################################
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20080724/b2c6c380/attachment.sig>