moving Connection/PipeConnection between processes

Randall Smith randall at tnr.cc
Sat Jun 13 03:23:37 EDT 2009


I've got a situation in which I'd like to hand one end of a pipe to 
another process.  First, in case you ask why, a spawner process is 
created early before many modules are imported.  That spawner process is 
responsible for creating new processes and giving a proxy to the parent 
process.

    (h1-1   <--  Pipe()   -->    h1-2)
   |------------------------> child1
   |(s1 < Pipe() > s2)
parent ->  spawner ->    |--> child2
   |(h2-1)  <--  Pipe()   -->   (h2-2)
   |----------------------|
   |------------------------> child3
    (h3-1   <--  Pipe()   -->    h3-2)

When I try to pass Connection h1-1 to the parent using Connection s1, I 
get an error:

TypeError: Required argument 'handle' (pos 1) not found

This error is from unpickling the Connection h1-1.  You can duplicate 
the error like this:

pickled_connection = pickle.dumps(h1-1, 2)
pickle.loads(pickled_connection) # raises the same error

Looking at the pickle docs, I wonder if this could be resolved by adding 
a __getnewargs__ method to _multiprocessing.Connection.  But even if 
that would work I couldn't do it now since it's an extension module. 
I've thought about trying to recreate the Connection.  Looks like it 
should be possible with Connection.fileno().  The Unix portion looks 
easy, but the win32 portion does not.

So if it's possible, what's the best way to pass a Connection to another 
process?

Randall




More information about the Python-list mailing list