How to initialize each multithreading Pool worker with an individual value?

Aahz aahz at pythoncraft.com
Thu Dec 30 11:01:52 EST 2010


In article <mailman.61.1291152972.2649.python-list at python.org>,
Valery Khamenya  <khamenya at gmail.com> wrote:
>
>However it doesn't look possible to use it to initialize each Pool's
>worker with some individual value (I'd wish to be wrong here)
>
>So, how to initialize each multithreading Pool worker with the
>individual values?
>
>The typical use case might be a connection pool, say, of 3 workers,
>where each of 3 workers has its own TCP/IP port.
>
>from multiprocessing.pool import Pool
>
>def port_initializer(_port):
>    global port
>    port = _port
>
>def use_connection(some_packet):
>    global _port
>    print "sending data over port # %s" % port
>
>if __name__ == "__main__":
>    ports=((4001,4002, 4003), )
>    p = Pool(3, port_initializer, ports) # oops... :-)

You probably can't use initargs here.  Your port_initializer needs to be
some kind of class instance that works with multiprocessing and emits one
port number when its __call__() method gets invoked.  (There may be other
ways to accomplish the same effect, but that's what springs to mind.)
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Think of it as evolution in action."  --Tony Rand



More information about the Python-list mailing list