How to initialize each multithreading Pool worker with an individual value?

[Adam Tauno Williams]

On Thu, 2010-12-30 at 08:01 -0800, Aahz wrote:
> In article <mailman.61.1291152972.2649.python-list at python.org>,
> Valery Khamenya  <khamenya at gmail.com> wrote:
> >However it doesn't look possible to use it to initialize each Pool's
> >worker with some individual value (I'd wish to be wrong here)
> >So, how to initialize each multithreading Pool worker with the
> >individual values?
> >The typical use case might be a connection pool, say, of 3 workers,
> >where each of 3 workers has its own TCP/IP port.
> >from multiprocessing.pool import Pool
> >def port_initializer(_port):
> >    global port
> >    port = _port
> >def use_connection(some_packet):
> >    global _port
> >    print "sending data over port # %s" % port
> >if __name__ == "__main__":
> >    ports=((4001,4002, 4003), )
> >    p = Pool(3, port_initializer, ports) # oops... :-)
> You probably can't use initargs here.  Your port_initializer needs to be
> some kind of class instance that works with multiprocessing and emits one
> port number when its __call__() method gets invoked.  (There may be other
> ways to accomplish the same effect, but that's what springs to mind.)

Maybe this is obvious; but it is possible to create a pool of workers
all listening on the same socket.  An idle worker will pick-up the
connection. Just open the socket in the initial process and then fork
your workers - they will inherit the file handle and can accept() on it.