How to initialize each multithreading Pool worker with an individual value?

[James Mills]

On Wed, Dec 1, 2010 at 7:35 AM, Valery Khamenya <khamenya at gmail.com> wrote:
> multithreading.pool Pool has a promissing initializer argument in its
> constructor.
> However it doesn't look possible to use it to initialize each Pool's
> worker with some individual value (I'd wish to be wrong here)
>
> So, how to initialize each multithreading Pool worker with the
> individual values?
>
> The typical use case might be a connection pool, say, of 3 workers,
> where each of 3 workers has its own TCP/IP port.
>
> from multiprocessing.pool import Pool
>
> def port_initializer(_port):
>    global port
>    port = _port
>
> def use_connection(some_packet):
>    global _port
>    print "sending data over port # %s" % port
>
> if __name__ == "__main__":
>    ports=((4001,4002, 4003), )
>    p = Pool(3, port_initializer, ports) # oops... :-)
>    some_data_to_send = range(20)
>    p.map(use_connection, some_data_to_send)

I assume you are talking about multiprocessing
despite you mentioning "multithreading" in the mix.

Have a look at the source code for multiprocessing.pool
and how the Pool object works and what it does
with the initializer argument. I'm not entirely sure it
does what you expect and yes documentation on this
is lacking...

cheers
James

-- 
-- James Mills
--
-- "Problems are solved by method"