How to initialize each multithreading Pool worker with an individual value?

Valery khamenya at gmail.com
Thu Dec 2 09:43:27 EST 2010


On Dec 1, 3:24 am, James Mills <prolo... at shortcircuit.net.au> wrote:
> I assume you are talking about multiprocessing
> despite you mentioning "multithreading" in the mix.

yes, sorry.


> Have a look at the source code for multiprocessing.pool
> and how the Pool object works and what it does
> with the initializer argument. I'm not entirely sure it
> does what you expect and yes documentation on this
> is lacking...

I see

I found my way "to seed" each member of Pool with own data. I do it
right after after initialization:

port = None
def port_seeder(port_val)
    from time import sleep
    sleep(1) # or less...
    global port
    port = port_val

if __name__ == '__main__':
    pool = Pool(3)
    pool.map(port_seeder, range(3), chunksize=1)
    # now child processes are initialized with individual values.

Another (a bit more heavier) approach would be via shared resource.

P.S. sorry, I found your answer only now.

reagrds
--
Valery



More information about the Python-list mailing list