Pool module -- does map pre-assign tasks to processes?

Luca luca.de.alfaro at gmail.com
Mon Sep 28 13:02:41 EDT 2009


I would like to use the Pool module, but my tasks take sometimes
unpredictably different time to complete.  The simplest way to write
the code would be to put all task descriptions in an array, then call

p = Pool(8)
p.map(f, a)

But I don't want to preassign process 0 with elements a[0], a[8], a
[16],  process 1 with elements a[1], a[9], a[17], and so forth.
Rather, I would like all tasks to be put into a queue, and I would
like the processes to each time grab the next task to be done, and do
it.  This would ensure fairly equal loading.

My question is: does the map() method of Pool pre-assign which task
gets done by each process, or is this done at runtime, on a get-first-
task-to-be-done basis?

Many thanks,

Luca



More information about the Python-list mailing list