multiprocessing: Correct usage of pool & queue?

Aahz aahz at pythoncraft.com
Thu Sep 10 10:37:25 EDT 2009


In article <mailman.974.1252080796.2854.python-list at python.org>,
Allen Fowler  <allen.fowler at yahoo.com> wrote:
>
>1) Have a maximum of 20 "in-flight" tasks.  (thus worker processes?)

Good bet.

>3) Certain tasks in my list must be processed in the correct order.  (I
>guess the asignment logic must somehow tag those to by done by the same
>worker?)

The simpler way to do this would be to bundle these tasks into a single
queue object that contains a list of tasks.  Each worker iterates over
the list of tasks that it receives, which could be a single task.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy at lion.austin.ibm.com



More information about the Python-list mailing list