[Python-Dev] PEP 3148 ready for pronouncement

[Brian Quinlan]

On 26 May 2010, at 22:50, Antoine Pitrou wrote:

> On Wed, 26 May 2010 22:32:33 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>
>>> Ha, I'm a bit surprised. Isn't it what "futures" already provides?
>>> (except that for some reason it insists on the "SomeExecutor" naming
>>> scheme)
>>> http://www.python.org/dev/peps/pep-3148/#processpoolexecutor
>>
>> Not really - a general purpose pool would be a lot more agnostic  
>> about
>> how you give the pooled threads/processes work to do and get the  
>> results
>> back.
>>
>> Executors are the kind of thing you would build on top of one  
>> though. If
>> concurrent.pool was added, then the existing processing pools in
>> multiprocessing and the executors in concurrent.future would be the
>> first use cases for it.
>
> I think I'm a bit ignorant, but how is the Executor abstraction (and
> its proposed implementations) not generic enough? You have a pool,
> submit one or several tasks, and can either repeatedly poll for
> completion or do a blocking wait.
>
> (after all, Glyph pointed out that it should be quite easy to wrap the
> resulting Futures into Deferred objects)

Interesting. Executor.submit() return a Future, which might not be  
useful in some ThreadPool fire-and-forget use cases but having them  
doesn't seem harmful.

Java does take this approach and it gives you a lot more ways to  
customize the Executor thread pool i.e. the minimum number of threads  
running, the maximum number, the amount of time that a thread can be  
idle before it is killed, the queueing strategy to use (e.g. LIFO,  
FIFO, priority).

Cheers,
Brian