[Python-Dev] futures API

[Brian Quinlan]

On Dec 9, 2010, at 2:39 PM, Raymond Hettinger wrote:

>
> On Dec 9, 2010, at 9:02 AM, Brian Quinlan wrote:
>
>>
>> On Dec 9, 2010, at 4:26 AM, Thomas Nagy wrote:
>>
>>> Hello,
>>>
>>> I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86 
>>> ) by the futures module which was announced in python 3.2 beta. I  
>>> am a bit stuck with it, so I have a few questions about the futures:
>>>
>>> 1. Is the futures API frozen?
>>
>> Yes.
>
> Yes, unless the current API is defective in some way.  A beta1  
> release is a chance for everyone to exercise the new API and  
> discover whether it is problematic in any real world applications.
>
>>
>>> 2. How hard would it be to return the tasks processed in an output  
>>> queue to process/consume the results while they are returned? The  
>>> code does not seem to be very open for monkey patching.
>>
>> You can associate a callback with a submitted future. That callback  
>> could add the future to your queue.
>
> The would be a good example for the docs.

I don't know what Thomas' use case is but I expect that taking the  
results of a future and asynchronously sticking it in another queue is  
not typical.

Cheers,
Brian

>>
>>> 3. How hard would it be to add new tasks dynamically (after a task  
>>> is executed) and have the futures object never complete?
>>
>> I'm not sure that I understand your question. You can submit new  
>> work to an Executor at until time until it is shutdown and a work  
>> item can take as long to complete as you want. If you are  
>> contemplating tasks that don't complete then maybe you could be  
>> better just scheduling a thread.
>>
>>> 4. Is there a performance evaluation of the futures code  
>>> (execution overhead) ?
>>
>> No. Scott Dial did make some performance improvements so he might  
>> have a handle on its overhead.
>
> FWIW, the source code is short and readable.  From my quick read, it  
> looks to be a relatively thin wrapper/adapter around existing  
> tools.  Most of the work still gets done by the threads or processes  
> themselves.  Think of this as a cleaner, more centralized API around  
> the current toolset -- there is no deep new technology under the hood.
>
> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brian%40sweetapp.com