[Python-Dev] PEP 3148 ready for pronouncement

[Brian Quinlan]

On May 28, 2010, at 1:39 PM, Scott Dial wrote:

> On 5/27/2010 4:13 AM, Brian Quinlan wrote:
>> On 27 May 2010, at 17:53, Floris Bruynooghe wrote:
>>> On Thu, May 27, 2010 at 01:46:07PM +1200, Greg Ewing wrote:
>>>> On 27/05/10 00:31, Brian Quinlan wrote:
>>>>> You have two semantic choices here:
>>>>> 1. let the interpreter exit with the future still running
>>>>> 2. wait until the future finishes and then exit
>>>>
>>>> I'd go for (1). I don't think it's unreasonable to
>>>> expect a program that wants all its tasks to finish
>>>> to explicitly wait for that to happen.
>>>
>>> I'd got for (1) as well, it's no more then reasonable that if you  
>>> want
>>> a result you wait for it.  And I dislike libraries doing magic you
>>> can't see, I'd prefer if I explicitly had to shut a pool down.
>>>
>>> I'm glad I'm not alone in preferring (1) tough.
>>
>> Keep in mind that this library magic is consistent with the library
>> magic that the threading module does - unless the user sets
>> Thread.daemon to True, the interpreter does *not* exit until the  
>> thread
>> does.
>
> Given your rationale, I don't understand from the PEP:
>
>> shutdown(wait=True)
>>
>> Signal the executor that it should free any resources that it is
>> using when the currently pending futures are done executing. Calls to
>> Executor.submit and Executor.map and made after shutdown will raise
>> RuntimeError.
>>
>> If wait is True then the executor will not return until all the
>> pending futures are done executing and the resources associated with
>> the executor have been freed.
>
> Can you tell me what is the expected execution time of the following:
>
>>>> executor = ThreadPoolExecutor(max_workers=1)
>>>> executor.submit(lambda: time.sleep(1000))
>>>> executor.shutdown(wait=False)
>>>> sys.exit(0)
>
> I believe it's 1000 seconds, which seems to defy my request of
> shutdown(wait=False) because "secretly" the Python exit is going to  
> wait
> anyways.

It would take 1000 seconds. "...then the executor will not return..."  
should read "...then the method will not return...".

> ISTM, it is much easier to get behavior #2 if you have behavior
> #1, and it would also seem rather trivial to make ThreadPoolExecutor
> take an optional argument specifying which behavior you want.

Adding a daemon option would be reasonable. If you don't shutdown your  
executors you are pretty much guaranteed to get random traceback  
output on exit through.

> Your reference implementation does not actually implement the
> specification given in the PEP, so it's quite impossible to check this
> myself. There is no wait=True option for shutdown() in the reference
> implementation, so I can only guess what that implementation might  
> look
> like.

Look at around line 129 in:
http://code.google.com/p/pythonfutures/source/browse/branches/feedback/python3/futures/thread.py

Cheers,
Brian