[stdlib-sig] futures - a new package for asynchronous execution

[Jeffrey Yasskin]

I'm not going to be the one jerk holding back this proposal, so go
ahead and submit it to python-dev. I'm not around again until
Saturday, so I won't get a chance to comment until then.

On Thu, Mar 4, 2010 at 8:02 PM, Brian Quinlan <brian at sweetapp.com> wrote:
>
> On Feb 26, 2010, at 2:26 PM, Jeffrey Yasskin wrote:
>
>> On Thu, Feb 25, 2010 at 7:10 PM, Brian Quinlan <brian at sweetapp.com> wrote:
>>>
>>> On Feb 26, 2010, at 4:27 AM, Jeffrey Yasskin wrote:
>>>>
>>>> Heh. If you're going to put that in the pep, at least make it correct
>>>> (sleeping is not synchronization):
>>>
>>> I can't tell if you are joking or not. Was my demonstration of a possible
>>> deadlock scenario really unclear?
>>
>> It's clear; it's just wrong code, even if the futures weren't a cycle.
>> Waiting using sleep in any decently-sized system is guaranteed to
>> cause problems. Yes, this example will work nearly every time
>> (although if you get your load high enough, you'll still see
>> NameErrors), but it's not the kind of thing we should be showing
>> users. (For that matter, communicating between futures using globals
>> is also a bad use of them, but it's not outright broken.)
>
> Hey Jeff,
>
> I'm trying to demonstrate a pattern of executor usage that is likely to lead
> to deadlock.
>
> If, looking at the example, people are clear that this may lead to deadlock
> then I don't think that is necessary to write an example that provably
> always leads to deadlock.
>
> In fact, I think that all of the extra locking code required really
> distracts from the core of the problem being demonstrated.
>
>
>>>> Thanks. I still disagree, and think users are much more likely to be
>>>> surprised by occasional deadlocks due to cycles of executors than they
>>>> are
>>>> about guaranteed deadlocks from cycles of futures, but I don't want to
>>>> be
>>>> the only one holding up the PEP by insisting on this.
>>>
>>> Cycles of futures are not guaranteed to deadlock. Remove the sleeps from
>>> my
>>> example and it will deadlock a small percentage of the time.
>>
>> It only fails to deadlock when it fails to create a cycle of futures.
>>
>> It sounds like Antoine also wants you to either have the threaded
>> futures steal work or detect executor cycles and raise an exception.
>
>
> I really don't like the idea of work stealing.
>
> Do you have a concrete proposal on how to detect cycles?
>
> Cheers,
> Brian
>



-- 
Namasté,
Jeffrey Yasskin
http://jeffrey.yasskin.info/