[Python-ideas] Protecting finally clauses of interruptions

[Paul Colomiets]

Hi Yury,

On Thu, Apr 5, 2012 at 12:24 AM, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> On 2012-04-04, at 4:43 PM, Paul Colomiets wrote:
>>> How does it work?  To what object are you actually attaching timeout?
>>>
>>
>> There is basically a "Coroutine" object. It's actually a list with
>> paused generators, with top of them being currently running
>> (or stopped for doing IO). It represents stack, because there
>> is no built-in stack for generators.
>
> Interesting.  I decided to go with a simple coroutine objects with
> a '_caller' pointer to maintain the stack virtually.
>

It doesn't matter. IIRC, that was to draw a tree of calls starting
with roots. But that's irrelevant to the topic of discussion.

>> This framework does timeout handling in described way:
>>
>> https://github.com/tailhook/zorro
>>
>> Although, it's using greenlets. The difference is that we
>> we don't need to keep a stack in our own scheduler
>> when using greenlets, but everything else applies.
>
> Are you using that particular framework (zorro)?  Or some modification
> of it that uses generators too?
>

Currently we experimenting with greenlets and zorro. My
description of yield-based coroutines from earlier project
(unfortunately non-public one).

> OK, we're on the same page.  '''"frames are not connected" from the
> interpreter point of view''', that essentially means that 'f_in_finally'
> will always be a flag related only to its own frame, right?  Any
> 'propagation' of this flag is the responsibility of framework developers.
>

Yes, that's ok for me.

>> So for example:
>>
>> def a():
>>  yield b()
>>
>> def b():
>>  yield
>>
>> If `a().with_timeout(0.1)` is interrupted when it's waiting for value
>> of `b()`, will `b()` continue it's execution?
>
> Well, in our framework, if a() is getting aborted after it's scheduled
> b(), but before it received the result of b(), we interrupt both of them
> (and those that b() might had called).
>

Exactly! And you don't want them to be interrupted in the case
`a()` rewritten as:

def a():
  try: pass
  finally: yield b()

Same with threads and greenlets.

>>> I think even if it's decided to implement just your proposal, I feel
>>> that 'f_in_finally' should indicate the state of only its *own* frame.
>>
>> That was original intention. But it requires stack traversing. Andrew
>> proposed to propagate this flag, which is another point of view
>> on the same thing (not sure which one to pick though)
>
> Again, if coroutines' frames aren't connected on the interpreter level
> (it's the responsibility of a framework), about what exact propagation
> are you and Andrew talking (in the sole context of the patch to cpython)?
>

For threads and greenlets flag can be implicitly propagated, and
for yield-based coroutines f_in_finally can be made writable, so you
can propagate it in you own scheduler

Not sure It's good idea, just describing it.

-- 
Paul