pre-PEP: Simple Thunks

[Brian Sabbey]

Bengt Richter wrote:
>> Good background on thunks can be found in ref. [1].
>
> UIAM most of that pre-dates decorators. What is the relation of thunks
> to decorators and/or how might they interact?

Hmm, I think you answered this below better than I could ;).

>> def f(thunk):
>>    before()
>>    thunk()
>>    after()
>>
>> do f():
>>    stuff()
>>
>> The above code has the same effect as:
>>
>> before()
>> stuff()
>> after()
> Meaning "do" forces the body of f to be exec'd in do's local space? What if there
> are assignments in f? I don't think you mean that would get executed in do's local space,
> that's what the thunk call is presumably supposed to do...

Yes, I see now that there is an ambiguity in this example that I did not 
resolve.  I meant that the suite of the 'do' statement gets wrapped up as 
an anonymous function.  This function gets passed to 'f' and can be used 
by 'f' in the same way as any other function.  Bindings created in 'f' are 
not visible from the 'do' statement's suite, and vice-versa.  That would 
be quite trickier than I intended.

>> Other arguments to 'f' get placed after the thunk:
>>
>> def f(thunk, a, b):
>>     # a == 27, b == 28
>>     before()
>>     thunk()
>>     after()
>>
>> do f(27, 28):
>>     stuff()
> I'm not sure how you intend this to work. Above you implied (ISTM ;-)
> that the entire body of f would effectively be executed locally. But is that
> true? What if after after() in f, there were a last statment hi='from last statement of f'
> Would hi be bound at this point in the flow (i.e., after d f(27, 28): stuff() )?

I didn't mean to imply that.  The body of 'f' isn't executed any 
differently if it were called as one normally calls a function.  All of 
its bindings are separate from the bindings in the 'do' statement's scope.

> I'm thinking you didn't really mean that. IOW, by magic at the time of calling thunk from the
> ordinary function f, thunk would be discovered to be what I call an executable suite, whose
> body is the suite of your do statement.

yes

> In that case, f iself should not be a callable suite, since its body is _not_ supposed to be called locally,
> and other than the fact that before and after got called, it was not quite exact to say it was _equivalent_ to
>
> before()
> stuff() # the do suite
> after()

yes, I said "same effect as" instead of "equivalent" so that too much 
wouldn't be read from that example.  I see now that I should have been 
more clear.

> In that case, my version would just not have a do, instead defining the do suite
> as a temp executable suite, e.g., if instead
>
>
> we make an asignment in the suite, to make it clear it's not just a calling thing, e.g.,
>
> do f(27, 28):
>      x = stuff()
>
> then my version with explict name callable suite would be
>
> def f(thunk, a, b):
>      # a == 27, b == 28
>      before()
>      thunk()
>      after()
>
> set_x():
>     x = stuff() # to make it plain it's not just a calling thing
>
> f(set_x, 27, 28)
> # x is now visible here as local binding
>
> but a suitable decorator and an anonymous callable suite (thunk defined my way ;-) would make this
>
> @f(27, 28)
> (): x = stuff()
>

Hmm, but this would require decorators to behave differently than they do 
now.  Currently, decorators do not automatically insert the decorated 
function into the argument list.  They require you to define 'f' as:

def f(a, b):
 	def inner(thunk):
 		before()
 		thunk()
 		after()
 	return inner

Having to write this type of code every time I want an thunk-accepting 
function that takes other arguments would be pretty annoying to me.

>>
>> Thunks can also accept arguments:
>>
>> def f(thunk):
>>    thunk(6,7)
>>
>> do x,y in f():
>>    # x==6, y==7
>>    stuff(x,y)
>
> IMO
>    @f
>    (x, y): stuff(x, y)   # like def foo(x, y): stuff(x, y)
>
> is clearer, once you get used to the missing def foo format
>

OK.  I prefer a new keyword because it seems confusing to me to re-use 
decorators for this purpose.  But I see your point that decorators can be 
used this way if one allows anonymous functions as you describe, and if 
one allows decorators to handle the case in which the function being 
decorated is anonymous.

>>
>> The return value can be captured
>>
> This is just a fallout of f's being an ordinary function right?
> IOW, f doesn't really know thunk is not an ordinary callable too?
> I imagine that is for the CALL_FUNCTION byte code implementation to discover?

yes

>
>> def f(thunk):
>>    thunk()
>>    return 8
>>
>> do t=f():
>>    # t not bound yet
>>    stuff()
>>
>> print t
>> ==> 8
> That can't be done very well with a decorator, but you could pass an
> explicit named callable suite, e.g.,
>
>     thunk(): stuff()
>     t = f(thunk)

But not having to do it that way was most of the purpose of thunks.

>>
>> Thunks blend into their environment
> ISTM this needs earlier emphasis ;-)
>
>>
>> def f(thunk):
>>    thunk(6,7)
>>
>> a = 20
>> do x,y in f():
>>    a = 54
>> print a,x,y
>>
>> ==> 54,6,7
>
> IMO that's more readable as
>
>    def f(thunk):
>        thunk(6, 7)
>    @f
>    (x, y):         # think def foo(x, y): with "def foo" missing to make it a thunk
>        a = 54
>    print a,x,y
>
> IMO we need some real use cases, or we'll never be able to decide what's really useful.
>>
>> Thunks can return values.  Since using 'return' would leave it unclear
>> whether it is the thunk or the surrounding function that is returning, a
>> different keyword should be used. By analogy with 'for' and 'while' loops,
>> the 'continue' keyword is used for this purpose:
> Gak ;-/
>>
>> def f(thunk):
>>    before()
>>    t = thunk()
>>    # t == 11
>>    after()
>>
>> do f():
>>    continue 11
>
> I wouldn't think return would be a problem if the compiler generated a
> RETURN_CS_VALUE instead of RETURN_VALUE when it saw the end of
> the callable suite (hence _CS_) (or thunk ;-)
> Then it's up to f what to do with the result. It might pass it to after() sometimes.

It wouldn't be a problem to use 'return' instead of 'continue' if people 
so desired, but I find 'return' more confusing because a 'return' in 'for' 
suites returns from the function, not from the suite.  That is, having 
'return' mean different things in these two pieces of code would be 
confusing:

for i in [1,2]:
 	return 10

do i in each([1,2]):
 	return 10


>
>>
>> Exceptions raised in the thunk pass through the thunk's caller's frame
>> before returning to the frame in which the thunk is defined:
> But it should be possible to have try/excepts within the thunk, IWT?

yes, it is possible and they are allowed in the example implementation.

>>
>> def catch_everything(thunk):
>>     try:
>>         thunk()
>>     except:
>>         pass    # SomeException gets caught here
>>
>> try:
>>     do catch_everything():
>>        raise SomeException
>> except:
>>     pass        # SomeException doesn't get caught here because it was
>> already caught
>>
>> Because thunks blend into their environment, a thunk cannot be used after
>> its surrounding 'do' statement has finished:
>>
>> thunk_saver = None
>> def f(thunk):
>>     global thunk_saver
>>     thunk_saver = thunk
>>
>> do f():
>>     pass
>>
>> thunk_saver() # exception, thunk has expired
> Why? IWT the above line would be equivalent to executing the suite (pass) in its place.

The restriction on saving thunks for later is for performance reasons and 
because I believe it is hacky to use thunks in that way.

> What happens if you defined
>
> def f(thunk):
>    def inner(it):
>        it()
>    inner(thunk)
>
> do f():
>    x = 123
>
> Of course, I'd spell it
>    @f
>    (): x = 123
>
> Is there a rule against that (passing thunk on to inner)?

No, that is fine, as long as execution has not yet left the 'do' 
statement.

>> 'break' and 'return' should probably not be allowed in thunks.  One could
>> use exceptions to simulate these, but it would be surprising to have
>> exceptions occur in what would otherwise be a non-exceptional situation.
>> One would have to use try/finally blocks in all code that calls thunks
>> just to deal with normal situations.  For example, using code like
>>
>> def f(thunk):
>>     thunk()
>>     prevent_core_meltdown()
>>
>> with code like
>>
>> do f():
>>     p = 1
>> return p
>>
>> would have a different effect than using it with
>>
>> do f():
>>     return 1
>>
>> This behavior is potentially a cause of bugs since these two examples
>> might seem identical at first glance.
> I think less so with decorator and anonymous callable suite format
>
>    @f
>    (): return 1  # as in def foo(): return 1  -- mnemonically removing "def foo"
>

In your syntax, 'return' would return from the thunk.  With the 'do' 
syntax, 'return' would return from the surrounding function.  So the issue 
does not arise with your syntax.

>>
>> The thunk evaluates in the same frame as the function in which it was
>> defined. This frame is accessible:
>>
>> def f(thunk):
>>     frame = thunk.tk_frame
>      # no connection with tkinter, right? Maybe thunk._frame would also say be careful ;-)
>      assert sys._getframe(1) is frame # ?? when does that fail, if it can?

'tk' is supposed to remind one of 'thunk'.  I believe that follows a 
standard naming convention in python (e.g. see generators).

I don't see how that assert can fail.

I see what you're getting at with decorators and anonymous functions, but 
there are a couple of things that, to me, make it worth coming up with a 
whole new syntax.  First, I don't like how one does not get the thunk 
inserted automatically into the argument list of the decorator, as I 
described above.  Also, I don't like how the return value of the decorator 
cannot be captured (and is already used for another purpose).  The fact 
that decorators require one more line of code is also a little bothersome. 
Decorators weren't intended to be used this way, so it seems somewhat 
hacky to do so.  Why not a new syntax?

-Brian