pre-PEP: Simple Thunks

[Brian Sabbey]

Here is a first draft of a PEP for thunks.  Please let me know what you 
think. If there is a positive response, I will create a real PEP.

I made a patch that implements thunks as described here. It is available 
at:
     http://staff.washington.edu/sabbey/py_do

Good background on thunks can be found in ref. [1].

Simple Thunks
-------------

Thunks are, as far as this PEP is concerned, anonymous functions that 
blend into their environment. They can be used in ways similar to code 
blocks in Ruby or Smalltalk. One specific use of thunks is as a way to 
abstract acquire/release code. Another use is as a complement to 
generators.

A Set of Examples
=================

Thunk statements contain a new keyword, 'do', as in the example below. The 
body of the thunk is the suite in the 'do' statement; it gets passed to 
the function appearing next to 'do'. The thunk gets inserted as the first 
argument to the function, reminiscent of the way 'self' is inserted as the 
first argument to methods.

def f(thunk):
    before()
    thunk()
    after()

do f():
    stuff()

The above code has the same effect as:

before()
stuff()
after()

Other arguments to 'f' get placed after the thunk:

def f(thunk, a, b):
     # a == 27, b == 28
     before()
     thunk()
     after()

do f(27, 28):
     stuff()

Thunks can also accept arguments:

def f(thunk):
    thunk(6,7)

do x,y in f():
    # x==6, y==7
    stuff(x,y)

The return value can be captured

def f(thunk):
    thunk()
    return 8

do t=f():
    # t not bound yet
    stuff()

print t
==> 8

Thunks blend into their environment

def f(thunk):
    thunk(6,7)

a = 20
do x,y in f():
    a = 54
print a,x,y

==> 54,6,7

Thunks can return values.  Since using 'return' would leave it unclear 
whether it is the thunk or the surrounding function that is returning, a 
different keyword should be used. By analogy with 'for' and 'while' loops, 
the 'continue' keyword is used for this purpose:

def f(thunk):
    before()
    t = thunk()
    # t == 11
    after()

do f():
    continue 11

Exceptions raised in the thunk pass through the thunk's caller's frame 
before returning to the frame in which the thunk is defined:

def catch_everything(thunk):
     try:
         thunk()
     except:
         pass    # SomeException gets caught here

try:
     do catch_everything():
        raise SomeException
except:
     pass        # SomeException doesn't get caught here because it was 
already caught

Because thunks blend into their environment, a thunk cannot be used after 
its surrounding 'do' statement has finished:

thunk_saver = None
def f(thunk):
     global thunk_saver
     thunk_saver = thunk

do f():
     pass

thunk_saver() # exception, thunk has expired

'break' and 'return' should probably not be allowed in thunks.  One could 
use exceptions to simulate these, but it would be surprising to have 
exceptions occur in what would otherwise be a non-exceptional situation. 
One would have to use try/finally blocks in all code that calls thunks 
just to deal with normal situations.  For example, using code like

def f(thunk):
     thunk()
     prevent_core_meltdown()

with code like

do f():
     p = 1
return p

would have a different effect than using it with

do f():
     return 1

This behavior is potentially a cause of bugs since these two examples 
might seem identical at first glance.

The thunk evaluates in the same frame as the function in which it was 
defined. This frame is accessible:

def f(thunk):
     frame = thunk.tk_frame

do f():
     pass

Motivation
==========

Thunks can be used to solve most of the problems addressed by PEP 310 [2] 
and PEP 288 [3].

PEP 310 deals with the abstraction of acquire/release code. Such code is 
needed when one needs to acquire a resource before its use and release it 
after.  This often requires boilerplate, it is easy to get wrong, and 
there is no visual indication that the before and after parts of the code 
are related.  Thunks solve these problems by allowing the acquire/release 
code to be written in a single, re-usable function.

def acquire_release(thunk):
    f = acquire()
    try:
       thunk(f)
    finally:
       f.release()

do t in acquire_release():
    print t

More generally, thunks can be used whenever there is a repeated need for 
the same code to appear before and after other code. For example,

do WaitCursor():
      compute_for_a_long_time()

is more organized, easier to read and less bug-prone than the code

DoWaitCursor(1)
compute_for_a_long_time()
DoWaitCursor(-1)

PEP 288 tries to overcome some of the limitations of generators.  One 
limitation is that a 'yield' is not allowed in the 'try' block of a 
'try'/'finally' statement.

def get_items():
     f = acquire()
     try:
         for i in f:
             yield i   # syntax error
     finally:
         f.release()

for i in get_items():
     print i

This code is not allowed because execution might never return after the 
'yield' statement and therefore there is no way to ensure that the 
'finally' block is executed.  A prohibition on such yields lessens the 
suitability of generators as a way to produce items from a resource that 
needs to be closed.  Of course, the generator could be wrapped in a class 
that closes the resource, but this is a complication one would like to 
avoid, and does not ensure that the resource will be released in a timely 
manner.  Thunks do not have this limitation because the thunk-accepting 
function is in control-- execution cannot break out of the 'do' statement 
without first passing through the thunk-accepting function.

def get_items(thunk):    # <-- "thunk-accepting function"
     f = acquire()
     try:
         for i in f:
             thunk(i)      # A-OK
     finally:
         f.release()

do i in get_items():
    print i

Even though thunks can be used in some ways that generators cannot, they 
are not nearly a replacement for generators.  Importantly, one has no 
analogue of the 'next' method of generators when using thunks:

def f():
     yield 89
     yield 91

g = f()
g.next()  # == 89
g.next()  # == 91

[1] see the "Extended Function syntax" thread, 
http://mail.python.org/pipermail/python-dev/2003-February/
[2] http://www.python.org/peps/pep-0310.html
[3] http://www.python.org/peps/pep-0288.html