pre-PEP: Simple Thunks

[Brian Sabbey]

Ron_Adam wrote:
> On Sat, 16 Apr 2005 17:25:00 -0700, Brian Sabbey
>> Yes, much of what thunks do can also be done by passing a function
>> argument.  But thunks are different because they share the surrounding
>> function's namespace (which inner functions do not), and because they can
>> be defined in a more readable way.
>
> Generally my reason for using a function is to group and separate code
> from the current name space.  I don't see that as a drawback.

I agree that one almost always wants separate namespaces when defining a 
function, but the same is not true when considering only callback 
functions.  Thunks are a type of anonymous callback functions, and so a 
separate namespace usually isn't required or desired.

> Are thunks a way to group and reuse expressions in the current scope?
> If so, why even use arguments?  Wouldn't it be easier to just declare
> what I need right before calling the group?  Maybe just informally
> declare the calling procedure in a comment.
>
> def thunkit:	  # no argument list defines a local group.
> 	# set thunk,x and y before calling.
> 	before()
> 	result = thunk(x,y)
> 	after()
>
> def foo(x,y):
> 	x, y = y, x
> 	return x,y
>
> thunk = foo
> x,y = 1,2
> do thunkit
> print result
>
> -> (2,1)
>
> Since everything is in local name space, you  really don't need to
> pass arguments or return values.

I'm kicking myself for the first example I gave in my original post in 
this thread because, looking at it again, I see now that it really gives 
the wrong impression about what I want thunks to be in python.  The 
'thunkit' function above shouldn't be in the same namespace as the thunk. 
It is supposed to be a re-usable function, for example, to acquire and 
release a resource.  On the other hand, the 'foo' function is supposed to 
be in the namespace of the surrounding code; it's not re-usable.  So your 
example above is pretty much the opposite of what I was trying to get 
across.

>
> The 'do' keyword says to evaluate the group.  Sort of like eval() or
> exec would, but in a much more controlled way.  And the group as a
> whole can be passed around by not using the 'do'.  But then it starts
> to look and act like a class with limits on it.  But maybe a good
> replacement for lambas?
>
> I sort of wonder if this is one of those things that looks like it
> could be useful at first, but it turns out that using functions and
> class's in the proper way, is also the best way. (?)

I don't think so.  My pickled_file example below can't be done as cleanly 
with a class.  If I were to want to ensure the closing of the pickled 
file, the required try/finally could not be encapsulated in a class or 
function.

>
>> You're right that, in this case, it would be better to just write
>> "f(stuff, 27, 28)".  That example was just an attempt at describing the
>> syntax and semantics rather than to provide any sort of motivation.  If
>> the thunk contained anything more than a call to 'stuff', though, it would
>> not be as easy as passing 'stuff' to 'f'.  For example,
>>
>> do f(27, 28):
>> 	print stuff()
>>
>> would require one to define and pass a callback function to 'f'.  To me,
>> 'do' should be used in any situation in which a callback *could* be used,
>> but rarely is because doing so would be awkward.  Probably the simplest
>> real-world example is opening and closing a file.  Rarely will you see
>> code like this:
>>
>> def with_file(callback, filename):
>> 	f = open(filename)
>> 	callback(f)
>> 	f.close()
>>
>> def print_file(file):
>> 	print file.read()
>>
>> with_file(print_file, 'file.txt')
>>
>> For obvious reasons, it usually appears like this:
>>
>> f = open('file.txt')
>> print f.read()
>> f.close()
>>
>> Normally, though, one wants to do a lot more than just print the file.
>> There may be many lines between 'open' and 'close'.  In this case, it is
>> easy to introduce a bug, such as returning before calling 'close', or
>> re-binding 'f' to a different file (the former bug is avoidable by using
>> 'try'/'finally', but the latter is not).  It would be nice to be able to
>> avoid these types of bugs by abstracting open/close.  Thunks allow you to
>> make this abstraction in a way that is more concise and more readable than
>> the callback example given above:
>
> How would abstracting open/close help reduce bugs?

I gave two examples of bugs that one can encounter when using open/close. 
Personally, I have run into the first one at least once.

> I'm really used to using function calls, so anything that does things
> differently tend to be less readable to me. But this is my own
> preference.  What is most readable to people tends to be what they use
> most. IMHO
>
>> do f in with_file('file.txt'):
>> 	print f.read()
>>
>> Thunks are also more useful than callbacks in many cases since they allow
>> variables to be rebound:
>>
>> t = "no file read yet"
>> do f in with_file('file.txt'):
>> 	t = f.read()
>>
>> Using a callback to do the above example is, in my opinion, more
>> difficult:
>>
>> def with_file(callback, filename):
>> 	f = open(filename)
>> 	t = callback(f)
>> 	f.close()
>> 	return t
>>
>> def my_read(f):
>> 	return f.read()
>>
>> t = with_file(my_read, 'file.txt')
>
> Wouldn't your with_file thunk def look pretty much the same as the
> callback?

It would look exactly the same.  You would be able to use the same 
'with_file' function in both situations.

> I wouldn't use either of these examples.  To me the open/read/close
> example you gave as the normal case would work fine, with some basic
> error checking of course.

But worrying about the error checking is what one wants to avoid.  Even if 
it is trivial to remember to close a file, it's annoying to have to think 
about this every time one wants to use a file.  It would be nice to be 
able to worry about closing the file exactly once.

It's also annoying to have to use try/finally over and over again as one 
would in many real-life situations.  It would be nice to be able to think 
about the try/finally code once and put it in a re-usable function.

The open/close file is just the simplest example.  Instead of a file, it 
maybe be a database or something more complex.

> Since Python's return statement can handle
> multiple values, it's no problem to put everything in a single
> function and return both the status with an error code if any, and the
> result.  I would keep the open, read/write, and close statements in
> the same function and not split them up.

What about try/finally?  What if it is more complex than just opening and 
closing a file?  The example that got me annoyed enough to write this 
pre-PEP is pickling and unpickling.  I want to unpickle a file, modify it, 
and immediately pickle it again.  This is a pretty easy thing to do, but 
if you're doing it over and over again, there gets to be a lot of 
boilerplate.  One can of course create a class to handle the boilerplate, 
but instantiating a class is still more complicated than it has to be. 
Here is an example of using thunks to do this:

def pickled_file(thunk, name):
 	f = open(name, 'r')
 	l = pickle.load(f)
 	f.close()
 	thunk(l)
 	f = open(name, 'w')
 	pickle.dump(l, f)
 	f.close()

Now I can re-use pickled_file whenever I have to modify a pickled file:

do data in pickled_file('pickled.txt'):
 	data.append('more data')
 	data.append('even more data')

In my opinion, that is easier and faster to write, more readable, and less 
bug-prone than any non-thunk alternative.

>>> When I see 'do', it reminds me of 'do loops'. That is 'Do' involves
>>> some sort of flow control.  I gather you mean it as do items in a
>>> list, but with the capability to substitute the named function.  Is
>>> this correct?
>>
>> I used 'do' because that's what ruby uses for something similar.
>>
>
> I could see using do as an inverse 'for' operator.  Where 'do' would
> give values to items in a list, verses taking them from a list. I'm
> not exactly sure how that would work. Maybe...
>
> def fa(a,b):
> 	return a+b
> def fb(c,d):
> 	return c*b
> def fc(e,f):
> 	return e**f
>
> fgroup:
> 	fa(a,b)
> 	fb(c,d)
> 	fc(e,f)
>
> results = do 2,3 in flist
>
> print results
> ->  (5, 6, 8)
>
> But this is something else, and not what your thunk is trying to do.
>
>
>
> So it looks to me you have two basic concepts here.
>
> (1.) Grouping code in local name space.
>
> I can see where this could be useful. It would be cool if the group
> inherited the name space it was called with.
>
> (2.) A way to pass values to items in the group.
>
> Since you are using local space, that would be assignment statements
> in place of arguments.
>
> Would this work ok for what you want to do?
>
> def with_file:				# no argument list, local group.
> 	f = open(filename)
> 	t = callback(f)
> 	f.close
>
> def my_read(f):
> 	return f.read()
>
> callback = my_read
> filename = 'filename'
> do with_file

This wouldn't work since with_file wouldn't be re-usable.  It also doesn't 
get rid of the awkwardness of defining a callback.

-Brian