[Python-ideas] adding an __exec__ method to context managers?

[Fernando Perez]

Hi Sturla!


On Tue, 13 Oct 2009 03:46:22 +0200, Sturla Molden wrote:


> It occurs to me that all this would be a lot cleaner if context managers
> had an optional __exec__ method. It would receive the body as a code
> object, together with the local and global dicts for controlled
> execution. If it does not exist, something like this would be assumed:

[...]

I had this *precise* conversation at SciPy'08 at length with Alex 
Martelli, and more briefly at SciPy'09 with Peter Norvig, because this is 
something I've been tossing around for quite a while.  I actually 
implemented something similar in IPython, but which is horribly brittle 
because it digs the  parent context out by source introspection (using a 
custom exception to stop the execution flow at __enter__ time).  It worked 
actually perfectly, but I know such hacks aren't robust enough to be 
trusted in real production code.

I am somewhat skeptical that this idea could really fly in the long haul, 
because of all the issues regarding exactly what gets passed in (for lots 
of neat things you might want the real sources and not just the code 
object), etc.  Alex raised a number of detailed points in our conversation 
I don't have at the top of my head right now, but I could try to think a 
bit harder about the details if need be.

However, recently I've found 'peace' with this topic by using a different 
approach, that takes care of the problem you highlight in your post with 
having to call the _() function afterwards.  In python 3 with the new  non-
local keyword, this approach is actually very functional, and it solves 
lots of problems.  It's simply a matter of having a decorator consume the 
called function directly.  

Rather than repeat all this, I'll point you to a page where I summarized 
the whole topic at a recent talk at our Berkeley Scientific Python users 
group (it also contains links to a detailed discussion on this topic we 
had on the ipython list):

https://cirl.berkeley.edu/fperez/py4science/decorators.html

My current thinking on this matter is that I'll use this approach for a 
while, to get a better feel for the possibilities.  The syntax isn't 
ideal, but it's not horrible either, and it is quite flexible.  I think 
some real-world experience with this approach can teach us a lot, in order 
to later revisit the question with a really solid proposal for either a 
with extension like __exec__ or something else.

I hope this is useful, thanks a lot for bringing this up here (I've 
discussed this *exact* idea multiple times with colleagues, but never had 
the energy to carry it further on-list due to being too swamped  with 
other things).

Cheers,

f