[Python-ideas] A couple of with statement ideas

[Nick Coghlan]

I've been considering which broad concerns I want to focus on for 3.3,
and have come to the conclusion that I'm going to have plenty to keep
me busy between helping to smooth over some of the rough edges left
over from the PEP 3118 implementation, as well as championing the
module aliasing concept I first brought up a few weeks ago.

That means there are a couple of with statement ideas that I still
like, but almost certainly won't have the time to champion myself. So,
I'm lobbing them over the fence in a half-baked form. If anyone feels
strongly enough about them to get them into a PEP-worthy state, go
right ahead (that's the whole point of this message). If nobody else
cares enough to pick them up... well, that will simply justify my
decision that they weren't important enough to worry about.

Idea 1: Implicit context managers

PEP 343 included the idea of a "__context__" method that permitted the
creation of objects with an implicit associated context manager. It
was eventually dropped because we couldn't find a good way to explain
it to users at the time. Instead, you see things like
"decimal.localcontext()" and various other objects that need to be
called every time you want to use them as a context manager. If you
want an object to have a "native" context manager, you have to either
write __enter__ and __exit__ manually, or use a mixin or class
decorator that adapts the __enter__/__exit__ interface to a single
method (e.g. the ContextManager class and manage_context method from
GarlicSim that Ram Rachum posted about here some time ago).

Now that everyone has had time to get used to the way context managers
work, I believe the concept may usefully make a return using the
"implicit context manager" terminology. While the "__context__" name
has since been claimed by PEP 3134, the alternative name used in PEP
346 ("__with__") is still a possibility.

The idea of bringing back this concept would be to allow an object to
implement *either* a __with__ method that returns a context manager,
or else implement __enter__/__exit__ directly. The behaviour would be
similar to the relationship between iterators and iterables, except
that a missing "__with__" implementation would imply the "return self"
semantics that iterators must implement explicitly.

Anyone picking up this idea should be prepared for a lot of pushback,
as the last few years have shown us that the with statement is
perfectly usable without this feature, and the GarlicSim example shows
that this is already feasible using existing mechanisms (even I am at
best lukewarm on the concept).


Idea 2: Son of PEP 377 (letting context managers skip the body of the
with statement)

The niggles that lead me to write PEP 377 still bug me. There are some
nested with statements that are perfectly valid as inline code but
will throw RuntimeError if you make them into a single context manager
via contextlib.contextmanager.

That PEP was rightly rejected as having too great an impact on
"normal" code for something that is a comparatively exotic corner
case. As I noted in issue 5251, I've since thought of an alternative,
lower impact solution that may prove more acceptable to Guido and
others: an optional "__entered__" method for context managers that, if
present, would be executed *inside* the scope of the try/finally block
before the body of the with statement itself started executing. Any
exceptions raised in that method would be passed to the __exit__
method, thus increasing the flexibility of the context management
protocol, while having minimal impact

Most significantly, contextlib.GeneratorContextManager could be
adjusted to make use of this feature to correctly skip the body of the
with statement when the internal generator doesn't yield (attempt to
invoke __enter__ a second time would still trigger RuntimeError,
though). The translation of the problematic nested with statements
into a single generator based context manager would then work
correctly, functioning in the same way as the equivalent inline code.

I really like this idea, since it provides a genuinely new capability
to the context management protocol in a relatively low impact manner.
However, as noted in the intro, there are other, more immediately
practical, questions I want to deal with first, so who knows if or
when I'll be able to devote a significant amount of time to this one.

So, if anyone is feeling particularly keen to champion a PEP and dig
into the internals of contextlib and CPython's with statement
implementation, there's a couple of ideas for you to mull over :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia