[Python-ideas] ScopeGuardStatement/Defer Proposal

[Nathan Rice]

> Since there have been a few proposals along these lines recently:
>
> Nothing is going to happen on the dedicated syntax front in the
> deferred execution space at least until I get contextlib.CallbackStack
> into Python 3.3 and we gather additional feedback on patterns of use
> (and, assuming that API addresses the relevant use cases the way I
> plan, these features will *never* need dedicated syntax).
>
> A preliminary version of the API is available in the contextlib2
> backport as ContextStack:
> http://contextlib2.readthedocs.org/en/latest/index.html#contextlib2.ContextStack
>
> See the issue tracker for the changes that are planned in order to
> update that to the new CallbackStack API:
> https://bitbucket.org/ncoghlan/contextlib2/issue/8/rename-contextstack-to-callbackstack-and
>
>> ... snip ...
>
> With ContextStack:
>
> def ordering_example():
>    with ContextStack() as stack:
>        print(1)
>        stack.register(print, 2)
>        stack.register(print, 3)
>        print(4)
>
>
> With the planned CallbackStack API:
>
> def ordering_example():
>    with CallbackStack() as stack:
>        print(1)
>        stack.push(print, 2)
>        stack.push(print, 3)
>        print(4)

Hi Nick,

I just wanted to chime in on this, because I understand the use cases
and benefits of this but the code is very semantically opaque and
imperative.  I also feel like a lot of C programming concepts and
semantics have leaked into the design.  Additionally, I feel that
there are some benefits to taking a step back and looking at this
problem as part of a bigger picture.

Fundamentally, context managers are ways to convert a block of code
into an event, with __enter__ analogous to "before_block" and __exit__
analogous to an "after_block".  There are a couple of problems with
context managers that I feel an event system handles more elegantly:

1.)  Context is ambiguous.  Context could be interpreted to mean a
thread, a scope, a point in time, etc.  Context managers only deal
with the narrow problem of a block of code being run.  This is
succinctly described as an event.

2.)  The context manager API requires you to fire events before and
after the code block (yes, you can pass) and does not provide other
options, such as (in an ideal world of python with well behaved
threads) an event that is fired/in the active state concurrent to the
block of code's execution.  There are a few ways to hack this behavior
but they're all bad, and interoperability between libraries is
unlikely.

3.) If you want to extend context management for a particular piece of
code, you have to modify the code to add another context manager, or
monkey patch the existing context manager.  Modifying the code is has
some thorny issues, for instance, if you need to modify the context
handling in a third party lib, all of a sudden you have to fork the
lib and manually patch every time you upgrade or redeploy.  Monkey
patching is easy, but from a conceptual/readability perspective it is
horrible.  If the lib fires events, you can just register an action on
the event in your code and live happily ever after.

4.) The way context managers are defined only allows you describe a
linear chain of events, because they are associated with a block of
code, and the act of association precludes other context managers from
firing events for that same block of code.  Because of this, you have
things like register and preserve that exist to add support for (weak)
non-linearity.

5.) Going back to event concurrency and touching on non-linearity
again, if I have two functions that I've asked to fire when an event
occurs, this provides a strong clue to the interpreter that the given
functions could potentially run in parallel.  Of course, there would
need to be other cues, but I don't think people want to be in the
business of explicitly writing parallel code forever.

6.) User interface coders going back 30 years understand events pretty
well, but will probably give you a blank stare for a second or two if
you mention context managers.

I feel that "with"/context managers are an elegant solution to the
simple problem, however it seems like the generalized solution based
on context managers is pretty awkward.  The right thing to do in my
opinion would be to go back to the drawing board, design an event
subsystem that maps to something like pi-calculus/interval temporal
logic in a human/pythonic way.  This will avoid the immediate issues
like the necessary goofiness of contextlib2, and lay the groundwork
for nice things like automatic parallelization.

Of course, I'm anal about getting things 100% right, and context
managers are very nice, simple, elegant 80% solution.  If 99% of
people are happy with the 80% solution, it is probably the right thing
to do just to force a ugly hack on the remaining 1%.

Take care,

Nathan