[Python-Dev] Allow __enter__() methods to skip the with statement body?

[Nick Coghlan]

An interesting discrepancy [1] has been noted when comparing
contextlib.nested (and contextlib.contextmanager) with the equivalent
nested with statements.

Specifically, the following examples behave differently if
cmB().__enter__() raises an exception which cmA().__exit__() then
handles (and suppresses):

  with cmA():
    with cmB():
      do_stuff()
  # This will resume here without executing "Do stuff"

  @contextlib.contextmanager
  def combined():
    with cmA():
      with cmB():
        yield

  with combined():
    do_stuff()
  # This will raise RuntimeError complaining that the underlying
  # generator didn't yield

  with contextlib.nested(cmA(), cmB()):
    do_stuff()
  # This will raise the same RuntimeError as the contextmanager
  # example (unsurprising, given the way nested() is implemented)

The problem arises any time it is possible to skip over the yield
statement in a contextlib.contextmanager based context manager without
raising an exception that can be seen by the code calling __enter__().

I think the right way to fix this (as suggested by the original poster
of the bug report) is to introduce a new flow control exception along
the lines of GeneratorExit (e.g. SkipContext) and tweak the expansion of
the with statement [2] to skip the body of the statement if __enter__()
throws that specific exception:

  mgr = (EXPR)
  exit = mgr.__exit__  # Not calling it yet
  try:
    value = mgr.__enter__()
  except SkipContext:
    pass # This exception handler is the new part...
  else:
    exc = True
    try:
      VAR = value  # Only if "as VAR" is present
      BLOCK
    except:
      # The exceptional case is handled here
      exc = False
      if not exit(*sys.exc_info()):
        raise
      # The exception is swallowed if exit() returns true
    finally:
      # The normal and non-local-goto cases are handled here
      if exc:
        exit(None, None, None)

Naturally, contextlib.contextmanager would then be modified to raise
SkipContext instead of RuntimeError if the generator doesn't yield. The
latter two examples would then correctly resume execution at the first
statement after the with block.

I don't see any other way to comprehensively fix the problem - without
it, there will always be some snippets of code which cannot correctly be
converted into context managers, and those snippets won't always be
obvious (e.g. the fact that combined() is potentially a broken context
manager implementation would surprise most people - it certainly
surprised me).

Thoughts? Do people hate the idea? Are there any backwards compatibility
problems that I'm missing? Should I write a PEP or just add the feature
to the with statement in 2.7/3.1?

Cheers,
Nick.

[1] http://bugs.python.org/issue5251

[2] http://www.python.org/dev/peps/pep-0343/
-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------