[Python-ideas] ExitStack: Allow exiting individual context managers

[Ram Rachum]

I would actually want a method that exits not just the last context
manager, but any context manager in the stack according to my choosing.
Maybe it clashes with the fact that you're using `deque`, but I'm not sure
that you have a compelling reason to use `deque`.

If you're asking about my use case: It's pretty boring. I have a sysadmin
script with a long function that does remote actions on a few servers. I
wrapped it all in an `ExitStack` since I use file-based locks and I want to
ensure they get released eventually. Now, at some point I want to release
the file-based lock manually, but I can't use a with statement, because
there's a condition around the place where I acquire the lock. It's
something like this:

if condition:

    exit_stack.enter_context(get_lock_1())

else:

    exit_stack.enter_context(get_lock_2())

So ideally I would want a method that takes a context manager and just
exits it. Maybe even add an optional argument `context_manager` to the
existing `close` method. Personally I don't care about exception-handling
in this case, and while I think it would be nice to
include exception-handling, I see that the existing close method doesn't
provide exception-handling either, so I wouldn't feel bad about it.

So maybe something like this:

def close(self, context_manager=None):
    """Immediately unwind the context stack"""

    if context_manager is None:

        self.__exit__(None, None, None)

    else:

        for _exit_wrapper in reversed(self._exit_callbacks):

            if _exit_wrapper.__self__ is context_manager:

                _exit_wrapper(None, None, None)

                self._exit_callbacks.remove(_exit_wrapper)

Maybe also support accepting a tuple of context managers.

On Mon, Dec 7, 2015 at 5:40 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 7 December 2015 at 10:32, Steven D'Aprano <steve at pearwood.info> wrote:
> > On Sun, Dec 06, 2015 at 11:48:41PM +0200, Ram Rachum wrote:
> >> Hi guys,
> >>
> >> I'm using `contextlib.ExitStack` today, and pushing context managers
> into
> >> it. I find myself wanting to exit specific context managers that I've
> >> pushed into it, while still inside the `with` suite of the `ExitStack`.
> In
> >> other words, I want to exit one of the context managers but still keep
> the
> >> `ExitStack`, and all other context managers, acquired. This isn't
> currently
> >> possible, right? What do you think about implementing this?
> >
> > I'm not entirely sure what you mean. Can you give an example?
>
> It's a concept I considered implementing, but decided to hold off on
> it because there are a few different design options here and I didn't
> have any use cases to guide the API design, nor the capacity to do
> usability testing to see if the additional API complexity actually
> made ExitStack easier to use overall.
>
> The first design option is the status quo: using multiple with
> statement blocks, potentially in conjunction with multiple ExitStack
> instances. The virtue of this approach is that it means that once a
> context manager is added to an ExitStack instance, that's it - its
> lifecycle is now coupled to that of the other context managers in the
> stack. You can postpone cleaning up all of them with "pop_all()"
> (transferring responsibility for the cleanup to a fresh ExitStack
> instance), but you can't selectively clean them up from the end. The
> usage guidelines are thus relatively simple: if you don't want to
> couple the lifecycles of two context managers together, then don't add
> them to the same ExitStack instance.
>
> However, there is also that "Stack" in the name, so it's natural for
> users to expect to be able to both push() *and* pop() individual
> context managers on the stack.
>
> The (on the surface) simplest design option to *add* to the API would
> be a single "pop()" operation that returned a new ExitStack instance
> (for return type consistency with pop_all()) that contained the last
> context manager pushed onto the stack. However, this API is
> problematic, as you've now decoupled the nesting of the context
> manager stack - the popped context manager may now survive beyond the
> closure of the original ExitStack. Since this kind a pop_all()
> inspired selective clean-up API would result in two different
> ExitStack instances anyway, the status quo seems cleaner to me than
> this option, as it's obvious from the start that there are seperate
> ExitStack instances with potentially distinct lifecycles.
>
> The next option would then be to offer a separate
> "exit_last_context()" method, that exited the last context manager
> pushed onto the stack. This would be a single-stepping counterpart to
> the existing close() method, that allowed you to dynamically descend
> and ascend the context management stack during normal operation, while
> still preserving the property that the entire stack will be cleaned up
> when encountering an exception.
>
> Assuming we went with that simpler in-place API, there would still be
> a number of further design questions to be answered:
>
> * Do we need to try to manage the reported exception context the way
> ExitStack.__exit__ does?
> * Does "exit_last_context()" need to accept exception details like
> __exit__ does?
> * Does "exit_last_context()" need to support the ability to suppress
> exceptions?
> * What, if anything, should the return value be?
> * What happens if there are no contexts on the stack to pop?
> * Should it become possible to query the number of registered callbacks?
>
> Taking them in order, as a matter of implementation feasibility, the
> answer to the first question likely needs to be "No". For consistency
> with calling __exit__ methods directly, the answers to the next three
> questions likely need to be "support the same thing __exit__
> supports".
>
> For the second last question, while it's reasonable to call close(),
> pop_all() or __exit__() on an empty stack and have it silently do
> nothing, if someone has taken it on themselves to manage the stack
> depth manually, then it's likely more helpful to complain that the
> stack is empty than it is to silently do nothing. Since
> exit_last_context() may behave differently depending on whether or not
> there are items on the stack, and the number of items on the stack
> would be useful for diagnostic purposes, then it likely also makes
> sense to implement a __len__ method that delegated to
> "len(self._exit_callbacks)".
>
> That all suggests a possible implementation along the lines of the
> following:
>
>     def exit_last_context(self, *exc_details):
>         if not self._exit_callbacks:
>             raise RuntimeError("Attempted to exit last context on
> empty ExitStack instance")
>         cb = self._exit_callbacks.pop()
>         return cb(*exc_details)
>
>     def __len__(self):
>         return len(self._exit_callbacks)
>
> What I don't know is whether or not that's actually a useful enough
> improvement over the status quo to justify the additional cognitive
> burden when learning the ExitStack API - the current API was designed
> around the ExitStack recipes in the documentation, which were all
> fairly common code patterns, but most cases where I might consider
> using an "exit_last_context()" method, I'd be more inclined to follow
> Steven's advice and use a separate context manager for the resource
> with an independent lifecycle.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151207/c725a2bf/attachment-0001.html>