[Async-sig] killing tasks that won't cancel

[Chris Jerdonek]

On Tue, Feb 19, 2019 at 7:41 PM Nathaniel Smith <njs at pobox.com> wrote:

> I'm not sure what a "higher priority" exception is...raising an
> exception is hard to miss.
>

Just quickly on this one point: that was just my colloquial way of saying
superclass (or at least not a subclass), to emphasize that if you're e.g.
catching CancelledError, this new exception would bubble up whereas
CancelledError wouldn't. A similar example is KeyboardInterrupt not being
caught by Exception. In the case of CancelledError, we probably would want
the two exceptions to be comparable to one another rather than
incomparable. I seem to recall for example there being an old discussion as
to whether CancelledError should inherit from Exception or not.

--Chris




>
> There are a few things Trio does differently here that might be
> relevant, but it depends on why Chris is having trouble cancelling
> tasks.
>
> 1. Trio's cancellation exception, trio.Cancelled, inherits from
> BaseException instead of Exception, like KeyboardInterrupt or
> StopIteration. So 'except Exception' doesn't catch it by accident.
>
> 2. Trio's cancellation is "stateful": if your code is in the cancelled
> state, then every time you try to do an async operation then it raises
> trio.Cancelled again. So you avoid the case where a tasks gets stuck,
> someone forces it to raise CancelledError, and then it has a 'finally'
> block that tries to do some cleanup... but the 'finally' block also
> gets stuck. In trio the 'finally' block can't accidentally get stuck.
>
> 3. Both of these features are somewhat dependent on trio using
> "delimited" cancellation. Before you can cancel something, you have to
> say how far you want to unwind. This means that there's never any
> reason for anyone to try to catch 'Cancelled' on purpose, because trio
> will catch it for you at the appropriate moment. And it's hard to do
> 'stateful' cancellation if you don't know how long the state is
> supposed to persist. And, you avoid cases where some code thinks it
> just threw in a CancelledError and is supposed to catch it, but
> actually it was thrown in from some other stack frame, and it ends up
> confusedly catching the wrong exception.
>
> I'm not sure how much of this could be adapted for asyncio. The
> obvious change would be to make asyncio.CancelledError a
> BaseException, though it seems borderline to me from a back-compat
> perspective. I think I remember Yury was thinking about changing it
> anyway, though? That would definitely help with the 'except Exception'
> kind of mistake.
>
> But the other issues are deeper. If you don't have a solid system for
> keeping track of what exactly is supposed to be cancelled, then it's
> easy to accidentally cancel too much, or cancel too little. Solving
> that requires a systematic approach. And unfortunately, asyncio
> already has 2 different sets of cancellation semantics (Future.cancel
> -> takes effect immediately, irrevocable & idempotent, doesn't
> necessarily cause the underlying machinery to stop processing, just
> stops it from reporting its result; Task.cancel -> doesn't take effect
> immediately or necessarily at all, can be called repeatedly and
> injects one CancelledError per call, tries to stop the underlying
> machinery, chains to other tasks/futures that the first task is
> await'ing). So if our goal is to make the system as a whole as
> reliable and predictable as possible within the constraints of
> back-compat... I don't know whether adding a third set of semantics
> would actually help, or make more code confused about what it was
> supposed to be catching.
>
> And I don't know if any of these actually address whatever problem
> you're having with uncancellable tasks. It's certainly possible to
> make an uncancellable task in Trio too. We just try to make it hard to
> do by accident.
>
> -n
>
> On Tue, Feb 19, 2019 at 3:53 PM Chris Jerdonek <chris.jerdonek at gmail.com>
> wrote:
> >
> > On Tue, Feb 19, 2019 at 12:33 PM Yury Selivanov <yselivanov at gmail.com>
> wrote:
> >>
> >> Unfortunately asyncio isn't super flexible around "cancellation with a
> timeout" kind of scenarios. The current assumption is that once the
> cancellation is requested, the Task will start cancelling and will do so in
> a timely manner.  Imposing a second layer of timeouts on the cancellation
> process itself isn't natively supported.  But to properly address this we
> don't need a very broadly defined Task.set_exception(); we need to rethink
> the cancellation in asyncio (perhaps draw some inspiration from Trio and
> other frameworks).
> >
> >
> > What options have you already considered for asyncio's API? A couple
> naive things that occur to me are adding task.kill() with higher priority
> than task.cancel() (or equivalently task.graceful_cancel() with lower
> priority). Was task.cancel() meant more to have the meaning of "kill" or
> "graceful cancel"? In addition, the graceful version of the two (whichever
> that may be) could accept a timeout argument -- after which the exception
> of higher priority is raised. I realize this is a more simplistic model
> compared to the options trio is considering, but asyncio has already gone
> down the path of the simpler approach.
> >
> > --Chris
> >
> >
> >>
> >>
> >> Yury
> >
> > _______________________________________________
> > Async-sig mailing list
> > Async-sig at python.org
> > https://mail.python.org/mailman/listinfo/async-sig
> > Code of Conduct: https://www.python.org/psf/codeofconduct/
>
>
>
> --
> Nathaniel J. Smith -- https://vorpus.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20190219/c1082d87/attachment-0001.html>