[Python-ideas] Thread exceptions and interruption

[Guido van Rossum]

Regarding the issue of exceptions in threads, I indeed see it as a
non-issue. It's easy enough to develop a subclass of threading.Thread
which catches any exceptions raised by run(), and stores the exception
as an instance variable from which it can be retrieved after join()
succeeds.

Regarding the proposal of branching the call stack, it reminds me too
much of the problems one has when a fork()'ed child raises an
exception which ends up being handled by an exception handler higher
up in the parent's call stack (which has been faithfully copied into
the child process by fork()). That has proven a major problem, leading
to various warnings to always catch all exceptions and call os._exit()
upon problems. I realize you're not proposing exactly that. I also
admit I don't exactly understand how you plan to deal with the
situation where one thread raises an exception which the spawning
location fails to handle, while another thread is still running (but
may raise another exception later). Is the spawning thread unwound?
Then what's left to catch the second thread's exception? But all in
all it gives me the heebie-jeebies.

Finally, may I suggest that you're perhaps too much in love with the
with-statement?

--Guido

On 9/18/07, Adam Olsen <rhamph at gmail.com> wrote:
> One of the core problems with threading is what to do with exceptions
> and how to gracefully exit when one goes unhandled.  My approach is to
> replace the independently spawned threads with "branches" off of your
> main thread's call stack.
>
> The standard example looks like this[1]:
>
> def handle_client(conn, addr):
>     with conn:
>         ...
>
> def accept_loop(server_conn):
>     with branch() as clients:
>         with server_conn:
>             while True:
>                 clients.add(handle_client, *server_conn.accept())
>
> The call stack will look something like this:
>
> main - accept_loop - server_conn.accept
>           |- handle_client
>           \- handle_client
>
> Here I use a with-statement[2] to create a branch point.  The branch
> point collects any exceptions from its children and interrupts the
> children when the first exception occurs.  Interruption is done
> somewhat similarly to posix cancellation; participating functions
> react to it.  However, I raise an Interrupted exception, which can
> lead to much more graceful cleanup than posix cancellation. ;)
>
> The __exit__ portion of branch's with-statement blocks until all child
> threads have exited.  It then reraises the exception, if any, or wraps
> it in MultipleError if several occurred.
>
> The branch construct serves only simple needs.  It does not attempt to
> limit the number of threads to the number of cores available, nor any
> related tricks.  Those can be added as a separate tool (perhaps
> wrapping branch.)
>
> Thoughts?  Competing ideas?  Disagreement that it's a "core problem" at all? ;)
>
>
> [1] I've previously (in private mostly) referred to the branch()
> function as collate().  I've recently decided to rename it.
>
> [2] Unfortunately, a with-statement lacks all the invariants that
> would be desirable for the branch construct.  It also has no direct
> way of handling generators-as-context-managers that themselves use
> branches.
>
> --
> Adam Olsen, aka Rhamphoryncus
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)