atexit + threads = bug?

[Tim Peters]

[David Rushby]
> Consider the following program (underscores are used to force
> indentation):
> ------------------------------------------------
> import atexit, threading, time
>
> def atExitFunc():
> ____print 'atExitFunc called.'
>
> atexit.register(atExitFunc)
>
> class T(threading.Thread):
> ____def run(self):
> ________assert not self.isDaemon()
> ________print 'T before sleep.'
> ________time.sleep(1.0)
> ________print 'T after sleep.'
>
> T().start()
> print 'Main thread finished.'
> ------------------------------------------------
>
> I would expect the program to print 'atExitFunc called.' after 'T after
> sleep.',

Why?  I expect very little ;-)

> but instead, it prints (on Windows XP with Python 2.3.5 or
> 2.4.2):
> ------------------------------------------------
> T before sleep.
> Main thread finished.
> atExitFunc called.
> T after sleep.
> ------------------------------------------------

That's not what I saw just now on WinXP Pro SP2.  With 2.3.5 and 2.4.2
I saw this order instead:

    Main thread finished
    atExitFunc called.
    T before sleep.
    T after sleep.

The relative order of "Main thread finished." and "T before sleep" is
purely due to timing accidents; it's even possible for "T after
sleep." to appear before "Main thread finished.", although it's not
possible for "T after sleep." to appear before "T before sleep.".  In
fact, there are only two orderings you can count on here:

    T before sleep < T after sleep
    Main thread finished < atExitFunc called

If you need more than that, you need to add synchronization code.

> atExitFunc is called when the main thread terminates, rather than when
> the process exits.

Is there a difference between "main thread terminates" and "the
process exits" on Windows?  Not in C.  It so happens that Python's
threading module _also_ registers an atexit callback, which does a
join() on all the threads you created and didn't mark as daemon
threads.  Because threading.py's atexit callback was registered first,
it gets called last when Python is shutting down, and it doesn't
return until it joins all the non-daemon threads still sitting around.
 Your atexit callback runs first because it was registered last.  That
in turn makes it _likely_ that you'll see (as we both saw) "at
exitFunc called." before seeing "T after sleep.", but doesn't
guarantee that.

Don't by fooled by _printing_ "Main thread finished", BTW:  that's
just a sequence of characters ;-).  The main thread still does a lot
of work after that point, to tear down the interpreter in a sane
order.  Part of that work is threading.py waiting for your threads to
finish.

> The atexit documentation contains several warnings,
> but nothing about this.  Is this a bug?

It doesn't look like a bug to me, and I doubt Python wants to make
stronger promises than it does now about the exact order of assorted
exit gimmicks.

You can reliably get "atExitFunc called." printed last by delaying
your import of the threading module until after you register your
atExitFunc callback.  If you register that first, it's called last,
and threading.py's wait-for-threads-to-end callback gets called first
then.  That callback won't return before your worker thread finishes.

There's no promise that will continue to work forever, though.  This
is fuzzy stuff vaguely covered by the atexit doc's "In particular,
other core Python modules are free to use atexit without the
programmer's knowledge."  threading.py happens to be such a module
today, but maybe it won't be tomorrow.