WaitForMultipleObjects in Python.

[David Bolen]

Peter Hansen <peter at engcorp.com> writes:

> I'm perfectly _uncertain_ that it's possible.  Doesn't support for
> that require some sort of unified mechanism right at the OS level?
> All the ways of "waiting" in Python probably use Python or C code
> wrapped around various underlying OS mechanisms.  If those
> mechanisms are not already unified at the OS level, I have a
> suspicion (been getting lots of those lately) that it is simply
> impossible to do a "multi-wait", no matter how tricky the Python
> code, or C code, or anything else, that one might do.

Yep - "uncertain" it is...

The problem is that at the Python level it doesn't even offer the
ability to wait on multiple of the same object type (e.g., wait on
multiple Event objects).  You'd have to write it as a polling loop.
Of course, some of the timeout wait delays even on standard
synchronization objects are themselves implemented as polling loops in
the library.

I believe this is mostly because Python depends on very few underlying
synchronization primitives from the OS.  I think that just about
everything is built on top of the thread module lock object, and
that's a fairly minimalistic primitive that supports acquisition, with
or without blocking, but without a timeout.  The lack of timeout at
this level is what makes other objects that offer timeouts have to
resort to polling, but it does place minimal demands on the underlying
OS.

For the OP, about the best you can do is dedicate a thread for each of
the operations that might result in an event or indication to wake up
the single piece of action code.  Each of those worker threads can
then put something into a shared Queue, which the action thread is
listening on.  The queue acts as the aggregation mechanism (and each
worker thread as your independently notifiable activity).

What this doesn't let you do is have a single thread that can wake up
to support multiple operations (such as an I/O thread handling data in
both directions).  For that you pretty much need to use some
OS-specific routines (such as a select based system for Unix, or the
actual WaitForMultipleObjects under Windows).

For example, when I wanted a SerialIO object that supported full
non-blocking I/O to the serial port under Windows, I ended up
implementing a thread using overlapped I/O with
WaitForMultipleObjects.  There just wasn't any equivalent in portable
Python.

> For example, if your OS doesn't use the same mechanism for
> waiting for serial port input, file input, socket input,
> keyboard or other user input, or timing input, then at least
> one of those will not play nicely with a multi-wait mechanism.

This is definitely true as well, but there have been cases that I'd be
just as happy to be able to wait on multiple Python threading objects
(e.g., block a thread until something shows up on a Queue or one of
several events fire without having to create a custom Queue message
format to carry the extra events).

-- David