[Python-ideas] Tulip / PEP 3156 - subprocess events

[Paul Moore]

On 17 January 2013 12:23, Paul Moore <p.f.moore at gmail.com> wrote:
> In general, it still feels to me like the socket use case is being
> treated as "special", and other data sources and sinks (subprocesses
> being my use case, but I'm sure others exist) are either second-class
> or require a whole set of their own specialised methods, which isn't
> practical.

Thinking about this some more. The key point is that for any event
loop there can only be one "source of events" in terms of the thing
that the event loop checks when there are no pending tasks. So the
event loop is roughly:

while True:
    process_ready_queue()
    new_events = block_on_event_source(src, timeout=N)
    add_to_ready_queue(new_events)
    add_timed_events_to_ready_queue()

The source has to be a unique object, as there's an OS-level wait in
there, and you can't do two of them at once.

As things stand, methods like add_reader on the event loop object
should really be methods on the event source object (and indeed,
that's more or less what Tulip does internally). Would it not make
more sense to explicitly expose the event source? This is (I guess)
what the section "Choosing an Event Loop Implementation" in the PEP is
about. But if the event source is a user-visible object, methods like
add_reader would no longer be optional event loop methods, but rather
they would be methods of the event source (but only for those event
sources for which they make sense).

The point here is that there's a lot of event loop machinery (ready
queue, timed events, run methods) that are independent of the precise
means by which you poll the OS to ask "has anything interesting
happened?" Abstracting out that machinery would seem to me to make the
design cleaner and more understandable.

Other benefits - our hypothetical person with a serial port device can
build his own event source and plug it into the event loop directly.
Or someone could offer a multiplexer that combines two separate
sources by running them in different threads and merging the output on
a queue (that may be YAGNI, though).

This is really just something to think about while I'm trying to build
a Linux development environment so that I can do a Unix proof of
concept. Once I get started on that, I'll think about the
protocol/transport stuff.

Paul