[Python-ideas] Tulip / PEP 3156 - subprocess events

[Paul Moore]

On 18 January 2013 22:22, Guido van Rossum <guido at python.org> wrote:
>> The protocol part is then about adapting the transport API to
>> coroutine friendly readlines/writelines API (the part that Guido
>> points out needs more detail in
>> http://www.python.org/dev/peps/pep-3156/#coroutines-and-protocols)
>>
>> As a rough untested sketch (the buffering here could likely be a lot smarter):
>
> I have a more-or-less working but probably incomplete version checked
> into the tulip repo:
> http://code.google.com/p/tulip/source/browse/tulip/subprocess_transport.py

Ha! You beat me to it.

OK, looking at your code, I see that you freely used the
add_reader/add_writer functions and friends, and the fact that the
Unix selectors handle pipes as well as sockets. With the freedom to do
that, your code looks both reasonable and pretty straightforward. I
was having trouble getting past the fact that this approach wouldn't
work on Windows, and confusing "nonportable" with "not allowed". My
apologies. You kept telling me that writing the code for Unix would be
helpful, but I kept thinking in terms of writing code that worked on
Unix but with portability to Windows in mind, which completely misses
the point. I knew that the transport/protocol code I'd end up writing
would look something like this, but TBH I'd not seen that as the
interesting part of the problem...

BTW, to avoid duplication of the fork/exec stuff, I would probably
have written the transport to take a subprocess.Popen object as its
only argument, then hooked up self._wstdin to popen.stdin and
self._rstdout to popen.stdout. That requires the user to have created
the Popen object with those file descriptors as pipes (I don't know if
it's possible to introspect a Popen object to check that) but avoids
duplicating the subprocess logic. I can probably fairly quickly modify
your code to demonstrate, but it's late and I don't want to start
booting my Unix environment now, so it'll have to wait till tomorrow
:-)

Paul.