Direct interaction with subprocess - the curse of blocking I/O

Pascal Chambon chambon.pascal at gmail.com
Mon Jun 29 15:15:52 EDT 2009


Hello everyone

I've had real issues with subprocesses recently : from a python script, 
on windows, I wanted to "give control" to a command line utility, i.e 
forward user in put to it and display its output on console. It seems 
simple, but I ran into walls :
- subprocess.communicate() only deals with a forecast input, not 
step-by-step user interaction
- pexpect module is unix-only, and for automation, not interactive input
- when wanting to do all the job manually (transfering data between the 
standard streams of the python program and the binary subprocess, I met 
the issue : select() works only on windows, and python's I/O are 
blocking, so I can't just, for example, get data from the subprocess' 
stdout and expect the function to return if no input is present - the 
requesting thread might instead block forever.

Browsing the web, I found some hints :
- use the advanced win32 api to create non-blocking I/O : rather 
complicated, non portable and far from the python normal files
- use threads that block on the different streams and eat/feed them 
without ever stopping : rather portable, but gives problems on shutdown 
(How to terminate these threads without danger ? On some OSes, a process 
never dies as long as any thread - even "daemonic" - lives, I've seen 
people complaining about it).

So well, I'd like to know, do you people know any solution to this 
simple problem - making a user interact directly with a subprocess ? Or 
would this really require a library handling each case separately (win32 
api, select().....) ?

Thanks a lot for your interest and advice,
regards,
Pascal



More information about the Python-list mailing list