[IPython-dev] ERROR can not use fork(), exec(), dup2() to communicate with child process

[Thomas Kluyver]

On 3 October 2014 11:22, Andy Davidson <Andy at santacruzintegration.com>
wrote:

> The subprocess  documentation suggests that you need to call communicate()
> and that this function will not return until the child sends EOF or
> terminates.


You don't actually *need* to use communicate(), it's just advised for most
cases to avoid deadlocks. If you need to e.g. read streaming data one line
at a time, you can use Popen.stdout.read(). Where you need to be careful is
if the process writes to both stdout and stderr, and you only read stdout,
it will fill up the stderr buffer and then block trying to write more
stderr, while you're waiting for output on stdout that never appears.
Similarly, if you're sending it data on stdin, it can get stuck waiting for
input while you're waiting for output. communicate() handles this for you,
but if necessary you can handle it yourself.

Depending on your process, you might want to pipe stderr to /dev/null to
discard it, or merge it with stdout so you read both at once.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20141003/2d1519ae/attachment.html>