[issue1260171] subprocess: more general (non-buffering) communication

[Martin Panter]

Martin Panter added the comment:

The non-blocking read and write features proposed in Issue 1191964 wouldn’t be sufficient. Perhaps they might be useful for implementing this feature on Windows though; I’m not sure. And a non-blocking write on Posix may be slightly more efficient as well, removing the limitation of copying in chunks of PIPE_BUF bytes.

Ian’s original patch posted here proposes a version of communicate() which accepts a file reader for the subprocess’s input, and file writers for the subprocess’s outputs. Another option could be something like my SubprocessWriter class <https://github.com/vadmium/pacman-tools/blob/9ffdd88/makeaur#L179>. It provides a file writer interface for the caller to write to the subprocess’s input, while copying data from the subprocess’s output behind the scenes. I used it to stream a tar file, as it is written by the “tarfile” module, into a GPG subprocess to create a digital signature, while writing the output of GPG into a BytesIO buffer. Using Unix command pipeline pseudocode, it is used a bit like this:

tarfile.open(mode="w|") | Popen("gpg") | BytesIO()

----------
nosy: +vadmium

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue1260171>
_______________________________________