Help to debug my free library

[Chris Angelico]

On Thu, Feb 1, 2018 at 6:26 AM, Victor Porton <porton at narod.ru> wrote:
> Chris Angelico wrote:
>
>> On Thu, Feb 1, 2018 at 5:58 AM, Victor Porton <porton at narod.ru> wrote:
>>> LibComCom is a C library which passes a string as stdin of an OS command
>>> and stores its stdout in another string.
>>
>> Something like the built-in subprocess module does?
>
> I was going to write: "It seems that subprocess module can cause deadlocks.
> For example, if it first writes a long string to "cat" command input (going
> to read cat's stdout later), then "cat" would block because its output is
> not read while writing input."
>
> But after reading the docs it seems that Popen.communicate() does the job.
>
> Well, I first wrote in Java. For Java there was no ready solution. Later I
> switched to Python and haven't checked the standard libraries.
>
> So, please help me to make sure if Popen.communicate() is a solution for my
> problem (namely that it does not deadlock, as in "cat" example above).
>
> I am interested in both Python 2.7 and 3.x.

I believe communicate() is indeed safe for this. It does retain all
data in memory, so with arbitrarily large output it may be better to
use a pipe and read from it progressively (similarly if the program's
going to run for a long time; communicate() will wait for the process
to terminate before giving you any output), but it won't just
deadlock.

Even if what you want can't be done with communicate(), I would
definitely recommend looking at the subprocess module. Worst case, you
do in Python what your C library is doing: create pipes to communicate
with the process. It's a lot easier in Python than in C, but the logic
would be similar.

Bonus: subprocess is available on all supported platforms (most
notably Windows and POSIX, which are drastically different; there are
a handful of Windows-only or Unix-only features, but the core is all
identical, despite the implementation differing), and you can still
support 2.7, although you have to forego the new and improved API in
Python 3.5+.

ChrisA