Python child process in while True loop blocks parent

[Cameron Simpson]

On 08Dec2021 18:11, Jen Kris <jenkris at tutanota.com> wrote:
>Python must terminate its write strings with \n, or read will block in 
>C waiting for something that never comes.

There are two aspects to this:
- if upstream is rding "lines of text" then you need a newline to 
  terminate the lines
- you (probably) should flush the output pipe (Python to C) after the 
  newline

I see you're using file descriptors and os.write() to sent data. This is 
unbuffered, so there is nothing to flush, so you have not encountered 
the second point above.

But if you shift to using a Python file object for your output (eg 
f=os.fdopen(pw)), which would let you use print() or any number of other 
things which do things with Python files) your file object would have a 
buffer and normally that would not be sent to the pipe unless it was 
full.

So your deadlock issue has 2 components:
- you need to terminate your records for upstream (C) to see complete 
  records. Your records are lines, so you need a newline character.
- you need to ensure the whole record has been sent upstream (been 
  written to the pipe). If you use a buffered Python file object for 
  your output, you need to flush it at your synchronisation points or 
  upstream will not receive the buffer contents. That synchronisation 
  point for you is the end of the record.

Hopefully this makes the flow considerations more clear.

Cheers,
Cameron Simpson <cs at cskk.id.au>