[New-bugs-announce] [issue35754] When writing/closing a closed Popen.stdin, I get OSError vs. BrokenPipeError randomly or depending on bufsize

[jimbo1qaz_ via Gmail]

New submission from jimbo1qaz_ via Gmail <jimbo1qaz at gmail.com>:

Windows 10 1709 x64, Python 3.7.1.

Minimal example and stack traces at https://gist.github.com/jimbo1qaz/75d7a40cac307f8239ce011fd90c86bf

Essentially I create a subprocess.Popen, using a process (msys2 head.exe) which closes its stdin after some amount of input, then write nothing but b"\n"*1000 bytes to its stdin.

If the bufsize is small (1000 bytes), I always get OSError: [Errno 22] Invalid argument

If the bufsize is large (1 million bytes), I always get BrokenPipeError: [Errno 32] Broken pipe. (This happens whether I write 1 million newlines or 1000 at a time).

Originally I created a ffmpeg->ffplay pipeline with a massive bufsize (around 1280*720*3 * 2 frames), then wrote 1280*720*3 bytes of video frames at a time. Closing ffplay's window usually created BrokenPipeError, but occasionally OSError. This was actually random.


------------

It seems that this is known to some extent, although I couldn't find any relevant issues on the bug tracker, and "having to catch 2 separate errors" isn't explained on the documentation. (Is it intended though undocumented behavior?)

Popen._communicate() calls Popen._stdin_write(), but specifically ignores BrokenPipeError and OSError where exc.errno == errno.EINVAL == 22 (the 2 cases I encountered).

But I don't call Popen.communicate() but instead write directly to stdin, since I have a loop that outputs 1 video frame at a time, and rely on pipe blocking to stop my application from running too far ahead of ffmpeg/ffplay.

------------

popen.stdin is a <_io.BufferedWriter name=3>.

https://docs.python.org/3/library/io.html#io.BufferedIOBase.write

>Write the given bytes-like object, b, and return the number of bytes written (always equal to the length of b in bytes, since if the write fails an OSError will be raised). Depending on the actual implementation, these bytes may be readily written to the underlying stream, or held in a buffer for performance and latency reasons.

The page doesn't mention BrokenPipeError at all (Ctrl+F). So why do I *sometimes* get a BrokenPipeError (subclasses ConnectionError subclasses OSError) instead?

----------
messages: 333792
nosy: jimbo1qaz_
priority: normal
severity: normal
status: open
title: When writing/closing a closed Popen.stdin, I get OSError vs. BrokenPipeError randomly or depending on bufsize
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35754>
_______________________________________