[issue22114] You cannot call communicate() safely after receiving an exception (EINTR or EAGAIN)

[Amrith Kumar]

Amrith Kumar added the comment:

After some debugging and reading code in python's subprocess.py (v2.7), here's what I'm seeing.

(a) the error has nothing to do with eventlet and monkey-patching.
(b) the code in _communicate_with_select() and potentially _communicate_with_poll() are the problem.

What's the problem?
-------------------

The code in _communicate_with_select() correctly sets up to handle the read and write calls without blocking on any of them. It does this by establishing the two (read and write) lists of descriptors and calls select without no timeout specified. 

When select returns, and indicates that a socket is in (for example) the read list, what that means is that an attempt to read() will not block. However, it is possible on some systems (Linux for sure) that 

(a) a socket is non-blocking, and
(b) a call to select indicates that the socket is ready to read, and
(c) a call to read the socket returns an error EAGAIN (aka EWOULDBLOCK).

Callers of read() and write() on non-blocking sockets should be prepared to handle this situation. 

The python code in _communicate_with_select() is not.

It assumes that if select() returns that a read fd is ready for read, that a call to read it will produce 0 or more bytes. The calls to read() for stdout and stderr are not guarded with exception handlers. However, if a socket is setup as non-blocking, any call can produce EWOULDBLOCK, EAGAIN, ...

Adding some debugging code it was possible to recreate the problem and show that the backtrace was (in this case it happened with python 2.6)

Traceback (most recent call last):
[...]
  File "trove/openstack/common/processutils.py", line 186, in execute
    result = obj.communicate()
  File "/usr/lib64/python2.6/subprocess.py", line 732, in communicate
    stdout, stderr = self._communicate(input, endtime)
  File "/usr/lib64/python2.6/subprocess.py", line 1318, in _communicate
    stdout, stderr = self._communicate_with_select(input, endtime)
  File "/usr/lib64/python2.6/subprocess.py", line 1483, in _communicate_with_select
    data = os.read(self.stdout.fileno(), 1024)
OSError: [Errno 11] Resource temporarily unavailable

The correct fix for this is to make _communicate_with_select() and maybe _communicate_with_poll() properly handle the read() and write() calls and be better prepared to handle a thrown condition of EAGAIN from os.read or os.write.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22114>
_______________________________________