asyncio and subprocesses

[David]

Is this a bug in the asyncio libraries?



This code:

'''
  proc = yield from asyncio.create_subprocess_exec(*cmd,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)

  # read all data from subprocess pipe, copy to nextCoro
  ln = yield from proc.stdout.read(1024)
  while ln:
      yield from nextCoro.process(ln)
      ln = yield from proc.stdout.read(1024)

'''

will throw this exception:

Traceback (most recent call last):
File "/usr/project/bulk_aio.py", line 52, in db_source
ln = yield from proc.stdout.read(1024)
File "/usr/lib/python3.4/asyncio/streams.py", line 462, in read
self._maybe_resume_transport()
File "/usr/lib/python3.4/asyncio/streams.py", line 349, in _maybe_resume_transport  self._transport.resume_reading()
File "/usr/lib/python3.4/asyncio/unix_events.py", line 364, in resume_reading  self._loop.add_reader(self._fileno, self._read_ready)
AttributeError: 'NoneType' object has no attribute 'add_reader'


The exception always happens at the end of the subprocess's run, in what would be the last read. Whether it happens correlates with the time needed for nextCoro.process.  If each iteration takes more than about a millisecond, the exception will be thrown.

It *seems* that when the transport loses the pipe connection, it schedules the event loop for removal immediately, and the _loop gets set to None before the data can all be read.  This is the sequence in unix_events.py _UnixReadPipeTransport._read_ready.

Is there a workaround to avoid this exception?  Is this a fixed bug, already?  I am using Python 3.4.2 as distributed in Ubuntu Lucid, with built-in asyncio.

Thank you.
David