[New-bugs-announce] [issue46335] asyncio.create_subprocess_exec throws RuntimeError yet still executes subprogram

[Clint Olsen]

New submission from Clint Olsen <clint.olsen at gmail.com>:

When stress testing my code in a process-limited environment I found that despite throwing an exception, it appears the process still executes. Attempting to catch/retry results in a duplicate.

Attached is a script that I was able to repro the problem on Linux. I cannot get it to behave similarly on MacOS.

The exception looks like:

/bin/sh: fork: retry: Resource temporarily unavailable
Traceback (most recent call last):
  File "/home/colsen/async/./example", line 16, in run
    proc = await asyncio.create_subprocess_exec('/bin/sh','-c', f'/bin/echo {_id} > {_uuid}.out')
  File "/home/utils/Python/3.9/3.9.7-20211101/lib/python3.9/asyncio/subprocess.py", line 236, in create_subprocess_exec
    transport, protocol = await loop.subprocess_exec(
  File "/home/utils/Python/3.9/3.9.7-20211101/lib/python3.9/asyncio/base_events.py", line 1661, in subprocess_exec
    transport = await self._make_subprocess_transport(
  File "/home/utils/Python/3.9/3.9.7-20211101/lib/python3.9/asyncio/unix_events.py", line 202, in _make_subprocess_transport
    watcher.add_child_handler(transp.get_pid(),
  File "/home/utils/Python/3.9/3.9.7-20211101/lib/python3.9/asyncio/unix_events.py", line 1381, in add_child_handler
    thread.start()
  File "/home/utils/Python/3.9/3.9.7-20211101/lib/python3.9/threading.py", line 892, in start
    _start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread

So, this script ended up producing 21 output files (duplicate on iteration 18).

I need a way to catch these errors, pause, and retry when they are recoverable.

----------
components: asyncio
files: example
messages: 410242
nosy: Clint Olsen, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.create_subprocess_exec throws RuntimeError yet still executes subprogram
versions: Python 3.9
Added file: https://bugs.python.org/file50553/example

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue46335>
_______________________________________