[New-bugs-announce] [issue44962] asyncio.create_task weakrefset race condition

[Thomas Grainger]

New submission from Thomas Grainger <tagrain at gmail.com>:

with the following demo script I can get a IndexError: pop from empty list

```
import itertools
import asyncio
import concurrent.futures
import sys
import threading

threads = 200

def test_all_tasks_threading() -> None:
    async def foo() -> None:
        await asyncio.sleep(0)

    async def create_tasks() -> None:
        for i in range(1000):
            asyncio.create_task(foo())

        await asyncio.sleep(0)

    results = []
    with concurrent.futures.ThreadPoolExecutor(threads) as tpe:
        for f in concurrent.futures.as_completed(
            tpe.submit(asyncio.run, create_tasks()) for i in range(threads)
        ):
            results.append(f.result())
    assert results == [None] * threads


def main():
    for i in itertools.count():
        test_all_tasks_threading()
        print(f"worked {i}")
    return 0


if __name__ == "__main__":
    sys.exit(main())
```



```
worked 0
worked 1
worked 2
worked 3
worked 4
worked 5
worked 6
worked 7
worked 8
worked 9
worked 10
worked 11
worked 12
worked 13
worked 14
worked 15
worked 16
worked 17
worked 18
Traceback (most recent call last):
  File "/home/graingert/projects/asyncio-demo/demo.py", line 36, in <module>
    sys.exit(main())
  File "/home/graingert/projects/asyncio-demo/demo.py", line 30, in main
    test_all_tasks_threading()
  File "/home/graingert/projects/asyncio-demo/demo.py", line 24, in test_all_tasks_threading
    results.append(f.result())
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 438, in result
    return self.__get_result()
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 390, in __get_result
    raise self._exception
  File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python3.9/asyncio/runners.py", line 48, in run
    loop.run_until_complete(loop.shutdown_asyncgens())
  File "/usr/lib/python3.9/asyncio/base_events.py", line 621, in run_until_complete
    future = tasks.ensure_future(future, loop=self)
  File "/usr/lib/python3.9/asyncio/tasks.py", line 667, in ensure_future
    task = loop.create_task(coro_or_future)
  File "/usr/lib/python3.9/asyncio/base_events.py", line 433, in create_task
    task = tasks.Task(coro, loop=self, name=name)
  File "/usr/lib/python3.9/_weakrefset.py", line 84, in add
    self._commit_removals()
  File "/usr/lib/python3.9/_weakrefset.py", line 57, in _commit_removals
    discard(l.pop())
IndexError: pop from empty list
sys:1: RuntimeWarning: coroutine 'BaseEventLoop.shutdown_asyncgens' was never awaited
Task was destroyed but it is pending!
task: <Task pending name='Task-74178' coro=<BaseEventLoop.shutdown_asyncgens() running at /usr/lib/python3.9/asyncio/base_events.py:530>>
```

here's a live demo on github actions: https://github.com/graingert/asyncio-backport/runs/3380502247#step:5:90

----------
components: asyncio
messages: 399969
nosy: asvetlov, graingert, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.create_task weakrefset race condition
versions: Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue44962>
_______________________________________