[New-bugs-announce] [issue33115] Asyncio loop blocks with a lot of parallel tasks
Marat Sharafutdinov
report at bugs.python.org
Wed Mar 21 11:48:58 EDT 2018
New submission from Marat Sharafutdinov <decaz89 at gmail.com>:
I want to schedule a lot of parallel tasks, but it becomes slow with loop blocking:
```python
import asyncio
task_count = 10000
async def main():
for x in range(1, task_count + 1):
asyncio.ensure_future(f(x))
async def f(x):
if x % 1000 == 0 or x == task_count:
print(f'Run f({x})')
await asyncio.sleep(1)
loop.call_later(1, lambda: asyncio.ensure_future(f(x)))
loop = asyncio.get_event_loop()
loop.set_debug(True)
loop.run_until_complete(main())
loop.run_forever()
```
Outputs:
```
Executing <Task finished coro=<main() done, defined at test_aio.py:5> result=None created at /usr/lib/python3.6/asyncio/base_events.py:446> took 0.939 seconds
...
Executing <TimerHandle when=1841384.785427177 _set_result_unless_cancelled(<Future finis...events.py:275>, None) at /usr/lib/python3.6/asyncio/futures.py:339 created at /usr/lib/python3.6/asyncio/tasks.py:480> took 0.113 seconds
...
Executing <Task pending coro=<f() running at test_aio.py:12> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7fe89344fcd8>()] created at /usr/lib/python3.6/asyncio/base_events.py:275> created at test_aio.py:13> took 0.100 seconds
...
```
What can be another way to schedule a lot of parallel tasks?
----------
components: asyncio
messages: 314207
nosy: asvetlov, decaz, yselivanov
priority: normal
severity: normal
status: open
title: Asyncio loop blocks with a lot of parallel tasks
type: resource usage
versions: Python 3.6
_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33115>
_______________________________________
More information about the New-bugs-announce
mailing list