[issue24882] ThreadPoolExecutor doesn't reuse threads until #threads == max_workers

[Thomas]

Thomas <bugs.python at zulan.net> added the comment:

We ran into this issue in the context of asyncio which uses an internal ThreadPoolExecutor to provide an asynchronous getaddrinfo / getnameinfo.

We observed an async application spawned more and more threads through several reconnects. With a maximum of 5 x CPUs these were dozens of threads which easily looked like a resource leak.

At least in this scenario I would strongly prefer to correctly reuse idle threads. 

Spawning all possible threads on initialization in such a transparent case would be quite bad. Imagine having a process-parallel daemon that running a apparently single-threaded asyncio loop but then getting these executors for doing a single asyncio.getaddrinfo. Now you run 80 instances on an 80 core machine you get 32.000 extra implicit threads.

Now you can argue whether the default executor in asyncio is good as is, but if the executors properly reuse threads, it would be quite unlikely to be a practical problem.

----------
nosy: +tilsche

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue24882>
_______________________________________