exit ThreadPoolExecutor immediately

[dieter]

Atul Johri <al.johri at gmail.com> writes:
> I am looking for a way to stop a ThreadPoolExecutor immediately under the
> assumption that I don't care about what's currently running or pending.
>
> ```
> limit = 2
> executor = ThreadPoolExecutor(10)
> posts = itertools.islice(mygen(executor=executor, **kwargs), 0, limit)
> for post in posts:
>   print(post)
> executor.shutdown(wait=False)
> ```
>
> Basically I have a generator, mygen, which is using the executor to submit
> many tasks in parallel and yield the result one at a time. I would like to
> be able to limit the generator and have the executor stop processing
> immediately.
>
> I was considering clearing the _work_queue or iterating over it and running
> future.cancel() on each future but both seem to be quite hacky and didn't
> work in my initial try.
>
> https://github.com/python/cpython/blob/master/Lib/concurrent/futures/thread.py#L83
>
> Any ideas?

I would implement my own class (inheriting from "ThreadPoolExecutor")
and give it appropriate API to realise what I need (using whatever
internal implementation details are necessary).

I should not be that difficult for tasks not yet started (i.e.
not yet run be a thread).
In earlier Python versions, it was not
possible from Python level to abort a thread; there was a C level
Python API function to abort a thread once it started to execute Python
code again. Thus, there was not a complete solution for your problem.
This might have changed with modern Python version, but I doubt it
(the main reason for the restriction has been the difficulty
to cleanly abort a thread in a platform independent way -- this
difficulty should remain, whatever the Python version).