[issue14119] Ability to adjust queue size in Executors

[Victor Varvariuc]

Victor Varvariuc added the comment:

Hi Brian,

In one my projects I had to monkey-patch module `concurrent.futures.thread:60`( http://hg.python.org/cpython/file/37caaf21f827/Lib/concurrent/futures/thread.py#l60) with:

def _worker(executor_reference, work_queue):
    try:
        while True:
            work_item = work_queue.get(block=True)
            if work_item is not None:
                work_item.run()
                work_queue.task_done()  # <-- added this line
                continue
            executor = executor_reference()
            # Exit if:
            #   - The interpreter is shutting down OR
            #   - The executor that owns the worker has been collected OR
            #   - The executor that owns the worker has been shutdown.
            if futures_thread._shutdown or executor is None or executor._shutdown:
                # Notice other workers
                work_queue.put(None)
                return
            del executor
    except BaseException:
        futures_thread._base.LOGGER.critical('Exception in worker', exc_info=True)

This helps me to control the state of the work queue -- I can see if there are any work items still being processed:

if executor._work_queue.unfinished_tasks:
    # executor is still producing something
    ...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue14119>
_______________________________________