Real-world use of concurrent.futures

[Marko Rauhamaa]

Andrew McLean <lists at andros.org.uk>:

> That's fine, but I suspect that isn't a helpful pattern if I have a
> very large number of tasks. In my case I could run out of memory if I
> tried submitting all of the tasks to the executor before processing
> any of the results.

This is related to flow control. You'll need an object for each flow
(transaction). When new work comes in from the network, you'll have to
see if you are hitting the maximum number of pending transactions, and
not start another one before previous transactions have been processed.

Whenever a transaction is completed, you pull in more work.


Marko