concurrent.futures vs gevent

jspicklemire at gmail.com jspicklemire at gmail.com
Mon Aug 24 12:08:51 EDT 2015


Some people have tried to make a case that concurrent.futures should be adopted as a replacement wherever greenlet based algorithms are in use. However, my experience is that greenelts combined with concurrent.futures provides significant advantages. In other words, to a degree the two approaches complement each other. 

Specifically, 'greening' a ThreadPoolExecutor delivers the simplicity of either a context manager, or a map, depending on the use case, hadling in a few lines of simple Python what might have otherwise been a complex call-back or 'twisted' solution. 

Please note, this usage applies only to ThreadPool, not ProcessPool.

for example:

import eventlet
eventlet.patcher.monkey_patch(os=False, socket=True, select=True, thread=True)

futures = eventlet.import_patched('concurrent.futures') # 'greening' futures,
fut_tp_exec = futures.ThreadPoolExecutor

# the lines above enable even nested map / context manager 'futures'
# adding the lines below enables a vast number of 'simultaneous' HTTP requests.

import requests
req_f_sess = eventlet.import_patched('requests_futures.sessions') # 'green' req
o_rest_sess = req_f_sess.FuturesSession(executor=fut_tp_exec(max_workers=5000))



More information about the Python-list mailing list