[stdlib-sig] futures - a new package for asynchronous execution

[Brian Quinlan]

On 17 Jan 2010, at 01:44, Anh Hai Trinh wrote:

>> 2. Do several different operations on different data e.g.  
>> parallelizing code
>> like this:
>>
>> db = setup_database(host, port)
>> data = parse_big_xml_file(request.body)
>> save_data_in_db(data, db)
>>
>> I'm trying to get a handle on how streams accommodates the second  
>> case. With
>> futures, I would write something like this:
>>
>> db_future = executor.submit(setup_database, host, port)
>> data_future = executor.submit(parse_big_xml_file, data)
>> # Maybe do something else here.
>> wait(
>>    [db_future, data_future],
>>    timeout=10,
>>    # If either function raises then we can't complete the operation  
>> so
>>    # there is no reason to make the user wait.
>>    return_when=FIRST_EXCEPTION)
>>
>> db = db_future.result(timeout=0)
>> data = data.result(timeout=0)
>> save_data_in_db(data, db)
>
> For this kind of scenario, I feel `futures` and friends are not
> needed. My solution is to explicit use different threads for different
> operations then use join() thread to wait for a particular operation.
> Threading concurrency means memory is shared and thread.join() can be
> used to synchronize events.

It is definitely true that you can roll your own implementation using  
threads but the purpose of the futures library is to make that  
unnecessary.

> Generally, I would be doubtful about any library that support
> parallelization of code that "do several different operations on
> different data". One could have put it as "write concurrent programs",
> to which the answer must be a complete concurrency model: threading,
> multiprocessing, Erlang, Goroutines and CSP channels, etc.

I don't understand your doubts. To me the example that I gave is  
simple and useful.

Cheers,
Brian