Adding a Par construct to Python?

[Steven D'Aprano]

On Sun, 17 May 2009 17:19:15 +0100, MRAB wrote:

>> But reduce()? I can't see how you can parallelize reduce(). By its
>> nature, it has to run sequentially: it can't operate on the nth item
>> until it is operated on the (n-1)th item.
>> 
> It can calculate the items in parallel, 

I don't understand what calculation you are talking about. Let's take a 
simple example:

reduce(operator.sub, [100, 50, 25, 5])  => 100-50-25-5 = 20

What calculations do you expect to do in parallel?


> but the final result must be
> calculated sequence, although if the final operation is commutative then
> some of them could be done in parallel.

But reduce() can't tell whether the function being applied is commutative 
or not. I suppose it could special-case a handful of special cases (e.g. 
operator.add for int arguments -- but not floats!) or take a caller-
supplied argument that tells it whether the function is commutative or 
not. But in general, you can't assume the function being applied is 
commutative or associative, so unless you're willing to accept undefined 
behaviour, I don't see any practical way of parallelizing reduce().


-- 
Steven