[SciPy-User] Speeding things up - how to use more than one computer core

[Gael Varoquaux]

On Sun, Apr 07, 2013 at 12:17:59AM +0200, Troels Emtekær Linnet wrote:
> Method was joblib delayed
> Done :0:00:00

Hum, this is fishy, isn't it?

>         elif method=='joblib delayed':
>             Parallel(n_jobs=-2) #Can also use '-1' for all cores, '-2' for all
> cores=-1
>             func,res = delayed(getsqrt), a

I have a hard time reading your code, but it seems to me that you haven't
computed anything here, just instanciated to Parallel object.

You need to do:

    res = Parallel(n_jobs=-2)(delayed(getsqrt)(i) for i in a)

I would expect joblib to be on the same order of magnitude speed-wise as
multiprocessing (hell, it's just a wrapper on multiprocessing). It's just
going to be more robust code than instanciating manually a Pool (deal
better with error, and optionally dispatching on-demand computation).

Gaël