[SciPy-user] Multithreading cookbook entry

[Anand Patil]

Bruce,

>  from numpy import ones, exp
>  import time
>
>  if __name__=='__main__':
>     def f(x):
>
>         y = ones(10000000)
>         exp(y)
>     t1=time.time()
>     foreach(f,range(100))
>     t2=time.time()
>     for ndx in range(100):
>
>         y = ones(10000000)
>         exp(y)
>     t3=time.time()
>     print 'Handythread / simple loop)=, (t3-t2)/(t2-t1)
>
>  With this code, the 'for loop' takes about 2.7 times as long as the
>  handythread loop for a quad-core system.

That's very interesting. I set the 'threads' option to 2, since I have
a dual-core system, and the handythread example is still only about
1.5x faster than the for-loop example, even though I can see that both
my cores are being fully utilized. That could be because my machine
devotes a good fraction of one of its cores to just being a Mac, but
it doesn't look like that's what is making the difference.


The strange thing is that for me the 'for-loop' version above takes
67s, whereas a version with f modified as follows:

def f(x):
     y = ones(10000000)
     # exp(y)

takes 13s whether I use handythread or a for-loop. I think that means
'ones' can only be executed by one thread at a time. Based on that, if
my machine had three free cores I would expect about a 2.16X speedup
tops, but you're seeing a 2.7X speedup.

That means our machines are doing something differently (yours is
better). Do you see any speedup from handythread with the modified
version of f?

Anand