[SciPy-user] Multithreading cookbook entry

[Anand Patil]

Hi all,

I have a question primarily for Anne Archibald, the author of the
cookbook entry on multithreading,
http://www.scipy.org/Cookbook/Multithreading.

I tried replacing the 'if name=='__main__' clause in the attachment
handythread.py with

    from numpy import ones, exp
    def f(x):
        print x
        y = ones(10000000)
        exp(y)

and the wall-clock time with foreach was 4.72s vs 6.68s for a simple for-loop.

First of all, that's amazing! I've been internally railing against the
GIL for months. But it looks like only a portion of f is being done
concurrently. In fact if I comment out the 'exp(y)', I don't see any
speedup at all.

It makes sense that you can't malloc simultaneously from different
threads... but if I replace 'ones' with 'empty', the time drops
precipitously, indicating that most of the time taken by 'ones' is
spent actually filling the array with ones. It seems like you should
be able to do that concurrently.

So my question is, what kinds of numpy functions tend to release the
GIL? Is there a system to it, so that one can figure out ahead of time
where a speedup is likely, or do you have to try and see? Do
third-party f2py functions with the 'threadsafe' option release the
GIL?

Thanks,
Anand