[SciPy-user] Multithreading cookbook entry

[Lou Pecora]

I have no answer, but I want to add another question
to Ms. Archibald or anyone.  Will the GIL prevent in
any way the threading of C extensions?  That is, I
want to call a C extension in several threads and the
program will stay there doing a long calculation and
then return to Python when finished.  Perhaps this is
obvious, but I admit I don't fully understand the GIL.
 Thanks for any info.

Simple example:  

I want to evaluate a function using a C extension
(implemented with ctypes) for several parameters in
the function.  The parameters are in a list.  Then I
use the handythread.py approach and for each thread
call the C extension function with a new parameter
value from the list and, when the thread returns, I
add the result (say, a float number) to a result list.
 Will, the GIL let the threads run independently? I
hope my example is clear.

Thanks for any info.

--- Anand Patil <anand.prabhakar.patil at gmail.com>
wrote:

> Hi all,
> 
> I have a question primarily for Anne Archibald, the
> author of the
> cookbook entry on multithreading,
> http://www.scipy.org/Cookbook/Multithreading.
> 
> I tried replacing the 'if name=='__main__' clause in
> the attachment
> handythread.py with
> 
>     from numpy import ones, exp
>     def f(x):
>         print x
>         y = ones(10000000)
>         exp(y)
> 
> and the wall-clock time with foreach was 4.72s vs
> 6.68s for a simple for-loop.
> 
> First of all, that's amazing! I've been internally
> railing against the
> GIL for months. But it looks like only a portion of
> f is being done
> concurrently. In fact if I comment out the 'exp(y)',
> I don't see any
> speedup at all.
> 
> It makes sense that you can't malloc simultaneously
> from different
> threads... but if I replace 'ones' with 'empty', the
> time drops
> precipitously, indicating that most of the time
> taken by 'ones' is
> spent actually filling the array with ones. It seems
> like you should
> be able to do that concurrently.
> 
> So my question is, what kinds of numpy functions
> tend to release the
> GIL? Is there a system to it, so that one can figure
> out ahead of time
> where a speedup is likely, or do you have to try and
> see? Do
> third-party f2py functions with the 'threadsafe'
> option release the
> GIL?
> 
> Thanks,
> Anand
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
>
http://projects.scipy.org/mailman/listinfo/scipy-user
> 


-- Lou Pecora,   my views are my own.


      ____________________________________________________________________________________
Looking for last minute shopping deals?  
Find them fast with Yahoo! Search.  http://tools.search.yahoo.com/newsearch/category.php?category=shopping