[SciPy-user] Multithreading cookbook entry

[Lou Pecora]

--- Anne Archibald <peridot.faceted at gmail.com> wrote:



> Yep. Oops. Fixed in the v2 versions of the files.
> The wiki doesn't
> make a very good version control system. Is it worth
> incorporating
> those files into scipy?

If you mean put a new version of the example code up
to SciPy cookbook, then yes because bugs confuse
newbies like me.  :-) 

> Depends what your function does, really. If your
> function takes half a
> second but never releases the GIL, it should take
> twice as long. If
> your function releases the GIL, then it should take
> about the same
> time. But it's quite tricky to write a function that
> works hard and
> releases the GIL. A good rule of thumb is to count
> the lines of python
> that are getting executed. If there are only a few -
> say you're doing
> sum(log(exp(arange(1000000)))) - there's a good
> chance the GIL will be
> released. If you're running millions of python
> instructions, the GIL
> is held all that time, and you won't get a speedup.

Hmmm... gotta think about that.

> >  (3) You mention that ctypes probably doesn't
> release
> >  the GIL. I would guess that too, since it would
> be
> >  dangerous as I (vaguely) understand the GIL.  But
> does
> >  the GIL have to be released in the Cextension or
> can
> >  it be release in the step just before I call the
> C
> >  extension from Python? I.e. is release on the
> Python
> >  side possible?  If not, I guess I will have to
> look
> >  over the numpy code as you suggest.  If possible,
> I
> >  suppose the GIL must be enabled immediately on
> return
> >  from the C extension.
> 
> You can't execute any python bytecodes without
> holding the GIL, so
> it's impossible for python code to release the GIL.
> But it would be
> perfectly possible, in principle, for SWIG, F2PY, or
> ctypes to put a
> "release the GIL" in their wrappers. This will be a
> problem for some
> functions - either ones that aren't reentrant, or
> ones that call back
> to python (though in principle it might be possible
> to reacquire the
> GIL for the duration of a callback). But for a
> typical C function that
> acts only on data you give it and that doesn't know
> anything about
> python, it should be safe to run it without the GIL
> engaged. It seems
> like f2py can actually do this for functions marked
> as threadsafe; I
> don't know about ctypes or SWIG.

Sounds like it's better to call those macros:

Py_BEGIN_ALLOW_THREADS
and
Py_END_ALLOW_THREADS

on the C side.  Is that all that's needed?  Then will
code like your handythread.py work with threads if f
calls a C extension that uses those macros?  Or is
there more that needs to be done to set this up?



-- Lou Pecora,   my views are my own.


      ____________________________________________________________________________________
Never miss a thing.  Make Yahoo your home page. 
http://www.yahoo.com/r/hs