[SciPy-user] conforming to Python GIL...

[Anne Archibald]

On 03/04/2008, Travis E. Oliphant <oliphant at enthought.com> wrote:
> Anne Archibald wrote:
>  > On 03/04/2008, fred <fredmfp at gmail.com> wrote:
>  >
>  >> Travis E. Oliphant a écrit :
>  >>
>  >>
>  >>  > This requires a bit of effort to solve.   We need to in multiple places...
>  >>
>  >> Hum, I do understand, but...
>  >>
>  >>  It's very hard to believe that I am the only guy in the entire universe
>  >>  who encounters this kind of issue, uh ? ;-)))
>  >>
>  >>  IOW... how do scipy users do ???
>  >>
>  >
>  > You're not; it's in the FAQ, and there's a page devoted to it:
>  > http://www.scipy.org/ParallelProgramming
>  >
>  > I am surprised by the *particular* problem you have, that is, that the
>  > GIL is not released *at all* for long periods by scipy; I think most
>  > large pieces of compiled code in numpy/scipy operate outside the GIL
>  > (and interpreted code releases the GIL frequently). This makes it not
>  > just a parallel processing issue.
>  >
>
> I'm surprised that f2py is not helping us release the GIL more often.
>  On the other-hand, releasing the GIL automatically is dangerous if the
>  code  that could possibly execute in the same thread is not re-entrant.
>
>  I had thought that f2py was releasing the GIL more often than it
>  actually is.   This is a problem and should be fixed -- perhaps by the
>  addition of a default semaphore for code that may not be re-entrant.

What are the ways in which FORTRAN code can fail to be reentrant? My
impression is that this is mostly the result of using
statically-allocated storage (which may be built into the code's
calling convention).

Can one safely assume (default to) that FORTRAN code does not
manipulate python interpreter internals, so that FORTRAN code without
special annotations could go under its own global semaphore (but not
under the GIL)?

Can one safely assume that FORTRAN code in separate compilation units
can't step on each other's toes? After all f2py knows rather a lot
about what is linked to what during compilation... And of course
letting a user easily create and use named locks would make it fairly
easy to specify which code needed to avoid being run concurrently.

Anne