Solving the problem of mutual recursion

[Ian Kelly]

On Sun, May 26, 2013 at 10:36 PM, Peter Brooks
<peter.h.m.brooks at gmail.com> wrote:
> This makes complete sense - any atomic action should be atomic, so two
> threads can't be doing it at the same time. They can be doing anything
> else though.
>
> If two threads create a new object at the same time, for example,
> there's potentially the problem that they'll get the same space, so
> the object will be owned by both. To prevent this, the new object call
> should be run in only one thread.
>
> If you have two objects running their methods on their own local
> variables, then there's no potential for conflict, so there's no need
> for them to be blocked.

That's not the way it works.  The CPython interpreter always runs with
the GIL held; the alternative would be to have individual mutex locks
on every Python object, which is expensive for performance due to the
reference counting mechanism.  Python functions can't release the GIL.
 C functions that are called from the interpreter *can* release the
GIL to allow concurrency, but are only permitted to do so as long as
they're not working with Python objects, e.g. waiting on I/O or
performing a long calculation on C data.

There are some more detailed slides on how the GIL works at:

http://www.dabeaz.com/python/UnderstandingGIL.pdf

Note that the description in Part 1 describes how the GIL worked prior
to Python 3.2.  The new GIL is described in Part 4, but the basic
underlying concept is the same.

> This is an interesting subject.. There's nothing wrong with the tool
> I'm using to report threads - 'Activity Monitor' is the standard
> process monitor. It counts cores as 'CPUs', which seems perfectly
> reasonable to me. As I said, other Unixes, such as HP-UX, do the same
> thing.

I have no problem with that.  It's also the default in Linux, where I
believe it is called "IRIX mode" (as opposed to "Solaris mode", where
the difference is just a factor equal to the number of cores).  What I
was questioning was whether the actual number being reported was
correct.  If it's the standard tool for the OS, then it probably is.