Status of Python threading support (GIL removal)?

[skip at pobox.com]

    Carl> I'm sure you think you're trying to be helpful, but you're coming
    Carl> off as really presumptuous with this casual dismissal of their
    Carl> concerns.

My apologies, but in most cases there is more than one way to skin a cat.

Trust me, if removing the global interpreter lock was easy, or probably even
it was simply hard, it almost certainly would have been done by now.
Continuing to harp on this particular aspect of the CPython implementation
doesn't help.

    Carl> I guess what really bothers me about this is how easily people
    Carl> throw out "shut up and use C" for some things, especially things
    Carl> that quite reasonably appear to be a silly limitation.

You completely misunderstand I think.  People don't throw out "shut up and
use C" out of ignorance.  In fact, I don't believe I've ever read a response
which took that tone.  The practical matter is that there has so far been no
acceptable patch to CPython which gets rid of the global interpreter lock.
Extremely smart people have tried.  More than once.  If Guido knew then (20
years ago) what he knows now:

    * that the chip manufacturers would have run out of clock speed
      improvements for a few years and resorted to multi-core CPUs as a way
      to make their computers "faster"

    * that garbage collection algorithms would have improved as much as they
      have in the past twenty years

I suspect he might well have considered garbage collection instead of
reference counting as a way to reclaim unreferenced memory and we might have
a GIL-less CPython implementation today.

    Carl> Maybe you don't intend to sound like you're saying "shut up and
    Carl> use C", but to me, that's how you come off.  If you're going to
    Carl> advise someone to use C, at least try to show some understanding
    Carl> for their concerns--it would go a long way.

Then you haven't been listening.  This topic comes up over and over and over
again.  It's a well-known limitation of the implementation.  Poking people
in the eye with it over and over doesn't help.  The reasons for the
limitation are explained every time the topic is raised.  In the absence of
a GIL-less CPython interpreter you are simply going to have to look
elsewhere for performance improvements I'm afraid.  Yes, I'll drag out the
same old saws:

    * code hot spots in C or C++

    * use tools like Pyrex, Cython, Psyco or Shed Skin

    * for array procesing, use numpy, preferably on top of a recent enough
      version of Atlas which does transparent multi-threading under the
      covers

    * use multiple processes

    * rewrite your code to use more efficient algorithms

I don't write those out of ignorance for your plight.  It's just that if you
want a faster Python program today you're going to have to look elsewhere
for your speedups.

-- 
Skip Montanaro - skip at pobox.com - http://www.smontanaro.net/
    when i wake up with a heart rate below 40, i head right for the espresso
    machine. -- chaos @ forums.usms.org