[Python-Dev] Fixing the GIL (with a BFS scheduler)

[Antoine Pitrou]

Le lundi 17 mai 2010 à 09:05 -0700, Bill Janssen a écrit :
> Antoine Pitrou <solipsis at pitrou.net> wrote:
> 
> > On Sun, 16 May 2010 15:13:44 PDT
> > Bill Janssen <janssen at parc.com> wrote:
> > > 
> > > So the patch to the threading code would presumably, for those OSs where
> > > the capability exists, try to put all created threads in the same
> > > affinity set.
> > 
> > This is not really a good idea.
> 
> Yes, fixing the GIL for multicore in 2.7 would be a better idea, IMO.
> But that might be too large a change.

Well, 2.7rc1 is scheduled in less than three weeks now. IMO any patch
changing fundamental threading properties is a no-no (even the processor
affinity proposal).

Someone had tried to backport the new GIL to 2.7 (there's a tracker
issue for it, I'm too lazy to search right now), but it was concluded
that compatibility requirements for Python 2.x (compatibility with
various legacy threading libraries) made things too complicated and
tedious.

> > There's some code which releases the GIL, precisely so that you can
> > run several threads (computations) at once.
> 
> If they can get hold of the GIL in the first place!  Yes, you'd want to
> be able to "unbind" threads if you knew what you were doing, so that
> they could run on other cores, and you'd want a switch to disable the
> affinity mechanism entirely.  But I'd be happy to have things in the
> naive case run as well as they do on single-core machines, and let
> experts do optimizations.

"Letting experts do optimizations" is a regression, though, because
right now you don't have to be an expert to take advantage of such
optimizations (just run a separate thread doing e.g. some zlib
compression).

Regards

Antoine.