Parallelization on muli-CPU hardware?

[exarkun at divmod.com]

On Tue, 12 Oct 2004 11:07:57 +0200, "Nicolas Lehuen" <nicolas.lehuen at thecrmcompany.com> wrote:
>
> "Neil Hodgson" <nhodgson at bigpond.net.au> a =E9crit dans le message de =
> news:1pDad.22529$5O5.5713 at news-server.bigpond.net.au...
> > Luis P Caamano:
> >=20
> > > Therefore, fixing the GIL was always in competition with existing =
> code
> > > and libraries, and it always lost because it was (and still is)
> > > considered "not worth the effort."
> >=20
> >    Greg Stein implemented a "free-threaded" version of Python in 1996 =
> and
> > had another look in 2000.
> >=20
> > http://python.tpnet.pl/contrib-09-Dec-1999/System/threading.README
> > http://mail.python.org/pipermail/python-dev/2000-April/003605.html
> >=20
> >     So, it is not the initial implementation of a GIL-free Python that =
> is
> > the stumbling block but the maintenance of the code. The poor =
> performance of
> > free-threaded Python contributed to the lack of interest.
> 
> Yeah, and the poor performance of the ActiveState Python for .NET =
> implementation led them to tell that the CLR could not be efficient in =
> implementing Python. Turns out that the problem was in the =
> implementation, not the CLR, and that IronPython gives pretty good =
> performances.
> 
> It's not because an implementation gives poor performance that the whole =
> free-threaded concept is bad.
> 
> Plus, the benchmarks is highly questionable. Compare a single threaded =
> CPU intensive program with a multi-threaded CPU intensive program, the =
> single threaded program wins. Now compare IO intensive (especially =
> network IO) programs, the multi-threaded program wins. Granted, you can =
> built high performing asynchronous frameworks (Twisted or medusa come to =
> mind), but sooner or later you'll have to use threads (see the DB =
> adapter for Twisted).

  Uh, no.  Threads are useful for supporting libraries with blocking APIs.  Nothing about database access requires threads.  Twisted uses threads DB-API 2.0 specifies a blocking API, and it is useful to support DB-API 2.0 modules.

  Several SQL modules provide an asynchronous API which can be used without threads.

  When it comes to network-bound tasks, single-threaded wins the performance contest.

  Jp