threading support in python

[sjdevnull at yahoo.com]

Sandra-24 wrote:
> The trouble is there are some environments where you are forced to use
> threads. Apache and mod_python are an example. You can't make use of
> mutliple CPUs unless you're on *nux and run with multiple processes AND
> you're application doesn't store large amounts of data in memory (which
> mine does) so you'd have to physically double the computer's memory for
> a daul-core, or quadruple it for a quadcore.

You seem to be confused about the nature of multiple-process
programming.

If you're on a modern Unix/Linux platform and you have static read-only
data, you can just read it in before forking and it'll be shared
between the processes..

If it's read/write data or you're not on a Unix platform, you can use
shared memory to shared it between many processes.

Threads are way overused in modern multiexecution programming.  The
decision on whether to use processes or threads should come down to
whether you want to share everything, or whether you have specific
pieces of data you want to share.  With processes + shm, you can gain
the security of protected memory for the majority of your code + data,
only sacrificing it where you need to share the data.

The entire Windows programming world tends to be so biased toward
multithreading that they often don't even acknowledge the existence of
generally superior alternatives.  I think that's in large part because
historically on Windows 3.1/95/98 there was no good way to create
processes without running a new binary, and so a culture of threading
grew up.  Even today many Windows programmers are unfamiliar with using
CreateProcessEx with SectionHandle=NULL for efficient copy-on-write
process creation.

> And forget about running a
> windows server, apache will not even run with multiple processes.

It used to run on windows with multiple processes.  If it really won't
now, use an older version or contribute a fix.

Now, the GIL is independent of this; if you really need threading in
your situation (you share almost everything and have hugely complex
data structures that are difficult to maintain in shm) then you're
still going to run into GIL serialization.  If you're doing a lot of
work in native code extensions this may not actually be a big
performance hit, if not it can be pretty bad.