Parallelization on muli-CPU hardware?

[Nicolas Lehuen]

"Alex Martelli" <aleaxit at yahoo.com> a écrit dans le message de news:1gljja1.1nxj82c1a25c1bN%aleaxit at yahoo.com...
> Nicolas Lehuen <nicolas at lehuen.com> wrote:
> 
> > problem was the same ; people expected mod_python to run in a
> > multi-process context, not a multi-threaded context (I guess this is
> > due to a Linux-centered mindset, forgetting about BSD, MacosX or Win32
> > OSes). When I asked questions and pointed problem, the answer was 'Duh
> > - use Linux with the forking MPM, not Windows with the threading MPM'.
> 
> Sorry, I don't get your point.  Sure, Windows makes process creation
> hideously expensive and has no forking.  But all kinds of BSD, including
> MacOSX, are just great at forking.  Why is a preference for multiple
> processes over threads "forgetting about BSD, MacOSX", or any other
> flavour of Unix for that matter?

Because when you have multithreaded programs, you can easily share objects between different threads, provided you carefully implement them. On the web framework I wrote, this means sharing and reusing the same DB connection pool, template cache, other caches and so on. This means a reduced memory footprint and increased performance. In a multi-process environment, you have to instantiate as many connections, caches, templates etc. that you have processes. This is a waste of time and memory.

BTW [shameless plug] here is the cookbook recipe I wrote about thread-safe caching.

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/302997
 
> > Well, this is not going to take us anywhere, especially with all the
> > multicore CPUs coming.
> 
> Again, I don't get it.  Why would multicore CPUs be any worse than
> current multi-CPU machines at multiple processes, and forking?

Obviously they won't be any worse. Well, to be precise, it still depends on the OS, because the scheduler must know the difference between 2 processors and a 2-core processor to efficiently balance the work, but anyway.

What I meant is that right now I'm writing this on a desktop PC with hyperthreading. This means that even on a desktop PC you can benefit from having multithreaded (or multi-processed) applications. With multicore or multiprocessor machines being more and more current, the pressure to have proper threading support in Python will grow and grow.

Regards,

Nicolas