Threads vs Processes

[Chance Ginger]

On Wed, 26 Jul 2006 10:54:48 -0700, Carl J. Van Arsdall wrote:

> Alright, based a on discussion on this mailing list, I've started to 
> wonder, why use threads vs processes.  So, If I have a system that has a 
> large area of shared memory, which would be better?  I've been leaning 
> towards threads, I'm going to say why.
> 
> Processes seem fairly expensive from my research so far.  Each fork 
> copies the entire contents of memory into the new process.  There's also 
> a more expensive context switch between processes.  So if I have a 
> system that would fork 50+ child processes my memory usage would be huge 
> and I burn more cycles that I don't have to.  I understand that there 
> are ways of IPC, but aren't these also more expensive?
> 
> So threads seems faster and more efficient for this scenario.  That 
> alone makes me want to stay with threads, but I get the feeling from 
> people on this list that processes are better and that threads are over 
> used.  I don't understand why, so can anyone shed any light on this?
> 
> 
> Thanks,
> 
> -carl

Not quite that simple. In most modern OS's today there is something
called COW - copy on write. What happens is when you fork a process
it will make an identical copy. Whenever the forked process does
write will it make a copy of the memory. So it isn't quite as bad.

Secondly, with context switching if the OS is smart it might not 
flush the entire TLB. Since most applications are pretty "local" as
far as execution goes, it might very well be the case the page (or 
pages) are already in memory. 

As far as Python goes what you need to determine is how much 
real parallelism you want. Since there is a global lock in Python
you will only execute a few (as in tens) instructions before
switching to the new thread. In the case of true process you 
have two independent Python virtual machines. That may make things
go much faster.

Another issue is the libraries you use. A lot of them aren't 
thread safe. So you need to watch out.

Chance