Memory consumption of multiprocessing.Pool

[Aahz]

In article <mailman.1926.1260872281.2873.python-list at python.org>,
Wolodja Wentland  <wentland at cl.uni-heidelberg.de> wrote:
>
>I have a problem with the memory consumption of multiprocessing.Pool()'s
>worker processes. I have a parent process that has to handle big data
>structures and would like to use a pool of processes for computations.
>
>The problem is, that all worker processes have the same memory
>requirement as the parent one, although they do *not* use any the parent
>processes data structures. 

What I would do is put your current parent process into its own
subprocess and have the main process handle communication between the
memory-heavy subprocess and the worker processes.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.