multiprocessing eats memory

Max Ivanov ivanov.maxim at gmail.com
Thu Sep 25 08:40:48 EDT 2008


I'm playing with pyprocessing module and found that it eats lot's of
memory. I've made small test case to show it. I pass ~45mb of data to
worker processes and than get it back slightly modified. At any time
in main process there are shouldn't be no more than two copies of data
(one original data and one result). I run it on 8-core server and top
shows me that main process eats ~220 Mb and worker processes eats 90
-150 mb. Isn't it too much?

Small test-case is uploaded to pastebin: http://pastebin.ca/1210523



More information about the Python-list mailing list