Memory utilization (linux v. openbsd)

[Michael Torrie]

nazgul at jsbsystems.com wrote:
> My problem is that the two process under OpenBSD are going to fail with 
> a MemoryError becaause the size just keeps getting larger and larger.  
> ulimit -d is 1G for each process.

The problem is that you can't get accurate memory use readings from top.

The reality is that your python program is probably using the same 
amount of memory on both platforms (in and of itself), but due to the 
way the memory management OS system works, the usage is being reported 
differently on BSD.  Basically you're seeing the memory footprint of the 
python executable, it's data (your program) and also all of the shared 
libraries that python is linked to.  Of course the shared libraries are 
shared, so it's not like python's using all this memory just itself.

> 
> Any idea why the memory utilization is so much higher under OpenBSD?
> 
> What I also find interesting is that the OpenBSD box seems to grind to a 
> halt (that's probably due to swapping since there's 500M of swap space 
> in use).  Those processes should both be over 90% since they are CPU 
> bound, and they are that way at the start of the run.  Gradually, the 
> system % creeps up as does idle, but there's no reason for them to be 
> idle.  It's a CPU bound process.  (While I'm typing this top is showing 
> cpu rats of 63% and 41% for the two jobs).

I can't answer your questions or solve your problem.  But I just wanted 
to point out that the problem is likely somewhere besides python itself. 
  Maybe there are settings in the OS.  Who knows.  My guess is the 
difference in overall memory footprint is due to differences in the C 
library and its dependencies.  You probably would want to bring this 
issue up on the OpenBSD lists as it's clear that your BSD machine is 
having problems generally (trashing to death).  This doesn't appear to 
be a python problem.

> 
> TIA,
> nazgul