python 2.5a2, gcc 4.1 and memory problems

[Tim Peters]

[Michele Petrazzo]
> I'm doing some tests on my debian testing and I see a very strange
> memory problem with py 2.5a2 (just downloaded) and compiled with gcc
> 4.1.0, but not with the gcc 3.3.5:
>
> My test are:
>
> #--test.py
> import sys
> if sys.version.startswith("2.3"):
>   from sets import Set as set
> b=set(range(5000000))
> a=set(range(100000))
> c = b-a
>
> for i in 3 4 5; do echo "python 2.$i" && time python2.$i test.py; done
>
> My configure args for compile py 2.5a2 are:
> ...
> Then I execute my test. The memory usage of 2.5a2 and gcc 3.3 that I see
> with "top", is the same (about VIRT: 260 MB and RES: 250MB ) that with
> the py 2.3 and 2.4, but then I recompile with 4.1 and execute the same
> test, my system "stop to work"... with "top" I can see that it use VIRT:
> 2440 MB and RES: 640MB RAM (I think all that I have into my pc)

FYI, here are stats I saw running the test program on Windows:

Python \ VM size at end (KB)  peak mem use (KB)
------   -------------------  -----------------
2.3.5                    262                313
2.4.3                    262                313
trunk                    196                230

"trunk" should be very close to 2.5a2 in memory use.  "How much
memory" the OS thinks you're using greatly depends on the platform C's
malloc/free implementation, so I'd suspect a relevant difference in
glibc (assuming that's the C library you're using).  As the results on
Windows show, it's unlikely that Python is _directly_ "consuming more
memory" here; it's quite possible that the platform free() is hanging
on to temp memory where in a different implementation it returned that
temp memory to the OS.