[issue11849] glibc allocator doesn't release all free()ed memory

[Antoine Pitrou]

Antoine Pitrou <pitrou at free.fr> added the comment:

The MALLOC_MMAP_THRESHOLD improvement is less visible here:

$ MALLOC_MMAP_THRESHOLD_=1024 ../opt/python issue11849_test.py 
*** Python 3.3.0 alpha
--- USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
  0 antoine   7703  0.0  0.1  57756  8560 pts/2    S+   01:16   0:00 ../opt/python issue11849_test.py
  1 antoine   7703 62.0  1.0 138892 86100 pts/2    S+   01:16   0:01 ../opt/python issue11849_test.py
  2 antoine   7703 84.6  2.0 213580 160552 pts/2   S+   01:16   0:02 ../opt/python issue11849_test.py
  3 antoine   7703 97.0  2.9 288080 234972 pts/2   S+   01:16   0:03 ../opt/python issue11849_test.py
  4 antoine   7703 85.6  3.9 362852 309408 pts/2   S+   01:16   0:05 ../opt/python issue11849_test.py
  5 antoine   7703 93.4  4.8 437616 383844 pts/2   S+   01:16   0:06 ../opt/python issue11849_test.py
  6 antoine   7703 99.0  5.7 512380 458276 pts/2   S+   01:16   0:07 ../opt/python issue11849_test.py
  7 antoine   7703 89.6  6.7 591360 535672 pts/2   S+   01:16   0:08 ../opt/python issue11849_test.py
  8 antoine   7703 94.9  7.6 661676 607156 pts/2   S+   01:16   0:10 ../opt/python issue11849_test.py
  9 antoine   7703 95.5  8.6 740652 684556 pts/2   S+   01:16   0:11 ../opt/python issue11849_test.py
END antoine   7703 96.1  7.5 650432 597736 pts/2   S+   01:16   0:13 ../opt/python issue11849_test.py
 GC antoine   7703 97.2  6.5 570316 519228 pts/2   S+   01:16   0:13 ../opt/python issue11849_test.py
*** antoine   7703 90.8  6.5 569876 518792 pts/2   S+   01:16   0:13 ../opt/python issue11849_test.py


By the way, an easy fix is to use cElementTree instead of ElementTree. It still won't release all memory but it will eat a lot less of it, and be much faster as well.

----------
title: ElementTree memory leak -> glibc allocator doesn't release all free()ed memory
versions: +Python 3.3 -Python 2.5

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11849>
_______________________________________