[Python-Dev] Reduce memory footprint of Python

[Victor Stinner]

2013/10/6 Benjamin Peterson <benjamin at python.org>:
> 2013/10/6 Victor Stinner <victor.stinner at gmail.com>:
>> Hi,
>>
>> Slowly, I'm trying to see if it would be possible to reduce the memory
>> footprint of Python using the tracemalloc module.
>>
>> First, I noticed that linecache can allocate more than 2 MB. What do
>> you think of adding a registry of "clear cache" functions? For
>> exemple, re.purge() and linecache.clearcache(). gc.collect() clears
>> free lists. I don't know if gc.collect() should be related to this new
>> registy (clear all caches) or not.
>
> What is the usecase for minimizing the memory usage of Python in the
> middle of a program?

If you know that your application uses a lot of memory, it is
interesting to sometimes (when the application is idle) try to release
some bytes to not use all the system memory. On embedded devices,
memory is expensive and very limited. Each byte is important :-)

>> unittest doesn't look to release memory (the TestCase class) after the
>> execution of a test.
>
> Is it important to optimize unittests for memory usage?

I was surprised that running the whole test suite use more and more
memory. I expected something more stable with some short peaks.

Reducing the memory footprint is useful for performances (of test
suite) and be able to run the test suite on system with low memory
(embedded devices).

Ezio pointed me the following issues:

http://bugs.python.org/issue17534
http://bugs.python.org/issue11798
http://bugs.python.org/issue9815

Victor