Program uses twice as much memory in Python 3.6 than in Python 3.5

[Jan Gosmann]

Hi,

I have a program which uses twice as much memory when I run it in Python 
3.6 than when I run it in Python 3.5 (about 60GB instead of 30GB). I 
tried to find the reason for that, but the cumulated size (measured with 
sys.getsizeof) of all objects returned by gc.get_objects accumulates 
only to about 17GB in both cases. The program also uses NumPy and I 
tried tracking allocations with the NumPy allocation tracker in the 
relevant part of the program, but again the number of allocations are 
almost identical and the reported maximum memory usage perfectly agrees 
(it is about 18GB).

Any ideas where this significant increase in memory consumption could 
come from? Or any ideas how to further debug this?

Looking at the changelog it seems that the only change in Python 3.6 
affecting memory usage is the new dict implementation which is supposed 
to be more memory efficient. In fact, this is what I find if I run the 
program with a smaller test cases. There the memory consumption is less 
with Python 3.6.

Cheers, Jan