memory consumption

[Inada Naoki]

On Mon, Mar 29, 2021 at 7:16 PM Alexey <zen.supagood at gmail.com> wrote:
>
> Problem. Before executing, my interpreter process weighs ~100Mb, after first run memory increases up to 500Mb
> and after second run it weighs 1Gb. If I will continue to run this class, memory wont increase, so I think
> it's not a memory leak, but rather Python wont release allocated memory back to OS. Maybe I'm wrong.
>

First of all, I recommend upgrading your Python. Python 3.6 is a bit old.

As you saying, Python can not return the memory to OS until the whole
arena become unused.
If your task releases all objects allocated during the run, Python can
release the memory.
But if your task keeps at least one object, it may prevent releasing
the whole arena (256KB).

Python manages only small (~256bytes) objects. Larger objects is
allocated by malloc().
And glibc malloc may not efficient for some usage. jemalloc is better
for many use cases.

You can get some hints from sys._debugmallocstats(). It prints
obmalloc (allocator for small objects) stats to stderr.
Try printing stats before and after 1st run, and after 2nd run. And
post it in this thread if you can. (no sensible information in the
stats).

That is all I can advise.

-- 
Inada Naoki  <songofacandy at gmail.com>