memory consumption

[Dieter Maurer]

Alexey wrote at 2021-3-29 06:26 -0700:
>понедельник, 29 марта 2021 г. в 15:57:43 UTC+3, Julio Oña:
>> It looks like the problem is on celery.
>> The mentioned issue is still open, so not sure if it was corrected.
>>
>> https://manhtai.github.io/posts/memory-leak-in-celery/
>
>As I mentioned in my first message, I tried to run
>this task(class) via Flask API calls, without Celery.
>And results are the same. Flask worker receives the API call and
>executes MyClass().run() inside of view. After a few calls
>worker size increases to 1Gb of RAM. In production I have 8 workers,
> so in idle they will hold 8Gb.

Depending on your system (this works for `glibc` systems),
you can instruct the memory management via the envvar
`MALLOC_ARENA_MAX` to use a common memory pool (called "arena")
for all threads.
It is known that this can drastically reduce memory consumption
in multi thread systems.