memory consumption

Marco Ippolito maroloccio at gmail.com
Wed Mar 31 04:52:22 EDT 2021


> > At which point does the problem start manifesting itself?
> The problem spot is my cache(dict). I simplified my code to just load 
> all the objects to this dict and then clear it.

What's the memory utilisation just _before_ performing this load? I am assuming
it's much less than this 1 GB you can't seem to drop under after you run your
`.clear()`.

> After loading "top"

You may be using `top` in command line mode already but in case you aren't,
consider sorting processes whose command name is `python` (or whatever filter
selects your program) by RSS, like so, for easier collection of
machine-readable statistics:

# ps -o rss,ppid,pid,args --sort -rss $(pgrep python)
  RSS    PPID     PID COMMAND
  32836   14130   14377 python3
  10644   14540   14758 python3

> For debugging I use Pycharm

Sounds good, you can then use the GUI to set the breakpoint and consult
external statistics-gathering programs (like the `ps` invocation above) as you
step through your code.

Pycharm also allows you to see which variables are in scope in a particular
stack frame, so you'll have an easier time reasoning about garbage collection
in terms of which references might be preventing GC.


More information about the Python-list mailing list