ESR "Waning of Python" post

[Marko Rauhamaa]

dieter <dieter at handshake.de>:
> Marko Rauhamaa <marko at pacujo.net> writes:
>> However, I challenge the notion that creating hundreds of thousands of
>> temporary objects is stupid. I suspect that the root cause of the
>> lengthy pauses is that the program maintains millions of *nongarbage*
>> objects in RAM (a cache, maybe?).
>
> Definitely. The application concerned was a long running web application;
> caching was an important feature to speed up its typical use cases.

As an optimization technique, I suggest turning the cache into a "binary
blob" opaque to GC, or using some external component like SQLite.
Keeping the number of long-term objects low is key.

Note that Python creates a temporary object every time you invoke a
method. CPython removes them quickly through reference counting, but
other Python implementations just let GC deal with them, and that's
generally ok.


Marko