ESR "Waning of Python" post

[dieter]

Marko Rauhamaa <marko at pacujo.net> writes:
> dieter <dieter at handshake.de>:
> ...
>> Definitely. The application concerned was a long running web application;
>> caching was an important feature to speed up its typical use cases.
>
> As an optimization technique, I suggest turning the cache into a "binary
> blob" opaque to GC, or using some external component like SQLite.

This was a Python application and as such, it was primarily working with
Python objects. And it was a complex application heavily depending
on subframeworks, many of them internally using caching to speed
things up, the details hidden from the application.
Redesigning the application to use an alternative caching was
out of question.


> Keeping the number of long-term objects low is key.

Right, if I need near realtime behaviour and must live
with [C]Python's garbage collector.

But, a web application does usually not need near realtime behaviour.
An occasional (maybe once in a few days) garbage collection and
associated reduced response time is acceptable.
A problem only arises if a badly designed component produces
quite frequently hundreds of thousands of temporary objects
likely triggering (frequent) garbage collections.


> Note that Python creates a temporary object every time you invoke a
> method. CPython removes them quickly through reference counting, but
> other Python implementations just let GC deal with them, and that's
> generally ok.

The initial point has been that you must carefully look at the context
for which you design a solution and choose appropriately (among
other the implementation language).
In my case, the web application framework was fixed ("Zope")
and therefore the language ("Python") and its implementation ("CPython").
"Normal" temporary objects did not pose a problem (due to reference
counting); only the mass creation of temporary objects can
be problematic (as GC is triggered before the temporary objects
are released again (due to reference counting).

>
>
> Marko