Design thought for callbacks

[Steven D'Aprano]

Marko Rauhamaa wrote:

> Chris Angelico <rosuav at gmail.com>:
> 
>> On Sun, Feb 22, 2015 at 7:34 PM, Marko Rauhamaa <marko at pacujo.net> wrote:
>>> Refloops are not to be worried about, let alone removed.
>>
>> Why?
> 
> Because the whole point of GC-languages is that you should stop worrying
> about memory. Trying to mastermind and micromanage GC in the application
> is, pardon my French, an antipattern.

While it would be nice to be able to stop worrying about memory, try to
calculate 1000**1000**1000 and see how that works for you.

Garbage collection enables us to *mostly* automate the allocation and
deallocation of memory. If doesn't mean we can forget about it. GC is an
abstraction that frees us from most of the grunt work of allocating memory,
but it doesn't mean that there is never any need to think about memory. GC
is a leaky abstraction. Depending on the implementation, it may cause
distracting and annoying pauses in your application and/or resource leaks.
Even if there are no pauses, GC still carries a performance penalty. Good
programmers need to be aware of the limitations of their tools, and be
prepared to code accordingly.

When writing programs for educational purposes, we should try to code in the
simplest and most elegant way with no thought given to annoying practical
matters. At least at first. But when writing programs for actual use, we
should write for the implication we have, not the one we wish we had.


>> They force the use of the much slower cycle-detecting GC, rather than
>> the quick and efficient CPython refcounter.
> 
> Java's Hotspot doesn't bother with refcounters but is much faster than
> Python. CPython's refcounters are a historical accident that a Python
> application developer shouldn't even be aware of.

I don't know about Java's Hotspot, but I do know that CPython's ref counting
garbage collector has at least one advantage over the GC used by Jython and
IronPython: unlike them, open files are closed as soon as they are no
longer in use. Code like this may run out of operating system file handles
in Jython:

i = 0
while True:
    f = open('/tmp/x%d' % i)
    i += 1

while CPython will just keep going. I suppose it will *eventually* run out
of some resource, but probably not file handles.

Oh, a bit of trivia: Apple is abandoning their garbage collector and going
back to a reference counter:

https://developer.apple.com/news/?id=02202015a

Word on Reddit is that Apple is concerned about performance and battery
life.

P.S. A reminder that reference counting *is* a form of garbage collection.


>> I don't know how other Pythons work, but mark-and-sweep has its own
>> costs, and I don't know of any system that's both prompt and able to
>> detect refloops.
> 
> It's exceedingly difficult (and pointless) to detect cycles in your
> object structures. Python is going to have to do a GC occasionally
> anyway. Yes, your worst-case response times are going to suffer, but
> that's the cost of doing business.

In *general*, you're right. Who wants to spend all their time worrying about
cycles when the GC can do it for you? But if cycles are rare, and in known
parts of your code where it is simple to break them when you're done,
there's no disadvantage to doing so. Leave the GC for the hard cases.

It's like explicitly closing a file, either with file.close() or a context
manager. When using CPython, it doesn't really matter whether you close the
file or not, since the ref counter will normally close it automatically as
soon as the file goes out of scope. But it is cheap and easy to do so, so
why not do it? Then, when it otherwise would matter, say you are running
under Jython, it doesn't because you've closed the file.


>> Helping it along means your program doesn't waste memory. Why such a
>> blanket statement?
> 
> Because worrying Python programmers with evil spirits (reference loops)
> leads to awkward coding practices and takes away one of the main
> advantages of Python as a high-level programming language.

I think you exaggerate a tad. We're not trying to scare beginners, we're a
group of moderately experienced coders discussing "best practice" (or at
least "reasonable practice") when using callbacks.


-- 
Steven