script uses up all memory

[Chris Angelico]

On Fri, Mar 7, 2014 at 10:53 AM, Marko Rauhamaa <marko at pacujo.net> wrote:
> Chris Angelico <rosuav at gmail.com>:
>
>> Can you give a useful example of a closure that does create a refloop?
>
> Just the other day, I mentioned the state pattern:
>
>    class MyStateMachine:
>        def __init__(self):
>            sm = self
>
>            class IDLE:
>                def ding(self):
>                    sm.open_door()
>                    sm.state = AT_DOOR()

Yeah, that's an extremely unusual way to do things. Why keep on
instantiating objects when you could just reference functions?

> In general, event-driven programming produces circular references left
> and right, and that might come into wider use with asyncio.

Nope; certainly not with closures. I do a whole lot of event-driven
programming (usually in Pike rather than Python, but they work the
same way in this), and there's no reference loop. Properly-done
event-driven programming should have two basic states: a reference
from some invisible thing that can trigger the event (eg a GUI widget)
to a callable, and a reference from that callable to its state. Once
the trigger is gone, the callable is dropped, its state is dropped,
and everything's cleaned up. You don't usually need a reference inside
the function to that function.

Don't forget, a closure need only hang onto the things it actually
uses. It doesn't need all its locals.

> I suspect generators might create circular references as well.

I doubt it.

>>> def foo(x):
    return ("x"*i for i in range(x))
>>> len([foo(5) for _ in range(1000)])
1000
>>> gc.collect()
0
>>> len([foo(5) for _ in range(1000)])
1000
>>> gc.collect()
0

Again, unless it keeps a reference to itself, there's no loop. It'll
need to hang onto some of its locals, but that's all.

> Any tree data structure with parent references creates cycles.

Yes, but how many of those do you actually have and drop? If you
create a GUI, you generally hold your entire widget tree stably. The
only issue is if you create a parent-child subtree and then drop it.
That shouldn't be being done in a tight loop. Most of the classic data
structures like trees are implemented at the C level, so again, your
code shouldn't be concerning itself with that.

> In fact, I would imagine most OO designs create a pretty tight mesh of
> back-and-forth references.

Examples, please? I can think of a handful of situations where I've
created reference loops, and they're sufficiently rare that I can put
comments against them and explicitly break them. For instance, I have
a "Subwindow" that has a "Connection". My window can have multiple
subwindows, a subwindow may or may not have a connection, and the
connection always references its subwindow. The subw->connection->subw
loop is explicitly broken when the connection is terminated. If the
window chooses to drop a subw, it first checks if there's a connection
(and prompts the user to confirm), and then will explicitly
disconnect, which breaks the refloop (as the connection's terminated).
I did a similar thing at work, again with explicit refloop breakage to
ensure clean removal. Apart from those two cases, I can't think of
anything in the last ten years where I've had a data structure with a
loop in it, where the whole loop could be dropped. (My MUD has a loop,
in that a character exists in a room, and the room keeps track of its
contents; but it's not logical to drop a room with characters in it,
and dropping a character is done by moving it to no-room, which breaks
the refloop.)

ChrisA