[Python-Dev] Quick-and-dirty weak references

[Tim Peters]

[about weakdicts and the possibility of building them on weak
 references; the obvious way doesn't clean up the dict itself by
 magic; maybe a weak object should be notified when its referent
 goes away
]

[M.-A. Lemburg]
> Perhaps one could fiddle something out of the Proxy objects
> in mxProxy (you know where...). These support a special __cleanup__
> protocol that I use a lot to work around circular garbage:
> the __cleanup__ method of the referenced object is called prior
> to destroying the proxy; even if the reference count on the
> object has not yet gone down to 0.
>
> This makes direct circles possible without problems: the parent
> can reference a child through the proxy and the child can reference the
> parent directly.

What you just wrote is:

    parent --> proxy --> child -->+
    ^                             v
    +<----------------------------+

Looks like a plain old cycle to me!

> As soon as the parent is cleaned up, the reference to
> the proxy is deleted which then automagically makes the
> back reference in the child disappear, allowing the parent
> to be deallocated after cleanup without leaving a circular
> reference around.

M-A, this is making less sense by the paragraph <wink>:  skipping the
middle, this says "as soon as the parent is cleaned up ... allowing the
parent to be deallocated after cleanup".  If we presume that the parent gets
cleaned up explicitly (since the reference from the child is keeping it
alive, it's not going to get cleaned up by magic, right?), then the parent
could just as well call the __cleanup__ methods of the things it references
directly without bothering with a proxy.  For that matter, if it's the
straightforward

    parent <-> child

kind of cycle, the parent's cleanup method can just do

    self.__dict__.clear()

and the cycle is broken without writing a __cleanup__ method anywhere
(that's what I usually do, and in this kind of cycle that clears the last
reference to the child, which then goes away, which in turn automagically
clears its back reference to the parent).

So, offhand, I don't see that the proxy protocol could help here.  In a
sense, what's really needed is the opposite:  notifying the *proxy* when the
*real* object goes away (which makes no sense in the context of what your
proxy objects were designed to do).

[about Java and its four reference strengths]

Found a good introductory writeup at (sorry, my mailer will break this URL,
so I'll break it myself at a sensible place):

http://developer.java.sun.com/developer/
    technicalArticles//ALT/RefObj/index.html

They have a class for each of the three "not strong" flavors of references.
For all three you pass the referenced object to the constructor, and all
three accept (optional in two of the flavors) a second ReferenceQueue
argument.  In the latter case, when the referenced object goes away the
weak/soft/phantom-ref proxy object is placed on the queue.  Which, in turn,
is a thread-safe queue with various put, get, and timeout-limited polling
functions.  So you have to write code to look at the queue from time to
time, to find the proxies whose referents have gone away.

The three flavors may (or may not ...) have these motivations:

soft:  an object reachable at strongest by soft references can go away at
any time, but the garbage collector strives to keep it intact until it can't
find any other way to get enough memory

weak:  an object reachable at strongest by weak references can go away at
any time, and the collector makes no attempt to delay its death

phantom:  an object reachable at strongest by phantom references can get
*finalized* at any time, but won't get *deallocated* before its phantom
proxy does something or other (goes away? wasn't clear).  This is the flavor
that requires passing a queue argument to the constructor.  Seems to be a
major hack to worm around Java's notorious problems with order of
finalization -- along the lines that you give phantom referents trivial
finalizers, and put the real cleanup logic in the phantom proxy.  This lets
your program take responsibility for running the real cleanup code in the
order-- and in the thread! --where it makes sense.

Java 1.2 *also* tosses in a WeakHashMap class, which is a dict with
under-the-cover weak keys (unlike Dieter's flavor with weak values), and
where the key+value pairs vanish by magic when the key object goes away.
The details and the implementation of these guys waren't clear to me, but
then I didn't download the code, just scanned the online docs.


Ah, a correction to my last post:

class _Weak:
    ...
    def __del__(self):
        # this is purely an optimization:  if self gets nuked,
        # exempt its referent from greater expense when *it*
        # dies
        if self.id is not None:
            __clear_weak_bit(__id2obj(self.id))
            del id2weak[self.id]

Root of all evil:  this method is useless, since the id2weak dict keeps each
_Weak object alive until its referent goes away (at which time self.id gets
set to None, so _Weak.__del__ doesn't do anything).  Even if it did do
something, it's no cheaper to do it here than in the systemt cleanup code
("greater expense" was wrong).

weakly y'rs  - tim


PS:  Ooh!  Ooh!  Fellow at work today was whining about weakdicts, and
called them "limp dicts".  I'm not entirely sure it was an innocent Freudian
slut, but it's a funny pun even if it wasn't (for you foreigners, it sounds
like American slang for "flaccid one-eyed trouser snake" ...).