ANN.: Beta 1.0 of Weak Reference Extension Module is now available

[Alex Martelli]

"Gareth McCaughan" <Gareth.McCaughan at pobox.com> wrote in message
news:slrn918ttg.qe.Gareth.McCaughan at g.local...
    [snip]
> Another use for weak references: You build a class that
> remembers what all its instances are, but you still want
> the instances to disappear when there are no "ordinary"
> references to them. In languages with support for finalization
> (like Python and C++) you do this in the __del__ method,
> or destructor, or whatever you want to call it; in languages
> without such support (like Common Lisp) you do it by making
> the class's references to its instances weak references.

While the example is good, I find the second half of this
paragraph flawed: a language may well have some support for
finalization (Python, Java), but still not allow you to
accomplish the task described in the paragraph's first half
except by weak references!

If all references are "strong" (i.e., ensure the object
referred-to is kept alive as long as there are reachable
references to it), then you can't easily arrange for "the
instances to disappear" while keeping in your class object
(which stays reachable) a reference to the instance.

In practice, you CAN "roll your own" weak-references,
but it takes some doing, e.g.:

Suppose you have an existing class RealClass that does
the actual jobs you want BUT fails to keep the "list of
all outstanding instances" that you now also require, as
per the first half of the quoted paragraph.  E.g., just
for example,

class RealClass:
    def __init__(self, name):
        print 'rc.ctor',id(self)
        self.name = name
        print "begin of",self
    def __str__(self):
        # print 'rc.str',id(self)
        return "Real(%s)" % self.name
    def __del__(self):
        print 'rc.dtor',id(self)
        print "finis of",self

Then, rename RealClass to _RealClass, e.g.:

_RealClass = RealClass
del RealClass

and wrap your own proxy around it, e.g.:

class RealClass:
    outstanding = []
    mangledreal = '_RealClass__real'
    def __init__(self, *args, **kwds):
        # print 'proxy.ctor',id(self)
        self.__dict__[self.mangledreal] = None
        real = _RealClass(*args, **kwds)
        self.__dict__[self.mangledreal] = real
        self.outstanding.append(real)
    def __setattr__(self, name, value):
        # print 'proxy.set',name,id(self)
        if name==self.mangledreal:
            self.__dict__[name] = value
        else:
            setattr(self.__real, name, value)
    def __getattr__(self, name):
        # print 'proxy.get',name,id(self)
        if self.__dict__[self.mangledreal] is None:
            raise AttributeError, name
        return getattr(self.__real, name)
    def __delattr__(self, name):
        # print 'proxy.del',name,id(self)
        return delattr(self.__real, name)
    def __del__(self):
        # print 'proxy.dtor',id(self)
        self.outstanding.remove(self.__real)


This will work because eack proxy-class instance
keeps a reference to the real-class instance, and
another such reference is queued to .outstanding;
while, client-code keeps references to the *proxy*
instances only.  This split lets the proxy's __del__
come into play when the last client-code reference
to a given instance goes away, and that __del__
can then also remove the *other* reference to
the underlying real-class instance, the one held
in .outstanding (the proxy instance -> real
instance reference need not be removed -- it
will go of its own accord right after the proxy's
__del__ is done).

(It can be seen as an example of the interesting
things that become possible by explicity and
deliberately decoupling identity and state in
various ways:-).

This stuff can be in a generic Python 'proxy for
weak reference purposes' class, needing the
classobject it's proxying for (here, _RealClass)
as its sole operating parameter.  *However*...

...this is all pretty delicate, and crucially
dependent on "everybody else except the proxy
class" never holding a reference to the real
object, but only to the proxy.  It's not fully
transparent -- it will show through immediately
if anybody asks for an object's __class__; its
_invasive_ nature then becomes evident.  And if
the real-object is doing fancy metaprogramming
stuff itself, or such stuff is being played on
it from elsewhere, things are likely to break!


The 'Weak Reference Extension Module' is *NOT*
invasive, nor is it particularly 'delicate' --
the only 'funny business' needs to be done by
those rather-special pieces of code that do
want a *weak* reference specifically... every
other piece of code can keep doing its thing,
blissfully unaware that weak refs may exist to
some object or other.

Its only weak point is that it's not part of
Python itself -- so you can't necessarily count
on having it around (e.g., in Jython or .NET).

But then, __del__-based tricks are also unlikely
to work seamlessly in Jython or .NET, which lack
reference-counting semantics and thus cannot
guarantee finalization and timing thereof (as
best I understand their current architecture,
at least!).

Having weak references as a core Python concept
would be a Very Good Thing, IMHO.  They could be
implemented in CPython by merging the Weak
Reference Extension Module (or some similar
idea) into the core, and in Jython and .NET
by relying on the native weak-reference mechanisms
supplied by modern JVM's and (I believe) .NET...
the Python programmer would not need to know
which kind of underlying implementation of WR
is being used, but could just USE it!-)


Alex