Magic function

[Ruediger]

dg.google.groups at thesamovar.net wrote:

> Hi Rüdiger,
> 
> Thanks for your message. I liked your approach and I've been trying
> something along exactly these sorts of lines, but I have a few
> problems and queries.
> 
> The first problem is that the id of the frame object can be re-used,
> so for example this code (where I haven't defined InstanceTracker and
> getInstances, but they are very closely based on the ideas in your
> message):
> 
> class A(InstanceTracker):
>     gval = 0
>     def __init__(self):
>         self.value = A.gval # each time you make a new object, give
>         A.gval += 1         # it a value one larger
>     def __repr__(self):
>         return str(self.value)
> 
> def f2():
>     a = A() # objects 0 and 2
>     return getInstances(A)
> 
> def f3():
>     a = A() # object 1
>     return f2()
> 
> inst2 = f2()
> inst3 = f3()
> print inst2
> print inst3
> 
> The output is:
> 
> [0]
> [0, 2]
> 
> The A-variable with value 0 is not being garbage collected because
> it's saved in the variable inst2, but it's also being returned by the
> second call to getInstances because the frame of f2 is the same each
> time (which makes sense, but may be implementation specific?). The

Yes and No. id basically returns the memory address of an object.
and yes this is implementation specific. As of my knowledge a stackframe is
of constant size in cPython. Though you get always the same id for the same
call level as you would always get the same number from your instance
tracker.

No A-variable with value 0 is reported the second time because it had been
created at the same call level  __and__ it is still accessible from that
call level.

If you do want such object's to be destroyed you must not create hard
references to them. This may be hard for your users.

However you could still do something like:

def f2():
    InstanceTracker.prepare() # <-- delete previously created Entrys
                              # here or calculate some magic hash value
                              # or random number.
    a = A() # objects 0 and 2
    return getInstances(A)

or

@managedInstance # <-- see above
def f2():
    a = A() # objects 0 and 2
    return getInstances(A)


> same problem doesn't exist when you use the stack searching method
> because from f2's point of view, the only bound instance of A is the
> one in that particular call of f2. If you had at the end instead of
> the inst2, inst3 stuff:
> 
> print f2()
> print f3()
> 
> The output is:
> 
> [0]
> [2]
 
You basically guess here how a user would write his programm.

what if your user's writes code like this?

>>> my_global_dict = dict()
>>> my_global_list = list()
>>>
>>> def f2():
...     my_global_dict["a"] = object()
...     my_global_list.append(object())
...     print locals()
...
>>> f2()
{}
>>>

you would not find such a references by inspecting the stack.


> Again, I guess this because A with value 0 is being garbage collected
> between print f2() and print f3(), but again I think this is
> implementation specific? You don't have a guarantee that this object
> will be garbage collected straight away do you?

Yes inspecting the stack is pure guesswork.
You don't know anything about your users program structures and inspecting
the stack won't tell you. 


> So my concern here is that this approach is actually less safe than
> the stack based approach because it depends on implementation specific
> details in a non-straightforward way. That said, I very much like the
> fact that this approach works if I write:
> 
> a = [A()]
> a = [[A()]]
> etc.
> 
> To achieve the same thing with the stack based approach you have to
> search through all containers to (perhaps arbitrary) depth.


Yes and as pointed out above you will also have to search the global
namespace and all available memory because an instance could have been
created by psyco, ctypes, Swig, Assembly code .....


> I also have another problem which is that I have a function decorator
> which returns a callable object (a class instance not a function).
> Unfortunately, the frame in which the callable object is created is
> the frame of the decorator, not the place where the definition is.
> I've written something to get round this, but it seems like a bit of a
> hack.
> 
> Can anyone suggest an approach that combines the best of both worlds,
> the instance tracking approach and the stack searching approach? Or do
> I need to just make a tradeoff here?

well that's my last example. I hope it will help.


from weakref import ref
from random import seed, randint
seed()

class ExtendedRef(ref):
    def __init__(self, ob, callback=None, **annotations):
        super(ExtendedRef, self).__init__(ob, callback)
        self.__id = 0

class WeakSet(set):
    __inst__ = 0
    def add(self, value ):
        wr = ExtendedRef(value, self.remove)
        wr.__id = WeakSet.__inst__
        set.add(self, wr)
    def get(self, _id=None):
        _id = _id if _id else WeakSet.__inst__
        return [ _() for _ in self if _.__id == _id]
    @classmethod
    def prepare(self):
        WeakSet.__inst__ = randint(0, 2**32-1)
        return WeakSet.__inst__

class bigobject(WeakSet):
    def run(self, _id=None):
        for obj in self.get(_id):
            # process object's
            print obj.value

class foo(object):
    __instances__ = bigobject()
    def __init__(self, value):
        foo.__instances__.add(self)
        self.value = value

def managed(fun):
    def new(*att, **katt):
        _id = WeakSet.prepare()
        _result = fun(*att, **katt)
        foo.__instances__.run(_id)
        return _result
    return new

@managed
def main( depth, txt ):
    obj1 = foo("%s obj1 at depth %s " % (txt, depth))
    obj2 = foo("%s obj2 at depth %s" % (txt, depth))
    print "processing objects created in %s at depth %s" % (txt, depth)
    foo.__instances__.run()
    if depth == 0:
        return
    else:
        main(depth-1, "foo")
        main(depth-1, "bar")

if __name__ == "__main__":
    _id = WeakSet.prepare()
    obj1 = foo("obj1 at __main__")
    main(3, "root")
    print "processing objects created in __main__"
    foo.__instances__.run(_id)


ruediger at linux-ehvh:~/tmp> python test12.py
processing objects created in root at depth 3
root obj1 at depth 3
root obj2 at depth 3
processing objects created in foo at depth 2
foo obj1 at depth 2
foo obj2 at depth 2
processing objects created in foo at depth 1
foo obj2 at depth 1
foo obj1 at depth 1
processing objects created in foo at depth 0
foo obj1 at depth 0
foo obj2 at depth 0
processing objects created in bar at depth 0
bar obj1 at depth 0
bar obj2 at depth 0
processing objects created in bar at depth 1
bar obj1 at depth 1
bar obj2 at depth 1
processing objects created in foo at depth 0
foo obj2 at depth 0
foo obj1 at depth 0
processing objects created in bar at depth 0
bar obj1 at depth 0
bar obj2 at depth 0
processing objects created in bar at depth 2
bar obj1 at depth 2
bar obj2 at depth 2
processing objects created in foo at depth 1
foo obj1 at depth 1
foo obj2 at depth 1
processing objects created in foo at depth 0
foo obj1 at depth 0
foo obj2 at depth 0
processing objects created in bar at depth 0
bar obj1 at depth 0
bar obj2 at depth 0
processing objects created in bar at depth 1
bar obj2 at depth 1
bar obj1 at depth 1
processing objects created in foo at depth 0
foo obj2 at depth 0
foo obj1 at depth 0
processing objects created in bar at depth 0
bar obj2 at depth 0
bar obj1 at depth 0
processing objects created in __main__
obj1 at __main__
ruediger at linux-ehvh:~/tmp>