Magic function

[dg.google.groups at thesamovar.net]

Hi all,

I'm part of a small team writing a Python package for a scientific
computing project. The idea is to make it easy to use for relatively
inexperienced programmers. As part of that aim, we're using what we're
calling 'magic functions', and I'm a little bit concerned that they
are dangerous code. I'm looking for advice on what the risks are (e.g.
possibility of introducing subtle bugs, code won't be compatible with
future versions of Python, etc.).

Quick background: Part of the way our package works is that you create
a lot of objects, and then you create a new object which collects
together these objects and operates on them. We originally were
writing things like:

obj1 = Obj(params1)
obj2 = Obj(params2)
...
bigobj = Bigobj(objects=[obj1,obj2])
bigobj.run()

This is fine, but we decided that for clarity of these programs, and
to make it easier for inexperienced programmers, we would like to be
able to write something like:

obj1 = Obj(params1)
obj2 = Obj(params2)
...
run()

The idea is that the run() function inspects the stack, and looks for
object which are instances of class Obj, creates a Bigobj with those
objects and calls its run() method.

So, any comments on that approach?

I'm including the code I've written to do this, and if you have time
to look through it, I'd also be very grateful for any more specific
comments about the way I've implemented it (in particular, can it be
made faster, is my program creating cycles that stop the garbage
collection from working, etc.). I hope the code will be formatted
correctly:

def
getInstances(instancetype,level=1,includeglobals=True,containersearchdepth=1,exclude={},predicate=lambda
x:True):
    """Find all instances of a given class at a given level in the
stack
    """
    vars = {}
    # Note: we use level+1 because level refers to the level relative
to the function calling this one
    if includeglobals: vars.update(stack()[level+1][0].f_globals)
    vars.update(stack()[level+1][0].f_locals)
    # Note that you can't extract the names from vars.itervalues() so
we provide via knownnames the names vars.iterkeys(),
    # containersearchdepth+1 is used because vars.itervalues() is the
initial container from the point of view of this
    # function, but not from the point of view of the person calling
getInstances
    objs, names =
extractInstances(instancetype,vars.itervalues(),containersearchdepth
+1,knownnames=vars.iterkeys(),exclude=exclude,predicate=predicate)
    return (objs,names)

def
extractInstances(instancetype,container,depth,containingname='vars()',knownnames=None,exclude={},predicate=lambda
x:True):
    if depth<=0: return ([],[])
    if isinstance(container,str): return ([],[]) # Assumption: no need
to search through strings
    # Ideally, this line wouldn't be here, but it seems to cause
programs to crash, probably because
    # some of the simulator objects are iterable but shouldn't be
iterated over normally
    # TODO: Investigate what is causing this to crash, and possibly
put in a global preference to turn this line off?
    if not isinstance(container,
(list,tuple,dict,type({}.itervalues()))): return ([],[])
    # Note that knownnames is only provided by the initial call of
extractInstances and the known
    # names are from the dictionary of variables. After the initial
call, names can only come from
    # the __name__ attribute of a variable if it has one, and that is
checked explicitly below
    if knownnames is None:
        knewnames = False
        knownnames = repeat(containingname)
    else:
        knewnames = True
    objs = []
    names = []
    try: # container may not be a container, if it isn't, we'll
encounter a TypeError
        for x,name in zip(container,knownnames):
            # Note that we always have a name variable defined, but if
knewnames=False then this is just
            # a copy of containingname, so the name we want to give it
in this instance is redefined in this
            # case. We have to use this nasty check because we want to
iterate over the pair (x,name) as
            # variables in the same position in the container have the
same name, and we can't necessarily
            # use __getitem__
            if hasattr(x,'__name__'): name = x.__name__
            elif not knewnames: name = 'Unnamed object, id =
'+str(id(x))+', contained in: '+containingname
            if isinstance(x,instancetype):
                if x not in exclude and predicate(x):
                    objs.append(x)
                    names.append(name)
            else: # Assumption: an object of the instancetype is not
also a container we want to search in.
                # Note that x may not be a container, but then
extractInstances will just return an empty list
                newobjs, newnames =
extractInstances(instancetype,x,depth-1,containingname=name,predicate=predicate)
                objs += newobjs
                names += newnames
        return (objs,names)
    except: # if we encounter a TypeError from the for loop, we just
return an empty pair, container wasn't a container
        return ([],[])

In case that doesn't work, here it is without the comments:

def
getInstances(instancetype,level=1,includeglobals=True,containersearchdepth=1,exclude={},predicate=lambda
x:True):
    vars = {}
    if includeglobals: vars.update(stack()[level+1][0].f_globals)
    vars.update(stack()[level+1][0].f_locals)
    objs, names =
extractInstances(instancetype,vars.itervalues(),containersearchdepth
+1,knownnames=vars.iterkeys(),exclude=exclude,predicate=predicate)
    return (objs,names)

def
extractInstances(instancetype,container,depth,containingname='vars()',knownnames=None,exclude={},predicate=lambda
x:True):
    if depth<=0: return ([],[])
    if isinstance(container,str): return ([],[])
    if not isinstance(container,
(list,tuple,dict,type({}.itervalues()))): return ([],[])
    if knownnames is None:
        knewnames = False
        knownnames = repeat(containingname)
    else:
        knewnames = True
    objs = []
    names = []
    try:
        for x,name in zip(container,knownnames):
            if hasattr(x,'__name__'): name = x.__name__
            elif not knewnames: name = 'Unnamed object, id =
'+str(id(x))+', contained in: '+containingname
            if isinstance(x,instancetype):
                if x not in exclude and predicate(x):
                    objs.append(x)
                    names.append(name)
            else:
                newobjs, newnames =
extractInstances(instancetype,x,depth-1,containingname=name,predicate=predicate)
                objs += newobjs
                names += newnames
        return (objs,names)
    except:
        return ([],[])