Macro preprocessor.

[Christian Tismer]

"Magnus L. Hetland" wrote:
> 
> Christian Tismer <tismer at appliedbiometrics.com> writes:
> 
> > Kaz Kylheku wrote:
> > [description of a cute macro processor skipped]
> >
> > > Okay, enough. :) I'll probably get hate mail now about being off topic.
> >
> > No hate mail, but some thoughts:
> [...]
> 
> > There are a few cases where I admit that I generate scriptlets
> > on-the-fly using the formatting operator.
> 
> Now you've piqued my interest... How do you do this, and in which
> cases (for what purposes)? Do you create strings that you execute, or
> what? (Sounds like a cool thing to do ;)

I wrote a helper class which fakes dynamic classes which
can be pickled and stored in database fields.
This was used for a glass fiber network simulation project, where
networks of nodes with lots of properties and direct or symbolic
references to other nodes have to be stored, edited and updated.

The first version of this used exec and formatting to generate
classes dynamically and provide attributes and methods.

Later I could replace almost everything with calls to the "new"
module, and dynnamic assignment of functions as methods.

This is a case where classes and instances are not so sharply
distinguished, it falls into the class of problems where
metaclasses are handy. One simulation defines a special
configuration, gathers data and is saved as an object in
the database, this makes it like an instance. But other
simulations can refer to it, use it as a building block
and inherit from it, which makes it look like a class.

This is a proprietary project, so I can show just some
excerpts from an outdated version of the source files.
System design has been completely redefined, so this is
history.

Here is a dynamic module generator:

----------------------------------------------------------------
#######################################################

# provide a virtual module which has no true file

DYN_MODULE_NAME = "dynamic node classes"

def __virtual_setup():
    import new, sys
    vm = new.module(DYN_MODULE_NAME)
    sys.modules[DYN_MODULE_NAME] = vm
    
__virtual_setup()
del __virtual_setup

DYN_MODULE = __import__(DYN_MODULE_NAME)

# an alternative could be to replace the standard __import__
# function with one that can behave as expected.
# Anyway, this works.

class NodeGenerator: # Generates *no* code to declare a new class
on-the-fly
    def __init__(self, base, tag):
        import new
        self.nodeLabel = cname = base + "_" + tag # name of the class to
generate
        self.classDef = cl = new.classobj(cname, (globals()[base],), {})
        def __init__(self, id):
            self.__class__.__bases__[0].__init__(self,id)
            # which has a slight semantical difference. Guess about it!
        cl.__init__ = __init__ # function becomes an unbound method
        cl.name = tag

        # Initialize empty parameter and port defs
        cl.parameterDefs = {}
        cl.portDefs = {}

        # provide a virtual module name
        cl.__module__ = DYN_MODULE_NAME
        
    def address (self, addr):
        # Record the persistent address that the class has
        self.classDef.address = addr

        
    def defParameter (self, name, type, default, unit, range, desc):
        # Record a parameter definition
        param = {}
        param[ "type" ] = type
        param[ "default" ] = default
        param[ "unit" ] = unit
        param[ "range" ] = range
        param[ "desc" ] = desc
        
        self.classDef.parameterDefs[ name ] = param
        

    def defPort (self, name, type, direction, cardinality, desc):
        # Record a port definition
        port = {}
        port[ "type" ] = type
        port[ "direction" ] = direction
        port[ "cardinality" ] = cardinality
        port[ "desc" ] = desc
        
        self.classDef.portDefs[ name ] = port


    def declareNode(self):
        # just return the ready thing
        return self.classDef

#######################################################
----------------------------------------------------------------

and here a factory for abstract nodes:

----------------------------------------------------------------

class AbstractNodeFactory:
    def __init__(self):
        self.knownNodes = {} # The known types of AbstractNodes keyed by
addresses
        
    def getNode(self, address):
        # Is the desired node known?
        if address not in self.knownNodes.keys():
            # No => generate on-the-fly
            generator = NodeGenerator( "AbstractNode", "SomeShortLabel_"
+ address )

            # Some dummy stuff
            generator.address( address )

            generator.defParameter( "Weight", "float", "42.0", "lbs",
"41.9-42.1", "The weight of the brain" )

            generator.defPort( "Eye", "optical", "input", "2", "Optical
input" )
            generator.defPort( "Ear", "acustical", "input", "2",
"Acustical input" )
            generator.defPort( "Mouth", "acustical", "output", "1",
"Acustical output" )

            # Declare the class and get to know it            
            generator.declareNode()            
            self.knownNodes[ address ] = generator.classDef # was
generatedClass
            # store in the dynamic module as well
            node = self.knownNodes[ address ]
            setattr(DYN_MODULE, node.__name__, node)
            # keep a reference to the factory instance as well
            node.Factory = self

        # Return the class object
        return self.knownNodes[ address ]
        
    # pickling support.
    # we don't store the generated classes, but the parameters
    # needed to restore them.
    # therefore, the __getstate__/__setstate__ method
    # avoid to make use of the __dict__
    def __getstate__(self):
        return self.knownNodes.keys()
        
    def __setstate__(self, state):
        self.__init__()
        map(self.getNode, state)

class persistency:
    """wrap a node instance suitable for pickling"""
    def __init__(self, node_inst):
        factories = {}
        instances = {} # maybe we must use id() in the future
        todo = [node_inst]
        while todo:
            lookat = todo[0]
            del todo[0]
            for port in lookat.ports.values():
                for conn in port.values():
                    inst = conn[0]
                    if not instances.has_key(inst):
                        todo.append(inst)
                        instances[inst] = 1
                        factories[inst.Factory] = 1
        self.factories = factories.keys()
        self.instances = instances.keys()
        self.saveset = (self.factories, node_inst)
        
    def __getstate__(self):
        return self.saveset
        
    def __setstate__(self, state):
        factories, instance = state
        self.__init__(instance)

#######################################################


-- 
Christian Tismer             :^)   <mailto:tismer at appliedbiometrics.com>
Applied Biometrics GmbH      :     Have a break! Take a ride on Python's
Kaiserin-Augusta-Allee 101   :    *Starship* http://starship.python.net
10553 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     we're tired of banana software - shipped green, ripens at home