simulating #include in python

[Nick Craig-Wood]

We are currently investigating whether to move the data files from our
application into python for ease of maintenance.  Each data item turns
into a class definition with some class data.  The python approach
looks great, but there is one feature that we'd like to have.

Currently the data files can include other data files.  Typically
thats used to import standard definitions from somewhere else (rather
like #include in C - conceptually its a completely textual import).
We use them something like this

  include("../../standard_definitions")
  include("../shared_definitions")

  class A(base_from_standard_definitions): pass
  class B(A): pass

  include("more_definitions")

The includes act much more like #include than import - all the symbols
from the file before the include() must be available to the included
file and the included file must export all its symbols back to the
parent.

These can of course be re-arranged to work in a pythonic way using
'from x import *' and putting "../.." on sys.path instead of the
includes.  However there are over 500 of these files so I'd prefer a
more automatic solution which doesn't require re-arrangement in the
interim. (The re-arrangement is needed because "more_definitions"
above might refer to A, B or anything defined in
standard/shared_definitions leading to mutually recursive imports and
all the pain they cause)

I have implemented a working prototype, but it seems such a horrendous
bodge that there must surely be a better way!  I'd really like to be
able to run an __import__ in the context of the file thats running the
include() but I haven't figured that out.

Here is the code (avert your eyes if you are of a sensitive nature ;-)
Any suggestions for improvement would be greatly appreciated!

def include(path):

    # Add the include directory onto sys.path
    native_path = path.replace("/", os.path.sep)
    directory, module_name = os.path.split(native_path)
    if module_name.endswith(".py"):
        module_name = module_name[:-3]
    old_sys_path = sys.path
    if directory != "":
        sys.path.insert(0, directory)

    # Introspect to find the parent
    # Each record contains a frame object, filename, line number, function
    # name, a list of lines of context, and index within the context.
    up = inspect.stack()[1]
    frame = up[0]
    parent_name = frame.f_globals['__name__']
    parent = sys.modules[parent_name]

    # Poke all the current definitions into __builtin__ so the module
    # uses them without having to import them
    old_builtin = __builtin__.__dict__.copy()
    overridden = {}
    poked = []
    for name in dir(parent):
        if not (name.startswith("__") and name.endswith("__")):
            if hasattr(__builtin__, name):
                overridden[name] = getattr(__builtin__, name)
            else:
                poked.append(name)
            setattr(__builtin__, name, getattr(parent, name))

    # import the code
    module =  __import__(module_name, parent.__dict__, locals(), [])

    # Undo the modifications to __builtin__
    for name in poked:
        delattr(__builtin__, name)
    for name, value in overridden.items():
        setattr(__builtin__, name, value)

    # check we did it right! Note __builtin__.__dict__ is read only so
    # can't be over-written
    if old_builtin != __builtin__.__dict__:
        raise AssertionError("Failed to restore __builtin__ properly")

    # Poke the symbols from the import back in
    for name in dir(module):
        if not (name.startswith("__") and name.endswith("__")):
            setattr(parent, name, getattr(module, name))

    # Restore sys.path
    sys.path = old_sys_path

-- 
Nick Craig-Wood <nick at craig-wood.com> -- http://www.craig-wood.com/nick