"Plugin" architecture - how to do?

[anglozaxxon at gmail.com]

I'm making a program that consists of a main engine + plugins.  Both
are in Python.  My question is, how do I go about importing arbitrary
code and have it be able to use the engine's functions, classes, etc?
First, here's how it's laid out on the filesystem:

-mainstarterscript.py
-<engine>
  -__init__.py
  -[whole bunch of files]
-<plugin>
  -__init__.py
  -main.py
  -[whole bunch of files]

So you'd execute the whole thing with ./mainstarterscript.py, which
imports engine.  Engine initializes, then executes plugin somehow.
Plugin needs to be able to call functions in engine, and vice versa.
Although various IPC methods would probably work, I think there's
gotta be an easier and more straightforward way to do it.

I use execfile to execute the plugin's __init__.py script.  I insert
some globals from engine into it so that __init__ can use them.
__init__.py also imports plugin/main.py, which uses functions the
engine functions, and so must import __init__.py.  The problem is,
when main imports __init__, the inserted globals aren't there, so it
dies.  So I've tried to make some globals in __init__ that simply map
them to the engine's globals.  Here's roughly how:
#plugin/__init__.py

engine_globals = {}

if __name__ == '__builtin__': #run via execfile
  global engine_globals
  engine_globals = globals() #I don't really want all the globals, but
in my real code I go through them one by one
else: #imported from one of the other scripts in plugin dir.
  pass

import main #among other things
# end __init__.py

# plugins/main.py
import __init__

__init__.engine_globals.['some_function'](args)
#do other stuff
#end main.py

#engine/__init__.py
def some_function(args):
  pass

sys.argv.append('plugins')
execfile('plugins/__init__.py')

When main.py imports plungin/__init__, it reinitializes engine_globals
to {}, of course.  But I can't think of a way not to initialize it to
something, thus overwriting the original, because it needs to be in
the namespace..  Essentially, I want a global to not reset itself if
it's already set.

So how do I go about doing this?  Am I on the right track?  Should I
be __import__'ing plugins instead of execfile'ing them?

Thanks,
Nick