Python plugins

[Christian Tismer]

Tobias Klausmann wrote:
> Hi there,
> 
> For an open source project that needs to be extensible by its
> users, I would like to create a plugin architecture. Mind you,
> I'm not trying to use Python as a plugin language only like XChat
> or Gnumeric do. Let me illustrate:
> 
> I have a directory somewhere in PYTHONPATH (let's just assume
> it's in the CWD) that contains lots and lots of *.py files with a
> defined set of functions (or, at least one: register()). What I'd
> like to do now is being able to load these plugins at program
> start and reload them if I'd like to later. Problem is that the
> program has to figure out by itself what *.py files are there -
> and I have to make sure somehow that I get a grip on the loaded
> files.

You can avoid exec at all.
The PIL package (Python Imaging Library) from Fredrik Lundh
has a similar problem:
There are lots of image drivers in a directory, and PIL
cannot know in advance what's there. Here an excerpt from
Image.py, that shows what it does:

"""
def init():
     "Load all file format drivers."

     global _initialized
     if _initialized >= 2:
         return

     visited = {}

     directories = sys.path + [os.path.dirname(__file__)]

     # only check directories (including current, if present in the path)
     for directory in filter(isDirectory, directories):
         fullpath = os.path.abspath(directory)
         if visited.has_key(fullpath):
             continue
         for file in os.listdir(directory):
             if file[-14:] == "ImagePlugin.py":
                 f, e = os.path.splitext(file)
                 try:
                     sys.path.insert(0, directory)
                     try:
                         __import__(f, globals(), locals(), [])
                     finally:
                         del sys.path[0]
                 except ImportError:
                     if DEBUG:
                         print "Image: failed to import",
                         print f, ":", sys.exc_value
         visited[fullpath] = None

     if OPEN or SAVE:
         _initialized = 2
"""

...

> Now I can just "import pi" and execute pi.rel() which in turn
> loads all the modules that are there ('cept the ones that only
> exist as pyc or pyo, but that's okay). But now I have two
> problems: first, I have no idea how to call, say
> pi.someplugin.dostuff(args) except constructing each and every
> call as a string and exec() it - which is both error-prone and
> awkward IMHO.

If you executed a proper __import__, all modules and funcs should
be accessible in the normal way.
There is no need to exec.

> Second, reloading just does not seem to work right as Python is
> too smart ;) for me: it knows all of them have been loaded. So I
> added the reload code above but to no avail.

Reloading should work. If you have imported some module into
your current one, reload(module) should work as expected.
You will get into trouble if you imported the contents of
a module like "from mod import *", since these identifiers will
not get refreshed when you reload. You would need to repeat
the from... import after reloading, since it copies the
identifiers again to your namespace.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/