Enumerating Classes in Modules

[Terry Hancock]

On Wednesday 17 November 2004 05:00 pm, Rob Snyder wrote:
> I have a situation where I need to be able to have a Python function that
> will take all the modules in a given directory and find all the classes
> defined in these modules by name. Ultimately, I would need to loop through
> all of these, instantiating each of the classes in turn and calling a
> pre-known method of each.
> 
> Finding the name of each module is not a problem, but loading the classes
> out the module once I know the name is where I'm lost.
> 
> The idea is basically to support "plug-ins" for an application I'm building.
> I want the end user to be able to sub-class an example plug in class, then
> throw the implementation in a given directory, where I'll find it
> automatically when I need it.
> 
> I *am* still relatively new to Python, so I fully appreciate that I may be
> simply going about this all the wrong way. Any guidance or redirection
> anyone can provide would be appreciated.

You've probably heard all you needed to, but I can show you how I did it:

When I started working on VarImage, I just wanted it to do resizing, but
I quickly realized I wanted to support a wide variety of image filtering
operators, so I devised a plugin system, very similar to what you describe.

The full source can be had at: http://sourceforge.net/projects/narya-project

But the plugin finding code ( Operators/__init__.py ) just does this:

# Find and load all available plugin modules:
 
operator_path = os.path.abspath(__path__[0])
for module_file in filter(
        lambda n: n[-3:]=='.py' and n not in ('__init__.py', 'Operators.py'),
        os.listdir(operator_path)):
    #print "Loading %s" % module_file
    f, e = os.path.splitext(module_file)
    __import__(f, globals(), locals(), [])
 
The magic actually happens in the Operators/Operators.py file where I define
a dictionary called "Ops" that holds all the plugin objects as they are loaded.

Abridgement of part of my "Operator" class:

Ops = {}

class Operator:
    """
    Image operator object.
    """
    def __init__(self, id, func, ...):
        self.id = id
        self.op = func
        # [... stuff to do with actually using the operator ...]
        # Register:
        Ops[id] = self

I didn't want the plugin author to have to understand OOP -- just write a
function with the right profile.  So I use an instantiation of the class to build
each operator plugin class instance based on the function they provide. All
the plugin author has to do is call this class at the end of their module,
with the appropriate function as argument (example from Operators/pil_crop.py):

The essentials are just:

import Operators

def trim(...):
    "Documentation text which my help method reads"
    return ...

Operator('trim', trim, 0, 1, (int,))
 
This says to define an operator with name "trim", calling the
function trim, which can take a minimum of 0 arguments and a
maximum of 1 argument, which must be an int. (not really
type-checking, it just says what it should use to convert
the string it will get before calling this function).

I think the main conceptual difference here is that instead
of trying to use magical class discovery methods, I just let
the plugin author tell me which functions he wants me to make
into operators, and try to make that as simple as possible.

Does this help you at all?

Cheers,
Terry

--
Terry Hancock ( hancock at anansispaceworks.com )
Anansi Spaceworks  http://www.anansispaceworks.com