Best practices for dynamically loading plugins at startup

[Jeff Schwab]

Christoph Haas wrote:
> Dear coders...
> 
> I'm working on an application that is supposed to support "plugins".
> The idea is to use the plugins as packages like this:
> 
> Plugins/
>   __init__.py
>   Plugin1.py
>   Plugin2.py
>   Plugin3.py
> 
> When the application starts up I want to have these modules loaded
> dynamically. Users can put their own plugin modules into the
> Plugins/ directory and the application should know about it.
> 
> Since I don't know which plugins have been put into that directory
> I cannot just "import Plugin1, Plugin2, Plugin3" in the "__init__.py".
> So I need to find out the *.py there and load them during startup.
> I could do that with a "walk" over that directory.
> 
> Each plugin is supposed to be a class derived from a general
> "Plugin" superclass. I just don't know how to 'register' every
> plugin. The main application needs to know which plugin classes
> there are. On IRC I was recommended walking through all objects
> and finding out if the class is a subclass of "Plugin". Another
> person recommended using metaclasses to automatically register
> the plugin in a global list.
> 
> Since I have only little real-life Python knowledge I wonder what the
> best practice for this kind of problem is.
> 
> I looked at the "supybot" IRC bot to get an idea how plugins are handled
> there. Unfortunately it was still a bit over my (python) head.

I recently came up against this exact problem.  My preference is to have 
the plugin writer call a method to register the plugins, as this allows 
him the most control.  Something along these lines:

	class A_plugin(Plugin):
		...

	class Another_plugin(Plugin):
		...

	register( A_plugin )
	register( Another_plugin, optional_custom_registration_parameters )


I also like the Mark Pilgrim approach of having the plugins follow a 
strict naming convention, then searching for them with hasattr or by 
grepping each plugin module's namespace.  E.g., I have a build script 
written in python that I use instead of the "make" utility; for each 
target, I define a method called make_<target> in my plugin module, like 
this:

	function = "make_%s" % target
	if hasattr(module, function):
		getattr(module, function)()
	else:
		print >>sys.stderr, 'Unknown target "%s"' % target