[Distutils] Adding entry points into Distutils ?

[Doug Hellmann]

On May 5, 2009, at 9:33 AM, Tarek Ziadé wrote:

> On Tue, May 5, 2009 at 2:41 PM, Doug Hellmann  
> <doug.hellmann at gmail.com> wrote:
>>> I am confused with the role of this "man in the middle". In my mind
>>> there are plugins on one side,
>>> and host applications that consumes them if they wish on they  
>>> other side.
>>>
>>> Do you have a use case we can share for mutual comprehension ?
>>
>> I think we have a different view about how the plugins should  
>> work.  It
>> sounds like you're advocating a model where all plugins are  
>> registered
>> globally, an application can search the global registry for plugins  
>> based on
>> categories, and then some administrator enables/disables them  
>> locally for
>> each app.
>>
>> I don't want new functionality available to an application just  
>> because
>> someone has permission to install a package somewhere in the  
>> PYTHONPATH.  I
>> would rather have plugins added to an app through an explicit  
>> configuration
>> step of some sort.
>
> That is basically how host applications are dealing with entry points:
> an explicit configuration. The implicit part is just happening when
> you look for the plugins.
>
> Let's take a real example : I am working on a program called  
> Atomisator
> which can be extended through plugins.
>
> For example people can provide a plugin to read data that are located
> somewhere (like a rss feed), and return
> a sequence of entries.
>
> So in atomisator, I have a configuration file where I decide which
> plugin to use to read my data:
>
> """
>   [atomisator]
>
>   reader = rss
> """
>
> When reading it, Atomisator knows that it needs the "rss" plugin, and
> will browse in entry points
> with a specific group called "atomisator.rss". If it finds it, it uses
> it, otherwise it throws an exception. ("Install it!")

> That allows people to create their own plugins in separate packages,
> and use them by tweaking the configuration
> file. The only think that entry point provided here is an automatic
> registration at installation time of the "rss" plugins,
> so my Atomisatior application can discover then load it at run time.

I don't think I understand the difference between the step you're  
calling "discover", scanning the registry, and actually loading the  
plugin.  Does "discovering" the plugin involve importing any of its  
code?

> So in your way of seeing thing, you'd rather see this registration
> mechanism at the application level,
> but the you need to provide specific installation instructions for
> people that want to add plugins.
> e.g. "put your package in this /plugins directory" for example.

Sort of.  I see the installation and configuration of entry points as  
2 steps:

1. some variation of "python setup.py install"
2. update the configuration of app to tell it where to find the plugin

Some applications may provide automated tools for step 2, and I could  
see great benefit to making the loading and registration of entry  
points part of the standard library, as long as the registration is  
maintained per-application instead of site-wide.

> I am advocating that the entry point mechanism is handy because it
> relies on an existing mechanism : "install your package"
> and it allows plugin sharing amongst applications.

How then do I install a package but *not* have it available for an  
application?  Suppose, for example, that I have several copies of a  
web app installed for different customers, but I want to prevent some  
of them from using an advanced plugin of some sort.

> But I see the caveats you are explaining, and I understand them now;
>
> So, what if we didn't have these entry points installed globally when
> the package is installed,
> but just a configuration file in the host application level to point  
> them ?
>
> a configuration file that reunites all entry points an application
> uses. For the Atomisator example:
>
>  [atomisator.reader]
>  rss = somepackage.somemodule:MyPluginClass

Yes!  We can figure out the exact spelling, but we're talking about  
the same thing now.  If we use dotted notation all the way  
("somepackage.somemodule.MyPluginClass") then it's simple to just  
import the thing directly.

> This would let the application consume the plugins pointed by this
> configuration file
> and this would remove the implicit part you don't like. I'd be very
> happy with such a plugin system on my side.

If you follow the model of the logging module and provide an easy  
"load plugins from an ini file", then all python apps have the feature  
of making it easy to load plugins, developers have a standard API to  
code to, and administrators don't have to worry about plugins  
"sneaking" into an app.

> And this would fit I think in Distutils needs since we can configure
> it through three levels of configuration files
> distutils.cfg, pydistutils.cfg and setup.cfg

That sounds good.

Doug