[Python-Dev] PEP 376 proposed changes for basic plugins support

[M.-A. Lemburg]

Michael Foord wrote:
> On 02/08/2010 13:31, exarkun at twistedmatrix.com wrote:
>> On 12:21 pm, mal at egenix.com wrote:
>>> Tarek Ziad� wrote:
>>>> On Mon, Aug 2, 2010 at 3:06 AM, P.J. Eby <pje at telecommunity.com> wrote:
>>>> ..
>>>>>
>>>>> So without specific examples of why this is a problem, it's hard to
>>>>> see why
>>>>> a special Python-specific set of configuration files is needed to
>>>>> resolve
>>>>> it, vs. say, encouraging application authors to use the available
>>>>> alternatives for doing plugin directories, config files, etc.
>>>>
>>>> I don't have a specific example in mind, and I must admit that if an
>>>> application does the right thing
>>>> (provide the right configuration file), this activate feature is not
>>>> useful at all. So it seems to be a bad idea.
>>>>
>>>> I propose that we drop the PLUGINS file idea and we add a new metadata
>>>> field called Provides-Plugin
>>>> in PEP 345, which will contain the info I've described minus the state
>>>> field. This will allow us to expose
>>>> plugins at PyPI.
>>>>
>>>> IOW, have entry points like setuptools provides, but in a metadata
>>>> field instead of a entry_points.txt file.
>>>
>>> Do we really need to make Python packaging even more complicated by
>>> adding support for application-specific plugin mechanisms ?
>>>
>>> Packages can already work as application plugins by simply defining
>>> a plugins namespace package and then placing the plugin packages
>>> into that namespace.
>>>
>>> See Zope for an example of how well this simply mechanism works out in
>>> practice: it simply scans the "Products" namespace for sub-packages and
>>> then loads each sub-package it finds to have it register itself with
>>> Zope.
>>
>> This is also roughly how Twisted's plugin system works.  One drawback,
>> though, is that it means potentially executing a large amount of
>> Python in order to load plugins.  This can build up to a significant
>> performance issue as more and more plugins are installed.
>>
> 
> unittest will solve this problem by having plugins explicitly enabled in
> its own configuration system, and possibly managed through a separate
> tool like a plugins subcommand. The full package list will *only* need
> to be scanned when managing plugins, not during normal execution.
> 
> Having this distutils2 supported "plugin declaration and discovery" will
> be extremely useful for the unittest plugin system. Given that plugins
> may need configuring after installation, and tools that handle both
> activation and configuration can be provided, it doesn't seem a heavy cost.
> 
> The downside to this is that installing and activating plugins are two
> separate steps. Given that each project can have a different set of
> plugins enabled I don't see a way round it.

You might want to take a look at the Trac plugin system which
works in more or less the same way:

http://trac.edgewall.org/wiki/TracPlugins


Since applications tend to have a rather diverse set of needs for
plugins, I don't think we should add plugins support to PEP 376.

Users of applications will not want to edit a single configuration
file to maintain plugins of many different applications (they might
break some other application doing so) and sys admins
will have trouble with such a setup as well (they usually want to
have control over which plugins get used for various reasons).

In the end, you'd have a system wide plugin configuration (maintained
by the sys admin), a per user one (with local customizations) and a
per application one (providing application-specific defaults) -
which only increases complexity and doesn't really solve anything.

Instead, I'd suggest to let each application do its own little thing
to manage plugins, in a complex or simple way, with or without
configuration, and have them all live happily side-by-side.

The stdlib should really only provide tools to applications and
make useful suggestions, not try to enforce application design
choices. I think that's simply out of scope for the stdlib

Tarek:

What you might want to do is add new type fields to PEP 345,
making it easier to identify and list packages that work as
plugins for applications, e.g.

Type: Plugin for MyCoolApp

The MyCoolApp could then use the Type-field to identify all
installed plugins, get their installation directories, etc.
and work on from there.

Whether or not to use an installed plugin is really not
without the scope of Python's packaging system. This is
something the application must provide in its config file,
together with possible additional sections to configure
a particular plugin.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 02 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/