[Web-SIG] Entry points and import maps (was Re: Scarecrow deployment config

[Phillip J. Eby]

[cc:'d to distutils-sig because this is mostly about cool uses for the new 
EntryPoint facility of setuptools/pkg_resources]

At 12:01 AM 7/26/2005 +0100, James Gardner wrote:
>Hi Phillip,
>
>>There's one other advantage: this format will hopefully become as 
>>successful as WSGI itself in adoption by servers and applications.
>>Hopefully within a year or so, *the* normal way to deploy a Python web 
>>app will be using a .wsgi file.
>>
>>Beyond that, we can hopefully begin to see "Python" rather than 
>>"framework X" as being what people write their web apps with.
>
>Well that would be absolutely wonderful but also looking fairly likely 
>which is great news. I've got to say a massive thank you for the eggs 
>format and easy install as well.. Python was really crying out for it and 
>it will be phenomenally useful. I've split all my code up as a result 
>because there is no need to worry about people having to install lots of 
>packages if it is all done automatically.
>
>One thought: I'd ideally like to be able to backup a WSGI deployment to 
>allow it to be easily redeployed on another server with a different 
>configuration or just restored in the event of data loss. This would 
>probably just involve making a zip file of all data files (including an 
>SQL dump) and then redistributing it with the .wsgi file. Have you had any 
>thoughts on how that could be achieved or is that something you wouldn't 
>want the .wsgi file to be used for? Whatever software installed the 
>dependencies of the .wsgi file would need to be aware of the data file and 
>what to do with it, perhaps simply by calling an install handler 
>somewhere? Umm, all getting a bit complicated but I just wondered if you 
>had had any thoughts of that nature?

Well, you could define another set of entry point groups, like 
"wsgi.middleware.backup_handlers", which would contain entry points with 
the same names as in middleware, that would get called with the same 
configuration arguments as the application factories, but would then do 
some kind of backing up.  Similarly, you could have an entry point group 
for restoration functions.  These would have to defined by the same egg as 
the one with the factory, of course, although perhaps we could make the 
entry point names be the entry point targets instead of using the original 
entry point names.  That additional level of indirection would let one egg 
define backup and restore services for another's factories.  Perhaps the 
backup functions would return the name of a directory tree to back up, and 
the restore functions would receive some kind of zipfile or archive.

Obviously that's a totally unbaked idea that would need a fair amount of 
thought, but there's nothing stopping anybody from fleshing it out as a 
tool and publishing a spec for the entry points it uses.


>Oh sorry, another quick question: Is there any work underway auto-document 
>eggs using some of the docutils code if an appropriate specifier is made 
>in the egg_info file saying the egg is in restructured text or similar? 
>Would that be something you would be willing to include as a feature of 
>easy_install or is it a bit too messy? I'd love to be able to distribute a 
>.wsgi file and have all the documentation for the downloaded modules auto 
>created. If only some of the modules supported it it would still be quite 
>handy.

I'm having a little trouble envisioning what you mean exactly.  All that's 
really coming across is the idea that "there's some way to generate 
documentation from eggs".  I'd certainly like to be able to see tools like 
epydoc or pydoc support generating documentation for an egg.  However, 
there's a fair amount of balkanization in how you specify inputs for Python 
documentation tools, not unlike the previous balkanization of web servers 
and web apps/frameworks.  Maybe somebody will come up with a similar lingua 
franca for documentation tools.

With respect to adding more cool features to setup(), I plan to add a 
couple of entry point groups to setuptools that would support what you have 
in mind, though.  There's already a distutils.commands group that allows 
you to register setup commands, but I also plan to add egg_info.writers and 
distutils.setup_args.  The setup_args entry points would have the name of a 
parameter you'd like setup() to have, and would be a function that would 
get called on the Distribution object during its finalize_options() (so you 
can validate the argument).  The egg_info.writers group will define entry 
points for writing metadata files as part of the egg_info command.

Last, but not least, I'll add a 'setup_requires' argument to setup() that 
will specify eggs that need to be present for the setup script to run.

With these three things in place, tools like the build_utils or py2exe and 
py2app won't have to monkeypatch the distutils in order to install 
themselves; they can instead just define entry points for setup() args and 
the new commands they add.  And for your documentation concept, this could 
include document-specification arguments and an egg_info.writers entry 
point to put it in the EGG-INFO.  Packages using the arguments would have 
to use 'setup_requires', however, to list the eggs needed to process those 
arguments.

My idea for stuff like this was mainly to support frameworks; for example 
if an application needs plugin metadata other than entry points, it can 
define an egg that extends setuptools with the necessary setup arguments 
and metadata writers.  Then, when you're building a plugin for the tool, 
you just setup_requires=["AppSetup"], where "AppSetup" is the egg with the 
setuptools extensions for "App".  (Most apps will want their setuptools 
extensions in a separate egg, because the app itself may need the same 
extensions in order to be built, which would lead to a hairy 
chicken-and-egg problem.  setuptools itself was a little tricky to 
bootstrap since it finds its own commands via entry points now!)


>Thanks for the answers anyway, the whole thing looks tremendously exciting!

That's because it is.  :)