[Web-SIG] PasteDeploy 0.1

[Ian Bicking]

Phillip J. Eby wrote:
> At 12:44 PM 8/22/2005 -0500, Ian Bicking wrote:
> 
>> Hmm... it's also just occurred to me that filters should be easier to
>> define.  In almost all cases I find I want to curry the configuration so
>> it can be applied at the same time the wrapped application is passed in.
>>   I might add another protocol for that.
> 
> 
> I think the format is improving, as it was now clear enough for me to 
> figure out what I'd like to change.  ;-)
> 
> I stole this example off your blog, and then rewrote it using a slightly 
> more advanced version of my last syntax proposal:
> 
>     # Put one login system in front of the entire site
>     #
>     [login wrapper from Paste]
>     database = "mysql://localhost/userdb"
>     table    = "users"
> 
>     # Then this passes different path prefixes to different apps
>     #
>     [urlmap from Paste]
>     "/"     = static()
>     "/cms"  = auth(filebrowser_app())
>     "/blog" = blog()

One aspect of paste.deploy that wasn't shown in that example is that 
it's easy to refer to other configuration files.  It would actually be 
more realistic to do:

   [composit:app]
   use = egg:Paste#urlmap
   / = config:static_root.ini
   /cms = config:filebrowser.ini
   /blog = config:blog.ini

And if filebrowser.ini defined an authentication filter named "auth", 
you could add this to blog.ini to reuse that configuration:

   [filter-app:main]
   use = config:filebrowser.ini#auth
   next = blog

   [app:blog]
   ....

And so forth.  I think this will be really useful to me (when I have my 
sysadmin/deployer hat on) -- it's something I left out of my own 
previous specs, but I think incorrectly.

>     # variables used later
>     #
>     [config = vars]
>     admin_email = "me at example.com"
>     document_root = "/home/me/htdocs"

This seems useful.  I had thought about some way of using the globals in 
expressions; but with pure-string expressions it's not easy to do much 
of interest.

>     # a very simple app...
>     #
>     [static = static from Paste]
>     document_root = config.document_root
> 
>     # the login filter should give us a username; this just restricts
>     # who can access
>     #
>     [auth = auth wrapper from Paste]
>     require_role = "admin"
>     admin_email = config.admin_email
> 
>     # this application is distributed in an egg
>     #
>     [filebrowser_app = filebrowser from FileBrowser]
>     document_root = config.document_root
>     admin_email = config.admin_email

However, in paste.deploy there does remain real global configuration, so 
you wouldn't have to manually copy in values from the globals.  While 
admittedly it makes the interface slightly less elegant from the Python 
side, I think it's an important feature.

>     # In this case the app isn't distributed as an Egg with
>     # entry_points, so we manually create a glue function blog_app
>     # and just invoke it here
>     #
>     [blog = myglue.apps:blog_app]
>     admin_email = config.admin_email
> 
> 
> Most of the above should be pretty obvious, but a few points anyway:
> 
> * This format is generic; it has nothing to do with WSGI in particular 
> and can be used to assemble any component tree.  It also supports 
> implementing the "wsgi services" concept.

Ditto paste.deploy.  Not all of the bits are well defined in the 
implementation, but there's nothing inside or out that's connected to WSGI.

> * Argument names can be either an identifier or a quoted string

I tried to avoid anything fancy; if I was going to do something fancy 
I'd feel a need to look at all the configuration formats currently for 
Python, and if not reuse them at least steal from them.

But it's clear that plain ConfigParser parsing is pretty lame.

> * You can use factories from a default group (e.g. 'vars' above might 
> effectively be short for 'vars from WSGIUtils')

How is that default group determined?  What is a "group"?

> * named sections ("[name = ...]") have to come after the unnamed 
> sections, and they are turned into "curried" factory objects that are 
> available in the eval() namespace used for all expressions.  When called 
> in an expression, they can accept keyword arguments to override the 
> defaults in the named section.  They have properties with the same names 
> as the values defined in that section.

The properties are fine; I can't say the calling syntax appeals to me 
particularly.

> * The first part of a section (after the "name=", if any) is an import 
> spec for a factory, or if it's followed by "from" or "wrapper from", 
> then it's the name of an entry point that advertises a factory.

How do you determine the entry point type?  Or is there one entry point 
type for anything available in a configuration file?  paste.deploy 
defines an entry point type for each kind of object.

> * "wrapper" means that the factory will be called with two positional 
> arguments; non-wrappers are called with one argument.  Named wrappers 
> can be passed a positional argument if used in an another factory 
> argument expression - this will be the object they should wrap.

This part is unclear to me.

> * The last unnamed section is the effective "result" of parsing the 
> file, although it will be wrapped by any contiguous preceding "wrapper" 
> sections

This isn't clear to me when reading the configuration file.  INI files 
are flat, and I wouldn't expect them to be usefully ordered, especially 
in a way that puts particular importance on the last unnamed section.

I'd feel more comfortable with a nested configuration format in that case.

> The parser for this format would of course be considerably more complex 
> than the Paste-Deploy parser (especially since evaluation would be done 
> lazily), but I think the syntax is both cleaner and more powerful.  The 
> factory signatures are:
> 
>     def non_wrapper_factory(parent_component, **kw):
>         ...
> 
>     def wrapper_factory(child_component, parent_component, **kw):
>         ...
> 
> With the parent/child parameters always being supplied positionally.  
> The idea is that parent_component will be used to create a chain of 
> service contexts, and child_component is an application to be wrapped by 
> middleware.
> 
> I've thought this through enough that I know how I could implement all 
> of the features shown, but it may be a week or two at least before I 
> could try hacking together an implementation.  Also, the services side 
> of it isn't really fleshed out yet, and it may also be that we need to 
> provide some simple "builtin" functions in the eval() namespace to do 
> things like lookup services or load other deployment files, etc.

I dunno... I can't say much about the services, because I don't really 
know what you intend with those.  These are some things I like about 
your example:

* More structured/richer section names could be good; paste.deploy's 
"use" could go as a result.

* A clear notion of evaluation and variables would be nice.

* A config format with good quoting rules is called for.  ConfigParser 
isn't anything more than a stop-gap.

But some things I don't like:

* Using ordering in a syntax that doesn't feel ordered or nested.

* Using function composition to represent application/filter 
composition.  But only sometimes.

* "name from egg_spec" reads nice on one level, but is vague on another 
level.  Even if "egg:egg_spec#name" doesn't read well, I think it is 
nicely self-describing.

* eval() scares me a bit; if I used eval() I would feel a need to keep 
sufficient information around to do proper tracebacks that include the 
source configuration file.  But all-strings isn't great either. 
Evaluation without conditionals seems like it goes only half-way; OTOH 
conditionals get to something too complex for configuration.  So however 
it goes, configuration should be somewhere in the middle of completely 
dumb (ConfigParser, unevaluated values), and completely general (Python 
code).  Where in the middle I'm unsure.

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org