[Web-SIG] wsgiconfig design

[Jim Fulton]

On Jul 6, 2007, at 11:41 PM, Ian Bicking wrote:

> Every so often I get in this cleanup/redux mood where I feel a need to
> revisit things I've done before in an attempt to Do Them Right.
>
> We've discussed Paste Deploy here before, and I'm thinking about  
> Redoing
> It Right.

Cool.

> I thought I'd share some thoughts on the design:
>
> I still am quite happy with the entry points from Paste Deploy, and  
> plan
> to keep them,

Me too.  In fact, I'd really like them to have their own identity  
independent if Paste Deploy. People should start supporting these  
entry points even if they use some other application to put them  
together.

I do have one potential complaint about the entry-point APIs.  The  
applications my company builds have configurations that are too  
complex to fit in a single config-parser section.  To handle these  
configurations, I'd need to be able to read multiple sections, or to  
refer to an external configuration.  I think the later is the current  
recommended approach for Paste Deploy.  If you want to keep that  
approach, then the existing entry-point APIs are fine.  (I  
personally, want to be able to put all of my configuration in a  
single file, but zc.buildout lets me do that, so I don't need Paste  
Deploy to do that for me.)

...

> I  will probably rename them, but also support the old Paste Deploy  
> names.

Why?

...

> There will be a few levels of pluggability.  The first is the loader
> itself, which will default to the wsgiconfig loader itself.  This is
> only applicable to file-based configuration.  When you load up the
> application from a file, it will search for a #! line that specifies a
> loader, which is the name of a distribution or module.  It will search
> the distribution for some entry point (wsgiconfig.config_loader:main),
> and that entry point is a callable like:
>
>    def config_loader(filename, options, object_type):
>        return wsgi_application

which section would it get these from?

> options is a flat dictionary of options that are passed in, which the
> config loader can use at its discretion.  A common way to use it would
> be for variable substitution.  This allows for things like "paster  
> serve
> config.ini var1=value var2=value", for ad hoc customization.  It  
> returns
> a single application.  (This does mean that a feature from Paste  
> Deploy
> is lost, where you could do "paster serve config.ini#section_name" to
> load a specific application from several defined in a file -- but you
> are less likely to get dead sections or confusing config file  
> factorings).
>
> object_type is the kind of object we want to get out.  Here I'll only
> specify 'wsgi.application'.  'wsgi.server' will probably also be
> implemented, but that's all I plan for now.  'wsgi.appserver' or
> something might be possible, for the process manager that runs an  
> entire
> application.

I don't really follow this. Maybe an example would help.

...

> Unlike Paste Deploy, section names will not be arbitrary.  A section
> name has a prefix and name.  The prefix, as in Paste Deploy, says what
> you are describing.  The default prefix is "app:"; you can also give
> "middleware:".  Prefixes not recognized will be ignored.  A possible
> prefix might be "logging:" for logging, which if I don't implement it
> will be initially ignored (but someone else could handle it).

IMO, it would be nice *not* to reinvent yet another logging  
configuration handler. The standard library already defines one. If  
we don't like it, we should make it better.

>   Similarly
> the server as with Paste Deploy can be defined with "server:".  For  
> now
> all we're concerned with is applications, middleware, and composites.

Maybe I'm missing your point, but I thought the value of Paste Deploy  
was to be able to have a way to define and end-to-end configuration  
of applications, middleware and server.

> The applications and middleware are grouped together using the names.
> That is, if you have an application "/" and a middleware  
> "middleware:/",
> then the middleware wraps that application.  Middleware sections can
> have trailing numbers to indicate ordering and keep section names
> unique.  Thus "middleware:/ 1", "middleware:/ 2", etc.  Negative  
> numbers
> and floats are allowed.  Anything but trailing numbers is considered
> part of the name; thus names can have parameters or other structure.

Hm.  Sounds a bit too magic to me.  Maybe an example will make it  
look better. :)

> All the applications in the server are put in a single dictionary, and
> that is based to the composer.  The composer by default is urlmap  
> (which
> also includes optional host-based dispatch).  You can specify another
> composer with a global option "composer = (specifier)"

I'm not sure how I feel about that.

> The sections still have a "use" option, which indicates what  
> implements
> the option.  It will take "egg:", and dotted module/object names, and
> module names may have trailing entry point specifiers (if the object
> doesn't implement the "preferred" interface for that kind of output).
> You cannot use "config:" anymore, instead that will be handled by the
> config file format.

Good. I found this dual use of "use" to be confusing.

> The config files will use INITools, which is similar to  
> ConfigParser but
> a bit nicer.  It will include improved string substitution, including
> similar constructs to what zc.buildout has.  While still keeping
> applications firmly separated from the config files, I'm planning on
> paying close attention to call signatures and exceptions to give good
> error messages.  E.g., if you raise something like TypeError("...  
> got an
> unexpected keyword argument 'X'") I'll figure out where the keyword
> arguments came from and improve the text of that error message.  Not
> perfect, but should be passable, and better than what Paste Deploy  
> does now.

One suggestion to improve error detection/warning is to use  
RawConfigParse rather than ConfigParse and to track option access.  A  
common mistake when writing configurations is to misspell something.   
You end up with options that are ignored because they are misspelled.  
This sort of error can be hard to spot.  Handlers could complain  
about unused options, but this is hard to do if a DEFAULT sections  
causes options to appear in all sections.  Also, expecting handlers  
to do this sort of error checking is a bit if a burden on handler  
writers. This may not be such a problem for Paste Deploy handlers as  
a is for buildout recipes.  In buildout, I decided to shift this  
error checking to buildout itself.  Buildout tracks option access and  
warns about unused options.  This can be very helpful and puts no  
burden on recipe writers.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org