[Web-SIG] WSGI deployment config

[Ian Bicking]

OK, this is starting to become a bit more clear to me...

Phillip J. Eby wrote:
>> A service would have to have some way of providing its useful code to 
>>utilities etc as well as deploying middleware.
> 
> 
> I think maybe you're confusing something here.  I'm suggesting that there 
> be a chain of service providers, and that the WSGI API to load a pipeline 
> should return both a top-down middleware-to-app chain, and a bottom-up 
> service-to-service chain.  Thus, a utility program could load a WSGI file 
> and gain access to the service chain, ignoring the middleware.
> 
> But, I'm not saying that services are *part* of the middleware chain; 
> middleware components get created with access to the middleware chain, but 
> the services themselves are not middleware.

So, thinking back to the transaction middleware I speculated about: 
http://blog.ianbicking.org/more-perfect-app-server-wsgi-transactions.html

In your model with services, I think you are suggesting some middleware 
like this will still exist.  In fact, it would look very close to the 
way it looks in that example, except that instead of putting the Manager 
in the WSGI environment, some service would create the manager, and both 
the middleware and a transaction-user would use this service to get the 
manager.  (In case it creates confusion, I think Zope uses a different 
term for the manager; maybe it is simply a "transaction", I can't 
remember now)

So for many services some middleware would still be necessary, if the 
service was able to do anything to the request.  That middleware would 
be mostly a shell.  That's fine with me -- that's how I'm writing most 
of my middleware anyway, except that the "service" part is relatively ad 
hoc, and if you use it outside of the web environment you have to wire 
up the configuration on your own.  Which isn't what I want either.

If you *don't* want a middleware for every request/response-modifying 
service, then you'd need some uber-middleware like I mentioned back in 
http://mail.python.org/pipermail/web-sig/2005-July/001532.html -- in 
addition to saving some frames in the call stack, that would probably 
make pipeline specification easier.  But maybe not a whole lot easier, 
as there's usually additional details (like ordering) that are necessary 
to specify in the context of a web request.

>>it is helpful for a local application to inherit configuration from 
>>another source of components, possibly in the way I suggested. I don't 
>>think branching really fits into that model so how are you envisaging 
>>deployments?
> 
> 
> The branching was for saying things like "/foo goes to pipeline A, and /bar 
> goes to pipeline B".

The spec I gave in "WSGI deployment: an experiment" 
(http://mail.python.org/pipermail/web-sig/2005-July/001598.html) handles 
arbitrary kinds of branching, basically by naming both applications and 
middleware filters, and allowing application factories to call back into 
the configuration file.  So pipeline is just another application 
factory, just like urlmap or other kinds of branching.

Maybe this could be handled with an application-building service since 
we're passing services around anyway.

> A "pipeline spec" describes how to deploy a WSGI application, optionally 
> with middleware filters and services, by providing parameters to designated 
> factories.  There are three kinds of factories: application, middleware, 
> and service.  All three kinds are invoked with the parameters defined in 
> the spec and the most-recently specified service object.  Middleware 
> factories also receive the *next* middleware or application component 
> defined below them in the spec.
> 
> An example middleware factory signature:
> 
>       def make_middleware(last_service, application_to_wrap, **params):

It might add to the consistency if make_middleware takes the same 
parameters as the other two factories, except it builds "middleware" (or 
"middleware filters" to make the term less enterprisy) which are 
functions that, when passed in an application, return an application 
that wraps that application.  Though I would not object to a method 
instead of just calling the factory; I think we risk a maze of function 
calls, all looking the same.

Then the higher-level operation is "build something of type foo", where 
foo is a WSGI application, a WSGI middleware filter, a service, or 
something else.

> Example application and service factory signatures:
> 
>       def make_app(last_service, **params):
> 
>       def make_service(last_service, **params):
> 
> Just as the middleware-to-application links form a "downward" chain of 
> responsibility for handling WSGI requests, the service-to-service links 
> form an "upward" chain of responsibility for acquiring service 
> components.  There needs to be a specification for how to search the chain; 
> for example we could have a 'get_service(key)' method required on service 
> components, and if the service doesn't recognize the key it just calls 
> 'last_service.get_service()'.
> 
> In some circumstances, the same value or object is needed as a factory 
> parameter for more than one component.  In these cases, it would be useful 
> to be able to have a way to specify shared parameters in the 
> specification.  Ordinarily, these shared parameters will be defined at some 
> "high level" of the overall system, such as in a server-wide pipeline spec, 
> and then acquired in "low level" pipeline specs for specific areas of the 
> server or individual application components.  We can thus envision a 
> "shared parameter service" interface for publishing values that need to be 
> used often, and an API in the pipeline spec to indicate that a parameter 
> should be retrieved from the nearest shared parameter service that offers a 
> value for a given name.
 >
> This approach is superior to using a common namespace for parameters, 
> because the level of abstraction at which shared parameters are defined is 
> more likely to be concepts like "system administrator e-mail", but that 
> value might then be used for more specific component parameters like "email 
> errors to" and "administrator login ID".  So, being able to say that the 
> "email_errors_to" parameter for a given component should be looked up from 
> "sysadmin_email" in the shared parameter service allows for parameters to 
> be cleanly shared between components.

I think this would address some of the configuration concerns I've had. 
  I don't mind being very explicit in my code about how configuration is 
acquired; I just don't want to push that work onto the person doing the 
configuration, and I want sensible (and possibly derivative) defaults.

While Zope 2 gets hairy in its use of Acquisition -- essentially adding 
dynamic scoping to the core of the system -- the basic technique is not 
necessary correct.  Lisps get by okay with dynamic scopes, but they 
clearly mark variables as being so typed (like *current-output-stream*). 
  If we add dynamic-scope-like-functionality, we just need to make sure 
it's clear where it's being used, and that it's not the default so it 
isn't used when not necessary.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org