[Web-SIG] WSGI deployment config

[Phillip J. Eby]

I imAt 12:47 PM 8/8/2005 -0500, Ian Bicking wrote:
>OK, this is starting to become a bit more clear to me...

Cool.  :)  Sometimes the best way to get rid of confusing communication is 
to make the communication even more difficult.  :)  (e.g., banning the word 
"configuration")


>In your model with services, I think you are suggesting some middleware 
>like this will still exist.  In fact, it would look very close to the way 
>it looks in that example, except that instead of putting the Manager in 
>the WSGI environment, some service would create the manager, and both the 
>middleware and a transaction-user would use this service to get the manager.

Yes.


>So for many services some middleware would still be necessary, if the 
>service was able to do anything to the request.

Well yeah, if you want to wrap an app rather than just use the service.


>   That middleware would be mostly a shell.  That's fine with me -- that's 
> how I'm writing most of my middleware anyway, except that the "service" 
> part is relatively ad hoc, and if you use it outside of the web 
> environment you have to wire up the configuration on your own.  Which 
> isn't what I want either.

Note that you can use pipeline specs to configure arbitrary service chains, 
without WSGI even being involved.  So, to a certain extent, services can 
stand on their own.  What's interesting about that (to me anyway) is that 
if there are bridges that allow PEAK or Zope services to be used as WSGI 
services, then pipelines can be used to bridge various frameworks' service 
systems - without a web application in sight.


>If you *don't* want a middleware for every request/response-modifying 
>service, then you'd need some uber-middleware like I mentioned back in 
>http://mail.python.org/pipermail/web-sig/2005-July/001532.html -- in 
>addition to saving some frames in the call stack, that would probably make 
>pipeline specification easier.  But maybe not a whole lot easier, as 
>there's usually additional details (like ordering) that are necessary to 
>specify in the context of a web request.

Well, to me, the "uber middleware" is just an object with a generic 
function for its __call__ method, that has "before", "after", and "around" 
methods registered to do stuff like transaction wrapping, error handling, 
and any other sort of middleware-ish things.  So, it's not very "uber" in 
implementation complexity from my POV to have such a thing, and it takes 
care of many of the stacking issues.


>The spec I gave in "WSGI deployment: an experiment" 
>(http://mail.python.org/pipermail/web-sig/2005-July/001598.html) handles 
>arbitrary kinds of branching, basically by naming both applications and 
>middleware filters, and allowing application factories to call back into 
>the configuration file.  So pipeline is just another application factory, 
>just like urlmap or other kinds of branching.
>
>Maybe this could be handled with an application-building service since 
>we're passing services around anyway.

Hm.  An interesting point.  I haven't yet seen a branching/alternatives 
syntax I like though.  The big problem IMO is that a branching mechanism 
requires nesting ability, whereas pipelines are "flat and happy".  :)

Unfortunately, .ini syntax rapidly breaks down when nesting begins, which 
makes me tend to think that we should have a separate "site map" file that 
maps locations and other rules to groups of pipelines.


>>A "pipeline spec" describes how to deploy a WSGI application, optionally 
>>with middleware filters and services, by providing parameters to 
>>designated factories.  There are three kinds of factories: application, 
>>middleware, and service.  All three kinds are invoked with the parameters 
>>defined in the spec and the most-recently specified service 
>>object.  Middleware factories also receive the *next* middleware or 
>>application component defined below them in the spec.
>>An example middleware factory signature:
>>       def make_middleware(last_service, application_to_wrap, **params):
>
>It might add to the consistency if make_middleware takes the same 
>parameters as the other two factories, except it builds "middleware" (or 
>"middleware filters" to make the term less enterprisy) which are functions 
>that, when passed in an application, return an application that wraps that 
>application.  Though I would not object to a method instead of just 
>calling the factory; I think we risk a maze of function calls, all looking 
>the same.

I don't see a problem with the signature being different, to be 
honest.  Making it the same implies a similarity that doesn't exist.  If we 
were to change for consistency's sake, we should instead change the 
application factory signature to match that of middleware, and use 'None' 
for the 'application_to_wrap' in that case.  Applications and middleware 
are more alike than either of them are like services.


>I think this would address some of the configuration concerns I've had.  I 
>don't mind being very explicit in my code about how configuration is 
>acquired; I just don't want to push that work onto the person doing the 
>configuration, and I want sensible (and possibly derivative) defaults.

You can do that, sure.


>While Zope 2 gets hairy in its use of Acquisition -- essentially adding 
>dynamic scoping to the core of the system -- the basic technique is not 
>necessary correct.  Lisps get by okay with dynamic scopes, but they 
>clearly mark variables as being so typed (like 
>*current-output-stream*).  If we add dynamic-scope-like-functionality, we 
>just need to make sure it's clear where it's being used, and that it's not 
>the default so it isn't used when not necessary.

Right - explicit indirection or redirection avoids a lot of problems here.