[Web-SIG] WSGI deployment use case

[Phillip J. Eby]

At 08:29 PM 7/25/2005 -0500, Ian Bicking wrote:
>Right now Paste hands around a fairly flat dictionary.  This dictionary is 
>passed around in full (as part of the WSGI environment) to every piece of 
>middleware, and actually to everything (via an import and threadlocal 
>storage).  It gets used all over the place, and the ability to draw in 
>configuration without passing it around is very important.  I know it 
>seems like heavy coupling, but in practice it causes unstable APIs if it 
>is passed around explicitly, and as long as you keep clever dynamic values 
>out of the configuration it isn't a problem.
>
>Anyway, every piece gets the full dictionary, so if any piece expected a 
>constrained set of keys it would break.  Even ignoring that there are 
>multiple consumers with different keys that they pull out, it is common to 
>create intermediate configuration values to make the configuration more 
>abstract.  E.g., I set a "base_dir", then derive "publish_dir" and 
>"template_dir" from that.  Apache configuration is a good anti-example 
>here; its lack of variables hurts me daily.  While some variables could be 
>declared "abstract" somehow, that adds complexity where the unconstrained 
>model avoids that complexity.

*shudder* I think someone just walked over my grave.  ;)

I'd rather add complexity to the deployment format (e.g. variables, 
interpolation, etc.) to handle this sort of thing than add complexity to 
the components.  I also find it hard to understand why e.g. multiple 
components would need the same "template_dir".  Why isn't there a template 
service component, for example?


>When one piece delegates to another, it passes the entire dictionary 
>through (by convention, and by the fact it gets passed around 
>implicitly).  It is certainly possible in some circumstances that a 
>filtered version of the configuration should be passed in; that hasn't 
>happened to me yet, but I can certainly imagine it being necessary 
>(especially when a larger amount of more diverse software is running in 
>the same process).
>
>One downside of this is that there's no protection from name 
>conflicts.  Though name conflicts can go both ways.  The Happy Coincidence 
>is when two pieces use the same name for the same purpose (e.g., it's 
>highly likely "smtp_server" would be the subject of a Happy 
>Coincidence).  An Unhappy Coincidence is when two pieces use the same 
>value for different purposes ("publish_dir" perhaps).  An Expected 
>Coincidence is when the same code, invoked in two separate call stacks, 
>consumes the same value.  Of course, I allow configuration to be 
>overwritten depending on the request, so high collision names (like 
>publish_dir) in practice are unlikely to be a problem.

I think you've just explained why this approach doesn't scale very well, 
even to a large team, let alone to inter-organization collaboration (i.e. 
open source projects).


>   For instance an application-specific middleware that could plausibly be 
> used more widely -- does it consume the application configuration, or 
> does it take its own configuration?  But even excluding those ambiguous 
> situations, the way my middleware is factored is an internal 
> implementation detail, and I don't feel comfortable pushing that 
> structure into the configuration.

That's what encapsulation is for.  Just create a factory that takes a set 
of application-level parameters (like template_dir, publish_dir, etc.) and 
then *passes* them to the lower level components.

Heck, we could even add that to the .wsgi format...

    # app template file
    [WSGI options]
    parameters = "template_dir", "publish_dir", ...

    [filter1 from foo]
    some_param = template_dir

    [filter2 from bar]
    other_param = publish_dir


    # deployment file
    [use file "app_template.wsgi"]
    template_dir = "/some/where"
    publish_dir = "/another/place"


>So that's the issue I'm concerned about.

I think the right way to fix it is parameterization; that way you don't 
push a global (and non type-checkable) namespace down into each 
component.  Components should have an extremely minimal configuration with 
fairly specific parameters, because it makes early error checking easier, 
and you don't have to search all over the place to find how a parameter is 
used, etc., etc.