[Web-SIG] PasteDeploy 0.1

[Ian Bicking]

Phillip J. Eby wrote:
>> It's important to me, and it's not intuitive to me what you envision. 
>> So I feel a need to services in action, replacing global configuration.
> 
> 
> Using a config service in a factory to get a default argument value:
> 
>     def some_app_factory(parent_component, **args):
>         config = parent_component.get_service("global_config")
>         args.setdefault('someparam', config['someparam'])

So "parent_component" is some special object created by the config loader...

> Registering a config service (old syntax):
> 
>     [globalconfigservice from SomeEgg]
>     someparam = "foo"

...and I assume in this case globalconfigservice does something along 
the lines:

def globalconfigservice(parent_component, next, **args):
     config = parent_component.get_service('global_config').copy()
     config.update(args)
     component = Component(parent_component)
     component.save_service('global_config', config)
     return next(component)

Obviously I'm making up the component interface here.


>     [...next component in the stack...]
> 
> The config service would respond to 'get_service("global_config")' by 
> returning self.
> 
> The idea is that when you chain non-wrapper components in a pipeline, 
> each one gets the previous component as its "parent component", so you 
> can "acquire" services from your parents.  Components nearer to you 
> (i.e. more local) can override more global service definitions.

OK, well now I'm a bit confused... is globalconfigservice a wrapper?  I 
assume globalconfigservice can't modify the parent_component it is 
passed, and has to create a new one?

>> # or you could do the whole thing like:
>>
>> filebrowser = pipeline:
>>     auth wrapper from Paste:
>>         require_role = "admin"
>>         admin_email = config.admin_email
>>     filebrowser from FileBrowser:
>>         document_root = static.document_root
>>
>> # maybe you could specialize/clone like:
>>
>> auth2 = auth:
>>     require_role = "editor"
> 
> 
> Interesting.  If we used "in" to include other files then you could 
> refer to e.g.:
> 
>     foo = main in "some.ini":
>         # override params

Hmm... it would be nice to allow configuration filenames to be 
variables.  Though "in" and "from" don't scream "config file" and "egg" 
to me -- they are both equally vague terms.  I'd rather see "in egg" and 
"in file".

> Also, I was thinking that in this syntax, you want to be able to leave 
> off the trailing ':' for simple definitions, so that this would be a 
> complete definition, without needing a body:
> 
>     foo = main in "some.ini"

Yes, that works well.

> Finally, I think we could drop the "pipeline" keyword and simply use a 
> ':' to define a name, which then gives us a way to stack components 
> inside a definition, e.g.:
> 
>     main:
>         login wrapper from Paste:
>             # blah
>         urlmap from Paste:
>             "/":     static
>             "/blog": main in "blog.ini"
>             "/cms":
>                  auth wrapper from Paste:
>                      require_role = "admin"
>                  filebrowser from FileBrowser:
>                      document_root = static.document_root
> 
> 
> The idea here is that if you want to pass a component or components to a 
> factory, you have to use ':' syntax, with either a one-line component 
> specifier on the same line, or a multi-component stack as an indented 
> suite.

This starts looking a lot like class statements (especially when class 
statements get reused as data definitions).  And of course a bit like 
YAML.  But then both those resemblences are okay.

> An interesting question is whether you should be able to refer to nested 
> definitions as factory prototypes (ala your auth2/auth) or whether only 
> top-level names should be usable.  For example in this:
> 
>     foo: bar from baz
> 
>     spam:
>         foo: snickety from lemon
>         scuzz: foo
>         sprim:
>             thingy: foo
> 
> Does "scuzz: foo" refer to the inner foo or the outer foo?  What about 
> "thingy: foo"?
> 
> I'm inclined to say that both refer to the spam: foo rather than the 
> outermost "foo".  i.e., more or less the same rules as Python scopes.

I agree.  Will spam.foo be an unambiguous representation?  It seems like 
it should be.  Would there be a global object, like globals.foo?

> One minor problem with this syntax overall, though, is that it's a bit 
> context-dependent.  Whether "foo:" means "define foo" or "create a foo" 
> is just a matter of alternating layers.  It would be better if the 
> syntax were less ambiguous, e.g.:

I don't see the distinction between "define" and "create".  By this 
distinction do you mean that pieces of the loading process lazy?  Can 
all parts be lazy?  (I.e., the config file defines named factories, the 
body of sections isn't evaluated until those factories are invoked)

>     main :=
>         login wrapper from Paste:
>             # blah
>         urlmap from Paste:
>             "/"     := static
>             "/blog" := main in "blog.ini"
>             "/cms"  :=
>                  auth wrapper from Paste:
>                      require_role = "admin"
>                  filebrowser from FileBrowser:
>                      document_root = static.document_root

...for instance, when this was "main = pipeline:", it was clear this was 
just another "create", except using "pipeline" to create the object, and 
pipeline looks at the section contents.  The unnamed sections below it 
are just like positional parameters (would named sections be ordered? -- 
I've always wanted ordered class statements, I imagine I'd like to keep 
order here too)

I don't have any attachment to "pipeline", but I think some word is fine 
in that position, and I don't see why this is a particularly "special" 
construct (except of course that it should be builtin).  Would this be 
allowed?:

main = urlmap from Paste:
   "/" = static from Paste:
     document_root = "/home/me/htdocs"

> But that doesn't actually seem to help visually, and makes it harder to 
> write because you have to remember all the time whether you need ":" or 
> ":=".  Maybe this would be better:
> 
>     main is:
>         login wrapper from Paste:
>             # blah
>         urlmap from Paste:
>             match_mode = "longest"
>             "/" is static
>             "/blog" is main in "blog.ini"
>             "/cms" is:
>                  auth wrapper from Paste:
>                      require_role = "admin"
>                  filebrowser from FileBrowser:
>                      document_root = static.document_root

While I'm not attached to "pipeline", "is" is about as vague as "in" and 
"from".

> What's nice about this is that now you can unambiguously create a 
> top-level object in the simple case:
> 
>     zapp from Zope:
>         cfg_file = "site.zcml"
> 
> Without needing to do:
> 
>     main is:
>         zapp from Zope:
>             cfg_file = "site.zcml"

You could do:

main = zapp from Zope:
     cfg_file = "site.zcml"

Assuming "main" was a special magic name for the primary application.  I 
would certainly assume that reading the config file (even I'd never seen 
these config files before).  I, for instance, do not like Python's "if 
__name__=='__main__'" idiom; I think using a conventional name to 
indicate the primary function of a file is just fine.

> Which is a pain, IMO.  Although I suppose we could allow:
> 
>     main is zapp from Zope:
>         cfg_file = "site.zcml"
> 
> which isn't too bad.
> 
> 
>>>> * "name from egg_spec" reads nice on one level, but is vague on 
>>>> another level.  Even if "egg:egg_spec#name" doesn't read well, I 
>>>> think it is nicely self-describing.
>>>
>>>
>>> Um, wha???  The only difference between the two is that one of them 
>>> has "egg:" in front of it, which seems a bit redundant to me.  That's 
>>> probably because I assume that in the long run eggs will be so 
>>> ubiquitous that it really will be redundant to explicitly refer to 
>>> them as such.  :)
>>
>>
>> In paste.deploy config files are full peers to Eggs, and can be used 
>> anywhere that an egg: reference can be used.  I think that's a neat 
>> feature.  I don't want to tack on referencing other config files, like 
>> a special loader factory or textual inclusion hacks or anything like 
>> that.
>>
>> Config files describe applications.  Egg entry points describe 
>> applications.  They should be peers.
> 
> 
> Okay.  The "in" syntax I gave above allows that, although I could also 
> go for only using "from", as long as config URLs are quoted strings.  I 
> also think the strings should be relative or absolute URLs, rather than 
> filenames.  (So that '/' has the same meaning on all platforms.)  That 
> will be something of a pain for Windows users who may need to include 
> drive letters, but oh well.  We can always treat the letters A-Z as a 
> special "file:" protocol to fix that.  :)

By URLs, do you just mean that they use URL syntax, URL quoting of 
filenames, etc?  That's fine by me; I normalize \ to / in paste.deploy 
and run urllib.unquote on the result already.  I'm not sure what to do 
with \'s; they are dumb and annoying and I hate them, but when they slip 
into the system it should at least handle them reasonably.

While it is slightly annoying to keep track of it, I think it's 
important that filenames be defined as relative to the config file that 
they are contained in.  The current working directory is useless, and 
always using absolute filenames makes config files very hard to reuse.

>>> Conversely, if I assume that some further description is required, I 
>>> would want to say "pypi:" or "project:" or something else of that 
>>> sort, because "egg" isn't the essential nature of the thing; the name 
>>> is a *project* name, while eggs are an implementation detail.
>>
>>
>> egg: is an access method, just like http: or whatever.  It doesn't say 
>> what the URI describes, just how to find it.
> 
> 
> Ah, but that's just it.  The project name is a URN, not a URL, precisely 
> because it *doesn't* describe how to locate the resource, it just names 
> the resource and tells the system to go find it.

Well, sure it says how to find it -- load pkg_resources, get the package 
by name, etc.  There's always a "system, go do stuff for me" step, 
that's how computers work.


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org