[Web-SIG] Standardized template API

[Phillip J. Eby]

At 03:01 PM 2/3/2006 -0800, Ben Bangert wrote:
>On Feb 3, 2006, at 2:24 PM, Phillip J. Eby wrote:
>
>>>Wait, then how does the template engine know what template to render?
>>
>>The framework gives the engine the source of the template, and asks
>>for a WSGI app object in return.  The framework is allowed to hold
>>on to this application object and reuse it as often as desired.
>
>I'm really not sure why the framework developer should be further
>burdened with trying to cache templates.

Well, somebody's got to do it.  :)  Not every template facility out there 
currently has any kind of location ability or caching.  And different 
frameworks have their own mechanisms for locating, caching, etc.

However, it may just be that there are more pieces here than I thought, and 
I've been lumping some of them together under the name "framework".


>>What I'd suggest is that this is actually an opportunity for Myghty
>>to put its lookup and caching on one side of the interface, and an
>>actual template rendering facility on the other side.  That is,
>>treat Myghty as an embedd*er* rather than the embedd*ee*, so that
>>the engine can be used with arbitrary apps or templates on the
>>embedded side.
>
>Myghty's lookup and caching is rather complex and requires
>significant synchronization efforts to ensure that under heavy load,
>if the template has changed, its only re-compiled once while the rest
>of the requests continue to use the prior compiled version until the
>updated one is ready. It's complex, and putting it on completely
>separate sides is beyond a nightmare of effort.

I don't understand.  Are you saying there's nothing that corresponds to 
reading source, compiling, and writing out a compiled version, that would 
just change to simply call those methods on a different object than the 
ones they're called on now?


>The lookup mechanism it uses is also very complex, and it doesn't
>belong outside of Myghty.

I don't understand what you mean by "outside of Myghty" in this 
context.  My suggestion was simply that would gain pluggability in one main 
area: the implementation of compiling and serialization of 
templates.  (i.e., the syntax/semantics of the templates used)


>Yes, but its 100% incompatible with Myghty. For the flexibility and
>reloading functionality that Myghty provides, splitting such
>functionality up will only cripple features of Myghty that I really
>enjoy.

I'm still not understanding.  Since Myghty will be in control of all the 
streams, it will have full ability to monitor changes etc.  The only thing 
that's different under the embedding scheme is that 1: it would support 
multiple formats for the actual templates, and 2: it would return WSGI apps 
that frontend access to an individual template.


>Not really, the one I interpreted kept vital steps together. Without
>those steps coupled closely, Myghty cannot be compatible without very
>significant changes to underlying structures.

It's not as different as you think.  In your idea, Myghty took each WSGI 
call.  In the actual idea, Myghty would take a call that returns a 
callable.  That callable gets called for each WSGI call.  You still have 
total control over what happens in each WSGI call; it's just that *which* 
template is to be used gets factored out by the first call.  It doesn't 
decouple any of the things you're talking about, as best I understand it.


>I thought Cheetah also compiled its templates and would reload them
>if it detected changes. If a template changes on disk, thats the
>great thing about template systems that handle stuff like this. You
>ask for the rendered template, and if it changes, it'll recompile/re- 
>cache it, and render it for you.

Ah.  I didn't realize this was so common.  I'm accustomed to template 
systems that don't control so much stuff, but which are more at the 
compile/save level.  That may be more of a mismatch, making my idea much 
less useful for the template use cases that people currently have in mind.  :(


>It might not be complex for whatever template system you use, but its
>a drastically different situation for others. A pure WSGI call to a
>template engine like I thought you were referring to was more
>agnostic on how the template language deals with it, the key appeal
>and flexibility to me.

I'm seeing now that the key problem with my view of the situation is that 
I've been assuming *frameworks* have control over where and how templates 
are stored, and considering the template systems to be just that - 
implementation of the template itself.

One reason why I also wanted us to focus on template/resource deployment, 
was because that would help unify caching/compilation concerns and reach a 
common way to address and access such resources across e.g. different eggs.

I think the best thing to do at this point is for me to try to actually 
write up a spec that's clear enough to be understood.  I think that 
probably the issue here is that there are actually *four* roles, not three 
or two, and I've been lumping some of them together to make three, and most 
everybody else has been lumping them into two, and so none of our divisions 
match each other or reality.  :)

Restating it in four roles, I'd say there's a publisher, a manager, a 
compiler, and a resource.  The publisher is that part of a framework that 
determines which template is needed - a name or symbol identifying it.  The 
manager is the thing that translates a template's identity to some form of 
storage, and manages compilation/caching/reloading/etc.  The compiler is 
the thing that actually does the compilation and knows how to write out a 
compiled template.  Finally, the resource is a WSGI wrapper around a 
compiled template.

So, on a given request, the publisher asks the manager for a resource by 
saying what template it wants.  The manager does whatever it needs to do, 
possibly invoking the compiler if it needs to.  The publisher then makes a 
WSGI call on the resource to render it.

Now, all four of these parts are there in the vars'n'strings proposal too, 
more or less.  It's just that it seems to bundle up everything other than 
the publisher into one unit, and instead of getting back a resource, the 
publisher receives the result of the rendering.  So we get "framework" 
(==publisher) and "engine" (==manager+compiler+resources).

I was at first assuming that "framework" meant the publisher and manager, 
and "engine" was the compiler and resource.  Then to explain better I split 
compiler and resource, but now I see that I should've split (conceptually) 
the publisher and manager.

Now, if I'm understanding correctly, Myghty is manager+compiler+resources, 
so from the publisher point of view there's no real change from your 
interpretation; a name goes in and a WSGI callable comes out.  And, from a 
WSGI embedding point of view, that's really enough.  The only change from 
your previous idea is that you return some object that the publisher makes 
a WSGI call on, instead of doing it as a single WSGI call.  However, Myghty 
is still integrated in its control over the handling.

Does that make sense now, or am I missing something again?