[ANN] HTMLTemplate 1.0.0

Thu Jun 3 11:15:24 EDT 2004

Peter Maas <peter.maas at mplusr.de> wrote in message news:<c9kfd3$854$1 at swifty.westend.com>...
> David Fraser schrieb:
> >> Did you find these alternatives unsatisfactory? If somebody wants
> >> to use a Python templating framework why should he prefer
> >> HTMLTemplate?
>  [..]
> > It looks cool because it doesn't embed Python code in the template.
> > Do any of the other frameworks have this approach?
> 
> That's your personal taste. 

While taste certainly plays a role, I think there are also sound
technical arguments to be made for complete code-markup separation.

> If this is what you like about HTMLTemplate
> that's fine. I don't bother wether a template language contains Python
> code or not. To answer your question:
> 
> Hamish's example rewritten in TAL (TAL lines below Hamish's equivalents):
> 
> <html>
>      <head>
>          <title node="con:title">TITLE</title>
> ###TAL  <title tal:content="here/title">TITLE</title>
>      </head>
>      <body>
>          <ul>
>              <li node="rep:item">
> ###TAL      <li tal:repeat="link here/objectValues">
>                  <a href="" node="con:link">LINK</a>
> ###TAL          <a tal:attributes="href link/name" tal:content="link/title">LINK</a>
>              </li>
>          </ul>
>      </body>
> </html>

This is embedded code, though in a custom language rather than Python.
This has a couple of advantages: it's terser (less typing) and less
abstract (easier for non-programmers to cope with).

The disadvantages are: you've another language to learn before you can
use the system, you're limited in what you can do and how you do it by
that language's featureset, and code (Controller) and markup (View)
are tightly coupled (loose coupling makes for a more flexible and
easier to maintain architecture, while the standard argument in favour
of tight coupling - raw performance - doesn't apply in this case).

Also, the terseness and minimal abstraction are only really advantages
in simple cases; with more complex systems the number of keystrokes
saved is an insignificant proportion of the total typing done, and the
lack of abstraction becomes a significant disadvantage (like having to
write a 500-line program as a single continuous main() routine without
procedures or namespaces to structure it).

I'm not sure if TAL supports callbacks into Python code. ZPT allows
you to do this, allowing you to fall back to Python for tasks that the
templating language isn't able to perform itself; your basic two-tier
hybrid. The two other approaches mini-language based systems seem to
take are:

- expand their templating languages till they become full
Turing-complete programming languages in their own right

- keep the templating language restricted to the absolute basics:
insertion, conditionals and iteration.

The fundamental folly of the first approach ought to be obvious
enough; see Greenspun's Tenth Rule.

The second seems quite sensible, at least on paper: state that it is
not the templating language's job to be a general-purpose scripting
language, limit its scope to providing only the core functionality
necessary to insert pre-prepared data into the template, and leave the
client to do that data preparation themselves.

In practice though, it seems terribly inefficient and somewhat
redundant to have your data in one highly structured form, e.g. a
collection of database tables, file objects, native data structures,
etc., and have to munge that into another highly structured form, e.g.
a large, deeply-nested lists-and-dicts structure directly
corresponding to the template's final structure, just so the
templating engine can morph this structure into yet another form. It
would be more sensible to skip the middle step, eliminating all
restrictions on the type or form of data that the template can
consume. The temptation then is to evolve the second type of system
into the first; but then you're back to chasing that never-ending
feature request list for more power, more flexibility, more
convenience, and its Greenspun's rule all over again.

...

> But I wanted to read Hamish's thoughts. 

Heh, no chance! My tinfoil beanie successfully blocks your devilish
CIA X-Ray Secret Mind Reading Abilities, so I'm afraid you'll just
have to stick to standard plaintext communication protocols. Damned
inefficient, I know, but I can't risk you finding out where the
treasure is buried. ;)

> That would be helpful for programmers because it takes some time 
> to find your way through the maze of Python template languages.

I am, of course, happy to wait till the rest finally declare that
"It's a fair cop, guv!" and gracefully concede defeat. (Like that's
ever gonna happen, hey?;)

> I prefer Cheetah. 

Teensy request: I think it'd help our readers if you could also show
your Python code for hooking up to these templates and feeding them
data; allow them to compare like-against-like. That'd be cool.

...

Hamish's example in Cheetah:
> 
> <html>
>      <head>
>          <title>$title</title>
>      </head>
>      <body>
>          <ul>
>              #for $link in $links
>              <li>
>                  <a href="$link.href">$link.text</a>
>              </li>
>              #end for
>          </ul>
>      </body>
> </html>

I like Cheetah less than TAL. TAL, like HTMLTemplate, treats the
template's HTML as a first-class citizen; Cheetah is much closer to
the original macro processors from which all PHP/ASP/Cheetah/etc.
style systems are basically descended and treat everything non-macro
pretty much as dumb, meaningless character streams.

Like I think I've said elsewhere, I've already designed, written,
shipped and canned this sort of system myself before moving to
DOM-derived designs, so I've firsthand experience from both sides of
the fence and am quite certain that while there's some situations
where the former is better, these are really just corner cases and the
center field would be completely owned by HTMLTemplate-style systems
if it hadn't been for those pesky PHP kids doing a BASIC upon us
first. (Ah well; fair fight, early worm, and all that. Can't blame 'em
for being successful; besides, it's never _too_ late to Change...:)

...

> Advantages:
> 
> - No magic

No magic to HTMLTemplate either. It simply uses a more sophisticated
model to describe and construct renderable templates.

I'd suggest a good comparison is Apple's excellent Interface Builder,
which allows desktop applications' Views to be constructed without
writing or generating a single line of program code. In Cocoa GUI
construction, everything is done by manipulating object instances
directly, assembling them into a live, fully-functional object model
and adding a smidgin of binding information to connect them to the
application's Controller layer. [1]

> - placeholders occupy the places of their replacements
> - logic is visible in the template
>    -> you see what's going on
> - logic is visible in the rendered html page
>    -> you see where the replacements will go

These aren't outright advantages; merely differences in goals. 

Plus other advantages, like the ability to mockup your HTML templates
to look like finished pages - very helpful for designers and clients
in visualising their designs - are conspicuous by their absence. (Note
that TAL and some other code-in-markup systems also share this
ability, so it's not an advantage unique to DOM-derived systems;
although they're certainly an excellent natural fit for it.
Conversely, Cheetah's at a decided disadvantage here, since making
Cheetah templates look anything like the pages they'll eventually
produce is going to be rather hard with all that macro-style stuff
strewn throughout.)

PHP-style systems' lack of abstraction makes them particularly
appealing to folk with little or no abstract design skills; i.e.
non-programmers (and crap programmers, if you're being brutally
honest;). This is certainly a valuable feature for those users, but I
think this is oversold as an advantage while its disadvantages -
particularly the ease with which is can yield the most amazing
spaghetti code when put to use in non-trivial tasks - are
simultaneously undersold to everybody else.

Your average Python user, for example, shouldn't have any conceptual
problems dealing with the stronger abstractions of DOM-derived systems
like HTMLTemplate; and really, anyone that can understand and write
basic DHTML should quickly find themselves at home.

By comparison, while HTMLTemplate presents newcomers with a slightly
greater conceptual first hurdle to pass, once over it there isn't much
else left to learn. Meanwhile, the system itself scales extremely
well: while trivial code examples like the introductory tutorial on my
site can always be rewritten to look smaller and simpler in an
embedded language (which I'd never dispute, btw), I think you'll
really start noticing the difference as you move to larger, more
complex "real-world" templating tasks. For example, it'd be most
interesting to see a code comparison between Cheetah/TAL/PSP/EmPy/PTL
and HTMLTemplate for something like the scripting terminology renderer
in my appscript package. (Though at several hundred lines of HTML and
code I won't mind if nobody takes me up on this; it'd take a good few
hours' work with no actual reward at the end of it.:)

...

Incidentally, one of the projects on my "Fun" list is to write a
simple template macro preprocessor that can read a simple TAL-like
template and generate HTMLTemplate-based View and Controller layers
from it. This could make HTMLTemplate a great sell for developers
writing "Aunt Millie" shrinkwrapped software (e.g. stuff like
Fogcreek's CityDesk): non-programmers and "lite" users could use a
simple embedded macro expansion system to produce simple HTML
pages/customise their own; advanced users could dive straight in at
the Python API level.

The advantage of this approach is that users who start off using the
macro system can seamlessly transition to the more powerful underlying
programmatic system; they can even take their previous work with them,
simply throwing away the original macro-based templates after
outputting the code-generated versions to file for future modification
and use.

So this is definitely something I've thought about; the only
difference is I'd answer it with a loosely coupled, multi-layered,
componentised solution, not a big monolithic system that needs a
chainsaw to reshape into any form not explicitly provided for by its
designers.
...

> - Cheetah templates everything, not only markup

HTMLTemplate's motto here would be: "Don't bite off more than you
really need to chew." As I think I've said elsewhere, most of its
development history has involved throwing _away_ functionality whose
presence did not belong in HTMLTemplate or whose cost could not be
sufficiently justified to merit inclusion. In this case,
responsibility for non-HTML templating has been turned over to
texttemplate, which has been deliberately designed as a
general-purpose templating engine rather than optimised specifically
for working with HTML. I could just as easily turn this argument
around by saying that Cheetah spreads itself too thinly, and adds
unnecessary baggage for users who only want to do HTML templating.

...

Anyway, I think you really need to approach HTMLTemplate on its own
terms. It was never intended to be
yet-another-PHP-style-macro-preprocessor-turned-into-full-blown-language
templating system. It's actually an example of convergent evolution:
while in the large it aims to solve the same problems as Cheetah
et-al, it does so from completely different first principles and
influences and with different detailed goals: total
designer/programmer separation; desktop application architecture
practices; DOM/Apple Event Manager object model; minimalist name-based
object-code bindings.

Now, the macro processor-style approach certainly works, and has an
advantage in working equally well with any template type, while
HTMLTemplate is limited to SGML-based markup (you'll need to use
texttemplate[2] for templating other types of text). But ultimately it
achieves its aims more through brute force than real elegance. It
generally has to expend quite a bit of effort to reach its goals, and
a lot of hard work that is, unfortunately, entirely self-imposed. A
quick comparison of the Cheetah and HTMLTemplate APIs and codebases
will easily demonstrate:

- Cheetah's API consists of a complete built-in custom scripting
language in addition to its Python-based "housekeeping" API, while the
codebase spans twenty-odd modules and a C extension containing several
thousand lines of code.

- HTMLTemplate's API consists of four public classes [3] with under a
dozen properties and methods between them, and the implementation is
precisely 333 lines long.

HTMLTemplate's entire API can fit comfortably within Cheetah's
housekeeping API, with room left over. Cheetah's setup scripts alone
run to a couple hundred LOC; HTMLTemplate is mostly implemented in
that. I think if you're going to measure the two, you seriously have
to consider these aspects as well, because they have major
implications for things like user learning curves, developer
maintenance, and, ultimately, reaching that magical v1.0 release that
says "This System is Officially Complete."

Cheetah is a helluva impressive achievement in itself; I can't imagine
how many man-hours, how much blood sweat and tears have gone into
getting it to where it is today. Though I'm willing to bet it's no
more than I've put into getting HTMLTemplate to where it is today. The
only difference is, looking at HTMLTemplate you can't actually see the
many hundreds of hours and thousands of lines of code that it took me
to get there, because I spent several hundred more hours getting rid
of almost all of those thousands of lines of code. Like being a
professional conjurer: you spend many long years making amazingly
sophisticated and skillful sleight of hand appear to all the punters
as if it took absolutely _no_ effort at all. Under those
circumstances, y'just can't win...;)

...

Funnily enough, if I'm worried about anything, it's that HTMLTemplates
comparatively tiny size and simple API will cause casual viewers to
assume it must be a crude, shallow, underpowered system only fit for
trivial templating tasks. Because, you know, Real Templating is a
Serious Business, and requires Seriously Large Templating Technologies
like PHP, Cheetah, Velocity, etc, etc. After all, all that heavy-duty
engineering must exist for a damn good reason, mustn't it? I mean,
folk surely wouldn't go to all the trouble of writing it if it
_weren't_?

The single most important lesson I've learnt from developing
HTMLTemplate's predecessors is that although learning how to factor
and refactor program code well is a valuable skill, learning how to
factor and refactor program _design_ is a hundred times _more_
valuable than that[4]. Because ultimately it doesn't matter that much
how talented a coder you are: if the problem is split along just the
right plane, even an average programmer like myself can, at the end of
the day, outstrip one with ten times the coding skills who misses the
natural grain and smacks it sideways instead. But anyway, I should
shut-up on this angle before I start sounding like some insufferable
Smalltalk/Lisp/Unix philosophy weenie. ;)

...

Anyway, hope this mental splurge has been of some use to you (and
anyone else that's managed this far<g>).

Cheers,

has

--

[1] There are some differences, of course: an IB GUI is assembled via
drag-n-drop whereas HTMLTemplate uses compiler directives that
indicate the type, name and relative location of each object to
create; and IB pickles the resulting View object models for later
rehydration and use, while HTMLTemplate users will generally compile
their Views immediately before use.[1]

[1a] Incidentally, the original HTMLTemplate did employ pickling, but
that's because it was written in AppleScript, where it takes several
seconds to compile a template (mostly because I had to resort to code
generation to get around its lack of deep language features like
introspection and dynamic slot addition; fortunately, its built-in
object serialisation abilities makes pickling a breeze - a Smalltalk
influence, I think). Eliminating the need for compiled template
pickling was a major aim of the Python port, and quite successful, I
think: HTMLTemplate/Python compiles templates in a few
milliseconds/tens of milliseconds.

[2] Note that the current texttemplate release, 0.1.0, has an older
more complex API that I've since reworked for HTMLTemplate and am in
the process of reworking for texttemplate; so I don't really recommend
using the 0.1.0 version as its API _will_ be changing soon.

[3] I fudge the exact number in the documentation; there's several
subclasses I don't bother to distinguish between as the user has no
need to know about these (one of the compromises I had to make in
porting a nice simple prototype-based OO design to a less flexible
class-based language).

[4] That's 10x difference in API complexity times 10x difference in
code length, if you're wondering...;)