[Mailman-Developers] Feature Request - Interactive HTML Digests

[Stephen J. Turnbull]

Tanstaafl writes:

 > The point I'm getting at is, I'd like to see the basic framework for a
 > new HTML digest added to MM. Barry said he is fine with it as long as it
 > is done right, and Mark seems to concur. Well, you have to start
 > somewhere, so why not just start with the really simple/basic #anchor
 > tags being the first 'feature',

No reason not to, but even doing that right is non-trivial.  The
problem that you are not acknowledging is that generating HTML is
*not* the problem (although turning it into a user-modifiable template
might not be easy, especially if you want to have reasonable reply-to
features).  It's what happens before you even *think* about generating
the digest itself, in terms of filtering out parts that can't be put
into HTML, etc.

It would be possible to do what you suggest as follows:

1. Reject (return to sender) all posts with a top-level MIME type
   other than text/plain.  No HTML, no attachments.  It probably would
   be wise to reject anything that lacks a Content-Type header as
   well.  (Now you know how to spell "draconian".)  Put the rest into
   a file in the usual "mbox" format.

2. Output the HTML preamble.

3. Split the mbox into messages.  (Mailman already needs to do this.)

4. For each message:

   1. Output links to next and previous messages, first message in
      this thread, and digest TOC (table of contents).

   2. Split the message into a list of header fields (From, To,
      Subject, etc, as well as the more esoteric stuff you normally
      don't see), plus the body text as a blob.  (Mailman already
      needs to do this.)

   3. Output a table of field name (eg, From) and content.  Wrap
      fields with address content with mailto links for each person.
      The big question here is how to arrange for quoting of the body.
      The obvious and safe kludge would result in copying the body
      into the mailto link once for every reply link, which would
      multiply the size of the digest by 2-10 times, depending on how
      many of those there were.  Wrap the Subject field content in an
      internal link to the beginning of the thread in the digest TOC
      (table of contents).  (Straightforward.)  Javascript could be
      used to make the burden of the mailto links reasonable, but I
      don't like the idea of including Javascript in mail messages
      much.

   4. Output the body wrapped in an HTML PRE element.

5. After the last message, output another navigation section, then any
   postamble stuff (footer text, etc.)

But this is just a barebones start.  It would really be bad if there
were more than about 20 messages in the digest, or if any of them are
more than a couple of windows tall.  And it might need to be
completely refactored to add reasonable reply functionality.

 > and just see what happens.

Not a chance of that unless you can find a volunteer to field the bug
reports.  There will be documentation to write, configuration to add
to the config file and the web interface, etc, etc.

 > Mark seems to believe that adding the Reply-To-List/Sender' mailto:
 > links that can also grab the in-reply-to references will be doable

Links based on header content are easy; the headers are well-defined,
and their contents have a reasonably well-understood syntax.  Quoting
the body text is very likely a major mess, even if the message is
text/plain.  Even formatting the body text will be non-trivial in the
presence of messages with long lines (ie, > 74 columns).

 > >> 2. Make it template based, so as to be easily modifiable by the MM
 > >> admin.
 > 
 > > Impossible.
 > 
 > Ahem... 'Nothing is impossible' my Dad would say. ;)

'There ain't no such thing as a free lunch,' my economics textbooks
say.  But then, you know that.

 > How does MM generate the two digests it supports now?  Does it
 > store the individual messages in some temporary location until it
 > is time to generate the digest, then do whatever it does to
 > generate it?

Yes.

 > Or does it process each message as it comes in, and cumulatively
 > add them until the trigger for sending the digest is pulled?

No.

 > The reason I ask this is, a way would have to be found to provide anyone
 > who chose to work on this a 'test bed' of messages to be used to
 > generate the digests to test any of their modifications, otherwise
 > testing modifications would be a royal pain...

This is not a problem.

 > > The MM admin can probably make it prettier, but I doubt that it will
 > > be possible to do this in such a way that they can change the basic
 > > functionality.
 > 
 > No offense, but I don't see that at all... 

That's because once again you've conveniently lost all the context.
We're talking about the HTML template which is available to the MM
admin, and not the

 > python (, C?) and HTML - and the code.

Of course somebody who can hack code can make it do anything.  But can
*you*?  You're the person I'm thinking of when I write "impossible"
referring to the MM admin changing the basic function.  Of course, you
*could* learn to hack code ("nothing is impossible"), but so far I
detect no such appetite in you.  OTOH, the people who can hack the
code very likely don't need this feature at all.

Of course, if you have say USD10,000 to offer, that would change
matters entirely I bet. :-)