[Mailman-Developers] Adding headers to mailman generated mails

[Barry Warsaw]

On Wed, 2004-01-21 at 21:21, Brad Knowles wrote:

> 	As part of the message personalization, add the appropriate 
> per-user information in the template footer for the list.  That 
> should hopefully deal with the problem.

I should mention that Mailman 2.1's full personalization support (as
opposed to VERP header support) isn't terribly efficient.  I have what I
think will be a very nice scheme to do this about as fast as you can do
in Python, but it requires Python 2.3 so it's slated for Mailman 3.  The
nice upside is that you could conceivably support the templatizing and
personalization of any content, not limited to the footer, header, or
mail headers.  I believe I can make it efficiently configurable so a
list owner could disable content personalization of say the original
message for discussion lists, but enable it for newsletters.  We'll
ignore for now the question of where the personalized content comes
from.

In a previous message, Brad gave great answers and links that are well
worth re-reading every few months.  So I won't rehash anything I agree
with.

> >  So is there a way to tweak Exim into sending the messages individually and
> >  allow the addition of a personalized footer without creating personalized
> >  bounce-back addresses?
> 
> 	I am not familiar with Exim.  I do not know what configuration 
> changes would be required to get it to add personalized information 
> in the headers.  You would need to talk to someone better acquainted 
> with Exim, presumably on an Exim-specific mailing list or newsgroup.

Exim has some very nice capabilities which can be used to embed an
interpreter like Python in your MTA.  For example, we use this on
incoming messages on python.org to filter everything throw spambayes and
do other programmatically interesting checks on the message.  Yes, it
slows down message acceptance a bit, but it's worth it for us.

Nigel can provide details, but I think the same embedding feature could
be used to have the MTA do the final stitching of content template and
personalized data.  It would be A Project to hack together, but I think
it could be a neat idea to play with, although I'm not sure how much it
would help.  Certainly, pushing the stitching down into the MTA and
closer to the external socket connection would reduce disk i/o on the
mail server, because then Mailman could go back to handing one copy of
the message to the MTA, plus some job description of where to get the
personalized information.  Imagine a SQL select statement for instance. 
If the MTA could do what Mailman does here -- not creating a disk image
for each instance of the message, but stitching it together in member as
it's going out on the wire -- I think you'd greatly improve disk
contention.  You wouldn't help bandwidth, but then if JC's evaluation is
accurate, that penalty is a mere <wink> doubling of bandwidth.

I don't think Postfix has the same embedding capabilities, although I
haven't looked at what Postfix 2.1 may provide.

> 	I can tell you that the default standard installation of sendmail 
> will do this for you, automatically.  If there is one and only one 
> recipient of a message, the "$u" macro will be defined, and the 
> identified username will be shown in the "Received:" headers.  If 
> there is more than one recipient for the message, then this macro 
> will not be defined, and no usernames will be displayed.

In a sense, that's what we've talked about before.  If there were a
standard language that the mail server and list manager could agree on
for both defining the template, and defining the per-recipient data
source, we could have a more efficient mechanism, with perhaps a hope of
mta agnosticism.

> 	Unfortunately, most headers tend to be missing from most 
> complaints, so your best bet would probably be to get Mailman to put 
> the message personalization information into the footer of the 
> message, which is more likely to survive.

As for stripping headers, I do think there's some value in being able to
more easily configure the headers to strip for both regular messages and
anonymized messages.  OTOH, it's easy to hack the source.  Cleanse.py in
Mailman/Handlers is the place to look.

-Barry