[Mailman-Developers] MTA interface for MM 2.1

[Barry A. Warsaw]

>>>>> "NM" == Nigel Metheringham <Nigel.Metheringham@Vdata.co.uk> writes:

    NM> People are probably familiar with teh current ability of exim
    NM> to be configured to automagically see lists and handle them
    NM> appropriately.  There is also a wrapper (in scripts/auto) in
    NM> MM 2.1 that attempts to do the same thing for postfix (BTW it
    NM> will need updating for the additional aliases now used).

I actually don't plan on updating auto for MM2.1.  To use it requires
you to configure Postfix in ways that could interact poorly with other
parts of your mail system, depending on your topology.  I'm not sure
if I should leave it around for posterity, or just zap it.

    NM> MM 2.1 uses some additional list entry points.  This means
    NM> that for an alias based approach to injecting mail there are
    NM> now more aliases, and worse they aren't regular.  Here's the
    NM> alias set for a list with basename x (I have truncated the
    NM> path on wrapper to make it fit without wrapping):-

    |   x                 |wrapper post x
    |   x-admin           |wrapper mailowner x
    |   x-request         |wrapper mailcmd x
    |   x-join            |wrapper join x
    |   x-leave           |wrapper leave x
    |   x-bounces         |wrapper bounces x
    |   x-owner           |wrapper mailowner x

    NM> Additionally if VERP is being used, the bounces alias should
    NM> also be matching stuff like
    NM> x-bounces+aperson=dom.ain@mailhost.dom

    NM> Exim (and I guess other mailers - I'm going to talk on what I
    NM> know and then let other folks talk on the other MTAs), can
    NM> handle stuff based on regular expressions, so given a list x
    NM> (matched by some magic - current config does it by checking if
    NM> the file /home/mailman/lists/x/config.db exists) its dead easy
    NM> to match an address suffix (I think the postfix folks call
    NM> this an extension ) -(admin|request|join|leave|bounces|owner)

Note that the current approach to auto-aliasing for Postfix actually
does set up each and every alias -- no regexp matching is done.  It's
just that I know the format Postfix uses and we can write that format
in Python, so updating them is hidden from the admin.

Aside 1: I believe Fil has an alternative approach for Postfix
that does use regexps.

Aside 2: My current approach for Postfix may cause some logistical
problems because it requires that the version of Berkeley DB that
Postfix is linked against be the same as the version that Python is
linked against, and it's notoriously hard to get this right in
Python.  Especially in Python 2.0.x which doesn't build the bsddb
module by default (though Python 2.1.x does, which is one of the
reasons I'm considering requiring it for MM2.1).

Aside 3: MM2.1 won't use marshal as it's serialization format any
more; it uses pickle so the landmark file needs to be config.pck
instead (although that file will only exist after the first time the
list is opened and saved -- we could make sure the upgrade script does
this for every list though).

Aside 4: There was an interesting idea, in SF I think, that maybe the
MTA should just delivery to an mbox file, and then Mailman's qrunners
should just walk the mbox file.  We'd have to coordinate locking b/w
Mailman and the MTA, so that could be problematic.  OTOH, it would
save having to fork the wrapper script just to dump the plain text in
the queue.y

    NM> However, since the argument to wrapper varies with the
    NM> extension I then need an additional lookup to convert that
    NM> extention to the wrapper command.

    NM> So, for exim, life would be much easier if I could handle
    NM> these (excluding the post command) by having

    |   (.+)-(admin|request|join|leave|bounces|owner)
    | map to
    |      |wrapper $2 $1

    NM> [$1 & $2 are the regexp matches from the expression above]

    NM> Barry - any chance of regularising the naming of the entry
    NM> points, or adding aliases for admin, request and owner??  I
    NM> would guess this approach would work for several different
    NM> mailers.

I've thought the same thing.  I avoided in the past because it would
have required regeneration of aliases, but now that we've bit that
bullet, I think it does make sense to regularize them.  I'll have to
see if I can get SF to make copies of the cvs files so I can preserve
the historical record, but I think it's a good idea.

    | BTW VERP could be handled by changing the intitial regexp to
    |   ([a-z0-9_-]*)-(admin|request|join|leave|bounces|owner)(\+.*)

VERP format will be very configurable, so as long as you can write a
Python regexp to pull it apart, you can basically tell Mailman to use
any VERP format you want.

    NM> An alternative approach might be that all addresses that are
    NM> caught by the MTA list matching stuff are passed direct into
    NM> the Mailman address handler, so you just invoked wrapper
    NM> mailman-address-handler <address>

    NM> and let the python do the magic from there on.

I've found it convenient to do a little bit of preprocessing in the
wrapper script and knowing beforehand which alias the message is being
delivered to.  One reason is because I don't want to force the
wrappers to parse the email messages.  Parsing is expensive, and
potentially error prone, and the wrapper can't gracefully handle the
errors without a lot of complexity.  Which slows things down.  Which
increases the possibility of mail filter timeout.  Which we definitely
want to avoid at all costs.

To sum up: let's regularize the wrapper script names so that they are
the same as the extension they feed off of (sans the separating dash).
One catch is for the -admin address; I want this to go to the bounces
script for backwards compatibility.  So I guess we need to make a
symlink from scripts/bounces -> scripts/admin ...?

    NM> 	Nigel.

-Barry

    NM> If it was then possible to

Maybe, but only if we