[Mailman-Developers] GSoC 2013 - GNU Mailman - Introduction and Project Discussion

[Barry Warsaw]

On Apr 11, 2013, at 01:52 PM, Terri Oda wrote:

>Writing individual pipelines may be trivial, but making a user interface for
>managing said pipelines is non-trivial.  Right now, our pipeline management
>interface is "there's a text box in postorius that lets you choose a
>pipeline.  It's not even a dropdown, and you may be screwed if you make a
>typo" which is obviously not how I want it when we release. ;)

Remember that in MM3 you have two processing queues.  Think of the first as
the "moderation" process and the second as the "modification" process.

http://pythonhosted.org/mailman/src/mailman/docs/8-miles-high.html#basic-messaging-handling-workflow

When the message is first accepted by the LMTP server, it is dumped into the
incoming queue.  The incoming runner will send the message through the rule
chain for target mailing list.  (There are actually two of these chains for
every mailing list - one for messages to the owner/moderator and one for
messages posted to the list.)  Rule chains are made up of individual links
where each link usually contains a rule and an action.  Rules only inspect the
message, never modify it (though they can add metadata to the associated
dictionary), and rules produce a binary decision.  If the rule "hits", the
action is take.  If the rule "misses", the next link in the chain is executed.

At the end of the rule chain is usually a "truth" rule which always hits, and
its action is to jump to the "accept" chain, which essentially just dumps the
message into the pipeline queue.

The pipeline runner only handles messages that have been approved for posting,
so it doesn't have to make any moderation decisions.  All it has to do is
modify the message for posting.  Again, there are two pipelines, one for owner
messages and one for list posts, but pipelines are much simpler.  They are
just sequences of handlers.  Each handler does whatever it wants to the
message, and it can use, add, or delete information from the metadata
dictionary to do this work.  E.g. handlers can remove or add RFC 822 headers,
filter content, add those informative footers, etc.  Handlers also copy the
messages to other queues, so that other runners can send the message to the
archives, NNTP, and the outgoing SMTPd.

I mention all this because to show that there are *lots* of ways the system
can be configured, and I'm not sure all should be exposed via Postorius.
Certainly the full power isn't something a list owner will usually want to be
confronted with ;).  Each chain and pipeline (as well as rules and handlers)
have a unique name, and the choice of chain and pipeline is part of a list's
style (i.e. initial setting), so that may be a better way to allow list owners
some flexibility in setting up their lists.

Anyway, some things to keep in mind. :)

-Barry