[Mailman-Users] [Mailman-Developers] Sender field

[Stephen J. Turnbull]

>>>>> "William" == William D Tallman <wtallman at olypen.com> writes:

    William> On Wed, May 03, 2006 at 11:11:22PM +0900, Stephen
    William> J. Turnbull wrote:

    >> I don't think that is the way that RFC writers in general
    >> think.

    William> Yes, so I gather.

:-)

    William> Which means that people really should learn to use email
    William> systems intelligently, with the MUA of choice as the
    William> means of control.

I firmly believe that, but unfortunately there are lots of MUAs that
don't really permit intelligent use.  Many people "inherit" an MUA
either from their OS or maybe their brother-in-law, and do not have
the desire or resources to change MUAs or even learn to use the one
they've got effectively.

There are a number of ways to look at this, but what the people who
write RFCs have come to insist is that you be strict with yourself
(always follow the rules and best practices), while being lenient with
others.  You can think of this as simply aikido on the Internet (==
self-defense), or some kind of generosity to those less clued than
oneself, but the rule works well whyever you follow it. :-)

    >> So a good mail client will initialize the address of a reply to
    >> the Reply-To, but provide a way for the user to override.  The
    >> RFC only specifies the former, though, and only as a
    >> suggestion.  Exactly how to handle this problem is a user
    >> interface issue, and the RFC remains silent on such issues.

    William> Implication here is that 'user' is a real human being,
    William> not a software agent.

In this particular case, user refers to "user of a good mail client",
presumably human (but it could easily be an Emacs Lisp program or an
expect script ... ok, ok, that's not "easy", that's "heroic" ... but
it could be heroicly!)  However, most of RFC 2822 doesn't refer to
how the headers should be treated by a mail client, just to what they
mean.  That meaning could be useful to a human, or a mailing list
server, or whatever.

    William> RFC2822, section 3.6.6 discusses re-sending fields as
    William> intended for use by a re-sending 'user'.  It also
    William> specifies that these fields are for informational
    William> purposes only, and MUST NOT be used to actively
    William> manipulate the message.

"Automatically."  There's nothing that says that you can't write a
mail client that has an "bounce followup" feature which looks for
sender, resent-sender and so on, and adds them to the "To" header, as
well as formatting the body with a summary of the progress of the
message by using Date and Resent-Date headers.

    William> So an email list server cannot act as a re-sender, IIUC.

I don't see why not.  I think you're overinterpreting the RFCs.
Certainly, in this case "human user" is a leading interpretation.  But
if the actions described could be executed by a program, then there's
no good reason not to interpret "user" as possibly being a program,
unless the RFC explicitly says so.

    William> The alternative is that the server actually initiates a
    William> new message as a 'forwarding' agent.

I don't think either of the meanings of "forward" suggested in RFC
2822 section 3.6.6 apply here.  ("New message with old message as
body" clearly applies to digests, but I think we're more interested in
non-digest delivery in this thread.)

[William gives an example]
    William> That means that the server must (MUST?) identify itself
    William> in the originator fields.

No, I think that's wrong.  If the server wants to claim responsibility
for injecting the message into the mail system, then it should put
itself in the Sender field.  This absolves the original Sender of all
responsibility for misformatting of the message, misdirection to wrong
addresses, etc, etc.  If the server doesn't change the body at all,
and only adds new headers, then I think it should not do this.  In the
grey areas like Mailman, it's unclear.

However, suppose Mailman is configured to leave the headers alone, and
to leave the body alone, forwarding verbatim to the addresses on the
mailing list (except for adding its List-XXXX headers, etc).  I would
argue that since Mailman doesn't automatically forward, but instead
checks for spam, whether you're subscribed or not, whether the
subscriber is already an addressee in the message, etc., it makes
decisions about what to send where, and is therefore a "user" in the
sense of section 3.6.6.

Mailman SHOULD add Resent-XXXX headers, because if delivery gets
screwed up, bugs in its logic should be considered a candidate cause.
Ie, those headers mean that Mailman accepts partial responsibility for
misdirecting the reinjection of the message into the mail transport
system.

For example, suppose Mailman hiccups and reinjects the same post
twice.  It would be useful to check whether the Resent-Message-Ids
differ.  If they do, you know for sure it was Mailman.  If they don't,
it doesn't prove it wasn't Mailman, but you do know that the phase
where the error occurred was fairly late in Mailman's delivery
pipeline.

    William> IIUC, that is.  Which apparently I do not, having read
    William> through the headers of a message from this list.

While I respect the Mailman authors for their efforts to conform to
the RFCs, nobody's perfect.  Specifically there is controversy about
whether Mailman's treatment of the Sender field is conforming to
RFCs.  There may be other errors (I don't know of any, but I don't
know everything :-).

    William> There is no Sender: field.

This implies that whoever is in From is Sender.

    William> The first field is apparently an unstructured field with
    William> no identifier with the canonical following colon.

This was added by *your* MTA when it delivered to your mailbox.  It
was not present at any time before the message reached your host.
This is a non-standard (but very frequently used) format for mailboxes
called "Unix mbox".

Here "non-standard" does not mean "does not conform to the standard"
for some value of "the".  It means "this is neither defined nor
prohibited by any formal standard."

    William> It contains the sender (mailman-users-bounces...) and the
    William> date, presumably of sending.

The date is a timestamp added when the message was delivered to your
mailbox.

    William> The second field is Return-Path: with an 'addr-spec'.

This was also added by your MTA.  This is a standard header, specified
by RFC 2821 (the RFC for the protocol spoken by one MTA to another).

    William> The originator fields are untouched.

This is not always true.  Mailman will sometimes alter the From header
to preserve privacy, IIRC.

    William> Which means that the list server is neither a re-sender
    William> or a forwarder,

Neither of these have a definition in RFC 2822 (the RFC for the
protocol spoken by MUAs, to each other and to human users).  There are
two suggested interpretations for forwarder, and nothing very specific
about resender, except that it is not the kind of thing the MTAs do.

Note that although technically when Mailman adds a footer it has
created a new message containing a copy of the original, I doubt
*people* think of it that way.  As long as there's a complete,
verbatim copy of the original, the header and footer are considered
decoration, not part of the message.  Even if HTML parts or images are
removed, I think *people* still consider the message "the same".  So I
don't think these operations necessarily disqualify Mailman as a
resender.  (If you convince me that people will often consider these
operations to create a *new* message, then I'd have to rethink.)

As far as I can tell, a re-sender is any agent that (1) receives a
message, (2) sends it elsewhere, and (3) adds Resent-XXXX headers to
the resent message.

You could argue that there's a condition that Mailman, as the
resender, be a "bona fide addressee" of the message.  Well, (1) it's
the envelope recipient and probably in the addressee headers as well,
and (2) Mailman often keeps file copies (AKA mailing list archives).
You could claim that (1) is just an accident of technology, but I
don't think you can explain away (2).

To put all this another way, the only intermediary actually defined by
these RFCs is the "mail transport agent" (MTA), which accepts a sender
address, a list of recipients, and message data from another agent,
and returns to that agent a promise to deliver.  It then contacts
other MTAs that it believes represent some of those recipients, and
passes on the list of (a subset of) recipients, the sender address and
the message data, or delivers the message data verbatim to mailboxes
it has responsibility for.  Once it has received promises to deliver
for all of the recipients on the list it received, it is done, and can
purge the message data and envelope information from its queues.  This
is not what Mailman does, so it's a "user".

Note that a quick search of the RFC site for "delivery agent" or "MDA"
turns up nothing relevant, and a search for "mailing list" turns up
only RFCs 2369 and 2919 (describing the List-XXXX headers, but not the
kind of thing we're talking about) and RFC 1429 which describes an
obsolete service called DISTRIBUTE that is something intermediate
between a mailing list and a Usenet news server (and hosted on Bitnet
*shiver*).  There are no relevant standards yet.

    William> my MUA (MDA, actually: Procmail) is forced to process
    William> this message to its final destination in my mail system
    William> illegally?

I don't really understand what you're trying to say.  I can offer the
following random comments:

"Illegal" of course is a slight overstatement.  :-)  The preferred
word is "nonconforming".

But I don't think it's very useful to talk about "nonconforming
delivery processing" here.  Nothing about the delivery process is
non-conforming for the simple reason that the *only* things that you
can see in the message that reflect delivery processing are the
Received and Return-Path headers, and the (nonstandard) Unix mbox
"From " headline.

The other headers are all for the use of the MUAs and the users
(including Mr. Procmail).

A general remark: You might find RFC 2369 useful as context (and in
the future, to understand Mailman's use of those headers, though it's
not relevant to this thread).


-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.