From mark at msapiro.net  Mon Sep  2 00:43:58 2013
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 01 Sep 2013 15:43:58 -0700
Subject: [Mailman-Developers] Can we send message with attachment using
 UserNotification?
In-Reply-To: <52152AEE.6010707@gmail.com>
References: <52152AEE.6010707@gmail.com>
Message-ID: <5223C32E.9010506@msapiro.net>

On 08/21/2013 02:02 PM, Nghia Vu Truong wrote:
> Hi there,
> Could you please help me with this question!
> Is there anyway we can attach a document to message, sending by
> UserNotification method?


No. Mailman.Message.UserNotification() creates a single part, plain text
message.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From stephen at xemacs.org  Mon Sep  2 03:52:31 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 02 Sep 2013 10:52:31 +0900
Subject: [Mailman-Developers] Can we send message with attachment using
 UserNotification?
In-Reply-To: <5223C32E.9010506@msapiro.net>
References: <52152AEE.6010707@gmail.com>
	<5223C32E.9010506@msapiro.net>
Message-ID: <8761ukc7mo.fsf@uwakimon.sk.tsukuba.ac.jp>

Mark Sapiro writes:
 > On 08/21/2013 02:02 PM, Nghia Vu Truong wrote:
 > > Hi there,
 > > Could you please help me with this question!
 > > Is there anyway we can attach a document to message, sending by
 > > UserNotification method?
 > 
 > 
 > No. Mailman.Message.UserNotification() creates a single part, plain text
 > message.

I think the odds are pretty good that a uuencoded appendix would get
through.  But it's probably risky these days, especially if people are
using recent entrants (< 10 years old) in the MUA market.

From p at sys4.de  Tue Sep  3 11:41:15 2013
From: p at sys4.de (Patrick Ben Koetter)
Date: Tue, 3 Sep 2013 11:41:15 +0200
Subject: [Mailman-Developers] Test: Please ignore
Message-ID: <20130903094112.GA1808@sys4.de>

Are the mailman footer links HTTPS-Links?

p at rick

-- 
[*] sys4 AG
 
http://sys4.de, +49 (89) 30 90 46 64
Franziskanerstra?e 15, 81669 M?nchen
 
Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263
Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
 

From mark at msapiro.net  Thu Sep  5 05:21:00 2013
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 04 Sep 2013 20:21:00 -0700
Subject: [Mailman-Developers] Send message having same "References" from
 archive
In-Reply-To: <CAEU5bHzMabtS5wV_erORQEG9-j0ATi-oMwS8+2Z+Z6StoZxLsA@mail.gmail.com>
References: <CAEU5bHzMabtS5wV_erORQEG9-j0ATi-oMwS8+2Z+Z6StoZxLsA@mail.gmail.com>
Message-ID: <5227F89C.1010807@msapiro.net>

On 08/18/2013 09:20 AM, VuNghia Truong wrote:
>
> Suppose that I have archive file, like "list_name.mbox".
> Now, I want to get the message with same "References" field, and send them
> out.
> How could I accomplish this problem?


from Mailman import Mailbox

mid = 'the message Id you're looking for'
fp = open('/path/to/list_name.mbox')
mb = Mailbox.Mailbox(fp)
msgs = []
for msg in mb:
    refs = msg.get_all('references')
        for ref in refs:
            if ref.find(mid) >= 0:
                msgs.append(msg)
                break

# At this point, msgs contains a list of email.Message.Message objects #
that contain 'the message Id you're looking for' in their References:
# Sending them is left as an exercise for the reader.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From raj.abhilash1 at gmail.com  Wed Sep 11 10:58:21 2013
From: raj.abhilash1 at gmail.com (Abhilash Raj)
Date: Wed, 11 Sep 2013 14:28:21 +0530
Subject: [Mailman-Developers] Testing different email structures with MUAs
Message-ID: <523030AD.2040509@gmail.com>

Hi,

So I was testing which kind of message structure would be the best fit
for sending out messages with more than one signature part. I tried 3
things:

1) Message according to [1]this(having two signature parts in a
'multipart/mixed' part inside 'multipart/signed' message. My
MUA(*mu4e*[2]) fails to find any signature part at all in this

2) A 'multipart/alternative' message with each part having same
structure and each one having different signature. Now although from
rfc2046[3] it is not a thumb rule to have different 'content-type' for
each alternative part in a 'multipart/alternative', but I think MUAs
would get confused if they find both alternative parts completely
same(in structure). My MUA shows both parts and thus repeats data.
Although it shows two signature parts too.

3) This third type was just my idea, does not follow rfc3156 obviously.
I simply added another signature part to the original message and my MUA
reads it nicely showing two signatures.

I have attached all 3 type of message, each in a different file. Please
can you place it in your maildir and check how your MUAs respond to it
and report here? The message signature will not be verified(the
signature text is actually gibberish), this experiment is just to check
how the MUAs handle the message with such a structure.



[1]: http://tools.ietf.org/html/draft-ietf-openpgp-multsig-02
[2]: http://www.djcbsoftware.nl/code/mu/mu4e.html
[3]: http://tools.ietf.org/html/rfc2046#section-5.1.4
-- 
Abhilash Raj
-------------- next part --------------
From: anne at example.com
To: test at example.com
Subject: A Message with non-ascii body
Message-ID: <ant>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="===============0165780934=="
protocol: multipart/mixed

--===============0165780934==
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

A message body.
--===============0165780934==
Content-Type: multipart/mixed; boundary="===============1396434060=="
MIME-Version: 1.0

--===============1396434060==
Content-Type: application/pgp-signature
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2

iQCVAwUBMJrRF2N9oWBghPDJAQE9UQQAtl7LuRVndBjrk4EqYBIb3h5QXIX/LC//
jJV5bNvkZIGPIcEmI5iFd9boEgvpirHtIREEqLQRkYNoBActFBZmh9GC3C041WGq
uMbrbxc+nIs1TIKlA08rVi9ig/2Yh7LFrK5Ein57U/W72vgSxLhe/zhdfolT9Brn
HOxEa44b+EI==ndaj
-----END PGP SIGNATURE-----

--===============1396434060==
Content-Type: application/pgp-signature
                          

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2

iQCVAwUBMJrRF2N9oWBghPDJAQE9UQQAtl7LuRVndBjrk4EqYBIb3h5QXIX/LC//
jJV5bNvkZIGPIcEmI5iFd9boEgvpirHtIREEqLQRkYNoBActFBZmh9GC3C041WGq
uMbrbxc+nIs1TIKlA08rVi9ig/2Yh7LFrK5Ein57U/W72vgSxLhe/zhdfolT9Brn
HOxEa44b+EI==ndaj
-----END PGP SIGNATURE-----
--===============1396434060==--
--===============0165780934==--
-------------- next part --------------
From: anne at example.com
To: test at example.com
Subject: A Message with non-ascii body
Message-ID: <ant>
MIME-Version: 1.0
content-type: multipart/alternative; boundary="===============1658465758=="

--===============1658465758==
MIME-Version: 1.0
Content-Type: multipart/signed; protocol="application/pgp-signature";
 boundary="===============1005232601=="

--===============1005232601==
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

A message body.
--===============1005232601==
Content-Type: application/pgp-signature
                          

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2

iQCVAwUBMJrRF2N9oWBghPDJAQE9UQQAtl7LuRVndBjrk4EqYBIb3h5QXIX/LC//
jJV5bNvkZIGPIcEmI5iFd9boEgvpirHtIREEqLQRkYNoBActFBZmh9GC3C041WGq
uMbrbxc+nIs1TIKlA08rVi9ig/2Yh7LFrK5Ein57U/W72vgSxLhe/zhdfolT9Brn
HOxEa44b+EI==ndaj
-----END PGP SIGNATURE-----
--===============1005232601==--
--===============1658465758==
MIME-Version: 1.0
Content-Type: multipart/signed; protocol="application/pgp-signature";
 boundary="===============1174598811=="

--===============1174598811==
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

A message body.
--===============1174598811==
Content-Type: application/pgp-signature
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2

iQCVAwUBMJrRF2N9oWBghPDJAQE9UQQAtl7LuRVndBjrk4EqYBIb3h5QXIX/LC//
jJV5bNvkZIGPIcEmI5iFd9boEgvpirHtIREEqLQRkYNoBActFBZmh9GC3C041WGq
uMbrbxc+nIs1TIKlA08rVi9ig/2Yh7LFrK5Ein57U/W72vgSxLhe/zhdfolT9Brn
HOxEa44b+EI==ndaj
-----END PGP SIGNATURE-----

--===============1174598811==--
--===============1658465758==--
-------------- next part --------------
From: anne at example.com
To: test at example.com
Subject: A Message with non-ascii body
Message-ID: <ant>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary=bar; micalg=pgp-md5;
 protocol="application/pgp-signature"

--bar
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

A message body.
--bar
Content-Type: application/pgp-signature
                          

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2

iQCVAwUBMJrRF2N9oWBghPDJAQE9UQQAtl7LuRVndBjrk4EqYBIb3h5QXIX/LC//
jJV5bNvkZIGPIcEmI5iFd9boEgvpirHtIREEqLQRkYNoBActFBZmh9GC3C041WGq
uMbrbxc+nIs1TIKlA08rVi9ig/2Yh7LFrK5Ein57U/W72vgSxLhe/zhdfolT9Brn
HOxEa44b+EI==ndaj
-----END PGP SIGNATURE-----
--bar
Content-Type: application/pgp-signature
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2

iQCVAwUBMJrRF2N9oWBghPDJAQE9UQQAtl7LuRVndBjrk4EqYBIb3h5QXIX/LC//
jJV5bNvkZIGPIcEmI5iFd9boEgvpirHtIREEqLQRkYNoBActFBZmh9GC3C041WGq
uMbrbxc+nIs1TIKlA08rVi9ig/2Yh7LFrK5Ein57U/W72vgSxLhe/zhdfolT9Brn
HOxEa44b+EI==ndaj
-----END PGP SIGNATURE-----

--bar--

From adam-mailman at amyl.org.uk  Wed Sep 11 12:57:28 2013
From: adam-mailman at amyl.org.uk (Adam McGreggor)
Date: Wed, 11 Sep 2013 11:57:28 +0100
Subject: [Mailman-Developers] Testing different email structures with
	MUAs
In-Reply-To: <523030AD.2040509@gmail.com>
References: <523030AD.2040509@gmail.com>
Message-ID: <20130911105728.GO3728@hendricks.amyl.org.uk>

On Wed, Sep 11, 2013 at 02:28:21PM +0530, Abhilash Raj wrote:
> I have attached all 3 type of message, each in a different file. Please
> can you place it in your maildir and check how your MUAs respond to it
> and report here? The message signature will not be verified(the
> signature text is actually gibberish), this experiment is just to check
> how the MUAs handle the message with such a structure.

A more representative sample of what users (real users!) might see,
might be better gained through some of the online tools; litmus
<http://litmus.com/email-testing> may be useful.


-- 
"Cabbage-- the opinions of taxi drivers."
  (new definitions, from `I'm Sorry I Haven't a Clue')

From dkg at fifthhorseman.net  Thu Sep 12 01:36:05 2013
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Wed, 11 Sep 2013 19:36:05 -0400
Subject: [Mailman-Developers] Testing different email structures with
 MUAs
In-Reply-To: <20130911105728.GO3728@hendricks.amyl.org.uk>
References: <523030AD.2040509@gmail.com>
 <20130911105728.GO3728@hendricks.amyl.org.uk>
Message-ID: <5230FE65.7080506@fifthhorseman.net>

On 09/11/2013 06:57 AM, Adam McGreggor wrote:
> On Wed, Sep 11, 2013 at 02:28:21PM +0530, Abhilash Raj wrote:
>> I have attached all 3 type of message, each in a different file. Please
>> can you place it in your maildir and check how your MUAs respond to it
>> and report here? The message signature will not be verified(the
>> signature text is actually gibberish), this experiment is just to check
>> how the MUAs handle the message with such a structure.

thanks, abhilash, i will try to test these shortly and give you feedback
about icedove and notmuch at least, both of which are OpenPGP-enabled
(notmuch is native, icedove with the enigmail plugin).

> A more representative sample of what users (real users!) might see,
> might be better gained through some of the online tools; litmus
> <http://litmus.com/email-testing> may be useful.

Litmus appears to be about HTML rendering, which isn't what abilhash is
asking about at all.  If you know of a comparable service that provides
this kind of comprehensive feedback about cryptographic signatures, i'd
be very interested in learning about it.

Regards,

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1027 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20130911/8c586213/attachment.sig>

From stephen at xemacs.org  Thu Sep 12 02:44:31 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 12 Sep 2013 09:44:31 +0900
Subject: [Mailman-Developers] Testing different email structures with MUAs
In-Reply-To: <523030AD.2040509@gmail.com>
References: <523030AD.2040509@gmail.com>
Message-ID: <87ppse7tsg.fsf@uwakimon.sk.tsukuba.ac.jp>

Abhilash Raj writes:

 > I have attached all 3 type of message, each in a different file. Please
 > can you place it in your maildir and check how your MUAs respond to it
 > and report here? The message signature will not be verified(the
 > signature text is actually gibberish), this experiment is just to check
 > how the MUAs handle the message with such a structure.

I don't understand what you think you will learn from this experiment.
We're not interested (for your purposes) in what MUAs do with broken
messages, including those that can't be validated.  Your code simply
should never emit them.  (OTOH, we are interested in what your code
will do with broken messages: it should trap them.)

In particular, in most cases the MUA will parse the multipart
structures and tell you what they've found in one way or another.
This is true of signed message parts.  It's not unreasonable to
suppose that some messages will contain additional content after the
signed part; the MUA needs to determine the boundaries of the part in
any case.


From dkg at fifthhorseman.net  Thu Sep 12 07:49:09 2013
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Thu, 12 Sep 2013 01:49:09 -0400
Subject: [Mailman-Developers] Testing different email structures with
	MUAs
In-Reply-To: <87ppse7tsg.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <523030AD.2040509@gmail.com>
 <87ppse7tsg.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <523155D5.1050207@fifthhorseman.net>

On 09/11/2013 08:44 PM, Stephen J. Turnbull wrote:
> Abhilash Raj writes:
> 
>  > I have attached all 3 type of message, each in a different file. Please
>  > can you place it in your maildir and check how your MUAs respond to it
>  > and report here? The message signature will not be verified(the
>  > signature text is actually gibberish), this experiment is just to check
>  > how the MUAs handle the message with such a structure.
> 
> I don't understand what you think you will learn from this experiment.
> We're not interested (for your purposes) in what MUAs do with broken
> messages, including those that can't be validated.  Your code simply
> should never emit them.  (OTOH, we are interested in what your code
> will do with broken messages: it should trap them.)

I think Abhilash is modeling what the messages emitted by a re-signing
mailing list might look like.

We've been discussing several options about whether a message should be
re-signed by the mailing list before being sent to the subscribers,
whether the original author's signature should be kept as well, and if
both signatures are present, how they should be attached to the message.

He's created these messages so that people can view them in their
OpenPGP-compatible MUAs and see how the message signatures are displayed
to the user.

> In particular, in most cases the MUA will parse the multipart
> structures and tell you what they've found in one way or another.
> This is true of signed message parts.  It's not unreasonable to
> suppose that some messages will contain additional content after the
> signed part; the MUA needs to determine the boundaries of the part in
> any case.

I'd actually argue that it *is* unreasonable in the current ecosystem to
include some non-signed content after the signed part.  Most
OpenPGP-compliant MUAs that i've seen (thunderbird+enigmail in
particular [0]) cannot clearly indicate to the user which part of a
multipart, partially-signed message was the signed part.

[0] see the thread starting at
https://lists.enigmail.net/pipermail/enigmail-users_enigmail.net/2013-March/000721.html

Mailman is perhaps the most common generator of messages like this, such
that icedove+enigmail currently makes the tradeoff to permit what is
effectively a known signature-validation spoofing attack rather than
make people think that mailman is stripping signatures from their messages.

If we could make mailman tuck its footer within a larger signature, that
would be great.

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1027 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20130912/b8982278/attachment.sig>

From stephen at xemacs.org  Thu Sep 12 09:11:33 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 12 Sep 2013 16:11:33 +0900
Subject: [Mailman-Developers] Testing different email structures
	with	MUAs
In-Reply-To: <523155D5.1050207@fifthhorseman.net>
References: <523030AD.2040509@gmail.com>
 <87ppse7tsg.fsf@uwakimon.sk.tsukuba.ac.jp>
 <523155D5.1050207@fifthhorseman.net>
Message-ID: <87mwni7bve.fsf@uwakimon.sk.tsukuba.ac.jp>

Daniel Kahn Gillmor writes:
 > On 09/11/2013 08:44 PM, Stephen J. Turnbull wrote:
 > > Abhilash Raj writes:
 > > 
 > >  > I have attached all 3 type of message, each in a different file. Please
 > >  > can you place it in your maildir and check how your MUAs respond to it
 > >  > and report here? The message signature will not be verified(the
 > >  > signature text is actually gibberish), this experiment is just to check
 > >  > how the MUAs handle the message with such a structure.
 > > 
 > > I don't understand what you think you will learn from this experiment.
 > > We're not interested (for your purposes) in what MUAs do with broken
 > > messages, including those that can't be validated.  Your code simply
 > > should never emit them.  (OTOH, we are interested in what your code
 > > will do with broken messages: it should trap them.)
 > 
 > I think Abhilash is modeling what the messages emitted by a re-signing
 > mailing list might look like.

Yes, I know that.  I don't think his proposed experiment is valid,
however; he needs to use messages with valid signatures, because that
is what Mailman will produce.

 > He's created these messages so that people can view them in their
 > OpenPGP-compatible MUAs and see how the message signatures are displayed
 > to the user.

And if they do anything but display a warning that the signed part
didn't correctly validate and are you really sure you want to look at
the possibly radioactive waste in the payload?, they're broken, no?
But that's not our problem, and it's especially not Abhilash's.

 > > In particular, in most cases the MUA will parse the multipart
 > > structures and tell you what they've found in one way or another.
 > > This is true of signed message parts.  It's not unreasonable to
 > > suppose that some messages will contain additional content after the
 > > signed part; the MUA needs to determine the boundaries of the part in
 > > any case.
 > 
 > I'd actually argue that it *is* unreasonable in the current ecosystem to
 > include some non-signed content after the signed part.

You've got the wrong end of the Postel principle.  I'm saying you will
*see* such mail, not that you should *produce* it.  You immediately
provide evidence that it's common, and that at least one OpenPGP-
wannabe does the wrong thing with it!

 > Most OpenPGP-compliant MUAs that i've seen (thunderbird+enigmail in
 > particular [0]) cannot clearly indicate to the user which part of a
 > multipart, partially-signed message was the signed part.

Wonderful. :-(

 > If we could make mailman tuck its footer within a larger signature, that
 > would be great.

So you're proposing this, I guess:

    multipart/signed
        multipart/mixed
            text/whatever               # optional mailman header
            multipart/signed
                text/whatever           # original signed content
                application/signature
            text/whatever               # optional mailman footer
        application/signature

But the question is not whether Mailman can do that; it's trivial to
produce it by moving the signing handler later in the pipeline.  The
question is whether OpenPGP-wannabe MUAs will do anything reasonable
with it, FVO of "reasonable" that include "letting the user know what
exactly was validated".  Do you really think MUAs will do anything
"reasonable" with it when they do this:

 > Mailman is perhaps the most common generator of messages like this,
 > such that icedove+enigmail currently makes the tradeoff to permit
 > what is effectively a known signature-validation spoofing attack
 > rather than make people think that mailman is stripping signatures
 > from their messages.

I don't believe my eyes.  The MUA is passing off invalid data as
valid, and you're saying Mailman should cater to that MUA?  The sooner
users realize such MUAs are broken by design, the better!  Better they
should bitch about Mailman (at least on Mailman channels, where we can
explain to them what the real problem is).

Steve

From dkg at fifthhorseman.net  Fri Sep 13 06:24:33 2013
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Fri, 13 Sep 2013 00:24:33 -0400
Subject: [Mailman-Developers] Testing different email structures with
 MUAs
In-Reply-To: <87mwni7bve.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <523030AD.2040509@gmail.com>
 <87ppse7tsg.fsf@uwakimon.sk.tsukuba.ac.jp>
 <523155D5.1050207@fifthhorseman.net>
 <87mwni7bve.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <52329381.3010607@fifthhorseman.net>

On 09/12/2013 03:11 AM, Stephen J. Turnbull wrote:

> So you're proposing this, I guess:
> 
>     multipart/signed
>         multipart/mixed
>             text/whatever               # optional mailman header
>             multipart/signed
>                 text/whatever           # original signed content
>                 application/signature
>             text/whatever               # optional mailman footer
>         application/signature

yes, that's exactly what i was proposing.  Abhilash, can your code
produce messages like this?

> But the question is not whether Mailman can do that; it's trivial to
> produce it by moving the signing handler later in the pipeline. 

Great!  That's well-structured data, that should be able to be
legitimately rendered by any OpenPGP-compliant MUA, even ones that can
only provide validation information for messages as a whole.

If Mailman did this regularly instead of creating the common anti-pattern:

         multipart/mixed
             text/whatever               # optional mailman header
             multipart/signed
                 text/whatever           # original signed content
                 application/signature
             text/whatever               # optional mailman footer

then those MUAs like icedove that currently do the wrong thing might be
less likely to try to do it anyway.

Note that Icedove/Thunderbird refuse to show any validation information
for S/MIME-signed messages that are forwarded through mailman with
headers or footers attached like the above structure.

> I don't believe my eyes.  The MUA is passing off invalid data as
> valid, and you're saying Mailman should cater to that MUA?  The sooner
> users realize such MUAs are broken by design, the better!  Better they
> should bitch about Mailman (at least on Mailman channels, where we can
> explain to them what the real problem is).

that's decidedly not what i'm saying.  I'm just pointing out that
mailman commonly produces what you've called "invalid data", and that
its common production of that "invalid data" is precisely what this
MUA's author cites as something he wants to be able to validate instead
of hiding the main message contents' openpgp signature entirely. [0]

I'm not saying the enigmail folks are doing the right thing here --
there's more than enough bugs and blame to go around here, if we want to
get testy :P  (including the fact that thunderbird's UI makes a total
botch of display of MIME parts themselves, which makes it difficult to
attach any verification UI element to anything but the message as a whole).

But producing messages is what mailman does, so maybe we fix the
message-producing mailman wackiness on the mailman list and save fixing
the enigmail message-displaying wackiness for the enigmail list :)

Regards,

	--dkg

[0]
http://thread.gmane.org/gmane.comp.mozilla.enigmail.general/17707/focus=17861

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1027 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20130913/c15a7682/attachment.sig>

From dkg at fifthhorseman.net  Fri Sep 13 06:29:24 2013
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Fri, 13 Sep 2013 00:29:24 -0400
Subject: [Mailman-Developers] Testing different email structures with
	MUAs
In-Reply-To: <523030AD.2040509@gmail.com>
References: <523030AD.2040509@gmail.com>
Message-ID: <523294A4.20500@fifthhorseman.net>

On 09/11/2013 04:58 AM, Abhilash Raj wrote:

> I have attached all 3 type of message, each in a different file. Please
> can you place it in your maildir and check how your MUAs respond to it
> and report here? The message signature will not be verified(the
> signature text is actually gibberish), this experiment is just to check
> how the MUAs handle the message with such a structure.

I've updated those files slightly so that they each have a different
Message-ID: and Subject: header, otherwise they won't be distinguishable
in some MUAs.  I've also added a Date: header for standards compliance.

Here are snapshots of icedove and notmuch (please excuse my horrid color
schemes):

  http://dkg.fifthhorseman.net/src/mailman/multisigned-images/

none of them look particularly good :/

If i get a chance to test other MUAs, i'll update that page and let you
know.

Regards,

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1027 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20130913/213dfb00/attachment.sig>

From dkg at fifthhorseman.net  Fri Sep 13 07:14:22 2013
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Fri, 13 Sep 2013 01:14:22 -0400
Subject: [Mailman-Developers] Testing different email structures with
 MUAs
In-Reply-To: <523294A4.20500@fifthhorseman.net>
References: <523030AD.2040509@gmail.com> <523294A4.20500@fifthhorseman.net>
Message-ID: <52329F2E.9080904@fifthhorseman.net>

On 09/13/2013 12:29 AM, Daniel Kahn Gillmor wrote:
>   http://dkg.fifthhorseman.net/src/mailman/multisigned-images/

I've added a fourth message, with a variant on the content wrapping
structure stephen and i were just talking about:

???multipart/signed 11903 bytes
 ???multipart/mixed 8561 bytes
 ????text/plain 97 bytes
 ????message/rfc822 inline 8133 bytes
 ? ???multipart/mixed 7921 bytes
 ?  ???multipart/signed 3270 bytes
 ?  ????text/plain 1825 bytes
 ?  ????application/pgp-signature attachment 897 bytes
 ?  ???text/plain inline 171 bytes
 ???application/pgp-signature attachment 1027 bytes

And i've tested them all with claws mail as well, using the pgpmime
plugin, and updated the display page to include the other screenshots.

hth,

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1027 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20130913/667d2ede/attachment.sig>

From stephen at xemacs.org  Fri Sep 13 07:23:00 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 13 Sep 2013 14:23:00 +0900
Subject: [Mailman-Developers] Testing different email structures with
 MUAs
In-Reply-To: <52329381.3010607@fifthhorseman.net>
References: <523030AD.2040509@gmail.com>
 <87ppse7tsg.fsf@uwakimon.sk.tsukuba.ac.jp>
 <523155D5.1050207@fifthhorseman.net>
 <87mwni7bve.fsf@uwakimon.sk.tsukuba.ac.jp>
 <52329381.3010607@fifthhorseman.net>
Message-ID: <87li3170sr.fsf@uwakimon.sk.tsukuba.ac.jp>

Daniel Kahn Gillmor writes:

 > I'm just pointing out that mailman commonly produces what you've
 > called "invalid data",

In the OpenPGP sense that the whole message cannot be considered to be
validly signed, even though it may contain a multipart/signed part
with a valid signature.

 > and that its common production of that "invalid data" is precisely
 > what this MUA's author cites as something he wants to be able to
 > validate instead of hiding the main message contents' openpgp
 > signature entirely. [0]

How is that relevant to us?  No matter how you slice it, if Mailman
does its thing of adding a header or footer, the MUA has to dig into
MIME structure and validate a subpart.  Sure, in Abhilash's scheme
Mailman will be validating the subpart as a service to lazy (?) or
anonymous subscribers, but a PUCT[1] will want to double-check that
Mailman did what it claims to do.

 > But producing messages is what mailman does, so maybe we fix the
 > message-producing mailman wackiness on the mailman list

It's *not* wackiness.  It's perfectly standard-conforming, and I see
no reason why people who currently don't sign messages, and don't want
to ask Mailman to do so because the necessary infrastructure is user-
hostile, should be punished or be criticized for producing such messages.

My point is that I have no objection to trying to create valid
messages that will validate correctly on as many MUAs as possible.
What I object vehemently to is the idea that what a broken MUA (such
as TB-E) does is a valid test of anything Mailman does.  Especially
not with a broken message.

I also have no objection to Mailman lists simply signing everything,
so that they can advertise that they do.  (OTOH, this is already more
or less fulfilled by DKIM, so it's a niche use case.)


Footnotes: 
[1]  Paranoid User of a Certain Type.  Ie, trusts the author but not
the list.



From franck at peachymango.org  Fri Sep 13 07:44:40 2013
From: franck at peachymango.org (Franck Martin)
Date: Thu, 12 Sep 2013 22:44:40 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
	2.1.16
Message-ID: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>

In the upcoming mailman 2.1.16 there has been the introduction of the optional feature author_is_list

"Replace the sender with the list address to conform with policies like ADSP and DMARC. It replaces the poster's address in the From: header with the list address and adds the poster to the Reply-To: header, but the anonymous_list and Reply-To: header munging settings below take priority. If setting this to Yes, it is advised to set the MTA to DKIM sign all emails. "

Usage of this feature on lists have indicated no adverse issue for the members, provided proper communication is done when the feature is enabled (as to allow inbox rules to be changed if needed).

In the 2.1.16 Release Candidate the feature author_is_list is controlled by the following switches in the mm_cfg.py

ALLOW_AUTHOR_IS_LIST = No
DEFAULT_AUTHOR_IS_LIST = No

The first one will enable the GUI to present an option to the list admin to enable the author_is_list feature, the second controls if new lists or upgraded lists should have the author_is_list feature set to yes

While it is better for the MTA to DKIM sign emails if author_is_list is enabled, this is not a requirement and the list will work fine without DKIM.

While DEFAULT_AUTHOR_IS_LIST is important to avoid new lists and upgraded lists change behavior, setting ALLOW_AUTHOR_IS_LIST to Yes does not change how any list is handled (author_is_list in the GUI is No by default). it only provides an option to the list admin to change the list behavior.

Unfortunately some list admins cannot edit mm_cfg.py (like CPANEL type install), therefore it would be nice to remove ALLOW_AUTHOR_IS_LIST or set it to Yes by default to let the list admin decide how he/she wants the list to behave. Otherwise the list admin needs to contact the mailman admin to enable this config.

Please indicates if you are ok to set ALLOW_AUTHOR_IS_LIST to Yes or remove this option from mm_cfg.py

Note: a few organizations have indicated they will provide the necessary translation of this feature in the GUI for the most common languages.


From sm at resistor.net  Fri Sep 13 08:30:22 2013
From: sm at resistor.net (SM)
Date: Thu, 12 Sep 2013 23:30:22 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
Message-ID: <6.2.5.6.2.20130912232809.0bb71f18@resistor.net>

Hi Franck,
At 22:44 12-09-2013, Franck Martin wrote:
>In the upcoming mailman 2.1.16 there has been the introduction of 
>the optional feature author_is_list
>
>"Replace the sender with the list address to conform with policies 
>like ADSP and DMARC. It replaces the poster's address in the From: 
>header with the list address and adds the poster to the Reply-To: 
>header, but the anonymous_list and Reply-To: header munging settings 
>below take priority. If setting this to Yes, it is advised to set 
>the MTA to DKIM sign all emails. "

There is an effort (not mailman-related) to mark ADSP as not recommended.

Regards,
-sm 


From franck at peachymango.org  Fri Sep 13 16:13:35 2013
From: franck at peachymango.org (Franck Martin)
Date: Fri, 13 Sep 2013 07:13:35 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
	2.1.16
In-Reply-To: <WM!6882d98413eeae4436d034da5fb9c7aa0b6d550ce296c24d434aac4544bfec48bba30ebe7f0fbc1cdb48fda16416856f!@asav-1.01.com>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <6.2.5.6.2.20130912232809.0bb71f18@resistor.net>
 <WM!6882d98413eeae4436d034da5fb9c7aa0b6d550ce296c24d434aac4544bfec48bba30ebe7f0fbc1cdb48fda16416856f!@asav-1.01.com>
Message-ID: <B69BB242-04FB-49A8-AAEE-16DF6E4C9409@peachymango.org>


On Sep 12, 2013, at 11:30 PM, SM <sm at resistor.net> wrote:

> Hi Franck,
> At 22:44 12-09-2013, Franck Martin wrote:
>> In the upcoming mailman 2.1.16 there has been the introduction of the optional feature author_is_list
>> 
>> "Replace the sender with the list address to conform with policies like ADSP and DMARC. It replaces the poster's address in the From: header with the list address and adds the poster to the Reply-To: header, but the anonymous_list and Reply-To: header munging settings below take priority. If setting this to Yes, it is advised to set the MTA to DKIM sign all emails. "
> 
> There is an effort (not mailman-related) to mark ADSP as not recommended.
> 
yes I'm aware, it will take a bit of time before the ADSP RFC move to historical status. We are not doing this feature for ADSP...but DMARC.  I guess we will drop the word ADSP for 2.1.17 ;)


From barry at list.org  Fri Sep 13 17:06:05 2013
From: barry at list.org (Barry Warsaw)
Date: Fri, 13 Sep 2013 11:06:05 -0400
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
Message-ID: <20130913110605.7ea598a2@anarchist>

On Sep 12, 2013, at 10:44 PM, Franck Martin wrote:

>Unfortunately some list admins cannot edit mm_cfg.py (like CPANEL type
>install), therefore it would be nice to remove ALLOW_AUTHOR_IS_LIST or set it
>to Yes by default to let the list admin decide how he/she wants the list to
>behave. Otherwise the list admin needs to contact the mailman admin to enable
>this config.
>
>Please indicates if you are ok to set ALLOW_AUTHOR_IS_LIST to Yes or remove
>this option from mm_cfg.py

I will leave it to Mark for final decision on this, but my own opinion is that
the mm_cfg.py option should stay.  cPanel already customizes their Mailman
installation, so I think they should set it to Yes when they upgrade their
systems to 2.1.16.

-Barry

From mark at msapiro.net  Fri Sep 13 20:31:44 2013
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 13 Sep 2013 11:31:44 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <20130913110605.7ea598a2@anarchist>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <20130913110605.7ea598a2@anarchist>
Message-ID: <52335A10.60603@msapiro.net>

On 09/13/2013 08:06 AM, Barry Warsaw wrote:
> 
> I will leave it to Mark for final decision on this, but my own opinion is that
> the mm_cfg.py option should stay.  cPanel already customizes their Mailman
> installation, so I think they should set it to Yes when they upgrade their
> systems to 2.1.16.


I don't feel strongly about this either way except for the general
principle of least surprise. Enabling this by default has three
downsides that I see. It can render a fully i18n translated General
Options page a bit ugly with one relatively large English paragraph; it
gives list owners yet another bullet with which to shoot themselves in
the foot, and it complicates list configuration by adding yet another
decision.

None of these is a deal breaker. I researched the i18n issue, and it
turns out only 4 languages currently have a fully translated General
Options page. One of these has already been updated and the other 3 are
being addressed. Most languages already have between 1 and 3
untranslated strings on this page from prior changes so it could be
argued that one more is not important.

The other two considerations are relatively minor, but I still lean
towards requiring overt action by the site admin to enable the feature.

I wanted this brought to mailman-developers in the hope that whatever
discussion ensued would lead to some consensus.

I confess, I'm not at all up to speed on DMARC. Franck has assured me
that this feature can be useful even in the absence of the DNS and MTA
changes necessary to DKIM sign outgoing list mail, but it seems to me
that enabling author_is_list will almost guarantee that any incoming
DKIM signatures will be broken (the From: is almost certainly included
in the signed headers) which will cause problems if the outgoing mail is
not signed with a valid DKIM signature.

Also, it seems that an installation would want to validate in some way
incoming mail before taking responsibility, even in a minor way, for
resending it.

All of this leads me to think that making this available to list owners
should be an installation decision rather than being done by default.

Please help me understand if I'm wrong.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From stephen at xemacs.org  Fri Sep 13 21:13:02 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 14 Sep 2013 04:13:02 +0900
Subject: [Mailman-Developers] Author_is_list option in upcoming
	mailman	2.1.16
In-Reply-To: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
Message-ID: <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>

Franck Martin writes:

 > In the upcoming mailman 2.1.16 there has been the introduction of
 > the optional feature author_is_list
 >
 > "Replace the sender

Before you release, s/sender/author/, please.  When discussing
Internet email, sender != author.  The name of the feature, "author is
list", is an obvious falsehood: lists don't write posts, they relay
them.  These policies do not conform to the email RFCs.  (Given the
semantics of "From" set out in RFC 5322, they may even constitute
copyright infringement in the absence of a license from posters
permitting From-munging.  But that's not the topic here.)

AFAICS there's an easy way for Mailman to adapt to DKIM without
violating RFC 5322: wrap every message in a MIME message/rfc822 part
(ie, every message is a one-post "digest").  The wrapper message
obviously can conform to DMARC ("From: LIST at DOMAIN" with appropriate
DKIM signature), and Mailman can DTRT with the wrapped message.

 > with the list address to conform with policies like ADSP and DMARC.
 > It replaces the poster's address in the From: header with the list
 > address and adds the poster to the Reply-To: header,

Another RFC violation. :-(  What if the poster already set Reply-To
because she *doesn't* want mail at the From address?  What if the
list's address *isn't* in Reply-to, but the author expects wide
replies to go to the list?

 > but the anonymous_list

This is probably OK.

 > and Reply-To: header munging settings below take priority.

Does this make sense?  See above.

 > If setting this to Yes, it is advised to set the MTA to DKIM sign
 > all emails. "

Please change this to "This setting is useful when your host signs
outgoing mail according using DKIM."  As written, the advice is
inaccurate anyway.  DKIM requires more than just MTA settings.  There
must be cooperation from the nameserver.

 > Usage of this feature on lists have indicated no adverse issue for
 > the members,

s/no adverse issue/only minor annoyance/ (I disagree with "minor", but
sure, Outlook users probably won't notice).

You should remember that "Reply-To munging" works as expected for
almost all subscribers almost all the time on most lists, and for that
reason is widely used despite its ex post obvious deficiencies.  When
it fails, though, it's at minimum a minor pain in the ass and at
maximum an inadvertant privacy violation.

Please go slowly on screwing with From, which (unlike Reply-To) is a
required field and so appears in *every* email and has well-defined
semantics *with which this feature is in deliberate non-conformance*.

 > provided proper communication is done when the feature is enabled
 > (as to allow inbox rules to be changed if needed).

Isn't this a lie?  If the From header is corrupt, then there is no
reliable indication of who the author is.  If you're lucky you can
fish it out of the body or .signature.  Reply-To won't do: not only
are its semantics such that there's no guarantee that it's an alias
for author, but it complicates the rules significantly because you
need different rules for From-corrupting lists and other lists and
non-list mail.

 > In the 2.1.16 Release Candidate the feature author_is_list is
 > controlled by the following switches in the mm_cfg.py
 > 
 > ALLOW_AUTHOR_IS_LIST = No
 > DEFAULT_AUTHOR_IS_LIST = No
 > 
 > The first one will enable the GUI to present an option to the list
 > admin to enable the author_is_list feature, the second controls if
 > new lists or upgraded lists should have the author_is_list feature
 > set to yes

"Upgraded" lists?  If upgrading to Mailman 2.1.16 causes all my lists
to be corrupted by this feature, I will not be pleased.  An option
called DEFAULT should only apply to new lists.

If you insist on making this a fallback if the list doesn't have an
explicit setting, call the option AUTHOR_IS_LIST.

 > While it is better for the MTA to DKIM sign emails if
 > author_is_list is enabled, this is not a requirement and the list
 > will work fine without DKIM.

But why would anybody want to do this in the absence of DKIM?  Mailman
already has the RFC 2369 and 2919 fields to tell MUAs that it's a list
post, and a plethora of ways (Subject, header, footer) to tell users
that it's a list post.  Anonymous list is already an option for
obscuring the author if that's desirable, and for an announce list
there's no problem, as the list (or an officer of the host) is already
the author.  I think you should just delete that sentence.

 > While DEFAULT_AUTHOR_IS_LIST is important to avoid new lists and
 > upgraded lists change behavior, setting ALLOW_AUTHOR_IS_LIST to Yes
 > does not change how any list is handled (author_is_list in the GUI
 > is No by default). it only provides an option to the list admin to
 > change the list behavior.
 > 
 > Unfortunately some list admins cannot edit mm_cfg.py (like CPANEL
 > type install), therefore it would be nice to remove
 > ALLOW_AUTHOR_IS_LIST or set it to Yes by default to let the list
 > admin decide how he/she wants the list to behave. Otherwise the
 > list admin needs to contact the mailman admin to enable this
 > config.

I'm reluctantly willing to accept a factory default that allows list
owners to decide to systematically violate RFC 5322 (as well as all
predecessors) only if the factory default of DEFAULT_AUTHOR_IS_LIST=No.

 > Please indicates if you are ok to set ALLOW_AUTHOR_IS_LIST to Yes
 > or remove this option from mm_cfg.py

I don't see any obvious loss from removing it, but in keeping with the
general "go slow" attitude toward implementing RFC-violating features,
I think we should keep it for now, but set it to Yes.

In fact, we SHOULD remove DEFAULT_AUTHOR_IS_LIST.  List owners should
make an informed decision to violate RFC 5322, not be defaulted into
doing so.  If site owners want to enforce it, let them hack code.



From franck at peachymango.org  Fri Sep 13 21:18:31 2013
From: franck at peachymango.org (Franck Martin)
Date: Fri, 13 Sep 2013 14:18:31 -0500 (CDT)
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <WM!7d7792df27ac7c6b5c227389aff7355284c982f9e0b3f907b23c32ab2282022a72acd54c0ac4876a657d99bf13ca8139!@asav-1.01.com>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <20130913110605.7ea598a2@anarchist> <52335A10.60603@msapiro.net>
 <WM!7d7792df27ac7c6b5c227389aff7355284c982f9e0b3f907b23c32ab2282022a72acd54c0ac4876a657d99bf13ca8139!@asav-1.01.com>
Message-ID: <463863369.23580.1379099911972.JavaMail.zimbra@peachymango.org>


----- Original Message -----
> From: "Mark Sapiro" <mark at msapiro.net>
> To: mailman-developers at python.org
> Sent: Friday, September 13, 2013 11:31:44 AM
> Subject: Re: [Mailman-Developers] Author_is_list option in upcoming mailman 2.1.16
> 
> On 09/13/2013 08:06 AM, Barry Warsaw wrote:
> > 
> > I will leave it to Mark for final decision on this, but my own opinion is
> > that
> > the mm_cfg.py option should stay.  cPanel already customizes their Mailman
> > installation, so I think they should set it to Yes when they upgrade their
> > systems to 2.1.16.
> 
> 
> I don't feel strongly about this either way except for the general
> principle of least surprise. Enabling this by default has three
> downsides that I see. It can render a fully i18n translated General
> Options page a bit ugly with one relatively large English paragraph; it
> gives list owners yet another bullet with which to shoot themselves in
> the foot, and it complicates list configuration by adding yet another
> decision.
> 
> None of these is a deal breaker. I researched the i18n issue, and it
> turns out only 4 languages currently have a fully translated General
> Options page. One of these has already been updated and the other 3 are
> being addressed. Most languages already have between 1 and 3
> untranslated strings on this page from prior changes so it could be
> argued that one more is not important.
> 
> The other two considerations are relatively minor, but I still lean
> towards requiring overt action by the site admin to enable the feature.
> 
> I wanted this brought to mailman-developers in the hope that whatever
> discussion ensued would lead to some consensus.
> 
> I confess, I'm not at all up to speed on DMARC. Franck has assured me
> that this feature can be useful even in the absence of the DNS and MTA
> changes necessary to DKIM sign outgoing list mail, but it seems to me
> that enabling author_is_list will almost guarantee that any incoming
> DKIM signatures will be broken (the From: is almost certainly included
> in the signed headers) which will cause problems if the outgoing mail is
> not signed with a valid DKIM signature.
> 

DKIM does not require that the d= to be linked to the domains in the From: that's what DMARC do.

Mailman breaks DKIM as soon as you add a footer or tag in the subject line, which a lot of lists do (including this one). The rule with DKIM is to consider any broken DKIM signature as if there was no signature at all. So for instance this list mailman-developers will break all DKIM signatures when resending emails. Mailman could in fact strip DKIM at reception and this would be same effect.

So let me explain what author_is_list achieves.

if I post to this list using any of these domains which have a p=reject DMARC policy (linkedin.com, paypal.com, twitter.com,...). The original DKIM signature will be broken, and SPF while being valid, will not be aligned, not the same organizational domain as in the From:

Return-Path: mailman-developers-bounces+franck=peachymango.org at python.org

So DMARC will fail, creating the email to be bounced when being resent to members at gmail,yahoo,hotmail,aol,comacast,.. Note: This add to the unsubscription count for these members.

With Author is list, the From: becomes (I simplified):
From: mailman-developers at python.org

python.org does not have a DMARC policy, therefore the email will not be bounced due to DMARC for members at gmail,yahoo,hotmail,...

For a receiver, the IP is known, the headers contains the List-Id, so what is in the From has not much impact (besides the DMARC check), because it is mainly about the reputation of the sending IP that makes the email delivered to your mailbox.

> Also, it seems that an installation would want to validate in some way
> incoming mail before taking responsibility, even in a minor way, for
> resending it.

This would be appreciated, but I'm not sure why adding this extra burden on author_is_list is needed. All installations of mailman should validate somehow messages before resending them, with or without author_is_list.

> 
> All of this leads me to think that making this available to list owners
> should be an installation decision rather than being done by default.
> 
I'm not bent on if this option stays in the config file, I find already awesome that the option is present and we (the people using DMARC) have the opportunity to build adoption. Just trying to reduce adoption friction ;)


From stephen at xemacs.org  Fri Sep 13 21:49:18 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 14 Sep 2013 04:49:18 +0900
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <52335A10.60603@msapiro.net>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <20130913110605.7ea598a2@anarchist> <52335A10.60603@msapiro.net>
Message-ID: <87ioy47b9d.fsf@uwakimon.sk.tsukuba.ac.jp>

Mark Sapiro writes:

 > Franck has assured me that this feature can be useful even in the
 > absence of the DNS and MTA changes necessary to DKIM sign outgoing
 > list mail,

That's nonsense.  There's no reason to do this in the absence of
correct DKIM signatures by Mailman or its MTA, and every reason
(starting with RFCs 724, 733, 822, 2822, and 5322) not to do so.  Of
course if the point is to violate the RFCs, then this will still
violate the RFCs without the MTA and DNS changes.  But surely that's
not what Franck means by "useful".

The whole point of this option is to allow lists to do what we've come
to expect them to do (discard or quarantine attachments, add header
and footer text, munge Subject) while still presenting a valid DKIM
signature to the receiver.

 > Also, it seems that an installation would want to validate in some way
 > incoming mail before taking responsibility, even in a minor way, for
 > resending it.

Too late.  The reason we should consider adding this feature at all is
that the big email providers are heading in the direction of imposing
that responsibility whether list owners want it or not.

The only real alternative to DKIM signingby Mailman or its MTA is to
pass the original message through, optionally with additional headers
(eg, RFC 2369), but otherwise verbatim, ensuring valid DKIM existing
signatures won't be corrupted.

There is an alternative to From-munging, though, which is to
encapsulate the whole message (either as message/rfc822 MIME type, or
as a one-message digest).  Then the Mailman host can DKIM sign the
wrapper message without invalidating the signature on the main
message.  In theory this could also be done with a multipart/mixed
(*not* multipart/digest), with a structure like

    multipart/mixed
        text/plain        ; Mailman list header
        message/rfc822
        text/plain        ; Mailman list footer

I have no idea what MUAs would do with that, though. :-(

 > All of this leads me to think that making this available to list
 > owners should be an installation decision rather than being done by
 > default.

If the Mailman host is using DKIM, then list owners will definitely
want the option available.  So site owners should have to make a
conscious decision to shut it off.  On the other hand, since it does
involve a serious systematic RFC violation, I think it would be a good
idea to eliminate the DEFAULT_AUTHOR_IS_LIST option.  Ie, require list
owners to set it explicitly, or site owners to hack code.

See my reply to Franck for more detailed comments on the actual
proposed changes.

Steve

From stephen at xemacs.org  Fri Sep 13 22:07:14 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 14 Sep 2013 05:07:14 +0900
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <463863369.23580.1379099911972.JavaMail.zimbra@peachymango.org>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <20130913110605.7ea598a2@anarchist> <52335A10.60603@msapiro.net>
 <WM!7d7792df27ac7c6b5c227389aff7355284c982f9e0b3f907b23c32ab2282022a72acd54c0ac4876a657d99bf13ca8139!@asav-1.01.com>
 <463863369.23580.1379099911972.JavaMail.zimbra@peachymango.org>
Message-ID: <87hado7afh.fsf@uwakimon.sk.tsukuba.ac.jp>

Franck Martin writes:

 > we (the people using DMARC) have the opportunity to build
 > adoption. Just trying to reduce adoption friction ;)

The direction you're heading will *create* adoption friction.  The
only way you're going to be able to sell this to admins like me is to
wait until our users start getting unsubscribed because of DKIM
bounces.  Otherwise, I'm going to prefer RFC conformance and not
messing up my users' rules (eg, killfiles) and autocitation functions.



From franck at peachymango.org  Sat Sep 14 02:53:54 2013
From: franck at peachymango.org (Franck Martin)
Date: Fri, 13 Sep 2013 17:53:54 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
	2.1.16
In-Reply-To: <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
Message-ID: <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>

One may argue that since the list is modifying the message, it is now the new author of it, this proposal just make it more clearly.

In an ideal world, lists should only forward the email, preserving the DKIM signature, but few lists are doing that except the notable exception of apache.org lists.

When spam is received from a list (which is rare), the onus is on the list administrator to maintain his/her list content and membership.

This option is off by default and at no time it is proposed to change the behavior of any list being in place or upgraded or imported without the list admin consent and action.


On Sep 13, 2013, at 12:13 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:

> Franck Martin writes:
> 
>> In the upcoming mailman 2.1.16 there has been the introduction of
>> the optional feature author_is_list
>> 
>> "Replace the sender
> 
> Before you release, s/sender/author/, please.  When discussing
> Internet email, sender != author.  The name of the feature, "author is
> list", is an obvious falsehood: lists don't write posts, they relay
> them.  These policies do not conform to the email RFCs.  (Given the
> semantics of "From" set out in RFC 5322, they may even constitute
> copyright infringement in the absence of a license from posters
> permitting From-munging.  But that's not the topic here.)
> 
> AFAICS there's an easy way for Mailman to adapt to DKIM without
> violating RFC 5322: wrap every message in a MIME message/rfc822 part
> (ie, every message is a one-post "digest").  The wrapper message
> obviously can conform to DMARC ("From: LIST at DOMAIN" with appropriate
> DKIM signature), and Mailman can DTRT with the wrapped message.
> 
>> with the list address to conform with policies like ADSP and DMARC.
>> It replaces the poster's address in the From: header with the list
>> address and adds the poster to the Reply-To: header,
> 
> Another RFC violation. :-(  What if the poster already set Reply-To
> because she *doesn't* want mail at the From address?  What if the
> list's address *isn't* in Reply-to, but the author expects wide
> replies to go to the list?
> 
>> but the anonymous_list
> 
> This is probably OK.
> 
>> and Reply-To: header munging settings below take priority.
> 
> Does this make sense?  See above.
> 
>> If setting this to Yes, it is advised to set the MTA to DKIM sign
>> all emails. "
> 
> Please change this to "This setting is useful when your host signs
> outgoing mail according using DKIM."  As written, the advice is
> inaccurate anyway.  DKIM requires more than just MTA settings.  There
> must be cooperation from the nameserver.
> 
>> Usage of this feature on lists have indicated no adverse issue for
>> the members,
> 
> s/no adverse issue/only minor annoyance/ (I disagree with "minor", but
> sure, Outlook users probably won't notice).
> 
> You should remember that "Reply-To munging" works as expected for
> almost all subscribers almost all the time on most lists, and for that
> reason is widely used despite its ex post obvious deficiencies.  When
> it fails, though, it's at minimum a minor pain in the ass and at
> maximum an inadvertant privacy violation.
> 
> Please go slowly on screwing with From, which (unlike Reply-To) is a
> required field and so appears in *every* email and has well-defined
> semantics *with which this feature is in deliberate non-conformance*.
> 
>> provided proper communication is done when the feature is enabled
>> (as to allow inbox rules to be changed if needed).
> 
> Isn't this a lie?  If the From header is corrupt, then there is no
> reliable indication of who the author is.  If you're lucky you can
> fish it out of the body or .signature.  Reply-To won't do: not only
> are its semantics such that there's no guarantee that it's an alias
> for author, but it complicates the rules significantly because you
> need different rules for From-corrupting lists and other lists and
> non-list mail.
> 
>> In the 2.1.16 Release Candidate the feature author_is_list is
>> controlled by the following switches in the mm_cfg.py
>> 
>> ALLOW_AUTHOR_IS_LIST = No
>> DEFAULT_AUTHOR_IS_LIST = No
>> 
>> The first one will enable the GUI to present an option to the list
>> admin to enable the author_is_list feature, the second controls if
>> new lists or upgraded lists should have the author_is_list feature
>> set to yes
> 
> "Upgraded" lists?  If upgrading to Mailman 2.1.16 causes all my lists
> to be corrupted by this feature, I will not be pleased.  An option
> called DEFAULT should only apply to new lists.
> 
> If you insist on making this a fallback if the list doesn't have an
> explicit setting, call the option AUTHOR_IS_LIST.
> 
>> While it is better for the MTA to DKIM sign emails if
>> author_is_list is enabled, this is not a requirement and the list
>> will work fine without DKIM.
> 
> But why would anybody want to do this in the absence of DKIM?  Mailman
> already has the RFC 2369 and 2919 fields to tell MUAs that it's a list
> post, and a plethora of ways (Subject, header, footer) to tell users
> that it's a list post.  Anonymous list is already an option for
> obscuring the author if that's desirable, and for an announce list
> there's no problem, as the list (or an officer of the host) is already
> the author.  I think you should just delete that sentence.
> 
>> While DEFAULT_AUTHOR_IS_LIST is important to avoid new lists and
>> upgraded lists change behavior, setting ALLOW_AUTHOR_IS_LIST to Yes
>> does not change how any list is handled (author_is_list in the GUI
>> is No by default). it only provides an option to the list admin to
>> change the list behavior.
>> 
>> Unfortunately some list admins cannot edit mm_cfg.py (like CPANEL
>> type install), therefore it would be nice to remove
>> ALLOW_AUTHOR_IS_LIST or set it to Yes by default to let the list
>> admin decide how he/she wants the list to behave. Otherwise the
>> list admin needs to contact the mailman admin to enable this
>> config.
> 
> I'm reluctantly willing to accept a factory default that allows list
> owners to decide to systematically violate RFC 5322 (as well as all
> predecessors) only if the factory default of DEFAULT_AUTHOR_IS_LIST=No.
> 
>> Please indicates if you are ok to set ALLOW_AUTHOR_IS_LIST to Yes
>> or remove this option from mm_cfg.py
> 
> I don't see any obvious loss from removing it, but in keeping with the
> general "go slow" attitude toward implementing RFC-violating features,
> I think we should keep it for now, but set it to Yes.
> 
> In fact, we SHOULD remove DEFAULT_AUTHOR_IS_LIST.  List owners should
> make an informed decision to violate RFC 5322, not be defaulted into
> doing so.  If site owners want to enforce it, let them hack code.
> 
> 


From mark at msapiro.net  Sat Sep 14 04:48:48 2013
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 13 Sep 2013 19:48:48 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <463863369.23580.1379099911972.JavaMail.zimbra@peachymango.org>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <20130913110605.7ea598a2@anarchist> <52335A10.60603@msapiro.net>
 <WM!7d7792df27ac7c6b5c227389aff7355284c982f9e0b3f907b23c32ab2282022a72acd54c0ac4876a657d99bf13ca8139!@asav-1.01.com>
 <463863369.23580.1379099911972.JavaMail.zimbra@peachymango.org>
Message-ID: <5233CE90.5000203@msapiro.net>

On 09/13/2013 12:18 PM, Franck Martin wrote:
> 
> Mailman breaks DKIM as soon as you add a footer or tag in the subject line, which a lot of lists do (including this one).


Not necessarily. It depends on the DKIM signature and how much of the
body is signed. Granted, you are correct in most cases, but it might be
of interest to some to go to
<https://mail.python.org/pipermail/mailman-developers/2007-February/>
and review the "dkim-signature headers" threads.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From stephen at xemacs.org  Sat Sep 14 10:27:52 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 14 Sep 2013 17:27:52 +0900
Subject: [Mailman-Developers] Author_is_list option in upcoming
	mailman	2.1.16
In-Reply-To: <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
Message-ID: <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>

Franck Martin writes:

 > One may argue that since the list is modifying the message, it is
 > now the new author of it, this proposal just make it more clearly. 

Nonsense.  Here's what RFC 5322 says:

   The "From:" field specifies the author(s) of the message, that is,
   the mailbox(es) of the person(s) or system(s) responsible for the
   writing of the message.

The list obviously isn't responsible for the writing of the message
body, and you could argue that in adding header/footer and munging
attachments and Subject field it's acting as the agent of the author,
who is therefore responsible for them too.[1]

If that's not convincing, ask any of your users if they think "the
list" is an author of their posts, or anybody else's.

OTOH, if you want to make an authorship claim validly, there's an easy
way to accomplish it: encapsulate the whole thing in message/rfc822.

Steve



Footnotes: 
[1]  Note that RFC 5322's phrasing also clearly refutes the same
argument when made for "Reply-To".


From franck at peachymango.org  Sat Sep 14 20:21:32 2013
From: franck at peachymango.org (Franck Martin)
Date: Sat, 14 Sep 2013 11:21:32 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
	2.1.16
In-Reply-To: <WM!07ce1f104baadf303b8e6ebae7fbeb72a3174a87769d5cfa6945a9800eeea4c1b78ffe94c7423dfc831b3643468795ba!@asav-1.01.com>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <20130913110605.7ea598a2@anarchist> <52335A10.60603@msapiro.net>
 <WM!7d7792df27ac7c6b5c227389aff7355284c982f9e0b3f907b23c32ab2282022a72acd54c0ac4876a657d99bf13ca8139!@asav-1.01.com>
 <463863369.23580.1379099911972.JavaMail.zimbra@peachymango.org>
 <5233CE90.5000203@msapiro.net>
 <WM!07ce1f104baadf303b8e6ebae7fbeb72a3174a87769d5cfa6945a9800eeea4c1b78ffe94c7423dfc831b3643468795ba!@asav-1.01.com>
Message-ID: <6A271E0A-CC85-4EE5-9AE1-1D28673E9241@peachymango.org>


On Sep 13, 2013, at 7:48 PM, Mark Sapiro <mark at msapiro.net> wrote:

> On 09/13/2013 12:18 PM, Franck Martin wrote:
>> 
>> Mailman breaks DKIM as soon as you add a footer or tag in the subject line, which a lot of lists do (including this one).
> 
> 
> Not necessarily. It depends on the DKIM signature and how much of the
> body is signed. Granted, you are correct in most cases, but it might be
> of interest to some to go to
> <https://mail.python.org/pipermail/mailman-developers/2007-February/>
> and review the "dkim-signature headers" threads.
> 
Unfortunately z= and especially l= are not used practically by senders because they create a risk. One could add an attachment containing malware to the message for instance.

Even cisco does not use them in its signatures anymore.

Jim Fenton did a good document on DKIM threats: http://tools.ietf.org/html/rfc4686

From stephen at xemacs.org  Sun Sep 15 02:16:13 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 15 Sep 2013 09:16:13 +0900
Subject: [Mailman-Developers] Author_is_list option in upcoming
	mailman	2.1.16
In-Reply-To: <6A271E0A-CC85-4EE5-9AE1-1D28673E9241@peachymango.org>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <20130913110605.7ea598a2@anarchist> <52335A10.60603@msapiro.net>
 <WM!7d7792df27ac7c6b5c227389aff7355284c982f9e0b3f907b23c32ab2282022a72acd54c0ac4876a657d99bf13ca8139!@asav-1.01.com>
 <463863369.23580.1379099911972.JavaMail.zimbra@peachymango.org>
 <5233CE90.5000203@msapiro.net>
 <WM!07ce1f104baadf303b8e6ebae7fbeb72a3174a87769d5cfa6945a9800eeea4c1b78ffe94c7423dfc831b3643468795ba!@asav-1.01.com>
 <6A271E0A-CC85-4EE5-9AE1-1D28673E9241@peachymango.org>
Message-ID: <877gej6isy.fsf@uwakimon.sk.tsukuba.ac.jp>

Franck Martin writes:

 > Unfortunately z= and especially l= are not used practically by
 > senders because they create a risk. One could add an attachment
 > containing malware to the message for instance.

Indeed, we have to assume that the MUAs are broken in this respect.
See Daniel Gillmor's posts on the problems MUAs have with indicating
which parts of a message are signed MIME parts in the "testing MUAs"
thread.

The basic state of the art seems to be that MUAs can't handle anything
safely except a signature that applies to the whole message.


From franck at peachymango.org  Sun Sep 15 07:59:41 2013
From: franck at peachymango.org (Franck Martin)
Date: Sat, 14 Sep 2013 22:59:41 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
	2.1.16
In-Reply-To: <WM!ad9000caa757501125359b1f010db79f2923222bd788b4531235de7567f8a743881fba00051cf613e780f018e4dfa1d4!@asav-1.01.com>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <20130913110605.7ea598a2@anarchist> <52335A10.60603@msapiro.net>
 <WM!7d7792df27ac7c6b5c227389aff7355284c982f9e0b3f907b23c32ab2282022a72acd54c0ac4876a657d99bf13ca8139!@asav-1.01.com>
 <463863369.23580.1379099911972.JavaMail.zimbra@peachymango.org>
 <5233CE90.5000203@msapiro.net>
 <WM!07ce1f104baadf303b8e6ebae7fbeb72a3174a87769d5cfa6945a9800eeea4c1b78ffe94c7423dfc831b3643468795ba!@asav-1.01.com>
 <6A271E0A-CC85-4EE5-9AE1-1D28673E9241@peachymango.org>
 <877gej6isy.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!ad9000caa757501125359b1f010db79f2923222bd788b4531235de7567f8a743881fba00051cf613e780f018e4dfa1d4!@asav-1.01.com>
Message-ID: <099412D7-B997-45A4-BC11-1D43DC730185@peachymango.org>


On Sep 14, 2013, at 5:16 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:

> Franck Martin writes:
> 
>> Unfortunately z= and especially l= are not used practically by
>> senders because they create a risk. One could add an attachment
>> containing malware to the message for instance.
> 
> Indeed, we have to assume that the MUAs are broken in this respect.
> See Daniel Gillmor's posts on the problems MUAs have with indicating
> which parts of a message are signed MIME parts in the "testing MUAs"
> thread.
> 
> The basic state of the art seems to be that MUAs can't handle anything
> safely except a signature that applies to the whole message.
> 


I'm not sure if DKIM was ever meant to be exposed to the end user, but the current trend is to try to protect the end user as much as possible and this is done best by MTAs than MUAs.

From franck at peachymango.org  Sun Sep 15 08:18:56 2013
From: franck at peachymango.org (Franck Martin)
Date: Sat, 14 Sep 2013 23:18:56 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
	2.1.16
In-Reply-To: <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
Message-ID: <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>

When a list goes bad, usually the members are not blamed but the list admin, therefore making the list the system responsible of the writing of the message.

Anyhow, it does not matter, this is a religious discussion. Please feel free to code and test your solution of encapsulating the message in a mime rfc822. This seems an interesting and good alternative. I'd like to see it in practice so we can compare data.

On Sep 14, 2013, at 1:27 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:

> Franck Martin writes:
> 
>> One may argue that since the list is modifying the message, it is
>> now the new author of it, this proposal just make it more clearly. 
> 
> Nonsense.  Here's what RFC 5322 says:
> 
>   The "From:" field specifies the author(s) of the message, that is,
>   the mailbox(es) of the person(s) or system(s) responsible for the
>   writing of the message.
> 
> The list obviously isn't responsible for the writing of the message
> body, and you could argue that in adding header/footer and munging
> attachments and Subject field it's acting as the agent of the author,
> who is therefore responsible for them too.[1]
> 
> If that's not convincing, ask any of your users if they think "the
> list" is an author of their posts, or anybody else's.
> 
> OTOH, if you want to make an authorship claim validly, there's an easy
> way to accomplish it: encapsulate the whole thing in message/rfc822.
> 
> Steve
> 
> 
> 
> Footnotes: 
> [1]  Note that RFC 5322's phrasing also clearly refutes the same
> argument when made for "Reply-To".
> 


From stephen at xemacs.org  Sun Sep 15 14:14:28 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 15 Sep 2013 21:14:28 +0900
Subject: [Mailman-Developers] Author_is_list option in upcoming
	mailman	2.1.16
In-Reply-To: <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
 <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
Message-ID: <8761u2704b.fsf@uwakimon.sk.tsukuba.ac.jp>

Franck Martin writes:

 > When a list goes bad, usually the members are not blamed but the
 > list admin, therefore making the list the system responsible of the
 > writing of the message.

Please stop being evasive.  The RFC's use of "responsible" is intended
to point to the person who wanted the content of the message injected
into the email system.  You know that, I know that, and you're just
looking for an excuse to let your patch escape from its responsibility
for undermining the standards on which electronic mail is founded.

 > Anyhow, it does not matter, this is a religious discussion.

Religious maybe, but it does matter.  Open source lives and dies by
open standards.  Microsoft can (and does) get away with ignoring
standards if they think that will enable them to destroy the
competition by making non-Microsoft software inable to interoperate
with Microsoft's.  (Consider the number of complaints we get about
Outlook's brain-damaged handling of the "Sender" field.)

Let's *not* do it to ourselves *if we can avoid it*.  Maybe we can't
avoid it, but we really ought to try.

 > Please feel free to code and test your solution of encapsulating
 > the message in a mime rfc822. This seems an interesting and good
 > alternative. I'd like to see it in practice so we can compare data.

Without funding, I probably can't do it soon.  My GSoC student
Abhilash might be willing to do it after GSoC though.


From stephen at xemacs.org  Sun Sep 15 14:31:23 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 15 Sep 2013 21:31:23 +0900
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <099412D7-B997-45A4-BC11-1D43DC730185@peachymango.org>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <20130913110605.7ea598a2@anarchist> <52335A10.60603@msapiro.net>
 <WM!7d7792df27ac7c6b5c227389aff7355284c982f9e0b3f907b23c32ab2282022a72acd54c0ac4876a657d99bf13ca8139!@asav-1.01.com>
 <463863369.23580.1379099911972.JavaMail.zimbra@peachymango.org>
 <5233CE90.5000203@msapiro.net>
 <WM!07ce1f104baadf303b8e6ebae7fbeb72a3174a87769d5cfa6945a9800eeea4c1b78ffe94c7423dfc831b3643468795ba!@asav-1.01.com>
 <6A271E0A-CC85-4EE5-9AE1-1D28673E9241@peachymango.org>
 <877gej6isy.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!ad9000caa757501125359b1f010db79f2923222bd788b4531235de7567f8a743881fba00051cf613e780f018e4dfa1d4!@asav-1.01.com>
 <099412D7-B997-45A4-BC11-1D43DC730185@peachymango.org>
Message-ID: <874n9m6zc4.fsf@uwakimon.sk.tsukuba.ac.jp>

Franck Martin writes:

 > I'm not sure if DKIM was ever meant to be exposed to the end user,
 > but the current trend is to try to protect the end user as much as
 > possible and this is done best by MTAs than MUAs.

I disagree fundamentally.  It's best done by *both* MTAs and MUAs.
Not all threats attack you from the outside, nor can MTAs stop
everything that comes at them without help.  That's why mail services
provide "spam folders" to quarantine suspect mail.

I agree that altogether too many MUA authors agree with you, so we
can't expect much good to happen if we try to do things that depend on
capable MUAs.  That doesn't mean we shouldn't lobby for better MUAs.
MUAs have the advantage of interacting with the user, and they can
take advantage of the user's knowledge and intuition.


From mark at msapiro.net  Sun Sep 15 17:24:10 2013
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 15 Sep 2013 08:24:10 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
 <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
Message-ID: <5235D11A.8040806@msapiro.net>

On 09/14/2013 11:18 PM, Franck Martin wrote:
> 
>  this is a religious discussion.


Religious or not, it is controversial, and this discussion has raised
valid points.

Because the issue remains controversial, I will soon release 2.1.16
final with the feature disabled by default, and will consider the
message encapsulation approach or other possibilities based on
experience with 2.1.16 for a 2.1.17 release perhaps early next year.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From barry at list.org  Wed Sep 18 02:18:30 2013
From: barry at list.org (Barry Warsaw)
Date: Tue, 17 Sep 2013 20:18:30 -0400
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <87ioy47b9d.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <20130913110605.7ea598a2@anarchist> <52335A10.60603@msapiro.net>
 <87ioy47b9d.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20130917201830.3620cfeb@anarchist>

On Sep 14, 2013, at 04:49 AM, Stephen J. Turnbull wrote:

>I have no idea what MUAs would do with that, though. :-(

Me neither, but experience indicates it wouldn't be pretty. :/

-Barry

From barry at list.org  Wed Sep 18 02:28:18 2013
From: barry at list.org (Barry Warsaw)
Date: Tue, 17 Sep 2013 20:28:18 -0400
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <5235D11A.8040806@msapiro.net>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
 <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
 <5235D11A.8040806@msapiro.net>
Message-ID: <20130917202818.0b39c69f@anarchist>

On Sep 15, 2013, at 08:24 AM, Mark Sapiro wrote:

>Because the issue remains controversial, I will soon release 2.1.16
>final with the feature disabled by default, and will consider the
>message encapsulation approach or other possibilities based on
>experience with 2.1.16 for a 2.1.17 release perhaps early next year.

This makes sense to me, although I would label the feature "provisional" or
"experimental".  There just isn't any good experience here we can draw on, so
it seems reasonable to provide the knobs that will allow motivated folks to
gather such experience, but generally keep it out of the way for the majority
of users.

I suspect we still have a long way to go before we understand how DKIM and
mailing lists will work best together in practice, from the end-user's point
of view.

-Barry

From mark at msapiro.net  Wed Sep 18 03:21:19 2013
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 17 Sep 2013 18:21:19 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <20130917202818.0b39c69f@anarchist>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
 <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
 <5235D11A.8040806@msapiro.net> <20130917202818.0b39c69f@anarchist>
Message-ID: <5239000F.1070501@msapiro.net>

On 09/17/2013 05:28 PM, Barry Warsaw wrote:
> On Sep 15, 2013, at 08:24 AM, Mark Sapiro wrote:
> 
>> Because the issue remains controversial, I will soon release 2.1.16
>> final with the feature disabled by default, and will consider the
>> message encapsulation approach or other possibilities based on
>> experience with 2.1.16 for a 2.1.17 release perhaps early next year.
> 
> This makes sense to me, although I would label the feature "provisional" or
> "experimental".  There just isn't any good experience here we can draw on, so
> it seems reasonable to provide the knobs that will allow motivated folks to
> gather such experience, but generally keep it out of the way for the majority
> of users.


I intend to so indicate in the NEWS about the feature.

I have given some thought to the encapsulation approach and have some
questions about it.

My thoughts on how to do it include the following if the feature is enabled.

CookHeaders saves the original Subject: before prefixing in the metadata.

A new handler before ToOutgoing but after ToDigest, ToArchive and
ToUsenet creates a new message From: the list with Content-Type:
message/rfc822, a unique Message-ID: and Subject:, References: and
In-Reply-To: copied from the current message. It then replaces the
Subject: in the current message with that saved in the metadata and sets
the payload of the new message to the current message.

This will result in a single-part message with a payload of the content
filtered original message. If content filtering hasn't removed anything,
the original message's DKIM signatures if any should still be valid.

The message then goes to ToOutgoing and eventually OutgoingRunner and
SMTPDirect which will call Decorate and if any msg_header or msg_footer
is added, these will be added as is currently done which will result in
the message becoming multipart/mixed with the addition of one or two
text/plain, Content-Disposition: inline parts containing the header
and/or footer.

The idea is the message/rfc822 part preserves as much of the original as
possible so its DKIM sigs if any may still validate and to put enough
into the headers of the new message so MUAs can still thread it properly
and render it nicely. Also, the message is unchanged over current
behavior for digests, archives and usenet.

The sticky questions are what to do with the original From: and
Reply-To: in the face of possible Reply-To: munging options and so on.
Presumably, we want to still be able to reply to the original author,
and preserve the behavior of a simple MUA 'reply' going to the original
author and not the list in the absence of Reply-To: munging, and that
could be accomplished by putting the original author's Reply-To: (or the
original From: if no original Reply-To:) in the new message's Reply-To:,
but it's not yet clear to me how to handle the munging options (maybe
just ignore them ;).

I could actually implement this approach for the 2.1.16 release, but
that would negate the i18n work that's already been done as the
descriptive string on General Options would change, so I'm more inclined
to label this feature as experimental and subject to change
significantly in a future release.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From franck at peachymango.org  Wed Sep 18 07:04:29 2013
From: franck at peachymango.org (Franck Martin)
Date: Tue, 17 Sep 2013 22:04:29 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
	2.1.16
In-Reply-To: <WM!b958f3c6aaa178131a195a9aa700701ede8194319412e08f7b6ea1220a92f3c534f9094eca1d9de977d249b110990650!@asav-1.01.com>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
 <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
 <5235D11A.8040806@msapiro.net> <20130917202818.0b39c69f@anarchist>
 <5239000F.1070501@msapiro.net>
 <WM!b958f3c6aaa178131a195a9aa700701ede8194319412e08f7b6ea1220a92f3c534f9094eca1d9de977d249b110990650!@asav-1.01.com>
Message-ID: <1F0D27F3-E960-4EFE-8EA4-8B18D59C73F2@peachymango.org>


On Sep 17, 2013, at 6:21 PM, Mark Sapiro <mark at msapiro.net> wrote:

> On 09/17/2013 05:28 PM, Barry Warsaw wrote:
>> On Sep 15, 2013, at 08:24 AM, Mark Sapiro wrote:
>> 
>>> Because the issue remains controversial, I will soon release 2.1.16
>>> final with the feature disabled by default, and will consider the
>>> message encapsulation approach or other possibilities based on
>>> experience with 2.1.16 for a 2.1.17 release perhaps early next year.
>> 
>> This makes sense to me, although I would label the feature "provisional" or
>> "experimental".  There just isn't any good experience here we can draw on, so
>> it seems reasonable to provide the knobs that will allow motivated folks to
>> gather such experience, but generally keep it out of the way for the majority
>> of users.
> 
> 
> I intend to so indicate in the NEWS about the feature.
> 
> I have given some thought to the encapsulation approach and have some
> questions about it.
> 
> My thoughts on how to do it include the following if the feature is enabled.
> 
> CookHeaders saves the original Subject: before prefixing in the metadata.
> 
> A new handler before ToOutgoing but after ToDigest, ToArchive and
> ToUsenet creates a new message From: the list with Content-Type:
> message/rfc822, a unique Message-ID: and Subject:, References: and
> In-Reply-To: copied from the current message. It then replaces the
> Subject: in the current message with that saved in the metadata and sets
> the payload of the new message to the current message.
> 
> This will result in a single-part message with a payload of the content
> filtered original message. If content filtering hasn't removed anything,
> the original message's DKIM signatures if any should still be valid.
> 
> The message then goes to ToOutgoing and eventually OutgoingRunner and
> SMTPDirect which will call Decorate and if any msg_header or msg_footer
> is added, these will be added as is currently done which will result in
> the message becoming multipart/mixed with the addition of one or two
> text/plain, Content-Disposition: inline parts containing the header
> and/or footer.
> 
> The idea is the message/rfc822 part preserves as much of the original as
> possible so its DKIM sigs if any may still validate and to put enough
> into the headers of the new message so MUAs can still thread it properly
> and render it nicely. Also, the message is unchanged over current
> behavior for digests, archives and usenet.
> 
> The sticky questions are what to do with the original From: and
> Reply-To: in the face of possible Reply-To: munging options and so on.
> Presumably, we want to still be able to reply to the original author,
> and preserve the behavior of a simple MUA 'reply' going to the original
> author and not the list in the absence of Reply-To: munging, and that
> could be accomplished by putting the original author's Reply-To: (or the
> original From: if no original Reply-To:) in the new message's Reply-To:,
> but it's not yet clear to me how to handle the munging options (maybe
> just ignore them ;).
> 
> I could actually implement this approach for the 2.1.16 release, but
> that would negate the i18n work that's already been done as the
> descriptive string on General Options would change, so I'm more inclined
> to label this feature as experimental and subject to change
> significantly in a future release.
> 

1) If you keep the From: header as it is then, we will still have the same problems
2) the purpose of the encapsulation message/rfc822 is not meant to preserve the DKIM signature, DKIM is not meant to be verified by MUAs
3) encapsulation is not there to provide a transitive trust, there is another method explored for that which is Original-Authentication-Results (OAR). There is an expired Internet Draft on it.

I think, it is risky to code this encapsulation method directly and now, it requires a branch some testing and then merging back into the main branch.

The author_is_list has had deployment and testing for over a year in a DMARC environment. Limited testing I agree but nevertheless proved not to break things ( besides people ideal view of email  ;) ) and be useable with many MTAs and especially MUAs

Here is a recent test, deployment and analysis: http://sys4.de/en/blog/2013/08/11/mailman-dmarc-konform-betreiben/

I'd like to see somebody operating a mailing list with this encapsulation method first, before merging.



From mark at msapiro.net  Wed Sep 18 07:36:34 2013
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 17 Sep 2013 22:36:34 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <1F0D27F3-E960-4EFE-8EA4-8B18D59C73F2@peachymango.org>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
 <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
 <5235D11A.8040806@msapiro.net> <20130917202818.0b39c69f@anarchist>
 <5239000F.1070501@msapiro.net>
 <WM!b958f3c6aaa178131a195a9aa700701ede8194319412e08f7b6ea1220a92f3c534f9094eca1d9de977d249b110990650!@asav-1.01.com>
 <1F0D27F3-E960-4EFE-8EA4-8B18D59C73F2@peachymango.org>
Message-ID: <52393BE2.7000709@msapiro.net>

On 09/17/2013 10:04 PM, Franck Martin wrote:
> 
> 1) If you keep the From: header as it is then, we will still have the same problems


Perhaps I wasn't clear. The From: of the outer message would contain the
list address. The From: of its message/rfc822 payload would be unchanged
from that of the incoming message.

My questions only had to do with the Reply-To: of the outer message in
cases where the list was set to mung Reply-To: in some way.


> 2) the purpose of the encapsulation message/rfc822 is not meant to preserve the DKIM signature, DKIM is not meant to be verified by MUAs


No, but that is a side effect, at least where content filtering has not
altered the message.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From franck at peachymango.org  Wed Sep 18 07:41:33 2013
From: franck at peachymango.org (Franck Martin)
Date: Tue, 17 Sep 2013 22:41:33 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
	2.1.16
In-Reply-To: <WM!5a3df7b37f1b1652016ff593340c75b952b7bef641ab403387de5d87b4eab0effcb98d500fec6cfdeaebc9ea58659b2e!@asav-3.01.com>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
 <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
 <5235D11A.8040806@msapiro.net> <20130917202818.0b39c69f@anarchist>
 <5239000F.1070501@msapiro.net>
 <WM!b958f3c6aaa178131a195a9aa700701ede8194319412e08f7b6ea1220a92f3c534f9094eca1d9de977d249b110990650!@asav-1.01.com>
 <1F0D27F3-E960-4EFE-8EA4-8B18D59C73F2@peachymango.org>
 <52393BE2.7000709@msapiro.net>
 <WM!5a3df7b37f1b1652016ff593340c75b952b7bef641ab403387de5d87b4eab0effcb98d500fec6cfdeaebc9ea58659b2e!@asav-3.01.com>
Message-ID: <77C3C8B3-A399-4FD6-AB7E-B3E297DE10BC@peachymango.org>


On Sep 17, 2013, at 10:36 PM, Mark Sapiro <mark at msapiro.net> wrote:

> On 09/17/2013 10:04 PM, Franck Martin wrote:
>> 
>> 1) If you keep the From: header as it is then, we will still have the same problems
> 
> 
> Perhaps I wasn't clear. The From: of the outer message would contain the
> list address. The From: of its message/rfc822 payload would be unchanged
> from that of the incoming message.
> 
> My questions only had to do with the Reply-To: of the outer message in
> cases where the list was set to mung Reply-To: in some way.
> 
> 
Ah, makes sense, sorry

>> 2) the purpose of the encapsulation message/rfc822 is not meant to preserve the DKIM signature, DKIM is not meant to be verified by MUAs
> 
> 
> No, but that is a side effect, at least where content filtering has not
> altered the message.

Yes indeed

From stephen at xemacs.org  Wed Sep 18 10:04:07 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 18 Sep 2013 17:04:07 +0900
Subject: [Mailman-Developers] Author_is_list option in upcoming
	mailman	2.1.16
In-Reply-To: <1F0D27F3-E960-4EFE-8EA4-8B18D59C73F2@peachymango.org>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
 <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
 <5235D11A.8040806@msapiro.net> <20130917202818.0b39c69f@anarchist>
 <5239000F.1070501@msapiro.net>
 <WM!b958f3c6aaa178131a195a9aa700701ede8194319412e08f7b6ea1220a92f3c534f9094eca1d9de977d249b110990650!@asav-1.01.com>
 <1F0D27F3-E960-4EFE-8EA4-8B18D59C73F2@peachymango.org>
Message-ID: <874n9i5zew.fsf@uwakimon.sk.tsukuba.ac.jp>

Franck Martin writes:

 > I think, it is risky to code this encapsulation method directly and
 > now, it requires a branch some testing and then merging back into
 > the main branch.

First, the risk is zero, except to volunteers, as long as it's not
default.

Second, it's been tested for decades.  A MIME digest is nothing but
one or more encapsulated message/rfc822 parts.  In multipart/digest
form, it has an obvious defect of requiring an extra click for readers
using the MUAs I refuse to use (the MUAs I use can be taught to
explode digests automatically, or read them as mini-folders).  It
would be nice to find alternative ways to accomplish the same goals,
but this is already proof of concept.

Third, it has the advantage of preserving as much or as little of the
original message as the list would like without interfering with DKIM
validation of the encapsulation.

 > The author_is_list has had deployment and testing for over a year
 > in a DMARC environment. Limited testing I agree but nevertheless
 > proved

Limited testing is not "proof" that something works.  Limited testing
can only *prove* that something is *broken*.  More extensive testing
is still not *proof*, but it can give you confidence that it's not
*too* broken.

 > Here is a recent test, deployment and analysis:
 > http://sys4.de/en/blog/2013/08/11/mailman-dmarc-konform-betreiben/

I don't read German, but I don't see anything that looks like data,
nor is there room for "analysis."  Nor does the blog by Patrick
Koettner referenced therein.  (The Google translations confirm that.)
Please show us something that looks like data and analysis.
Specifically of interest:

    Number of lists, number of users on each list (min, mean, max
    would do), duration of operation in this mode, type of users (mail
    admins vs. general technical vs non-technical), the MUAs in use,
    any discussion from the users themselves.

 > I'd like to see somebody operating a mailing list with this
 > encapsulation method first, before merging.

Any list with MIME digests enabled and in use is a test of the basic
usability.  Do so on a site with DKIM-signing and you're done.  All my
proposal does is tune the encapsulation a bit.  It might or might not
work.



From mark at msapiro.net  Thu Sep 19 04:49:19 2013
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 18 Sep 2013 19:49:19 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <5239000F.1070501@msapiro.net>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
 <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
 <5235D11A.8040806@msapiro.net> <20130917202818.0b39c69f@anarchist>
 <5239000F.1070501@msapiro.net>
Message-ID: <523A662F.8020708@msapiro.net>

On 09/17/2013 06:21 PM, Mark Sapiro wrote:
> 
> I could actually implement this approach for the 2.1.16 release, but
> that would negate the i18n work that's already been done as the
> descriptive string on General Options would change, so I'm more inclined
> to label this feature as experimental and subject to change
> significantly in a future release.
>

I have had another thought. I will look at provisionally making
ALLOW_AUTHOR_IS_LIST a 3 way switch for 2.1.16 with 0 or False (No)
meaning current (2.1.15) behavior, 1 or True (Yes) meaning the 2.1.16rc1
behavior and 2 meaning the encapsulated message behavior.

If implementation doesn't turn into a can of worms or delay the release
too long, I'll do that and encourage interested people to try it and
report so we can get some practical experience with which to compare the
different approaches.

It will still be labeled experimental and subject to future change.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From stephen at xemacs.org  Thu Sep 19 05:15:53 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 19 Sep 2013 12:15:53 +0900
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <523A662F.8020708@msapiro.net>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
 <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
 <5235D11A.8040806@msapiro.net> <20130917202818.0b39c69f@anarchist>
 <5239000F.1070501@msapiro.net> <523A662F.8020708@msapiro.net>
Message-ID: <87eh8l4i3a.fsf@uwakimon.sk.tsukuba.ac.jp>

Mark Sapiro writes:

 > I have had another thought. I will look at provisionally making
 > ALLOW_AUTHOR_IS_LIST a 3 way switch for 2.1.16 with 0 or False (No)
 > meaning current (2.1.15) behavior, 1 or True (Yes) meaning the 2.1.16rc1
 > behavior and 2 meaning the encapsulated message behavior.

If you do, please change the name to "DKIM_CONFORMANCE_METHOD" or
similar.  But I don't think this is a very good API, because
"AUTHOR_IS_LIST"[1] and "DKIM_ARMOR" (= encapsulate messages) are
independent.  Of course if encapsulation is used, there's very little
point in rewriting "From".  But it doesn't interfere with doing so.


Footnotes: 
[1]  Barf - can we name this something that isn't a lie, like
"LIST_REWRITES_FROM", or better "LIST_HIJACKS_FROM" to distinguish it
from the behavior of anonymous lists?


From superuser at gmail.com  Fri Sep 20 01:06:00 2013
From: superuser at gmail.com (Murray S. Kucherawy)
Date: Thu, 19 Sep 2013 16:06:00 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
	2.1.16
In-Reply-To: <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAL0qLwY8Vf0-Wx5XUo4MNrpLzJhYR=5GehfVGPKPteymHxmqiQ@mail.gmail.com>

I really wish I could keep up with all the lists where interesting stuff is
going on.

We produced an RFC a few years ago that tries to tackle the names and
definitions of all the various roles (RFC 5598).  That document
deliberately avoided defining what a Sender is because that word has become
so overloaded as to be hyper-ambiguous.  Thus:

On Fri, Sep 13, 2013 at 12:13 PM, Stephen J. Turnbull <stephen at xemacs.org>wrote:

> Franck Martin writes:
>
>  > In the upcoming mailman 2.1.16 there has been the introduction of
>  > the optional feature author_is_list
>  >
>  > "Replace the sender
>
> Before you release, s/sender/author/, please.  When discussing
> Internet email, sender != author.  The name of the feature, "author is
> list", is an obvious falsehood: lists don't write posts, they relay
> them.  These policies do not conform to the email RFCs.  (Given the
> semantics of "From" set out in RFC 5322, they may even constitute
> copyright infringement in the absence of a license from posters
> permitting From-munging.  But that's not the topic here.)
>

I disagree.  Formally, Mailman is creating a new message using (likely) a
large portion of the original message.  Unless the output is completely
identical, Mailman is an Author.  So I think the name is right, unless you
want to tie the name of the feature to the list's configuration, and I'm
sure you don't want to do that.

This isn't absolute of course.  There are mushy topics like "Did the
Message-Id change?"  (One could argue that if the Author changes, a new
Message-Id should be generated.)  "Should a new Message-Id have been
generated?"  (Yes, if there was any meaningful content change whatsoever.)

Either way, I think the name is right as-is.

-MSK

From superuser at gmail.com  Fri Sep 20 01:13:57 2013
From: superuser at gmail.com (Murray S. Kucherawy)
Date: Thu, 19 Sep 2013 16:13:57 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
	2.1.16
In-Reply-To: <87ioy47b9d.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <20130913110605.7ea598a2@anarchist> <52335A10.60603@msapiro.net>
 <87ioy47b9d.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAL0qLwY7k-8a6Y515gqAbMf0Bf-htKmV0HCV3GiDm9ztmhFb8Q@mail.gmail.com>

On Fri, Sep 13, 2013 at 12:49 PM, Stephen J. Turnbull <stephen at xemacs.org>wrote:

> That's nonsense.  There's no reason to do this in the absence of
> correct DKIM signatures by Mailman or its MTA, and every reason
> (starting with RFCs 724, 733, 822, 2822, and 5322) not to do so.  Of
> course if the point is to violate the RFCs, then this will still
> violate the RFCs without the MTA and DNS changes.  But surely that's
> not what Franck means by "useful".
>

I'm familiar with RFC 822 and up, but can you specify what exactly is being
violated?  I'm not saying I disagree, I just want to go to the
reference/rule you have in mind.

If the concern is with replacing the From: field on a relayed message, then
one could argue Mailman is issuing a new message anyway, so it's free to
replace or rewrite anything it chooses.  Again referring to RFC 5598, it's
a Mediator, not a Relay, though it could also be configured to act as a
Relay.  But if it were doing that, DKIM signatures would almost always
survive.

If it's something else, I'd like to understand.


The only real alternative to DKIM signingby Mailman or its MTA is to
> pass the original message through, optionally with additional headers
> (eg, RFC 2369), but otherwise verbatim, ensuring valid DKIM existing
> signatures won't be corrupted.
>

Relay vs. Mediator mode, basically.


>
> There is an alternative to From-munging, though, which is to
> encapsulate the whole message (either as message/rfc822 MIME type, or
> as a one-message digest).  Then the Mailman host can DKIM sign the
> wrapper message without invalidating the signature on the main
> message.  In theory this could also be done with a multipart/mixed
> (*not* multipart/digest), with a structure like
>
>     multipart/mixed
>         text/plain        ; Mailman list header
>         message/rfc822
>         text/plain        ; Mailman list footer
>

Right, that's an option.


>
> I have no idea what MUAs would do with that, though. :-(
>

Me either.


>
>  > All of this leads me to think that making this available to list
>  > owners should be an installation decision rather than being done by
>  > default.
>
> If the Mailman host is using DKIM, then list owners will definitely
> want the option available.  So site owners should have to make a
> conscious decision to shut it off.  On the other hand, since it does
> involve a serious systematic RFC violation, I think it would be a good
> idea to eliminate the DEFAULT_AUTHOR_IS_LIST option.  Ie, require list
> owners to set it explicitly, or site owners to hack code.
>

I definitely agree that it should be off by default as it violates the
Principle of Least Astonishment.

-MSK

From aurelien at bompard.org  Fri Sep 20 12:46:54 2013
From: aurelien at bompard.org (Aurelien Bompard)
Date: Fri, 20 Sep 2013 12:46:54 +0200
Subject: [Mailman-Developers] Migration script from MM2 to MM3
Message-ID: <CAHmoxtVdK1NZyqtDppoSc2NeaGKWvzQEk+M=DJb9GJfufSQhig@mail.gmail.com>

Hi all,

I'm in the process of testing the conversion from Fedora's current mailman
2.1 instance to a test 3.0 instance. I realize the import21 command is
currently pretty naive and does not migrate most of the settings, nor the
member lists.
I've looked at bug 965532 [1] and more recently the email thread about
creating such a script during GSOC [2], and unless I'm mistaken nothing
moved in that general area during the summer. Am I right or did I miss
something?

[1] https://bugs.launchpad.net/mailman/+bug/965532
[2] http://www.mail-archive.com/mailman-developers at python.org/msg13464.html

I'd like to work on that, if that's the right time and no-one else is
working on it.
I'll probably extend the import21 command to convert the other settings. I
understand the tricky part is to convert email addresses in 2.1 to users in
3.0, so that's probably going to be a separate module with its own tests.
At least it'll serve as a point of reference and we'll be able to refine
the use cases and the algorithm later on.

What do you think? Bad timing? Any existing work I'm missing?

Aur?lien

From barry at list.org  Sat Sep 21 17:48:10 2013
From: barry at list.org (Barry Warsaw)
Date: Sat, 21 Sep 2013 11:48:10 -0400
Subject: [Mailman-Developers] Migration script from MM2 to MM3
In-Reply-To: <CAHmoxtVdK1NZyqtDppoSc2NeaGKWvzQEk+M=DJb9GJfufSQhig@mail.gmail.com>
References: <CAHmoxtVdK1NZyqtDppoSc2NeaGKWvzQEk+M=DJb9GJfufSQhig@mail.gmail.com>
Message-ID: <20130921114810.3b4648c6@anarchist>

On Sep 20, 2013, at 12:46 PM, Aurelien Bompard wrote:

>What do you think? Bad timing? Any existing work I'm missing?

I think it's a *great* idea.

No, perfect timing.

Not that I know of! :)

What we've said in the past is that we weren't going to promise conversion
scripts from 2.1 to 3.0, partly because that work was incomplete, but also
because we wanted to caution users not to just blindly upgrade as soon as 3.0
is released.  But I think having a reliable, well-tested, non-destructive
option to upgrade would be wonderful, especially because I fully hope that a
site could run 2.1 and 3.0 together for a while.

Any help you need, let me know.

(I've been ridiculously swamped at work these past few weeks.  Apologies for
being rather quiet lately.  I hope that'll improve in the next couple of
weeks.)

Cheers,
-Barry

From barry at list.org  Sun Sep 22 23:12:46 2013
From: barry at list.org (Barry Warsaw)
Date: Sun, 22 Sep 2013 17:12:46 -0400
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <874n9i5zew.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
 <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
 <5235D11A.8040806@msapiro.net> <20130917202818.0b39c69f@anarchist>
 <5239000F.1070501@msapiro.net>
 <WM!b958f3c6aaa178131a195a9aa700701ede8194319412e08f7b6ea1220a92f3c534f9094eca1d9de977d249b110990650!@asav-1.01.com>
 <1F0D27F3-E960-4EFE-8EA4-8B18D59C73F2@peachymango.org>
 <874n9i5zew.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20130922171246.418324ff@anarchist>

On Sep 18, 2013, at 05:04 PM, Stephen J. Turnbull wrote:

>I don't read German, but I don't see anything that looks like data,
>nor is there room for "analysis."  Nor does the blog by Patrick
>Koettner referenced therein.  (The Google translations confirm that.)
>Please show us something that looks like data and analysis.
>Specifically of interest:
>
>    Number of lists, number of users on each list (min, mean, max
>    would do), duration of operation in this mode, type of users (mail
>    admins vs. general technical vs non-technical), the MUAs in use,
>    any discussion from the users themselves.

Indeed.  I'm skeptical about how well encapsulation will go over with
end-users who have no understanding about these issues (nor should they).

End users just care about how the email looks in their mail readers.  I'm
concerned that this will be a nice, RFC-compliant feature that makes things
easy and workable for all the automated systems involved, but will look
horrible to end-users and just make them upset.  If that's the case then IMHO,
it a failure.

OTOH, maybe we won't know for sure until it gets *a lot* more testing.  But I
think it's a mistake to say "well, we just have to force MUA developers to
catch up".  As we've seen with something presumably as simple as
reply-to-list, it (almost) never happens.

-Barry

From mark at msapiro.net  Mon Sep 23 05:32:58 2013
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 22 Sep 2013 20:32:58 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
 2.1.16
In-Reply-To: <20130922171246.418324ff@anarchist>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
 <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
 <5235D11A.8040806@msapiro.net> <20130917202818.0b39c69f@anarchist>
 <5239000F.1070501@msapiro.net>
 <WM!b958f3c6aaa178131a195a9aa700701ede8194319412e08f7b6ea1220a92f3c534f9094eca1d9de977d249b110990650!@asav-1.01.com>
 <1F0D27F3-E960-4EFE-8EA4-8B18D59C73F2@peachymango.org>
 <874n9i5zew.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20130922171246.418324ff@anarchist>
Message-ID: <523FB66A.2020909@msapiro.net>

On 09/22/2013 02:12 PM, Barry Warsaw wrote:
> 
> OTOH, maybe we won't know for sure until it gets *a lot* more testing.  But I
> think it's a mistake to say "well, we just have to force MUA developers to
> catch up".  As we've seen with something presumably as simple as
> reply-to-list, it (almost) never happens.


I think what we already know for sure is that people will continue to
use their favorite, brain dead MUA regardless of how well or how often
we point out the better choices. I think we have some limited influence,
but we are definitely not the 600 lb. gorilla in this jungle.

Because of that and because of the need for better real world data, I am
convinced that I need to release 2.1.16 final with the options of going
with simple From: munging (as in RC2) or encapsulation and a default of
neither.

To that end, I will work on getting it out as soon as I can and will
definitely describe the feature as experimental and subject to change in
future releases, but encourage people to try it and report so their
experience can influence future direction.

OT - I just registered for PyCon. I haven't been to Montreal since 1975.
I'm committed. I'm stoked!

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From raj.abhilash1 at gmail.com  Mon Sep 23 21:26:43 2013
From: raj.abhilash1 at gmail.com (Abhilash Raj)
Date: Tue, 24 Sep 2013 00:56:43 +0530
Subject: [Mailman-Developers] GSoC Final report
In-Reply-To: <523F31A7.8040701@gmail.com>
References: <523F31A7.8040701@gmail.com>
Message-ID: <524095F3.3080609@gmail.com>


Hi,

So I have pushed my final commit for now for my GSoC project. I will try
to be brief on all the accomplishments and mention concisely
what-is-working.

* Signature rule following strict rfc 3156 can now be verified, only
pgp-mime messages are accepted and messages without a (valid) signature
are held.

* SignMessage handler creates multipart/signed message, first of whose
part is the original message formed after all the processing(in
default-posting-pipeline this handler is at last before the message is
send out or copied to nntp or any other queue). It expects a
`multipart/signed` or `multipart/mixeed` message. The signature is
calculated on whole part using the list's secret key.

* A List's key can be created from postorius and other signature
parameters can also be changed from there( only 'signature_max_age for
now).

* A user can import his public key from public keyservers( defaults to
ha.pool.sks-keyservers.net) by entering their public key id or they can
copy paste their public key data to upload their public key.( Exceptions
handling is not implemented yet in postorius, if works fine in perfect
situations).


I hope to improve all these things even more but seems like I have a
short of time now, I will update you( and the mm-dev list) with what
other plans I have to furthur improve these parts.

P.S.: There is a working version of my copy of mailman here[1] if you
want to have a look.

-- 
Abhilash Raj

[1]: http://maxking.emoir.co.in




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 901 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20130924/006c14ba/attachment.sig>

From stephen at xemacs.org  Tue Sep 24 05:52:32 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 24 Sep 2013 12:52:32 +0900
Subject: [Mailman-Developers]  GSoC Final report
In-Reply-To: <524095F3.3080609@gmail.com>
References: <523F31A7.8040701@gmail.com>
	<524095F3.3080609@gmail.com>
Message-ID: <87txha27wf.fsf@uwakimon.sk.tsukuba.ac.jp>

Abhilash Raj writes:

 > P.S.: There is a working version of my copy of mailman here[1] if you
 > want to have a look.

 > [1]: http://maxking.emoir.co.in

A request: Abhilash is busy with exams for the next several days, so
please post questions about the implementation to mailman-developers
(strongly preferred) or write to me.  I'll answer as best I can until
he can come back online.

I believe you can sign up for his list using Mozilla Persona.  If you
can't or won't (for whatever reason), I have an installation at
mailto:maxking-test at turnbull.sk.tsukuba.ac.jp and
http://turnbull.sk.tsukuba.ac.jp/postorius and I can sign you up by
hand.

Steve



From stephen at xemacs.org  Tue Sep 24 19:40:12 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 25 Sep 2013 02:40:12 +0900
Subject: [Mailman-Developers] Exim4 support for Mailman 3
Message-ID: <87ob7i15kz.fsf@uwakimon.sk.tsukuba.ac.jp>

Hi all,

I have successfully added Exim 4 support to Mailman 3.  The branch is
here:

    lp:~stephen-xemacs/mailman/exim4

There's an instance at maxking-test at turnbull.sk.tsukuba.ac.jp which
also contains Abhilash's signed-list feature.  Since it's very strict
about checking signatures, it probably won't be much fun to use. :-P

Postorius is running at

    http://turnbull.sk.tsukuba.ac.jp/postorius

It's a separate process group from my main site, so the URLs you'll
see in the address bar aren't very pretty, it needs the extra
"postorius" in front to distinguish the top page, and I haven't
bothered to fix the rest of the name space.

No HyperKitty yet, sorry.

From superuser at gmail.com  Wed Sep 25 19:16:57 2013
From: superuser at gmail.com (Murray S. Kucherawy)
Date: Wed, 25 Sep 2013 10:16:57 -0700
Subject: [Mailman-Developers] Author_is_list option in upcoming mailman
	2.1.16
In-Reply-To: <20130922171246.418324ff@anarchist>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
 <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
 <5235D11A.8040806@msapiro.net> <20130917202818.0b39c69f@anarchist>
 <5239000F.1070501@msapiro.net>
 <WM!b958f3c6aaa178131a195a9aa700701ede8194319412e08f7b6ea1220a92f3c534f9094eca1d9de977d249b110990650!@asav-1.01.com>
 <1F0D27F3-E960-4EFE-8EA4-8B18D59C73F2@peachymango.org>
 <874n9i5zew.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20130922171246.418324ff@anarchist>
Message-ID: <CAL0qLwbTg1TWh7HCoZh_eZ9KyON0H4K97p2LnsxJvSJ7SWLbHg@mail.gmail.com>

On Sun, Sep 22, 2013 at 2:12 PM, Barry Warsaw <barry at list.org> wrote:

> End users just care about how the email looks in their mail readers.  I'm
> concerned that this will be a nice, RFC-compliant feature that makes things
> easy and workable for all the automated systems involved, but will look
> horrible to end-users and just make them upset.  If that's the case then
> IMHO,
> it a failure.
>
> OTOH, maybe we won't know for sure until it gets *a lot* more testing.
>  But I
> think it's a mistake to say "well, we just have to force MUA developers to
> catch up".  As we've seen with something presumably as simple as
> reply-to-list, it (almost) never happens.
>
>
We as developers and standards people often avoid engaging UI people (MUAs,
in our case) and issues specifically because it's a space that doesn't
follow rules, which is what we're used to.  That partition allows us to be
able to declare victory on our side of the line most of the time, but it
leaves us with the frustrations you've described here.

I wonder how long we can hold out before we start trying to drag them into
our conversations, which might be the only way to solve these pain points
long term.  It seems to me that Gmail, Yahoo Mail, Thunderbird, etc., must
have either a team or an individual that spends time thinking about and
testing user-visible solutions to these problems, so perhaps it's time to
start asking them for help.

-MSK

From stephen at xemacs.org  Thu Sep 26 01:20:14 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 26 Sep 2013 08:20:14 +0900
Subject: [Mailman-Developers] Author_is_list option in upcoming
	mailman	2.1.16
In-Reply-To: <CAL0qLwbTg1TWh7HCoZh_eZ9KyON0H4K97p2LnsxJvSJ7SWLbHg@mail.gmail.com>
References: <1CD37233-E121-41CB-958A-0340AEB57FCC@peachymango.org>
 <87k3ik7cxt.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!a8b381fa98fb09207589e10195a70eb76a0df19eee3429d2a179170d73489bdf8d41c8047b1f52f0f40f7aed14e67fd9!@asav-1.01.com>
 <7BE6AB29-52D7-448B-BC2A-9BE496A94E4A@peachymango.org>
 <87eh8r7qpj.fsf@uwakimon.sk.tsukuba.ac.jp>
 <WM!b4f8ace2e12cf53c4918b66d22ac6703023b30e37330594c4a0f297ae8dcd1f3b94ce961471baddf90705e3b8f91ccce!@asav-2.01.com>
 <22E89E0A-C44D-43F4-8321-206F9B02B842@peachymango.org>
 <5235D11A.8040806@msapiro.net> <20130917202818.0b39c69f@anarchist>
 <5239000F.1070501@msapiro.net>
 <WM!b958f3c6aaa178131a195a9aa700701ede8194319412e08f7b6ea1220a92f3c534f9094eca1d9de977d249b110990650!@asav-1.01.com>
 <1F0D27F3-E960-4EFE-8EA4-8B18D59C73F2@peachymango.org>
 <874n9i5zew.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20130922171246.418324ff@anarchist>
 <CAL0qLwbTg1TWh7HCoZh_eZ9KyON0H4K97p2LnsxJvSJ7SWLbHg@mail.gmail.com>
Message-ID: <87eh8c1ob5.fsf@uwakimon.sk.tsukuba.ac.jp>

Murray S. Kucherawy writes:

 > I wonder how long we can hold out before we start trying to drag
 > [the MUA developers] into our conversations, which might be the
 > only way to solve these pain points long term.  It seems to me that
 > Gmail, Yahoo Mail, Thunderbird, etc., must have either a team or an
 > individual that spends time thinking about and testing user-visible
 > solutions to these problems, so perhaps it's time to start asking
 > them for help.

To be honest, I don't think they really care.  Even larsi, who never
saw an internet-draft he wouldn't implement, is not terribly
systematic about it -- as long as things are smooth between Gnus
users, well, tough luck for the rest of the world.  When I was talking
to MUA developers about best practices for dealing with reply-to-list
the basic response was "we already have a function for that" or
"sounds great, patches welcome".  The Mozilla people were clearly much
more interested in features like calendars and vcards.

I think it's important we get started on it (and maybe I can
contribute something now that GSoC is almost over), but I don't have
much hope that the MUA developers will put a high priority on it.
It's not really in their purview (see Franck Martin's comments about
MUA irrelevance in this thread, for example, and he's not even an MUA
developer).

And of course the worst offender is Microsoft, which AFAICS is
somewhat actively undermining the RFC 822 standard for reasons I don't
understand.

Regards,

From mgill25 at outlook.com  Thu Sep 26 17:09:37 2013
From: mgill25 at outlook.com (Manish Gill)
Date: Thu, 26 Sep 2013 20:39:37 +0530
Subject: [Mailman-Developers] GSoC progress report
Message-ID: <BLU0-SMTP299F14083AFDBD3F1B7294CB8280@phx.gbl>

This is the final progress report for my project, Authenticated REST API
for Postorius/Django.

The code is available at https://bitbucket.org/mgill25/mm-rest

I'll go over the major points of the work done so far:

* The public-facing REST Interface is mostly done. Most business use
cases, things like creating/deleting
lists, subscriptions etc are working via REST.

* The REST Interface currently works with Basic Auth, but other
authentication schemes are easily pluggable
via the Django REST framework.

* Postorius has been modified in order to use the MM-REST models instead
of directly interacting with
Mailman Core.

* Hence, in the case of no connection available to Core, MM-REST, and
consequently Postorius,
continue to work on their own, saving data in their own level. Which
brings me to...

* The Caching Interface b/w Mailman Core and MM-REST, which is yet to be
completely finished.
I have a (somewhat) generic interface, but haven't managed to make it
work for all the models, primarily Memberships.

* Workaround in mm-rest or modifications in Core for allowing backing up
of certain Models,
which is currently limited by the Core REST API. For example:
Registering new Email addresses for a User.

* And there's a small todo file in the project containing some things to
be done yet.


The project is not 100% finished yet, and after the GSoC is formally
over, I shall continue working on it.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 555 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20130926/4a91da1e/attachment.sig>

From mark at msapiro.net  Sun Sep 29 05:13:36 2013
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 28 Sep 2013 20:13:36 -0700
Subject: [Mailman-Developers] [Mailman-Announce] Mailman 2.1.16 release.
In-Reply-To: <51E30DF7.8020306@msapiro.net>
References: <51DF5301.7070802@msapiro.net> <51E30DF7.8020306@msapiro.net>
Message-ID: <52479AE0.2090103@msapiro.net>

I am pleased to announce the third release candidate for Mailman 2.1.16.

Python 2.4 is the minimum supported, but Python 2.7 is recommended.

This release has a few new features and several bug fixed as detailed in
the attached README.

Differences from the second candidate are the renaming of the
author_is_list feature to from_is_list to more accurately describe what
it does. This feature has also been extended to provide the option of
leaving the original message intact and wrapping it in an outer part
with Content-Type: message/rfc822.

The from_is_list feature is considered experimental and subject to
change in future releases. It is being released now as a "candidate" to
allow some exposure before the final 2.1.16 release which should occur
in October.

Mailman is free software for managing email mailing lists and
e-newsletters. Mailman is used for all the python.org and
SourceForge.net mailing lists, as well as at hundreds of other sites.

For more information, please see:

http://www.list.org
http://www.gnu.org/software/mailman
http://mailman.sourceforge.net/

Mailman 2.1.16rc3 can be downloaded from

https://launchpad.net/mailman/2.1/
http://ftp.gnu.org/gnu/mailman/
https://sourceforge.net/projects/mailman/

I anticipate that the 2.1.16 final release will be in Mid October.
Bugs found between now and then will be fixed if possible, but I hope
that most if not all changes between now and then will be i18n updates.
Please send any updates to the templates and/or message catalogs in the
2.1.16rc3 release directly to me no later than 15 Oct 2013.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
  New Features

    - There is a new list attribute from_is_list to either rewrite the From:
      header of posts replacing the posters address with that of the list or
      wrap the message in an outer message From: the list for compatability
      with DMARC and or ADSP.  There is a new mm_cfg.py setting
      DEFAULT_FROM_IS_LIST to control the default for new lists, and the
      existing REMOVE_DKIM_HEADERS setting has been extended to allow removing
      those headers only for certain from_is_list lists.  This feature must
      be enabled by setting ALLOW_FROM_IS_LIST to Yes in mm_cfg.py.  See the 
      description of these settings in Defaults.py for more detail.  This
      feature is experimental in 2.1.16, and it is subject to change or to
      become just one of the two methods in a subsequent release. People 
      interested in this feature are encouraged to try it and report their
      experiences to the mailman-users at python.org list.

    - There is a new DISPLAY_HELD_SUMMARY_SORT_BUTTONS setting which if set
      in mm_cfg.py will display a set of radio buttons in the admindb held
      message summary to select how the held messages are sorted and grouped
      for display. The exact setting determines the default grouping and
      sorting.  See the description in Defaults.py for details.

    - Setting digest_size_threshhold to zero now means no digests will be
      sent based on size instead of a digest being sent with every post.
      (LP: #558274)

    - There is a new mm_cfg.py setting SUBSCRIBE_FORM_SECRET which will put
      a dynamically generated, hidden hash in the listinfo subscribe form and
      check it upon submission.  Setting this will prevent automated processes
      (bots) from successfully POSTing web subscribes without first retrieving
      and parsing the form from the listinfo page.  The form must also be
      submitted no later than FORM_LIFETIME nor no earlier than
      SUBSCRIBE_FORM_MIN_TIME after retrieval.  Note that enabling this will
      break any static subscribe forms on your site.  See the description in
      Defaults.py for more info.  (LP: #1082746)

    - add_members now has an option to add members with mail delivery disabled
      by admin.  (LP: #1070574)

    - IncomingRunner now logs rejected messages to the vette log.
      (LP: #1068837)

    - The name of the mailmanctl master lock file is now congigurable via the
      mm_cfg.py setting MASTER_LOCK_FILE.  (LP: #1082308)

    - list_lists now has an option to list only lists with public archives.
      (LP: #1082711)

  Contributed programs

    - A new import_majordomo_into_mailman.pl script has been contributed by
      Geoff Mayes.  (LP: #1129742)

    - A new "sitemap" bash script has been contributed by Tomasz Chmielewski
      <mangoo at wpkg.org> to generate a sitemap.xml file of an installation's
      public archives for submission to search engines.

  i18n

    - A Farsi (Persian) translation has been added thanks to Javad Hoseini and
      Mahyar Moghimi.

    - Fixed several misspelled or garbled string replacements in the Spanish
      message catalog.  (LP: #1160138)

    - pt_BR message catalog has two new and an updated message per Hugo Koji
      Kobayashi.  (LP: #1138578)

    - German message catalog has been updated per Ralf Hildebrandt.

    - Corrected typo in templates/it/private.html.

  Bug Fixes and other patches

    - Fixed cron/disabled to send a fresh cookie when notifying disabled
      members.  (LP: #1203200)

    - Added "message_id" to the interpolation dictionary for the Article.html
      template.  (LP: #725498)

    - Changed the admin GUI to report only the bad entries in a list of email
      addresses if any are bad.  (LP: #558253)

    - Added logging for template errors in HyperArch.py.  (LP: #558254)

    - Added more explanation to the bad owner address message from
      bin/newlist.  (LP: #1200763)

    - Fixed a bug causing the admin web interface to fail CSRF checking if
      the list name contains a '+' character.  (LP: #1190802)

    - Fixed bin/mailmanctl -s to not remove the master lock if it can't be
      determined to be truly stale.  (LP: #1189558)

    - It is no longer possible to add 'invalid' addresses to the ban_list
      and the *_these_nonmembers filters from the check boxes on the admindb
      interface.  (LP: #1187201)

    - Backported recognition for mail.ru DSNs and minor bug fixes from
      lp:flufl.bounce.  (LP: #1074592, LP: #1079249 and #1079254)

    - Defended against buggy web servers that don't include an empty
      QUERY_STRING in the CGI environment.  (LP: #1160647)

    - The Switchboard.finish() method now logs the text of the exception when
      it fails to unlink/preserve a .bak file.  (LP: #1165589)

    - The pending (un)subscriptions waiting approval are now sorted by email
      address in the admindb interface as intended.  (LP: #1164160)

    - The subscribe log entry for a bin/add_members subscribe now identifies
      bin/add_members as the source.  (LP: #1161642)

    - Fixed a bug where the Subject: of the user notification of a
      bin/remove_members unsubscribe was not in the user's language.
      (LP: #1161445)

    - Fixed a bug where BounceRunner could create and leave behind zero length
      bounce-events files.  (LP: #1161610)

    - Added recognition for another Yahoo bounce format.  (LP: #1157961)

    - Changed configure's method for getting Python's include directory from
      distutils.sysconfig.get_config_var('CONFINCLUDEPY') to
      distutils.sysconfig.get_python_inc().  (LP: #1098162)

    - Added an Auto-Generated: header to password reminders.  (LP: #558240)

    - Fixed a bug where non-ascii characters in the real name in a subscription
      request could throw a UnicodeEncodeError upon subscription approval and
      perhaps in other situations too.  (LP: #1047100)

    - The query fragments send_unsub_notifications_to_list_owner and
      send_unsub_ack_to_this_batch will now assume default values if not set
      in mass unsubscribe URLs.  (LP: #1032378)

    - Replaced utf-8 encoded characters in newly added German templates with
      HTML entities. (LP: #1018208)