From tanstaafl at libertytrek.org  Thu May  1 12:25:34 2014
From: tanstaafl at libertytrek.org (Tanstaafl)
Date: Thu, 01 May 2014 06:25:34 -0400
Subject: [Mailman-Developers] GSOC 2014: CI tool for the Mailman suite
 and postorius improvements
In-Reply-To: <20140430101117.11ec9b77@anarchist.wooz.org>
References: <CAPkTf0KaWNJzrpQQ=bhrU1hfBht6-1muWTk4hXzs440nQXFUuw@mail.gmail.com>
 <20140430101117.11ec9b77@anarchist.wooz.org>
Message-ID: <5362211E.2000702@libertytrek.org>

On 4/30/2014 10:11 AM, Barry Warsaw <barry at list.org> wrote:
>> Thanks for giving me opportunity to work with mailman community this
>> summer. I'm an undergraduate student from Manipal Institute Of Technology,
>> India and i'll be working on project "CI tool for the Mailman suite and
>> postorius improvements".
>> I would love to get feedback and suggestions from the community regarding
>> my project which involves:
>> 1. CI tool for Mailman Suite (mailman core, postorius, hyperkitty)
>> 2. The UI Testing Framework for Postorius
>> I am in process of discussing the best possible implementation for this
>> project with my mentors and advice from the community for the same will be
>> very helpful.

> One of the things I think is critical for the core, is testing it against
> supported the two supported db backends, SQLite and PostgreSQL.

? Not fully supporting the most popular (mysql/mariadb)?

From barry at list.org  Thu May  1 15:18:47 2014
From: barry at list.org (Barry Warsaw)
Date: Thu, 1 May 2014 09:18:47 -0400
Subject: [Mailman-Developers] GSOC 2014: CI tool for the Mailman suite
 and postorius improvements
In-Reply-To: <5362211E.2000702@libertytrek.org>
References: <CAPkTf0KaWNJzrpQQ=bhrU1hfBht6-1muWTk4hXzs440nQXFUuw@mail.gmail.com>
 <20140430101117.11ec9b77@anarchist.wooz.org>
 <5362211E.2000702@libertytrek.org>
Message-ID: <20140501091847.2b5c2a17@anarchist.wooz.org>

On May 01, 2014, at 06:25 AM, Tanstaafl wrote:

>? Not fully supporting the most popular (mysql/mariadb)?

Contributions welcome!

the-obvious-response-ly y'rs,
-Barry

From rajeevs1992 at gmail.com  Fri May  2 03:06:03 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Fri, 2 May 2014 06:36:03 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
Message-ID: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>

Hi,

I will be using the following branch for the Mailman CLI project

*https://code.launchpad.net/~rajeevs1992/mailman.client/mailmancli
<https://code.launchpad.net/~rajeevs1992/mailman.client/mailmancli>*

I have created and committed the basic structure of the project in the
branch.Also, as the first step I have implemented the "create list"
functionality for the command line tools.

A few snapshots:

1.As discussed in stage of proposal review, the command mmclient (currently
script) triggers the shell if no argument is specified and performs the
action when an argument is supplied.

2.The argparse module has been used for parsing arguments. Shell is built
using Cmd module, currently no functionality.

3.The code has been verified with pep8 *and* flake8 tools. It also passes
most of the guidelines mentioned in Barry's styleguide. Some of the
guidelines are yet to be met, like the licensing block
and stuff like __all__. Also the ^L at major sections are also not
added.(Is it still necessary?)

4.The code rests in mailman.client/src/mailmanclient/cli


I guess it would be easier to discuss the design and architecture based on
this.

*Regards,*
*Rajeev S*
*Government Engineering College,Thrissur*
*http://rajeevs.tk <http://rajeevs.tk>*

From barry at list.org  Fri May  2 03:43:22 2014
From: barry at list.org (Barry Warsaw)
Date: Thu, 1 May 2014 21:43:22 -0400
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
Message-ID: <20140501214322.7637aa17@limelight.wooz.org>

On May 02, 2014, at 06:36 AM, Rajeev S wrote:

>3.The code has been verified with pep8 *and* flake8 tools. It also passes
>most of the guidelines mentioned in Barry's styleguide. Some of the
>guidelines are yet to be met, like the licensing block
>and stuff like __all__. Also the ^L at major sections are also not
>added.(Is it still necessary?)

Files should definitely all have licensing blocks.  ^Ls are not required.  I
find them useful for Emacs navigation, but I understand they may be
distracting for other folks in other editors.  I really need to update the
style guide for Python 3 and not-quite-Python-3-yet.

-Barry

From rajeevs1992 at gmail.com  Fri May  2 17:36:29 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Fri, 2 May 2014 21:06:29 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <20140501214322.7637aa17@limelight.wooz.org>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
Message-ID: <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>

Hi,

I have added the licensing blocks to the files and pushed the code.


*Regards,Rajeev S*
*Government Engineering College,Thrissur*
*http://rajeevs.tk <http://rajeevs.tk>*


On Fri, May 2, 2014 at 7:13 AM, Barry Warsaw <barry at list.org> wrote:

> On May 02, 2014, at 06:36 AM, Rajeev S wrote:
>
> >3.The code has been verified with pep8 *and* flake8 tools. It also passes
> >most of the guidelines mentioned in Barry's styleguide. Some of the
> >guidelines are yet to be met, like the licensing block
> >and stuff like __all__. Also the ^L at major sections are also not
> >added.(Is it still necessary?)
>
> Files should definitely all have licensing blocks.  ^Ls are not required.
>  I
> find them useful for Emacs navigation, but I understand they may be
> distracting for other folks in other editors.  I really need to update the
> style guide for Python 3 and not-quite-Python-3-yet.
>
> -Barry
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers at python.org
> https://mail.python.org/mailman/listinfo/mailman-developers
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Searchable Archives:
> http://www.mail-archive.com/mailman-developers%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-developers/rajeevs1992%40gmail.com
>
> Security Policy: http://wiki.list.org/x/QIA9
>

From rajeevs1992 at gmail.com  Sat May  3 09:49:13 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Sat, 3 May 2014 13:19:13 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
Message-ID: <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>

Hi,

I have added two more methods, *create domain* and *list mailing lists*.
The listing feature is performed using the `tabulate` module, which I have
added to the install_requires.

Also, the usage of the CLI is explained in the cli/docs/using.txt.


*Regards,Rajeev S*
*Government Engineering College,Thrissur*
*http://rajeevs.tk <http://rajeevs.tk>*


On Fri, May 2, 2014 at 9:06 PM, Rajeev S <rajeevs1992 at gmail.com> wrote:

> Hi,
>
> I have added the licensing blocks to the files and pushed the code.
>
>
> *Regards,Rajeev S*
> *Government Engineering College,Thrissur*
> *http://rajeevs.tk <http://rajeevs.tk>*
>
>
> On Fri, May 2, 2014 at 7:13 AM, Barry Warsaw <barry at list.org> wrote:
>
>> On May 02, 2014, at 06:36 AM, Rajeev S wrote:
>>
>> >3.The code has been verified with pep8 *and* flake8 tools. It also passes
>> >most of the guidelines mentioned in Barry's styleguide. Some of the
>> >guidelines are yet to be met, like the licensing block
>> >and stuff like __all__. Also the ^L at major sections are also not
>> >added.(Is it still necessary?)
>>
>> Files should definitely all have licensing blocks.  ^Ls are not required.
>>  I
>> find them useful for Emacs navigation, but I understand they may be
>> distracting for other folks in other editors.  I really need to update the
>> style guide for Python 3 and not-quite-Python-3-yet.
>>
>> -Barry
>> _______________________________________________
>> Mailman-Developers mailing list
>> Mailman-Developers at python.org
>> https://mail.python.org/mailman/listinfo/mailman-developers
>> Mailman FAQ: http://wiki.list.org/x/AgA3
>> Searchable Archives:
>> http://www.mail-archive.com/mailman-developers%40python.org/
>> Unsubscribe:
>> https://mail.python.org/mailman/options/mailman-developers/rajeevs1992%40gmail.com
>>
>> Security Policy: http://wiki.list.org/x/QIA9
>>
>
>

From tom.browder at gmail.com  Sat May  3 13:41:27 2014
From: tom.browder at gmail.com (Tom Browder)
Date: Sat, 3 May 2014 06:41:27 -0500
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
Message-ID: <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>

On Sat, May 3, 2014 at 2:49 AM, Rajeev S <rajeevs1992 at gmail.com> wrote:
> Hi,
>
> I have added two more methods, *create domain* and *list mailing lists*.

I haven't watched this entire thread, but I hope someone is
considering setting appropriate defaults for at least two mailing list
types such as:

  read-only (news)
  read-post (moderated or not)

Excuse my imprecise terminology.

Best regards,

-Tom

From stephen at xemacs.org  Sat May  3 14:29:04 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 03 May 2014 21:29:04 +0900
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
Message-ID: <87fvkrkpv3.fsf@uwakimon.sk.tsukuba.ac.jp>

Rajeev S writes:
 > Hi,
 > 
 > I have added two more methods, *create domain* and *list mailing lists*.
 > The listing feature is performed using the `tabulate` module, which I have
 > added to the install_requires.
 > 
 > Also, the usage of the CLI is explained in the cli/docs/using.txt.

Great!

I have $DAYJOB stuff that will prevent me from reviewing until at
least Monday evening (Japan time), but thank you for posting and
keeping all of us updated.

From stephen at xemacs.org  Sat May  3 15:44:50 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 03 May 2014 22:44:50 +0900
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
Message-ID: <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>

Tom Browder writes:

 > I haven't watched this entire thread, but I hope someone is
 > considering setting appropriate defaults for at least two mailing list
 > types such as:
 > 
 >   read-only (news)
 >   read-post (moderated or not)

Not in that terminology, no.  What do you hope to type, and what
effect do you want it to have?


From tom.browder at gmail.com  Sat May  3 16:15:01 2014
From: tom.browder at gmail.com (Tom Browder)
Date: Sat, 3 May 2014 09:15:01 -0500
Subject: [Mailman-Developers] ANNOUNCE: The GNU Mailman 3 suite,
	beta 1 preview
In-Reply-To: <20140424101938.0efc11d4@anarchist.wooz.org>
References: <20140424101938.0efc11d4@anarchist.wooz.org>
Message-ID: <CAFMGiz_-NaCp+EOzit+w6g2xnJYkK-Jtc6DP4CPJ-+hXSrv3vw@mail.gmail.com>

On Thu, Apr 24, 2014 at 9:19 AM, Barry Warsaw <barry at list.org> wrote:
...
> On behalf of the Mailman development team, I am happy
> to announce the first beta release of the full Mailman 3 suite.
>
> This release includes:
...
> Core
>     project  - https://launchpad.net/mailman
...

I apologize for my confusion, but I'm not used to the bazaar model for
multiple developers.  I'm used to the svn model for C/C++ code where
we keep the trunk as the latest development version (but we keep the
trunk so it always builds without error).

If I want the latest development version of Mailman should I use the
announced lp address above or the one I'm using now:

Related branches:
  parent branch: http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/
revision-id: barry at list.org-20140101145942-b9vqi0xmcewpk4mb
date: 2014-01-01 09:59:42 -0500
build-date: 2014-05-03 10:07:35 -0400
revno: 7232

Thanks.

Best,

-Tom

From tom.browder at gmail.com  Sat May  3 16:59:08 2014
From: tom.browder at gmail.com (Tom Browder)
Date: Sat, 3 May 2014 09:59:08 -0500
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>

On Sat, May 3, 2014 at 8:44 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Tom Browder writes:
>
>  > I haven't watched this entire thread, but I hope someone is
>  > considering setting appropriate defaults for at least two mailing list
>  > types such as:
>  >
>  >   read-only (news)
>  >   read-post (moderated or not)
>
> Not in that terminology, no.  What do you hope to type, and what
> effect do you want it to have?

We had a little discussion last year about list styles.  See this thread:

  http://www.mail-archive.com/mailman-developers%40python.org/msg13369.html

I think this GSoC project might be the place to at least start with
the option of choosing a list "style" such as a defined  "read-only"
mailing list.

Using the latest cli commands as shown in Rajeev's
/src/mailmanclient/cli/docs/using.txt would be for the "default" list
style:

  test_one = example.create_list('test-one')

Then, for other styles, we could use something like (pardon my pseudo code):

  test_one = example.create_list('test-one', style='read-only')

Looking briefly at the cli code I see nothing about the list
attributes Barry mentioned in last year's thread referenced above, so
I'm not sure how this would all fit together, but I think it needs to
be considered.

I hope this makes some sense from a very interested user.

Best regards,

-Tom

From mark at msapiro.net  Sat May  3 17:22:05 2014
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 03 May 2014 08:22:05 -0700
Subject: [Mailman-Developers] ANNOUNCE: The GNU Mailman 3 suite,
 beta 1 preview
In-Reply-To: <CAFMGiz_-NaCp+EOzit+w6g2xnJYkK-Jtc6DP4CPJ-+hXSrv3vw@mail.gmail.com>
References: <20140424101938.0efc11d4@anarchist.wooz.org>
 <CAFMGiz_-NaCp+EOzit+w6g2xnJYkK-Jtc6DP4CPJ-+hXSrv3vw@mail.gmail.com>
Message-ID: <5365099D.1080702@msapiro.net>

On 05/03/2014 07:15 AM, Tom Browder wrote:
> On Thu, Apr 24, 2014 at 9:19 AM, Barry Warsaw <barry at list.org> wrote:
> ...
>> On behalf of the Mailman development team, I am happy
>> to announce the first beta release of the full Mailman 3 suite.
>>
>> This release includes:
> ...
>> Core
>>     project  - https://launchpad.net/mailman
> ...
> 
> I apologize for my confusion, but I'm not used to the bazaar model for
> multiple developers.  I'm used to the svn model for C/C++ code where
> we keep the trunk as the latest development version (but we keep the
> trunk so it always builds without error).
> 
> If I want the latest development version of Mailman should I use the
> announced lp address above or the one I'm using now:
> 
> Related branches:
>   parent branch: http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/
> revision-id: barry at list.org-20140101145942-b9vqi0xmcewpk4mb
> date: 2014-01-01 09:59:42 -0500
> build-date: 2014-05-03 10:07:35 -0400
> revno: 7232


Now I'm confused, but it looks like you are looking at a local (to you)
branch that you branched at rev 7232 from lp:mailman or
http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/ or some other
alias for that branch.

If you now cd to that branch on your local machine and do 'bzr pull' it
should bring you up to the head of the branch on launchpad at rev 7251.

OTOH, if that is not your local branch, but a branch you obtained
elsewhere, then you probably should not be using it as the 'official'
branch is lp:mailman.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From tom.browder at gmail.com  Sat May  3 17:36:59 2014
From: tom.browder at gmail.com (Tom Browder)
Date: Sat, 3 May 2014 10:36:59 -0500
Subject: [Mailman-Developers] ANNOUNCE: The GNU Mailman 3 suite,
	beta 1 preview
In-Reply-To: <5365099D.1080702@msapiro.net>
References: <20140424101938.0efc11d4@anarchist.wooz.org>
 <CAFMGiz_-NaCp+EOzit+w6g2xnJYkK-Jtc6DP4CPJ-+hXSrv3vw@mail.gmail.com>
 <5365099D.1080702@msapiro.net>
Message-ID: <CAFMGiz9dLK05cu-CeBP0EqEdc5fW5uFh_KymDcPQ89uSr8-o7w@mail.gmail.com>

On Sat, May 3, 2014 at 10:22 AM, Mark Sapiro <mark at msapiro.net> wrote:
> On 05/03/2014 07:15 AM, Tom Browder wrote:
>> ...
>> If I want the latest development version of Mailman should I use the
>> announced lp address above or the one I'm using now:
>>
>> Related branches:
>>   parent branch: http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/
>> revision-id: barry at list.org-20140101145942-b9vqi0xmcewpk4mb
>> date: 2014-01-01 09:59:42 -0500
>> build-date: 2014-05-03 10:07:35 -0400
>> revno: 7232
>
>
> Now I'm confused, but it looks like you are looking at a local (to you)
> branch that you branched at rev 7232 from lp:mailman or
> http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/ or some other
> alias for that branch.

That's correct--the aliases are very confusing to me, but I apparently
have the right remote repo branched.

After I did a 'bzr pull' I got the proper revision.

I had done a 'bzr update' but got no response--hence my confusion
since I branched from the right place.

Now I see I am 'unbound', so I have executed 'bzr bind <...>' and can
now do 'bzr update' successfully and am at rev 7251.

I assume a 'checkout' instead of a 'branch' would have put me in the
same state as I am now.

Thanks, Mark.

Best regards,

-Tom

From stephen at xemacs.org  Sat May  3 19:17:13 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 04 May 2014 02:17:13 +0900
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
Message-ID: <877g62lr3a.fsf@uwakimon.sk.tsukuba.ac.jp>

Tom Browder writes:

 > I think this GSoC project might be the place to at least start with
 > the option of choosing a list "style" such as a defined  "read-only"
 > mailing list.

Ah, OK, I see what you're getting at now.

I think this is out of scope for the CLI project, although it's
possible that providing some kind of scriptable interface (even a
simple way to store a sequence of commands acting on the "current"
list should suffice, where "current list" is a concept his proposal
already includes) would allow it to be implemented relatively easily.
I'll discuss that with Rajeev later.

Still, I would expect that the "right" place to expose a "list style"
capability is in Postorius.  Most list admins are not even going to
have access to the CLI, most likely, because we have *no* security
model for it yet, except shell access to the list host.  The CLI is
going to be a "power user's" and sysadmin feature more than "the" way
to manage lists for most people, AIUI.  (Again, this is negotiable
with the student.)

Also, list "styles" probably should be "stored" in the Mailman core so
they can be accessed from any client (which is what takes it right out
of the scope of this GSoC IMO, although that's negotiable, again
according to Rajeev's interest).

HTH,
Steve

From barry at list.org  Sat May  3 19:21:32 2014
From: barry at list.org (Barry Warsaw)
Date: Sat, 3 May 2014 13:21:32 -0400
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
Message-ID: <20140503132132.33b88006@anarchist.wooz.org>

On May 03, 2014, at 09:59 AM, Tom Browder wrote:

>Using the latest cli commands as shown in Rajeev's
>/src/mailmanclient/cli/docs/using.txt would be for the "default" list
>style:
>
>  test_one = example.create_list('test-one')
>
>Then, for other styles, we could use something like (pardon my pseudo code):
>
>  test_one = example.create_list('test-one', style='read-only')
>
>Looking briefly at the cli code I see nothing about the list
>attributes Barry mentioned in last year's thread referenced above, so
>I'm not sure how this would all fit together, but I think it needs to
>be considered.

Using MM3's internal API:

from mailman.app.lifecycle import create_list
mylist = create_list('test at example.com', style='legacy-announce')

Using the REST API

http://pythonhosted.org//mailman/src/mailman/rest/docs/lists.html#apply-a-style-at-list-creation-time

The CLI (i.e. `mailman create` command) does not yet accept a --style
argument, but that would be pretty trivial to add.

Note that currently only the legacy-default and legacy-announce styles are
defined by default.

Cheers,
-Barry

From tom.browder at gmail.com  Sat May  3 19:40:46 2014
From: tom.browder at gmail.com (Tom Browder)
Date: Sat, 3 May 2014 12:40:46 -0500
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <20140503132132.33b88006@anarchist.wooz.org>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
Message-ID: <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>

On Sat, May 3, 2014 at 12:21 PM, Barry Warsaw <barry at list.org> wrote:
> On May 03, 2014, at 09:59 AM, Tom Browder wrote:
...
>>Then, for other styles, we could use something like (pardon my pseudo code):
>>
>>  test_one = example.create_list('test-one', style='read-only')
...
> The CLI (i.e. `mailman create` command) does not yet accept a --style
> argument, but that would be pretty trivial to add.
>
> Note that currently only the legacy-default and legacy-announce styles are
> defined by default.

Great, the 'legacy-announce' style sounds like a 'read-only'/'news'
list!  I hope so because that's just what I'm looking for.

Best,

-Tom

From mark at msapiro.net  Sat May  3 20:30:36 2014
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 03 May 2014 11:30:36 -0700
Subject: [Mailman-Developers] Mailman 2.1.18 final release
Message-ID: <536535CC.9080908@msapiro.net>

I'm pleased to announce the final release for Mailman 2.1.18.

This release has new features to help with mitigation of the impacts of
DMARC on mailing lists as well as fixing several bugs.

Python 2.4 is the minimum supported, but Python 2.7 is recommended.

There are significant new i18n strings associated with the DMARC
mitigation features. If you are interested in helping with the
translations of these strings, see <http://wiki.list.org/x/MQ>.

There is also a new dependency associated with these features. Namely,
the new Privacy options -> Sender filters -> dmarc_moderation_action
feature requires that the dnspython <http://www.dnspython.org/> package
be available in Python.

See the attached README for more details.

Mailman is free software for managing email mailing lists and
e-newsletters. Mailman is used for all the python.org and
SourceForge.net mailing lists, as well as at hundreds of other sites.

For more information, please see:

http://www.list.org
http://www.gnu.org/software/mailman
http://mailman.sourceforge.net/

Mailman 2.1.18 can be downloaded from

https://launchpad.net/mailman/2.1/
http://ftp.gnu.org/gnu/mailman/
https://sourceforge.net/projects/mailman/

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan
-------------- next part --------------
2.1.18 (03-May-2014)

  Dependencies

    - There is a new dependency associated with the new Privacy options ->
      Sender filters -> dmarc_moderation_action feature discussed below.
      This requires that the dnspython <http://www.dnspython.org/> package
      be available in Python.  This package can be downloaded from the above
      site or from the CheeseShop <https://pypi.python.org/pypi/dnspython/>
      or installed with pip.

  New Features

    - The from_is_list feature introduced in 2.1.16 is now unconditionally
      available to list owners.  There is also, a new Privacy options ->
      Sender filters -> dmarc_moderation_action feature which applies to list
      messages where the From: address is in a domain which publishes a DMARC
      policy of reject or possibly quarantine.  This is a list setting with
      values of Accept, Wrap Message, Munge From, Reject or Discard. There is
      a new DEFAULT_DMARC_MODERATION_ACTION configuration setting to set the
      default for this, and the list admin UI is not able to set an action
      which is 'less' than the default.  The prior ALLOW_FROM_IS_LIST setting
      has been removed and is effectively always Yes. There is a new
      dmarc_quarantine_moderation_action list setting with default set by a
      new DEFAULT_DMARC_QUARANTINE_MODERATION_ACTION configuration setting
      which in turn defaults to Yes.  The list setting can be set to No to
      exclude domains with DMARC policy of quarantine from
      dmarc_moderation_action.

      dmarc_moderation_action and from_is_list interact in the following way.
      If the message is From: a domain to which dmarc_moderation_action applies
      and if dmarc_moderation_action is other than Accept,
      dmarc_moderation_action applies to that message.  Otherwise the
      from_is_list action applies.

      Also associated with dmarc_moderation_action are configuration settings
      DMARC_RESOLVER_TIMEOUT and DMARC_RESOLVER_LIFETIME. These are described
      in more detail in Defaults.py.  There are also new vette log entries
      written when dmarc_moderation_action is found to apply to a post.

  i18n

    - Added missing <mm-digest-question-start> tag to French listinfo template.
      (LP: #1275964)

  Bug Fixes and other patches

    - Removed HTML tags from the title of a couple of rmlist.py pages because
      browsers don't render tags in the title.  (LP: #265848)

    - Most Mailman generated notices to list owners and moderators are now
      sent as Precedence: list instead of bulk.  (LP: #1313146)

    - The Reply-To: munging options weren't honored if there was no
      from_is_list action.  (LP: #1313010)

    - Changed from_is_list actions to insert the list address in Cc: if the
      list is fully personalized.  Otherwise, the list address is only in
      From: and Reply-To: overrides it.  (LP: #1312970)

    - Fixed the Munge From action to only Munge the From: and/or Reply-To: in
      the outgoing message and not in archives, digests and messages sent via
      the usenet gateway.  (LP: #1311431)

    - Fixed a long standing issue in which a notice sent to a user whose
      language is other than that of the list can cause subsequent things
      which should be in the list's language to be in the user's language
      instead.  (LP: #1308655)

    - Fixed the admin Membership List so a search string if any is not lost
      when visiting subsequent fragments of a chunked list.  (LP: #1307454)

    - For from_is_list feature, use email address from original From: if
      original From: has no display name and strip domain part from resultant
      names that look like email addresses.  (LP: #1304511)

    - Added the list name to the vette log "held message approved" entry.
      (LP: 1295875)

    - Added the CGI module name to various "No such list" error log entries.
      (LP: 1295875)

    - Modified contrib/mmdsr to report module name if present in "No such list
      error log entries.

    - Fixed a NameError exception in cron/nightly_gzip when it tries to print
      the usage message.  (LP: #1291038)

    - Fixed a bug in ListAdmin._handlepost that would crash when trying to
      preserve a held message for the site admin if HOLD_MESSAGES_AS_PICKLES
      is False.  (LP: #1282365)

    - The from_is_list header munging feature introduced in Mailman 2.1.16 is
      no longer erroneously applied to Mailman generated notices.
      (LP: #1279667)

    - Changed the message from the confirm CGI to not indicate approval is
      required for an acceptance of an invitation.  (LP: #1277744)

    - Fixed POSTFIX_STYLE_VIRTUAL_DOMAINS to be case-insensitiive.
      (LP: #1267003)

    - Added recognition for another simple warning to bounce processing.
      (LP: #1263247)

    - Fixed a few failing tests in tests/test_handlers.py.  (LP: #1262950)

    - Fixed bin/arch to not create scrubbed attachments for messages skipped
      when processing the --start= option.  (LP: #1260883)

    - Fixed email address validation to do a bit better in obscure cases.
      (LP: #1258703)

    - Fixed a bug which caused some authentication cookies to expire too soon
      if AUTHENTICATION_COOKIE_LIFETIME is non-zero.  (LP: #1257112)
 
    - Fixed a possible TypeError in bin/sync_members introduced in 2.1.17.
      (LP: #1243343)

  Miscellaneous

    - Added to the contrib directory, a script from Alain Williams to count
      posts in a list's archive.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20140503/fddbe41e/attachment.sig>

From rajeevs1992 at gmail.com  Sun May  4 05:01:18 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Sun, 04 May 2014 08:31:18 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <877g62lr3a.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <877g62lr3a.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <5365AD7E.90105@gmail.com>


Stephen J. Turnbull wrote:
> Ah, OK, I see what you're getting at now.
>
> I think this is out of scope for the CLI project, although it's
> possible that providing some kind of scriptable interface (even a
> simple way to store a sequence of commands acting on the "current"
> list should suffice, where "current list" is a concept his proposal
> already includes) would allow it to be implemented relatively easily.
> I'll discuss that with Rajeev later.
>
> Still, I would expect that the "right" place to expose a "list style"
> capability is in Postorius.  Most list admins are not even going to
> have access to the CLI, most likely, because we have *no* security
> model for it yet, except shell access to the list host.  The CLI is
> going to be a "power user's" and sysadmin feature more than "the" way
> to manage lists for most people, AIUI.  (Again, this is negotiable
> with the student.)
Adding a list style attribute would be simple to implement, as Barry
had mentioned.
> Also, list "styles" probably should be "stored" in the Mailman core so
> they can be accessed from any client (which is what takes it right out
> of the scope of this GSoC IMO, although that's negotiable, again
> according to Rajeev's interest).
I have not concentrated much on the mailman core as it has
least to do with my proposal. All I have done with the mailman
core is to go through the table schema as it might be useful for
the command shell. However, I will give it a try.

> HTH,
> Steve
>


From raj.abhilash1 at gmail.com  Sun May  4 09:12:44 2014
From: raj.abhilash1 at gmail.com (Abhilash Raj)
Date: Sun, 4 May 2014 12:42:44 +0530
Subject: [Mailman-Developers] Mailman-Suite demo server
Message-ID: <CAF2AkX=-ZEA9OSaXbXOfxjKZV1bN2BFOk6bZrWoPtZ3ZkgAjbw@mail.gmail.com>

Hi all,

I have setup a mailman-suite demo server here[1]. It uses the latest
packages
at pypi for postorius, mailman and hyperkitty. I have deployed using
mailman-
bundler and used its latest code from launchpad.

All are invited to poke around ;-)

thanks,
Abhilash

[1]: mailman.asynchronous.in

From johnl at taugh.com  Sun May  4 15:26:29 2014
From: johnl at taugh.com (John Levine)
Date: 4 May 2014 13:26:29 -0000
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
Message-ID: <20140504132629.1234.qmail@joyce.lan>

I realize I'm a bit late to this party, but this is a technique that I
don't think has been addressed here.  On my lists I've fixed the DMARC
bounces by rewriting From: lines of DMARC'ed domains like this on the
way out:

  From: Marissa <mmeyer at yahoo.com>

to

  From: Marissa <mmeyer at yahoo.com.invalid>

Before you tell me I'm nuts, hear me out.  I've actually implemented
this, and it works.

Advantages:

* Really fixes DMARC problems

* Everyone can still post to lists

* Doesn't change the way users in non-DMARC'ed domains work

* Leaves the author's address in the headers, so you can reply and use
  MUA sorting rules, give or take usenet-style unmunging.  (This is a
  reason we dislike putting the list address in the From: line.)

* Code is simple

* Gives people a hint of where the actual problem lies

R's,
John

PS: I use .invalid because RFC 6761 promises it will never be
allocated to anyone.

From sylvain at opensource-expert.com  Sun May  4 11:53:55 2014
From: sylvain at opensource-expert.com (Sylvain Viart)
Date: Sun, 04 May 2014 11:53:55 +0200
Subject: [Mailman-Developers] Mailman-Suite demo server
In-Reply-To: <CAF2AkX=-ZEA9OSaXbXOfxjKZV1bN2BFOk6bZrWoPtZ3ZkgAjbw@mail.gmail.com>
References: <CAF2AkX=-ZEA9OSaXbXOfxjKZV1bN2BFOk6bZrWoPtZ3ZkgAjbw@mail.gmail.com>
Message-ID: <53660E33.5000408@opensource-expert.com>

Hi,

On 04/05/2014 09:12, Abhilash Raj wrote:
> I have setup a mailman-suite demo server here
https://mailman.asynchronous.in/archives/
> All are invited to poke around ;-)

No registration form?

Is it a default setup?

Regards,
Sylvain.

From mark at msapiro.net  Sun May  4 16:45:52 2014
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 04 May 2014 07:45:52 -0700
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <20140504132629.1234.qmail@joyce.lan>
References: <20140504132629.1234.qmail@joyce.lan>
Message-ID: <536652A0.3010104@msapiro.net>

On 05/04/2014 06:26 AM, John Levine wrote:
> I realize I'm a bit late to this party, but this is a technique that I
> don't think has been addressed here.  On my lists I've fixed the DMARC
> bounces by rewriting From: lines of DMARC'ed domains like this on the
> way out:
> 
>   From: Marissa <mmeyer at yahoo.com>
> 
> to
> 
>   From: Marissa <mmeyer at yahoo.com.invalid>


It's been mentioned on mailman-users.

See the post at
<https://mail.python.org/pipermail/mailman-users/2014-May/076761.html>
and the two following replies. It came too late for 2.1.18, but we would
be interested in your experience for a follow-up release.

Our concerns are the possibility of mail being rejected by recipient
MTAs because of the invalid From: address and user complaints about
difficulty in replying to the poster.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From raj.abhilash1 at gmail.com  Sun May  4 16:47:06 2014
From: raj.abhilash1 at gmail.com (Abhilash Raj)
Date: Sun, 4 May 2014 20:17:06 +0530
Subject: [Mailman-Developers] Mailman-Suite demo server
In-Reply-To: <53660E33.5000408@opensource-expert.com>
References: <CAF2AkX=-ZEA9OSaXbXOfxjKZV1bN2BFOk6bZrWoPtZ3ZkgAjbw@mail.gmail.com>
 <53660E33.5000408@opensource-expert.com>
Message-ID: <CAF2AkXm2rUSx_GooF5JoG8R=M0gdDGinc8BSreZe=56S67vmmw@mail.gmail.com>

Hi Sylvian,

Currently there is no links to migrate between postorius(the mailman3 web
ui) and
hyperkitty(the archiver).

Postorius: https://mailman.asynchronous.in/mailman3/
Hyperkitty: https://mailman.asynchronous.in/archives/

thanks,
Abhilash


On Sun, May 4, 2014 at 3:23 PM, Sylvain Viart <sylvain at opensource-expert.com
> wrote:

> Hi,
>
>
> On 04/05/2014 09:12, Abhilash Raj wrote:
>
>> I have setup a mailman-suite demo server here
>>
> https://mailman.asynchronous.in/archives/
>
>  All are invited to poke around ;-)
>>
>
> No registration form?
>
> Is it a default setup?
>
> Regards,
> Sylvain.
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers at python.org
> https://mail.python.org/mailman/listinfo/mailman-developers
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Searchable Archives: http://www.mail-archive.com/
> mailman-developers%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-
> developers/raj.abhilash1%40gmail.com
>
> Security Policy: http://wiki.list.org/x/QIA9
>

From johnl at taugh.com  Sun May  4 17:16:38 2014
From: johnl at taugh.com (John Levine)
Date: 4 May 2014 15:16:38 -0000
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <536652A0.3010104@msapiro.net>
Message-ID: <20140504151638.1724.qmail@joyce.lan>

>>   From: Marissa <mmeyer at yahoo.com.invalid>

>Our concerns are the possibility of mail being rejected by recipient
>MTAs because of the invalid From: address and user complaints about
>difficulty in replying to the poster.

Those are exactly the things I was worried about, too.

I've seen no rejections at all due to the address.  In an earlier
iteration I tried using a null group address like this, which got
rejected all over the place:

  From: "Marissa <mmeyer at yahoo.com>" :;

(That is legal syntax since March 2013 per RFC 6854, but few MTAs have
been updated to take note, and anything that does DMARC processing
tends to freak out if there is not exactly one From: address.)

The .invalid hack seems fine, no bounces, and no complaints about
disappearing mail.  There are mutant versions of this hack where you
append a name with a wildcard that resolves but has an MTA that
rejects all the mail, and a really evil one where you append a name
that points to a server that rewrites the address and remails it, e.g.
mmeyer at yahoo.com.remail.lists.org -> mmeyer at yahoo.com.

For replies, I expected complaints, since I'm using it on some busy
lists for my church where people complain about every little burp, but
to my surprise I've gotten none.  I think one reason is that you can
still use an unmunged Reply-To, which a lot of users do, and the other
is that it's pretty obvious what to do to get the address to work,
unlike trying to guess the author's address if the From: is the list.

Something I have considered but not implemented is to add a fake Cc:
line with the unmunged address so reply-to-all will work.  It's not
clear whether that would be more confusing than useful.

Regards,
John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

From johnl at taugh.com  Sun May  4 18:35:25 2014
From: johnl at taugh.com (John R Levine)
Date: 4 May 2014 12:35:25 -0400
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <6.2.5.6.2.20140504092739.0bd68140@resistor.net>
References: <536652A0.3010104@msapiro.net>
 <20140504151638.1724.qmail@joyce.lan>
 <6.2.5.6.2.20140504092739.0bd68140@resistor.net>
Message-ID: <alpine.BSF.2.00.1405041235060.1925@joyce.lan>

> Was there any occurrence of the ".invalid" in replies which were posted to 
> the mailing list [1]?

Not that I recall.

Regards,
John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

From sm at resistor.net  Sun May  4 18:32:34 2014
From: sm at resistor.net (SM)
Date: Sun, 04 May 2014 09:32:34 -0700
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <20140504151638.1724.qmail@joyce.lan>
References: <536652A0.3010104@msapiro.net>
 <20140504151638.1724.qmail@joyce.lan>
Message-ID: <6.2.5.6.2.20140504092739.0bd68140@resistor.net>

Hi John,
At 08:16 04-05-2014, John Levine wrote:
>The .invalid hack seems fine, no bounces, and no complaints about
>disappearing mail.  There are mutant versions of this hack where you
>append a name with a wildcard that resolves but has an MTA that
>rejects all the mail, and a really evil one where you append a name
>that points to a server that rewrites the address and remails it, e.g.
>mmeyer at yahoo.com.remail.lists.org -> mmeyer at yahoo.com.
>
>For replies, I expected complaints, since I'm using it on some busy
>lists for my church where people complain about every little burp, but
>to my surprise I've gotten none.  I think one reason is that you can
>still use an unmunged Reply-To, which a lot of users do, and the other
>is that it's pretty obvious what to do to get the address to work,
>unlike trying to guess the author's address if the From: is the list.

Was there any occurrence of the ".invalid" in replies which were 
posted to the mailing list [1]?

Regards,
-sm

1. I assume that it would be caught on message submission.  I am 
asking the question as what happens in practice might be different. 


From stephen at xemacs.org  Sun May  4 20:08:50 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 05 May 2014 03:08:50 +0900
Subject: [Mailman-Developers]  Fixing DMARC problems with .invalid munge
In-Reply-To: <20140504132629.1234.qmail@joyce.lan>
References: <20140504132629.1234.qmail@joyce.lan>
Message-ID: <874n15l8lp.fsf@uwakimon.sk.tsukuba.ac.jp>

John Levine writes:

 > Before you tell me I'm nuts, hear me out.  I've actually implemented
 > this, and it works.

You're not nuts.  However, your definition of "works" is necessarily
limited to what you personally can see, in only a couple of weeks.  It
does *not* take into account the potential damage to interoperability
over time of this hack becoming wide-spread.

 > Advantages:
 > 
 > * Really fixes DMARC problems

That's a matter of opinion.  The DMARC-using domains will disagree, I
think, as it still means that you are "impersonating" their users (see
below), and making DMARC ineffective as a means of reducing spam and
phishing.  But we'll see about that soon enough.

Disadvantages:

The big one is that we can only guess what the long-run consequences
are going to be.  In particular, I think in the short-run we can
definitely foresee (1) spreading invalid addresses, which will bounce
if automatically inserted as destination addresses, all over the net,
possibly creating more backscatter, and (2) a tendency for people to
see "@yahoo.com.invalid" as a *valid* form of Yahoo! address
("computers are so weird").  Tools, most of them simple, will be
created to hide and to strip ".invalid" automatically, meaning you can
hide any arbitrary domain from DMARC behind ".invalid".  Spammers and
phishers will pick up on this, with an eventual adverse effect on your
lists' ability to deliver mail (not to mention whatever responsibility
we have for any victims we have trained to ignore ".invalid").

Note that these effects, if operational, tend to hurt everybody on the
net.  Removing yahoo.com, and other domains with "p=reject" policies,
entirely from "From" hurts Yahoo! users and people who want to
communicate with Yahoo! users, but really, the bad effects will stop
there, I think.  Yahoo! gets what conspiracy theorists (Hi, Lindsay!) 
think they want, namely locking their users into Yahoo! groups, but I
think they're going to find that the opposite effect (of annoying
their users so they move to a more flexible domain) is more important,
and DMARC will fall of its own weight without leaving too much of a
mark on the rest of the mail system.

OTOH, the natural response of the DMARC folks is going to be to try to
control this.  I can't imagine, if they succeed in agreeing on how to
do so, that it will cause less damage to interoperability than their
current behavior.

My bottom line is that I would prefer not teaching Mailman to violate
RFC 5322 at all (yes, I know that ship has long since sailed), and if
we must we should limit it to what the "owners" of those addresses
recommend.

Remember, the DMARC people are by and large employed by companies with
*no vested interest* in the continued success of the email system,
especially for forums: in most cases they'd be happier if mailing lists
(and newsgroups) went away and their users were locked into their web
forums.  We, on the other hand, depend very much on the smooth working
of the email system -- the only effect of fighting fire with fire is
to increase the damage *we* suffer from first- to second-degree burns.

If third parties' experiments with .invalid become overwhelmingly
popular, then practically speaking Mailman will cave on an option to
munge From by appending ".invalid" the same way it had to cave on
Reply-To munging.  But let's not lead the charge of the Light Brigade.

Regards,
Steve


From johnl at taugh.com  Sun May  4 20:46:15 2014
From: johnl at taugh.com (John R Levine)
Date: 4 May 2014 14:46:15 -0400
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <874n15l8lp.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20140504132629.1234.qmail@joyce.lan>
 <874n15l8lp.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <alpine.BSF.2.00.1405041437410.1925@joyce.lan>

> > * Really fixes DMARC problems
>
> That's a matter of opinion.  The DMARC-using domains will disagree, I
> think, as it still means that you are "impersonating" their users (see
> below), and making DMARC ineffective as a means of reducing spam and
> phishing.  But we'll see about that soon enough.

No, I can see that mail using this hack is delivered into my AOL and Yahoo 
accounts.  I know a people at AOL and Yahoo, and they quietly admit that 
screwing up mailing lists was not the goal, it's a side effect of a clumsy 
attempt to clean up after security breaches.  See my blog entry at 
http://jl.ly/Email/aoldmarc.html.  The conspiracy theories about 
Yahoogroups are just that.

> Note that these effects, if operational, tend to hurt everybody on the
> net.  Removing yahoo.com, and other domains with "p=reject" policies,
> entirely from "From" hurts Yahoo! users and people who want to
> communicate with Yahoo! users, but really, the bad effects will stop
> there, I think.

The DMARC cartel includes the largest mailbox providers in the world. 
There may be pockets of the net where you can thumb your nose at them, but 
in the world I live in, people depend on the lists I run, and it is simply 
not an option to tell all the Yahoo and AOL users to go away, much though 
we might think they deserve it.

By the way, in the long run, the plan is to persuade the DMARC crowd to 
mitigate the damage themselves, if not by dropping inappropriate DMARC 
policies, by figuring out how to whitelist the 30,000 (Yahoo's count) list 
and other providers that they've screwed up.  One advantage of this hack 
is that you can just turn it off when you don't need it, much easier than 
the stuff that puts the list address in the From: line which affects 
everyone.

Regards,
John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

From markr at signal100.com  Sun May  4 21:46:40 2014
From: markr at signal100.com (Mark Rousell)
Date: Sun, 04 May 2014 20:46:40 +0100
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <874n15l8lp.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20140504132629.1234.qmail@joyce.lan>
 <874n15l8lp.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <53669920.1010508@signal100.com>

Hi,

This is my first post here so please be gentle. I know I am risking
telling certain people how to suck eggs.

I'm not currently a Mailman user but will probably begin to use it soon.
I've followed the Mailman-Developers list for some time to familiarise
myself with how Mailman works.

Apologies also for the long message.

On 04/05/2014 19:08, Stephen J. Turnbull wrote:
>  > Advantages:
>  > 
>  > * Really fixes DMARC problems
> 
> That's a matter of opinion.  The DMARC-using domains will disagree, I
> think, as it still means that you are "impersonating" their users (see
> below), and making DMARC ineffective as a means of reducing spam and
> phishing.  But we'll see about that soon enough.

Surely that is a case of "so be it". DMARC (well, to be precise, Yahoo's
particular implementation of it combined with Yahoo's size in the
market) has broken longstanding and valid usage patterns and, for simple
practical purposes, needs to be routed around in a way that still
retains interoperability with Yahoo.

I do not think that this method of working around Yahoo's DMARC
implementation will necessarily impact the usefulness of DMARC as an
impersonated-spam/spoofing reduction tool. Whilst it can be argued that
adding ".INVALID" (or something similar, see also below) is still
impersonating Yahoo users, it can also be argued (more persuasively in
my opinion) that "yahoo.com.INVALID" or "yahoo.com.REMOVEME" or similar
is quite clearly not impersonating any valid Yahoo user *when sent from
a mail list*.

> Disadvantages:
> 
> The big one is that we can only guess what the long-run consequences
> are going to be.  In particular, I think in the short-run we can
> definitely foresee (1) spreading invalid addresses, which will bounce
> if automatically inserted as destination addresses, all over the net,
> possibly creating more backscatter

But who is going to be emailing using those addresses? Surely the only
people who would be emailing from such addresses for the purposes of
spoofing are spammers and any additional backscatter from such a source
would surely only be an incremental additional to the problem of
spamming. DMARC is, in any case, intended to reduce such backscatter.

> and (2) a tendency for people to
> see "@yahoo.com.invalid" as a *valid* form of Yahoo! address
> ("computers are so weird").

To my mind this is a valid criticism of the choice of words (i.e.
".invalid" can quickly start to be taken to be valid!) but not of the
idea in general.

Perhaps a better form of words can be found. For example:

.REMOVEME
.ThisMailProvidersDMARCImplementationIsBroken
.RemoveThisToEmail
.MungedForDMARCPleaseRemoveToEmail

In other words, something that non-technical end users will not so
easily become inured to.

> Tools, most of them simple, will be
> created to hide and to strip ".invalid" automatically, meaning you can
> hide any arbitrary domain from DMARC behind ".invalid".  Spammers and
> phishers will pick up on this

This is surely going to happen anyway if spammers want to harvest mail
lists as spam target sources or to harvest identities for spoofing
(which, as I understand it, is what DMARC is particularly intended to
prevent).

I don't see why it should affect mail lists and I don't see why it
should prevent DMARC from working to prevent impersonated/spoofed spam.
Indeed, it is the *removal* of suffixes like ".INVALID" (or whatever)
that DMARC is intended to eventually make worthless (not the addition of
them).

> Yahoo! gets what conspiracy theorists (Hi, Lindsay!) 
> think they want, namely locking their users into Yahoo! groups, but I
> think they're going to find that the opposite effect (of annoying
> their users so they move to a more flexible domain) is more important,
> and DMARC will fall of its own weight without leaving too much of a
> mark on the rest of the mail system.

Sadly I suspect that the vast majority Yahoo users will be entirely
oblivious to it.

Out of the very few (relatively speaking) Yahoo users who use services
which are impacted by Yahoo's p=reject DMARC implementation many will
blame the services themselves. The remaining few who understand the
issue will move away to better providers, but I suspect this will be a
negligible proportion of Yahoo's user base.

As far as I can see, DMARC is set to work as Yahoo and its other backers
intend for it. They have the market size to push it through regardless
of standards.

> OTOH, the natural response of the DMARC folks is going to be to try to
> control this.  I can't imagine, if they succeed in agreeing on how to
> do so, that it will cause less damage to interoperability than their
> current behavior.

But do they actually need to? As I understand it, they wish to use DMARC
to prevent spam that impersonates or spoofs their users or clients.
Spammers who strip off munged email suffixes and spoof-send using the
unmunged email address will indeed be blocked by DMARC.

By munging email addresses we are surely going along with the aim of not
meaningfully impersonating users of DMARC-using mail providers. We are
making it clear who the original sender is but in a form that does not
make it look like the email was sent by a third party spoofing the
original sender in question.

> and if
> we must we should limit it to what the "owners" of those addresses
> recommend.

The recommendations that dmarc.org makes for mailing list providers at
http://dmarc.org/faq.html#s_3 could be helpful in some cases but do not
address the full issue that adding a suffix is intended to work around
(and it is an issue which *needs* be to worked around somehow). That
said, adding a suffix is simply a variation of the suggestion in section
3 of the linked FAQ.

The reference in the FAQ to Original Authentication Results at
http://tools.ietf.org/html/draft-kucherawy-original-authres-00 looks
rather interesting too.

> Remember, the DMARC people are by and large employed by companies with
> *no vested interest* in the continued success of the email system,
> especially for forums: in most cases they'd be happier if mailing lists
> (and newsgroups) went away and their users were locked into their web
> forums.  We, on the other hand, depend very much on the smooth working
> of the email system -- the only effect of fighting fire with fire is
> to increase the damage *we* suffer from first- to second-degree burns.

And yet we need somehow to route around the problem they have caused us.
Adding suffixes seems like a kludge but a not unreasonable one and it is
also one that should not negatively impact DMARC or users of DMARC, as
far as I can see.

> If third parties' experiments with .invalid become overwhelmingly
> popular, then practically speaking Mailman will cave on an option to
> munge From by appending ".invalid" the same way it had to cave on
> Reply-To munging.

Would this really be a problem? If it become a de facto standard the all
well and good (but let's get the wording right). Mail providers such as
Yahoo could even play along with it by recognising it and it would not
negatively impact the effectiveness of DMARC from their perspective or
spam filtering, from what I can see.





-- 
Mark Rousell

PGP public key: http://www.signal100.com/markr/pgp
Key ID: C9C5C162
 
 
 


From stephen at xemacs.org  Mon May  5 09:38:08 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 05 May 2014 16:38:08 +0900
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <alpine.BSF.2.00.1405041437410.1925@joyce.lan>
References: <20140504132629.1234.qmail@joyce.lan>
 <874n15l8lp.fsf@uwakimon.sk.tsukuba.ac.jp>
 <alpine.BSF.2.00.1405041437410.1925@joyce.lan>
Message-ID: <87wqe0k74v.fsf@uwakimon.sk.tsukuba.ac.jp>

John R Levine writes:

 > One advantage of this hack is that you can just turn it off when
 > you don't need it, much easier than the stuff that puts the list
 > address in the From: line which affects everyone.

You're wrong on both counts.  In Mailman 2.1.18, "From" munging is
equally easy to turn off -- two clicks on an admin page.  In Mailman
2.1, it's unlikely that anything will ever get easier than that (not a
lot of Web 2.0 fancy click-once-and-be-done-with-it stuff in 2.1).
And Mailman 2.1.18 has an option that munges "From" only in case of a
published "p=reject" DNS record.

As for the rest of your post, you didn't say anything new and you
didn't rebut my point about the limitations on what will be visible to
individual postmasters in the short run.  Until we have more evidence
(or at least much better theory than I've seen so far, including from
myself) on the long-run system-wide effects of ".invalid", I'm going
to oppose implementing it *in Mailman*.

To give you one example that demonstrates how little we know about
system-wide effects, I don't recall predictions of mass unsubscribes
of third parties when a major freemail provider changed policy to
"p=reject".  That's an "oops".  I doubt that ".invalid" will have that
obvious an effect in the short run, but I think there is a real,
worrisome possibility that it will have a more subtle, infrequent
effect that cumulates over time.

Note that I am not telling *you* what to do or not to do.  I am saying
that the *Mailman Project* should act according to "Look Before You
Leap" (and IMO preferably not leap ever, but that's just my opinion),
rather than being seduced by "Easier to Ask Forgiveness than
Permission."  It may be *easy* for Mailman to ask, but if things do go
south, forgiveness may not be forthcoming.


From stephen at xemacs.org  Mon May  5 10:07:09 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 05 May 2014 17:07:09 +0900
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <53669920.1010508@signal100.com>
References: <20140504132629.1234.qmail@joyce.lan>
 <874n15l8lp.fsf@uwakimon.sk.tsukuba.ac.jp>
 <53669920.1010508@signal100.com>
Message-ID: <87vbtkk5si.fsf@uwakimon.sk.tsukuba.ac.jp>

Mark Rousell writes:

 > I do not think that this method of working around Yahoo's DMARC
 > implementation will necessarily

Nor do I.  I point to the *possibility* and our lack of ability to
predict effects.  The RFCs have proven over time to give us a system
that works smoothly.  We have rules of thumb that help to understand
why they work as well as they do, but the most important one is that
RFCs must be shown to work in practice before they become Proposed
Standards.  Ie, don't expect something to work until you see it.

 > it can also be argued (more persuasively in my opinion) that
 > "yahoo.com.INVALID" or "yahoo.com.REMOVEME" or similar is quite
 > clearly not impersonating any valid Yahoo user *when sent from a
 > mail list*.

Ah, but what matters here is not your opinion or mine, nor even those
of the DMARC folks or the sysadmins at Yahoo.  What matters is what
"just plain folks" think.  Remember, according to AOL, 2% of such
users were suckered by the recent phishing attack at AOL, and that
presumably translates to around 50,000 users!  If 2% of those 2% adopt
the interpretation I fear they will, that's 1000 users.  I'm not
willing to allow Mailman to be (partly) responsible for that without
trying to find a much better way first.

 > But do they actually need to? As I understand it, they wish to use DMARC
 > to prevent spam that impersonates or spoofs their users or clients.
 > Spammers who strip off munged email suffixes and spoof-send using the
 > unmunged email address will indeed be blocked by DMARC.

No, that's the whole point.  They will *not* strip the suffix, and
instead prefix the phishing attack with

    We have repeatedly attempted to reach your email address, but our
    mail has been rejected due to your ISP's DMARC configuration.
    Thus we have used the .invalid convention to work around this
    problem for this important message.

You saw it here first!  And don't think that the phishers are too dumb
to figure it out for themselves.  N.B.  This trick doesn't work if the
list-post address is in From.

 > The recommendations that dmarc.org makes for mailing list providers at
 > http://dmarc.org/faq.html#s_3 could be helpful in some cases but do not
 > address the full issue that adding a suffix is intended to work around
 > (and it is an issue which *needs* be to worked around somehow).

The wrap_message feature implemented in Mailman 2.1.18 is such a
workaround.  It completely takes "ownership" (aargh!) of the message
in a way which is 100% RFC-conformant and causes no loss of
information.

The main problem is that it is known to be inconvenient for users of
at least some versions of the Apple MUA on iPhones (in general, for
anybody whose MUA can't handle MIME digests well).

 > Adding suffixes seems like a kludge but a not unreasonable one and it is
 > also one that should not negatively impact DMARC or users of DMARC, as
 > far as I can see.

That is not good enough for me.  When you can say, "here's a $10,000
prize for the first report of a serious negative effect of this
kludge, and it's in escrow with a reputable service", then I'll
consider this kind of argument.  $10,000 is a quite reasonable
criterion, given that if you're wrong, you may be causing $1 worth of
annoyance to each of 500 million people.


From aurelien at bompard.org  Mon May  5 10:00:44 2014
From: aurelien at bompard.org (Aurelien Bompard)
Date: Mon, 5 May 2014 10:00:44 +0200
Subject: [Mailman-Developers] Mailman-Suite demo server
In-Reply-To: <CAF2AkXm2rUSx_GooF5JoG8R=M0gdDGinc8BSreZe=56S67vmmw@mail.gmail.com>
References: <CAF2AkX=-ZEA9OSaXbXOfxjKZV1bN2BFOk6bZrWoPtZ3ZkgAjbw@mail.gmail.com>
 <53660E33.5000408@opensource-expert.com>
 <CAF2AkXm2rUSx_GooF5JoG8R=M0gdDGinc8BSreZe=56S67vmmw@mail.gmail.com>
Message-ID: <CAHmoxtXn_i+5qzjZn5nrdeFf+j4Vbo0rEetT1ipJ2RAQrEvovw@mail.gmail.com>

> Currently there is no links to migrate between postorius(the mailman3 web
> ui) and hyperkitty(the archiver).

Yeah, and that's something I'll fix as soon as I'm done squashing the
main bugs I have in my pile.

Aur?lien

From stephen at xemacs.org  Mon May  5 10:18:08 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 05 May 2014 17:18:08 +0900
Subject: [Mailman-Developers] Handling non-standard or obfuscated DSNs for
	DMARC rejects
Message-ID: <87tx94k5a7.fsf@uwakimon.sk.tsukuba.ac.jp>

It just occurred to me that (at the expense of a DNS query) many sites
can distinguish DMARC rejects pretty well by doing a DNS query for the
>From address (which I think we should be able to get, at least most of
the time).  This may not help much if you have most of your users
posting from p=reject sites (you can't easily distinguish an
obfuscated DMARC reject from a broken recipient address), but could
save some of your users trouble if you have enough non-reject posters,
or those users have standard DSNs that specify a DMARC reject.

Of course you'd want to cache the result, I guess -- and actually, it
would already available from when you accepted the post in the first
place (if you have a non-null dmarc_moderation_action).


From sm at resistor.net  Mon May  5 11:44:43 2014
From: sm at resistor.net (SM)
Date: Mon, 05 May 2014 02:44:43 -0700
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <87vbtkk5si.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20140504132629.1234.qmail@joyce.lan>
 <874n15l8lp.fsf@uwakimon.sk.tsukuba.ac.jp>
 <53669920.1010508@signal100.com>
 <87vbtkk5si.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <6.2.5.6.2.20140505023744.0b0f9f38@resistor.net>

Hi Stephen,
At 01:07 05-05-2014, Stephen J. Turnbull wrote:
>Nor do I.  I point to the *possibility* and our lack of ability to
>predict effects.  The RFCs have proven over time to give us a system
>that works smoothly.  We have rules of thumb that help to understand
>why they work as well as they do, but the most important one is that
>RFCs must be shown to work in practice before they become Proposed
>Standards.  Ie, don't expect something to work until you see it.

This is a nit.  There isn't any requirement that RFCs have to be 
shown to work in practice  before they become Proposed Standards.

>of the DMARC folks or the sysadmins at Yahoo.  What matters is what
>"just plain folks" think.  Remember, according to AOL, 2% of such

Yes.

>No, that's the whole point.  They will *not* strip the suffix, and
>instead prefix the phishing attack with
>
>     We have repeatedly attempted to reach your email address, but our
>     mail has been rejected due to your ISP's DMARC configuration.
>     Thus we have used the .invalid convention to work around this
>     problem for this important message.

Yes.

Regards,
-sm


From johnl at taugh.com  Mon May  5 15:50:07 2014
From: johnl at taugh.com (John R Levine)
Date: 5 May 2014 09:50:07 -0400
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <87wqe0k74v.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20140504132629.1234.qmail@joyce.lan>
 <874n15l8lp.fsf@uwakimon.sk.tsukuba.ac.jp>
 <alpine.BSF.2.00.1405041437410.1925@joyce.lan>
 <87wqe0k74v.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <alpine.BSF.2.00.1405050947300.56860@joyce.lan>

> > One advantage of this hack is that you can just turn it off when
> > you don't need it, much easier than the stuff that puts the list
> > address in the From: line which affects everyone.
>
> You're wrong on both counts.  In Mailman 2.1.18, "From" munging is
> equally easy to turn off -- two clicks on an admin page.

You're missing the point.  If you configure a list to put the list's 
address in the From: line, users have to change they way they work for all 
mail from the list, and if you change it back, the users have to change 
again.

Regards,
John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

From rajeevs1992 at gmail.com  Mon May  5 17:52:07 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Mon, 05 May 2014 21:22:07 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <5365AD7E.90105@gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <877g62lr3a.fsf@uwakimon.sk.tsukuba.ac.jp> <5365AD7E.90105@gmail.com>
Message-ID: <5367B3A7.8030604@gmail.com>

Hi,

I have elaborated the implementation and the motivations behind the 
current approach in the following blog post, in case it might help 
reviewing the code better.

http://myfossblog.blogspot.in/2014/05/lets-talk-over-cup-of-code.html



From tim.marx at xrammit.de  Mon May  5 22:22:57 2014
From: tim.marx at xrammit.de (Tim Marx)
Date: Mon, 05 May 2014 22:22:57 +0200
Subject: [Mailman-Developers] [Bug 1130957] Re: Unicode errors in
	mailman3
In-Reply-To: <20140430175340.277d4238@anarchist.wooz.org>
References: <20130220231858.9018.27307.malonedeb@gac.canonical.com>
 <20140430213828.4471.26296.malone@soybean.canonical.com>
 <20140430175340.277d4238@anarchist.wooz.org>
Message-ID: <5367F321.7030702@xrammit.de>

Am 30.04.2014 23:53, schrieb Barry Warsaw:
> On Apr 30, 2014, at 09:38 PM, Tim Marx wrote:
>
>> I'm using mailman 3.0.0b4 (with mailman-bundler) and ran into the same
>> problem as described above.  Luckily I found this bug report and could manage
>> to fix it with the patches above.
>>
>> May I ask anybody (with more bazar-knowledge than me) to merge this two
>> patches into the mailman 3 branch?  Thank you!
> I think we just need some test cases.  Tim, can you provide any details on how
> you it the bug?
I had exactly the same stacktraces and error messages as Aur?lien 
described in his bug report.
They appeared after I had sent mails from list-member email adresses to 
my mailing list server. I have used 'normal' email addresses without any 
special characters.
I'm sorry if I could not really help you with this!
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers at python.org
> https://mail.python.org/mailman/listinfo/mailman-developers
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/tim%40xrammit.de
>
> Security Policy: http://wiki.list.org/x/QIA9
>


From tim.marx at xrammit.de  Tue May  6 00:49:20 2014
From: tim.marx at xrammit.de (Tim Marx)
Date: Tue, 06 May 2014 00:49:20 +0200
Subject: [Mailman-Developers] mailman-bundler-3.0b2: WSGI server error with
	zope.interface
Message-ID: <53681570.6080706@xrammit.de>

Hello,

I wanted to test the new mailman-bundler-3.0b2 and ran into a problem I 
could not solve.
I'm not sure if it's a problem with the mailman-bundler or with my 
machine. So I'm sorry for bothering you if the problem might be only on 
my server.

I installed the latest revision (#13) of the branch 
lp:~abompard/mailman-bundler/mailman-bundler in a fresh virtualenv with 
flag --no-site-packages (because of earlier install attempts that 
failed) located under /var/tmp/mailman.
I've created a user and a group ('mailman') who own the directory 
/var/tmp/mailman and I've added the user 'www-data' to the group 
'mailman' so there might be no permission problem in my opinion.
I followed the steps in START.rst (as 'production' installation) and 
everything worked fine so far.
Then I wanted to setup the Postorius/Hyperkitty frontend access with 
Apache and mod_wsgi (3.3). I used the supplied ~/deployment/apache.conf 
as basis and added VirtualHost directives and custom logging and so on 
but I didn't change the WSGI... directives.
Now when I try to access the configured domain I see always a "Server 
Error (500)" and the following stacktrace in my error log:
-------------------------------------------------------------------------------------
ERROR 2014-05-05 23:46:03,342 base 5588 140076680513280 Internal Server 
Error: /
Traceback (most recent call last):
   File 
"/var/tmp/mailman/mailman-bundler/eggs/Django-1.6.4-py2.7.egg/django/core/handlers/base.py", 
line 90, in get_response
     response = middleware_method(request)
   File 
"/var/tmp/mailman/mailman-bundler/eggs/HyperKitty-0.1.7-py2.7.egg/hyperkitty/middleware.py", 
line 67, in process_request
     store = kittystore.get_store(settings)
   File 
"/var/tmp/mailman/mailman-bundler/eggs/KittyStore-0.1.7-py2.7.egg/kittystore/__init__.py", 
line 52, in get_store
     from kittystore.storm import get_storm_store
   File 
"/var/tmp/mailman/mailman-bundler/eggs/KittyStore-0.1.7-py2.7.egg/kittystore/storm/__init__.py", 
line 11, in <module>
     from .store import StormStore
   File 
"/var/tmp/mailman/mailman-bundler/eggs/KittyStore-0.1.7-py2.7.egg/kittystore/storm/store.py", 
line 30, in <module>
     from kittystore.scrub import Scrubber
   File 
"/var/tmp/mailman/mailman-bundler/eggs/KittyStore-0.1.7-py2.7.egg/kittystore/scrub.py", 
line 26, in <module>
     from mailman.utilities.string import oneline
   File 
"/var/tmp/mailman/mailman-bundler/eggs/mailman-3.0.0b4-py2.7.egg/mailman/utilities/string.py", 
line 39, in <module>
     from zope.component import getUtility
   File 
"/var/tmp/mailman/mailman-bundler/eggs/zope.component-4.2.1-py2.7.egg/zope/component/__init__.py", 
line 19, in <module>
     from zope.interface import named
ImportError: cannot import name named
---------------------------------------------------------------------------------------
I have already found out, that this error occurs in zope.interface < 
4.1.0, but I've definitly installed version 4.1.1.
One interesting fact is, when I start the Django development server 
(with activated virtualenv) with './bin/mailman-web-django-admin 
runserver' this error doesn't occur.

Thank you very much for any ideas and help!
Tim Marx

From johnl at taugh.com  Tue May  6 05:41:12 2014
From: johnl at taugh.com (John Levine)
Date: 6 May 2014 03:41:12 -0000
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <87vbtkk5si.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20140506034112.60036.qmail@joyce.lan>

>    We have repeatedly attempted to reach your email address, but our
>    mail has been rejected due to your ISP's DMARC configuration.
>    Thus we have used the .invalid convention to work around this
>    problem for this important message.

They already do about a million similar things.  Among the various
attacks that DMARC does not address are these:

 From: Paypal Security <boris at rbn.ru> 
 From: Paypal Security <security at paypaI.com>

(Keep in mind that many MUAs, probably a majority, don't show the
address at all, only the comment.)

I wouldn't waste time worrying about whether various hacks might make
it 0.0001% easier to phish people.

R's,
John

From stephen at xemacs.org  Tue May  6 07:30:32 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 06 May 2014 14:30:32 +0900
Subject: [Mailman-Developers] mailman-bundler-3.0b2: WSGI server error
	with	zope.interface
In-Reply-To: <53681570.6080706@xrammit.de>
References: <53681570.6080706@xrammit.de>
Message-ID: <87r447jwxz.fsf@uwakimon.sk.tsukuba.ac.jp>

Tim Marx writes:
 > "/var/tmp/mailman/mailman-bundler/eggs/zope.component-4.2.1-py2.7.egg/zope/component/__init__.py", 
 > line 19, in <module>
 >      from zope.interface import named
 > ImportError: cannot import name named
 > ---------------------------------------------------------------------------------------
 > I have already found out, that this error occurs in zope.interface < 
 > 4.1.0, but I've definitly installed version 4.1.1.
 > One interesting fact is, when I start the Django development server 
 > (with activated virtualenv) with './bin/mailman-web-django-admin 
 > runserver' this error doesn't occur.

I think you may have answered your own question.  Try starting a new
shell (to get out of the virtualenv) and typing "./bin/python" (or
wherever the virtualenv python lives) at the shell prompt, then

    >>> import zope.interface
    >>> from zope.interface import named

If the first succeeds but the second doesn't, you have another
zope.interface on your PYTHON_PATH.  You need to ensure that Apache
and mod_wsgi see zope.interface version 4.1.1, not the other one.

I cheat, I have a virtualenv that manipulates the system site-packages
because I don't care about anything but Mailman.  However, you should
be careful that you don't have packages that depend incompatibly on an
earlier version of zope.interface.

From mark at msapiro.net  Tue May  6 08:11:38 2014
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 05 May 2014 23:11:38 -0700
Subject: [Mailman-Developers] Mailman 2.1.18 final release
In-Reply-To: <536535CC.9080908@msapiro.net>
References: <536535CC.9080908@msapiro.net>
Message-ID: <53687D1A.5030302@msapiro.net>

On 05/03/2014 11:30 AM, Mark Sapiro wrote:
> I'm pleased to announce the final release for Mailman 2.1.18.


It appears that the from_is_list and dmarc_moderation_actions Wrap
Message actions may run afoul of this issue
<http://bugs.python.org/issue7970> in the Python email library in
versions older than 2.6.x where x is some number < 5. I.e. I know the
bug is fixed in Python 2.7 and 2.6.5 and not in any 2.5.x or older. I'm
not sure about 2.6.1 - 2.6.4.

I have attached a patch to Mailman/Message.py which I think will fix
this issue if you have it. You will know if you do because all outgoing
mail will be shunted with the exception "TypeError: Expected list, got
<type 'str'>" when SMTPDirect.py invokes the as_string() method on the
message object.

I think this will only occur with those older Pythons and when a Wrap
Message action is applied.

As soon as I get confirmation from the original reporter that the patch
solves the problem, I will release a fixed version.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan
-------------- next part --------------
--- /var/MM/2.1/Mailman/Message.py	2014-04-26 21:29:42.282766984 -0700
+++ /var/MM/21/Mailman/Message.py	2014-05-05 22:40:32.412348756 -0700
@@ -59,6 +59,25 @@
         return self.__class__(fp, self._mangle_from_,
                 self.__children_maxheaderlen, self.__children_maxheaderlen)
 
+    # This is the _handle_message method with the fix for bug 7970.
+    def _handle_message(self, msg):
+        s = StringIO()
+        g = self.clone(s)
+        # The payload of a message/rfc822 part should be a multipart sequence
+        # of length 1.  The zeroth element of the list should be the Message
+        # object for the subpart.  Extract that object, stringify it, and
+        # write it out.
+        # Except, it turns out, when it's a string instead, which happens when
+        # and only when HeaderParser is used on a message of mime type
+        # message/rfc822.  Such messages are generated by, for example,
+        # Groupwise when forwarding unadorned messages.  (Issue 7970.)  So
+        # in that case we just emit the string body.
+        payload = msg.get_payload()
+        if isinstance(payload, list):
+            g.flatten(msg.get_payload(0), unixfrom=False)
+            payload = s.getvalue()
+        self._fp.write(payload)
+
 
 

 class Message(email.Message.Message):
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20140505/130222ae/attachment-0001.sig>

From stephen at xemacs.org  Tue May  6 08:23:09 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 06 May 2014 15:23:09 +0900
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <alpine.BSF.2.00.1405050947300.56860@joyce.lan>
References: <20140504132629.1234.qmail@joyce.lan>
 <874n15l8lp.fsf@uwakimon.sk.tsukuba.ac.jp>
 <alpine.BSF.2.00.1405041437410.1925@joyce.lan>
 <87wqe0k74v.fsf@uwakimon.sk.tsukuba.ac.jp>
 <alpine.BSF.2.00.1405050947300.56860@joyce.lan>
Message-ID: <87ppjrjuia.fsf@uwakimon.sk.tsukuba.ac.jp>

John R Levine writes:

 > > > One advantage of this hack is that you can just turn it off when
 > > > you don't need it, much easier than the stuff that puts the list
 > > > address in the From: line which affects everyone.
 > >
 > > You're wrong on both counts.  In Mailman 2.1.18, "From" munging is
 > > equally easy to turn off -- two clicks on an admin page.
 > 
 > You're missing the point.

No, I just know that "From" munging in Mailman[1] is designed to avoid
the problem you're talking about.  See below.

 > If you configure a list to put the list's address in the From:
 > line, users have to change they way they work for all mail from the
 > list,

It won't be "all mail" in the great majority of cases, I believe, as
the preferred (perhaps eventually default) configuration will be
dmarc_moderation_action = wrap_message (or mung_from).

 > and if you change it back, the users have to change  again.

I suspect you misunderstand both the results of your experiment and
how "wrap_message" and "mung_from" actually work.  That is, the reason
you've had no complaints from your users about ".invalid" is that they
very rarely reply to author anyway.  From your description of the
lists you serve I would expect that users rarely feel a need for a
personal reply (as opposed to a public reply directed to a person).  I
could be wrong, but how do you get data to show I'm wrong?

In any case, here are the reply-to-author algorithms for each form of
>From munging:

.invalid in From, non-Reply-To munging list:

    1a.  Hit "r" (or click button labeled "Reply"), as usual.
    2a.  Mark and delete ".invalid", different from pre-DMARC.
    3a.  Compose and send message, as usual.

.invalid in From, Reply-To munging list:

    1b.  Hit "r" (or click button labeled "Reply"), bs usual.
    2b1. Mark author address except ".invalid", different.
    2b2. Copy it.
    2b3. Delete or mark list-post.
    2b4. Paste author address.
    3b.  Compose and send message, as usual.

list-post in From, author in Reply-To (non-Reply-To-munging):

    1c.  Hit "r" (or click button labeled "Reply"), as usual.
    2c.  No-op, as usual.
    3c.  Compose and send message, as usual.

list-post in From, author and list-post in Reply-To:

    1d.  Hit "r" (or click button labeled "Reply"), as usual.
    2d.  Mark and delete <list-post>, as usual for Reply-To-munging
         lists which add author to Reply-To.  In GMail this is
         actually easier than marking, as you can click on a
         delete-address button.
    3d.  Compose and send message, as usual.

There is one case that may be different, which is a "replace Reply-To
with list-post" Reply-To-munging list, where IIUC the user's identity
will be in the display name as "A. User via list-post" but her address
may not be.  If so, I think the answer is "Don't do that" if the list-
owner cares about reply-to-author, or perhaps Mailman's treatment can
be improved.  Note that the ugly display name in "From" should never
be automatically used by a "reply" function, because "Reply-To" will
always be present, and that overrides "From".  I believe that all
major MUAs conform.

A similar analysis can be done for reply-to-list, with I believe
similar results (the dog wants to be walked, gotta go).

I see no real difference between 2a and 2d, and I doubt users will be
particularly unhappy with switching from 2b to 2d.  It's true that
users will have to learn to trust that author is indeed in reply-to
(most MUAs don't display it), but I think they will, quite quickly.

For wrap_message, there is *no change at all* once you're actually
reading the message, but you probably have to take some action to read
the encapsulated message.

Footnotes: 
[1]  At least, is intended to work.  Mailman 2.1.16 and 2.1.17 had
some problems but I believe 2.1.18 now works as intended.

From stephen at xemacs.org  Tue May  6 08:28:41 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 06 May 2014 15:28:41 +0900
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <6.2.5.6.2.20140505023744.0b0f9f38@resistor.net>
References: <20140504132629.1234.qmail@joyce.lan>
 <874n15l8lp.fsf@uwakimon.sk.tsukuba.ac.jp>
 <53669920.1010508@signal100.com>
 <87vbtkk5si.fsf@uwakimon.sk.tsukuba.ac.jp>
 <6.2.5.6.2.20140505023744.0b0f9f38@resistor.net>
Message-ID: <87oazbju92.fsf@uwakimon.sk.tsukuba.ac.jp>

SM writes:

 > >RFCs must be shown to work in practice before they become Proposed
 > >Standards.  Ie, don't expect something to work until you see it.
 > 
 > This is a nit.  There isn't any requirement that RFCs have to be 
 > shown to work in practice  before they become Proposed Standards.

Don't you have that backwards?  It's pointing out lack of a formal
hard requirement that is nit-picking.  After all, Postel's Principle
isn't written in any IETF procedure manual.  Would you call that one a
"nit", too?


From stephen at xemacs.org  Tue May  6 08:39:36 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 06 May 2014 15:39:36 +0900
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <20140506034112.60036.qmail@joyce.lan>
References: <87vbtkk5si.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140506034112.60036.qmail@joyce.lan>
Message-ID: <87mwevjtqv.fsf@uwakimon.sk.tsukuba.ac.jp>

John Levine writes:

 > I wouldn't waste time worrying about whether various hacks might make
 > it 0.0001% easier to phish people.

Will you please stop focusing on *your* logic, and start thinking
about what happens if people with different interpretations of the
facts take action on those interpretations?

*I* am not really worried about 0.0001% easier to phish (although I
think my "2%" is a more accurate estimate).  What I worry about is
"what if Yahoo! and AOL think ...".  We already know that they think
differently from us.  They are desperate and grasping at straws, as
far as I can see.  The whole SPF-ADDoS-DKIM-DMARC path shows that they
are unwilling to bite the bullet of the obvious (and obviously
correct) solution: proper per-author digital signatures by default.

DMARC, as far as I can see (and have previously argued), is a good
optimization for corporate authors, where users of a mailbox in a
domain are delegates of the corporate owner of the domain.  Where that
is not true, it sucks for a whole slew of reasons, yet AOL and Yahoo!
are trying to apply it.

From sm at resistor.net  Tue May  6 12:16:41 2014
From: sm at resistor.net (SM)
Date: Tue, 06 May 2014 03:16:41 -0700
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <87oazbju92.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20140504132629.1234.qmail@joyce.lan>
 <874n15l8lp.fsf@uwakimon.sk.tsukuba.ac.jp>
 <53669920.1010508@signal100.com>
 <87vbtkk5si.fsf@uwakimon.sk.tsukuba.ac.jp>
 <6.2.5.6.2.20140505023744.0b0f9f38@resistor.net>
 <87oazbju92.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <6.2.5.6.2.20140506030149.0b9d5840@resistor.net>

Hi Stephen,
At 23:28 05-05-2014, Stephen J. Turnbull wrote:
>Don't you have that backwards?  It's pointing out lack of a formal
>hard requirement that is nit-picking.  After all, Postel's Principle
>isn't written in any IETF procedure manual.  Would you call that one a
>"nit", too?

I labelled my previous comment as a nit (the previous comment was 
unimportant).  I would not describe anything someone else writes as a nit.

Regards,
-sm 


From stephen at xemacs.org  Tue May  6 14:07:00 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 06 May 2014 21:07:00 +0900
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <6.2.5.6.2.20140506030149.0b9d5840@resistor.net>
References: <20140504132629.1234.qmail@joyce.lan>
 <874n15l8lp.fsf@uwakimon.sk.tsukuba.ac.jp>
 <53669920.1010508@signal100.com>
 <87vbtkk5si.fsf@uwakimon.sk.tsukuba.ac.jp>
 <6.2.5.6.2.20140505023744.0b0f9f38@resistor.net>
 <87oazbju92.fsf@uwakimon.sk.tsukuba.ac.jp>
 <6.2.5.6.2.20140506030149.0b9d5840@resistor.net>
Message-ID: <87ha53jel7.fsf@uwakimon.sk.tsukuba.ac.jp>

SM writes:
 > Hi Stephen,
 > At 23:28 05-05-2014, Stephen J. Turnbull wrote:
 > >Don't you have that backwards?  It's pointing out lack of a formal
 > >hard requirement that is nit-picking.  After all, Postel's Principle
 > >isn't written in any IETF procedure manual.  Would you call that one a
 > >"nit", too?
 > 
 > I labelled my previous comment as a nit (the previous comment was 
 > unimportant).  I would not describe anything someone else writes as a nit.

OK, thanks for the apology.  And sorry on my part for assuming you
were calling the (informal) rule a nit.


From johnl at taugh.com  Tue May  6 16:40:11 2014
From: johnl at taugh.com (John R Levine)
Date: 6 May 2014 10:40:11 -0400
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <87mwevjtqv.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <87vbtkk5si.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140506034112.60036.qmail@joyce.lan>
 <87mwevjtqv.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <alpine.BSF.2.00.1405061039130.27813@joyce.lan>

> > I wouldn't waste time worrying about whether various hacks might make
> > it 0.0001% easier to phish people.
>
> Will you please stop focusing on *your* logic, and start thinking
> about what happens if people with different interpretations of the
> facts take action on those interpretations?

My apologies.  My imagination is sadly limited by 20 years of running 
mailing lists for real people, and extensive conversations with the people 
who designed and use DMARC.

Regards,
John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

From mark at msapiro.net  Tue May  6 19:37:58 2014
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 06 May 2014 10:37:58 -0700
Subject: [Mailman-Developers] Mailman 2.1.18 final release
Message-ID: <53691DF6.1040101@msapiro.net>

A critical incompatibility between the Mailman 2.1.18 final release and
Python versions older than 2.6.5 or thereabouts affecting the DMARC Wrap
Message action was discovered and fixed. This incompatibility also
existed in the 2.1.16 and 2.1.17 releases.

Thus, I have released Mailman 2.1.18-1 with a fix for this
incompatibility. Please use 2.1.18-1 and not 2.1.18.

These releases have new features to help with mitigation of the impacts
of DMARC on mailing lists as well as fixing several bugs.

Python 2.4 is the minimum supported, but Python 2.7 is recommended.

There are significant new i18n strings associated with the DMARC
mitigation features. If you are interested in helping with the
translations of these strings, see <http://wiki.list.org/x/MQ>.

There is also a new dependency associated with these features. Namely,
the new Privacy options -> Sender filters -> dmarc_moderation_action
feature requires that the dnspython <http://www.dnspython.org/> package
be available in Python.

See the attached README for more details.

Mailman is free software for managing email mailing lists and
e-newsletters. Mailman is used for all the python.org and
SourceForge.net mailing lists, as well as at hundreds of other sites.

For more information, please see:

http://www.list.org
http://www.gnu.org/software/mailman
http://mailman.sourceforge.net/

Mailman 2.1.18-1 can be downloaded from

https://launchpad.net/mailman/2.1/
http://ftp.gnu.org/gnu/mailman/
https://sourceforge.net/projects/mailman/

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
2.1.18-1 (06-May-2014)

  Bug fixes and other patches

    - A critical incompatibility between the DMARC Wrap Message action and
      Python versions older than 2.6.x for some x <= 5 existed and caused
      Wrapped message to be shunted.  This is fixed.  (LP: #1316682)

    - Sender: headers are no longer removed in from_is_list Munge From
      actions.  (LP: #1315970)

2.1.18 (03-May-2014)

  Acknowledgements

    - Thanks to Jim Popovitch and Phil Pennock for the branch that formed the
      basis of the dmarc_moderation_action feature.

    - Thanks to Franck Martin et al for the branch that formed the basis of
      the from_is_list feature.

  Dependencies

    - There is a new dependency associated with the new Privacy options ->
      Sender filters -> dmarc_moderation_action feature discussed below.
      This requires that the dnspython <http://www.dnspython.org/> package
      be available in Python.  This package can be downloaded from the above
      site or from the CheeseShop <https://pypi.python.org/pypi/dnspython/>
      or installed with pip.

  New Features

    - The from_is_list feature introduced in 2.1.16 is now unconditionally
      available to list owners.  There is also, a new Privacy options ->
      Sender filters -> dmarc_moderation_action feature which applies to list
      messages where the From: address is in a domain which publishes a DMARC
      policy of reject or possibly quarantine.  This is a list setting with
      values of Accept, Wrap Message, Munge From, Reject or Discard. There is
      a new DEFAULT_DMARC_MODERATION_ACTION configuration setting to set the
      default for this, and the list admin UI is not able to set an action
      which is 'less' than the default.  The prior ALLOW_FROM_IS_LIST setting
      has been removed and is effectively always Yes. There is a new
      dmarc_quarantine_moderation_action list setting with default set by a
      new DEFAULT_DMARC_QUARANTINE_MODERATION_ACTION configuration setting
      which in turn defaults to Yes.  The list setting can be set to No to
      exclude domains with DMARC policy of quarantine from
      dmarc_moderation_action.

      dmarc_moderation_action and from_is_list interact in the following way.
      If the message is From: a domain to which dmarc_moderation_action applies
      and if dmarc_moderation_action is other than Accept,
      dmarc_moderation_action applies to that message.  Otherwise the
      from_is_list action applies.

      Also associated with dmarc_moderation_action are configuration settings
      DMARC_RESOLVER_TIMEOUT and DMARC_RESOLVER_LIFETIME. These are described
      in more detail in Defaults.py.  There are also new vette log entries
      written when dmarc_moderation_action is found to apply to a post.

  i18n

    - Added missing <mm-digest-question-start> tag to French listinfo template.
      (LP: #1275964)

  Bug Fixes and other patches

    - Removed HTML tags from the title of a couple of rmlist.py pages because
      browsers don't render tags in the title.  (LP: #265848)

    - Most Mailman generated notices to list owners and moderators are now
      sent as Precedence: list instead of bulk.  (LP: #1313146)

    - The Reply-To: munging options weren't honored if there was no
      from_is_list action.  (LP: #1313010)

    - Changed from_is_list actions to insert the list address in Cc: if the
      list is fully personalized.  Otherwise, the list address is only in
      From: and Reply-To: overrides it.  (LP: #1312970)

    - Fixed the Munge From action to only Munge the From: and/or Reply-To: in
      the outgoing message and not in archives, digests and messages sent via
      the usenet gateway.  (LP: #1311431)

    - Fixed a long standing issue in which a notice sent to a user whose
      language is other than that of the list can cause subsequent things
      which should be in the list's language to be in the user's language
      instead.  (LP: #1308655)

    - Fixed the admin Membership List so a search string if any is not lost
      when visiting subsequent fragments of a chunked list.  (LP: #1307454)

    - For from_is_list feature, use email address from original From: if
      original From: has no display name and strip domain part from resultant
      names that look like email addresses.  (LP: #1304511)

    - Added the list name to the vette log "held message approved" entry.
      (LP: 1295875)

    - Added the CGI module name to various "No such list" error log entries.
      (LP: 1295875)

    - Modified contrib/mmdsr to report module name if present in "No such list
      error log entries.

    - Fixed a NameError exception in cron/nightly_gzip when it tries to print
      the usage message.  (LP: #1291038)

    - Fixed a bug in ListAdmin._handlepost that would crash when trying to
      preserve a held message for the site admin if HOLD_MESSAGES_AS_PICKLES
      is False.  (LP: #1282365)

    - The from_is_list header munging feature introduced in Mailman 2.1.16 is
      no longer erroneously applied to Mailman generated notices.
      (LP: #1279667)

    - Changed the message from the confirm CGI to not indicate approval is
      required for an acceptance of an invitation.  (LP: #1277744)

    - Fixed POSTFIX_STYLE_VIRTUAL_DOMAINS to be case-insensitiive.
      (LP: #1267003)

    - Added recognition for another simple warning to bounce processing.
      (LP: #1263247)

    - Fixed a few failing tests in tests/test_handlers.py.  (LP: #1262950)

    - Fixed bin/arch to not create scrubbed attachments for messages skipped
      when processing the --start= option.  (LP: #1260883)

    - Fixed email address validation to do a bit better in obscure cases.
      (LP: #1258703)

    - Fixed a bug which caused some authentication cookies to expire too soon
      if AUTHENTICATION_COOKIE_LIFETIME is non-zero.  (LP: #1257112)

    - Fixed a possible TypeError in bin/sync_members introduced in 2.1.17.
      (LP: #1243343)

  Miscellaneous

    - Added to the contrib directory, a script from Alain Williams to count
      posts in a list's archive.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20140506/1e4118d3/attachment.sig>

From jimpop at gmail.com  Tue May  6 20:43:28 2014
From: jimpop at gmail.com (Jim Popovitch)
Date: Tue, 6 May 2014 14:43:28 -0400
Subject: [Mailman-Developers] Mailman 2.1.18 final release
In-Reply-To: <53691DF6.1040101@msapiro.net>
References: <53691DF6.1040101@msapiro.net>
Message-ID: <CAGfsgR3cvFQ8qAc10MaajppVsyPJhAt5XONaGtmMES54CCtN0A@mail.gmail.com>

On Tue, May 6, 2014 at 1:37 PM, Mark Sapiro <mark at msapiro.net> wrote:
> A critical incompatibility between the Mailman 2.1.18 final release and
> Python versions older than 2.6.5 or thereabouts affecting the DMARC Wrap
> Message action was discovered and fixed. This incompatibility also
> existed in the 2.1.16 and 2.1.17 releases.
>
> Thus, I have released Mailman 2.1.18-1 with a fix for this
> incompatibility. Please use 2.1.18-1 and not 2.1.18.

Thank you Mark, and thank you for the huge effort in getting the
Mailman 2.18.x release out the door.  I know everyone thinks this, I
just felt it needed to be stated.

-Jim P.

From barry at list.org  Wed May  7 04:15:24 2014
From: barry at list.org (Barry Warsaw)
Date: Tue, 6 May 2014 22:15:24 -0400
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <20140504132629.1234.qmail@joyce.lan>
References: <20140504132629.1234.qmail@joyce.lan>
Message-ID: <20140506221524.6a12dccd@anarchist.wooz.org>

On May 04, 2014, at 01:26 PM, John Levine wrote:

>I realize I'm a bit late to this party, but this is a technique that I
>don't think has been addressed here.  On my lists I've fixed the DMARC
>bounces by rewriting From: lines of DMARC'ed domains like this on the
>way out:
>
>  From: Marissa <mmeyer at yahoo.com>
>
>to
>
>  From: Marissa <mmeyer at yahoo.com.invalid>

I have some sympathy for this approach, as I mentioned over in mailman-users.
It violates RFCs so I'm not sure Mailman should adopt it, but it's worth
experimenting with, and I'm glad you (John) are doing so, and providing
feedback here.

I'm not personally concerned about the effects of .invalid on phishing, since
I largely agree with John's later statement that there are plenty of "pretty
close" domains you can stick in the From header that will fool most
non-technical users.  Heck, I see dozens per day and some are clever enough to
even fool me before close inspection reveals the subterfuge.  Add to that, as
others have observed, that many MUAs don't even display the actual email
address.

Of course, adding .invalid doesn't really solve the problem, and I'm quite
uncomfortable with overloading even more operations onto Reply-To.  As seen on
mailman-users, the interactions with the various options is a mess, difficult
to get right, fragile, and difficult to understand all the implications.

Message wrapping is the safest but equally unsatisfying.  It's pretty clear to
me that there are *no* good solution today to DMARC's affect on mailing lists,
only less bad ones.

-Barry

From stephen at xemacs.org  Wed May  7 04:59:12 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 07 May 2014 11:59:12 +0900
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <alpine.BSF.2.00.1405061039130.27813@joyce.lan>
References: <87vbtkk5si.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140506034112.60036.qmail@joyce.lan>
 <87mwevjtqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <alpine.BSF.2.00.1405061039130.27813@joyce.lan>
Message-ID: <87bnvajnun.fsf@uwakimon.sk.tsukuba.ac.jp>

John R Levine writes:

 > My apologies.  My imagination is sadly limited by 20 years of
 > running mailing lists for real people, and extensive conversations
 > with the people who designed and use DMARC.

Experience doesn't limit imagination, it's desperation to solve a
difficult problem in a hurry that does that.  That said, your years of
experience are undoubtedly valuable, as is the information from your
experiments with .INVALID.

I do not say you "should" take a more global view (I said "please",
didn't I?)  I'd like the benefit of your thoughts from that global
point of view, but it's up to you whether you want to go to that
effort.

I'm not trying to criticize you (though I've expressed myself badly --
I'm frustrated with the "damn the torpedos" atmosphere created by
Yahoo! and AOL, that's also coloring a lot of thinking here).  It's
not your responsibility to think globally -- you're an operator with a
problem to address, not responsible for creating a standard or for
maintaining infrastructure that implements that standard.  I support
your experiments with .INVALID, although I believe them to be
technically not conforming to RFC.  "Code is law", but *RFCs aren't*.

What'm trying to do is explain why Mailman should (IMHO) take a quite
different, much more conservative, stance toward implementing this,
and why I criticize DMARC.  The DMARC folks *are* creating a standard,
and Mailman *is* attempting to implement it.  It *is* *our* (and
*their*!) responsibility to think globally, even as we act locally.

BTW, we've had conversations with several of those "designers and
users" here, and they clearly do suffer from the "we have a problem we
need to fix, and we'll deal with the consequences for other people --
which we don't think are a big deal -- later if need be" attitude.
That includes several folks who do know better to do than what the big
portals are doing, so I figure they're REALLY desperate (or at least
the folks at email providers which are abusing p=reject are).

Note that AOL and Yahoo! need to do this because they have ambitions
of being e-commerce platforms, and so their domain names can be used
to scam money out of people.  I don't think their business model
justifies the harm their refusal to conform to RFC does to our
subscribers.


From johnl at taugh.com  Wed May  7 05:23:15 2014
From: johnl at taugh.com (John R Levine)
Date: 6 May 2014 23:23:15 -0400
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <87bnvajnun.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <87vbtkk5si.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140506034112.60036.qmail@joyce.lan>
 <87mwevjtqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <alpine.BSF.2.00.1405061039130.27813@joyce.lan>
 <87bnvajnun.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <alpine.BSF.2.00.1405062318070.3290@joyce.lan>

> What'm trying to do is explain why Mailman should (IMHO) take a quite
> different, much more conservative, stance toward implementing this,
> and why I criticize DMARC.

I don't know anyone who thinks the way that Yahoo and AOL are using DMARC 
is a good idea.

But you're nuts if you think that every Mailman list is going to kick off 
every Yahoo and AOL user, no matter how much the two providers might 
deserve it.  As I said, some of us run lists to get work done rather than 
to make a political point, and the only way to continue to get work done 
at this point is to route around the DMARC damage somehow.

> Note that AOL and Yahoo! need to do this because they have ambitions
> of being e-commerce platforms, and so their domain names can be used
> to scam money out of people.

We're deep enough into tin-foil hat territory here that we're done. 
Should you want to know what AOL and Yahoo are actually doing, see the 
most recent entry in my blog at http://jl.ly/

Regards,
John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

From stephen at xemacs.org  Wed May  7 06:30:43 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 07 May 2014 13:30:43 +0900
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <alpine.BSF.2.00.1405062318070.3290@joyce.lan>
References: <87vbtkk5si.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140506034112.60036.qmail@joyce.lan>
 <87mwevjtqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <alpine.BSF.2.00.1405061039130.27813@joyce.lan>
 <87bnvajnun.fsf@uwakimon.sk.tsukuba.ac.jp>
 <alpine.BSF.2.00.1405062318070.3290@joyce.lan>
Message-ID: <871tw6jjm4.fsf@uwakimon.sk.tsukuba.ac.jp>

John R Levine writes:

 > But you're nuts if you think that every Mailman list is going to
 > kick off every Yahoo and AOL user,

You can stop the ad hominem innuendo right there (that's an RFC 2119
MUST NOT).  There is plenty of documentary evidence on Mailman lists
that I'm fully aware that that is neither possible nor desirable.

If you would like to address what *Mailman* should do in polite terms,
I'm ready.




From stephen at xemacs.org  Wed May  7 06:41:59 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 07 May 2014 13:41:59 +0900
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <alpine.BSF.2.00.1405062318070.3290@joyce.lan>
References: <87vbtkk5si.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140506034112.60036.qmail@joyce.lan>
 <87mwevjtqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <alpine.BSF.2.00.1405061039130.27813@joyce.lan>
 <87bnvajnun.fsf@uwakimon.sk.tsukuba.ac.jp>
 <alpine.BSF.2.00.1405062318070.3290@joyce.lan>
Message-ID: <87zjiui4iw.fsf@uwakimon.sk.tsukuba.ac.jp>

John R Levine writes:

 > > Note that AOL and Yahoo! need to do this because they have
 > > ambitions of being e-commerce platforms, and so their domain
 > > names can be used to scam money out of people.
 > 
 > We're deep enough into tin-foil hat territory here that we're
 > done. Should you want to know what AOL and Yahoo are actually
 > doing, see the most recent entry in my blog at http://jl.ly/

*shrug* I'm a business school professor, and do analysis like the
analysis behind my statement above for a living.  It's a fairly
obvious extension of my posts on use of DMARC by financial
institutions.  And it helps explain why GMail seems to be actively
dissenting, and Hotmail has not followed yet.  Those email providers
have a different approach to being portals, and use different trade
names for their financial activites to the extent that I am aware of
them.  I'm not going to yield on it quite yet.

So, call it "tin-foil hat territory" if you like.  Being an expert on
Internet business models doesn't make me *right*, but name-calling
isn't going to bother me.

From johnl at taugh.com  Wed May  7 14:18:00 2014
From: johnl at taugh.com (John Levine)
Date: 7 May 2014 12:18:00 -0000
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <20140506221524.6a12dccd@anarchist.wooz.org>
Message-ID: <20140507121800.4398.qmail@joyce.lan>

>>  From: Marissa <mmeyer at yahoo.com.invalid>
>
>I have some sympathy for this approach, as I mentioned over in mailman-users.
>It violates RFCs so I'm not sure Mailman should adopt it, but it's worth
>experimenting with, and I'm glad you (John) are doing so, and providing
>feedback here.

I know the guy who wrote the RFC, and I'm reasonably sure he'd forgive
us under the circumstances.  There is, after all, plenty of mail from
addresses like donotreply at bigbank.com already, and this does the same
thing with slightly different and syntax.  Note that putting the
list's address in From: is also an RFC violation since the list isn't
the author by any normal understanding of the term.

>Message wrapping is the safest but equally unsatisfying.  It's pretty clear to
>me that there are *no* good solution today to DMARC's affect on mailing lists,
>only less bad ones.

The good solution is for the members of the DMARC group to set up and
use a shared DMARC-bypass whitelist for mailing lists and other
legitimate mail sources that don't match DMARC's narrow security model.
I'm working on it, but in the meantime, .INVALID has the advantage of
reminding people where the source of the problem lies.

R's,
John

From raj.abhilash1 at gmail.com  Thu May  8 11:05:54 2014
From: raj.abhilash1 at gmail.com (Abhilash Raj)
Date: Thu, 8 May 2014 14:35:54 +0530
Subject: [Mailman-Developers] Mailman-Suite demo server
In-Reply-To: <536B467D.3080604@opensource-expert.com>
References: <CAF2AkX=-ZEA9OSaXbXOfxjKZV1bN2BFOk6bZrWoPtZ3ZkgAjbw@mail.gmail.com>
 <53660E33.5000408@opensource-expert.com>
 <CAF2AkXm2rUSx_GooF5JoG8R=M0gdDGinc8BSreZe=56S67vmmw@mail.gmail.com>
 <53670C5F.7060202@opensource-expert.com>
 <CAF2AkX=S_T=LXFD+zuemmubgfZz9-TkwMvD6z_2OV7JfYHudbg@mail.gmail.com>
 <536B467D.3080604@opensource-expert.com>
Message-ID: <CAF2AkX=jkQk7iOBuHXgdmFfc-dVoBKN=6O8XOy4E9jQ_2kuJWA@mail.gmail.com>

On Thu, May 8, 2014 at 2:25 PM, Sylvain Viart <sylvain at opensource-expert.com
> wrote:

> Hi Abhilash,
>
>
> On 08/05/2014 10:33, Abhilash Raj wrote:
>
>> the mm3 web-ui) is a django app which uses mozzila persona for user
>> authentication purposes. ( Google and
>> other links were not working the last time I checked ).
>>
>
> Ok. Interesting.
>
>
>
>> Well theoretically it is possible to add a separate authentication module
>> to
>> postorius and hyperkitty as they both use the same django project while
>> deploying
>> using mailman-bundler. But AFAIK it is not among one of the todos. (You
>> will find
>> a long discussion on user authentication if you dig though mm-dev
>> archives)
>>
>
> A bit surprising to me. If I understand what you said correctly, you mean
> that mailman standalone (local) auth for mailing list subscription is, for
> now, delegated to some foreign generic auth module ? Here mozilla stuff.
>
> So I can't subscribe to you test list, without sharing that  with mozilla.
>
> I'm right?
>
>
Yes, authentication is delegated to mozzila persona as of now.

Funny? And scary. :-)
>
> Thanks for your reply and setting up a demo.
> The UI is a Sooo great improvement of the old good, but boring, mailman
> cgi. :-P
>
> Sylvain.
>
Yes it is a great improvement I agree. If you are so worried about sharing
your
email with mozzila I can create an account for you and you can see the
after-login
part in postorius and HK?

thanks,
Abhilash

From rajeevs1992 at gmail.com  Sun May 11 19:07:49 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Sun, 11 May 2014 22:37:49 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <536FAE65.6030002@gmail.com>



On Friday 09 May 2014 11:12 PM, Stephen J. Turnbull wrote:
> I would like to post this to Mailman-Developers, with your permission.
>
> Rajeev S writes:
>
>   > Hi Steve,Can I go forward with the current design?
>
> Looking at the code you've produced so far[1] I can say it's clean and
> (mostly) Pythonic, easy to read, making good use of packaged modules
> (and good choices of the ones to use).
>
> But to be honest, you haven't really *presented* a design yet.  A file
> tree as in your blog[2] isn't a design, and the descriptions of
> commands really amount to a set of example use cases, but hardly
> complete.  The design is implicit in patterns in the examples, and I
> have a feeling that it isn't yet coherent in your mind.
I only meant to expose my approach towards the project. Now, at least you
know how I have planned to approach the task at hand.
> So you can go forward, but if you continue to write code "as you go
> along", I think every couple of weeks we're going to have to refactor.
> That's OK if that's what you want (as you say in your blog
> "discussions over a piece of code would be more effective than just
> talk").  The alternative is to systematically explore the space of
> command + argument + option combinations, and try to come up with a
> language that compactly expresses the most common operations.  I
> recommend this approach if it works for you, because various options
> are starting to proliferate, and I would rather we keep them to a
> minimum so that users don't need to remember them all or be
> continuously consulting the documentation.  It may *not* work for you,
> and that's OK, we can go with the refactor-until-happy strategy then.
I too would prefer to go by the systematic approach. I felt the best way to
express my thoughts was to implement some part of it. I am refactoring
too much already.
> A couple of examples of the kind of things I think it would be useful
> to think about in advance:
>
> (1) You already have --domainname and --listname options, and of
>      course the pattern suggests --username for sure, and maybe there
>      will be additional --foonames.  I wonder if it wouldn't be better
>      to have a single --name option so that instead of
>
>      mmclient list create --domainname list.example.com --listname foo
>
>      the user types
>
>      mmclient list create --name foo --in-domain list.example.com
>
>      or
>
>      mmclient list create --namefoo at list.example.com
I doubt the usability of a common option `name`. As per the above 
snippet, you have
used a `in-domain` to specify the domain name. There are many such
instances where you would have to use more than one `name`, for instance
adding moderators for a list. In such cases, options like `domainname`, 
`listname`
and `username` would be more intuitive than using options like `in-domain`.

Further, the option `name` is ambiguous for a `user`.

>      or even
>
>      mmclient list createfoo at list.example.com
>
>      Note that the last is "Pythonic" in the sense that your
>      ArgumentParser will check for the right number of arguments.
This is a good way to do it, a more pythonic way,as mentioned.

Imposing a fair enough rule like the list must be specified only in
`list at domain` format can reduce a number of arguments in a *lot*
of commands.

By using a third (optional) positional argument that specifies
the `name` of the instance being managed, the commands look
more cleaner.

However, some commands still need more names to appear on the command
which I prefer *not* to replace by a single `--name`. The commands are 
mentioned below.

> (2) Should the scope (in [domain, list, user]) come first as in your
>      current design, "mmclient list create", or should the action come
>      first as in English, "mmclient create list"?  I think the latter
>      will be easier for non-technical admins to get used to, but it may
>      not be -- we need a list of "things we want to say" to decide that.
Commands that makes sense in English would suit most situations,
in fact, all situations in current list of use cases.

As the first step of the systematic design process, I present the list of
things we would want to say and how the command must look like and
what possible arguments it should accept.

I have modified the CLI to use English like commands and hence will
use them hereafter.

*list*

The command lists the entities and should be available for users,mailing 
lists and domains.

mmclient list list [list at domain.org] [-v for verbose]
mmclient list user [list at domain.org] [-v for verbose] [--role ROLE]
mmclient list domain [domain.org] [-v for verbose]

The ROLE would be any of member,owner or moderator.

*create*

Used to create an instance and should be available for domains, lists 
and users.

mmclient create domain domain.org [--contact CONTACT_EMAIL]
mmclient create list list at domain.org
mmclient create user foo at bar.com --name DISPLAY_NAME --password PASSWORD

*delete*

Removes a domain, list or user.

mmclient delete domain domain.org
mmclient delete list list at domain.org
mmclient delete user foo at bar.com

*Role management*

User roles can be managed by two actions, addrole and
removerole, rather than 6 separate actions for subscribe, unsubscribe,
addowner, addmoderator, removeowner, removemoderator

mmclient addrole user foo at bar.com --list list at domain.org --role 
<member,owner,moderator>
mmclient removerole user foo at bar.com --list list at domain.org --role 
<member,owner,moderator>

The above commands are an argument longer than the commands like

mmclient addmoderator list list at domain.org --user foo at bar.com

but I feel former approach better, as it looks better and reduces the number
of commands to learn/remember.

*Preferences/settings*

Settings would form a new `scope`, commands for which can be implemented
in a fairly straightforward way.

mmclient set setting <setting name> --value <value>

*Edit (user)*

Changing user credentials like email, password and display name

mmclient edit user foo at bar.com --email foo1 at bar.com
mmclient edit user foo at bar.com --name foo
mmclient edit user foo at bar.com --password bar

A common method that will support multiple arguments together
can be built for this action.
> I feel that it's likely to be quite annoying to refactor a ton of code
> if we decide to change those aspects of the design.  That's why I feel
> it's better to do a more detailed design now.
>
> A few unconnected comments on various aspects:
>
> - In your code you have some short variable names (d, l).  This is
>    mildly un-Pythonic.  Try reading diffs (eg, cmdparser.py in r52) and
>    I think you'll see.  If you don't see right off, I can explain
>    precisely why I stumbled in a couple of places, but I'd rather let
>    you see it for yourself.
Fixed.
> - Short names for long options like --ll are a definite no-no.  They
>    make the code harder to understand, and on the command line they are
>    arcane.  We need to cater to users who may not be so flexible.
I have replaced --ll with -v and --verbose.
> - The change you make in r52 is interesting.  I'm a little leery of
>    the use of getattr(), it's ugly, but that's probably easily fixed.
> - Once you have the parsed command, you look up the command and then
>    apply it to arguments retrieved from your ArgumentParser instance.
>    *These lookups can fail*, and you need to catch them (for
>    interactive shell mode).  I don't think ArgumentParser will check
>    that all syntactically required arguments are present, so you need
>    to do it yourself.  Think carefully about this -- you may want to
>    revise or even revert the change in r52 because it implies doing
>    this argument checking in each action rather than in the command
>    parser.
Why not leave the argument verification to the action method itself? I 
feel that
it's better to leave it there as it would make future addition of 
actions easier.

All the methods in the current version have a initial argument checking 
part. The
current version already `prints` error messages if required arguments 
are absent.

I will be replacing it with an `exception` so that it would be easier to 
integrate
it with the interactive shell.

Also argparse returns a dictionary with keys for *all* the possible 
options, whether or
not they are specified by the user. If the value of option is not 
specified by the user,
then the value of corresponding key would be `None`.

However, extra arguments are ignored without warning.
> - For detailed listings you add header in get_listing.  But I think
>    you should have a no-header option -- if you ever want to pipe that
>    output to a script the header is annoying.  My instinct was to move
>    it out of that function, but I guess you have to do it there to get
>    columnization right.
>
Adding --no-header requires just a trivial change.Will do.
> Some nitpicking about names:
>
>
> - "Operate" is pretty general, the conventional word for managing
>    lists of things is "Manage".
>
> - "Long list" is ambiguous ("one 'long list'" vs. a "paged list", for
>    example).  "Detailed" (your word) or maybe "verbose" (conventional)
>    is better.
>
> - I prefer the word "scope" for [domain, list, user] rather than
>    "type" or "instance".
Fixed.
> - Having "list" be both the name of a scope and the name of a command
>    in the same ArgumentParser means that you need to check syntax
>    (order of the words), not just presence of the words, to be sure the
>    user has given all the required arguments.  If the scope names and
>    command names are disjoint, then you could even let the user choose
>    whether to type "create list" or "list create".  (That may not be a
>    great idea.)
The so called positional arguments are parsed in the order that
they are listed in the initialize_options method. Hence the
order checking is performed by the argparse itself.
`create domain` is not same as `domain create`.

The presence of all required arguments is verified at
the method corresponding to an action.
> Footnotes:
> [1]http://bazaar.launchpad.net/~rajeevs1992/mailman.client/mailmancli/changes/
>
> [2]http://myfossblog.blogspot.in/2014/05/lets-talk-over-cup-of-code.html
>
>
In addition, I will be partitioning the commands/arguments to groups 
based on the
entity, by making use of the argparse sub-parsers. This would enhance 
the readability
of the help strings.

TODO list from this mail:

- no-header option for list
- positional argument for instance name
- exceptions on argument lookup failure
   - printing error messages on lookup failure
- partitioning arguments using sub-parsers

All parts mentioned fixed are fixed and will form r53, after some more 
cleaning, like the docs,
probably by tomorrow.The items mentioned in TODO lists will
be built once the design process is complete.

From rajeevs1992 at gmail.com  Mon May 12 05:04:04 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Mon, 12 May 2014 08:34:04 +0530
Subject: [Mailman-Developers]  [GSoC 2014] Mailman CLI Project
In-Reply-To: <536FAE65.6030002@gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <536FAE65.6030002@gmail.com>
Message-ID: <CAPq4ZOLvPxcbcOhheZ9o8_fCGfbvxFoGHFsfxSeJ=mG-0Nmw_g@mail.gmail.com>

---------- Forwarded message ----------
From: "Rajeev S" <rajeevs1992 at gmail.com>
Date: May 11, 2014 10:38 PM
Subject: Re: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
To: "Stephen J. Turnbull" <stephen at xemacs.org>, <
mailman-developers at python.org>



On Friday 09 May 2014 11:12 PM, Stephen J. Turnbull wrote:

> I would like to post this to Mailman-Developers, with your permission.
>
> Rajeev S writes:
>
>   > Hi Steve,Can I go forward with the current design?
>
> Looking at the code you've produced so far[1] I can say it's clean and
> (mostly) Pythonic, easy to read, making good use of packaged modules
> (and good choices of the ones to use).
>
> But to be honest, you haven't really *presented* a design yet.  A file
> tree as in your blog[2] isn't a design, and the descriptions of
> commands really amount to a set of example use cases, but hardly
> complete.  The design is implicit in patterns in the examples, and I
> have a feeling that it isn't yet coherent in your mind.
>
I only meant to expose my approach towards the project. Now, at least you
know how I have planned to approach the task at hand.

 So you can go forward, but if you continue to write code "as you go
> along", I think every couple of weeks we're going to have to refactor.
> That's OK if that's what you want (as you say in your blog
> "discussions over a piece of code would be more effective than just
> talk").  The alternative is to systematically explore the space of
> command + argument + option combinations, and try to come up with a
> language that compactly expresses the most common operations.  I
> recommend this approach if it works for you, because various options
> are starting to proliferate, and I would rather we keep them to a
> minimum so that users don't need to remember them all or be
> continuously consulting the documentation.  It may *not* work for you,
> and that's OK, we can go with the refactor-until-happy strategy then.
>
I too would prefer to go by the systematic approach. I felt the best way to
express my thoughts was to implement some part of it. I am refactoring
too much already.

 A couple of examples of the kind of things I think it would be useful
> to think about in advance:
>
> (1) You already have --domainname and --listname options, and of
>      course the pattern suggests --username for sure, and maybe there
>      will be additional --foonames.  I wonder if it wouldn't be better
>      to have a single --name option so that instead of
>
>      mmclient list create --domainname list.example.com --listname foo
>
>      the user types
>
>      mmclient list create --name foo --in-domain list.example.com
>
>      or
>
>      mmclient list create --namefoo at list.example.com
>
I doubt the usability of a common option `name`. As per the above snippet,
you have
used a `in-domain` to specify the domain name. There are many such
instances where you would have to use more than one `name`, for instance
adding moderators for a list. In such cases, options like `domainname`,
`listname`
and `username` would be more intuitive than using options like `in-domain`.

Further, the option `name` is ambiguous for a `user`.

      or even
>
>      mmclient list createfoo at list.example.com
>
>
>      Note that the last is "Pythonic" in the sense that your
>      ArgumentParser will check for the right number of arguments.
>
This is a good way to do it, a more pythonic way,as mentioned.

Imposing a fair enough rule like the list must be specified only in
`list at domain` format can reduce a number of arguments in a *lot*
of commands.

By using a third (optional) positional argument that specifies
the `name` of the instance being managed, the commands look
more cleaner.

However, some commands still need more names to appear on the command
which I prefer *not* to replace by a single `--name`. The commands are
mentioned below.


 (2) Should the scope (in [domain, list, user]) come first as in your
>      current design, "mmclient list create", or should the action come
>      first as in English, "mmclient create list"?  I think the latter
>      will be easier for non-technical admins to get used to, but it may
>      not be -- we need a list of "things we want to say" to decide that.
>
Commands that makes sense in English would suit most situations,
in fact, all situations in current list of use cases.

As the first step of the systematic design process, I present the list of
things we would want to say and how the command must look like and
what possible arguments it should accept.

I have modified the CLI to use English like commands and hence will
use them hereafter.

*list*

The command lists the entities and should be available for users,mailing
lists and domains.

mmclient list list [list at domain.org] [-v for verbose]
mmclient list user [list at domain.org] [-v for verbose] [--role ROLE]
mmclient list domain [domain.org] [-v for verbose]

The ROLE would be any of member,owner or moderator.

*create*

Used to create an instance and should be available for domains, lists and
users.

mmclient create domain domain.org [--contact CONTACT_EMAIL]
mmclient create list list at domain.org
mmclient create user foo at bar.com --name DISPLAY_NAME --password PASSWORD

*delete*

Removes a domain, list or user.

mmclient delete domain domain.org
mmclient delete list list at domain.org
mmclient delete user foo at bar.com

*Role management*

User roles can be managed by two actions, addrole and
removerole, rather than 6 separate actions for subscribe, unsubscribe,
addowner, addmoderator, removeowner, removemoderator

mmclient addrole user foo at bar.com --list list at domain.org --role
<member,owner,moderator>
mmclient removerole user foo at bar.com --list list at domain.org --role
<member,owner,moderator>

The above commands are an argument longer than the commands like

mmclient addmoderator list list at domain.org --user foo at bar.com

but I feel former approach better, as it looks better and reduces the number
of commands to learn/remember.

*Preferences/settings*

Settings would form a new `scope`, commands for which can be implemented
in a fairly straightforward way.

mmclient set setting <setting name> --value <value>

*Edit (user)*

Changing user credentials like email, password and display name

mmclient edit user foo at bar.com --email foo1 at bar.com
mmclient edit user foo at bar.com --name foo
mmclient edit user foo at bar.com --password bar

A common method that will support multiple arguments together
can be built for this action.

 I feel that it's likely to be quite annoying to refactor a ton of code
> if we decide to change those aspects of the design.  That's why I feel
> it's better to do a more detailed design now.
>
> A few unconnected comments on various aspects:
>
> - In your code you have some short variable names (d, l).  This is
>    mildly un-Pythonic.  Try reading diffs (eg, cmdparser.py in r52) and
>    I think you'll see.  If you don't see right off, I can explain
>    precisely why I stumbled in a couple of places, but I'd rather let
>    you see it for yourself.
>
Fixed.

 - Short names for long options like --ll are a definite no-no.  They
>    make the code harder to understand, and on the command line they are
>    arcane.  We need to cater to users who may not be so flexible.
>
I have replaced --ll with -v and --verbose.

 - The change you make in r52 is interesting.  I'm a little leery of
>    the use of getattr(), it's ugly, but that's probably easily fixed.
> - Once you have the parsed command, you look up the command and then
>    apply it to arguments retrieved from your ArgumentParser instance.
>    *These lookups can fail*, and you need to catch them (for
>    interactive shell mode).  I don't think ArgumentParser will check
>    that all syntactically required arguments are present, so you need
>    to do it yourself.  Think carefully about this -- you may want to
>    revise or even revert the change in r52 because it implies doing
>    this argument checking in each action rather than in the command
>    parser.
>
Why not leave the argument verification to the action method itself? I feel
that
it's better to leave it there as it would make future addition of actions
easier.

All the methods in the current version have a initial argument checking
part. The
current version already `prints` error messages if required arguments are
absent.

I will be replacing it with an `exception` so that it would be easier to
integrate
it with the interactive shell.

Also argparse returns a dictionary with keys for *all* the possible
options, whether or
not they are specified by the user. If the value of option is not specified
by the user,
then the value of corresponding key would be `None`.

However, extra arguments are ignored without warning.

 - For detailed listings you add header in get_listing.  But I think
>    you should have a no-header option -- if you ever want to pipe that
>    output to a script the header is annoying.  My instinct was to move
>    it out of that function, but I guess you have to do it there to get
>    columnization right.
>
>  Adding --no-header requires just a trivial change.Will do.

 Some nitpicking about names:
>
>
> - "Operate" is pretty general, the conventional word for managing
>    lists of things is "Manage".
>
> - "Long list" is ambiguous ("one 'long list'" vs. a "paged list", for
>    example).  "Detailed" (your word) or maybe "verbose" (conventional)
>    is better.
>
> - I prefer the word "scope" for [domain, list, user] rather than
>    "type" or "instance".
>
Fixed.

 - Having "list" be both the name of a scope and the name of a command
>    in the same ArgumentParser means that you need to check syntax
>    (order of the words), not just presence of the words, to be sure the
>    user has given all the required arguments.  If the scope names and
>    command names are disjoint, then you could even let the user choose
>    whether to type "create list" or "list create".  (That may not be a
>    great idea.)
>
The so called positional arguments are parsed in the order that
they are listed in the initialize_options method. Hence the
order checking is performed by the argparse itself.
`create domain` is not same as `domain create`.

The presence of all required arguments is verified at
the method corresponding to an action.

 Footnotes:
> [1]http://bazaar.launchpad.net/~rajeevs1992/mailman.
> client/mailmancli/changes/
>
> [2]http://myfossblog.blogspot.in/2014/05/lets-talk-over-cup-of-code.html
>
>
>  In addition, I will be partitioning the commands/arguments to groups
based on the
entity, by making use of the argparse sub-parsers. This would enhance the
readability
of the help strings.

TODO list from this mail:

- no-header option for list
- positional argument for instance name
- exceptions on argument lookup failure
  - printing error messages on lookup failure
- partitioning arguments using sub-parsers

All parts mentioned fixed are fixed and will form r53, after some more
cleaning, like the docs,
probably by tomorrow.The items mentioned in TODO lists will
be built once the design process is complete.

From barry at list.org  Mon May 12 17:35:47 2014
From: barry at list.org (Barry Warsaw)
Date: Mon, 12 May 2014 11:35:47 -0400
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <536FAE65.6030002@gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <536FAE65.6030002@gmail.com>
Message-ID: <20140512113547.509c953e@anarchist.wooz.org>

On May 11, 2014, at 10:37 PM, Rajeev S wrote:

>I doubt the usability of a common option `name`. As per the above snippet,
>you have used a `in-domain` to specify the domain name. There are many such
>instances where you would have to use more than one `name`, for instance
>adding moderators for a list. In such cases, options like `domainname`,
>`listname` and `username` would be more intuitive than using options like
>`in-domain`.

You'll want to take a cue from the `mailman create` command, and as a general
rule, I think the shell-cli should be as similar as possible to the rest-cli.
I'm open to modifying the former in order to better align them, as long as we
have a consistent view of commands and options across both tools.  Let's not
go down the accretion-without-design path of git. :)

-Barry

From barry at list.org  Mon May 12 17:42:11 2014
From: barry at list.org (Barry Warsaw)
Date: Mon, 12 May 2014 11:42:11 -0400
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <536FAE65.6030002@gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <536FAE65.6030002@gmail.com>
Message-ID: <20140512114211.1c34421d@anarchist.wooz.org>

On May 11, 2014, at 10:37 PM, Rajeev S wrote:

>I have modified the CLI to use English like commands and hence will
>use them hereafter.
>
>*list*
>
>The command lists the entities and should be available for users,mailing
>lists and domains.
>
>mmclient list list [list at domain.org] [-v for verbose]
>mmclient list user [list at domain.org] [-v for verbose] [--role ROLE]
>mmclient list domain [domain.org] [-v for verbose]

A better name might be `show` since the term "list" is so overloaded in this
context.  Here's it's being used as a verb and a noun to refer to different
concepts, and I think that's confusing.

Also as a general rule, I think we want just one level of subcommand, so that
`mmclient show --list` would be the template.  (That's open to discussion.)

>*Role management*
>
>User roles can be managed by two actions, addrole and
>removerole, rather than 6 separate actions for subscribe, unsubscribe,
>addowner, addmoderator, removeowner, removemoderator

Yuck.  Sorry but I'd like to discourage the use of made up or concatenated
words.  It's okay for some options to be multi-word separated by dashes,
e.g. --affect-bar or --change-foo.

I don't have a good alternative atm.

>*Preferences/settings*
>
>Settings would form a new `scope`, commands for which can be implemented
>in a fairly straightforward way.
>
>mmclient set setting <setting name> --value <value>

E.g.

mmclient set --key <name> --value <value>

-Barry

From stephen at xemacs.org  Mon May 12 18:45:54 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 13 May 2014 01:45:54 +0900
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <20140512114211.1c34421d@anarchist.wooz.org>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
Message-ID: <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>

Barry Warsaw writes:

 > Also as a general rule, I think we want just one level of
 > subcommand, so that `mmclient show --list` would be the template.
 > (That's open to discussion.)

I wonder about this in the context of argparse and the command line,
because argparse makes a strong distinction (and corresponding
associations) between *required positional* arguments and *optional*
keyword-like arguments (ie, any argument with leading dashes).

In the model Rajeev has shown so far, the "scope" argument (list,
domain, user) hasn't been optional.

 > mmclient set --key <name> --value <value>

This seems unnecessarily verbose on the one hand, and to not actually
correspond to an actual use case, on the other: there's no scope
mentioned.  I feel the scope should be mandatory, even if it's
sitewide:

    mmclient set --site-wide --key CAN_PERSONALIZE --value No
    mmclient set --domain=python.org --key CAN_PERSONALIZE --value Yes

(after the first one, the second would be an error, I guess, but in
other cases a site-wide setting would be interpreted as a default).

I guess this horse has already bolted the barn, but I wonder about a
syntax like

    mmclient set --site-wide --key PERSONALIZE --value Permitted
    mmclient set --domain=python.org --key PERSONALIZE --value Permitted
    mmclient set --list=mailman-users at python.org --key PERSONALIZE --value No

for resource constraining settings.  (Permitted could probably be an
alias for False.)

Steve




From sylvain at opensource-expert.com  Mon May 12 18:03:44 2014
From: sylvain at opensource-expert.com (Sylvain Viart)
Date: Mon, 12 May 2014 18:03:44 +0200
Subject: [Mailman-Developers] modifying payload using same encoding and
	Content-Transfer-Encoding
Message-ID: <5370F0E0.8030303@opensource-expert.com>

Hi,

I don't manage to figure out how to keep the same 
Content-Transfer-Encoding, while rewriting the content?

msg.set_payload() doesn't seems to trigger any encoding, base64 for 
example was set.

not using decode=True, return a base64 bloc? I just want to write it back.
I should be able to know that the payload was encoded?


             related = MIMEMultipart('related')

             html_footer = HTML_ATTACHMENT_HOLDER % {'HTML_HERE': 
html_attach}
             html_footer += '</body>'
             old_content = msg.get_payload(decode=True)
             new_content = re.sub(r'</body>', html_footer, old_content)
             if old_content != new_content:
                 syslog('debug', 'add html footer')
             else:
                 syslog('debug', 'no html footer added')

             charset = msg.get_content_charset()
             syslog('debug', 'get_content_charset: %s, 
Content-Transfer-Encoding: %s', charset, msg['Content-Transfer-Encoding'])

             msg.set_payload(new_content, charset)
             related.attach(msg)
             related.attach(clip_payload)


The output produced must be wrong:

Content-Transfer-Encoding: base64
MIME-Version: 1.0
Content-Type: text/html; charset="utf-8"

<html>
   <head>
     <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
   </head>
   <body bgcolor="#FFFFFF" text="#000000">

sysloged output:
May 12 17:45:32 2014 (1795) get_content_charset: utf-8, 
Content-Transfer-Encoding: base64

From mark at msapiro.net  Tue May 13 04:50:47 2014
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 12 May 2014 19:50:47 -0700
Subject: [Mailman-Developers] modifying payload using same encoding and
 Content-Transfer-Encoding
In-Reply-To: <5370F0E0.8030303@opensource-expert.com>
References: <5370F0E0.8030303@opensource-expert.com>
Message-ID: <53718887.7000507@msapiro.net>

On 05/12/2014 09:03 AM, Sylvain Viart wrote:
> 
> I don't manage to figure out how to keep the same
> Content-Transfer-Encoding, while rewriting the content?


Do you really want that? That's not what your example code does.


> msg.set_payload() doesn't seems to trigger any encoding, base64 for
> example was set.


It will if there is a charset parameter and the message contains no
Content-Transfer-Encoding: header.

See
<https://docs.python.org/2/library/email.message.html#email.message.Message.set_payload>
and the implicit
<https://docs.python.org/2/library/email.message.html#email.message.Message.set_charset>.


> not using decode=True, return a base64 bloc? I just want to write it back.
> I should be able to know that the payload was encoded?
> 
> 
>             related = MIMEMultipart('related')
> 
>             html_footer = HTML_ATTACHMENT_HOLDER % {'HTML_HERE':
> html_attach}
>             html_footer += '</body>'
>             old_content = msg.get_payload(decode=True)
>             new_content = re.sub(r'</body>', html_footer, old_content)
>             if old_content != new_content:
>                 syslog('debug', 'add html footer')
>             else:
>                 syslog('debug', 'no html footer added')
> 
>             charset = msg.get_content_charset()
>             syslog('debug', 'get_content_charset: %s,
> Content-Transfer-Encoding: %s', charset, msg['Content-Transfer-Encoding'])
> 


Here, add
            del msg['content-transfer-encoding']
Don't delete Content-Type or it will get set to text/plain.


>             msg.set_payload(new_content, charset)
>             related.attach(msg)
>             related.attach(clip_payload)
> 
> 
> The output produced must be wrong:
...

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From rajeevs1992 at gmail.com  Tue May 13 11:57:46 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Tue, 13 May 2014 15:27:46 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp> <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <5371EC9A.1030804@gmail.com>

On Monday 12 May 2014 10:15 PM, Stephen J. Turnbull wrote:
> Barry Warsaw writes:
>
>   > Also as a general rule, I think we want just one level of
>   > subcommand, so that `mmclient show --list` would be the template.
>   > (That's open to discussion.)
>
> I wonder about this in the context of argparse and the command line,
> because argparse makes a strong distinction (and corresponding
> associations) between *required positional* arguments and *optional*
> keyword-like arguments (ie, any argument with leading dashes).

Positional arguments *can be made* optional, also be supplied
with a default value, in case the argument was not specified.

In my opinion, I don't like the `one level of sub command` as
neither the user nor the developer is benefited of such a design.
The user ends up typing the same strings as before plus an extra
`--` followed by the same set of arguments.

And from the angle of implementation, each of the *scope* name
would require a different optional argument, followed by a set of
if-else's to land at the right *scope* to manage.

Further there is a possibility of the user specifying multiple scopes,

mmclient show --list --listname "list at domain.org" --domain

which makes the outcome dependent on the order in which the
if-else's are written. This is a serious bug when actions like `delete`
are being used.

> In the model Rajeev has shown so far, the "scope" argument (list,
> domain, user) hasn't been optional.

I assumed this model was OK since I had received no comments
against that part, since the beginning. I strongly believe that it is quite
effective to mention the scope this way.

>   > mmclient set --key <name> --value <value>
>
> This seems unnecessarily verbose on the one hand, and to not actually
> correspond to an actual use case, on the other: there's no scope
> mentioned.  I feel the scope should be mandatory, even if it's
> sitewide:
>
>      mmclient set --site-wide --key CAN_PERSONALIZE --value No
>      mmclient set --domain=python.org --key CAN_PERSONALIZE --value Yes
>
> (after the first one, the second would be an error, I guess, but in
> other cases a site-wide setting would be interpreted as a default).

Got a bit confused with the use of *scope* in this context.
Anyways, if the scope is not specified, apply the setting on a
default *scope*, `default=site-wide` makes sense, while others
do not.

> I guess this horse has already bolted the barn, but I wonder about a
> syntax like
>
>      mmclient set --site-wide --key PERSONALIZE --value Permitted
>      mmclient set --domain=python.org --key PERSONALIZE --value Permitted
>      mmclient set --list=mailman-users at python.org --key PERSONALIZE --value No
>
> for resource constraining settings.  (Permitted could probably be an
> alias for False.)


Sorry about the horse :). As I said, I assumed it was OK, and It was a
mistake from my part not to discuss the command syntax before
working on it.

Also, the above is still possible with the current version. The *scope* 
positional argument can
be made to default to a *scope* that has no solid structure, `settings` for
example. More generally, it could be defaulted to a `general` scope, managed
by a `General` class, that inherits from multiple classes like `Settings`,
`Backup/restore` etc.

And as the final word, I am ready to change the command style,

mmclient <action> <scope> <arguments>

if there is some serious disagreement with it.

> Steve
>
>
>

From rajeevs1992 at gmail.com  Tue May 13 12:04:17 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Tue, 13 May 2014 15:34:17 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <20140512114211.1c34421d@anarchist.wooz.org>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp> <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
Message-ID: <5371EE21.1030700@gmail.com>

On Monday 12 May 2014 09:12 PM, Barry Warsaw wrote:
> A better name might be `show` since the term "list" is so overloaded in this
> context.  Here's it's being used as a verb and a noun to refer to different
> concepts, and I think that's confusing.

Yes, its confusing. In fact, I was looking for a replacement for that. 
Thanks.

> Also as a general rule, I think we want just one level of subcommand, so that
> `mmclient show --list` would be the template.  (That's open to discussion.)
>
> ...
>
> Yuck.  Sorry but I'd like to discourage the use of made up or concatenated
> words.  It's okay for some options to be multi-word separated by dashes,
> e.g. --affect-bar or --change-foo.
>
> I don't have a good alternative atm.

Will do.

> E.g.
>
> mmclient set --key <name> --value <value>
>
>
I have answered this part and the `one level subcommand` part in the 
reply to Steve's mail.

From barry at list.org  Tue May 13 15:33:54 2014
From: barry at list.org (Barry Warsaw)
Date: Tue, 13 May 2014 09:33:54 -0400
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20140513093354.3bc16cb4@anarchist.wooz.org>

On May 13, 2014, at 01:45 AM, Stephen J. Turnbull wrote:

>In the model Rajeev has shown so far, the "scope" argument (list,
>domain, user) hasn't been optional.

If it's truly non-optional in the sense that there's no default, and the scope
is required, then maybe it's okay.  It just doesn't look "normal" to me.

> > mmclient set --key <name> --value <value>
>
>This seems unnecessarily verbose on the one hand, and to not actually
>correspond to an actual use case, on the other: there's no scope
>mentioned.  I feel the scope should be mandatory, even if it's
>sitewide:
>
>    mmclient set --site-wide --key CAN_PERSONALIZE --value No
>    mmclient set --domain=python.org --key CAN_PERSONALIZE --value Yes

This seems different than what Rajeev wrote, where he mentions that the
'setting' argument is the scope:

On May 11, 2014, at 10:37 PM, Rajeev S wrote:

>*Preferences/settings*
>
>Settings would form a new `scope`, commands for which can be implemented
>in a fairly straightforward way.
>
>mmclient set setting <setting name> --value <value>

On May 13, 2014, at 01:45 AM, Stephen J. Turnbull wrote:

>(after the first one, the second would be an error, I guess, but in
>other cases a site-wide setting would be interpreted as a default).
>
>I guess this horse has already bolted the barn, but I wonder about a
>syntax like
>
>    mmclient set --site-wide --key PERSONALIZE --value Permitted
>    mmclient set --domain=python.org --key PERSONALIZE --value Permitted
>    mmclient set --list=mailman-users at python.org --key PERSONALIZE --value No

I think that would be fine (probably --site is sufficient, and a bit
shorter).  Also I would suggest allowing either List-ID or posting address as
an argument to --list, e.g. --list=mailman-users.python.org would also be
acceptable.  Usually there will be no difference, but lists can be renamed and
the List-ID will never change.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20140513/3cf20ebc/attachment.sig>

From barry at list.org  Tue May 13 15:42:29 2014
From: barry at list.org (Barry Warsaw)
Date: Tue, 13 May 2014 09:42:29 -0400
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <5371EC9A.1030804@gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com>
Message-ID: <20140513094229.3ce30980@anarchist.wooz.org>

On May 13, 2014, at 03:27 PM, Rajeev S wrote:

>Further there is a possibility of the user specifying multiple scopes,
>
>mmclient show --list --listname "list at domain.org" --domain

Would --list be implied by seeing a `--listname=list at example.com`?  E.g. would
this be just as useful, and a little shorter:

mmclient show --listname=list at example.com --domain=example.org

?

>which makes the outcome dependent on the order in which the
>if-else's are written. This is a serious bug when actions like `delete`
>are being used.

Destructive actions should probably be more constrained in what they allow, so
that there's no possibility of ambiguity on either the part of the user or the
code.  "In the face of ambiguity, refuse the temptation to guess."

>Got a bit confused with the use of *scope* in this context.
>Anyways, if the scope is not specified, apply the setting on a
>default *scope*, `default=site-wide` makes sense, while others
>do not.

Hmm.  If scope is optional (because it has a default), then it's not a
required positional argument, right?  So shouldn't a --switch should be used?

>Sorry about the horse :). As I said, I assumed it was OK, and It was a
>mistake from my part not to discuss the command syntax before working on it.
>
>Also, the above is still possible with the current version. The *scope*
>positional argument can be made to default to a *scope* that has no solid
>structure, `settings` for example. More generally, it could be defaulted to a
>`general` scope, managed by a `General` class, that inherits from multiple
>classes like `Settings`, `Backup/restore` etc.
>
>And as the final word, I am ready to change the command style,
>
>mmclient <action> <scope> <arguments>
>
>if there is some serious disagreement with it.

I just want it to be consistent, easily described, and easily understood by
users.  If it makes sense for the mmclient CLI to different from the
shell-access mailman command, then we at least need to be "translatable"
between the two.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20140513/9f7d0af5/attachment.sig>

From sylvain at opensource-expert.com  Tue May 13 16:14:37 2014
From: sylvain at opensource-expert.com (Sylvain Viart)
Date: Tue, 13 May 2014 16:14:37 +0200
Subject: [Mailman-Developers] modifying payload using same encoding and
 Content-Transfer-Encoding
In-Reply-To: <53718887.7000507@msapiro.net>
References: <5370F0E0.8030303@opensource-expert.com>
 <53718887.7000507@msapiro.net>
Message-ID: <537228CD.5080400@opensource-expert.com>

Hi,

Thanks for your reply. Unfortunately it doesn't work. I don't know why.
I did read the doc many times, without the expected result.

>  See 
> <https://docs.python.org/2/library/email.message.html#email.message.Message.set_payload> 
> and the implicit 
> <https://docs.python.org/2/library/email.message.html#email.message.Message.set_charset>. 
>

My code is inside this custom handler. It works, but not for keeping the 
encoding.
https://github.com/Sylvain303/mailman-AttachmentMove/blob/master/AttachmentMove.py#L338

It could be related to this specific usage inside the recursive change?

To solve it, I've forced the encoding explicitly:

             msg.set_payload(new_content)
             del msg['content-transfer-encoding']
             encoders.encode_7or8bit(msg)

I hope it will handle it correctly.


This code outside mailman is working: took from 
http://bugs.python.org/issue1525919

from email.Message import Message
from email.Charset import Charset, QP
text = "="
msg = Message()
charset = Charset("utf-8")
charset.header_encoding = QP
charset.body_encoding = QP
msg.set_charset(charset)
msg.set_payload(text)
print msg.as_string()



I've tried with no success:

from email.Charset import Charset, QP, BASE64
[?]
             del msg['content-transfer-encoding']
             charset = Charset("utf-8")
             charset.header_encoding = BASE64
             charset.body_encoding = BASE64
             msg.set_charset(charset)
             msg.set_payload(new_content)

I got the output below.  But it's not base64? just the plain new_content 
I've produced.
I don't need the base64 encoding specially, just that the content match 
what the header is saying.


    MIME-Version: 1.0
    Content-Type: text/html; charset="utf-8"
    Content-Transfer-Encoding: base64

    <html>
       <head>[?]


If I call the encoding explicitly it works, in conjunction with 
msg.set_charset(charset), I got header Content-Transfer-Encoding: twice.

             msg.set_payload(new_content)
             encoders.encode_base64(msg)

Regards,
Sylvain.

From mark at msapiro.net  Wed May 14 02:45:47 2014
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 13 May 2014 17:45:47 -0700
Subject: [Mailman-Developers] modifying payload using same encoding and
 Content-Transfer-Encoding
In-Reply-To: <537228CD.5080400@opensource-expert.com>
References: <5370F0E0.8030303@opensource-expert.com>
 <53718887.7000507@msapiro.net> <537228CD.5080400@opensource-expert.com>
Message-ID: <5372BCBB.6070401@msapiro.net>

On 05/13/2014 07:14 AM, Sylvain Viart wrote:
> Hi,
> 
> Thanks for your reply. Unfortunately it doesn't work. I don't know why.
> I did read the doc many times, without the expected result.


It works for me. The following withlist interaction based on a stripped
down version of what you originally posted shows it works.

I create a message with utf-8, base64 encoded content. I get its decoded
payload, delete the existing Content-Transfer-Encoding: header and set a
new payload (not actually different, but could be) with the charset from
the original message and the body is base64 encoded and the
Content-Transfer-Encoding: header is set appropriately. And the Python
email package's encoding for utf-8 is base64, so the encoding will be
base64 if you just include the utf-8 charset argument in the call to
set_payload().

This

>             del msg['content-transfer-encoding']
>             charset = Charset("utf-8")
>             charset.header_encoding = BASE64
>             charset.body_encoding = BASE64
>             msg.set_charset(charset)

All gets undone when you do this

>             msg.set_payload(new_content)


$ /var/MM/21/bin/withlist -i
No list name supplied.
Python 2.7.5+ (default, Feb 27 2014, 19:37:08)
[GCC 4.8.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import email
>>> from Mailman.Message import Message
>>> msg = email.message_from_string("""From: me
... To: you
... Subject: test message
... Content-Transfer-Encoding: base64
... MIME-Version: 1.0
... Content-Type: text/html; charset="utf-8"
...
...
PGh0bWw+CjxoZWFkPgo8L2hlYWQ+Cjxib2R5PgpUaGlzIGlzIGEgdGVzdAo8L2JvZHk+CjwvaHRt
... bD4K
... """, Message)
>>> msg
>From nobody Tue May 13 17:30:12 2014
From: me
To: you
Subject: test message
Content-Transfer-Encoding: base64
MIME-Version: 1.0
Content-Type: text/html; charset="utf-8"

PGh0bWw+CjxoZWFkPgo8L2hlYWQ+Cjxib2R5PgpUaGlzIGlzIGEgdGVzdAo8L2JvZHk+CjwvaHRt
bD4K

>>> old_content = msg.get_payload(decode=True)
>>> old_content
'<html>\n<head>\n</head>\n<body>\nThis is a test\n</body>\n</html>\n'
>>> new_content = old_content
>>> charset = msg.get_content_charset()
>>> del msg['content-transfer-encoding']
>>> msg.set_payload(new_content, charset)
>>> msg
>From nobody Tue May 13 17:32:32 2014
From: me
To: you
Subject: test message
MIME-Version: 1.0
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64

PGh0bWw+CjxoZWFkPgo8L2hlYWQ+Cjxib2R5PgpUaGlzIGlzIGEgdGVzdAo8L2JvZHk+CjwvaHRt
bD4K

>>>



-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From stephen at xemacs.org  Wed May 14 05:01:01 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 14 May 2014 12:01:01 +0900
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <5371EC9A.1030804@gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com>
Message-ID: <87r43xdpxu.fsf@uwakimon.sk.tsukuba.ac.jp>

BTW, unless specifically mentioned that I'm speaking as mentor, I'm
speaking as an ordinary developer, and you should feel free to argue
with me, or agree with me, or reserve comment until you feel
comfortable discussing issues.  Also, I apologize if I end up talking
"down" to you.  I don't know you very well yet, so feel free to let me
know you already understand what I'm talking about.

Rajeev S writes:

 > Positional arguments *can be made* optional, also be supplied
 > with a default value, in case the argument was not specified.

Sure, I'm just saying that argparse "likes" to associate "positional"
with "required", and keyword-like with "optional".  We should consider
carefully whether we want to deviate from that practice because
there's probably good reason for it.

 > In my opinion, I don't like the `one level of sub command`

That's fine by me, and as your mentor I advise that you continue to
discuss this with Barry.  I hope that you and he will come to
agreement on this point.  In the end, however, Barry is The Client,
and if you don't come to a meeting of minds the default is to do it
his way.  OK?

Taking my mentor hat off, now.

 > Further there is a possibility of the user specifying multiple scopes,
 > 
 > mmclient show --list --listname "list at domain.org" --domain

I think this is a syntax error, and should be reported that way.

 > > In the model Rajeev has shown so far, the "scope" argument (list,
 > > domain, user) hasn't been optional.
 > 
 > I assumed this model was OK since I had received no comments
 > against that part, since the beginning. I strongly believe that it
 > is quite effective to mention the scope this way.

I agree with that.  Note, however, that it has been suggested that in
the shell it should be possible to

   >>> set scope list=foo-list at domain.org

so that after that only list-scope commands are allowed and only
foo-list will be affected.

 > Sorry about the horse :). As I said, I assumed it was OK, and It
 > was a mistake from my part not to discuss the command syntax before
 > working on it.

Again speaking as mentor, I would not call it a "mistake" in your
case, but rather say it is up to your personal preference.  You have
demonstrated that you do write code that can be refactored and that
you refactor spontaneously.  It's acceptable for developers to "write
code for discussion" to the extent that they are comfortable with such
refactoring.  Many clients will find that a great convenience (it's a
simple form of "prototype").

Conversely it's unacceptable for you to respond to The Client's
requirements by saying "I've already written the code and it doesn't
do that."  (Except, of course, if the existing code was written to a
specification accepted by the client and he's changing his mind.  Then
both sides have a responsibility to negotiate.)

 > Also, the above is still possible with the current version. The
 > *scope* positional argument can be made to default to a *scope*
 > that has no solid structure, `settings` for example. More
 > generally, it could be defaulted to a `general` scope, managed by a
 > `General` class, that inherits from multiple classes like
 > `Settings`, `Backup/restore` etc.

Wearing my developer hat, I don't much like an implicit default
scope.  The user should either specify the scope in each command, or
explicitly specify a default scope.

Regards,
Steve

From stephen at xemacs.org  Wed May 14 05:19:48 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 14 May 2014 12:19:48 +0900
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <20140513094229.3ce30980@anarchist.wooz.org>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com>
 <20140513094229.3ce30980@anarchist.wooz.org>
Message-ID: <87ppjhdp2j.fsf@uwakimon.sk.tsukuba.ac.jp>

Barry Warsaw writes:

 > Would --list be implied by seeing a `--listname=list at example.com`?
 > E.g. would this be just as useful, and a little shorter:
 > 
 > mmclient show --listname=list at example.com --domain=example.org

Are you thinking that this is equivalent to 

    mmclient show --list --listname=list at example.com
    mmclient show --domain=example.org

which would display the set of subscribers for list at example.com and
the set of lists for example.org?  I can see that as a minor
convenience, but it doesn't seem useful enough to allow.

 > "In the face of ambiguity, refuse the temptation to guess."

Rajeev, do you know about the Zen of Python?  If not yet, try
"python -m this".  :-)

 > I just want it to be consistent, easily described, and easily understood by
 > users.  If it makes sense for the mmclient CLI to different from the
 > shell-access mailman command, then we at least need to be "translatable"
 > between the two.

What do you mean by "shell-access mailman command"?
src/mailman/bin/mailman.py?


From raj.abhilash1 at gmail.com  Thu May 15 11:01:46 2014
From: raj.abhilash1 at gmail.com (Abhilash)
Date: Thu, 15 May 2014 14:31:46 +0530
Subject: [Mailman-Developers] Dev version of mailman-bundler
Message-ID: <5374827A.5070002@gmail.com>

Hi All,

Right now I mailman-bundler uses the PyPI packages for Mailman3, 
Postorius, HK, KittyStore and MailmanClient to fetch and deploy the 
entire mailman suite. I think it would be nice to have a version of it 
where latest code is pulled from the respective sources. I will try to 
work on it it but, I just wanted to know if it will be used at all by 
anyone else?

-- 
thanks,
Abhilash

From sylvain at opensource-expert.com  Thu May 15 10:17:32 2014
From: sylvain at opensource-expert.com (Sylvain Viart)
Date: Thu, 15 May 2014 10:17:32 +0200
Subject: [Mailman-Developers] modifying payload using same encoding and
 Content-Transfer-Encoding
In-Reply-To: <5372BCBB.6070401@msapiro.net>
References: <5370F0E0.8030303@opensource-expert.com>
 <53718887.7000507@msapiro.net> <537228CD.5080400@opensource-expert.com>
 <5372BCBB.6070401@msapiro.net>
Message-ID: <5374781C.90902@opensource-expert.com>

Hi,

Le 14/05/2014 02:45, Mark Sapiro a ?crit :
> It works for me. The following withlist interaction based on a 
> stripped down version of what you originally posted shows it works. 

Thanks a lot, it finally works as you said?

I was pretty sure to have tested that code already, but I should have 
missed a small part?

This code works:

             charset = msg.get_content_charset()
             old_content = msg.get_payload(decode=True)
             new_content = re.sub(r'</body>', html_footer, old_content)
             del msg['content-transfer-encoding']
             msg.set_payload(new_content, charset)


See the full code in the repository:
https://github.com/Sylvain303/mailman-AttachmentMove/blob/master/AttachmentMove.py#L352

Regards,
Sylvain.

From tim at xrammit.de  Thu May 15 12:08:39 2014
From: tim at xrammit.de (Tim Marx)
Date: Thu, 15 May 2014 12:08:39 +0200
Subject: [Mailman-Developers] Dev version of mailman-bundler
In-Reply-To: <5374827A.5070002@gmail.com>
References: <5374827A.5070002@gmail.com>
Message-ID: <53749227.3010303@xrammit.de>

> Right now I mailman-bundler uses the PyPI packages for Mailman3, 
> Postorius, HK, KittyStore and MailmanClient to fetch and deploy the 
> entire mailman suite. I think it would be nice to have a version of it 
> where latest code is pulled from the respective sources. I will try to 
> work on it it but, I just wanted to know if it will be used at all by 
> anyone else? 
Yes, I would use it too! Great idea!

From sylvain at opensource-expert.com  Thu May 15 17:50:24 2014
From: sylvain at opensource-expert.com (Sylvain Viart)
Date: Thu, 15 May 2014 17:50:24 +0200
Subject: [Mailman-Developers] syslog(debug) disable logging in production?
Message-ID: <5374E240.5030307@opensource-expert.com>

Hi,

I think that syslog() call don't pass through system syslog.
I use Mailman/Logging/Syslog.py

from Mailman.Logging.Syslog import syslog

Is there a way to keep the code without disabling it?
In the mailman/Mailman folder (2.1.15) debug tagged syslog calls are 
commented.

Of course I can write some code:

DEBUG = False

def debug(msg, *args, **kws):
     if DEBUG == 1:
         syslog.write_ex('debug', msg, args, kws)

def process(mlist, msg, msgdata=None):
     global DEBUG
     if hasattr(mlist, 'debug'):
         DEBUG = mlist.debug
[?]

enabling with: config_list -i <(echo mlist.debug=1) listname

I did, in fact? :-)

Did I miss some nice feature?

Regards,
Sylvain.


From mark at msapiro.net  Fri May 16 02:35:06 2014
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 15 May 2014 17:35:06 -0700
Subject: [Mailman-Developers] syslog(debug) disable logging in
	production?
In-Reply-To: <5374E240.5030307@opensource-expert.com>
References: <5374E240.5030307@opensource-expert.com>
Message-ID: <53755D3A.7050701@msapiro.net>

On 05/15/2014 08:50 AM, Sylvain Viart wrote:
> Hi,
> 
> I think that syslog() call don't pass through system syslog.
> I use Mailman/Logging/Syslog.py
> 
> from Mailman.Logging.Syslog import syslog
> 
> Is there a way to keep the code without disabling it?
> In the mailman/Mailman folder (2.1.15) debug tagged syslog calls are
> commented.
> 
> Of course I can write some code:
> 
> DEBUG = False
> 
> def debug(msg, *args, **kws):
>     if DEBUG == 1:
>         syslog.write_ex('debug', msg, args, kws)
> 
> def process(mlist, msg, msgdata=None):
>     global DEBUG
>     if hasattr(mlist, 'debug'):
>         DEBUG = mlist.debug
> [?]
> 
> enabling with: config_list -i <(echo mlist.debug=1) listname
> 
> I did, in fact? :-)
> 
> Did I miss some nice feature?


No, but you could just leave your syslog('debug', ...) calls intact and do

ln -s /dev/null /path/to/mailman/logs/debug

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From johnl at taugh.com  Fri May 16 17:50:51 2014
From: johnl at taugh.com (John Levine)
Date: 16 May 2014 15:50:51 -0000
Subject: [Mailman-Developers] Anyone tried the DMARC mail address
	translucent forwarder hack?
Message-ID: <20140516155051.10446.qmail@joyce.lan>

> and a really evil one where you append a name
>that points to a server that rewrites the address and remails it, e.g.
>mmeyer at yahoo.com.remail.lists.org -> mmeyer at yahoo.com.

This is apparently what LISTSERV does, give or take details of the
syntax of the forwarding address.

Has anyone tried this with Mailman?

R's,
John

PS: You don't have to tell me all the reasons it's not a good idea.
I'm just wondering how bad an idea it turns out to be in practice.

From mark at msapiro.net  Fri May 16 18:05:52 2014
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 16 May 2014 09:05:52 -0700
Subject: [Mailman-Developers] Anyone tried the DMARC mail address
 translucent forwarder hack?
In-Reply-To: <20140516155051.10446.qmail@joyce.lan>
References: <20140516155051.10446.qmail@joyce.lan>
Message-ID: <53763760.80400@msapiro.net>

On 05/16/2014 08:50 AM, John Levine wrote:
>> and a really evil one where you append a name
>> that points to a server that rewrites the address and remails it, e.g.
>> mmeyer at yahoo.com.remail.lists.org -> mmeyer at yahoo.com.
> 
> This is apparently what LISTSERV does, give or take details of the
> syntax of the forwarding address.
> 
> Has anyone tried this with Mailman?
> 
> R's,
> John
> 
> PS: You don't have to tell me all the reasons it's not a good idea.
> I'm just wondering how bad an idea it turns out to be in practice.


I'm not very expert in this area, but it seems at least with the above,
you'd need DNS entries for yahoo.com.remail.lists.org,
aol.com.remail.lists.org, thenextone.com.remail.lists.org,
theoneafterthat.com.remail.lists.org, ... and that would be a real pain.
Something like mmeyer=yahoo.com at remail.lists.org for the address might
be better.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From johnl at taugh.com  Fri May 16 21:53:26 2014
From: johnl at taugh.com (John Levine)
Date: 16 May 2014 19:53:26 -0000
Subject: [Mailman-Developers] Anyone tried the DMARC mail address
	translucent forwarder hack?
In-Reply-To: <53763760.80400@msapiro.net>
Message-ID: <20140516195326.11053.qmail@joyce.lan>

>>> that points to a server that rewrites the address and remails it, e.g.
>>> mmeyer at yahoo.com.remail.lists.org -> mmeyer at yahoo.com.

>I'm not very expert in this area, but it seems at least with the above,
>you'd need DNS entries for yahoo.com.remail.lists.org,
>aol.com.remail.lists.org, thenextone.com.remail.lists.org,
>theoneafterthat.com.remail.lists.org, ... and that would be a real pain.

You just need one DNS entry, for *.remail.lists.org.  Believe it or
not, that's legal, valid, standard, etc.  It's often used to allow
tag at fred.provider.com instead of fred+tag at provider.com.

>Something like mmeyer=yahoo.com at remail.lists.org for the address might
>be better. ...

You could do that, but the syntax details aren't all that important.
There are two issues that I was wondering about.

One is that if you do this in a naive way, you have a wide open relay
for bad guys to use.  You'd have to manage it, probably with a
combination of of only allowing mail to addresses you've rewritten,
rate limiting, and spam filtering.

The other is that if you do this very much, the rewritten addresses
will find their way into people's address books, and now you're stuck
being a semi-public mail forwarder forever.  You could limit how long
each address works, and after that put a note in the bounce message
telling people what address to use, but it has the potential to be
very confusing to the users.

R's,
John



From franck at peachymango.org  Sat May 17 04:16:31 2014
From: franck at peachymango.org (Franck Martin)
Date: Fri, 16 May 2014 21:16:31 -0500 (CDT)
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <WM!5354d7076943f5cea091f4b03025728f2ee6ce5fc0b5612c9dee8dbb28a067289b7275bfbcf85f31ecfca629d653f72c!@asav-1.01.com>
References: <20140507121800.4398.qmail@joyce.lan>
 <WM!5354d7076943f5cea091f4b03025728f2ee6ce5fc0b5612c9dee8dbb28a067289b7275bfbcf85f31ecfca629d653f72c!@asav-1.01.com>
Message-ID: <1856298671.144791.1400292991012.JavaMail.zimbra@peachymango.org>

The trouble with .invalid is that it is a domain that do not accept emails. Therefore why should you accept emails from a domain that does not allow you to reply to it?

It is bound in the future to create issues when people move to more serious/ubiquitous domain reputation schemes.

From johnl at taugh.com  Sat May 17 05:08:27 2014
From: johnl at taugh.com (John Levine)
Date: 17 May 2014 03:08:27 -0000
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <1856298671.144791.1400292991012.JavaMail.zimbra@peachymango.org>
Message-ID: <20140517030827.13231.qmail@joyce.lan>

In article <1856298671.144791.1400292991012.JavaMail.zimbra at peachymango.org> you write:
>The trouble with .invalid is that it is a domain that do not accept emails. Therefore why should you accept emails from a domain
>that does not allow you to reply to it?
>
>It is bound in the future to create issues when people move to more serious/ubiquitous domain reputation schemes.

Everyone I know who's tried to do spam filtering by SMTP callbacks to
verify sender addresses has stopped, for the dual reasons that it
doesn't work, and it's abusive.  I can't imagine why you think people
are likely to resume using it.  Considering the amount of mail I get
with non-replyable addresses like donotreply at bigbank.com, I'm not the
only person who doesn't think this is a problem.

Or if they do, it's easy enough to add domain suffixes that satisfy
whatever even more broken reputation scheme we need to defeat.

R's,
John

From bob at nleaudio.com  Sat May 17 05:36:42 2014
From: bob at nleaudio.com (Bob Puff)
Date: Fri, 16 May 2014 23:36:42 -0400
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <20140517030827.13231.qmail@joyce.lan>
References: <1856298671.144791.1400292991012.JavaMail.zimbra@peachymango.org>
 <20140517030827.13231.qmail@joyce.lan>
Message-ID: <20140517033502.M92538@nleaudio.com>

So guys... Is there a simple little hack we can do within MM 2.1 to try to
mitigate this issue, by adding .invalid or some other extension?  I've got a
few lists that are getting to the point where MM sends the probe email, and
then figures it is not a bouncing address, but a lot of emails are not being
delivered.

Bob


From franck at peachymango.org  Sat May 17 06:27:17 2014
From: franck at peachymango.org (Franck Martin)
Date: Fri, 16 May 2014 23:27:17 -0500 (CDT)
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <WM!46ee542221cec7ae9ef1ff535bf00c9d673c5e41519abdf3ed070b5c584be24b30b63b43c99fa6484204743af03d62aa!@asav-3.01.com>
References: <1856298671.144791.1400292991012.JavaMail.zimbra@peachymango.org>
 <20140517030827.13231.qmail@joyce.lan> <20140517033502.M92538@nleaudio.com>
 <WM!46ee542221cec7ae9ef1ff535bf00c9d673c5e41519abdf3ed070b5c584be24b30b63b43c99fa6484204743af03d62aa!@asav-3.01.com>
Message-ID: <BC27EC9C-07FB-4FE7-B293-9DD7D2462495@peachymango.org>

Upgrade to 2.1.18

Toute connaissance est une r?ponse ? une question.

> On May 16, 2014, at 20:43, "Bob Puff" <bob at nleaudio.com> wrote:
> 
> So guys... Is there a simple little hack we can do within MM 2.1 to try to
> mitigate this issue, by adding .invalid or some other extension?  I've got a
> few lists that are getting to the point where MM sends the probe email, and
> then figures it is not a bouncing address, but a lot of emails are not being
> delivered.
> 
> Bob
> 
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers at python.org
> https://mail.python.org/mailman/listinfo/mailman-developers
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/franck%40peachymango.org
> 
> Security Policy: http://wiki.list.org/x/QIA9

From franck at peachymango.org  Sat May 17 06:27:58 2014
From: franck at peachymango.org (Franck Martin)
Date: Fri, 16 May 2014 23:27:58 -0500 (CDT)
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <WM!8657222fba3d9ead8ea18ae4c7e5875724ba6b320b2b39320668c7d40bacc6c94a9ab8ccd8d90d18bd92fe119ddcb2e1!@asav-3.01.com>
References: <20140517030827.13231.qmail@joyce.lan>
 <WM!8657222fba3d9ead8ea18ae4c7e5875724ba6b320b2b39320668c7d40bacc6c94a9ab8ccd8d90d18bd92fe119ddcb2e1!@asav-3.01.com>
Message-ID: <62C70823-648C-40F0-956F-DC1BF01E2989@peachymango.org>

You are really mixing everything...

Toute connaissance est une r?ponse ? une question.

> On May 16, 2014, at 20:09, "John Levine" <johnl at taugh.com> wrote:
> 
> In article <1856298671.144791.1400292991012.JavaMail.zimbra at peachymango.org> you write:
>> The trouble with .invalid is that it is a domain that do not accept emails. Therefore why should you accept emails from a domain
>> that does not allow you to reply to it?
>> 
>> It is bound in the future to create issues when people move to more serious/ubiquitous domain reputation schemes.
> 
> Everyone I know who's tried to do spam filtering by SMTP callbacks to
> verify sender addresses has stopped, for the dual reasons that it
> doesn't work, and it's abusive.  I can't imagine why you think people
> are likely to resume using it.  Considering the amount of mail I get
> with non-replyable addresses like donotreply at bigbank.com, I'm not the
> only person who doesn't think this is a problem.
> 
> Or if they do, it's easy enough to add domain suffixes that satisfy
> whatever even more broken reputation scheme we need to defeat.
> 
> R's,
> John
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers at python.org
> https://mail.python.org/mailman/listinfo/mailman-developers
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/franck%40peachymango.org
> 
> Security Policy: http://wiki.list.org/x/QIA9

From mark at msapiro.net  Sat May 17 06:40:54 2014
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 16 May 2014 21:40:54 -0700
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <20140517033502.M92538@nleaudio.com>
References: <1856298671.144791.1400292991012.JavaMail.zimbra@peachymango.org>
 <20140517030827.13231.qmail@joyce.lan> <20140517033502.M92538@nleaudio.com>
Message-ID: <5376E856.9020200@msapiro.net>

On 05/16/2014 08:36 PM, Bob Puff wrote:
> So guys... Is there a simple little hack we can do within MM 2.1 to try to
> mitigate this issue, by adding .invalid or some other extension?  I've got a
> few lists that are getting to the point where MM sends the probe email, and
> then figures it is not a bouncing address, but a lot of emails are not being
> delivered.


Exactly how to patch this depends on what Mailman version you're
starting with, but you basically want some code like this.

    name, addrs = parseaddr(msg.get('from'))
    addrs += '.invalid'
    del msg['from']
    msg['From'] = formataddr((name, addrs))

If you put it in Mailman/Handlers/Cleanse.py or
Mailman/Handlers/CookHeaders.py, parseaddr and formataddr are already
imported from email.Utils so the above 4 lines added to the
process(mlist, msg, msgdata) function are all you need.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sat May 17 06:43:35 2014
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 16 May 2014 21:43:35 -0700
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <BC27EC9C-07FB-4FE7-B293-9DD7D2462495@peachymango.org>
References: <1856298671.144791.1400292991012.JavaMail.zimbra@peachymango.org>
 <20140517030827.13231.qmail@joyce.lan> <20140517033502.M92538@nleaudio.com>
 <WM!46ee542221cec7ae9ef1ff535bf00c9d673c5e41519abdf3ed070b5c584be24b30b63b43c99fa6484204743af03d62aa!@asav-3.01.com>
 <BC27EC9C-07FB-4FE7-B293-9DD7D2462495@peachymango.org>
Message-ID: <5376E8F7.4010609@msapiro.net>

On 05/16/2014 09:27 PM, Franck Martin wrote:
> Upgrade to 2.1.18


If one is going to upgrade, one should upgrade to Mailman 2.1.18-1, but
presumably the OP knows that and wants a different mitigation.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From bob at nleaudio.com  Sat May 17 07:13:38 2014
From: bob at nleaudio.com (Bob Puff)
Date: Sat, 17 May 2014 01:13:38 -0400
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <5376E856.9020200@msapiro.net>
References: <1856298671.144791.1400292991012.JavaMail.zimbra@peachymango.org>
 <20140517030827.13231.qmail@joyce.lan> <20140517033502.M92538@nleaudio.com>
 <5376E856.9020200@msapiro.net>
Message-ID: <20140517050835.M60706@nleaudio.com>

 
> Exactly how to patch this depends on what Mailman version you're
> starting with, but you basically want some code like this.
> 
>     name, addrs = parseaddr(msg.get('from'))
>     addrs += '.invalid'
>     del msg['from']
>     msg['From'] = formataddr((name, addrs))
> 
> If you put it in Mailman/Handlers/Cleanse.py or
> Mailman/Handlers/CookHeaders.py, parseaddr and formataddr are already
> imported from email.Utils so the above 4 lines added to the
> process(mlist, msg, msgdata) function are all you need.
> 

(Including your other email)

Right, I have a few highly-customized MM installs, and need to do this by hand
- can't just install the latest and greatest.

Thanks, I'll start testing now.

Bob


From franck at peachymango.org  Sat May 17 07:28:42 2014
From: franck at peachymango.org (Franck Martin)
Date: Sat, 17 May 2014 00:28:42 -0500 (CDT)
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <WM!5dbc5eb9ac245aa5cd3b9711daae2c8ad21270d79b0b7c4aacd0d19f828308defa24e0d6c3d8a79348a8da8d5f7d5579!@asav-2.01.com>
References: <1856298671.144791.1400292991012.JavaMail.zimbra@peachymango.org>
 <20140517030827.13231.qmail@joyce.lan> <20140517033502.M92538@nleaudio.com>
 <5376E856.9020200@msapiro.net> <20140517050835.M60706@nleaudio.com>
 <WM!5dbc5eb9ac245aa5cd3b9711daae2c8ad21270d79b0b7c4aacd0d19f828308defa24e0d6c3d8a79348a8da8d5f7d5579!@asav-2.01.com>
Message-ID: <1479716833.145194.1400304522987.JavaMail.zimbra@peachymango.org>

You can also apply this patch:

http://bazaar.launchpad.net/~mlm-author/mailman/2.1-author/revision/1341?remember=1338&compare_revid=1338

Rather than injecting an invalid domain in the From: and weakening more the security of email...



From bob at nleaudio.com  Sat May 17 07:56:50 2014
From: bob at nleaudio.com (Bob Puff)
Date: Sat, 17 May 2014 01:56:50 -0400
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <5376E856.9020200@msapiro.net>
References: <1856298671.144791.1400292991012.JavaMail.zimbra@peachymango.org>
 <20140517030827.13231.qmail@joyce.lan> <20140517033502.M92538@nleaudio.com>
 <5376E856.9020200@msapiro.net>
Message-ID: <20140517054946.M67202@nleaudio.com>


>     name, addrs = parseaddr(msg.get('from'))
>     addrs += '.invalid'
>     del msg['from']
>     msg['From'] = formataddr((name, addrs))
> 
> If you put it in Mailman/Handlers/Cleanse.py or
> Mailman/Handlers/CookHeaders.py, parseaddr and formataddr are already
> imported from email.Utils so the above 4 lines added to the
> process(mlist, msg, msgdata) function are all you need.

Hey Mark,

I'm getting: NameError: global name 'parseaddr' is not defined
Should something else be imported?

Bob


From stephen at xemacs.org  Sat May 17 09:44:08 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 17 May 2014 16:44:08 +0900
Subject: [Mailman-Developers] Anyone tried the DMARC mail
	address	translucent forwarder hack?
In-Reply-To: <20140516195326.11053.qmail@joyce.lan>
References: <53763760.80400@msapiro.net> <20140516195326.11053.qmail@joyce.lan>
Message-ID: <87k39kdf3r.fsf@uwakimon.sk.tsukuba.ac.jp>

John Levine writes:

 > You just need one DNS entry, for *.remail.lists.org.  Believe it or
 > not, that's legal, valid, standard, etc.

Legal, valid, and useful, yes.  However, it's generally considered a
poor practice because it means that all of those domains exist, which
makes it hard to debug misrouted connections.

In this case it's sufficiently useful that it's worth considering.

 > You could do that, but the syntax details aren't all that important.

Indeed.

 > One is that if you do this in a naive way, you have a wide open
 > relay for bad guys to use.  You'd have to manage it, probably with
 > a combination of only allowing mail to addresses you've rewritten,
 > rate limiting, and spam filtering.

Yes.  This is true regardless of the syntax used, of course.  You also
need to be sure that such addresses that are not in "From" get
rewritten to original form (subject to the same checks), and so on
both in distributed posts and in your archives.

 > The other is that if you do this very much, the rewritten addresses
 > will find their way into people's address books, and now you're stuck
 > being a semi-public mail forwarder forever.

Also into archives of mailing lists and the like.

I hope anybody thinking about doing this takes the above warnings to
heart, especially about the potential for abuse as an open relay.


From stephen at xemacs.org  Sat May 17 09:59:53 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 17 May 2014 16:59:53 +0900
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <20140517033502.M92538@nleaudio.com>
References: <1856298671.144791.1400292991012.JavaMail.zimbra@peachymango.org>
 <20140517030827.13231.qmail@joyce.lan>
 <20140517033502.M92538@nleaudio.com>
Message-ID: <87iop4dedi.fsf@uwakimon.sk.tsukuba.ac.jp>

Bob Puff writes:

 > So guys... Is there a simple little hack we can do within MM 2.1 to
 > try to mitigate this issue, by adding .invalid or some other
 > extension?  I've got a few lists that are getting to the point
 > where MM sends the probe email, and then figures it is not a
 > bouncing address, but a lot of emails are not being delivered.

In Mailman 2.1.16 and later, you don't need a "hack", just to set the
appropriate options[1], and as of 2.1.18-1 Mailman 2 provides four
options (munging "From" to the list-post address and Reply-To to
poster, or wrapping the whole post in a MIME part of a message "From"
the list-post address, with the option of applying the chosem
transformation either to all posts or just to those publishing a DMARC
p=reject policy).

All variants are known to be effective in getting past p=reject.  What
are your other requirements?  Also, is it possible that the problem is
not due to DMARC p=reject, but a different issue that arose at this
time by coincidence?



Footnotes: 
[1]  In Mailman 2.1.16 you also need to enable the options in
mm_cfg.py, but in later versions the DMARC mitigation options are
available to each list individually.  Checking for p=reject requires
installing dnspython, a Python package not part of Mailman or the
Python standard library.


From stephen at xemacs.org  Sat May 17 10:33:20 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 17 May 2014 17:33:20 +0900
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <1479716833.145194.1400304522987.JavaMail.zimbra@peachymango.org>
References: <1856298671.144791.1400292991012.JavaMail.zimbra@peachymango.org>
 <20140517030827.13231.qmail@joyce.lan>
 <20140517033502.M92538@nleaudio.com> <5376E856.9020200@msapiro.net>
 <20140517050835.M60706@nleaudio.com>
 <WM!5dbc5eb9ac245aa5cd3b9711daae2c8ad21270d79b0b7c4aacd0d19f828308defa24e0d6c3d8a79348a8da8d5f7d5579!@asav-2.01.com>
 <1479716833.145194.1400304522987.JavaMail.zimbra@peachymango.org>
Message-ID: <87ha4odctr.fsf@uwakimon.sk.tsukuba.ac.jp>

Franck Martin writes:

 > You can also apply this patch:
 > 
 > http://bazaar.launchpad.net/~mlm-author/mailman/2.1-author/revision/1341?remember=1338&compare_revid=1338
 > 
 > Rather than injecting an invalid domain in the From: and weakening
 > more the security of email...

If your *primary* concern is preserving the integrity of the email
system, the right thing to do is go straight to Privacy | SPAM Filters
and add "[.@]aol\.com$" and "[.@]yahoo\.com$" with a HOLD action
(can't "reject", unfortunately, because as far as I know significant
amounts of spam etc still originate from those domains).  Then reject
genuine posts and discard spam.

This is completely in accord with the "p=reject" policies published by
those domains, which not only will result in rejection by most ESPs,
but also threaten denial of service to other subscribers.  If their
users have a complaint about nondelivery, they should make it to their
ESPs which publish p=reject.

For "security" of email, the right thing to do is to use DKIM and/or
strongly encourage your users to use personal digital signatures, and
allow recipients to use that information to secure themselves.  In my
experience GMail does a very good job -- I don't get spam and I don't
lose authentic mail as far as I can tell.  I don't know what others
think.  I do know GMail is the haven chosen by all of the people I
know who've chosen to leave Yahoo! and AOL recently.

Regards,


From sca at andreasschulze.de  Sat May 17 13:34:35 2014
From: sca at andreasschulze.de (Andreas Schulze)
Date: Sat, 17 May 2014 13:34:35 +0200
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <20140517033502.M92538@nleaudio.com>
References: <1856298671.144791.1400292991012.JavaMail.zimbra@peachymango.org>
 <20140517030827.13231.qmail@joyce.lan>
 <20140517033502.M92538@nleaudio.com>
Message-ID: <20140517113435.GC25649@solar.andreasschulze.de>

Bob Puff:
> So guys... Is there a simple little hack we can do within MM 2.1 to try to
> mitigate this issue, by adding .invalid or some other extension?  I've got a
> few lists that are getting to the point where MM sends the probe email, and
> then figures it is not a bouncing address, but a lot of emails are not being
> delivered.


Bob,

you may also reconfigure your lists:
$subject_prefix = ''
$msg_footer = ''

mostly no message modifications -> dkim sigs stay valid -> dmarc pass.

Andreas

From johnl at taugh.com  Sat May 17 16:12:26 2014
From: johnl at taugh.com (John Levine)
Date: 17 May 2014 14:12:26 -0000
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <5376E856.9020200@msapiro.net>
Message-ID: <20140517141226.99065.qmail@joyce.lan>

>Exactly how to patch this depends on what Mailman version you're
>starting with, but you basically want some code like this.
>
>    name, addrs = parseaddr(msg.get('from'))
>    addrs += '.invalid'
>    del msg['from']
>    msg['From'] = formataddr((name, addrs))
>
>If you put it in Mailman/Handlers/Cleanse.py or
>Mailman/Handlers/CookHeaders.py, parseaddr and formataddr are already
>imported from email.Utils so the above 4 lines added to the
>process(mlist, msg, msgdata) function are all you need.

How do you limit it to just addresses with DMARC problems?  There's no
benefit to doing it to everyone.

R's,
John

PS: My experiments have been with mj2 because I know the code better.

From stephen at xemacs.org  Sat May 17 18:37:49 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 18 May 2014 01:37:49 +0900
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <20140517141226.99065.qmail@joyce.lan>
References: <5376E856.9020200@msapiro.net>
 <20140517141226.99065.qmail@joyce.lan>
Message-ID: <87wqdkbbtu.fsf@uwakimon.sk.tsukuba.ac.jp>

Quoth Mark Sapiro:

 > >Exactly how to patch this depends on what Mailman version you're
 > >starting with, but you basically want some code like this.

snip

John Levine writes:

 > How do you limit it to just addresses with DMARC problems?  There's no
 > benefit to doing it to everyone.

Probably a hard-coded regexp (or list thereof) matching against the
address in "From".  The OP has a special situation where he's got lots
of local mods to Mailman, so it's not convenient to upgrade.  So he
wants a quick and dirty approach, and he also knows enough Python that
he can work out the details for himself.



From fmouse at fmp.com  Sat May 17 17:05:30 2014
From: fmouse at fmp.com (Lindsay Haisley)
Date: Sat, 17 May 2014 10:05:30 -0500
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <20140517141226.99065.qmail@joyce.lan>
References: <20140517141226.99065.qmail@joyce.lan>
Message-ID: <1400339130.11452.70.camel@pudina.fmp.com>

On Sat, 2014-05-17 at 14:12 +0000, John Levine wrote:
> How do you limit it to just addresses with DMARC problems?  There's no
> benefit to doing it to everyone.
> 
Because a DMARC record is published in DNS, Mailman must use a Python
module capable of querying DNS.  MM 2.1.18 uses the dnspython
<http://www.dnspython.org/> package for this.  I believe PyDNS will also
work.

-- 
Lindsay Haisley       | "Everything works if you let it"
FMP Computer Services |
512-259-1190          |          --- The Roadie
http://www.fmp.com    |


From johnl at taugh.com  Sat May 17 19:14:05 2014
From: johnl at taugh.com (John R Levine)
Date: 17 May 2014 13:14:05 -0400
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <87wqdkbbtu.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <5376E856.9020200@msapiro.net>
 <20140517141226.99065.qmail@joyce.lan>
 <87wqdkbbtu.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <alpine.BSF.2.00.1405171313250.23413@joyce.lan>

> > How do you limit it to just addresses with DMARC problems?  There's no
> > benefit to doing it to everyone.
>
> Probably a hard-coded regexp (or list thereof) matching against the
> address in "From".  The OP has a special situation where he's got lots
> of local mods to Mailman, so it's not convenient to upgrade.  So he
> wants a quick and dirty approach, and he also knows enough Python that
> he can work out the details for himself.

Isn't there code in 2.18 to check for DMARC problems to catch messages on 
the way in?  I'd think you could adapt that.

Regards,
John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

From mark at msapiro.net  Sat May 17 19:19:41 2014
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 17 May 2014 10:19:41 -0700
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <20140517054946.M67202@nleaudio.com>
References: <1856298671.144791.1400292991012.JavaMail.zimbra@peachymango.org>
 <20140517030827.13231.qmail@joyce.lan> <20140517033502.M92538@nleaudio.com>
 <5376E856.9020200@msapiro.net> <20140517054946.M67202@nleaudio.com>
Message-ID: <53779A2D.2060206@msapiro.net>

On 05/16/2014 10:56 PM, Bob Puff wrote:
> 
>>     name, addrs = parseaddr(msg.get('from'))
>>     addrs += '.invalid'
>>     del msg['from']
>>     msg['From'] = formataddr((name, addrs))
>>
>> If you put it in Mailman/Handlers/Cleanse.py or
>> Mailman/Handlers/CookHeaders.py, parseaddr and formataddr are already
>> imported from email.Utils so the above 4 lines added to the
>> process(mlist, msg, msgdata) function are all you need.
> 
> Hey Mark,
> 
> I'm getting: NameError: global name 'parseaddr' is not defined
> Should something else be imported?


You need

from email.Utils import parseaddr, formataddr

I said Cleanse.py and Cookheaders.py already contains this, but pre
2.1.16 versions of Cleanse.py only have

from email.Utils import formataddr

so if you're using Cleanse.py, add ', parseaddr' to that.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From rajeevs1992 at gmail.com  Sat May 17 19:21:55 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Sat, 17 May 2014 22:51:55 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <20140513094229.3ce30980@anarchist.wooz.org>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>	<20140501214322.7637aa17@limelight.wooz.org>	<CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>	<CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>	<CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>	<87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>	<CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>	<20140503132132.33b88006@anarchist.wooz.org>	<CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>	<CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>	<87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>	<536FAE65.6030002@gmail.com>	<20140512114211.1c34421d@anarchist.wooz.org>	<87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>	<5371EC9A.1030804@gmail.com>
 <20140513094229.3ce30980@anarchist.wooz.org>
Message-ID: <53779AB3.4070603@gmail.com>


On Tuesday 13 May 2014 07:12 PM, Barry Warsaw wrote:
> On May 13, 2014, at 03:27 PM, Rajeev S wrote:
>
> Would --list be implied by seeing a `--listname=list at example.com`?  E.g. would
> this be just as useful, and a little shorter:
>
> mmclient show --listname=list at example.com --domain=example.org
>
> ?

In the command *mmclient <scope> <action>* , the scope denotes
the class name and the action denotes the class method to be invoked.

The following format suggested by you,

mmclient show --listname=list at example.com --domain=example.org

 From the developer point of view, its difficult to map an action to a 
class
by using this format. For eg, the above command can be associated
with the class domain or list, as the order of the arguments is 
insignificant.

Further, the commands do not offer much advantage to the user in terms
of usability. Most of the current (frequent) commands are quite 
straightforward
and similar to the ones to be used in the shell, for eg, *create list 
list at domain.org*
compared to *create --listname="list at domain.org"*.

Some commands like *add-role* are less intuitive in this format,yet provide
a tolerable level of understandability.

>> which makes the outcome dependent on the order in which the
>> if-else's are written. This is a serious bug when actions like `delete`
>> are being used.
> Destructive actions should probably be more constrained in what they allow, so
> that there's no possibility of ambiguity on either the part of the user or the
> code.  "In the face of ambiguity, refuse the temptation to guess."
>
>> Got a bit confused with the use of *scope* in this context.
>> Anyways, if the scope is not specified, apply the setting on a
>> default *scope*, `default=site-wide` makes sense, while others
>> do not.
> Hmm.  If scope is optional (because it has a default), then it's not a
> required positional argument, right?  So shouldn't a --switch should be used?


A switch would work best when the number of possible choices are
very few. Here we have the following choices for the scope, site-wide, 
domain
and list. It seems OK to use a switch here.

However, using switches limits the extendibility of the scopes. If a new 
scope
is to be added some day (something server-wide?), integrating new 
switches is not an easy task, when
compared to the addition of a new choice for scope in current approach.

>> And as the final word, I am ready to change the command style,
>>
>> mmclient <action> <scope> <arguments>
>>
>> if there is some serious disagreement with it.
> I just want it to be consistent, easily described, and easily understood by
> users.  If it makes sense for the mmclient CLI to different from the
> shell-access mailman command, then we at least need to be "translatable"
> between the two.

The current format of commands,mmclient <scope> <action> <arguments> is
directly translatable to the shell. In fact, they are almost similar, 
except for the `--`
in the commands.


From johnl at taugh.com  Sat May 17 19:33:05 2014
From: johnl at taugh.com (John Levine)
Date: 17 May 2014 17:33:05 -0000
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <1400339130.11452.70.camel@pudina.fmp.com>
Message-ID: <20140517173305.21605.qmail@joyce.lan>

>> How do you limit it to just addresses with DMARC problems?  There's no
>> benefit to doing it to everyone.
>> 
>Because a DMARC record is published in DNS, Mailman must use a Python
>module capable of querying DNS.  MM 2.1.18 uses the dnspython
><http://www.dnspython.org/> package for this.  I believe PyDNS will also
>work.

Well, yes, obviously.  Since DNS checks can be slow, it would be nice
to reuse the answers that Mailman probably already has.  Where are
they stored?

R's,
John




From mark at msapiro.net  Sat May 17 19:50:00 2014
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 17 May 2014 10:50:00 -0700
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <20140517173305.21605.qmail@joyce.lan>
References: <20140517173305.21605.qmail@joyce.lan>
Message-ID: <5377A148.7090002@msapiro.net>

On 05/17/2014 10:33 AM, John Levine wrote:
> 
> Well, yes, obviously.  Since DNS checks can be slow, it would be nice
> to reuse the answers that Mailman probably already has.  Where are
> they stored?


They aren't stored anywhere in Mailman, but they are likely cached in a
local name server.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From fmouse at fmp.com  Sat May 17 22:40:04 2014
From: fmouse at fmp.com (Lindsay Haisley)
Date: Sat, 17 May 2014 15:40:04 -0500
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <87wqdkbbtu.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <5376E856.9020200@msapiro.net>
 <20140517141226.99065.qmail@joyce.lan>
 <87wqdkbbtu.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <1400359204.13050.30.camel@pudina.fmp.com>

On Sun, 2014-05-18 at 01:37 +0900, Stephen J. Turnbull wrote:
>  > How do you limit it to just addresses with DMARC problems?  There's no
>  > benefit to doing it to everyone.
> 
> Probably a hard-coded regexp (or list thereof) matching against the
> address in "From".  The OP has a special situation where he's got lots
> of local mods to Mailman, so it's not convenient to upgrade.  So he
> wants a quick and dirty approach, and he also knows enough Python that
> he can work out the details for himself.

Quick 'n dirty indeed!  We keep discovering domain names publishing
DMARC p=reject policies, and there's no guarantee that others won't
follow suit.  So far, AOL and Yahoo are the only public ESPs we know of,
but that could change.  The only guaranteed method is to query DNS.
Queries are cached, so repeated lookups of the same names should have a
fairly low latency.

I have a lot of mods to Mailman too.  Patching is easy using the gnu.org
diff and patch tools and can easily be scripted, although if your OP
hasn't kept diff records of his patches then this isn't an option.
> 
-- 
Lindsay Haisley       | "Everything works if you let it"
FMP Computer Services |
512-259-1190          |          --- The Roadie
http://www.fmp.com    |


From stephen at xemacs.org  Sun May 18 02:00:53 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 18 May 2014 09:00:53 +0900
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <alpine.BSF.2.00.1405171313250.23413@joyce.lan>
References: <5376E856.9020200@msapiro.net>
 <20140517141226.99065.qmail@joyce.lan>
 <87wqdkbbtu.fsf@uwakimon.sk.tsukuba.ac.jp>
 <alpine.BSF.2.00.1405171313250.23413@joyce.lan>
Message-ID: <87r43sklai.fsf@uwakimon.sk.tsukuba.ac.jp>

John R Levine writes:

 > Isn't there code in 2.18 to check for DMARC problems to catch messages on 
 > the way in?  I'd think you could adapt that.

I'm not sure what "on the way in" means, but 2.1.18 does have code to
catch a p=reject record and handle it just once per post.  The OP
can't use 2.1.18.  It also requires a 3rd-party module (easily
available on PyPI, and probably that doesn't bother the OP, but it
does bother some people).

I think I suggested a cache in Mailman earlier, but as Mark suggested
a local caching nameserver might be just as effective, since mail
handling nowadays requires a ton of DNS lookups aside from DMARC.
Caches are tricky beasts to manage, especially when somebody else
might decide to invalidate them without telling you.





From stephen at xemacs.org  Sun May 18 02:08:14 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 18 May 2014 09:08:14 +0900
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <1400359204.13050.30.camel@pudina.fmp.com>
References: <5376E856.9020200@msapiro.net>
 <20140517141226.99065.qmail@joyce.lan>
 <87wqdkbbtu.fsf@uwakimon.sk.tsukuba.ac.jp>
 <1400359204.13050.30.camel@pudina.fmp.com>
Message-ID: <87ppjckky9.fsf@uwakimon.sk.tsukuba.ac.jp>

Lindsay Haisley writes:

 > I have a lot of mods to Mailman too.  Patching is easy using the gnu.org
 > diff and patch tools and can easily be scripted,

I'm sure the OP knows that, and may even have a bzr (or git) repo.
However, any change can require resolving conflicts, and some require
changing lots of list configs.  All of this may require testing with
your local mods.  That adds up to time, which not everybody has
*today*.  But DMARC needs to be fixed *yesterday*.

I'm not recommending that; 2.1.18-1 already has a better solution in
dmarc_moderate_action.  I'm just saying that if he's not willing to
upgrade to 2.1.18-1 yet, quick and dirty seems to be the way to go.


From rajeevs1992 at gmail.com  Tue May 20 03:15:45 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Tue, 20 May 2014 06:45:45 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <537A88E5.4080608@gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp> <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp> <5371EC9A.1030804@gmail.com>
 <537A88E5.4080608@gmail.com>
Message-ID: <537AACC1.9040604@gmail.com>

Hi,

I have pushed the revision 53, that has the following changes.

1. Refactoring as mentioned in the comments by Steve
2. Grouping of options and sub commands using subparsers
3. Changed the format of the command from *mmclient <scope> <action>*
     to *mmclient <action> <scope>*
4. Replaced ambiguous action *list* with *show*


The change mentioned in point 2 solves many/most of the problems
raised during the recent discussions.

The options are now paired with their corresponding sub commands.
The implications of this change is that the extra options/arguments
raise command syntax errors, which were ignored earlier. Further,
an great extent of leniency can be allowed in the command format.
To be specific, the parser can support the commands

*mmclient create list --list list1 --domain domain.org*

and

*mmclient set --key "foo" --value "bar"*

at the same time. Note that the second command has not specified a
scope.(PS: scope as in list, domain, user etc, and not site-wide, domain 
etc).

The change also prevents argparse from generating a dictionary
with all possible options (which may be huge).

The agreed change to add a positional argument for the *name*
(commands of the form *mmclient create list list at domain.org*)
has been postponed to the next revision, as this revision mostly
contains code refactoring.

http://bazaar.launchpad.net/~rajeevs1992/mailman.client/mailmancli/revision/53



From rajeevs1992 at gmail.com  Wed May 21 02:25:11 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Wed, 21 May 2014 05:55:11 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <537AACC1.9040604@gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp> <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp> <5371EC9A.1030804@gmail.com>
 <537A88E5.4080608@gmail.com> <537AACC1.9040604@gmail.com>
Message-ID: <537BF267.8070605@gmail.com>

As discussed, I have completed the r54, that does the following

1. The arguments will be specified as positional arguments
wherever necessary and possible. The ideal way to use
positional arguments here is "if the argument to the command
is the name of an instance the scope, use positional args else use
optional args"

E.g,
To filter mailing lists based on domains, do

  mmclient show list --domain domain.org

and not

   mmclient show list domain.org

To create a list, do

   mmclient create list list at domain.org

instead of

   mmclient create list --list list --domain domain.org

2. Added a no-header option to the detailed listing so
that it becomes easier to pip the output.

I will be making the following changes for r55

1.Delete

Quite a straightforward one.

   delete list list at domain.org
   delete domain domain.org

2. Begin the third `scope` user and add methods like create,
delete and list.

   create user foo at bar.com --password PASS --name NAME
   list user [--verbose] [--no-header]

The user class would be built and managed in the same way
as other classes.

r55 would be an easy one. Once the above changes are approved,
r55 can be completed in a day.

From iane at sussex.ac.uk  Wed May 21 18:14:59 2014
From: iane at sussex.ac.uk (Ian Eiloart)
Date: Wed, 21 May 2014 16:14:59 +0000
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <20140517030827.13231.qmail@joyce.lan>
References: <20140517030827.13231.qmail@joyce.lan>
Message-ID: <BE9B9093-19FD-4290-B95E-479556AAEAD1@sussex.ac.uk>


On 17 May 2014, at 04:08, John Levine <johnl at taugh.com> wrote:

> Everyone I know who's tried to do spam filtering by SMTP callbacks to
> verify sender addresses has stopped,

Not me.

> for the dual reasons that it
> doesn't work, and it's abusive.

It is helpful, we get almost no complaints. Complaints are usually resolved by the fixing of the third party sender. NOREPLY addresses may be rejected, but not usually until after DATA: at which point we?ve already dropped the callout connection. 

And, it?s not abusive if appropriate SPF checks are done first: obviously, you don?t do the callout if you get an SPF fail. A callout with an SPF pass isn?t abusive: if the domain sent me an email, then it should be able to handle a callout. 

If there?s no SPF record, then the domain isn?t protected: publishing SPF records will protect against excessive callouts. For SPF neutral or softfail results, a similar argument applies: the domain isn?t properly protected. 

-- 
Ian Eiloart
Postmaster, University of Sussex
+44 (0) 1273 87-3148


From johnl at taugh.com  Wed May 21 19:27:32 2014
From: johnl at taugh.com (John R Levine)
Date: 21 May 2014 13:27:32 -0400
Subject: [Mailman-Developers] callbacks,
 Fixing DMARC problems with .invalid munge
In-Reply-To: <BE9B9093-19FD-4290-B95E-479556AAEAD1@sussex.ac.uk>
References: <20140517030827.13231.qmail@joyce.lan>
 <BE9B9093-19FD-4290-B95E-479556AAEAD1@sussex.ac.uk>
Message-ID: <alpine.BSF.2.00.1405211326050.20064@joyce.lan>

> And, it?s not abusive if appropriate SPF checks are done first: obviously, you don?t do the callout if you get an SPF fail. A callout with an SPF pass isn?t abusive: if the domain sent me an email, then it should be able to handle a callout.

Let me just say that a lot of people at large ISPs disagree with you.

Considering that about 90% of mail is spam, and about 99% of spam has 
forged return addresses, in practice all of your callbacks are annoying 
people who had nothing to do with the spam.  If you're not getting 
complaints, it's because you're too small to be worth blocking.

Regards,
John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

From tanstaafl at libertytrek.org  Wed May 21 21:06:36 2014
From: tanstaafl at libertytrek.org (Tanstaafl)
Date: Wed, 21 May 2014 15:06:36 -0400
Subject: [Mailman-Developers] Fixing DMARC problems with .invalid munge
In-Reply-To: <BE9B9093-19FD-4290-B95E-479556AAEAD1@sussex.ac.uk>
References: <20140517030827.13231.qmail@joyce.lan>
 <BE9B9093-19FD-4290-B95E-479556AAEAD1@sussex.ac.uk>
Message-ID: <537CF93C.6080407@libertytrek.org>

On 5/21/2014 12:14 PM, Ian Eiloart <iane at sussex.ac.uk> wrote:
> And, it?s not abusive if appropriate SPF checks are done first:
> obviously, you don?t do the callout if you get an SPF fail. A callout
> with an SPF pass isn?t abusive: if the domain sent me an email, then
> it should be able to handle a callout.

You are wrong.

Minimizing the use of SAV is always best if you are going to use it, but 
you should only use it with sites that you have an understanding and 
agreement in place where they formally tell you its ok.

Anything else is, indeed, abusive, regardless of your personal opinion 
on the matter.

From varunsharmalive at gmail.com  Fri May 23 19:41:40 2014
From: varunsharmalive at gmail.com (varun sharma)
Date: Fri, 23 May 2014 23:11:40 +0530
Subject: [Mailman-Developers] GSOC 2014: CI tool for the Mailman suite
 and postorius improvements
Message-ID: <CAPkTf0JrsGa-wrjmA5FrBCPnazx2j0_8b+zM+K70URkdjEht7w@mail.gmail.com>

Sorry for being out of touch for so long, i was having exams during
community bonding period,so i couldn't spare time for discussions. But
after doing 3 irc meetings with my mentors,i have now pretty clear
understanding on how to proceed with my project but in order to get further
suggestions from community, i am posting all the details of my project:

1. CI tool for Mailman Suite (name yet to decide):
The first part that i will be doing for next 4 weeks is CI tool for Mailman
Suite. I have already started work on that and created a launchpad
repository at https://code.launchpad.net/~varun/mailman/ci_tool. I will try
to adhere to my timeline, so by the end of this week i'll be finishing with
all the unit tests of ci-tool. Right now i've written the rough
implementation of pull and test commands using python's optparse module
http://bazaar.launchpad.net/~varun/mailman/ci_tool/view/head:/ci-tool:

ci-tool -p all
(will pull all the projects which includes mailman, mailman.client,
postorius, postorius_standalone, hyperkitty and kittystore)

ci-tool -p hyperkitty
(for pulling a particular project)

ci-tool -t all
(will run all the unit tests from all the projects)

ci-tool -t postorius
(run tests specifically)

The tool is running fine but buildbot is unable to detect the failed tests
in case of running them from ci-tool, i guess i need to write plugin for
nose2 to fix this problem. Link to one of such build is
http://buildbot.sharmalabs.com/builders/runtests/builds/116
The basic commands that i think it should have are 'pull', 'test' and 'try'
but suggestions are welcomed for any new addition.

I request all the creative (and non-creative) members of the community to
suggest some cool names for the ci-tool :)


2. The UI Testing Framework for Postorius:
I have also started working on UI framework and after discussing the
possible testing frameworks that we can use in that, we decided to go with
selenium. I wrote 5 basis tests for testing the ui of postorius with
selenium webdriver (
http://bazaar.launchpad.net/~varun/mailman/ci_tool/view/head:/test_postorius.py)
and the i am yet to expand it to hyperkitty. The idea is to include this
framework into ci-tool as an optional feature.

I will be very happy to hear the feedback from the community, so that i'll
be able to make changes to my project accordingly and at right time.
Many thanks to my mentors Florian, Steve and Sneha for helping me in making
this project feasible.

@abhilsh: Sorry, i was really busy during that time,so couldn't replay
accordingly and i'll keep that in mind for future discussions.
@barry: Thanks barry, I'll make sure the test suite works fine with all the
major MTAs, DBMSs and *nix platforms.


Thanks
Varun

From raj.abhilash1 at gmail.com  Tue May 27 08:57:24 2014
From: raj.abhilash1 at gmail.com (Abhilash Raj)
Date: Tue, 27 May 2014 12:27:24 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <537BF267.8070605@gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com> <537A88E5.4080608@gmail.com>
 <537AACC1.9040604@gmail.com> <537BF267.8070605@gmail.com>
Message-ID: <20140527122724.618d5531@angel>

Rajeev S <rajeevs1992 at gmail.com> wrote:

> As discussed, I have completed the r54, that does the following
> 
> 1. The arguments will be specified as positional arguments
> wherever necessary and possible. The ideal way to use
> positional arguments here is "if the argument to the command
> is the name of an instance the scope, use positional args else use
> optional args"
> 
> E.g,
> To filter mailing lists based on domains, do
> 
>   mmclient show list --domain domain.org
> 
> and not
> 
>    mmclient show list domain.org
> 
> To create a list, do
> 
>    mmclient create list list at domain.org
> 
> instead of
> 
>    mmclient create list --list list --domain domain.org

Both of these options looks clean to me as a user. So if I am right you
have completed

* Listing of domains
* Listing of mailing lists(filtered by domain also)
* Creating domains
* Creating mailing lists

Right?

Since this tool is meant for the users, you should write better
documentation. Like in using.txt#39 what does long listing mean? What
does `--verbose` do for listing a domain? Or even for listing all the
lists?

As I understand using.txt is more of a command reference than documentation?

Are there only these two options for lists and domains? What about
editing any list? Adding and removing user roles will be possible after
you have create the `user` scope but editing of other parameters can be
done?

Also maybe you can try making your tool a little more smart? Like lets
say I try to create a list abhilash at raj.com and there is no domain
raj.com in the database, so instead of just showing error maybe you can
ask the user:

"The domain raj.com does not exist, Do you want to create one? [y/n]"

Just adding all the options using argparse is really not a very tough
job (and with your pace, it is definitely not going to last more than
one month ;-). Try to put some more thought to how you can make this
tool more intuitive for the end user.

> 2. Added a no-header option to the detailed listing so
> that it becomes easier to pip the output.
> 
> I will be making the following changes for r55
> 
> 1.Delete
> 
> Quite a straightforward one.
> 
>    delete list list at domain.org
>    delete domain domain.org

What I belive is that deleting a domain should not strightforward, any
destructive command should not be. Would it not be nice to prompt the
user once before deleting? as in

"There are 100 lists associated with this domain. Once deleted you
cannot undo it, Do you really want to delete xxx at yyy.com?"[y/n]

Or maybe it could schedule a deletion after a pre-defined time with a
reasonable default lets say "1 Day"? And for an urgency(to delete) there
could be --force argument?

> 2. Begin the third `scope` user and add methods like create,
> delete and list.
> 
>    create user foo at bar.com --password PASS --name NAME
>    list user [--verbose] [--no-header]
> 
> The user class would be built and managed in the same way
> as other classes.
> 
> r55 would be an easy one. Once the above changes are approved,
> r55 can be completed in a day.

I roughly went through your code, and I have a few more points:

* You should write tests, before writing more code. Infact you should
  follow TDD (ofcourse if you are comfortable doing it) since the
  outcome of your commands is less likely to change, even though your
  code does. 
* The commit messages should be written in a language that tells a
  reader "What this commit does?". Like "Adds documentation file
  using.txt". Instad of answering "What I have done in this commit?"
  Although it is my point of view, AFAIK it is also preferred by many
  other developers.(See commit messages of barry on lp:mailman)
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers at python.org
> https://mail.python.org/mailman/listinfo/mailman-developers
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/raj.abhilash1%40gmail.com
> 
> Security Policy: http://wiki.list.org/x/QIA9



-- 
thanks,
Abhilash Raj

From rajeevs1992 at gmail.com  Wed May 28 00:50:14 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Wed, 28 May 2014 04:20:14 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <20140527122724.618d5531@angel>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp> <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp> <5371EC9A.1030804@gmail.com>
 <537A88E5.4080608@gmail.com> <537AACC1.9040604@gmail.com>
 <537BF267.8070605@gmail.com> <20140527122724.618d5531@angel>
Message-ID: <538516A6.6080802@gmail.com>

On Tuesday 27 May 2014 12:27 PM, Abhilash Raj wrote:
> Rajeev S <rajeevs1992 at gmail.com> wrote:
>
>
> Both of these options looks clean to me as a user. So if I am right you
> have completed
>
> * Listing of domains
> * Listing of mailing lists(filtered by domain also)
> * Creating domains
> * Creating mailing lists
>
> Right?
>
> Since this tool is meant for the users, you should write better
> documentation. Like in using.txt#39 what does long listing mean? What
> does `--verbose` do for listing a domain? Or even for listing all the
> lists?
>
> As I understand using.txt is more of a command reference than documentation?
>
> Are there only these two options for lists and domains? What about
> editing any list? Adding and removing user roles will be possible after
> you have create the `user` scope but editing of other parameters can be
> done?

No, there are more options for lists and domains, many of which
are dependent on the users. Some of the other options are list-members,
edit and export(to CSV)/import. Hence, these would be easier to build
after a basic user class is available.

> Also maybe you can try making your tool a little more smart? Like lets
> say I try to create a list abhilash at raj.com and there is no domain
> raj.com in the database, so instead of just showing error maybe you can
> ask the user:
>
> "The domain raj.com does not exist, Do you want to create one? [y/n]"
>
> Just adding all the options using argparse is really not a very tough
> job (and with your pace, it is definitely not going to last more than
> one month ;-). Try to put some more thought to how you can make this
> tool more intuitive for the end user.
> What I belive is that deleting a domain should not strightforward, any
> destructive command should not be. Would it not be nice to prompt the
> user once before deleting? as in
>
> "There are 100 lists associated with this domain. Once deleted you
> cannot undo it, Do you really want to delete xxx at yyy.com?"[y/n]
>
> Or maybe it could schedule a deletion after a pre-defined time with a
> reasonable default lets say "1 Day"? And for an urgency(to delete) there
> could be --force argument?
>
> I roughly went through your code, and I have a few more points:
>
> * You should write tests, before writing more code. Infact you should
>    follow TDD (ofcourse if you are comfortable doing it) since the
>    outcome of your commands is less likely to change, even though your
>    code does.
> * The commit messages should be written in a language that tells a
>    reader "What this commit does?". Like "Adds documentation file
>    using.txt". Instad of answering "What I have done in this commit?"
>    Although it is my point of view, AFAIK it is also preferred by many
>    other developers.(See commit messages of barry on lp:mailman)

Everything else seems fine. Will apply the necessary changes.



From barry at list.org  Wed May 28 21:50:30 2014
From: barry at list.org (Barry Warsaw)
Date: Wed, 28 May 2014 15:50:30 -0400
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <20140527122724.618d5531@angel>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com> <537A88E5.4080608@gmail.com>
 <537AACC1.9040604@gmail.com> <537BF267.8070605@gmail.com>
 <20140527122724.618d5531@angel>
Message-ID: <20140528155030.081e0b41@anarchist.wooz.org>

Although I haven't had time to go through the code, I'm liking what I'm seeing
here on the mailing list.  Just a quick comment.

On May 27, 2014, at 12:27 PM, Abhilash Raj wrote:

>Since this tool is meant for the users, you should write better
>documentation. Like in using.txt

Ideally, because this is a command line tool aimed and users, there should be
a manpage for mmclient.  Fortunately, it's *really* easy to write manpages
in reST with Sphinx.

http://sphinx-doc.org/builders.html#sphinx.builders.manpage.ManualPageBuilder

And since examples never hurt:

http://tinyurl.com/pwa2dfh

Cheers,
-Barry

From barry at list.org  Wed May 28 21:53:57 2014
From: barry at list.org (Barry Warsaw)
Date: Wed, 28 May 2014 15:53:57 -0400
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <87ppjhdp2j.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com>
 <20140513094229.3ce30980@anarchist.wooz.org>
 <87ppjhdp2j.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20140528155357.4eee4c18@anarchist.wooz.org>

On May 14, 2014, at 12:19 PM, Stephen J. Turnbull wrote:

>Barry Warsaw writes:
> > I just want it to be consistent, easily described, and easily understood by
> > users.  If it makes sense for the mmclient CLI to different from the
> > shell-access mailman command, then we at least need to be "translatable"
> > between the two.
>
>What do you mean by "shell-access mailman command"?
>src/mailman/bin/mailman.py?

Yes, as much as makes sense.  When installed into a virtual environment,
you'll have `<venv>/bin/mailman` with lots of subcommands, many of which are
similar to mmclient.  It may not be possible to give them identical
signatures, and that's fine, but it would be great if when someone learns the
commands for one, the other will not break their intuition, especially for
things that are "close".

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20140528/3ba80a60/attachment.sig>

From rajeevs1992 at gmail.com  Thu May 29 08:05:10 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Thu, 29 May 2014 11:35:10 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <20140528155030.081e0b41@anarchist.wooz.org>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp> <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com> <537A88E5.4080608@gmail.com>
 <537AACC1.9040604@gmail.com>
 <537BF267.8070605@gmail.com> <20140527122724.618d5531@angel>
 <20140528155030.081e0b41@anarchist.wooz.org>
Message-ID: <CAPq4ZO+9i70AETo254bwNgO+==8mPQddkc9SXkVzXMF3qKJmdg@mail.gmail.com>

Hi Barry,


Although I haven't had time to go through the code, I'm liking what I'm
> seeing
> here on the mailing list.  Just a quick comment.
> [...]
> Ideally, because this is a command line tool aimed and users, there should
> be
> a manpage for mmclient.  Fortunately, it's *really* easy to write manpages
> in reST with Sphinx.
>
>
Sure it is necessary. And writing the man page is mentioned in my proposal
as a project
deliverable.


http://sphinx-doc.org/builders.html#sphinx.builders.manpage.ManualPageBuilder
>
> And since examples never hurt:
>
> http://tinyurl.com/pwa2dfh
>
>
Thanks for the links.

From stephen at xemacs.org  Thu May 29 15:30:25 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 29 May 2014 22:30:25 +0900
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <538516A6.6080802@gmail.com>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <20140501214322.7637aa17@limelight.wooz.org>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com> <537A88E5.4080608@gmail.com>
 <537AACC1.9040604@gmail.com> <537BF267.8070605@gmail.com>
 <20140527122724.618d5531@angel> <538516A6.6080802@gmail.com>
Message-ID: <87r43c7lvy.fsf@uwakimon.sk.tsukuba.ac.jp>

Rajeev S writes:

 > > Also maybe you can try making your tool a little more smart? Like lets
 > > say I try to create a list abhilash at raj.com and there is no domain
 > > raj.com in the database, so instead of just showing error maybe you can
 > > ask the user:
 > >
 > > "The domain raj.com does not exist, Do you want to create one? [y/n]"

I'm not sure I like this approach.  Creating a domain should be a
heavyweight operation, and eventually should include a bunch of sanity
checks, like existence of domains and MX records.  I have visions
(nightmares) of users coming to us saying

    User: I said yes, but mail never arrives.
    Dev:  ....  Oh, is there a proper entry in the DNS?
    User: Doesn't Mailman create the domain?

 > > Or maybe it could schedule a deletion after a pre-defined time with a
 > > reasonable default lets say "1 Day"? And for an urgency(to delete) there
 > > could be --force argument?

Deleting a list should be immediate, but I agree it should be confirmed.


From p at sys4.de  Thu May 29 17:20:07 2014
From: p at sys4.de (Patrick Ben Koetter)
Date: Thu, 29 May 2014 17:20:07 +0200
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <87r43c7lvy.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com> <537A88E5.4080608@gmail.com>
 <537AACC1.9040604@gmail.com> <537BF267.8070605@gmail.com>
 <20140527122724.618d5531@angel> <538516A6.6080802@gmail.com>
 <87r43c7lvy.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20140529152007.GA22170@sys4.de>

* Stephen J. Turnbull <stephen at xemacs.org>:
> Rajeev S writes:
> 
>  > > Also maybe you can try making your tool a little more smart? Like lets
>  > > say I try to create a list abhilash at raj.com and there is no domain
>  > > raj.com in the database, so instead of just showing error maybe you can
>  > > ask the user:
>  > >
>  > > "The domain raj.com does not exist, Do you want to create one? [y/n]"
> 
> I'm not sure I like this approach.  Creating a domain should be a
> heavyweight operation, and eventually should include a bunch of sanity
> checks, like existence of domains and MX records.  I have visions
> (nightmares) of users coming to us saying
> 
>     User: I said yes, but mail never arrives.
>     Dev:  ....  Oh, is there a proper entry in the DNS?
>     User: Doesn't Mailman create the domain?

I doubt anyone that igorant of e-mail and how it works will ever make it to
the MM3 command line client, but yes, such cases do exist. We should have
public pillory that receives name, mail and date, whenever someone confirms
the above question. ;)

However I think the use case "prepare Mailman to handle mail for a domain it
doesn't handle mail for yet" exists and we should find a way to deal with it.

Perhaps we could improve this, if we used better wording that doesn't lead the
operator to believe Mailman will connect a domain, setup DNS, do all the other
foo and finally configure it self to handle mailing lists for that domain?

>  > > Or maybe it could schedule a deletion after a pre-defined time with a
>  > > reasonable default lets say "1 Day"? And for an urgency(to delete) there

Deleting a list/domain requires an (internal) scheduler. Does Mailman have
one? A broom job that can be called via cron?


>  > > could be --force argument?
> 
> Deleting a list should be immediate, but I agree it should be confirmed.

... and it should be possible to pass the confirmation in the command to make
it useful in scripts.

p at rick


-- 
[*] sys4 AG
 
https://sys4.de, +49 (89) 30 90 46 64
Franziskanerstra?e 15, 81669 M?nchen
 
Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
 

From barry at list.org  Thu May 29 17:30:19 2014
From: barry at list.org (Barry Warsaw)
Date: Thu, 29 May 2014 11:30:19 -0400
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <20140529152007.GA22170@sys4.de>
References: <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com> <537A88E5.4080608@gmail.com>
 <537AACC1.9040604@gmail.com> <537BF267.8070605@gmail.com>
 <20140527122724.618d5531@angel> <538516A6.6080802@gmail.com>
 <87r43c7lvy.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140529152007.GA22170@sys4.de>
Message-ID: <20140529113019.358c0391@anarchist.wooz.org>

On May 29, 2014, at 05:20 PM, Patrick Ben Koetter wrote:

>Deleting a list/domain requires an (internal) scheduler. Does Mailman have
>one? A broom job that can be called via cron?

Sort of, but the way these are handled currently are individual scripts that
each have to be added to cron.

>... and it should be possible to pass the confirmation in the command to make
>it useful in scripts.

Many such scripts have a --yes option to pre-confirm the action.

-Barry

From stephen at xemacs.org  Thu May 29 20:41:45 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 30 May 2014 03:41:45 +0900
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <20140529152007.GA22170@sys4.de>
References: <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com> <537A88E5.4080608@gmail.com>
 <537AACC1.9040604@gmail.com> <537BF267.8070605@gmail.com>
 <20140527122724.618d5531@angel> <538516A6.6080802@gmail.com>
 <87r43c7lvy.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140529152007.GA22170@sys4.de>
Message-ID: <87egzc77h2.fsf@uwakimon.sk.tsukuba.ac.jp>

Patrick Ben Koetter writes:

 > I doubt anyone that igorant of e-mail and how it works will ever
 > make it to the MM3 command line client, but yes, such cases do
 > exist.

I think they're actually likely to be reasonably common.

 > However I think the use case "prepare Mailman to handle mail for a
 > domain it doesn't handle mail for yet" exists and we should find a
 > way to deal with it.

Indeed.  I'm suggesting that the routine could check the DNS and a few
other things, and present the list operator with a TODO list.  Sortof
like check_perms.

 > > Deleting a list should be immediate, but I agree it should be confirmed.
 > 
 > ... and it should be possible to pass the confirmation in the command to make
 > it useful in scripts.

Ooh, that is a very good point.  Bessides "possible," the syntax for
doing that should be regular.

I wonder about distinguishing "yes to this prompt" and "yes to all"?
Probably scripts should not be using commands that need multiple
confirmations, though.


From barry at list.org  Thu May 29 20:50:27 2014
From: barry at list.org (Barry Warsaw)
Date: Thu, 29 May 2014 14:50:27 -0400
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <87egzc77h2.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com> <537A88E5.4080608@gmail.com>
 <537AACC1.9040604@gmail.com> <537BF267.8070605@gmail.com>
 <20140527122724.618d5531@angel> <538516A6.6080802@gmail.com>
 <87r43c7lvy.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140529152007.GA22170@sys4.de>
 <87egzc77h2.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20140529145027.050e3068@anarchist.wooz.org>

On May 30, 2014, at 03:41 AM, Stephen J. Turnbull wrote:

>Probably scripts should not be using commands that need multiple
>confirmations, though.

Agreed!
-Barry

From p at sys4.de  Thu May 29 21:16:33 2014
From: p at sys4.de (Patrick Ben Koetter)
Date: Thu, 29 May 2014 21:16:33 +0200
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <87egzc77h2.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com> <537A88E5.4080608@gmail.com>
 <537AACC1.9040604@gmail.com> <537BF267.8070605@gmail.com>
 <20140527122724.618d5531@angel> <538516A6.6080802@gmail.com>
 <87r43c7lvy.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140529152007.GA22170@sys4.de>
 <87egzc77h2.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20140529191633.GD22170@sys4.de>

* Stephen J. Turnbull <stephen at xemacs.org>:
> Patrick Ben Koetter writes:
> 
>  > I doubt anyone that igorant of e-mail and how it works will ever
>  > make it to the MM3 command line client, but yes, such cases do
>  > exist.
> 
> I think they're actually likely to be reasonably common.
> 
>  > However I think the use case "prepare Mailman to handle mail for a
>  > domain it doesn't handle mail for yet" exists and we should find a
>  > way to deal with it.
> 
> Indeed.  I'm suggesting that the routine could check the DNS and a few
> other things, and present the list operator with a TODO list.  Sortof
> like check_perms.

Where would you start with the TODO list? Where would you end?


>  > > Deleting a list should be immediate, but I agree it should be confirmed.
>  > 
>  > ... and it should be possible to pass the confirmation in the command to make
>  > it useful in scripts.
> 
> Ooh, that is a very good point.  Bessides "possible," the syntax for
> doing that should be regular.


I'm hopping in on this and I certainly missed most of the discussion on the
command line client: Do we have interface guidelines for commands? Shortopts,
longopts, interaction, non-interactive behaviour/requirements?

> I wonder about distinguishing "yes to this prompt" and "yes to all"?

Me too. 

> Probably scripts should not be using commands that need multiple
> confirmations, though.

Hmmm, Unix: Do one thing. Do it well. ?
 

-- 
[*] sys4 AG
 
https://sys4.de, +49 (89) 30 90 46 64
Franziskanerstra?e 15, 81669 M?nchen
 
Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
 

From rajeevs1992 at gmail.com  Fri May 30 14:42:18 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Fri, 30 May 2014 18:12:18 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <20140529191633.GD22170@sys4.de>
References: <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com>
 <537A88E5.4080608@gmail.com> <537AACC1.9040604@gmail.com>
 <537BF267.8070605@gmail.com>
 <20140527122724.618d5531@angel> <538516A6.6080802@gmail.com>
 <87r43c7lvy.fsf@uwakimon.sk.tsukuba.ac.jp> <20140529152007.GA22170@sys4.de>
 <87egzc77h2.fsf@uwakimon.sk.tsukuba.ac.jp> <20140529191633.GD22170@sys4.de>
Message-ID: <CAPq4ZOL9X9e1EsJbt46isbRNmd1c1ys=x_EeDsS62FnM3zsVPQ@mail.gmail.com>

Hi,

I have made more additions to the CLI, like the `delete` command and the
`user` scope.

Following is the list of changes brought about

1.Introduce User scope

Actions supported by User scope currently are create, show and delete.
show users, like other scopes, supports verbose and no-header flags. However
the detailed listing of users is not quite effective as most of the fields
are lists
rather than single valued attributes. The non detailed version prints one
of the
email ids of each user. The show list command takes an optional argument
list name that gives the users who are members of the list.


2.Introduce a cli/lib directory, to store procedures common across the
application.

The directory is to hold the procedures and classes that are reused among
scripts.
It currently holds a class that is used to colorize the output printed on
the terminal.

3.Updated and improved docs/using.txt. It was obsolete in r54.


4.Added delete action for all objects

The action is confirmed by a question to user, which can be overridden by a
--yes switch.

I will be committing and pushing r55 tonight (30/05/2014 around 16:30 UST,
I am away from
the computer right now).
____

As mentioned by Steve and agreed by Barry, I too believe that, in the
scenario of the CLI
multiple confirmations won't be necessary. In a situation in which such
multiple confirmations
come necessary, I believe that the commands will be better if the
confirmations are replaced
by switches.

I agree that an override for the confirmation message is necessary so that
the CLI
commands can be used in scripts and I would like to follow Barry's
suggestion of using
a `--yes` or `--force` switch. (Isn't --force more common than --yes, as in
`rm -f` ?)

About the create domain suggestion, Is the follow up question on `*create
domain*`, upon missing domain,
such a bad idea? A user already can `add a domain` by using the web
interface. If that is not ambiguous,
neither is this, right?

However I see this issue, if a newbie user types in an email address in
place of the list name, probably
on misreading the command format, he would end up creating a domain for his
mail provider.

*$./mmclient create list --help*
*Creates the list list at domain.org <list at domain.org>*
 *$./mmclient create list user at gmail.com <user at gmail.com>*
*The domain gmail.com <http://gmail.com> does not exist.Create one?[y/n] y*
 *$*

@Abhilash

I have written unittests for methods like connect and create, but have to
refine them more.
Hence the tests won't come up in this revision.

Also, I would like to ask how the tests should be triggered. Should
there be a `mmclient test` with a set of possible switches like `all`
,`domain`, `list`, `user` etc?
Or should the customary `python setup.py test` be used?


I also propose to build a a *`describe user`* command that gives a detailed
profile of the user.
Verbose listing using tables is not so effective users as most of the
attributes are lists.

Thanks!

From raj.abhilash1 at gmail.com  Fri May 30 15:24:17 2014
From: raj.abhilash1 at gmail.com (Abhilash Raj)
Date: Fri, 30 May 2014 18:54:17 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <87r43c7lvy.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAPq4ZO+1A00UUmopp4OrKQvUBLo1xV5w_wAxEDL6x6f3oLMWqw@mail.gmail.com>
 <CAPq4ZO+_bUb8RE95XKXBjqKLT-ReNH4diykwmmvtxPakAdGppg@mail.gmail.com>
 <CAPq4ZOLC3Zarskn39MavozNxECKb8ddmXedCovQbh-Vvq7NL3A@mail.gmail.com>
 <CAFMGiz9EZoB1CyeR2wuGsOybSfoZ1HqMcMjP6gYzE7D108kNUw@mail.gmail.com>
 <87bnvfkmct.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CAFMGiz8C-yHRqzXqNZzqW7U3MMgwavOWjdnT3PxZfCzuum10fQ@mail.gmail.com>
 <20140503132132.33b88006@anarchist.wooz.org>
 <CAFMGiz9Gp8qU0P5K3sD_Ytren2R2h=R2T6cWcpz-aod2960gyw@mail.gmail.com>
 <CAPq4ZO+1_VE=9KLk6J9cjWJtyYbxxQJP3mFi45-_vYdetkjDbw@mail.gmail.com>
 <87lhuahmqv.fsf@uwakimon.sk.tsukuba.ac.jp>
 <536FAE65.6030002@gmail.com>
 <20140512114211.1c34421d@anarchist.wooz.org>
 <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com> <537A88E5.4080608@gmail.com>
 <537AACC1.9040604@gmail.com> <537BF267.8070605@gmail.com>
 <20140527122724.618d5531@angel> <538516A6.6080802@gmail.com>
 <87r43c7lvy.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20140530185417.2f101b63@angel>

On Thu, 29 May 2014 22:30:25 +0900
"Stephen J. Turnbull" <stephen at xemacs.org> wrote:

>  > > Also maybe you can try making your tool a little more smart? Like lets
>  > > say I try to create a list abhilash at raj.com and there is no domain
>  > > raj.com in the database, so instead of just showing error maybe you can
>  > > ask the user:
>  > >
>  > > "The domain raj.com does not exist, Do you want to create one? [y/n]"
> 
> I'm not sure I like this approach.  Creating a domain should be a
> heavyweight operation, and eventually should include a bunch of sanity
> checks, like existence of domains and MX records.  I have visions
> (nightmares) of users coming to us saying
> 
>     User: I said yes, but mail never arrives.
>     Dev:  ....  Oh, is there a proper entry in the DNS?
>     User: Doesn't Mailman create the domain?

I agree to your point that it should be a heavyweight operation, I
guess i was not able to express myself. I don't know if this CLI client
can add domains to mailman(I mean add domain to mailman before and not
*create*), there could be a prompt saying "This mailman installation
is not configured to use this domain, do you want to do it now?" and
then maybe it will walk the user through the "usual" process of adding
a new domain? 

> 
>  > > Or maybe it could schedule a deletion after a pre-defined time with a
>  > > reasonable default lets say "1 Day"? And for an urgency(to delete) there
>  > > could be --force argument?
> 
> Deleting a list should be immediate, but I agree it should be confirmed.
> 



-- 
thanks,
Abhilash Raj

From raj.abhilash1 at gmail.com  Fri May 30 15:39:23 2014
From: raj.abhilash1 at gmail.com (Abhilash Raj)
Date: Fri, 30 May 2014 19:09:23 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <CAPq4ZOL9X9e1EsJbt46isbRNmd1c1ys=x_EeDsS62FnM3zsVPQ@mail.gmail.com>
References: <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com> <537A88E5.4080608@gmail.com>
 <537AACC1.9040604@gmail.com> <537BF267.8070605@gmail.com>
 <20140527122724.618d5531@angel> <538516A6.6080802@gmail.com>
 <87r43c7lvy.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140529152007.GA22170@sys4.de>
 <87egzc77h2.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140529191633.GD22170@sys4.de>
 <CAPq4ZOL9X9e1EsJbt46isbRNmd1c1ys=x_EeDsS62FnM3zsVPQ@mail.gmail.com>
Message-ID: <20140530190923.05ffaeda@angel>

On Fri, 30 May 2014 18:12:18 +0530
Rajeev S <rajeevs1992 at gmail.com> wrote:

> I agree that an override for the confirmation message is necessary so that
> the CLI
> commands can be used in scripts and I would like to follow Barry's
> suggestion of using
> a `--yes` or `--force` switch. (Isn't --force more common than --yes, as in
> `rm -f` ?)

No, as I think, `--yes` and `--force` or `-f` have different use cases.
For conformations `--yes` is used most of the times, it says 'hey I
know you are going to ask this question and my answer would be yes'.
While `--force` is to actually *force* any operation that cannot be
completed otherwise.

> About the create domain suggestion, Is the follow up question on `*create
> domain*`, upon missing domain,
> such a bad idea? A user already can `add a domain` by using the web
> interface. If that is not ambiguous,
> neither is this, right?
> 
> However I see this issue, if a newbie user types in an email address in
> place of the list name, probably
> on misreading the command format, he would end up creating a domain for his
> mail provider.
> 
> *$./mmclient create list --help*
> *Creates the list list at domain.org <list at domain.org>*
>  *$./mmclient create list user at gmail.com <user at gmail.com>*
> *The domain gmail.com <http://gmail.com> does not exist.Create one?[y/n] y*
>  *$*
> 
> @Abhilash
> 
> I have written unittests for methods like connect and create, but have to
> refine them more.
> Hence the tests won't come up in this revision.

Good to hear that. Did you give TDD a try?
 
> Also, I would like to ask how the tests should be triggered. Should
> there be a `mmclient test` with a set of possible switches like `all`
> ,`domain`, `list`, `user` etc?
> Or should the customary `python setup.py test` be used?

You can use `nose2` for that, like mailman does. It automatically
discovers all the tests in directories specified and runs them.

> 
> I also propose to build a a *`describe user`* command that gives a detailed
> profile of the user.
> Verbose listing using tables is not so effective users as most of the
> attributes are lists.

What detail profile are you talking about? Apart from addresses and
name(optional) does mailman store any other data about a user? You can
show the lists he is subscribed to, but `describe user` may not be an
appropriate command for that. Maybe you can add it in users scope
itself like
  `mmclient user a at b.org --list-subscriptions`
Others may have different thoughts though.

-- 
thanks,
Abhilash Raj

From raj.abhilash1 at gmail.com  Fri May 30 15:45:09 2014
From: raj.abhilash1 at gmail.com (Abhilash Raj)
Date: Fri, 30 May 2014 19:15:09 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <CAPq4ZOL9X9e1EsJbt46isbRNmd1c1ys=x_EeDsS62FnM3zsVPQ@mail.gmail.com>
References: <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com> <537A88E5.4080608@gmail.com>
 <537AACC1.9040604@gmail.com> <537BF267.8070605@gmail.com>
 <20140527122724.618d5531@angel> <538516A6.6080802@gmail.com>
 <87r43c7lvy.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140529152007.GA22170@sys4.de>
 <87egzc77h2.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20140529191633.GD22170@sys4.de>
 <CAPq4ZOL9X9e1EsJbt46isbRNmd1c1ys=x_EeDsS62FnM3zsVPQ@mail.gmail.com>
Message-ID: <20140530191509.00d5aae2@angel>

On Fri, 30 May 2014 18:12:18 +0530
Rajeev S <rajeevs1992 at gmail.com> wrote:

When you are not citing any context from previous posts, you should
start a new thread instead of re-posting on same old one. It makes it
difficult to find a specific post with this long thread.

-- 
thanks,
Abhilash Raj

From superuser at gmail.com  Fri May 30 19:35:12 2014
From: superuser at gmail.com (Murray S. Kucherawy)
Date: Fri, 30 May 2014 10:35:12 -0700
Subject: [Mailman-Developers] Thinking about list footers
Message-ID: <CAL0qLwa-6+5KbGtNuH768k7=J1WTfmtY=hZBOyp0RQOe85YqXA@mail.gmail.com>

Right now list footers are a common feature of Mailman and other MLMs.  The
typical way of doing this is to just append some content, probably after
"--", that indicates the location of archives and maybe some administrative
information (unsubscribe instructions, for example).

What's the expertise on the idea of adding footers in a new MIME text/plain
part rather than just bolting it onto the text as-is?  (Or is that already
done?)  What do MUAs generally do with multipart text/plain bodies?

And along those lines, do any MUAs do useful things with the various List-*
fields, other than permitting one to sort on them?

I'm brainstorming on some possible solutions to DMARC and DKIM matters, and
I don't want to reinvent any wheels.

Thanks,
-MSK

From barry at list.org  Fri May 30 21:21:37 2014
From: barry at list.org (Barry Warsaw)
Date: Fri, 30 May 2014 15:21:37 -0400
Subject: [Mailman-Developers] Thinking about list footers
In-Reply-To: <CAL0qLwa-6+5KbGtNuH768k7=J1WTfmtY=hZBOyp0RQOe85YqXA@mail.gmail.com>
References: <CAL0qLwa-6+5KbGtNuH768k7=J1WTfmtY=hZBOyp0RQOe85YqXA@mail.gmail.com>
Message-ID: <20140530152137.73a2a073@limelight.wooz.org>

I'm really sorry I haven't had time to catch up on the various DMARC threads,
but I'm hoping to do so early next week.

On May 30, 2014, at 10:35 AM, Murray S. Kucherawy wrote:

>What's the expertise on the idea of adding footers in a new MIME text/plain
>part rather than just bolting it onto the text as-is?  (Or is that already
>done?)  What do MUAs generally do with multipart text/plain bodies?

We do that already if we can't just bolt them on, e.g. if the charset or MIME
type for the footer doesn't match the text body.  Unfortunately, we also get
complaints from users because MUAs don't handle them very well.

http://wiki.list.org/pages/viewpage.action?pageId=4030707

>And along those lines, do any MUAs do useful things with the various List-*
>fields, other than permitting one to sort on them?

Some do.  Mine at least handles List-Post pretty well.

Cheers,
-Barry

From mark at msapiro.net  Fri May 30 21:28:51 2014
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 30 May 2014 12:28:51 -0700
Subject: [Mailman-Developers] Thinking about list footers
In-Reply-To: <CAL0qLwa-6+5KbGtNuH768k7=J1WTfmtY=hZBOyp0RQOe85YqXA@mail.gmail.com>
References: <CAL0qLwa-6+5KbGtNuH768k7=J1WTfmtY=hZBOyp0RQOe85YqXA@mail.gmail.com>
Message-ID: <5388DBF3.4030101@msapiro.net>

On 05/30/2014 10:35 AM, Murray S. Kucherawy wrote:
> 
> What's the expertise on the idea of adding footers in a new MIME text/plain
> part rather than just bolting it onto the text as-is?  (Or is that already
> done?)  What do MUAs generally do with multipart text/plain bodies?


This is what Mailman currently does with msg_header and msg_footer in
all cases *except* the common one where the message after content
filtering is a single-part text/plain message with known character set.

Mailman could easily do this in all cases, but there would be many
complaints due to common MUAs that don't display this well.

MUA's deal with this differently. Some are quite reasonable and just
display all the parts inline in the sequence in which they appear,
although they may also indicate that there are 'attachments'.

Others display only the 'body' and one is required to open or download
and open the attached footer.

Really pathological ones, including IIRC MS Outlook will in cases where
a msg_header part is added, display the msg_header part as the text of
the message with the original body and footer as attachments requiring
additional user action to see.

Also, it is not clear what the legal situation would be in jurisdictions
that require an unsubscribe link in list mail. Is a link in a separate
MIME part that may not be displayed inline by some MUAs satisfaction of
that requirement.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From rajeevs1992 at gmail.com  Sat May 31 04:50:26 2014
From: rajeevs1992 at gmail.com (Rajeev S)
Date: Sat, 31 May 2014 08:20:26 +0530
Subject: [Mailman-Developers] [GSoC 2014] Mailman CLI Project
In-Reply-To: <20140530190923.05ffaeda@angel>
References: <87d2fjeyil.fsf@uwakimon.sk.tsukuba.ac.jp>
 <5371EC9A.1030804@gmail.com> <537A88E5.4080608@gmail.com>
 <537AACC1.9040604@gmail.com> <537BF267.8070605@gmail.com>
 <20140527122724.618d5531@angel> <538516A6.6080802@gmail.com>
 <87r43c7lvy.fsf@uwakimon.sk.tsukuba.ac.jp> <20140529152007.GA22170@sys4.de>
 <87egzc77h2.fsf@uwakimon.sk.tsukuba.ac.jp> <20140529191633.GD22170@sys4.de>
 <CAPq4ZOL9X9e1EsJbt46isbRNmd1c1ys=x_EeDsS62FnM3zsVPQ@mail.gmail.com>
 <20140530190923.05ffaeda@angel>
Message-ID: <53894372.4050300@gmail.com>

On Friday 30 May 2014 07:09 PM, Abhilash Raj wrote:
> On Fri, 30 May 2014 18:12:18 +0530
> Rajeev S <rajeevs1992 at gmail.com> wrote:
>
>> I agree that an override for the confirmation message is necessary so that
>> the CLI
>> commands can be used in scripts and I would like to follow Barry's
>> suggestion of using
>> a `--yes` or `--force` switch. (Isn't --force more common than --yes, as in
>> `rm -f` ?)
> No, as I think, `--yes` and `--force` or `-f` have different use cases.
> For conformations `--yes` is used most of the times, it says 'hey I
> know you are going to ask this question and my answer would be yes'.
> While `--force` is to actually *force* any operation that cannot be
> completed otherwise.
>

Ok. I will go with the --yes.

> Good to hear that. Did you give TDD a try?

Not yet, I wrote a few test cases for the existing units using
python untitest.

I will try to focus on following TDD for the next set of units.

>   
>> Also, I would like to ask how the tests should be triggered. Should
>> there be a `mmclient test` with a set of possible switches like `all`
>> ,`domain`, `list`, `user` etc?
>> Or should the customary `python setup.py test` be used?
> You can use `nose2` for that, like mailman does. It automatically
> discovers all the tests in directories specified and runs them.

Ok.

> What detail profile are you talking about? Apart from addresses and
> name(optional) does mailman store any other data about a user? You can
> show the lists he is subscribed to, but `describe user` may not be an
> appropriate command for that. Maybe you can add it in users scope
> itself like
>    `mmclient user a at b.org --list-subscriptions`
> Others may have different thoughts though.
>

The following are the details that are available for a user

1.Addresses
2. Subscriptions
(3. Subscription list id's)
4.Preferences
5.Self link

@all
BTW I have pushed r55.

http://bazaar.launchpad.net/~rajeevs1992/mailman.client/mailmancli/revision/55



From stephen at xemacs.org  Sat May 31 13:30:18 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 31 May 2014 20:30:18 +0900
Subject: [Mailman-Developers]  Thinking about list footers
In-Reply-To: <CAL0qLwa-6+5KbGtNuH768k7=J1WTfmtY=hZBOyp0RQOe85YqXA@mail.gmail.com>
References: <CAL0qLwa-6+5KbGtNuH768k7=J1WTfmtY=hZBOyp0RQOe85YqXA@mail.gmail.com>
Message-ID: <87tx865gol.fsf@uwakimon.sk.tsukuba.ac.jp>

Murray S. Kucherawy writes:

 > What's the expertise on the idea of adding footers in a new MIME text/plain
 > part rather than just bolting it onto the text as-is?  (Or is that already
 > done?)  What do MUAs generally do with multipart text/plain bodies?

As Mark and Barry point out, the MUAs-for-people-most-vulnerable-to-
email-fraud often handle them poorly.

Also, the last time partial signatures came up, it was pointed out
that there are *no* MUAs that differentiate between signed parts and
unsigned parts.  You don't get a warning when your eyes move from a
signed part to an unsigned part or vice-versa the way you do when
following a link from an HTTP URL to an HTTPS URL in a browser.  The
DKIM advocates have not liked the idea of signatures that don't apply
to the whole message at all.

 > And along those lines, do any MUAs do useful things with the
 > various List-* fields, other than permitting one to sort on them?

I think many do.  I had a proposal some years back that I discussed
with the Mozilla people.  The idea was to devise an algorithm for MUAs
that would get rid of Reply-To munging in most cases, and an optional
header field that would allow lists to express a preference.  They
thought it would be nice if someone would write up a document but
weren't much interested in helping or implementing, they thought their
products already did a good job.  There were some refinements but the
basic idea was

1.  If there is a Reply-To:, use that address, otherwise
2.  if there is a List-Post:, use that address, otherwise,
3.  reply to the address in From:.

The optional field (I even forget the name, it was something like
List-Prefer-Reply) allowed giving From: priority over List-Post:.

I'm pretty sure that Thunderbird, Mutt, and Emacs/Gnus implement
either a "smart" reply algorithm or or a reply-to-list function.
Other Emacs-based MUAs probably do, and it would be trivial to add in
most cases.  I don't know about KMail, Sylpheed, and Evolution.

On the other hand Windows and web-based MUAs didn't do much useful at
the time, and probably don't now, either.


From sca at andreasschulze.de  Sat May 31 13:34:08 2014
From: sca at andreasschulze.de (Andreas Schulze)
Date: Sat, 31 May 2014 13:34:08 +0200
Subject: [Mailman-Developers] Thinking about list footers
In-Reply-To: <CAL0qLwa-6+5KbGtNuH768k7=J1WTfmtY=hZBOyp0RQOe85YqXA@mail.gmail.com>
References: <CAL0qLwa-6+5KbGtNuH768k7=J1WTfmtY=hZBOyp0RQOe85YqXA@mail.gmail.com>
Message-ID: <20140531133408.Horde.gCEEqRjPCcW6YGh7dvk6kg2@horde.andreasschulze.de>


Murray S. Kucherawy:

> And along those lines, do any MUAs do useful things with the various List-*
> fields, other than permitting one to sort on them?

Hello,

I think, that is the real problem: There are RFCs 2369 and 2919 and  
virtually no MUA
implement them *usefull*.

I do sort my inbound messages by list-id too. But no MUA 'show' me
  * unsubscribe
  * request list-help
  * open list-archive

And because no MUA show such information to endusers
MLM still has to modify subject and append a footer :-/

Let's talk to MUA developers!

Andreas