From fmouse at fmp.com  Fri Jan  5 03:21:17 2018
From: fmouse at fmp.com (Lindsay Haisley)
Date: Fri, 05 Jan 2018 02:21:17 -0600
Subject: [Mailman-Developers] Small contrib script
Message-ID: <1515140477.46954.10.camel@fmp.com>

I have a small script which may be appropriate for the contrib section
in Mailman 2. Basically, it simply assembles a few build-time options,
including the option string given to the configure script, and spits
them out to stdout when the script is executed.

Modifications to configure.in are accomplished using a patch file, and
autoconf needs to be run to regenerate the configure script.

In what form and fashion should I submit this contribution, which
consists of about 3 files. I can put these in a tarball and attach
them, attach them seprately, put them on my public server and post the
URL to this list, or upload them, whatever's appropriate. This is a
small package, less than 3K.

Let me know.

-- 
Lindsay Haisley       | "The first casualty when
FMP Computer Services |         war comes is truth."
512-259-1190          |            
http://www.fmp.com    |     -- Hiram W Johnson


From mark at msapiro.net  Fri Jan  5 23:05:55 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 5 Jan 2018 20:05:55 -0800
Subject: [Mailman-Developers] Small contrib script
In-Reply-To: <1515140477.46954.10.camel@fmp.com>
References: <1515140477.46954.10.camel@fmp.com>
Message-ID: <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>

On 01/05/2018 12:21 AM, Lindsay Haisley wrote:
> 
> In what form and fashion should I submit this contribution, which
> consists of about 3 files. I can put these in a tarball and attach
> them, attach them seprately, put them on my public server and post the
> URL to this list, or upload them, whatever's appropriate. This is a
> small package, less than 3K.


The ideal way is to register at launchpad, get a copy of the branch at
<https://code.launchpad.net/~mailman-coders/mailman/2.1>, make your
additions, push that to a branch in your account and create a merge
proposal, but if that seems too complex, you can send me a tarball off list.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From fmouse at fmp.com  Sat Jan  6 17:51:52 2018
From: fmouse at fmp.com (Lindsay Haisley)
Date: Sat, 06 Jan 2018 16:51:52 -0600
Subject: [Mailman-Developers] Small contrib script
In-Reply-To: <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>
References: <1515140477.46954.10.camel@fmp.com>
 <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>
Message-ID: <1515279112.118708.35.camel@fmp.com>

On Fri, 2018-01-05 at 20:05 -0800, Mark Sapiro wrote:
> On 01/05/2018 12:21 AM, Lindsay Haisley wrote:
> > 
> > 
> > In what form and fashion should I submit this contribution, which
> > consists of about 3 files. I can put these in a tarball and attach
> > them, attach them seprately, put them on my public server and post the
> > URL to this list, or upload them, whatever's appropriate. This is a
> > small package, less than 3K.
> 
> The ideal way is to register at launchpad, get a copy of the branch at
> <https://code.launchpad.net/~mailman-coders/mailman/2.1>, make your
> additions, push that to a branch in your account and create a merge
> proposal, but if that seems too complex, you can send me a tarball off list.

This isn't too complex, and I'm almost there, but I haven't worked with
version control since I was working with cvs some years ago (showing my
age, I guess) so bzr is new to me.

I've made a local copy of the MM 2.1 source tree using 'bzr branch
lp:mailman/2.1" and made appropriate revisions to it incorporating my
mailman-config.py into the contrib directory of the source tree and
modifying configure and configure.in to do the proper substitutions for
the working copy that's generated in build/contrib at build time.

I'm a bit at sea on the last two parts of your instruction. How do I
properly push my branch back to my launchpad account? Do I create a
PPA?

How do I create a merge proposal?

This isn't, as they say, rocket science, and I've been a Linux system
administrator for close to 20 years, so if I know what I'm doing I can
easily handle it, but a few tips from you folks will save me a bunch of
research. I don't want to go mucking around on a community resource
without knowing what I'm doing :)

-- 
Lindsay Haisley       | "The first casualty when
FMP Computer Services |         war comes is truth."
512-259-1190          |            
http://www.fmp.com    |     -- Hiram W Johnson


From mark at msapiro.net  Sat Jan  6 18:25:55 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 6 Jan 2018 15:25:55 -0800
Subject: [Mailman-Developers] Small contrib script
In-Reply-To: <1515279112.118708.35.camel@fmp.com>
References: <1515140477.46954.10.camel@fmp.com>
 <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>
 <1515279112.118708.35.camel@fmp.com>
Message-ID: <df8ca0c5-3e02-07d6-4e92-98e9ec07c90d@msapiro.net>

On 01/06/2018 02:51 PM, Lindsay Haisley wrote:
> 
> I've made a local copy of the MM 2.1 source tree using 'bzr branch
> lp:mailman/2.1" and made appropriate revisions to it incorporating my
> mailman-config.py into the contrib directory of the source tree and
> modifying configure and configure.in to do the proper substitutions for
> the working copy that's generated in build/contrib at build time.
> 
> I'm a bit at sea on the last two parts of your instruction. How do I
> properly push my branch back to my launchpad account? Do I create a
> PPA?


No, you don't need to create a PPA. It's been a while since I've done
this, so I may not have it exactly right, but assume your launchpad user
name is "example" and assuming you have committed all your changes in
your bzr branch, first ensure you have uploaded your ssh public key to
your Launchpad account.

Then in your bzr branch do

bzr push lp:~example/mailman/name
"example" is your actual user name. "mailman" is the project this is
associated with and "name" will be the Launchpad name of your branch and
can be any name that is meaningful to you.


> How do I create a merge proposal?

Once you have pushed your branch, go to it in Launchpad. It will be at
<https://code.launchpad.net/~example/mailman/name> or you will also find
it listed at <https://code.launchpad.net/~example/>.

On your branch's code page will be a "Propose for merging" link. Follow
that, set the Target Branch to ~mailman-coders/mailman/2.1, add a
description if you like and "Propose Merge"


-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From fmouse at fmp.com  Sat Jan  6 19:28:13 2018
From: fmouse at fmp.com (Lindsay Haisley)
Date: Sat, 06 Jan 2018 18:28:13 -0600
Subject: [Mailman-Developers] Small contrib script
In-Reply-To: <df8ca0c5-3e02-07d6-4e92-98e9ec07c90d@msapiro.net>
References: <1515140477.46954.10.camel@fmp.com>
 <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>
 <1515279112.118708.35.camel@fmp.com>
 <df8ca0c5-3e02-07d6-4e92-98e9ec07c90d@msapiro.net>
Message-ID: <1515284893.118708.59.camel@fmp.com>

On Sat, 2018-01-06 at 15:25 -0800, Mark Sapiro wrote:
> Once you have pushed your branch, go to it in Launchpad. It will be at
> <https://code.launchpad.net/~example/mailman/name> or you will also find
> it listed at <https://code.launchpad.net/~example/>.
> 
> On your branch's code page will be a "Propose for merging" link. Follow
> that, set the Target Branch to ~mailman-coders/mailman/2.1, add a
> description if you like and "Propose Merge"

OK, done. I had to frog around a bit and mentally review my work from
some years ago with cvs, which bzr resembles. There's one deleted
revision in the revision history for mailman (sorry) since I was
learning by doing.

mailman-config.py, which this branch adds, is the result of a recent
discussion in the mailman-users list. I use Courier as my mail suite
(IMAP, MTA, Custom spam filtering, virtual mailboxes, etc.) and since
Courier is often built from source outside of a distribution's package
management, the Courier developers (notably Sam Varshavchik) have
wisely included a "courier-config" utility which, when executed, shows
the build-time configure options used in building the package plus a
few other things.

MM is another suite that I generally install from source, since stuff
such as DMARC mitigation doesn't make it down the pike into the Ubuntu
package until long after it's needed.

If mailman-config.py is of interest, it's easy enough to make it show
other build-time options. At present, all it shows is the MM version,
the build date/time, prifix, var_prefix, mailman_user, mailman_group,
mail_group, cgi_group and the list of options provided to configure. If
it's of interest, and more output information is needed, let me know
and I'll add it.

-- 
Lindsay Haisley       | "The first casualty when
FMP Computer Services |         war comes is truth."
512-259-1190          |            
http://www.fmp.com    |     -- Hiram W Johnson


From mark at msapiro.net  Sat Jan  6 20:16:32 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 6 Jan 2018 17:16:32 -0800
Subject: [Mailman-Developers] Small contrib script
In-Reply-To: <1515284893.118708.59.camel@fmp.com>
References: <1515140477.46954.10.camel@fmp.com>
 <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>
 <1515279112.118708.35.camel@fmp.com>
 <df8ca0c5-3e02-07d6-4e92-98e9ec07c90d@msapiro.net>
 <1515284893.118708.59.camel@fmp.com>
Message-ID: <003f7141-3dae-3f60-ff55-baf932735834@msapiro.net>

On 01/06/2018 04:28 PM, Lindsay Haisley wrote:
> On Sat, 2018-01-06 at 15:25 -0800, Mark Sapiro wrote:
>>
>> On your branch's code page will be a "Propose for merging" link. Follow
>> that, set the Target Branch to ~mailman-coders/mailman/2.1, add a
>> description if you like and "Propose Merge"
> 
> OK, done. I had to frog around a bit and mentally review my work from
> some years ago with cvs, which bzr resembles. There's one deleted
> revision in the revision history for mailman (sorry) since I was
> learning by doing.


I'm looking at it now.


> mailman-config.py, which this branch adds, is the result of a recent
> discussion in the mailman-users list. I use Courier as my mail suite
> (IMAP, MTA, Custom spam filtering, virtual mailboxes, etc.) and since
> Courier is often built from source outside of a distribution's package
> management, the Courier developers (notably Sam Varshavchik) have
> wisely included a "courier-config" utility which, when executed, shows
> the build-time configure options used in building the package plus a
> few other things.


The problem with what you currently have done is the configured script
only exists in the source build/contrib directory which will disappear
if one runs 'make distclean' and which is somewhat obscure to find in
any case.

If this is to be truly useful, it should be a bin/ command so it gets
into the installed Mailman.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From fmouse at fmp.com  Sat Jan  6 20:54:16 2018
From: fmouse at fmp.com (Lindsay Haisley)
Date: Sat, 06 Jan 2018 19:54:16 -0600
Subject: [Mailman-Developers] Small contrib script
In-Reply-To: <003f7141-3dae-3f60-ff55-baf932735834@msapiro.net>
References: <1515140477.46954.10.camel@fmp.com>
 <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>
 <1515279112.118708.35.camel@fmp.com>
 <df8ca0c5-3e02-07d6-4e92-98e9ec07c90d@msapiro.net>
 <1515284893.118708.59.camel@fmp.com>
 <003f7141-3dae-3f60-ff55-baf932735834@msapiro.net>
Message-ID: <1515290056.26461.14.camel@fmp.com>

On Sat, 2018-01-06 at 17:16 -0800, Mark Sapiro wrote:
> The problem with what you currently have done is the configured script
> only exists in the source build/contrib directory which will disappear
> if one runs 'make distclean' and which is somewhat obscure to find in
> any case.

True, that.?

> If this is to be truly useful, it should be a bin/ command so it gets
> into the installed Mailman.

I did give that some thought, and it should be a trivial change to put
it in bin rather than build/contrib. I quite agree with you, but I
wasn't sure if this utility conformed to policy for files in the bin
directory. I'll look at the make install code, but I'd guess anything
in bin in the source tree pretty much just gets configured (AC_SUBST)
and then copied to ~mailman/bin during the install (assuming it's
listed in SCRIPTS in Makefile.in).

-- 
Lindsay Haisley       | "The only unchanging certainty
FMP Computer Services |    is the certainty of change"
512-259-1190          |
http://www.fmp.com    | - Ancient wisdom, all cultures


From fmouse at fmp.com  Sun Jan  7 00:41:34 2018
From: fmouse at fmp.com (Lindsay Haisley)
Date: Sat, 06 Jan 2018 23:41:34 -0600
Subject: [Mailman-Developers] Small contrib script
In-Reply-To: <003f7141-3dae-3f60-ff55-baf932735834@msapiro.net>
References: <1515140477.46954.10.camel@fmp.com>
 <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>
 <1515279112.118708.35.camel@fmp.com>
 <df8ca0c5-3e02-07d6-4e92-98e9ec07c90d@msapiro.net>
 <1515284893.118708.59.camel@fmp.com>
 <003f7141-3dae-3f60-ff55-baf932735834@msapiro.net>
Message-ID: <1515303694.118708.66.camel@fmp.com>

On Sat, 2018-01-06 at 17:16 -0800, Mark Sapiro wrote:
> The problem with what you currently have done is the configured script
> only exists in the source build/contrib directory which will disappear
> if one runs 'make distclean' and which is somewhat obscure to find in
> any case.
> 
> If this is to be truly useful, it should be a bin/ command so it gets
> into the installed Mailman.

This has been changed as per your suggestion and pushed to Launchpad. I
tried to post a merge request again, after making the changes, but it
was disallowed by the system since I'd already done this for the
original branch push. I presume mailman coders were notified of the
change.

Sorry if I'm a bit clumsy at this. I'm not familiar with bzr and
Launchpad version control, and community development techniques
associated with them.

-- 
Lindsay Haisley       | "The first casualty when
FMP Computer Services |         war comes is truth."
512-259-1190          |            
http://www.fmp.com    |     -- Hiram W Johnson

From mark at msapiro.net  Sun Jan  7 11:25:04 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 7 Jan 2018 08:25:04 -0800
Subject: [Mailman-Developers] Small contrib script
In-Reply-To: <1515303694.118708.66.camel@fmp.com>
References: <1515140477.46954.10.camel@fmp.com>
 <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>
 <1515279112.118708.35.camel@fmp.com>
 <df8ca0c5-3e02-07d6-4e92-98e9ec07c90d@msapiro.net>
 <1515284893.118708.59.camel@fmp.com>
 <003f7141-3dae-3f60-ff55-baf932735834@msapiro.net>
 <1515303694.118708.66.camel@fmp.com>
Message-ID: <66ec2919-b918-87af-d479-90294e15780c@msapiro.net>

On 01/06/2018 09:41 PM, Lindsay Haisley wrote:
> 
> This has been changed as per your suggestion and pushed to Launchpad. I
> tried to post a merge request again, after making the changes, but it
> was disallowed by the system since I'd already done this for the
> original branch push. I presume mailman coders were notified of the
> change.


When you push additional changes to your branch, your merge proposal is
automatically updated.

I'll look at the changes in detail within the next few days.


> Sorry if I'm a bit clumsy at this. I'm not familiar with bzr and
> Launchpad version control, and community development techniques
> associated with them.


That's OK. You're doing fine. However, try not to invest too much effort
into learning bzr and Launchpad since we have moved to git and GitLab
for Mailman 3.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From fmouse at fmp.com  Sun Jan  7 13:13:32 2018
From: fmouse at fmp.com (Lindsay Haisley)
Date: Sun, 07 Jan 2018 12:13:32 -0600
Subject: [Mailman-Developers] Small contrib script
In-Reply-To: <66ec2919-b918-87af-d479-90294e15780c@msapiro.net>
References: <1515140477.46954.10.camel@fmp.com>
 <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>
 <1515279112.118708.35.camel@fmp.com>
 <df8ca0c5-3e02-07d6-4e92-98e9ec07c90d@msapiro.net>
 <1515284893.118708.59.camel@fmp.com>
 <003f7141-3dae-3f60-ff55-baf932735834@msapiro.net>
 <1515303694.118708.66.camel@fmp.com>
 <66ec2919-b918-87af-d479-90294e15780c@msapiro.net>
Message-ID: <1515348812.118708.74.camel@fmp.com>

On Sun, 2018-01-07 at 08:25 -0800, Mark Sapiro wrote:
> When you push additional changes to your branch, your merge proposal is
> automatically updated.

Thanks.

> I'll look at the changes in detail within the next few days.

Fine. The mailman-config program just provides a bit of convenient
information. Code-wise, it's trivial. The parts that took time were
boning up on autoconf and learning Launchpad's version control system.

> > Sorry if I'm a bit clumsy at this. I'm not familiar with bzr and
> > Launchpad version control, and community development techniques
> > associated with them.
> 
> That's OK. You're doing fine. However, try not to invest too much effort
> into learning bzr and Launchpad since we have moved to git and GitLab
> for Mailman 3.

Learning anything such as languages (programming or spoken), musical
instruments and version control systems has the property that after
you've learned one or two, learning others becomes much easier.

-- 
Lindsay Haisley       | "Behold! Our way lies through a
FMP Computer Services |    dark wood whence in which
512-259-1190          |      weirdness may wallow!?
http://www.fmp.com    |           --Beauregard


From fmouse at fmp.com  Sun Jan  7 15:28:51 2018
From: fmouse at fmp.com (Lindsay Haisley)
Date: Sun, 07 Jan 2018 14:28:51 -0600
Subject: [Mailman-Developers] Small contrib script - Autoconf
In-Reply-To: <66ec2919-b918-87af-d479-90294e15780c@msapiro.net>
References: <1515140477.46954.10.camel@fmp.com>
 <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>
 <1515279112.118708.35.camel@fmp.com>
 <df8ca0c5-3e02-07d6-4e92-98e9ec07c90d@msapiro.net>
 <1515284893.118708.59.camel@fmp.com>
 <003f7141-3dae-3f60-ff55-baf932735834@msapiro.net>
 <1515303694.118708.66.camel@fmp.com>
 <66ec2919-b918-87af-d479-90294e15780c@msapiro.net>
Message-ID: <1515356931.118708.96.camel@fmp.com>

What's the proper way to run autoconf on configure.in in MM2 in order
to _not_ generate runstatedir options and code in the configure script?
I note that the distributed configure script in MM2 doesn't have them,
but simply running autoconf on configure.in produces a configure script
that does. This option is harmless enough, since MM apparently doesn't
need or use it, but for simplicity I'd like to be able to generate a
configure script without it.

-- 
Lindsay Haisley ? ? ? | "I felt a great disturbance in the Force,
FMP Computer Services | ? as if millions of voices suddenly cried out
512-259-1190 ? ? ? ? ?| ? ? in terror and were suddenly silenced."
http://www.fmp.com? ? | ? ? ? ? ? ? ?- Obi-Wan Kenobi

From mark at msapiro.net  Tue Jan  9 18:55:57 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 9 Jan 2018 15:55:57 -0800
Subject: [Mailman-Developers] Small contrib script - Autoconf
In-Reply-To: <1515356931.118708.96.camel@fmp.com>
References: <1515140477.46954.10.camel@fmp.com>
 <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>
 <1515279112.118708.35.camel@fmp.com>
 <df8ca0c5-3e02-07d6-4e92-98e9ec07c90d@msapiro.net>
 <1515284893.118708.59.camel@fmp.com>
 <003f7141-3dae-3f60-ff55-baf932735834@msapiro.net>
 <1515303694.118708.66.camel@fmp.com>
 <66ec2919-b918-87af-d479-90294e15780c@msapiro.net>
 <1515356931.118708.96.camel@fmp.com>
Message-ID: <387e81a2-f547-bb34-e677-39d1e7a20295@msapiro.net>

On 01/07/2018 12:28 PM, Lindsay Haisley wrote:
> What's the proper way to run autoconf on configure.in in MM2 in order
> to _not_ generate runstatedir options and code in the configure script?
> I note that the distributed configure script in MM2 doesn't have them,
> but simply running autoconf on configure.in produces a configure script
> that does. This option is harmless enough, since MM apparently doesn't
> need or use it, but for simplicity I'd like to be able to generate a
> configure script without it.


In my case, my autoconf which didn't generate those options in 2014 when
I generated the current configure and which then was probably
debian/ubuntu version 2.69-7 is now debian/ubuntu version 2.69-9 and it
does generate them. See
<https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=759647>.

The bottom line is this is something that newer autoconf does and if
your autoconf does it, I don't know a way to turn it off.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From fmouse at fmp.com  Tue Jan  9 19:01:35 2018
From: fmouse at fmp.com (Lindsay Haisley)
Date: Tue, 09 Jan 2018 18:01:35 -0600
Subject: [Mailman-Developers] Small contrib script - Autoconf
In-Reply-To: <387e81a2-f547-bb34-e677-39d1e7a20295@msapiro.net>
References: <1515140477.46954.10.camel@fmp.com>
 <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>
 <1515279112.118708.35.camel@fmp.com>
 <df8ca0c5-3e02-07d6-4e92-98e9ec07c90d@msapiro.net>
 <1515284893.118708.59.camel@fmp.com>
 <003f7141-3dae-3f60-ff55-baf932735834@msapiro.net>
 <1515303694.118708.66.camel@fmp.com>
 <66ec2919-b918-87af-d479-90294e15780c@msapiro.net>
 <1515356931.118708.96.camel@fmp.com>
 <387e81a2-f547-bb34-e677-39d1e7a20295@msapiro.net>
Message-ID: <1515542495.94448.0.camel@fmp.com>

On Tue, 2018-01-09 at 15:55 -0800, Mark Sapiro wrote:
> On 01/07/2018 12:28 PM, Lindsay Haisley wrote:
> > 
> > What's the proper way to run autoconf on configure.in in MM2 in order
> > to _not_ generate runstatedir options and code in the configure script?
> > I note that the distributed configure script in MM2 doesn't have them,
> > but simply running autoconf on configure.in produces a configure script
> > that does. This option is harmless enough, since MM apparently doesn't
> > need or use it, but for simplicity I'd like to be able to generate a
> > configure script without it.
> 
> In my case, my autoconf which didn't generate those options in 2014 when
> I generated the current configure and which then was probably
> debian/ubuntu version 2.69-7 is now debian/ubuntu version 2.69-9 and it
> does generate them. See
> <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=759647>.
> 
> The bottom line is this is something that newer autoconf does and if
> your autoconf does it, I don't know a way to turn it off.

I looked at the code and that's kinda what I figured. Thanks

-- 
Lindsay Haisley       | "The first casualty when
FMP Computer Services |         war comes is truth."
512-259-1190          |            
http://www.fmp.com    |     -- Hiram W Johnson


From turnbull.stephen.fw at u.tsukuba.ac.jp  Tue Jan  9 22:13:20 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Wed, 10 Jan 2018 12:13:20 +0900
Subject: [Mailman-Developers] Small contrib script - Autoconf
In-Reply-To: <1515542495.94448.0.camel@fmp.com>
References: <1515140477.46954.10.camel@fmp.com>
 <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>
 <1515279112.118708.35.camel@fmp.com>
 <df8ca0c5-3e02-07d6-4e92-98e9ec07c90d@msapiro.net>
 <1515284893.118708.59.camel@fmp.com>
 <003f7141-3dae-3f60-ff55-baf932735834@msapiro.net>
 <1515303694.118708.66.camel@fmp.com>
 <66ec2919-b918-87af-d479-90294e15780c@msapiro.net>
 <1515356931.118708.96.camel@fmp.com>
 <387e81a2-f547-bb34-e677-39d1e7a20295@msapiro.net>
 <1515542495.94448.0.camel@fmp.com>
Message-ID: <23125.34000.325169.284867@turnbull.sk.tsukuba.ac.jp>

Lindsay Haisley writes:

 > > The bottom line is this is something that newer autoconf does and
 > > if your autoconf does it, I don't know a way to turn it off.
 > 
 > I looked at the code and that's kinda what I figured. Thanks

The autotools authors are *extremely* prescriptive, and try to
implement as much of the "GNU standard" in code as they can.  The
first macro in an autoconf script is ac_init, which IIRC is
boilerplate setup and environment consistency checks.  Most likely one
of the next two or three macro invocations sets up the standard
options, and that will invoke a macro to set up these directories.  Or
it might be another layer deeper.  Generally the way to turn stuff off
is to unroll the macros and only use what you need.

All of the autotools stuff is deeply entwined.  I suspect we only use
autoconf, so surgery on the configure.in to only call the stuff we use
is probably OK.  But if we use anything like automake, it's likely
there will be ripple effects if you remove GNU standard setup as Make
variables that are defined by the standard and used in the Makefile go
undefined.

It's probably not worth doing more.

Steve


From fmouse at fmp.com  Tue Jan  9 23:36:56 2018
From: fmouse at fmp.com (Lindsay Haisley)
Date: Tue, 09 Jan 2018 22:36:56 -0600
Subject: [Mailman-Developers] Small contrib script - Autoconf
In-Reply-To: <23125.34000.325169.284867@turnbull.sk.tsukuba.ac.jp>
References: <1515140477.46954.10.camel@fmp.com>
 <ed8bfc61-f2ea-1001-d2af-f5c95aa06289@msapiro.net>
 <1515279112.118708.35.camel@fmp.com>
 <df8ca0c5-3e02-07d6-4e92-98e9ec07c90d@msapiro.net>
 <1515284893.118708.59.camel@fmp.com>
 <003f7141-3dae-3f60-ff55-baf932735834@msapiro.net>
 <1515303694.118708.66.camel@fmp.com>
 <66ec2919-b918-87af-d479-90294e15780c@msapiro.net>
 <1515356931.118708.96.camel@fmp.com>
 <387e81a2-f547-bb34-e677-39d1e7a20295@msapiro.net>
 <1515542495.94448.0.camel@fmp.com>
 <23125.34000.325169.284867@turnbull.sk.tsukuba.ac.jp>
Message-ID: <1515559016.94448.56.camel@fmp.com>

On Wed, 2018-01-10 at 12:13 +0900, Stephen J. Turnbull wrote:
> Lindsay Haisley writes:
> 
> ?> > The bottom line is this is something that newer autoconf does and
> ?> > if your autoconf does it, I don't know a way to turn it off.
> ?>?
> ?> I looked at the code and that's kinda what I figured. Thanks
> 
> All of the autotools stuff is deeply entwined.??I suspect we only use
> autoconf, so surgery on the configure.in to only call the stuff we use
> is probably OK.??But if we use anything like automake, it's likely
> there will be ripple effects if you remove GNU standard setup as Make
> variables that are defined by the standard and used in the Makefile go
> undefined.
> 
> It's probably not worth doing more.

Frankly, my main issue is my timidity, and not wanting to make any
changes not directly related to my proposed mailman-config script. If I
ran autoconf in the source root without changing configure.in I wanted
the resulting configure script to be byte for byte the same as the one
distributed with every version of MM2 since dirt was new (or at least
several version back). It wasn't. On the other hand, the only
difference was code supporting a (relatively) new config
variable,?runstatedir, which allows flexible assignment of the location
of temporary runtime process information such as the PID file and such.
When this was introduced, there was a transition underway, maybe with
the FSF standard too, moving files from /var/run to /run in some
distributions. There's a FSF announcement about it at?
https://lists.gnu.org/archive/html/bug-autoconf/2013-09/msg00003.html

MM2 doesn't appear to use anything in /run or /var/run, so runstatedir
is kind of moot, and looking at the supporting macros I didn't see any
code which allowed reference to it to be removed without hacking a
macro file, which didn't seem appropriate.

I had to make changes to configure.in, and I've generated the configure
file with the autoconf I have here - v2.69. Going forward setting?
--runstatedir will be a configure option. As far as I can see, it's
harmless, so I'll let it be and move on. If it needs attention down the
road it can be dealt with then. If reference to runstatedir is
surgically removed from the configure script here it becomes
"politically correct" and byte for byte identical in all other respects
to the old version of it.

Ciao,
lh

-- 
Lindsay Haisley ? ? ? | "Dissent is what rescues democracy
FMP Computer Services | ? from a quiet death behind?
512-259-1190 ? ? ? ? ?| ? ? closed doors"?
http://www.fmp.com ? ?| ? ? ? - Molly Ivins


From turnbull.stephen.fw at u.tsukuba.ac.jp  Wed Jan 17 00:01:54 2018
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Wed, 17 Jan 2018 14:01:54 +0900
Subject: [Mailman-Developers] Firefox loves us!  Leftmost "top site"!
Message-ID: <23134.55490.996676.299158@turnbull.sk.tsukuba.ac.jp>

>From the Silly but Encouraging Fact of the Day Department

http://turnbull.sk.tsukuba.ac.jp/MailmanFTW.png

Note the empty "Highlights" box: I haven't used Firefox in a long
time, just checking that another browser isn't broken on one of my
employer's sites.  So this *isn't* because I visit Mailman a lot.

<wink/>

Steve


From mark at msapiro.net  Fri Jan 19 19:24:54 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 19 Jan 2018 16:24:54 -0800
Subject: [Mailman-Developers] Issues with SQLAlchemy >= 1.2 All CI fails.
Message-ID: <efca147e-7011-3fc6-51e5-11ca9ecbf08e@msapiro.net>

One specific test is failing when run with SQLAlchemy >= 1.2. The test
test_absorb_memberships in mailman.model.tests.test_user.TestUser throws
sqlalchemy.orm.exc.StaleDataError.

I have a small MR I'm working on. I'm going to push that and require
sqlalchemy < 1.2 for the time being.

I still haven't figured out whether we have a real incompatibility or if
it's just a problem with the test.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sat Jan 20 15:43:21 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 20 Jan 2018 12:43:21 -0800
Subject: [Mailman-Developers] Mailman 2.1.26 Security release Feb 4, 2018
Message-ID: <68476c0b-f481-1ff3-4cd1-0b5f01dbc5cf@msapiro.net>

An XSS vulnerability in the Mailman 2.1 web UI has been reported and
assigned CVE-2018-5950 which is not yet public.

I plan to release Mailman 2.1.26 along with a patch for older releases
to fix this issue on Feb 4, 2018. At that time, full details of the
vulnerability will be public.

This is advance notice of the upcoming release and patch for those that
need a week or two to prepare. The patch will be small and only affect
one module.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20180120/e786295c/attachment.sig>

From holger at merlinux.eu  Sun Jan 21 00:47:09 2018
From: holger at merlinux.eu (holger krekel)
Date: Sun, 21 Jan 2018 06:47:09 +0100
Subject: [Mailman-Developers] opportunistically encrypted mailman lists with
 Autocrypt
Message-ID: <20180121054709.GT16901@beto>

Hi all,

maybe some of you know me for my works on pypy, tox or pytest but
this mail will be about something else ...

In the last year i co-instigated a new opportunistic mail encryption
effort called Autocrypt (https://autocrypt.org). With Autocrypt Level 1, 
mail clients (e.g. enigmail, K-9 mail, Delta.chat, ...) send and parse
public keys in "Autocrypt" e-mail headers and offer single-click
encryption.  Releases are upcoming in spring 2018, we have been
doing fun and well-received user-testing sessions already 
with in-development versions. 

In 2018, I'd like to tackle basic integration of Mailman and Autocrypt, 
to achieve opportunistically encrypted mailing lists.  The main idea is to
grow a mailman mode/plugin to:

1) have mailman lists maintain a public pgp identity that
   is added through Autocrypt headers to outgoing mails. 

2) have mailman keep track of "incoming" autocrypt keys
   and decrypt incoming mails if they are encrypted.

3) encrypt to those subscribers where we have a proper Autocrypt key

Code wise, i'd like to tackle this based on the the new and evolving
(and quite unpublic so far) "muacrypt" project:
https://muacrypt.readthedocs.io 

>From looking at archives and the GSOC idea page i see there were related
efforts and ideas.  Are there pointers to draft implementations that
are still somewhat up to date (with mm3)? 

Also, i am considering to submit a project proposal for the
Mozilla Mission Partners program which would include a few more
things ... OnionMX routing for postfix, and documentation on
how to setup a best-practise MM3 mail instance, and maybe organizing
a sprint around the mentioned topics.

collaboration welcome'ly yours,

holger

P.S.: i had discussed mm-encryption with Barry 2-3 years ago
and feel now, with Autocrypt getting out the door, it's all 
becoming more feasible.


From mark at msapiro.net  Sun Jan 21 19:04:27 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 21 Jan 2018 16:04:27 -0800
Subject: [Mailman-Developers] Issues with SQLAlchemy >= 1.2 All CI fails.
In-Reply-To: <efca147e-7011-3fc6-51e5-11ca9ecbf08e@msapiro.net>
References: <efca147e-7011-3fc6-51e5-11ca9ecbf08e@msapiro.net>
Message-ID: <d37c7802-2ea3-7b3f-52b3-e7d3945ab637@msapiro.net>

On 01/19/2018 04:24 PM, Mark Sapiro wrote:
> One specific test is failing when run with SQLAlchemy >= 1.2. The test
> test_absorb_memberships in mailman.model.tests.test_user.TestUser throws
> sqlalchemy.orm.exc.StaleDataError.
> 
> I have a small MR I'm working on. I'm going to push that and require
> sqlalchemy < 1.2 for the time being.
> 
> I still haven't figured out whether we have a real incompatibility or if
> it's just a problem with the test.


I have posted this issue at
<https://groups.google.com/forum/#!topic/sqlalchemy/2E3TPXvmcsA>. There
is a follow-up there from Mike Bayer asking for more information. I will
not have time to work on this before next weekend - I have an
out-of-town guest visiting this coming week.

If someone else can look at the thread and provide what Mike is asking,
that would be cool.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From holger at merlinux.eu  Tue Jan 23 09:04:32 2018
From: holger at merlinux.eu (holger krekel)
Date: Tue, 23 Jan 2018 15:04:32 +0100
Subject: [Mailman-Developers] opportunistically encrypted mailman lists
 with Autocrypt
In-Reply-To: <20180121054709.GT16901@beto>
References: <20180121054709.GT16901@beto>
Message-ID: <20180123140432.GJ16901@beto>

Hi again,

To be a bit clearer wrt my questions:

- are there any active efforts wrt mailman and encryption?
  any "post mortems" on former ones? 

- are some of you interested in mailman+encryption efforts? 

I btw expect autocrypt integration to be significantly 
simpler than previous efforts that dealt with keyservers IISIC.

holger


On Sun, Jan 21, 2018 at 06:47 +0100, holger krekel wrote:
> Hi all,
> 
> maybe some of you know me for my works on pypy, tox or pytest but
> this mail will be about something else ...
> 
> In the last year i co-instigated a new opportunistic mail encryption
> effort called Autocrypt (https://autocrypt.org). With Autocrypt Level 1, 
> mail clients (e.g. enigmail, K-9 mail, Delta.chat, ...) send and parse
> public keys in "Autocrypt" e-mail headers and offer single-click
> encryption.  Releases are upcoming in spring 2018, we have been
> doing fun and well-received user-testing sessions already 
> with in-development versions. 
> 
> In 2018, I'd like to tackle basic integration of Mailman and Autocrypt, 
> to achieve opportunistically encrypted mailing lists.  The main idea is to
> grow a mailman mode/plugin to:
> 
> 1) have mailman lists maintain a public pgp identity that
>    is added through Autocrypt headers to outgoing mails. 
> 
> 2) have mailman keep track of "incoming" autocrypt keys
>    and decrypt incoming mails if they are encrypted.
> 
> 3) encrypt to those subscribers where we have a proper Autocrypt key
> 
> Code wise, i'd like to tackle this based on the the new and evolving
> (and quite unpublic so far) "muacrypt" project:
> https://muacrypt.readthedocs.io 
> 
> >From looking at archives and the GSOC idea page i see there were related
> efforts and ideas.  Are there pointers to draft implementations that
> are still somewhat up to date (with mm3)? 
> 
> Also, i am considering to submit a project proposal for the
> Mozilla Mission Partners program which would include a few more
> things ... OnionMX routing for postfix, and documentation on
> how to setup a best-practise MM3 mail instance, and maybe organizing
> a sprint around the mentioned topics.
> 
> collaboration welcome'ly yours,
> 
> holger
> 
> P.S.: i had discussed mm-encryption with Barry 2-3 years ago
> and feel now, with Autocrypt getting out the door, it's all 
> becoming more feasible.
> 
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers at python.org
> https://mail.python.org/mailman/listinfo/mailman-developers
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/holger%40merlinux.eu
> 
> Security Policy: http://wiki.list.org/x/QIA9

From barry at list.org  Tue Jan 23 11:51:18 2018
From: barry at list.org (Barry Warsaw)
Date: Tue, 23 Jan 2018 11:51:18 -0500
Subject: [Mailman-Developers] opportunistically encrypted mailman lists
 with Autocrypt
In-Reply-To: <20180121054709.GT16901@beto>
References: <20180121054709.GT16901@beto>
Message-ID: <5F7A232A-DAE1-49CB-B34C-6888BCBCC8A7@list.org>

Hi Holger.  Very cool initiative.  I just skimmed the specs for now, but I do like how it takes an opportunistic approach to key management, in order to simplify the UX and increase adoption.

> On Jan 21, 2018, at 00:47, holger krekel <holger at merlinux.eu> wrote:
> 
> maybe some of you know me for my works on pypy, tox or pytest but
> this mail will be about something else ?

Indeed.  If autocrypt becomes even a fraction as amazing, joyful to use, and essential to our toolkits as those others that you work on, it will undoubtedly succeed.

> In the last year i co-instigated a new opportunistic mail encryption
> effort called Autocrypt (https://autocrypt.org). With Autocrypt Level 1,
> mail clients (e.g. enigmail, K-9 mail, Delta.chat, ...) send and parse
> public keys in "Autocrypt" e-mail headers and offer single-click
> encryption.  Releases are upcoming in spring 2018, we have been
> doing fun and well-received user-testing sessions already
> with in-development versions.

Have there been any talks about creating plugins for commercial MUAs like Mail.app, Postbox, and Outlook?  Do you have plans for webmail clients like Gmail or Outlook?

> In 2018, I'd like to tackle basic integration of Mailman and Autocrypt,
> to achieve opportunistically encrypted mailing lists.  The main idea is to
> grow a mailman mode/plugin to:
> 
> 1) have mailman lists maintain a public pgp identity that
>   is added through Autocrypt headers to outgoing mails.
> 
> 2) have mailman keep track of "incoming" autocrypt keys
>   and decrypt incoming mails if they are encrypted.
> 
> 3) encrypt to those subscribers where we have a proper Autocrypt key
> 
> Code wise, i'd like to tackle this based on the the new and evolving
> (and quite unpublic so far) "muacrypt" project:
> https://muacrypt.readthedocs.io
> 
>> From looking at archives and the GSOC idea page i see there were related
> efforts and ideas.  Are there pointers to draft implementations that
> are still somewhat up to date (with mm3)?

I?m not really sure what the state of those previous efforts are, i.e. whether they?d be applicable to the current code base without a serious rebase.  ISTM that at least some of the changes in general would still be applicable, e.g. model changes to manage keys.

> Also, i am considering to submit a project proposal for the
> Mozilla Mission Partners program which would include a few more
> things ... OnionMX routing for postfix, and documentation on
> how to setup a best-practise MM3 mail instance, and maybe organizing
> a sprint around the mentioned topics.

Cool.  I think for best-practices in standing up an instance, you should definitely look at Abhilash?s docker images.  A sprint would be fun for sure.  Are you thinking Pycon US 2018 or elsewhere?

> collaboration welcome'ly yours,
> 
> holger
> 
> P.S.: i had discussed mm-encryption with Barry 2-3 years ago
> and feel now, with Autocrypt getting out the door, it's all
> becoming more feasible.

Indeed; having a specification and library for many of the details would help quite a bit.

I?ll say this; there have been countless discussions on the mailing list about whether list traffic encryption really helps or not, how easy it would be to subvert, and other related topics.  I still think it?s an interesting and useful feature that will satisfy enough use cases to make it worth working on.  Then we have to ask whether this is of general utility to make it a built-in Mailman 3 feature.  That decision can certainly be deferred because you can probably get pretty far with the much better plugin support in Mailman 3.  I?m sure you could do a pretty good PoC as a plugin.

Cheers,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20180123/fbbeded6/attachment.sig>

From holger at merlinux.eu  Thu Jan 25 06:21:27 2018
From: holger at merlinux.eu (holger krekel)
Date: Thu, 25 Jan 2018 12:21:27 +0100
Subject: [Mailman-Developers] opportunistically encrypted mailman lists
 with Autocrypt
In-Reply-To: <5F7A232A-DAE1-49CB-B34C-6888BCBCC8A7@list.org>
References: <20180121054709.GT16901@beto>
 <5F7A232A-DAE1-49CB-B34C-6888BCBCC8A7@list.org>
Message-ID: <20180125112127.GM16901@beto>

Hi Barry, all,

thanks for your encouragement and feedback!  more inline ...
also would appreciate some quick feedback on the rough
technical design at the end. 

On Tue, Jan 23, 2018 at 11:51 -0500, Barry Warsaw wrote:
> > In the last year i co-instigated a new opportunistic mail encryption
> > effort called Autocrypt (https://autocrypt.org). With Autocrypt Level 1,
> > mail clients (e.g. enigmail, K-9 mail, Delta.chat, ...) send and parse
> > public keys in "Autocrypt" e-mail headers and offer single-click
> > encryption.  Releases are upcoming in spring 2018, we have been
> > doing fun and well-received user-testing sessions already
> > with in-development versions.
> 
> Have there been any talks about creating plugins for commercial MUAs like Mail.app, Postbox, and Outlook?  Do you have plans for webmail clients like Gmail or Outlook?

We are working with Mailvelope and Roundcube developers, also have contacts
to several others. It's going to be a long game ... Getting 3-4 MUAs
to seemlessly work and have happy users will hopefully convince others
to join efforts. Autocrypt itself has no funding and does not employ
anyone at this point.

> > Also, i am considering to submit a project proposal for the
> > Mozilla Mission Partners program which would include a few more
> > things ... OnionMX routing for postfix, and documentation on
> > how to setup a best-practise MM3 mail instance, and maybe organizing
> > a sprint around the mentioned topics.
> 
> Cool.  I think for best-practices in standing up an instance, you should definitely look at Abhilash?s docker images.  A sprint would be fun for sure.  Are you thinking Pycon US 2018 or elsewhere?

I am thinking of one in Freiburg (germany, black forest) where i live :)

> I?ll say this; there have been countless discussions on the mailing list about whether list traffic encryption really helps or not, how easy it would be to subvert, and other related topics.  I still think it?s an interesting and useful feature that will satisfy enough use cases to make it worth working on.  Then we have to ask whether this is of general utility to make it a built-in Mailman 3 feature.  That decision can certainly be deferred because you can probably get pretty far with the much better plugin support in Mailman 3.  I?m sure you could do a pretty good PoC as a plugin.

Indeed, i guess a plugin should get us >90% there.  Here are my
current technical considerations in a quick list: 

- Autocrypt L1 specifies how to generate an Autocrypt key, transfer and 
  parse public keys and settings through headers of regular e-mails.  

- Mailman is to create and maintain a per-list Autocrypt account 
  (managed through "muacrypt") which keeps track of 
  subscriber's encryption settings by parsing incoming mail messages. 
  muacrypt in turn uses "gpg" or "gpg2" and would maintain 
  keyring/account data on a per-list basis.  muacrypt also implements
  mime-encryption and decryption and needs to intercept incoming/outgoing
  mails.  Can a plugin modify outgoing messages on a per-recipient basis? 

- No special interface is needed on the mailing list web page
  maybe except from enabling/disabling the plugin/support. 

- With the current Autocrypt specification, a mailman list 
  will need to have the DMARC-mitigation "replace-from" action enabled
  to a) allow the list's public key to be added to outgoing mails 
  in an Autocrypt compliant way b) send properly signed and encrypted mails
  to those subscribers which are Autocrypt-capable and have a "mutual"
  prefer-encrypt settings (see Autocrypt spec for details "prefer-encrypt").
  If we get this to work nicely i think we can see about advancing the
  Autocrypt spec to support other ML modes (i.e. not replacing From).

- For a mixed set of subscribers (some with and some without Autocrypt
  support) mailman will send out encrypted and unencrypted mails
  respectively.  The status of the group might be added to a group's
  footer so that subscribers know about the group's security status
  (and get incentivized to upgrade their e-mail program).

Does that sound sensible? And achievable through a plugin, leaving the
core (largely) untouched? 

holger

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 474 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20180125/cced0db7/attachment.sig>

From mark at msapiro.net  Tue Jan 30 15:08:01 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 30 Jan 2018 12:08:01 -0800
Subject: [Mailman-Developers] [Merge] lp:~dsieborger/mailman/recaptcha
 into lp:mailman/2.1
In-Reply-To: <151723856051.29798.2371709095739885124.launchpad@ackee.canonical.com>
References: <151723856051.29798.2371709095739885124.launchpad@ackee.canonical.com>
Message-ID: <2021988c-47e2-d645-6777-ed252f306dfb@msapiro.net>

On 01/29/2018 07:09 AM, David Sieb?rger wrote:
> David Sieb?rger has proposed merging lp:~dsieborger/mailman/recaptcha into lp:mailman/2.1.


FYI, I have made additional changes to your merged code. See
<https://bugs.launchpad.net/mailman/+bug/1746189> and
<http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1740/Mailman/Cgi/subscribe.py#Mailman/Cgi/subscribe.py>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From johny at neuromancer.sk  Wed Jan 31 09:28:23 2018
From: johny at neuromancer.sk (Jan Jancar)
Date: Wed, 31 Jan 2018 15:28:23 +0100
Subject: [Mailman-Developers] opportunistically encrypted mailman lists
 with Autocrypt
In-Reply-To: <20180125112127.GM16901@beto>
References: <20180121054709.GT16901@beto>
 <5F7A232A-DAE1-49CB-B34C-6888BCBCC8A7@list.org> <20180125112127.GM16901@beto>
Message-ID: <9e2f0d94-51cd-0145-22d5-28b510560dc2@neuromancer.sk>

Hi all!

On 01/25/2018 12:21 PM, holger krekel wrote:
> Indeed, i guess a plugin should get us >90% there.  Here are my
> current technical considerations in a quick list: 
> 
> - Autocrypt L1 specifies how to generate an Autocrypt key, transfer and 
>   parse public keys and settings through headers of regular e-mails.  
> 
> - Mailman is to create and maintain a per-list Autocrypt account 
>   (managed through "muacrypt") which keeps track of 
>   subscriber's encryption settings by parsing incoming mail messages. 
>   muacrypt in turn uses "gpg" or "gpg2" and would maintain 
>   keyring/account data on a per-list basis.  muacrypt also implements
>   mime-encryption and decryption and needs to intercept incoming/outgoing
>   mails.  Can a plugin modify outgoing messages on a per-recipient basis? 

Yes a plugin can do that, although it has to go through some hoops as
when I was making the delivery part of the plugin support I didn't want
to change Mailman's delivery mechanisms too much, which means a plugin
doing this will have some code duplication/boilerplate. As there are
only 2 plugins currently([mailman-pgp] and [mailman-rest-events]), I
think the plugin API could still evolve.

If you would like to use a Python OpenPGP implementation you could look
at [PGPy] and how I used it in mailman-pgp.
> 
> - No special interface is needed on the mailing list web page
>   maybe except from enabling/disabling the plugin/support. 

Plugin configuration is done through the Mailman configuration and those
are read-only through the REST interface. However a plugin might supply
it's own REST endpoints for example for per-list setup/configuration.

> 
> - With the current Autocrypt specification, a mailman list 
>   will need to have the DMARC-mitigation "replace-from" action enabled
>   to a) allow the list's public key to be added to outgoing mails 
>   in an Autocrypt compliant way b) send properly signed and encrypted mails
>   to those subscribers which are Autocrypt-capable and have a "mutual"
>   prefer-encrypt settings (see Autocrypt spec for details "prefer-encrypt").
>   If we get this to work nicely i think we can see about advancing the
>   Autocrypt spec to support other ML modes (i.e. not replacing From).
> 
> - For a mixed set of subscribers (some with and some without Autocrypt
>   support) mailman will send out encrypted and unencrypted mails
>   respectively.  The status of the group might be added to a group's
>   footer so that subscribers know about the group's security status
>   (and get incentivized to upgrade their e-mail program).
> 
> Does that sound sensible? And achievable through a plugin, leaving the
> core (largely) untouched? 

I definitely think so.

[mailman-pgp]: https://gitlab.com/J08nY/mailman-pgp
[mailman-rest-events]: https://gitlab.com/J08nY/mailman-rest-events
[PGPy]: https://github.com/SecurityInnovation/PGPy

Cheers,
-- 
Jan
______________________________________________________
   /\  # PGP: 362056ADA8F2F4E421565EF87F4A448FE68F329D
  /__\  # https://neuromancer.sk
 /\  /\  # Eastern Seaboard Phishing Authority
/__\/__\  #

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 862 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20180131/bd22b212/attachment.sig>

From holger at merlinux.eu  Wed Jan 31 12:27:24 2018
From: holger at merlinux.eu (holger)
Date: Wed, 31 Jan 2018 18:27:24 +0100
Subject: [Mailman-Developers] opportunistically encrypted mailman lists
 with Autocrypt
In-Reply-To: <9e2f0d94-51cd-0145-22d5-28b510560dc2@neuromancer.sk>
References: <20180121054709.GT16901@beto>
 <5F7A232A-DAE1-49CB-B34C-6888BCBCC8A7@list.org>
 <20180125112127.GM16901@beto>
 <9e2f0d94-51cd-0145-22d5-28b510560dc2@neuromancer.sk>
Message-ID: <20180131172724.GY3489@beto>

Hi Jan,

On Wed, Jan 31, 2018 at 15:28 +0100, Jan Jancar wrote:
> Hi all!
> 
> On 01/25/2018 12:21 PM, holger krekel wrote:
> > Indeed, i guess a plugin should get us >90% there.  Here are my
> > current technical considerations in a quick list: 
> > 
> > - Autocrypt L1 specifies how to generate an Autocrypt key, transfer and 
> >   parse public keys and settings through headers of regular e-mails.  
> > 
> > - Mailman is to create and maintain a per-list Autocrypt account 
> >   (managed through "muacrypt") which keeps track of 
> >   subscriber's encryption settings by parsing incoming mail messages. 
> >   muacrypt in turn uses "gpg" or "gpg2" and would maintain 
> >   keyring/account data on a per-list basis.  muacrypt also implements
> >   mime-encryption and decryption and needs to intercept incoming/outgoing
> >   mails.  Can a plugin modify outgoing messages on a per-recipient basis? 
> 
> Yes a plugin can do that, although it has to go through some hoops as
> when I was making the delivery part of the plugin support I didn't want
> to change Mailman's delivery mechanisms too much, which means a plugin
> doing this will have some code duplication/boilerplate. As there are
> only 2 plugins currently([mailman-pgp] and [mailman-rest-events]), I
> think the plugin API could still evolve.

sounds good!
 
> If you would like to use a Python OpenPGP implementation you could look
> at [PGPy] and how I used it in mailman-pgp.

It's under consideration here: https://github.com/hpk42/muacrypt/issues/32
Are your experiences with pgpy indicating it's compatible with
enigmail and k9-mail? (see the questions on that issue)

> > - No special interface is needed on the mailing list web page
> >   maybe except from enabling/disabling the plugin/support. 
> 
> Plugin configuration is done through the Mailman configuration and those
> are read-only through the REST interface. However a plugin might supply
> it's own REST endpoints for example for per-list setup/configuration.

I guess read-only REST would allow for a command line interface for 
debugging or other low-level configuration wrt to autocrypt key
status for peers.  Can a plugin add per-list configuration 
options (enable/disable, maybe a choice between 2-3 policies?)

> > - With the current Autocrypt specification, a mailman list 
> >   will need to have the DMARC-mitigation "replace-from" action enabled
> >   to a) allow the list's public key to be added to outgoing mails 
> >   in an Autocrypt compliant way b) send properly signed and encrypted mails
> >   to those subscribers which are Autocrypt-capable and have a "mutual"
> >   prefer-encrypt settings (see Autocrypt spec for details "prefer-encrypt").
> >   If we get this to work nicely i think we can see about advancing the
> >   Autocrypt spec to support other ML modes (i.e. not replacing From).
> > 
> > - For a mixed set of subscribers (some with and some without Autocrypt
> >   support) mailman will send out encrypted and unencrypted mails
> >   respectively.  The status of the group might be added to a group's
> >   footer so that subscribers know about the group's security status
> >   (and get incentivized to upgrade their e-mail program).
> > 
> > Does that sound sensible? And achievable through a plugin, leaving the
> > core (largely) untouched? 
> 
> I definitely think so.
> 
> [mailman-pgp]: https://gitlab.com/J08nY/mailman-pgp
> [mailman-rest-events]: https://gitlab.com/J08nY/mailman-rest-events
> [PGPy]: https://github.com/SecurityInnovation/PGPy

thanks a bunch for your feedback and links.

holger
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 474 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20180131/4171102e/attachment.sig>

From johny at neuromancer.sk  Wed Jan 31 13:00:59 2018
From: johny at neuromancer.sk (Jan Jancar)
Date: Wed, 31 Jan 2018 19:00:59 +0100
Subject: [Mailman-Developers] opportunistically encrypted mailman lists
 with Autocrypt
In-Reply-To: <20180131172724.GY3489@beto>
References: <20180121054709.GT16901@beto>
 <5F7A232A-DAE1-49CB-B34C-6888BCBCC8A7@list.org> <20180125112127.GM16901@beto>
 <9e2f0d94-51cd-0145-22d5-28b510560dc2@neuromancer.sk>
 <20180131172724.GY3489@beto>
Message-ID: <d04bfd56-d131-70b3-5dfc-4e8b1083f7f0@neuromancer.sk>


>> If you would like to use a Python OpenPGP implementation you could look
>> at [PGPy] and how I used it in mailman-pgp.
> 
> It's under consideration here: https://github.com/hpk42/muacrypt/issues/32
> Are your experiences with pgpy indicating it's compatible with
> enigmail and k9-mail? (see the questions on that issue)

PGPy is a quite complete OpenPGP(RFC4880) implementation, its support
table shows that:

https://pgpy.readthedocs.io/en/latest/progress.html

The unsupported packets are very rarely used nowadays and are only
really produced by very old PGP clients, afaik. PGPy also has a quite
extensive test suite that works with gpg internally.

I suggest you look at PGPy issue tracker to see what it lacks currently,
the most painful issue I think is the missing support for writing
partial length packets. It can read them just well but not output them.

> 
>>> - No special interface is needed on the mailing list web page
>>>   maybe except from enabling/disabling the plugin/support. 
>>
>> Plugin configuration is done through the Mailman configuration and those
>> are read-only through the REST interface. However a plugin might supply
>> it's own REST endpoints for example for per-list setup/configuration.
> 
> I guess read-only REST would allow for a command line interface for 
> debugging or other low-level configuration wrt to autocrypt key
> status for peers.  Can a plugin add per-list configuration 
> options (enable/disable, maybe a choice between 2-3 policies?)

Yes definitely, the configuration will be handled completely inside the
pkugin. I'm thinking along the lines of:

 - Expose a custom REST endpoint for per-list Autocrypt configuration,
that can be read-write, however is only protected by one global REST
user-password pair. That will be accessed by an Autocrypt Django app,
similarly to how I implemented configuration for mailman-pgp in
[django-pgpmailman]. That app can be then run alongside Postorius and
HyperKitty which provide Mailman's configuration and archives. So this
gives list admins a simple web UI for per-list Autocrypt configuration.

 - Provide a CLI command component which can manage the per-list
Autocrypt configuration of the Mailman instance locally.

That should make per-list configuration of Autocrypt quite nice(I did
mailman-pgp per-list config exactly this way). You can also read more on
my blog, which contains my GSoC reports and goes into a bit more depth,
https://neuromancer.sk/article/18 and other GSoC articles.

> 
>>> - With the current Autocrypt specification, a mailman list 
>>>   will need to have the DMARC-mitigation "replace-from" action enabled
>>>   to a) allow the list's public key to be added to outgoing mails 
>>>   in an Autocrypt compliant way b) send properly signed and encrypted mails
>>>   to those subscribers which are Autocrypt-capable and have a "mutual"
>>>   prefer-encrypt settings (see Autocrypt spec for details "prefer-encrypt").
>>>   If we get this to work nicely i think we can see about advancing the
>>>   Autocrypt spec to support other ML modes (i.e. not replacing From).
>>>
>>> - For a mixed set of subscribers (some with and some without Autocrypt
>>>   support) mailman will send out encrypted and unencrypted mails
>>>   respectively.  The status of the group might be added to a group's
>>>   footer so that subscribers know about the group's security status
>>>   (and get incentivized to upgrade their e-mail program).
>>>
>>> Does that sound sensible? And achievable through a plugin, leaving the
>>> core (largely) untouched? 
>>
>> I definitely think so.
>>
>> [mailman-pgp]: https://gitlab.com/J08nY/mailman-pgp
>> [mailman-rest-events]: https://gitlab.com/J08nY/mailman-rest-events
>> [PGPy]: https://github.com/SecurityInnovation/PGPy
> 
> thanks a bunch for your feedback and links.
> 
> holger
> 

Glad that I could help!

[django-pgpmailman]: https://gitlab.com/J08nY/django-pgpmailman

Cheers,
-- 
Jan
______________________________________________________
   /\  # PGP: 362056ADA8F2F4E421565EF87F4A448FE68F329D
  /__\  # https://neuromancer.sk
 /\  /\  # Eastern Seaboard Phishing Authority
/__\/__\  #

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 862 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20180131/df7354cd/attachment.sig>

From holger at merlinux.eu  Wed Jan 31 15:03:02 2018
From: holger at merlinux.eu (holger)
Date: Wed, 31 Jan 2018 21:03:02 +0100
Subject: [Mailman-Developers] opportunistically encrypted mailman lists
 with Autocrypt
In-Reply-To: <d04bfd56-d131-70b3-5dfc-4e8b1083f7f0@neuromancer.sk>
References: <20180121054709.GT16901@beto>
 <5F7A232A-DAE1-49CB-B34C-6888BCBCC8A7@list.org>
 <20180125112127.GM16901@beto>
 <9e2f0d94-51cd-0145-22d5-28b510560dc2@neuromancer.sk>
 <20180131172724.GY3489@beto>
 <d04bfd56-d131-70b3-5dfc-4e8b1083f7f0@neuromancer.sk>
Message-ID: <20180131200302.GZ3489@beto>

Hi Jan,

On Wed, Jan 31, 2018 at 19:00 +0100, Jan Jancar wrote:
> >> If you would like to use a Python OpenPGP implementation you could look
> >> at [PGPy] and how I used it in mailman-pgp.
> > 
> > It's under consideration here: https://github.com/hpk42/muacrypt/issues/32
> > Are your experiences with pgpy indicating it's compatible with
> > enigmail and k9-mail? (see the questions on that issue)
> 
> PGPy is a quite complete OpenPGP(RFC4880) implementation, its support
> table shows that:
> 
> https://pgpy.readthedocs.io/en/latest/progress.html
> 
> The unsupported packets are very rarely used nowadays and are only
> really produced by very old PGP clients, afaik. PGPy also has a quite
> extensive test suite that works with gpg internally.
> 
> I suggest you look at PGPy issue tracker to see what it lacks currently,
> the most painful issue I think is the missing support for writing
> partial length packets. It can read them just well but not output them.

Good to know! I added the links to the muacrypt issue i linked above. 
dkg and vincent (from k-9 mail) tell me write support of partial 
length header is not needed for e-mails, so that's good
because i'd like muacrypt, my in-progress autocrypt implementation,
to work with pgpy sometime (https://github.com/hpk42/muacrypt/issues/32 )

> >>> - No special interface is needed on the mailing list web page
> >>>   maybe except from enabling/disabling the plugin/support. 
> >>
> >> Plugin configuration is done through the Mailman configuration and those
> >> are read-only through the REST interface. However a plugin might supply
> >> it's own REST endpoints for example for per-list setup/configuration.
> > 
> > I guess read-only REST would allow for a command line interface for 
> > debugging or other low-level configuration wrt to autocrypt key
> > status for peers.  Can a plugin add per-list configuration 
> > options (enable/disable, maybe a choice between 2-3 policies?)
> 
> Yes definitely, the configuration will be handled completely inside the
> pkugin. I'm thinking along the lines of:
> 
>  - Expose a custom REST endpoint for per-list Autocrypt configuration,
> that can be read-write, however is only protected by one global REST
> user-password pair. That will be accessed by an Autocrypt Django app,
> similarly to how I implemented configuration for mailman-pgp in
> [django-pgpmailman]. That app can be then run alongside Postorius and
> HyperKitty which provide Mailman's configuration and archives. So this
> gives list admins a simple web UI for per-list Autocrypt configuration.
> 
>  - Provide a CLI command component which can manage the per-list
> Autocrypt configuration of the Mailman instance locally.

sounds good.  Is it also possible to hook into the standard mm3 configuration,
for adding a per-list configuration item that can then be processed by 
plugin code? 

holger

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 474 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20180131/d3729516/attachment-0001.sig>

From barry at list.org  Wed Jan 31 15:37:38 2018
From: barry at list.org (Barry Warsaw)
Date: Wed, 31 Jan 2018 15:37:38 -0500
Subject: [Mailman-Developers] opportunistically encrypted mailman lists
 with Autocrypt
In-Reply-To: <9e2f0d94-51cd-0145-22d5-28b510560dc2@neuromancer.sk>
References: <20180121054709.GT16901@beto>
 <5F7A232A-DAE1-49CB-B34C-6888BCBCC8A7@list.org> <20180125112127.GM16901@beto>
 <9e2f0d94-51cd-0145-22d5-28b510560dc2@neuromancer.sk>
Message-ID: <032743B5-8A1B-43E9-83F5-72D38BA423F0@list.org>

On Jan 31, 2018, at 09:28, Jan Jancar <johny at neuromancer.sk> wrote:

> Plugin configuration is done through the Mailman configuration and those
> are read-only through the REST interface. However a plugin might supply
> it's own REST endpoints for example for per-list setup/configuration.

Yep; configuration options are read-only through REST because they are specified in the mailman.cfg file.  You wouldn?t want to (and probably even can?t) allow REST clients to change things that aren?t kept in the database.

On Jan 31, 2018, at 15:03, holger <holger at merlinux.eu> wrote:

> sounds good.  Is it also possible to hook into the standard mm3 configuration,
> for adding a per-list configuration item that can then be processed by
> plugin code?

Per-list configurations are maintained in the database, configuration files (e.g. mailman.cfg) configure site-wide static settings.  That?s because lists are dynamic; they can be created, deleted, and modified.  But your plugin should be able to create its own tables and cross reference the mailing list tables (via the SQLAlchemy ORM).

Static configurations can be added to a [plugins.autocrypt] section, or better a separate autocrypt.cfg file, much like the way we separate out the postfix.cfg settings.

Cheers,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20180131/b05f7c8f/attachment.sig>