From mark at msapiro.net  Sun Feb  4 12:51:37 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 4 Feb 2018 09:51:37 -0800
Subject: [Mailman-Developers] Mailman 2.1.26 Security release Feb 4, 2018
In-Reply-To: <68476c0b-f481-1ff3-4cd1-0b5f01dbc5cf@msapiro.net>
References: <68476c0b-f481-1ff3-4cd1-0b5f01dbc5cf@msapiro.net>
Message-ID: <1d05c305-c440-a07b-b992-85e8bf6c3bb0@msapiro.net>

I am pleased to announce the release of Mailman 2.1.26.

Python 2.4 is the minimum supported, but Python 2.7 is strongly recommended.

This is a security and bug fix release with a couple of new features.
See the attached README.txt for details.

For those who are concerned about the security vulnerability and can't
upgrade immediately, there is a patch at
<https://bugs.launchpad.net/mailman/+bug/1747209/+attachment/5048344/+files/options.patch>
to fix the security issue. More information on the issue itself is in
the bug report at <https://bugs.launchpad.net/mailman/+bug/1747209>.

Mailman is free software for managing email mailing lists and
e-newsletters. Mailman is used for all the python.org and
SourceForge.net mailing lists, as well as at hundreds of other sites.

For more information, please see our web site at one of:

http://www.list.org
https://www.gnu.org/software/mailman
http://mailman.sourceforge.net/
https://mirror.list.org/

Mailman 2.1.26 can be downloaded from

https://launchpad.net/mailman/2.1/
https://ftp.gnu.org/gnu/mailman/
https://sourceforge.net/projects/mailman/

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan
-------------- next part --------------
2.1.26 (04-Feb-2018)

  Security

    - An XSS vulnerability in the user options CGI could allow a crafted URL
      to execute arbitrary javascript in a user's browser.  A related issue
      could expose information on a user's options page without requiring
      login.  These are fixed.  Thanks to Calum Hutton for the report.
      CVE-2018-5950  (LP: #1747209)

  New Features

    - Thanks to David Sieb?rger who adapted an existing patch by Andrea
      Veri to use Google reCAPTCHA v2 there is now the ability to add
      reCAPTCHA to the listinfo subscribe form.  There are two new mm_cfg.py
      settings for RECAPTCHA_SITE_KEY and RECAPTCHA_SECRET_KEY, the values
      for which you obtain for your domain(s) from Google at
      <https://www.google.com/recaptcha/admin>.

    - Thanks to Lindsay Haisley, there is a new bin/mailman-config command
      to display various information about this Mailman version and how it
      was configured.

  i18n

    - The Japanese message catalog has been updated for added strings by
      Yasuhito FUTATSUKI.

    - The German translation of a couple of templates has been updated by
      Thomas Hochstein.

    - The Japanese translation of Defaults.py.in has been updated by
      Yasuhito FUTATSUKI.

  Bug fixes and other patches

    - Fixed an i18n bug in the reCAPTCHA feature.  (LP: #1746189)

    - Added a few more environment variables to the list of those passed
      to CGIs to support an nginx/uwsgi configuration.  (LP #1744739)

    - Mailman 2.1.22 introduced a Python 2.7 dependency that could affect
      bin/arch processing a message without a valid Date: header.  The
      dependency has been removed.  (LP: #1740543)

    - Messages held for header_filter_rules now show the matched regexp in
      the hold reason.  (LP: #1737371)

    - When updating the group and mode of a .db file with Mailman's Postfix
      integration, a missing file is ignored.  (LP: #1734162)

    - The DELIVERY_RETRY_WAIT setting is now effective.  (LP: #1729472)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20180204/76561c18/attachment.sig>

From Hagedorn at uni-koeln.de  Wed Feb  7 07:01:55 2018
From: Hagedorn at uni-koeln.de (Sebastian Hagedorn)
Date: Wed, 07 Feb 2018 13:01:55 +0100
Subject: [Mailman-Developers] Mailman 2.1.26 Security release Feb 4, 2018
In-Reply-To: <1d05c305-c440-a07b-b992-85e8bf6c3bb0@msapiro.net>
References: <68476c0b-f481-1ff3-4cd1-0b5f01dbc5cf@msapiro.net>
 <1d05c305-c440-a07b-b992-85e8bf6c3bb0@msapiro.net>
Message-ID: <53D40D5A1AD42E71A32893DE@tyrion.rrz.uni-koeln.de>

Hi,

FWIW, I just installed 2.1.26 on a test system, and that command does not 
work for me:

$ mailman-config
  File "/usr/lib/mailman/bin/mailman-config", line 34
    print "mail_group:      %s" % ""mail", "postfix", "mailman", "nobody", 
"daemon""
                                       ^
SyntaxError: invalid syntax

Both that line and the following (for cgi_group) are double-quoted:

print "cgi_group:       %s" % ""apache""

If I remove the quotes in bin/mailman-config prior to running configure, 
the resulting script works fine:

print "mail_group:      %s" % @MAIL_GROUP@
print "cgi_group:       %s" % @CGI_GROUP@

$ build/bin/mailman-config
Configuration and build information for Mailman

Mailman version: 2.1.26
Build Date:      Wed Feb  7 12:58:18 CET 2018

prefix:          /usr/local/mailman
var_prefix:      /usr/local/mailman
mailman_user:    mailman
mailman_group:   mailman
mail_group:      mailman other mail daemon
cgi_group:       www www-data nobody

configure_opts: "--with-python=/usr/bin/python2.7 --without-permcheck"

--On 4. Februar 2018 um 09:51:37 -0800 Mark Sapiro <mark at msapiro.net> wrote:

>     - Thanks to Lindsay Haisley, there is a new bin/mailman-config command
>       to display various information about this Mailman version and how it
>       was configured.

-- 
    .:.Sebastian Hagedorn - Weyertal 121 (Geb?ude 133), Zimmer 2.02.:.
                 .:.Regionales Rechenzentrum (RRZK).:.
   .:.Universit?t zu K?ln / Cologne University - ? +49-221-470-89578.:.

From mark at msapiro.net  Wed Feb  7 13:42:45 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 7 Feb 2018 10:42:45 -0800
Subject: [Mailman-Developers] Mailman 2.1.26 Security release Feb 4, 2018
In-Reply-To: <53D40D5A1AD42E71A32893DE@tyrion.rrz.uni-koeln.de>
References: <68476c0b-f481-1ff3-4cd1-0b5f01dbc5cf@msapiro.net>
 <1d05c305-c440-a07b-b992-85e8bf6c3bb0@msapiro.net>
 <53D40D5A1AD42E71A32893DE@tyrion.rrz.uni-koeln.de>
Message-ID: <58aa1368-8525-ef8f-8a91-c8376510930f@msapiro.net>

On 02/07/2018 04:01 AM, Sebastian Hagedorn wrote:
> Hi,
> 
> FWIW, I just installed 2.1.26 on a test system, and that command does
> not work for me:
> 
> $ mailman-config
> ?File "/usr/lib/mailman/bin/mailman-config", line 34
> ?? print "mail_group:????? %s" % ""mail", "postfix", "mailman",
> "nobody", "daemon""
> ????????????????????????????????????? ^
> SyntaxError: invalid syntax
> 
> Both that line and the following (for cgi_group) are double-quoted:
> 
> print "cgi_group:?????? %s" % ""apache""
...
> configure_opts: "--with-python=/usr/bin/python2.7 --without-permcheck" 



The issue is you're running configure --without-permcheck and you
haven't specified --with-mail-gid and --with-cgi-gid.

If you don't specify those, configure looks for an existing group for
mail-gid from "mailman other mail daemon" and for cgi-gid from "www
www-data nobody". If you don't specify --without-permcheck, configure
complains if it doesn't find a group in the system from the list, but if
you do specify --without-permcheck and configure doesn't find a group it
uses the whole "mailman other mail daemon" and/or "www www-data nobody"
string which causes the issue you're seeing in mailman-config.

There will also be group mismatch errors in the configured wrappers in
this case.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From Hagedorn at uni-koeln.de  Wed Feb  7 16:38:00 2018
From: Hagedorn at uni-koeln.de (Sebastian Hagedorn)
Date: Wed, 07 Feb 2018 22:38:00 +0100
Subject: [Mailman-Developers] Mailman 2.1.26 Security release Feb 4, 2018
In-Reply-To: <58aa1368-8525-ef8f-8a91-c8376510930f@msapiro.net>
References: <68476c0b-f481-1ff3-4cd1-0b5f01dbc5cf@msapiro.net>
 <1d05c305-c440-a07b-b992-85e8bf6c3bb0@msapiro.net>
 <53D40D5A1AD42E71A32893DE@tyrion.rrz.uni-koeln.de>
 <58aa1368-8525-ef8f-8a91-c8376510930f@msapiro.net>
Message-ID: <E7F327C9A7674225AA6312F8@Sebbis-iMac.local>

>> FWIW, I just installed 2.1.26 on a test system, and that command does
>> not work for me:
>>
>> $ mailman-config
>> ?File "/usr/lib/mailman/bin/mailman-config", line 34
>> ?? print "mail_group:????? %s" % ""mail", "postfix", "mailman",
>> "nobody", "daemon""
>> ????????????????????????????????????
>> ? ^ SyntaxError: invalid syntax
>>
>> Both that line and the following (for cgi_group) are double-quoted:
>>
>> print "cgi_group:?????? %s" % ""apache""
> ...
>> configure_opts: "--with-python=/usr/bin/python2.7 --without-permcheck"
>
>
>
> The issue is you're running configure --without-permcheck and you
> haven't specified --with-mail-gid and --with-cgi-gid.
>
> If you don't specify those, configure looks for an existing group for
> mail-gid from "mailman other mail daemon" and for cgi-gid from "www
> www-data nobody". If you don't specify --without-permcheck, configure
> complains if it doesn't find a group in the system from the list, but if
> you do specify --without-permcheck and configure doesn't find a group it
> uses the whole "mailman other mail daemon" and/or "www www-data nobody"
> string which causes the issue you're seeing in mailman-config.
>
> There will also be group mismatch errors in the configured wrappers in
> this case.

Hm, part of that was an artifact of running configure manually instead of 
using the SPEC file I usually use to build Mailman. With the latter and my 
"fix" I get the following:

$ mailman-config
Configuration and build information for Mailman

Mailman version: 2.1.26
Build Date:      Wed Feb  7 13:23:45 CET 2018

prefix:          /usr/lib/mailman
var_prefix:      /var/lib/mailman
mailman_user:    mailman
mailman_group:   mailman
mail_group:      mail postfix mailman nobody daemon
cgi_group:       apache

configure_opts: "--prefix=/usr/lib/mailman 
--with-var-prefix=/var/lib/mailman --with-config-dir=/etc/mailman 
--with-lock-dir=/var/lock/mailman --with-log-dir=/var/log/mailman 
--with-pid-dir=/var/run/mailman --with-queue-dir=/var/spool/mailman 
--with-python=/usr/bin/python2.7 --with-mail-gid=mail postfix mailman 
nobody daemon --with-cgi-id=apache --with-cgi-gid=apache 
--with-mailhost=localhost.localdomain --with-urlhost=localhost.localdomain 
--without-permcheck"

So it's still using --without-permcheck, but the other options are there.
--
Sebastian Hagedorn - Weyertal 121, Zimmer 2.02
Regionales Rechenzentrum (RRZK)
Universit?t zu K?ln / Cologne University - Tel. +49-221-470-89578

From mark at msapiro.net  Wed Feb  7 18:22:24 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 7 Feb 2018 15:22:24 -0800
Subject: [Mailman-Developers] Mailman 2.1.26 Security release Feb 4, 2018
In-Reply-To: <E7F327C9A7674225AA6312F8@Sebbis-iMac.local>
References: <68476c0b-f481-1ff3-4cd1-0b5f01dbc5cf@msapiro.net>
 <1d05c305-c440-a07b-b992-85e8bf6c3bb0@msapiro.net>
 <53D40D5A1AD42E71A32893DE@tyrion.rrz.uni-koeln.de>
 <58aa1368-8525-ef8f-8a91-c8376510930f@msapiro.net>
 <E7F327C9A7674225AA6312F8@Sebbis-iMac.local>
Message-ID: <44ba5838-54a2-0d9d-930a-d25dedec5973@msapiro.net>

On 02/07/2018 01:38 PM, Sebastian Hagedorn wrote:
> 
> Hm, part of that was an artifact of running configure manually instead
> of using the SPEC file I usually use to build Mailman. With the latter
> and my "fix" I get the following:
> 
> $ mailman-config
> Configuration and build information for Mailman
> 
> Mailman version: 2.1.26
> Build Date:????? Wed Feb? 7 13:23:45 CET 2018
> 
> prefix:????????? /usr/lib/mailman
> var_prefix:????? /var/lib/mailman
> mailman_user:??? mailman
> mailman_group:?? mailman
> mail_group:????? mail postfix mailman nobody daemon
> cgi_group:?????? apache
> 
> configure_opts: "--prefix=/usr/lib/mailman
> --with-var-prefix=/var/lib/mailman --with-config-dir=/etc/mailman
> --with-lock-dir=/var/lock/mailman --with-log-dir=/var/log/mailman
> --with-pid-dir=/var/run/mailman --with-queue-dir=/var/spool/mailman
> --with-python=/usr/bin/python2.7 --with-mail-gid=mail postfix mailman
> nobody daemon --with-cgi-id=apache --with-cgi-gid=apache
> --with-mailhost=localhost.localdomain
> --with-urlhost=localhost.localdomain --without-permcheck"
> 
> So it's still using --without-permcheck, but the other options are there.


When I run that command without having made any changes in the unpacked
tarball on a machine without a 'mailman' user, but with a 'mail' group,
I get this from configure

configure: WARNING: unrecognized options: --with-config-dir,
--with-lock-dir, --with-log-dir, --with-pid-dir, --with-queue-dir,
--with-cgi-id

This is expected because those options to configure were added by RedHat
as part of their FHS compliance patch. See
<https://wiki.list.org/x/8486953> and
<https://mail.python.org/pipermail/mailman-developers/2004-October/017343.html>

I get this from the bin/mailman-config command

Configuration and build information for Mailman

Mailman version: 2.1.26
Build Date:      Wed Feb  7 14:19:11 PST 2018

prefix:          /usr/lib/mailman
var_prefix:      /var/lib/mailman
mailman_user:
mailman_group:
mail_group:      mail
cgi_group:       apache

configure_opts: "--prefix=/usr/lib/mailman
--with-var-prefix=/var/lib/mailman --with-config-dir=/etc/mailman
--with-lock-dir=/var/lock/mailman --with-log-dir=/var/log/mailman
--with-pid-dir=/var/run/mailman --with-queue-dir=/var/spool/mailman
--with-python=/usr/bin/python2.7 --with-mail-gid=mail postfix mailman
nobody daemon --with-cgi-id=apache --with-cgi-gid=apache
--with-mailhost=localhost.localdomain
--with-urlhost=localhost.localdomain --without-permcheck"

The empty mailman_user and mailman_group is because there is no
'mailman' user/group on the system I ran it on and the fact that I get
mail_group = 'mail' rather than 'mail postfix mailman nobody daemon' is
because there is a 'mail' group (it picks the first group that exists
from that list and only yields the whole list as the result if none exist.

I suspect that your actual configure command options rather than the
ones reported by your bin/mailman-config are something like

--prefix=/usr/lib/mailman --with-var-prefix=/var/lib/mailman
--with-config-dir=/etc/mailman --with-lock-dir=/var/lock/mailman
--with-log-dir=/var/log/mailman --with-pid-dir=/var/run/mailman
--with-queue-dir=/var/spool/mailman --with-python=/usr/bin/python2.7
--with-mail-gid='"mail postfix mailman nobody daemon"'
--with-cgi-id=apache --with-cgi-gid=\"apache\"
--with-mailhost=localhost.localdomain
--with-urlhost=localhost.localdomain --without-permcheck

and that's where the extraneous quotes are coming from. I think the
passing of a list to --with-mail-gid relies on another RedHat
modification to the mail wrapper to be able to dynamically configure the
mail group and not build it in to the RPM.

The question is what are you trying to do.

If you just want to build a working Mailman 2.1.26 installation, I
suggest removing the --with-config-dir, --with-lock-dir, --with-log-dir,
--with-pid-dir, --with-queue-dir, --with-cgi-id options and setting
--with-mail-gid and --with-cgi-gid to the appropriate single groups
without any quotes.

If you are trying to build a RHEL FHS compliant Mailman, start by
porting the patch in the attachment to
<https://mail.python.org/pipermail/mailman-developers/2004-October/017343.html>
and applying it, running autoconf to regenerate configure from
configure.in and take any resultant issues to RedHat. Note, the last I
knew, John Dennis was still at RedHat, but was no longer working with
Mailman, but that was a long time ago. In case it isn't obvious, I
recommend the first approach.

If you're just trying to fix CVE-2018-5950, just apply the patch
attached to <https://bugs.launchpad.net/mailman/+bug/1747209>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From okan at demirmen.com  Wed Feb  7 19:10:26 2018
From: okan at demirmen.com (Okan Demirmen)
Date: Wed, 7 Feb 2018 19:10:26 -0500
Subject: [Mailman-Developers] mailman 2.x user/group name
Message-ID: <20180208001026.GA2143@carbon.khaoz.org>

Hi all,

I noticed a conversation on the recent thread releasing 2.1.26, but
wanted to start a new thread that'll hopefully stay on-topic.

Because the mailman configure requires the user and group to exist prior
to running, as well as hard coding the actual values once run, it
becomes very hard for porters/packagers when trying to support multiple
mail servers and their respective user/group configurations (sendmail,
postfix, exim, openstmpd, etc); and additionally the various web
servers. Either we have to create a package for every combination or
come up with another solution.

What I did for the OpenBSD ports tree was to rewrite check_caller() in
src/common.c (with src/common.h addition). It allows one to set the
username, groupname, cgi-gid and mail-gid all via configure without
these having to be setup beforehand. From the log message:

- Rewrite src/common.c:check_caller() for the cgi/mail wrapper to now                               
  look at the defined group membership instead, _mailmanq (a new group).                            
  This allows the administrator to switch mail servers and web servers                              
  without requiring a FLAVOR for each combination; but rather, by simply                            
  adding the cgi/mail user to the _mailmanq group. This is a diversion                              
  from upstream, but will be proposed. At least sthen@ and dlg@ agree to                            
  go in this general direction.

Here as well:
https://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/mail/mailman/patches/patch-src_common_c
https://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/mail/mailman/patches/patch-src_common_h

In the OpenBSD ports tree, configure is now run with:
	--with-username=_mailman \
	--with-groupname=_mailman \
	--with-cgi-gid=_mailmanq \
	--with-mail-gid=_mailmanq

and the admin can change/migrate mail servers as well as web servers by
just mucking with group memberships; no need to rebuild - allows mailman
to be packaged basically.

I realize now I never sent this upstream, so better late than never :)

Inline patch to src/common.[ch] to follow, hopefully in the same style
as the original.

Considerations, thoughts?

Thanks,
Okan

$OpenBSD: patch-src_common_h,v 1.1 2016/04/22 16:42:14 okan Exp $
--- src/common.h.orig	Sun Mar 20 13:48:18 2016
+++ src/common.h	Sun Mar 20 13:53:00 2016
@@ -27,6 +27,7 @@
 #include <errno.h>
 #include <sys/types.h>
 #include <grp.h>
+#include <pwd.h>
 #include <unistd.h>
 
 /* GETGROUPS_T gets set in the makefile by configure */
@@ -52,6 +53,7 @@ extern const char* logident;
 #define MAIL_ILLEGAL_COMMAND 6
 #define ADDALIAS_USAGE_ERROR 7
 #define GROUP_NAME_NOT_FOUND 8
+#define USER_NAME_NOT_FOUND 9
 
 
 /*
$OpenBSD: patch-src_common_c,v 1.1 2016/04/22 16:42:14 okan Exp $
--- src/common.c.orig	Sun Feb 28 15:47:44 2016
+++ src/common.c	Sun Mar 20 16:22:35 2016
@@ -119,45 +119,39 @@ fatal(const char* ident, int exitcode, char* format, .
 void
 check_caller(const char* ident, const char* parentgroup)
 {
-        GID_T mygid = getgid();
-        struct group *mygroup = getgrgid(mygid);
-        char* option;
-        char* server;
+	struct passwd *pw;
+	struct group *gr;
+	char **g;
+	int ok = 0;
         char* wrapper;
 
-        if (running_as_cgi) {
-                option = "--with-cgi-gid";
-                server = "web";
-                wrapper = "CGI";
-        }
-        else {
-                option = "--with-mail-gid";
-                server = "mail";
-                wrapper = "mail";
-        }
+	pw = getpwuid(getuid());
+	if (pw == NULL)
+		fatal(ident, USER_NAME_NOT_FOUND,
+		      "Failure to find username");
 
-        if (!mygroup)
-                fatal(ident, GROUP_NAME_NOT_FOUND,
-                      "Failure to find group name for GID %d.  Mailman\n"
-                      "expected the %s wrapper to be executed as group\n"
-                      "\"%s\", but the system's %s server executed the\n"
-                      "wrapper as GID %d for which the name could not be\n"
-                      "found.  Try adding GID %d to your system as \"%s\",\n"
-                      "or tweak your %s server to run the wrapper as group\n"
-                      "\"%s\".",
-                      mygid, wrapper, parentgroup, server, mygid, mygid,
-                      parentgroup, server, parentgroup);
+	gr = getgrnam(parentgroup);
+	if (gr == NULL)
+		fatal(ident, GROUP_NAME_NOT_FOUND,
+		      "Failure to find \"%s\" group", parentgroup);
 
-        if (strcmp(parentgroup, mygroup->gr_name))
-                fatal(ident, GROUP_MISMATCH,
-                      "Group mismatch error.  Mailman expected the %s\n"
-                      "wrapper script to be executed as group \"%s\", but\n"
-                      "the system's %s server executed the %s script as\n"
-                      "group \"%s\".  Try tweaking the %s server to run the\n"
-                      "script as group \"%s\", or re-run configure, \n"
-                      "providing the command line option `%s=%s'.",
-                      wrapper, parentgroup, server, wrapper, mygroup->gr_name,
-                      server, parentgroup, option, mygroup->gr_name);
+	for (g = gr->gr_mem; *g; g++) {
+		if (strcmp(pw->pw_name, *g) == 0) {
+			ok = 1;
+			break;
+		}
+	}
+
+	if (running_as_cgi)
+		wrapper = "CGI";
+	else
+		wrapper = "mail";
+
+	if (ok == 0)
+		fatal(ident, GROUP_MISMATCH,
+		      "Group mismatch error.  Mailman expected the %s\n"
+		      "wrapper script to be executed by a member of\n"
+		      "\"%s\" group.", wrapper, parentgroup);
 }
 
 

From mark at msapiro.net  Wed Feb  7 22:10:39 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 7 Feb 2018 19:10:39 -0800
Subject: [Mailman-Developers] mailman 2.x user/group name
In-Reply-To: <20180208001026.GA2143@carbon.khaoz.org>
References: <20180208001026.GA2143@carbon.khaoz.org>
Message-ID: <60c82803-abf5-06c6-c8d2-b5f39a7bf5ec@msapiro.net>

On 02/07/2018 04:10 PM, Okan Demirmen wrote:
> 
> Because the mailman configure requires the user and group to exist prior
> to running, as well as hard coding the actual values once run, it
> becomes very hard for porters/packagers when trying to support multiple
> mail servers and their respective user/group configurations (sendmail,
> postfix, exim, openstmpd, etc); and additionally the various web
> servers. Either we have to create a package for every combination or
> come up with another solution.


This is a common problem faced by all downstream packagers. While I
appreciate your sharing your solution, at this point in the lifecycle of
Mailman 2.1, I think most if not all downstream packagers have addressed
this in their own similar but incompatible ways.

I think for the Mailman 2.1 developers (read "me") to try to implement a
solution upstream at this point would help only the one downstream that
was already doing it that way and just make more work for the others.

Note that there are many packaging issues to be faced for Mailman 3
(Debian for one is working on it and the GNU Mailman project is
distributing Docker containers), but at least there are no SETGID
compiled wrappers and group mismatch errors to contend with.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From Hagedorn at uni-koeln.de  Thu Feb  8 05:32:39 2018
From: Hagedorn at uni-koeln.de (Sebastian Hagedorn)
Date: Thu, 08 Feb 2018 11:32:39 +0100
Subject: [Mailman-Developers] Mailman 2.1.26 Security release Feb 4, 2018
In-Reply-To: <44ba5838-54a2-0d9d-930a-d25dedec5973@msapiro.net>
References: <68476c0b-f481-1ff3-4cd1-0b5f01dbc5cf@msapiro.net>
 <1d05c305-c440-a07b-b992-85e8bf6c3bb0@msapiro.net>
 <53D40D5A1AD42E71A32893DE@tyrion.rrz.uni-koeln.de>
 <58aa1368-8525-ef8f-8a91-c8376510930f@msapiro.net>
 <E7F327C9A7674225AA6312F8@Sebbis-iMac.local>
 <44ba5838-54a2-0d9d-930a-d25dedec5973@msapiro.net>
Message-ID: <810914DEEA59BBA48D213CB2@tyrion.rrz.uni-koeln.de>

Thank you for this detailed analysis that goes far above and beyond what 
one can reasonably expect! I did not expect the issue to be caused by the 
patches specific to Red Hat, otherwise I wouldn't have bothered you with 
this.

--On 7. Februar 2018 um 15:22:24 -0800 Mark Sapiro <mark at msapiro.net> wrote:

> On 02/07/2018 01:38 PM, Sebastian Hagedorn wrote:
>>
>> Hm, part of that was an artifact of running configure manually instead
>> of using the SPEC file I usually use to build Mailman. With the latter
>> and my "fix" I get the following:
>>
>> $ mailman-config
>> Configuration and build information for Mailman
>>
>> Mailman version: 2.1.26
>> Build Date:????? Wed Feb? 7 13:23:45 CET 2018
>>
>> prefix:????????? /usr/lib/mailman
>> var_prefix:????? /var/lib/mailman
>> mailman_user:??? mailman
>> mailman_group:?? mailman
>> mail_group:????? mail postfix mailman nobody daemon
>> cgi_group:?????? apache
>>
>> configure_opts: "--prefix=/usr/lib/mailman
>> --with-var-prefix=/var/lib/mailman --with-config-dir=/etc/mailman
>> --with-lock-dir=/var/lock/mailman --with-log-dir=/var/log/mailman
>> --with-pid-dir=/var/run/mailman --with-queue-dir=/var/spool/mailman
>> --with-python=/usr/bin/python2.7 --with-mail-gid=mail postfix mailman
>> nobody daemon --with-cgi-id=apache --with-cgi-gid=apache
>> --with-mailhost=localhost.localdomain
>> --with-urlhost=localhost.localdomain --without-permcheck"
>>
>> So it's still using --without-permcheck, but the other options are there.
>
>
> When I run that command without having made any changes in the unpacked
> tarball on a machine without a 'mailman' user, but with a 'mail' group,
> I get this from configure
>
> configure: WARNING: unrecognized options: --with-config-dir,
> --with-lock-dir, --with-log-dir, --with-pid-dir, --with-queue-dir,
> --with-cgi-id
>
> This is expected because those options to configure were added by RedHat
> as part of their FHS compliance patch. See
> <https://wiki.list.org/x/8486953> and
> <https://mail.python.org/pipermail/mailman-developers/2004-October/017343
> .html>

Right, we've been using the Red Hat patches for many years. Even on RHEL 7 
Red Hat only provides an RPM for 2.1.15, so at some point I took the SPEC 
file from their source RPM and have been updating it myself.

> I get this from the bin/mailman-config command
>
> Configuration and build information for Mailman
>
> Mailman version: 2.1.26
> Build Date:      Wed Feb  7 14:19:11 PST 2018
>
> prefix:          /usr/lib/mailman
> var_prefix:      /var/lib/mailman
> mailman_user:
> mailman_group:
> mail_group:      mail
> cgi_group:       apache
>
> configure_opts: "--prefix=/usr/lib/mailman
> --with-var-prefix=/var/lib/mailman --with-config-dir=/etc/mailman
> --with-lock-dir=/var/lock/mailman --with-log-dir=/var/log/mailman
> --with-pid-dir=/var/run/mailman --with-queue-dir=/var/spool/mailman
> --with-python=/usr/bin/python2.7 --with-mail-gid=mail postfix mailman
> nobody daemon --with-cgi-id=apache --with-cgi-gid=apache
> --with-mailhost=localhost.localdomain
> --with-urlhost=localhost.localdomain --without-permcheck"
>
> The empty mailman_user and mailman_group is because there is no
> 'mailman' user/group on the system I ran it on and the fact that I get
> mail_group = 'mail' rather than 'mail postfix mailman nobody daemon' is
> because there is a 'mail' group (it picks the first group that exists
> from that list and only yields the whole list as the result if none exist.
>
> I suspect that your actual configure command options rather than the
> ones reported by your bin/mailman-config are something like
>
> --prefix=/usr/lib/mailman --with-var-prefix=/var/lib/mailman
> --with-config-dir=/etc/mailman --with-lock-dir=/var/lock/mailman
> --with-log-dir=/var/log/mailman --with-pid-dir=/var/run/mailman
> --with-queue-dir=/var/spool/mailman --with-python=/usr/bin/python2.7
> --with-mail-gid='"mail postfix mailman nobody daemon"'
> --with-cgi-id=apache --with-cgi-gid=\"apache\"
> --with-mailhost=localhost.localdomain
> --with-urlhost=localhost.localdomain --without-permcheck
>
> and that's where the extraneous quotes are coming from. I think the
> passing of a list to --with-mail-gid relies on another RedHat
> modification to the mail wrapper to be able to dynamically configure the
> mail group and not build it in to the RPM.

In our SPEC file it looks like this:

./configure \
        --prefix=%{mmdir} \
        --with-var-prefix=%{varmmdir} \
        --with-config-dir=%{configdir} \
        --with-lock-dir=%{lockdir} \
        --with-log-dir=%{logdir} \
        --with-pid-dir=%{piddir} \
        --with-queue-dir=%{queuedir} \
        --with-python=/usr/bin/python2.7 \
        --with-mail-gid=%{mailgroup} \
        --with-cgi-id=%{cgiuser} \
        --with-cgi-gid=%{cgigroup} \
        --with-mailhost=localhost.localdomain \
        --with-urlhost=localhost.localdomain \
        --without-permcheck

The variables in question are defined like this:

# Now, the groups your mail spoolers run as.  Sendmail uses 'mail'(12)
# and postfix used to use 'nobody', but now uses 'postfix'
%define mailgroup  "mail postfix mailman nobody daemon"

# Now, the user and group the CGIs will expect to be run under.  This should
# match the user and group the web server is configured to run as.  The 
scripts
# will error out if they are invoked by any other user.
%define cgiuser    apache
%define cgigroup   apache

Which explains the quotes for mailgroup, although I don't understand where 
the quotes for cgigroup come from ...

> The question is what are you trying to do.
>
> If you just want to build a working Mailman 2.1.26 installation, I
> suggest removing the --with-config-dir, --with-lock-dir, --with-log-dir,
> --with-pid-dir, --with-queue-dir, --with-cgi-id options and setting
> --with-mail-gid and --with-cgi-gid to the appropriate single groups
> without any quotes.

That would be fine for a new installation but I'm wary of such a change on 
an active system. In hindsight we should never have used the RPMs provided 
by Red Hat in the first place.

> If you are trying to build a RHEL FHS compliant Mailman, start by
> porting the patch in the attachment to
> <https://mail.python.org/pipermail/mailman-developers/2004-October/017343
> .html> and applying it, running autoconf to regenerate configure from
> configure.in

That's what we're already doing.

> and take any resultant issues to RedHat.

That part is useless, unfortunately.

> Note, the last I
> knew, John Dennis was still at RedHat, but was no longer working with
> Mailman, but that was a long time ago. In case it isn't obvious, I
> recommend the first approach.

As I mentioned I "fixed" this by removing the quotes around @MAIL_GROUP@ 
and @CGI_GROUP at . I have included that patch in our SPEC file with a note 
not to bother the community with that issue.
-- 
    .:.Sebastian Hagedorn - Weyertal 121 (Geb?ude 133), Zimmer 2.02.:.
                 .:.Regionales Rechenzentrum (RRZK).:.
   .:.Universit?t zu K?ln / Cologne University - ? +49-221-470-89578.:.

From okan at demirmen.com  Thu Feb  8 08:44:47 2018
From: okan at demirmen.com (Okan Demirmen)
Date: Thu, 8 Feb 2018 08:44:47 -0500
Subject: [Mailman-Developers] mailman 2.x user/group name
In-Reply-To: <60c82803-abf5-06c6-c8d2-b5f39a7bf5ec@msapiro.net>
References: <20180208001026.GA2143@carbon.khaoz.org>
 <60c82803-abf5-06c6-c8d2-b5f39a7bf5ec@msapiro.net>
Message-ID: <20180208134447.GA7434@carbon.khaoz.org>

On Wed 2018.02.07 at 19:10 -0800, Mark Sapiro wrote:
> On 02/07/2018 04:10 PM, Okan Demirmen wrote:
> > 
> > Because the mailman configure requires the user and group to exist prior
> > to running, as well as hard coding the actual values once run, it
> > becomes very hard for porters/packagers when trying to support multiple
> > mail servers and their respective user/group configurations (sendmail,
> > postfix, exim, openstmpd, etc); and additionally the various web
> > servers. Either we have to create a package for every combination or
> > come up with another solution.
> 
> 
> This is a common problem faced by all downstream packagers. While I
> appreciate your sharing your solution, at this point in the lifecycle of
> Mailman 2.1, I think most if not all downstream packagers have addressed
> this in their own similar but incompatible ways.
> 
> I think for the Mailman 2.1 developers (read "me") to try to implement a
> solution upstream at this point would help only the one downstream that
> was already doing it that way and just make more work for the others.
> 
> Note that there are many packaging issues to be faced for Mailman 3
> (Debian for one is working on it and the GNU Mailman project is
> distributing Docker containers), but at least there are no SETGID
> compiled wrappers and group mismatch errors to contend with.

I agree. There's probably not much use in fleshing out a solution for a
version that's on the edge of retirement. I just noticed the
conversation and shared my solution (since it seems I never did when I
said was going to!) - I'm sure there's a pile of others that equally
work.

Thanks for your comments.

From mark at msapiro.net  Fri Feb 23 12:14:30 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 23 Feb 2018 09:14:30 -0800
Subject: [Mailman-Developers] Gitlab CI stuck.
Message-ID: <9b1eb734-7461-daaf-c21e-a927760ca774@msapiro.net>

I'm seeing

This job is stuck, because you don't have any active runners that can
run this job.
Go to Runners page

I go to the runners page and I see



Runners activated for this project

    678886ce

#9640

docker-runner: using maxking/mailman-ci-runner image. Has Mysql and
Postgresql images linked.


The status circle left of 678886ce is grey and if I hover over it I see
"Runner is offline, last contact was 10 days ago"


-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From maxking at asynchronous.in  Fri Feb 23 12:17:23 2018
From: maxking at asynchronous.in (Abhilash Raj)
Date: Fri, 23 Feb 2018 09:17:23 -0800
Subject: [Mailman-Developers] Gitlab CI stuck.
In-Reply-To: <9b1eb734-7461-daaf-c21e-a927760ca774@msapiro.net>
References: <9b1eb734-7461-daaf-c21e-a927760ca774@msapiro.net>
Message-ID: <1519406243.3795467.1281161648.67F8A325@webmail.messagingengine.com>

On Fri, Feb 23, 2018, at 9:14 AM, Mark Sapiro wrote:
> I'm seeing
> 
> This job is stuck, because you don't have any active runners that can
> run this job.
> Go to Runners page
> 
> I go to the runners page and I see


Can you please Shared-Runners for your repo?

The CI machine has been down for a few days and we don't know why. I (and Barry) have been using the Shared Runners.

thanks,
Abhilash 

> 
> 
> 
> Runners activated for this project
> 
>     678886ce
> 
> #9640
> 
> docker-runner: using maxking/mailman-ci-runner image. Has Mysql and
> Postgresql images linked.
> 
> 
> The status circle left of 678886ce is grey and if I hover over it I see
> "Runner is offline, last contact was 10 days ago"
> 
> 
> -- 
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers at python.org
> https://mail.python.org/mailman/listinfo/mailman-developers
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Searchable Archives: 
> http://www.mail-archive.com/mailman-developers%40python.org/
> Unsubscribe: 
> https://mail.python.org/mailman/options/mailman-developers/raj.abhilash1%40gmail.com
> 
> Security Policy: http://wiki.list.org/x/QIA9


-- 
  Abhilash Raj
  maxking at asynchronous.in

From mark at msapiro.net  Fri Feb 23 13:57:35 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 23 Feb 2018 10:57:35 -0800
Subject: [Mailman-Developers] Gitlab CI stuck.
In-Reply-To: <1519406243.3795467.1281161648.67F8A325@webmail.messagingengine.com>
References: <9b1eb734-7461-daaf-c21e-a927760ca774@msapiro.net>
 <1519406243.3795467.1281161648.67F8A325@webmail.messagingengine.com>
Message-ID: <0e76d45c-f278-a430-fa67-8b20aa70bceb@msapiro.net>

On 02/23/2018 09:17 AM, Abhilash Raj wrote:
> 
> Can you please Shared-Runners for your repo?
> 
> The CI machine has been down for a few days and we don't know why. I (and Barry) have been using the Shared Runners.


I've now enabled shared runners and CI is now running.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Fri Feb 23 15:10:18 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 23 Feb 2018 12:10:18 -0800
Subject: [Mailman-Developers] Gitlab CI failure
Message-ID: <2f2c278f-5022-7aeb-948e-218046aabe30@msapiro.net>

I enabled shared runners and CI now runs, but I get a failure only on
pgsql. This appears consistent, i.e. the same tests failed in both
py35-nocov-pg and py36-nocov-pg on two separate runs, apparently because
of messages like

    +
/builds/msapiro/mailman/.tox/py36-nocov-pg/lib/python3.6/site-packages/psycopg2/__init__.py:144:
UserWarning: The psycopg2 wheel package will be renamed from release
2.8; in order to keep installing from binary please use "pip install
psycopg2-binary" instead. For details see:
<http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
    +   """)

I don't see this failure when I run py35-nocov-pg locally even though
both my local tox and CI install psycopg2==2.7.4

I am at a loss as to how to proceed.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Fri Feb 23 15:27:51 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 23 Feb 2018 12:27:51 -0800
Subject: [Mailman-Developers] Gitlab CI failure
In-Reply-To: <2f2c278f-5022-7aeb-948e-218046aabe30@msapiro.net>
References: <2f2c278f-5022-7aeb-948e-218046aabe30@msapiro.net>
Message-ID: <5576d602-4ba5-1f34-c426-d13785ff29f4@msapiro.net>

On 02/23/2018 12:10 PM, Mark Sapiro wrote:
> 
> I am at a loss as to how to proceed.


Actually, I was only lost when I grepped for psycopg2 to see where it
was required and somehow overlooked tox.ini.

I'm going to try again with requiring psycopg2-binary. Local test
running now. I'll push to gitlab when it passes.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Feb 25 21:03:46 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 25 Feb 2018 18:03:46 -0800
Subject: [Mailman-Developers] Postorius, Hyperkitty and python support
Message-ID: <a9dae172-c173-4d88-25c2-545fdd478c17@msapiro.net>

What is the state of the current Postorius, Hyperkitty and
django_mailman3 branches with respect to Python version support.

I am trying to do some things with Postorius at the moment and with
Python 2 I get import errors, specifically on

from urllib.error import HTTPError

which seems to work only in Python 3.

Are these supposed to be bi-lingual or Python 3 only, or are they just
unstable and not working at the moment?

My immediate concern is to try to bring
<https://gitlab.com/mailman/postorius/merge_requests/186> up to date. If
I run tox on an unmodified gitlab head, It only runs py35 tests (it
would run py36, but I don't have that installed) and they all fail like
this:

ERROR: test_middleware_request
(postorius.tests.mailman_api_tests.test_middleware.TestMiddleware)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site-packages/mock/mock.py",
line 1305, in patched
    return func(*args, **keywargs)
  File
"/var/MM/3/master/postorius/src/postorius/tests/mailman_api_tests/test_middleware.py",
line 47, in test_middleware_request
    response.context['error'])
  File
"/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site-packages/django/test/utils.py",
line 73, in __getitem__
    raise KeyError(key)
KeyError: 'error'


(that from py35-djangolatest, but py35-django111 and py35-django20 have
the same failure and there are no py27 tests defined.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From maxking at asynchronous.in  Sun Feb 25 21:30:55 2018
From: maxking at asynchronous.in (Abhilash Raj)
Date: Sun, 25 Feb 2018 18:30:55 -0800
Subject: [Mailman-Developers] Postorius, Hyperkitty and python support
In-Reply-To: <a9dae172-c173-4d88-25c2-545fdd478c17@msapiro.net>
References: <a9dae172-c173-4d88-25c2-545fdd478c17@msapiro.net>
Message-ID: <1519612255.2561977.1283111552.28C74042@webmail.messagingengine.com>

On Sun, Feb 25, 2018, at 6:03 PM, Mark Sapiro wrote:
> What is the state of the current Postorius, Hyperkitty and
> django_mailman3 branches with respect to Python version support.

Postorius and django-mailman3 are both now ported to Python3 and supports only Django 1.11+

Hyperkitty's python3 branch is ready and I will merge it soon.

> 
> I am trying to do some things with Postorius at the moment and with
> Python 2 I get import errors, specifically on
> 
> from urllib.error import HTTPError
> 
> which seems to work only in Python 3.
> 
> Are these supposed to be bi-lingual or Python 3 only, or are they just
> unstable and not working at the moment?

They are (or soon going to be) all Python 3 only.

> 
> My immediate concern is to try to bring
> <https://gitlab.com/mailman/postorius/merge_requests/186> up to date. If
> I run tox on an unmodified gitlab head, It only runs py35 tests (it
> would run py36, but I don't have that installed) and they all fail like
> this:
> 
> ERROR: test_middleware_request
> (postorius.tests.mailman_api_tests.test_middleware.TestMiddleware)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
> "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site-
> packages/mock/mock.py",
> line 1305, in patched
>     return func(*args, **keywargs)
>   File
> "/var/MM/3/master/postorius/src/postorius/tests/mailman_api_tests/
> test_middleware.py",
> line 47, in test_middleware_request
>     response.context['error'])
>   File
> "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site-
> packages/django/test/utils.py",
> line 73, in __getitem__
>     raise KeyError(key)
> KeyError: 'error'
> 
> 
> (that from py35-djangolatest, but py35-django111 and py35-django20 have
> the same failure and there are no py27 tests defined.

I will go check out this and see if I can reproduce this on my machine. I think I know the reason behind this, it is mostly because of old or new style middlewares in Django.

The CI passed properly when I merged the branch though.

-- 
  Abhilash Raj
  maxking at asynchronous.in

From barry at python.org  Sun Feb 25 21:33:06 2018
From: barry at python.org (Barry Warsaw)
Date: Sun, 25 Feb 2018 18:33:06 -0800
Subject: [Mailman-Developers] Postorius, Hyperkitty and python support
In-Reply-To: <1519612255.2561977.1283111552.28C74042@webmail.messagingengine.com>
References: <a9dae172-c173-4d88-25c2-545fdd478c17@msapiro.net>
 <1519612255.2561977.1283111552.28C74042@webmail.messagingengine.com>
Message-ID: <E9C48743-589A-4FC7-AE4F-82A682932486@python.org>

On Feb 25, 2018, at 18:30, Abhilash Raj <maxking at asynchronous.in> wrote:
>> Are these supposed to be bi-lingual or Python 3 only, or are they just
>> unstable and not working at the moment?
> 
> They are (or soon going to be) all Python 3 only.

How awesome is that!  Once this lands, we should definitely advertise this on the social medias.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20180225/0a62eeae/attachment.sig>

From maxking at asynchronous.in  Sun Feb 25 23:14:18 2018
From: maxking at asynchronous.in (Abhilash Raj)
Date: Sun, 25 Feb 2018 20:14:18 -0800
Subject: [Mailman-Developers] Postorius, Hyperkitty and python support
In-Reply-To: <a9dae172-c173-4d88-25c2-545fdd478c17@msapiro.net>
References: <a9dae172-c173-4d88-25c2-545fdd478c17@msapiro.net>
Message-ID: <1519618458.2597668.1283178192.3827056D@webmail.messagingengine.com>

On Sun, Feb 25, 2018, at 6:03 PM, Mark Sapiro wrote:
> What is the state of the current Postorius, Hyperkitty and
> django_mailman3 branches with respect to Python version support.
> 
> I am trying to do some things with Postorius at the moment and with
> Python 2 I get import errors, specifically on
> 
> from urllib.error import HTTPError
> 
> which seems to work only in Python 3.
> 
> Are these supposed to be bi-lingual or Python 3 only, or are they just
> unstable and not working at the moment?
> 
> My immediate concern is to try to bring
> <https://gitlab.com/mailman/postorius/merge_requests/186> up to date. If
> I run tox on an unmodified gitlab head, It only runs py35 tests (it
> would run py36, but I don't have that installed) and they all fail like
> this:
> 
> ERROR: test_middleware_request
> (postorius.tests.mailman_api_tests.test_middleware.TestMiddleware)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File
> "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site-
> packages/mock/mock.py",
> line 1305, in patched
>     return func(*args, **keywargs)
>   File
> "/var/MM/3/master/postorius/src/postorius/tests/mailman_api_tests/
> test_middleware.py",
> line 47, in test_middleware_request
>     response.context['error'])
>   File
> "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site-
> packages/django/test/utils.py",
> line 73, in __getitem__
>     raise KeyError(key)
> KeyError: 'error'
> 
> 
> (that from py35-djangolatest, but py35-django111 and py35-django20 have
> the same failure and there are no py27 tests defined.


Are you sure you are trying from the git-head of postorius?

I can't reproduce this issue in my machine. I also tried it in the docker container used for CI (maxking/mailman-ci-runner), and all the tests seem to passs with python{35,36}-django{111,20,latest}. 

I also merged a branch this morning in Postorius and the CI passed for it as well.


-- 
  Abhilash Raj
  maxking at asynchronous.in

From mark at msapiro.net  Mon Feb 26 00:45:40 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 25 Feb 2018 21:45:40 -0800
Subject: [Mailman-Developers] Postorius, Hyperkitty and python support
In-Reply-To: <1519618458.2597668.1283178192.3827056D@webmail.messagingengine.com>
References: <a9dae172-c173-4d88-25c2-545fdd478c17@msapiro.net>
 <1519618458.2597668.1283178192.3827056D@webmail.messagingengine.com>
Message-ID: <b05c9f8e-f674-d635-d8db-fc2b363aa654@msapiro.net>

On 02/25/2018 08:14 PM, Abhilash Raj wrote:
> On Sun, Feb 25, 2018, at 6:03 PM, Mark Sapiro wrote:
>>
>> ERROR: test_middleware_request
>> (postorius.tests.mailman_api_tests.test_middleware.TestMiddleware)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>>   File
>> "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site-
>> packages/mock/mock.py",
>> line 1305, in patched
>>     return func(*args, **keywargs)
>>   File
>> "/var/MM/3/master/postorius/src/postorius/tests/mailman_api_tests/
>> test_middleware.py",
>> line 47, in test_middleware_request
>>     response.context['error'])
>>   File
>> "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site-
>> packages/django/test/utils.py",
>> line 73, in __getitem__
>>     raise KeyError(key)
>> KeyError: 'error'
>>
>>
>> (that from py35-djangolatest, but py35-django111 and py35-django20 have
>> the same failure and there are no py27 tests defined.
> 
> 
> Are you sure you are trying from the git-head of postorius?


I'll do some more checking tomorrow. I seem to be having various
anomalous issues such as python_dateutil-1.5 (not compatible with Python
3) getting installed in my Python 3 virtualenv.

The issue I'm chasing now is editing domains doesn't work. I can add
domains and delete domains, but if I try to edit I get the edit page
which doesn't have a box for the mail_host which is the name of the
domain I'm editing and it says "Please enter a domain name" which is the
error for a missing (required) mail_host, and if I submit the form, all
I get is Please check the errors below.

What I can't figure out is why the mail_host box is not on the form.

Anyway, I have to put this aside until tomorrow, but I'll check further
then.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Mon Feb 26 17:38:09 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 26 Feb 2018 14:38:09 -0800
Subject: [Mailman-Developers] Postorius, Hyperkitty and python support
In-Reply-To: <b05c9f8e-f674-d635-d8db-fc2b363aa654@msapiro.net>
References: <a9dae172-c173-4d88-25c2-545fdd478c17@msapiro.net>
 <1519618458.2597668.1283178192.3827056D@webmail.messagingengine.com>
 <b05c9f8e-f674-d635-d8db-fc2b363aa654@msapiro.net>
Message-ID: <94891791-d4db-9a46-f26a-e9da825f3738@msapiro.net>

On 02/25/2018 09:45 PM, Mark Sapiro wrote:
> On 02/25/2018 08:14 PM, Abhilash Raj wrote:
>> On Sun, Feb 25, 2018, at 6:03 PM, Mark Sapiro wrote:
>>>
>>> ERROR: test_middleware_request
>>> (postorius.tests.mailman_api_tests.test_middleware.TestMiddleware)
>>> ----------------------------------------------------------------------
>>> Traceback (most recent call last):
>>>   File
>>> "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site-
>>> packages/mock/mock.py",
>>> line 1305, in patched
>>>     return func(*args, **keywargs)
>>>   File
>>> "/var/MM/3/master/postorius/src/postorius/tests/mailman_api_tests/
>>> test_middleware.py",
>>> line 47, in test_middleware_request
>>>     response.context['error'])
>>>   File
>>> "/var/MM/3/master/postorius/.tox/py35-djangolatest/lib/python3.5/site-
>>> packages/django/test/utils.py",
>>> line 73, in __getitem__
>>>     raise KeyError(key)
>>> KeyError: 'error'
>>>
>>>
>>> (that from py35-djangolatest, but py35-django111 and py35-django20 have
>>> the same failure and there are no py27 tests defined.
>>
>>
>> Are you sure you are trying from the git-head of postorius?
> 
> 
> I'll do some more checking tomorrow. I seem to be having various
> anomalous issues such as python_dateutil-1.5 (not compatible with Python
> 3) getting installed in my Python 3 virtualenv.


The python_dateutil-1.5 turned out to be HyperKitty not yet being Python
3 ready and requiring python_dateutil<2.0.

I just did a 'git clone git at gitlab.com:mailman/postorius into a new
directory and ran tox and I get the same error.


> The issue I'm chasing now is editing domains doesn't work. I can add
> domains and delete domains, but if I try to edit I get the edit page
> which doesn't have a box for the mail_host which is the name of the
> domain I'm editing and it says "Please enter a domain name" which is the
> error for a missing (required) mail_host, and if I submit the form, all
> I get is Please check the errors below.
> 
> What I can't figure out is why the mail_host box is not on the form.


I have filed <https://gitlab.com/mailman/postorius/issues/257> for this one.


-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From leon at green-side.de  Mon Feb 26 18:14:56 2018
From: leon at green-side.de (Leon Merten Lohse)
Date: Tue, 27 Feb 2018 00:14:56 +0100
Subject: [Mailman-Developers] Postorius, Hyperkitty and python support
In-Reply-To: <94891791-d4db-9a46-f26a-e9da825f3738@msapiro.net>
References: <a9dae172-c173-4d88-25c2-545fdd478c17@msapiro.net>
 <1519618458.2597668.1283178192.3827056D@webmail.messagingengine.com>
 <b05c9f8e-f674-d635-d8db-fc2b363aa654@msapiro.net>
 <94891791-d4db-9a46-f26a-e9da825f3738@msapiro.net>
Message-ID: <3471206f-3357-faa3-f7d4-767c6197cfb7@green-side.de>

On 02/26/2018 11:38 PM, Mark Sapiro wrote:

> The python_dateutil-1.5 turned out to be HyperKitty not yet being Python
> 3 ready and requiring python_dateutil<2.0.

The Python 3 port of HyperKitty sits in the 'python3'-branch and does 
correctly require python-dateutil>=2.0.
To test it, you have to checkout that branch first. As of two days ago 
all tests pass with Python3.5 and Python3.6 using Django-1.11. 
Compatibility with Django-2.0 apparently requires some more work.

The merge will happen *soon*, hopefully.

Best
Leon

From bob at nleaudio.com  Tue Feb 27 23:37:29 2018
From: bob at nleaudio.com (Bob Puff)
Date: Tue, 27 Feb 2018 23:37:29 -0500
Subject: [Mailman-Developers] Subject munging with non-ascii
Message-ID: <022b3982-53ea-cf93-7d32-a34b74805726@nleaudio.com>

Greetings,

I just got a report today from one of my Mailman 2 lists, where I have a subject prefix set (like 
[Listname] ).  It seems that if the subject line has non-ascii characters in it, Mailman is not 
messing with it, and adding in the requested prefix.  Could this be an easy fix?

Bob

From mark at msapiro.net  Wed Feb 28 00:14:25 2018
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 27 Feb 2018 21:14:25 -0800
Subject: [Mailman-Developers] Subject munging with non-ascii
In-Reply-To: <022b3982-53ea-cf93-7d32-a34b74805726@nleaudio.com>
References: <022b3982-53ea-cf93-7d32-a34b74805726@nleaudio.com>
Message-ID: <a2ffc004-4051-02ef-787f-def3dc5b9e3f@msapiro.net>

On 02/27/2018 08:37 PM, Bob Puff wrote:
> Greetings,
> 
> I just got a report today from one of my Mailman 2 lists, where I have a
> subject prefix set (like [Listname] ).? It seems that if the subject
> line has non-ascii characters in it, Mailman is not messing with it, and
> adding in the requested prefix.? Could this be an easy fix?


This shouldn't be happening. Is the non-ascii subject properly RFC 2047
encoded or does it actually have non-ascii bytes?

If the former, prefixing should work. If the latter, the message is
non-compliant and all bets are off (we don't even know the character set
of the subject), but we should try to make it work anyway.

In order to know how to fix it if it can be fixed, I'd like to see the
original raw subject as it arrives to Mailman, and what Mailman version
is this?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan