From sm at noisynotes.com  Sat Aug  1 15:48:38 2015
From: sm at noisynotes.com (Steve Matzura)
Date: Sat, 01 Aug 2015 09:48:38 -0400
Subject: [Mailman-Users] mailpasswd failure
Message-ID: <8oipratms4mnuosjejas95l9o021qdhodn@4ax.com>

With some excellent help, all my Mailman problems up to today have
been solved. and my mailing lists have been behaving normally. Since
it's a new month, mailpasswd ran earlier this morning. The email of
the cron job reported a permissions error on one of the config.pck
files for one of the lists. Investigation proved the offender was
owned by root, not mailman. Interesting that check_perms didn't think
this was a problem. I found several others that were like this and,
figuring the one reported in the cron email was only the tip of the
iceberg, I changed the others as well and forced the cron job to run
again. As a result, I got the following:

Traceback (most recent call last):
  File "/usr/local/mailman/cron/mailpasswds", line 243, in <module>
    main()
  File "/usr/local/mailman/cron/mailpasswds", line 237, in main
    'verp'    : mm_cfg.VERP_PASSWORD_REMINDERS,
  File "/usr/local/mailman/Mailman/Message.py", line 313, in send
    self._enqueue(mlist, **_kws)
  File "/usr/local/mailman/Mailman/Message.py", line 325, in _enqueue
    **_kws)
  File "/usr/local/mailman/Mailman/Queue/Switchboard.py", line 136, in
enqueue
    fp = open(tmpfile, 'w')
IOError: [Errno 13] Permission denied:
'/usr/local/mailman/qfiles/virgin/1438435501.992054+ccd0ca1df3e432f3edbc589c470ce7b47276cc9d.pck.tmp'

As you can tell, this is a hand-built installation of Mailman, which
may or may not have something to do with this. check_perms reports no
problems found. The directory /usr/local/mailman/qfiles/virgin, which
is currently empty, is owned by root and is a member of the mailman
group. Its protection mask is 770. Looking through the mm
documentation isn't doing it for me at this point. Everything on the
new system looks like it matches everything on the old system with
regard to protection masks and ownerships. What else should I be
checking?

From jmck at ece.cmu.edu  Sun Aug  2 15:00:06 2015
From: jmck at ece.cmu.edu (Jim McKinney)
Date: Sun, 2 Aug 2015 09:00:06 -0400
Subject: [Mailman-Users] command line get list of members moderation status?
Message-ID: <55BE1456.6060301@ece.cmu.edu>

Hi,

We're running Mailman 2.1.12.  We have a list that is largely moderated,
but there is a small group of posters who should not be.  Because of the
size of the list the Membership management forms are a little cumbersome
for checking all of these.  Is there command line method to get a list
of members who have their moderation flag either set, or unset if that's
easier?  Alternatively, is there a way to list all the members of the
list with their moderation flag as part of the output?  I did look
around a bit, but my google mojo is failing me...

Thanks in advance for any help.

thanx
 -jmck

-- 
Jim McKinney
Information Technology Services (ITS)
Department of Electrical and Computing Engineering
Carnegie Mellon University
412.268.5141


From heller at deepsoft.com  Sun Aug  2 19:15:57 2015
From: heller at deepsoft.com (Robert Heller)
Date: Sun, 2 Aug 2015 13:15:57 -0400
Subject: [Mailman-Users] Changing a subscriber's E-Mail address
Message-ID: <201508021715.t72HFvid030679@sharky2.deepsoft.com>

Is it true that the quickest way for a list admin to change a subscriber's 
E-Mail address is to do a 'mass removal' then a 'mass subscription?

I have a subscriber who needs to change her E-Mail address for a list, and
since she does not (yet?) know how to use her (new) iPhone, and is basically
unable to respond to mailman confirmation message. There is no obvious place
to change a subscriber's E-Mail address that does not entail going through the
mailman confirmation process.

-- 
Robert Heller             -- 978-544-6933
Deepwoods Software        -- Custom Software Services
http://www.deepsoft.com/  -- Linux Administration Services
heller at deepsoft.com       -- Webhosting Services
                                                                                       

From stephen at xemacs.org  Mon Aug  3 05:19:52 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 03 Aug 2015 12:19:52 +0900
Subject: [Mailman-Users]  Changing a subscriber's E-Mail address
In-Reply-To: <201508021715.t72HFvid030679@sharky2.deepsoft.com>
References: <201508021715.t72HFvid030679@sharky2.deepsoft.com>
Message-ID: <874mkhcb3b.fsf@uwakimon.sk.tsukuba.ac.jp>

Robert Heller writes:

 > Is it true that the quickest way for a list admin to change a
 > subscriber's E-Mail address is to do a 'mass removal' then a 'mass
 > subscription?

Yes.  Addresses don't change frequently, are even less frequently
reassigned to other users.  In most cases the user does it for
herself.

 > There is no obvious place to change a subscriber's E-Mail address
 > that does not entail going through the mailman confirmation
 > process.

Somebody needs to authenticate, otherwise all kinds of mischief are
possible (and in the past occurred frequently).  If the user can't, it
has to be the admin, and administrative removal plus administrative
subscription is the obvious way to change a subscriber's address,
since Mailman 2 has no concept of "subscriber", only "subscribed
address".

Mailman 3 does have a notion of "user" apart from a subscription, so
it is possible to change the address of a user.


From adam-mailman at amyl.org.uk  Mon Aug  3 15:07:53 2015
From: adam-mailman at amyl.org.uk (Adam McGreggor)
Date: Mon, 3 Aug 2015 14:07:53 +0100
Subject: [Mailman-Users] Changing a subscriber's E-Mail address
In-Reply-To: <201508021715.t72HFvid030679@sharky2.deepsoft.com>
References: <201508021715.t72HFvid030679@sharky2.deepsoft.com>
Message-ID: <20150803130753.GJ9573@hendricks.amyl.org.uk>

On Sun, Aug 02, 2015 at 01:15:57PM -0400, Robert Heller wrote:
> Is it true that the quickest way for a list admin to change a subscriber's 
> E-Mail address is to do a 'mass removal' then a 'mass subscription?
> 

If you have shell access,
https://www.msapiro.net/scripts/change_member_address.py is quite
nifty.


-- 
"There is a providence that protects idiots, drunkards,
 children and the United States of America." 
    -- Otto von Bismarck

From Peter.Heitzer at rz.uni-regensburg.de  Mon Aug  3 17:54:17 2015
From: Peter.Heitzer at rz.uni-regensburg.de (Peter Heitzer)
Date: Mon, 03 Aug 2015 17:54:17 +0200
Subject: [Mailman-Users] command line get list of members moderation
 status?
In-Reply-To: <55BE1456.6060301@ece.cmu.edu>
References: <55BE1456.6060301@ece.cmu.edu>
Message-ID: <55BFAAC90200002C0002F560@gwsmtp1.uni-regensburg.de>

>>> On 08/02/2015 at 15:00, <jmck at ece.cmu.edu> wrote: 
> Hi,
> 
> We're running Mailman 2.1.12.  We have a list that is largely moderated,
> but there is a small group of posters who should not be.  Because of the
> size of the list the Membership management forms are a little cumbersome
> for checking all of these.  Is there command line method to get a list
> of members who have their moderation flag either set, or unset if that's
> easier?  Alternatively, is there a way to list all the members of the
> list with their moderation flag as part of the output?  I did look
> around a bit, but my google mojo is failing me...
> 
> Thanks in advance for any help.
You could do this with a small python script called via withlist
At http://www.msapiro.net/scripts/ there are a lot of small utility scripts. 

https://www.msapiro.net/scripts/set_mod.py sets or removes the flag for given members.
If you already know the members to set moderated this would be the way to do it.

https://www.msapiro.net/scripts/list_mod.py
Lists the moderated on unmoderated members. Thats perhaps fullfills your wish best.





From nick.peckham at googlemail.com  Mon Aug  3 10:20:50 2015
From: nick.peckham at googlemail.com (Nick Stevens)
Date: Mon, 3 Aug 2015 09:20:50 +0100
Subject: [Mailman-Users] Getting back to default settings
Message-ID: <CAMtaSBZuiGFzQoTqXxDJqOtuH-LQPgb6M931c=D-M0DWdymQeg@mail.gmail.com>

Hi All,
I was experimenting with settings to try and sort out attachments, and I've
completely lost track of what I have changed.

I think a good starting point would be to get back to defaults on
everything.

Is there a quick way to do that?
Or failing that somewhere I can find the defaults listed, so I can do it
manually?

Thanks,
Nick

-- 
*My page on the International Space Art
Network: http://spaceart1.ning.com/profile/NickStevens
<http://spaceart1.ning.com/profile/NickStevens>.*
My secure email: starbase1 at protonmail.ch

From jaybird at bluegrasspals.com  Wed Aug  5 09:15:54 2015
From: jaybird at bluegrasspals.com (Jayson Smith)
Date: Wed, 05 Aug 2015 03:15:54 -0400
Subject: [Mailman-Users] Rejecting spam at the SMTP level?
Message-ID: <55C1B82A.9050007@bluegrasspals.com>

Hello,

I've been running several Mailman lists for the past ten years or so. 
Unfortunately, in recent months some list addresses have fallen into the 
hands of spammers, which is a real headache. Even worse, one particular 
-owner address is receiving nothing but spam, and I don't assume there's 
anything I can do about that without disabling that address for 
legitimate use, as well as for use by Mailman itself.

A few months ago I did some Googling and found a project called 
Mailman-Milter which claims to reject unwanted list mail i.e., nonmember 
postings at the SMTP level. I would love to use this, but there's 
virtually no documentation and I've never been able to get it working. 
Has anyone else had any better luck, or do you know of any other 
solutions? I'm using Sendmail, by the way.

I would set all my lists to just discard nonmember traffic, but I want 
to allow for legitimate users who perhaps don't understand that they 
need to subscribe to lists before posting, legitimate members sending 
from wrong accounts, etc. My current compromise solution is to set my 
lists to hold such messages for moderation, but not send the posters 
messages telling them such, so as to avoid backscatter caused by trying 
to send "You're not a member, your message is being held" messages to 
bogus spammer Email addresses. When I get a held message notice, if it's 
a spammer I discard it and add the address to the discard list, for what 
good that does, and if it's legitimate I reject it, perhaps explaining 
in my own words why the message was rejected, especially if I know the 
person and know they tend to post from the wrong account, etc.

Thanks for any help!

Jayson


From mark at msapiro.net  Sat Aug  8 02:47:09 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 07 Aug 2015 17:47:09 -0700
Subject: [Mailman-Users] Getting back to default settings
In-Reply-To: <CAMtaSBZuiGFzQoTqXxDJqOtuH-LQPgb6M931c=D-M0DWdymQeg@mail.gmail.com>
References: <CAMtaSBZuiGFzQoTqXxDJqOtuH-LQPgb6M931c=D-M0DWdymQeg@mail.gmail.com>
Message-ID: <55C5518D.4010705@msapiro.net>

On 08/03/2015 01:20 AM, Nick Stevens via Mailman-Users wrote:
> Hi All,
> I was experimenting with settings to try and sort out attachments, and I've
> completely lost track of what I have changed.
> 
> I think a good starting point would be to get back to defaults on
> everything.
> 
> Is there a quick way to do that?
> Or failing that somewhere I can find the defaults listed, so I can do it
> manually?


I suggest creating a new list and then running bin/config_list to dump
the settings. This will give you a file with defaults which you can just
edit to remove the list specific settings such as real_name[1] and then
use bin/config_list to apply those defaults to your list and then remove
the newly created list.

[1] You can just remove the things you don't want to change. config_list
doesn't change anything not mentioned in its input.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sat Aug  8 03:25:45 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 07 Aug 2015 18:25:45 -0700
Subject: [Mailman-Users] command line get list of members moderation
	status?
In-Reply-To: <55BE1456.6060301@ece.cmu.edu>
References: <55BE1456.6060301@ece.cmu.edu>
Message-ID: <55C55A99.4020808@msapiro.net>

On 08/02/2015 06:00 AM, Jim McKinney wrote:
> 
> Is there command line method to get a list
> of members who have their moderation flag either set, or unset if that's
> easier?


See <https://www.msapiro.net/scripts/list_mod.py> (mirrored at
<http://fog.ccsf.edu/~msapiro/scripts/list_mod.py>).


>  Alternatively, is there a way to list all the members of the
> list with their moderation flag as part of the output?  I did look
> around a bit, but my google mojo is failing me...


See <https://www.msapiro.net/scripts/mailman-subscribers.py> (mirrored
at <http://fog.ccsf.edu/~msapiro/scripts/mailman-subscribers.py>).

This latter script is referenced from the FAQ at
<http://wiki.list.org/x/4030569>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sat Aug  8 03:33:20 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 07 Aug 2015 18:33:20 -0700
Subject: [Mailman-Users] Changing a subscriber's E-Mail address
In-Reply-To: <201508021715.t72HFvid030679@sharky2.deepsoft.com>
References: <201508021715.t72HFvid030679@sharky2.deepsoft.com>
Message-ID: <55C55C60.60602@msapiro.net>

On 08/02/2015 10:15 AM, Robert Heller wrote:
> Is it true that the quickest way for a list admin to change a subscriber's 
> E-Mail address is to do a 'mass removal' then a 'mass subscription?


Unless you have Mailman 2.1.20 or newer in which case there is an
Address Change subsection in the admin Membership Management... section.

With command line access, you can use Mailman's bin/clone_member, but
many (most?) list admins don't have this access.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From thomas at ifi.uio.no  Sat Aug  8 04:09:46 2015
From: thomas at ifi.uio.no (Thomas Gramstad)
Date: Sat, 8 Aug 2015 04:09:46 +0200 (CEST)
Subject: [Mailman-Users] "Downgrading" Mailman
Message-ID: <alpine.DEB.2.11.1508080403480.20151@bia>

I am moving some lists from a server that will be shut down to 
another server, and the other server is running an older version 
of Mailman -- version 2.1.9 vs. 2.1.14. That's actually fine with 
me because 2.1.9 works with the listadmin script, while 2.1.14 
doesn't. My question is: I will also move the list archives into 
the "new" server -- are there any issues to consider moving a 
list archive from 2.1.14 into a 2.1.9 installation?

Thomas Gramstad

From mark at msapiro.net  Sat Aug  8 06:10:31 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 07 Aug 2015 21:10:31 -0700
Subject: [Mailman-Users] "Downgrading" Mailman
In-Reply-To: <alpine.DEB.2.11.1508080403480.20151@bia>
References: <alpine.DEB.2.11.1508080403480.20151@bia>
Message-ID: <55C58137.2090408@msapiro.net>

On 08/07/2015 07:09 PM, Thomas Gramstad wrote:
> My question is: I
> will also move the list archives into the "new" server -- are there any
> issues to consider moving a list archive from 2.1.14 into a 2.1.9
> installation?


Moving archives from 2.1.14 into a 2.1.9 installation should be no
different form moving them to another 2.1.14 installation.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sat Aug  8 06:28:05 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 07 Aug 2015 21:28:05 -0700
Subject: [Mailman-Users] mailpasswd failure
In-Reply-To: <8oipratms4mnuosjejas95l9o021qdhodn@4ax.com>
References: <8oipratms4mnuosjejas95l9o021qdhodn@4ax.com>
Message-ID: <55C58555.70609@msapiro.net>

On 08/01/2015 06:48 AM, Steve Matzura wrote:
> With some excellent help, all my Mailman problems up to today have
> been solved. and my mailing lists have been behaving normally. Since
> it's a new month, mailpasswd ran earlier this morning. The email of
> the cron job reported a permissions error on one of the config.pck
> files for one of the lists. Investigation proved the offender was
> owned by root, not mailman. Interesting that check_perms didn't think
> this was a problem.


Because it shouldn't be. The file should be in Mailman's group and group
writable and the cron should run as the Mailman user:group.


> I found several others that were like this and,
> figuring the one reported in the cron email was only the tip of the
> iceberg, I changed the others as well and forced the cron job to run
> again. As a result, I got the following:
> 
> Traceback (most recent call last):
>   File "/usr/local/mailman/cron/mailpasswds", line 243, in <module>
>     main()
>   File "/usr/local/mailman/cron/mailpasswds", line 237, in main
>     'verp'    : mm_cfg.VERP_PASSWORD_REMINDERS,
>   File "/usr/local/mailman/Mailman/Message.py", line 313, in send
>     self._enqueue(mlist, **_kws)
>   File "/usr/local/mailman/Mailman/Message.py", line 325, in _enqueue
>     **_kws)
>   File "/usr/local/mailman/Mailman/Queue/Switchboard.py", line 136, in
> enqueue
>     fp = open(tmpfile, 'w')
> IOError: [Errno 13] Permission denied:
> '/usr/local/mailman/qfiles/virgin/1438435501.992054+ccd0ca1df3e432f3edbc589c470ce7b47276cc9d.pck.tmp'


So whatever user:group you ran the cron as this time doesn't have write
permission on /usr/local/mailman/qfiles/virgin/ .


> As you can tell, this is a hand-built installation of Mailman, which
> may or may not have something to do with this. check_perms reports no
> problems found. The directory /usr/local/mailman/qfiles/virgin, which
> is currently empty, is owned by root and is a member of the mailman
> group. Its protection mask is 770. Looking through the mm
> documentation isn't doing it for me at this point. Everything on the
> new system looks like it matches everything on the old system with
> regard to protection masks and ownerships. What else should I be
> checking?


How did you run the cron? Was it running as the mailman user?

Are there security policy issues, e.g. SELinux or apparmor?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sat Aug  8 07:02:34 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 07 Aug 2015 22:02:34 -0700
Subject: [Mailman-Users] Rejecting spam at the SMTP level?
In-Reply-To: <55C1B82A.9050007@bluegrasspals.com>
References: <55C1B82A.9050007@bluegrasspals.com>
Message-ID: <55C58D6A.5010400@msapiro.net>

On 08/05/2015 12:15 AM, Jayson Smith wrote:
> 
> A few months ago I did some Googling and found a project called
> Mailman-Milter which claims to reject unwanted list mail i.e., nonmember
> postings at the SMTP level. I would love to use this, but there's
> virtually no documentation and I've never been able to get it working.
> Has anyone else had any better luck, or do you know of any other
> solutions? I'm using Sendmail, by the way.
> 
> I would set all my lists to just discard nonmember traffic, but I want
> to allow for legitimate users who perhaps don't understand that they
> need to subscribe to lists before posting, legitimate members sending
> from wrong accounts, etc.


So, if you don't want to set the lists to discard non-member posts for
the reasons you explain, is it that much better to reject them at SMTP
time. Granted this avoids the issue of backscatter to bogus addresses,
but it isn't very helpful to the non-member senders of 'legitimate' posts.

A better solution is to use various sendmail tools to identify and
reject spam in general rather than just non-member list posts.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From andrew at hodgsonfamily.org  Sat Aug  8 12:28:56 2015
From: andrew at hodgsonfamily.org (Andrew Hodgson)
Date: Sat, 8 Aug 2015 10:28:56 +0000
Subject: [Mailman-Users] Rejecting spam at the SMTP level?
In-Reply-To: <55C1B82A.9050007@bluegrasspals.com>
References: <55C1B82A.9050007@bluegrasspals.com>
Message-ID: <DB5PR04MB147810B7E430979C403C1F9EC5720@DB5PR04MB1478.eurprd04.prod.outlook.com>

Jayson Smith wrote:

>I've been running several Mailman lists for the past ten years or so. 
>Unfortunately, in recent months some list addresses have fallen into the hands of spammers, which is a real headache. Even worse, one particular -owner address is receiving nothing but spam, and I don't assume there's >anything I can do about that without disabling that address for legitimate use, as well as for use by Mailman itself.

>A few months ago I did some Googling and found a project called Mailman-Milter which claims to reject unwanted list mail i.e., nonmember postings at the SMTP level. I would love to use this, but there's virtually no >documentation and I've never been able to get it working. 
>Has anyone else had any better luck, or do you know of any other solutions? I'm using Sendmail, by the way.

We had the same issue with our lists and I decided to install SpamAssassin on the server that rejects at SMTP level.  After a small bit of tuning we have this working really well, it doesn't interfere with list traffic, but rejects a lot of rubbish to the owner and other addresses.

HTH.
Andrew. 

From andrew at hodgsonfamily.org  Sat Aug  8 12:30:49 2015
From: andrew at hodgsonfamily.org (Andrew Hodgson)
Date: Sat, 8 Aug 2015 10:30:49 +0000
Subject: [Mailman-Users] Getting back to default settings
In-Reply-To: <CAMtaSBZuiGFzQoTqXxDJqOtuH-LQPgb6M931c=D-M0DWdymQeg@mail.gmail.com>
References: <CAMtaSBZuiGFzQoTqXxDJqOtuH-LQPgb6M931c=D-M0DWdymQeg@mail.gmail.com>
Message-ID: <DB5PR04MB1478D0288264586F538BE8A3C5720@DB5PR04MB1478.eurprd04.prod.outlook.com>

Nick Stevens via Mailman-Users wrote: 

>I think a good starting point would be to get back to defaults on everything.

The method I used was similar to Mark's, but I exported the settings from both the new and the current list to a text file and ran diff on those files to see exactly what people had changed.

Thanks.
Andrew. 

From sm at noisynotes.com  Sat Aug  8 17:12:18 2015
From: sm at noisynotes.com (Steve Matzura)
Date: Sat, 08 Aug 2015 11:12:18 -0400
Subject: [Mailman-Users] Permissions error on all Mailman overnights
Message-ID: <l24csatv9bogjv7g63cutgssod5h8dedo7@4ax.com>

Every night, I get the following error at the bottom of a mail message
to my mailman list telling me that checkdbs, digest and disabled
failed:

IOError: [Errno 13] Permission denied:
'/usr/local/mailman/locks/mailman.lock.{my-FQDN}.{some-number}.0'

This also happened on the one and only password reminder message that
went out over a week ago, which no one received.

{some-number} is presumably the process ID of the job's process.

Here are descriptions of the locks directory on the old and new
systems--i.e., the one that works, and the one with the job failures.
The old system is Debian 4, from 2009, which has not received any
software upgrades since 2010. 'list' is the name of the Mailman user
and group. The locks directory, /var/lib/mailman/locks, is defined
thus:

lrwxrwxrwx 1 root root 18 2009-05-01 02:31 locks -> ../../lock/mailman
drwxrwxrwt 4 root root 4096 2015-08-08 02:56 lock
drwxrwsr-x 2 root list 4096 2015-08-08 12:00 mailman

On the new Fedora 20 system with hand-built Mailman up to date,
'mailman' is its username and group, and the locks directory,
/usr/local/mailman/locks, is defined thus:

drwxrwsr-x 2 root mailman 4096 Aug  8 14:37 locks

The new system's permissions don't define the locks directory as
temporary, although at this very moment there are files in
it--master-qrunner, and master-qrunner.{my-FQDN}.{some-number}, both
of which have *tomorrow's* dates on them. ???

I'm thinking the locks directory may be at fault here, and if it is,
what was done wrong when Mailman was built or configured? This is
important not just because the mail system is not running correctly at
this time, but because I will be installing a second instance of
Mailman to support a new domain I'll be hosting, so I'm thinking
whatever I did wrong the first time will happen again, unless I
prevent it.

As always, thanks in advance for assistance.

From jimpop at gmail.com  Sat Aug  8 17:43:35 2015
From: jimpop at gmail.com (Jim Popovitch)
Date: Sat, 8 Aug 2015 11:43:35 -0400
Subject: [Mailman-Users] Rejecting spam at the SMTP level?
In-Reply-To: <55C1B82A.9050007@bluegrasspals.com>
References: <55C1B82A.9050007@bluegrasspals.com>
Message-ID: <CAGfsgR2m3xVkMrjK4Ln+hJ=N61GaRfphQqP2=-Oo-iQaQkfotQ@mail.gmail.com>

On Wed, Aug 5, 2015 at 3:15 AM, Jayson Smith <jaybird at bluegrasspals.com> wrote:
>
> A few months ago I did some Googling and found a project called
> Mailman-Milter which claims to reject unwanted list mail i.e., nonmember
> postings at the SMTP level. I would love to use this, but there's virtually
> no documentation and I've never been able to get it working. Has anyone else
> had any better luck, or do you know of any other solutions? I'm using
> Sendmail, by the way.

I'm not exactly sure how to do it in Sendmail (been a long time since
I've used that) but in Postfix it can be done by setting up a custom
master.cf service(s) and restriction class(es).

/etc/postfix/master.cf:
    # Mailman list member checks
    127.0.0.1:27015 inet    n       n       n       -       1       spawn
           user=mailman
argv=/usr/local/lib/mailman/scripts/check_subscriber mylist
    127.0.0.1:27016 inet    n       n       n       -       1       spawn
           user=mailman
argv=/usr/local/lib/mailman/scripts/check_subscriber mylist2

You can get a copy of the check_subscriber script here:
http://paste.debian.net/plainh/95fd6c08

You then need to setup a restriction class that matches a mailinglist
posting address and then hooks into the master.cf service on
127.0.0.1:27015 listed above.  The restriction class matches just the
mailinglist posting address (not the -bounce, -owner, -subscribe,
etc).

/etc/postfix/check_recipients:
mylist at mydomain.tld             mailman_subscribers_mylist
mylist2 at mydomain.tld           mailman_subscribers_mylist2

/etc/postfix/main.cf
    smtpd_client_restrictions=
        permit_mynetworks,
        ......
        check_recipient_access hash:/etc/postfix/check_recipients,
        permit,

    # declare our custom restriction classes
    smtpd_restriction_classes =
        mailman_subscribers_mylist
        mailman_subscribers_mylist2

   # define our custom restriction class for mylist
  mailman_subscribers_mylist =
        check_sender_access tcp:127.0.0.1:27015,
        permit

   # define our custom restriction class for mylist2
   mailman_subscribers_mylist2 =
        check_sender_access tcp:127.0.0.1:27016,
        permit


I would be interested in hearing/learning about how to make that work
in sendmail if you get the oppty to shoehorn something like this it
into your setup.

-Jim P.

From mark at msapiro.net  Sat Aug  8 18:19:27 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 08 Aug 2015 09:19:27 -0700
Subject: [Mailman-Users] Permissions error on all Mailman overnights
In-Reply-To: <l24csatv9bogjv7g63cutgssod5h8dedo7@4ax.com>
References: <l24csatv9bogjv7g63cutgssod5h8dedo7@4ax.com>
Message-ID: <55C62C0F.4020804@msapiro.net>

On 08/08/2015 08:12 AM, Steve Matzura wrote:
> Every night, I get the following error at the bottom of a mail message
> to my mailman list telling me that checkdbs, digest and disabled
> failed:
> 
> IOError: [Errno 13] Permission denied:
> '/usr/local/mailman/locks/mailman.lock.{my-FQDN}.{some-number}.0'


So cron/checkdbs cant create a lock.

How are these crons running? Are they run form a user crontab
(/var/spool/cron/mailman ?) or a system crontab (/etc/cron.d/mailman ?)?

If the latter, what's the user nsame in the crontab entries.

> This also happened on the one and only password reminder message that
> went out over a week ago, which no one received.
> 
> {some-number} is presumably the process ID of the job's process.


Yes.


> Here are descriptions of the locks directory on the old and new
> systems--i.e., the one that works, and the one with the job failures.
> The old system is Debian 4, from 2009, which has not received any
> software upgrades since 2010. 'list' is the name of the Mailman user
> and group. The locks directory, /var/lib/mailman/locks, is defined
> thus:
> 
> lrwxrwxrwx 1 root root 18 2009-05-01 02:31 locks -> ../../lock/mailman
> drwxrwxrwt 4 root root 4096 2015-08-08 02:56 lock
> drwxrwsr-x 2 root list 4096 2015-08-08 12:00 mailman


This looks OK, but presumably is irrelevant as it's not the system with
the issue, right?


> On the new Fedora 20 system with hand-built Mailman up to date,
> 'mailman' is its username and group, and the locks directory,
> /usr/local/mailman/locks, is defined thus:
> 
> drwxrwsr-x 2 root mailman 4096 Aug  8 14:37 locks


This looks OK too.


> The new system's permissions don't define the locks directory as
> temporary,


That shouldn't be necessary.


> although at this very moment there are files in
> it--master-qrunner, and master-qrunner.{my-FQDN}.{some-number}, both
> of which have *tomorrow's* dates on them. ???


Those are the master qrunner locks and should be there if Mailman is
running, and the future date is normal. That's the time at which the
lock expires.


> I'm thinking the locks directory may be at fault here, and if it is,
> what was done wrong when Mailman was built or configured? This is
> important not just because the mail system is not running correctly at
> this time, but because I will be installing a second instance of
> Mailman to support a new domain I'll be hosting, so I'm thinking
> whatever I did wrong the first time will happen again, unless I
> prevent it.


Your locks directory is fine. The master qrunner can create its locks
and the running Mailman and presumably the web CGIs can lock and unlock
lists which is done repeatedly. Your only issue seems to be with crons.
This must be either an issue with the user:group running the crons or a
security policy (SELinux or ?) issue.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From jimpop at gmail.com  Sat Aug  8 19:21:59 2015
From: jimpop at gmail.com (Jim Popovitch)
Date: Sat, 8 Aug 2015 13:21:59 -0400
Subject: [Mailman-Users] Rejecting spam at the SMTP level?
In-Reply-To: <CAGfsgR2m3xVkMrjK4Ln+hJ=N61GaRfphQqP2=-Oo-iQaQkfotQ@mail.gmail.com>
References: <55C1B82A.9050007@bluegrasspals.com>
 <CAGfsgR2m3xVkMrjK4Ln+hJ=N61GaRfphQqP2=-Oo-iQaQkfotQ@mail.gmail.com>
Message-ID: <CAGfsgR14x90Gu85whFMxiE3fsFL4is3+fTvf_mA+BHK+F6XFeQ@mail.gmail.com>

On Sat, Aug 8, 2015 at 11:43 AM, Jim Popovitch <jimpop at gmail.com> wrote:
> On Wed, Aug 5, 2015 at 3:15 AM, Jayson Smith <jaybird at bluegrasspals.com> wrote:
>>
>> A few months ago I did some Googling and found a project called
>> Mailman-Milter which claims to reject unwanted list mail i.e., nonmember
>> postings at the SMTP level. I would love to use this, but there's virtually
>> no documentation and I've never been able to get it working. Has anyone else
>> had any better luck, or do you know of any other solutions? I'm using
>> Sendmail, by the way.
>
> I'm not exactly sure how to do it in Sendmail (been a long time since
> I've used that) but in Postfix it can be done by setting up a custom
> master.cf service(s) and restriction class(es).
>
> /etc/postfix/master.cf:
>     # Mailman list member checks
>     127.0.0.1:27015 inet    n       n       n       -       1       spawn
>            user=mailman
> argv=/usr/local/lib/mailman/scripts/check_subscriber mylist
>     127.0.0.1:27016 inet    n       n       n       -       1       spawn
>            user=mailman
> argv=/usr/local/lib/mailman/scripts/check_subscriber mylist2
>
> You can get a copy of the check_subscriber script here:
> http://paste.debian.net/plainh/95fd6c08
>
> You then need to setup a restriction class that matches a mailinglist
> posting address and then hooks into the master.cf service on
> 127.0.0.1:27015 listed above.  The restriction class matches just the
> mailinglist posting address (not the -bounce, -owner, -subscribe,
> etc).
>
> /etc/postfix/check_recipients:
> mylist at mydomain.tld             mailman_subscribers_mylist
> mylist2 at mydomain.tld           mailman_subscribers_mylist2
>
> /etc/postfix/main.cf
>     smtpd_client_restrictions=
>         permit_mynetworks,
>         ......
>         check_recipient_access hash:/etc/postfix/check_recipients,
>         permit,
>
>     # declare our custom restriction classes
>     smtpd_restriction_classes =
>         mailman_subscribers_mylist
>         mailman_subscribers_mylist2
>
>    # define our custom restriction class for mylist
>   mailman_subscribers_mylist =
>         check_sender_access tcp:127.0.0.1:27015,
>         permit
>
>    # define our custom restriction class for mylist2
>    mailman_subscribers_mylist2 =
>         check_sender_access tcp:127.0.0.1:27016,
>         permit
>
>
> I would be interested in hearing/learning about how to make that work
> in sendmail if you get the oppty to shoehorn something like this it
> into your setup.
>

BTW, here's what happens as you get older:   you forget that this has
already been documented  :-)

http://bazaar.launchpad.net/~jimpop/mailman/check_subscriber/view/head:/README.check_subscriber

-Jim P.

From larry at acbradio.org  Sat Aug  8 19:26:30 2015
From: larry at acbradio.org (Larry Turnbull)
Date: Sat, 08 Aug 2015 17:26:30 +0000
Subject: [Mailman-Users] Permissions error on all Mailman overnights
In-Reply-To: <55C62C0F.4020804@msapiro.net>
References: <l24csatv9bogjv7g63cutgssod5h8dedo7@4ax.com>
 <55C62C0F.4020804@msapiro.net>
Message-ID: <012d01d0d1ff$5fdf4dc0$1f9de940$@acbradio.org>

I am working with Steve on this one as well.

I just checked the system and I see /var/spool/crontab/mailman.

/etc/cron.d/mailman does not exist so it is running under the first
scenario.

If I type crontab -u mailman -l I can see the list of mailman cron jobs that
are scheduled to run.

I am trying to figure out what user the cron system is trying to run the
mailman crons as as that is where I think the problem lies.

Running /usr/local/mailman/bin/check_perms reports no problems so I think we
are good on how permissions are set on the files and folders.

The lists themselves are working fine it is just the cron jobs where things
are a bit odd.

Larry


-----Original Message-----
From: Mailman-Users
[mailto:mailman-users-bounces+larry=acbradio.org at python.org] On Behalf Of
Mark Sapiro
Sent: Saturday, August 08, 2015 11:19 AM
To: mailman-users at python.org
Subject: Re: [Mailman-Users] Permissions error on all Mailman overnights

On 08/08/2015 08:12 AM, Steve Matzura wrote:
> Every night, I get the following error at the bottom of a mail message 
> to my mailman list telling me that checkdbs, digest and disabled
> failed:
> 
> IOError: [Errno 13] Permission denied:
> '/usr/local/mailman/locks/mailman.lock.{my-FQDN}.{some-number}.0'


So cron/checkdbs cant create a lock.

How are these crons running? Are they run form a user crontab
(/var/spool/cron/mailman ?) or a system crontab (/etc/cron.d/mailman ?)?

If the latter, what's the user nsame in the crontab entries.

> This also happened on the one and only password reminder message that 
> went out over a week ago, which no one received.
> 
> {some-number} is presumably the process ID of the job's process.


Yes.


> Here are descriptions of the locks directory on the old and new 
> systems--i.e., the one that works, and the one with the job failures.
> The old system is Debian 4, from 2009, which has not received any 
> software upgrades since 2010. 'list' is the name of the Mailman user 
> and group. The locks directory, /var/lib/mailman/locks, is defined
> thus:
> 
> lrwxrwxrwx 1 root root 18 2009-05-01 02:31 locks -> ../../lock/mailman 
> drwxrwxrwt 4 root root 4096 2015-08-08 02:56 lock drwxrwsr-x 2 root 
> list 4096 2015-08-08 12:00 mailman


This looks OK, but presumably is irrelevant as it's not the system with the
issue, right?


> On the new Fedora 20 system with hand-built Mailman up to date, 
> 'mailman' is its username and group, and the locks directory, 
> /usr/local/mailman/locks, is defined thus:
> 
> drwxrwsr-x 2 root mailman 4096 Aug  8 14:37 locks


This looks OK too.


> The new system's permissions don't define the locks directory as 
> temporary,


That shouldn't be necessary.


> although at this very moment there are files in it--master-qrunner, 
> and master-qrunner.{my-FQDN}.{some-number}, both of which have 
> *tomorrow's* dates on them. ???


Those are the master qrunner locks and should be there if Mailman is
running, and the future date is normal. That's the time at which the lock
expires.


> I'm thinking the locks directory may be at fault here, and if it is, 
> what was done wrong when Mailman was built or configured? This is 
> important not just because the mail system is not running correctly at 
> this time, but because I will be installing a second instance of 
> Mailman to support a new domain I'll be hosting, so I'm thinking 
> whatever I did wrong the first time will happen again, unless I 
> prevent it.


Your locks directory is fine. The master qrunner can create its locks and
the running Mailman and presumably the web CGIs can lock and unlock lists
which is done repeatedly. Your only issue seems to be with crons.
This must be either an issue with the user:group running the crons or a
security policy (SELinux or ?) issue.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan
------------------------------------------------------
Mailman-Users mailing list Mailman-Users at python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy:
http://wiki.list.org/x/QIA9 Searchable Archives:
http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe:
https://mail.python.org/mailman/options/mailman-users/larry%40acbradio.org

From mark at msapiro.net  Sat Aug  8 21:39:19 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 08 Aug 2015 12:39:19 -0700
Subject: [Mailman-Users] Permissions error on all Mailman overnights
In-Reply-To: <012d01d0d1ff$5fdf4dc0$1f9de940$@acbradio.org>
References: <l24csatv9bogjv7g63cutgssod5h8dedo7@4ax.com>
 <55C62C0F.4020804@msapiro.net> <012d01d0d1ff$5fdf4dc0$1f9de940$@acbradio.org>
Message-ID: <55C65AE7.6040900@msapiro.net>

On 08/08/2015 10:26 AM, Larry Turnbull wrote:
> 
> I just checked the system and I see /var/spool/crontab/mailman.


OK, this is the user crontab for the user 'mailman'.


> /etc/cron.d/mailman does not exist so it is running under the first
> scenario.


Normally /etc/cron.d/mailman does not exist in a source install. Some
packages, e.g. Debian/Ubuntu, install this, but a source install doesn't.


> If I type crontab -u mailman -l I can see the list of mailman cron jobs that
> are scheduled to run.


And what that does is just list the contents of /etc/cron.d/mailman.


> I am trying to figure out what user the cron system is trying to run the
> mailman crons as as that is where I think the problem lies.


crons in crontabs in files /var/spool/crontab/XXX are run as user XXX.


> Running /usr/local/mailman/bin/check_perms reports no problems so I think we
> are good on how permissions are set on the files and folders.
> 
> The lists themselves are working fine it is just the cron jobs where things
> are a bit odd.


What does

groups mailman

report? It should be 'mailman : mailman'. If so, and the crons are
getting permission problems, check if SELinux is enabled and if so, try
disabling it.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From sm at noisynotes.com  Sun Aug  9 03:22:43 2015
From: sm at noisynotes.com (Steve Matzura)
Date: Sat, 08 Aug 2015 21:22:43 -0400
Subject: [Mailman-Users] Permissions error on all Mailman overnights
In-Reply-To: <55C65AE7.6040900@msapiro.net>
References: <l24csatv9bogjv7g63cutgssod5h8dedo7@4ax.com>
 <55C62C0F.4020804@msapiro.net> <012d01d0d1ff$5fdf4dc0$1f9de940$@acbradio.org>
 <55C65AE7.6040900@msapiro.net>
Message-ID: <poadsahardhq593rng8tlk1dmqtgd4cb73@4ax.com>

On Sat, 08 Aug 2015 12:39:19 -0700, Mark asked:

>What does
>
>groups mailman
>
>report? It should be 'mailman : mailman'. If so, and the crons are
>getting permission problems, check if SELinux is enabled and if so, try
>disabling it.

It's "mailman : mailman postfix", and SELinux is disabled according to
getenforce.


From sm at noisynotes.com  Sun Aug  9 03:22:44 2015
From: sm at noisynotes.com (Steve Matzura)
Date: Sat, 08 Aug 2015 21:22:44 -0400
Subject: [Mailman-Users] Permissions error on all Mailman overnights
In-Reply-To: <55C62C0F.4020804@msapiro.net>
References: <l24csatv9bogjv7g63cutgssod5h8dedo7@4ax.com>
 <55C62C0F.4020804@msapiro.net>
Message-ID: <3padsat55lvjbeujagt2diqaglgjq9hrim@4ax.com>

On Sat, 08 Aug 2015 09:19:27 -0700, Mark Sapiro <mark at msapiro.net>
wrote:

>On 08/08/2015 08:12 AM, Steve Matzura wrote:
>> Every night, I get the following error at the bottom of a mail message
>> to my mailman list telling me that checkdbs, digest and disabled
>> failed:
>> 
>> IOError: [Errno 13] Permission denied:
>> '/usr/local/mailman/locks/mailman.lock.{my-FQDN}.{some-number}.0'
>
>
>So cron/checkdbs cant create a lock.
>
>How are these crons running? Are they run form a user crontab
>(/var/spool/cron/mailman ?) or a system crontab (/etc/cron.d/mailman ?)?

The former.

>Your locks directory is fine. The master qrunner can create its locks
>and the running Mailman and presumably the web CGIs can lock and unlock
>lists which is done repeatedly. Your only issue seems to be with crons.
>This must be either an issue with the user:group running the crons or a
>security policy (SELinux or ?) issue.

No SELinux. getenforce says "Disabled".

From mark at msapiro.net  Sun Aug  9 03:47:29 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 08 Aug 2015 18:47:29 -0700
Subject: [Mailman-Users] Permissions error on all Mailman overnights
In-Reply-To: <poadsahardhq593rng8tlk1dmqtgd4cb73@4ax.com>
References: <l24csatv9bogjv7g63cutgssod5h8dedo7@4ax.com>
 <55C62C0F.4020804@msapiro.net> <012d01d0d1ff$5fdf4dc0$1f9de940$@acbradio.org>
 <55C65AE7.6040900@msapiro.net> <poadsahardhq593rng8tlk1dmqtgd4cb73@4ax.com>
Message-ID: <55C6B131.8010402@msapiro.net>

On 08/08/2015 06:22 PM, Steve Matzura wrote:
> On Sat, 08 Aug 2015 12:39:19 -0700, Mark asked:
> 
>> What does
>>
>> groups mailman
>>
>> report? It should be 'mailman : mailman'. If so, and the crons are
>> getting permission problems, check if SELinux is enabled and if so, try
>> disabling it.
> 
> It's "mailman : mailman postfix", and SELinux is disabled according to
> getenforce.


But it was "mailman : postfix". Larry Turnbull  fixed it by adding the
'mailman' group.

He reported this to me in an off-list message which I deleted thinking
it had gone to the list.

You should be OK now. If not, post tracebacks from runs that abort going
forward.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From sm at noisynotes.com  Sun Aug  9 17:25:33 2015
From: sm at noisynotes.com (Steve Matzura)
Date: Sun, 09 Aug 2015 11:25:33 -0400
Subject: [Mailman-Users] Permissions error on all Mailman overnights
In-Reply-To: <55C6B131.8010402@msapiro.net>
References: <l24csatv9bogjv7g63cutgssod5h8dedo7@4ax.com>
 <55C62C0F.4020804@msapiro.net> <012d01d0d1ff$5fdf4dc0$1f9de940$@acbradio.org>
 <55C65AE7.6040900@msapiro.net> <poadsahardhq593rng8tlk1dmqtgd4cb73@4ax.com>
 <55C6B131.8010402@msapiro.net>
Message-ID: <g2sesa9uuvfrqqdv7gmavbuj5cbm98vnh8@4ax.com>

On Sat, 08 Aug 2015 18:47:29 -0700, Mark Sapiro <mark at msapiro.net>
wrote:

>On 08/08/2015 06:22 PM, Steve Matzura wrote:
>> On Sat, 08 Aug 2015 12:39:19 -0700, Mark asked:
>> 
>>> What does
>>>
>>> groups mailman
>>>
>>> report? It should be 'mailman : mailman'. If so, and the crons are
>>> getting permission problems, check if SELinux is enabled and if so, try
>>> disabling it.
>> 
>> It's "mailman : mailman postfix", and SELinux is disabled according to
>> getenforce.
>
>
>But it was "mailman : postfix". Larry Turnbull  fixed it by adding the
>'mailman' group.

No errors on the overnights, Things appear to be working correctly.
I'm trying to run the password reminder job now, /var/log/messages
shows it started twenty-three minutes ago, I haven't received any mail
messages for password reminders on any of the lists on this system, so
for all I know it may still be running and I'll just have to wait. If
and when it finishes, I'll report.

From mark at msapiro.net  Sun Aug  9 17:44:56 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 09 Aug 2015 08:44:56 -0700
Subject: [Mailman-Users] Permissions error on all Mailman overnights
In-Reply-To: <g2sesa9uuvfrqqdv7gmavbuj5cbm98vnh8@4ax.com>
References: <l24csatv9bogjv7g63cutgssod5h8dedo7@4ax.com>
 <55C62C0F.4020804@msapiro.net> <012d01d0d1ff$5fdf4dc0$1f9de940$@acbradio.org>
 <55C65AE7.6040900@msapiro.net> <poadsahardhq593rng8tlk1dmqtgd4cb73@4ax.com>
 <55C6B131.8010402@msapiro.net> <g2sesa9uuvfrqqdv7gmavbuj5cbm98vnh8@4ax.com>
Message-ID: <55C77578.9090307@msapiro.net>

On 08/09/2015 08:25 AM, Steve Matzura wrote:
> 
> No errors on the overnights, Things appear to be working correctly.
> I'm trying to run the password reminder job now, /var/log/messages
> shows it started twenty-three minutes ago, I haven't received any mail
> messages for password reminders on any of the lists on this system, so
> for all I know it may still be running and I'll just have to wait. If
> and when it finishes, I'll report.


It shouldn't take that long. What is the exact command you issued (or
did you run it by modifying Mailman's crontab)? What if anything does

ps -fwwC python | grep mailpasswds

report?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From sm at noisynotes.com  Sun Aug  9 21:28:27 2015
From: sm at noisynotes.com (Steve Matzura)
Date: Sun, 09 Aug 2015 15:28:27 -0400
Subject: [Mailman-Users] Permissions error on all Mailman overnights
Message-ID: <ldafsa9mnpf8bf1lto5roddnbv88kbgaae@4ax.com>

On Sun, 09 Aug 2015 08:44:56 -0700, Mark wrote:

>On 08/09/2015 08:25 AM, Steve Matzura wrote:
>> 
>> No errors on the overnights, Things appear to be working correctly.
>> I'm trying to run the password reminder job now, /var/log/messages
>> shows it started twenty-three minutes ago, I haven't received any mail
>> messages for password reminders on any of the lists on this system, so
>> for all I know it may still be running and I'll just have to wait. If
>> and when it finishes, I'll report.
>
>
>It shouldn't take that long. What is the exact command you issued (or
>did you run it by modifying Mailman's crontab)? What if anything does
>
>ps -fwwC python | grep mailpasswds

I changed the crontab to a few minutes later than current time on
today's day of the month, and I did try the process show command and
it showed nothing running.

The command in the crontab is:

/usr/bin/python -S /usr/local/mailman/cron/mailpasswds

>report?

From mark at msapiro.net  Sun Aug  9 21:43:40 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 09 Aug 2015 12:43:40 -0700
Subject: [Mailman-Users] Permissions error on all Mailman overnights
In-Reply-To: <ldafsa9mnpf8bf1lto5roddnbv88kbgaae@4ax.com>
References: <ldafsa9mnpf8bf1lto5roddnbv88kbgaae@4ax.com>
Message-ID: <55C7AD6C.9020903@msapiro.net>

On 08/09/2015 12:28 PM, Steve Matzura wrote:
> 
> I changed the crontab to a few minutes later than current time on
> today's day of the month, and I did try the process show command and
> it showed nothing running.
> 
> The command in the crontab is:
> 
> /usr/bin/python -S /usr/local/mailman/cron/mailpasswds


OK. So mailpasswds is no longer running. Presumably, you are getting
emailed output from cron as you were seeing the prior errors, so if you
received nothing, then the cron produced no output which is normal.

Do you actually have lists with members with the list's send_reminders =
Yes and at least some members "Get password reminder email for this
list?" = Yes?

Is Mailman running and delivering mail?

Is there anything in Mailman's error log (/usr/local/mailman/logs/error)?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From dennisthetiger at chez-vrolet.net  Mon Aug 10 03:11:53 2015
From: dennisthetiger at chez-vrolet.net (Dennis Carr)
Date: Sun, 9 Aug 2015 18:11:53 -0700
Subject: [Mailman-Users] "Too many recipients" hold with only one recipient
	- and I'm owner
Message-ID: <20150809181153.cdcce401be2383f14b847020@chez-vrolet.net>

Today, I posted to a small list for site updates that I have, and I got
a hold message saying I needed to approve my own message on account of
"too many recipients" - which required me to approve my own message. 

I'm list owner, so this should be easy to fix.  However, I cannot find
the option to fix this in the webui.  

Can somebody point me in the right direction?

Thanks,
-Dennis Carr

From mark at msapiro.net  Mon Aug 10 06:38:34 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 09 Aug 2015 21:38:34 -0700
Subject: [Mailman-Users] "Too many recipients" hold with only one
 recipient - and I'm owner
In-Reply-To: <20150809181153.cdcce401be2383f14b847020@chez-vrolet.net>
References: <20150809181153.cdcce401be2383f14b847020@chez-vrolet.net>
Message-ID: <55C82ACA.4080107@msapiro.net>

On 08/09/2015 06:11 PM, Dennis Carr wrote:
> Today, I posted to a small list for site updates that I have, and I got
> a hold message saying I needed to approve my own message on account of
> "too many recipients" - which required me to approve my own message. 
> 
> I'm list owner, so this should be easy to fix.  However, I cannot find
> the option to fix this in the webui.  
> 
> Can somebody point me in the right direction?


Privacy options... -> Recipient filters -> max_num_recipients

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From sm at noisynotes.com  Wed Aug 12 03:12:06 2015
From: sm at noisynotes.com (Steve Matzura)
Date: Tue, 11 Aug 2015 21:12:06 -0400
Subject: [Mailman-Users] No luck on the overnights
Message-ID: <fp6lsal4p9sf76plfg9l6g3fifgl8vujqi@4ax.com>

I'm now thinking none of the overnight cron jobs are running. I set
digest mode to a couple lists, posted to them, then modified the
digest job's run time from 12:00 every day to a minute or two from
current time, waited ten minutes, and got nothing in response. Then I
did the same for the monthly password reminder job. Same thing--no
response. I'm thinking the best way to find out what's going on is to
log into the mailman account interactively and manually run one of the
cron commands to find out what happens. Any other suggestions most
welcome. Probably whatever's wrong with one job is wrong with them
all.

From mark at msapiro.net  Wed Aug 12 03:28:59 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 11 Aug 2015 18:28:59 -0700
Subject: [Mailman-Users] No luck on the overnights
In-Reply-To: <fp6lsal4p9sf76plfg9l6g3fifgl8vujqi@4ax.com>
References: <fp6lsal4p9sf76plfg9l6g3fifgl8vujqi@4ax.com>
Message-ID: <55CAA15B.2010903@msapiro.net>

On 08/11/2015 06:12 PM, Steve Matzura wrote:
> I'm now thinking none of the overnight cron jobs are running. I set
> digest mode to a couple lists, posted to them, then modified the
> digest job's run time from 12:00 every day to a minute or two from
> current time, waited ten minutes, and got nothing in response. Then I
> did the same for the monthly password reminder job. Same thing--no
> response. I'm thinking the best way to find out what's going on is to
> log into the mailman account interactively and manually run one of the
> cron commands to find out what happens. Any other suggestions most
> welcome. Probably whatever's wrong with one job is wrong with them
> all.


You can run a mailman cron just the way cron runs it by

sudo -u mailman python -S /usr/local/mailman/cron/xxx

where xxx is the cron job you want to run.

Note that if cron runs the job and it produces any output, it is mailed
to the 'mailman' user which in most cases is the Mailman site list.
Thus, to see this, you need to ensure that you are a member of the site
list with delivery enabled and that the site list's
generic_nonmember_action is accept.

The alternative is to add

MAILTO=you at your.domain

to the beginning of Mailman's crontab to tell cron to mail any output to
you instead of 'mailman'.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From pete at bossley.me  Wed Aug 12 15:21:58 2015
From: pete at bossley.me (Peter Bossley)
Date: Wed, 12 Aug 2015 13:21:58 +0000
Subject: [Mailman-Users] DKIM Failures cause posts from gmail users to not
 be relayed to the list
Message-ID: <BY1PR12MB04879850D83926537DE4660DA57E0@BY1PR12MB0487.namprd12.prod.outlook.com>

This is going to be a lengthy explanation, as I've spent a bit of time troubleshooting this issue.
I am running Mailman 2.1.20 as part of a server running WHM/cpanel.
The MTA is Exim.
The MTA was configured to reject DKIM failures.
The domain was configured to sign outgoing messages with DKIM.
We noticed that when messages were posted by gmail users, they would appear in the list archives but they would not be delivered to any list members. Posts by other domains such as my custom office 365 domain worked fine and were delivered to everyone including gmail users.
Of course my first stop was the logs, and I saw entries like this in the smtp-failure log:
Aug 11 22:06:50 2015 (3128) SMTP session failure: 550, DKIM: encountered the following problem validating gmail.com:
signature_incorrect, msgid: <CAHtjcYNyqX8Na44GC9GKUsS=2FbS=HD1ofu3GqcJkZuRomwreQ at mail.gmail.com>
Aug 11 22:06:50 2015 (3128) SMTP session failure: 550, DKIM: encountered the following problem validating gmail.com:
signature_incorrect, msgid: <CAHtjcYNyqX8Na44GC9GKUsS=2FbS=HD1ofu3GqcJkZuRomwreQ at mail.gmail.com>
Thinking that our signing of DKIM was causing issues, I shut that off. That didn't change anything.
So, next, thinking that the DMARC issues that have been plaguing the internet lately were to blame, I tried changing the DMARC_Moderation setting to munge. This failed to change the situation as well.
I then attempted to set this setting to wrap message, which again did not fix the issue.
At this point, I moved on to the from as list global setting, and tried munge here as well. This didn't work.
Last, I tried wrap message, which did seem to work. Given the functionality issues this created, however, I decided to keep investigating.
It was at this point that I decided to turn off DKIM failure rejection. I initially dismissed this course of action because I felt that changing the from as list setting to munge should have prevented this from becoming an issue. Since the initial posts were making it to the web-based archives I figured the gmail signature was fine.
I'm at a loss of where to go from here. I would like to still reject DKIM failures, but my mailing lists need to work properly as well. Does anyone have any suggestions or ideas on why the Munge setting didn't seem to have an impact?





From brad at fineby.me.uk  Wed Aug 12 15:44:12 2015
From: brad at fineby.me.uk (Brad Rogers)
Date: Wed, 12 Aug 2015 14:44:12 +0100
Subject: [Mailman-Users] DKIM Failures cause posts from gmail users to
 not be relayed to the list
In-Reply-To: <BY1PR12MB04879850D83926537DE4660DA57E0@BY1PR12MB0487.namprd12.prod.outlook.com>
References: <BY1PR12MB04879850D83926537DE4660DA57E0@BY1PR12MB0487.namprd12.prod.outlook.com>
Message-ID: <20150812144412.58171444@abydos.stargate.org.uk>

On Wed, 12 Aug 2015 13:21:58 +0000
Peter Bossley <pete at bossley.me> wrote:

Hello Peter,

>mailing lists need to work properly as well. Does anyone have any
>suggestions or ideas on why the Munge setting didn't seem to have an
>impact?

I'm far from being an expert regarding DKIM, DKMS and mailman, but what
I can say is this;

*All* lists run from list.debian.org are to have their footers turned off
because of valid DKIM signature breakage.

Maybe this is an option you could also explore.

See
<https://lists.debian.org/debian-devel-announce/2015/08/msg00003.html>
for the announcement.  Sadly, very little in the way of details, but the
poster of the message may be able to help you.

-- 
 Regards  _
         / )           "The blindingly obvious is
        / _)rad        never immediately apparent"
Drums quite good, bass is too loud, and I can't hear the words
Sound Of The Suburbs - Members
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20150812/86bed09d/attachment.sig>

From barry at list.org  Wed Aug 12 16:04:14 2015
From: barry at list.org (Barry Warsaw)
Date: Wed, 12 Aug 2015 10:04:14 -0400
Subject: [Mailman-Users] DKIM Failures cause posts from gmail users to
 not be relayed to the list
In-Reply-To: <20150812144412.58171444@abydos.stargate.org.uk>
References: <BY1PR12MB04879850D83926537DE4660DA57E0@BY1PR12MB0487.namprd12.prod.outlook.com>
 <20150812144412.58171444@abydos.stargate.org.uk>
Message-ID: <20150812100414.3bc943b9@anarchist.wooz.org>

On Aug 12, 2015, at 02:44 PM, Brad Rogers wrote:

>*All* lists run from list.debian.org are to have their footers turned off
>because of valid DKIM signature breakage.

FWIW, lists.debian.org does not run Mailman.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20150812/d06c07ae/attachment.sig>

From mark at msapiro.net  Wed Aug 12 16:26:05 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 12 Aug 2015 07:26:05 -0700
Subject: [Mailman-Users] DKIM Failures cause posts from gmail users to
 not be relayed to the list
In-Reply-To: <BY1PR12MB04879850D83926537DE4660DA57E0@BY1PR12MB0487.namprd12.prod.outlook.com>
References: <BY1PR12MB04879850D83926537DE4660DA57E0@BY1PR12MB0487.namprd12.prod.outlook.com>
Message-ID: <55CB577D.9080209@msapiro.net>

On 08/12/2015 06:21 AM, Peter Bossley wrote:

> The MTA was configured to reject DKIM failures.

This is wrong and is the cause of your issue. See RFC 6376
<http://www.rfc-editor.org/rfc/rfc6376.txt> sec 4.4, sec 6.1 and sec 6.3.

The issue is your mail list transformations break gmail's DKIM signature
and you are rejecting the outgoing mail because of the invalid
signature, in spit of the fact that it may also contain a valid
signature. Even if it doesn't also contain a valid signature, mail
should not be rejected just because of an invalid DKIM signature. In
most cases an invalid DKIM signature should be treated the same as no
signature.


> The domain was configured to sign outgoing messages with DKIM.


OK.


> So, next, thinking that the DMARC issues that have been plaguing the internet lately were to blame, I tried changing the DMARC_Moderation setting to munge. This failed to change the situation as well.


This is not a DMARC issue per se as gmail's DMARC policy is p=none.


> I then attempted to set this setting to wrap message, which again did not fix the issue.


Because gmail's DMARC policy is p=none, dmarc_moderation_action won't
apply to this mail.


> At this point, I moved on to the from as list global setting, and tried munge here as well. This didn't work.
> Last, I tried wrap message, which did seem to work.


Because the outer wrapper message only contains your DKIM signature.
Gmail's is in the wrapped message which is part of the message body and
not checked by your MTA.


> Given the functionality issues this created, however, I decided to keep investigating.
> It was at this point that I decided to turn off DKIM failure rejection. I initially dismissed this course of action because I felt that changing the from as list setting to munge should have prevented this from becoming an issue.


No. Turning off DKIM failure rejection or at least changing it to ignore
a failure if there is also a valid DKIM sig present was the correct
solution.


> Since the initial posts were making it to the web-based archives I figured the gmail signature was fine.


The sig was fine in the incoming mail, but transformations like subject
prefixing and the addition of a message header or footer break the sig
in the outgoing mail.


> I'm at a loss of where to go from here. I would like to still reject DKIM failures, but my mailing lists need to work properly as well. Does anyone have any suggestions or ideas on why the Munge setting didn't seem to have an impact?


All Munging the From: does is create one more failure in gmail's DKIM
sig. This is not a DMARC issue. Do not reject messages just because they
happen to contain one invalid DKIM sig. This is wrong. Read the RFC
sections I refer to above.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Wed Aug 12 16:32:51 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 12 Aug 2015 07:32:51 -0700
Subject: [Mailman-Users] DKIM Failures cause posts from gmail users to
 not be relayed to the list
In-Reply-To: <20150812144412.58171444@abydos.stargate.org.uk>
References: <BY1PR12MB04879850D83926537DE4660DA57E0@BY1PR12MB0487.namprd12.prod.outlook.com>
 <20150812144412.58171444@abydos.stargate.org.uk>
Message-ID: <55CB5913.5090703@msapiro.net>

On 08/12/2015 06:44 AM, Brad Rogers wrote:
> 
> *All* lists run from list.debian.org are to have their footers turned off
> because of valid DKIM signature breakage.


In order to avoid DKIM signature breakage, you also have to turn off
subject prefixing, content filtering, reply-to header munging and
message headers.

See item 2) at <http://wiki.list.org/x/17891458> although the OP's issue
is not with DMARC; it is with his own outgoing MTA being too fussy about
a broken sig.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20150812/d184f238/attachment-0001.sig>

From brad at fineby.me.uk  Wed Aug 12 16:40:54 2015
From: brad at fineby.me.uk (Brad Rogers)
Date: Wed, 12 Aug 2015 15:40:54 +0100
Subject: [Mailman-Users] DKIM Failures cause posts from gmail users to
 not be relayed to the list
In-Reply-To: <20150812100414.3bc943b9@anarchist.wooz.org>
References: <BY1PR12MB04879850D83926537DE4660DA57E0@BY1PR12MB0487.namprd12.prod.outlook.com>
 <20150812144412.58171444@abydos.stargate.org.uk>
 <20150812100414.3bc943b9@anarchist.wooz.org>
Message-ID: <20150812154054.3db472c6@abydos.stargate.org.uk>

On Wed, 12 Aug 2015 10:04:14 -0400
Barry Warsaw <barry at list.org> wrote:

Hello Barry,

>FWIW, lists.debian.org does not run Mailman.

Fair enough.  Seems to me to be less likely that Peter's problem is the
same, as other list owners of mailman run lists would probably be
reporting similar errors.

-- 
 Regards  _
         / )           "The blindingly obvious is
        / _)rad        never immediately apparent"
Sign away your life
Tin Soldiers - Stiff Little Fingers
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20150812/67b06914/attachment.sig>

From martin at skjoldebrand.eu  Thu Aug 13 09:17:55 2015
From: martin at skjoldebrand.eu (Martin =?ISO-8859-1?Q?Skj=F6ldebrand?=)
Date: Thu, 13 Aug 2015 09:17:55 +0200
Subject: [Mailman-Users] client denied by server configuration:
 /usr/lib/cgi-bin/mailman/listinfo
Message-ID: <1439450275.10391.6.camel@skjoldebrand.eu>

Hi,

Trying to get mailman to behave using this guide:https://www.howtoforg
e.com/how-to-install-and-configure-mailman-with-postfix-on-debian
-squeeze even though I'm on Jessie.

I get the above error in /var/log/apache2/list-error 

[Thu Aug 13 09:09:10.563936 2015] [access_compat:error] [pid 30067]
[client 37.196.46.254:58989] AH01797: client denied by server
configuration: /usr/lib/cgi-bin/mailman/listinfo
 
when trying to access the mailman site, and think I've narrowed it
down to the following line in the setup

ScriptAlias / /usr/lib/cgi-bin/mailman/listinfo

as commenting it out actually displays a crippled site. Obviously this
will not do. Permissions on the directory looks correct

# ls -l /usr/lib/cgi-bin/mailman/
total 132
-rwxr-xr-x 1 www-data list 10200 Apr  6 17:45 admin
-rwxr-xr-x 1 www-data list 10200 Apr  6 17:45 admindb
-rwxr-xr-x 1 www-data list 10200 Apr  6 17:45 confirm
-rwxr-xr-x 1 www-data list 10200 Apr  6 17:45 create
-rwxr-xr-x 1 www-data list 10200 Apr  6 17:45 edithtml
-rwxr-xr-x 1 www-data list 10200 Apr  6 17:45 listinfo
-rwxr-xr-x 1 www-data list 10200 Apr  6 17:45 options
-rwxr-xr-x 1 www-data list 10200 Apr  6 17:45 private
-rwxr-xr-x 1 www-data list 10200 Apr  6 17:45 rmlist
-rwxr-xr-x 1 www-data list 10200 Apr  6 17:45 roster
-rwxr-xr-x 1 www-data list 10200 Apr  6 17:45 subscribe

Or at least not more fubar than the others =)

Any pointers as to where to take troubleshooting from here?

/Martin S
-- 
*****************************************************************This address is for technical mail lists only!Please you my main address at the .org domain for all other matters.*****************************************************************

From mark at msapiro.net  Thu Aug 13 15:40:00 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 13 Aug 2015 06:40:00 -0700
Subject: [Mailman-Users] client denied by server configuration:
	/usr/lib/cgi-bin/mailman/listinfo
In-Reply-To: <1439450275.10391.6.camel@skjoldebrand.eu>
References: <1439450275.10391.6.camel@skjoldebrand.eu>
Message-ID: <55CC9E30.7060606@msapiro.net>

On 08/13/2015 12:17 AM, Martin Skj?ldebrand wrote:
> Hi,
> 
> Trying to get mailman to behave using this guide:https://www.howtoforg
> e.com/how-to-install-and-configure-mailman-with-postfix-on-debian
> -squeeze even though I'm on Jessie.


See the FAQ at <http://wiki.list.org/x/12812344>


> I get the above error in /var/log/apache2/list-error 
> 
> [Thu Aug 13 09:09:10.563936 2015] [access_compat:error] [pid 30067]
> [client 37.196.46.254:58989] AH01797: client denied by server
> configuration: /usr/lib/cgi-bin/mailman/listinfo


See the installation manual at
<http://www.list.org/mailman-install/node10.html>.

Somewhere in your apache config, maybe in /etc/apache2/conf.d/mailman,
you need a

<Directory "/usr/lib/cgi-bin/">

block an within that block you probably need (for apache 2.4)

    Require all granted

If you have

        Order allow,deny
        Allow from all

that's only for older apache versions.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From adam-mailman at amyl.org.uk  Thu Aug 13 15:40:43 2015
From: adam-mailman at amyl.org.uk (Adam McGreggor)
Date: Thu, 13 Aug 2015 14:40:43 +0100
Subject: [Mailman-Users] client denied by server configuration:
 /usr/lib/cgi-bin/mailman/listinfo
In-Reply-To: <1439450275.10391.6.camel@skjoldebrand.eu>
References: <1439450275.10391.6.camel@skjoldebrand.eu>
Message-ID: <20150813134043.GV9573@hendricks.amyl.org.uk>

On Thu, Aug 13, 2015 at 09:17:55AM +0200, Martin Skj?ldebrand wrote:
> Hi,
> 
> Trying to get mailman to behave using this guide:https://www.howtoforg
> e.com/how-to-install-and-configure-mailman-with-postfix-on-debian
> -squeeze even though I'm on Jessie.
> 
> I get the above error in /var/log/apache2/list-error 
> 
> [Thu Aug 13 09:09:10.563936 2015] [access_compat:error] [pid 30067]
> [client 37.196.46.254:58989] AH01797: client denied by server
> configuration: /usr/lib/cgi-bin/mailman/listinfo

See the 'Access control' section of

    http://httpd.apache.org/docs/2.4/upgrading.html

perhaps

-- 
"Ink is handicapped, in a way, because you can blow up a man with gunpowder in
 half a second, while it may take twenty years to blow him up with a book. But
 the gunpowder destroys itself along with its victim, while a book can keep on
 exploding for centuries."
    -- Christopher Morley

From tom.browder at gmail.com  Fri Aug 14 14:38:30 2015
From: tom.browder at gmail.com (Tom Browder)
Date: Fri, 14 Aug 2015 07:38:30 -0500
Subject: [Mailman-Users] Moving MM2 to a new server (revisited).
Message-ID: <CAFMGiz9jeHQzFvONv_fa4h=RtdM+w0mCQbZUkXVeW9NHDUca+w@mail.gmail.com>

I saw several recent msgs about moving MM2 from one server to another,
but I need some more specific advice for my situation:

Old server (Ubuntu 10.04 LTS, 64-bit)
===========================

apache:    version 2.2.14 (2.2.14-5ubuntu8.15)
postfix:     version 2.7.0 (2.7.0-1ubuntu0.2)
mailman:  version 2.1.13 (1:2.1.13-1ubuntu0.2)

MM2/Postfix stopped sending messages back in April. I was clueless
until after I shutdown postfix a couple of days ago and have done
nothing to the old server's files.  I'm almost positive there is
undelivered mail there.

New server (Debian 7, 64bit)
====================

openssl: version 1.0.2d

I am successfully running Apache 2.4.16 with several virtual hosts.

I am going to install the latest Postfix (from src).  I will install
the latest MM2 (from src).

Plan
===

1. Get a working MM2 with one or more new lists associated with one or
more virtual hosts (with no new names conflicting with names from the
dormant lists).

2. Follow previous threads on transferring old lists to the new, working MM2.

Question:
=======

Is is necessary to restart Apache2, Postfix, and MM2 on the old server
in order to recover the undelivered mail, or can I somehow magically
copy files to the new server before that effort.  Note I haven't yet
looked at the logs on the old server.

Thanks.

Best regards,

-Tom

From mark at msapiro.net  Sat Aug 15 02:13:51 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 14 Aug 2015 17:13:51 -0700
Subject: [Mailman-Users] Moving MM2 to a new server (revisited).
In-Reply-To: <CAFMGiz9jeHQzFvONv_fa4h=RtdM+w0mCQbZUkXVeW9NHDUca+w@mail.gmail.com>
References: <CAFMGiz9jeHQzFvONv_fa4h=RtdM+w0mCQbZUkXVeW9NHDUca+w@mail.gmail.com>
Message-ID: <55CE843F.3000008@msapiro.net>

On 08/14/2015 05:38 AM, Tom Browder wrote:
> 
> Question:
> =======
> 
> Is is necessary to restart Apache2, Postfix, and MM2 on the old server
> in order to recover the undelivered mail, or can I somehow magically
> copy files to the new server before that effort.  Note I haven't yet
> looked at the logs on the old server.


No. You shouldn't need to start the web server, mail server or Mailman
on the old server.

What you do need to do depends on where this undelivered mail is. There
are 4 possibilities - queued in Postfix to be delivered to Mailman, in
Mailman's qfiles/in queue waiting to be processed by Mailman, in some
combination of Mailman's qfiles/out and qfiles/retry queues waiting to
be delivered to Postfix or queued in Postfix for delivery to list members.

As far as Mailman's queues are concerned, you should be able to just
move the qfiles/*/*.pck files to the new server AFTER moving the
lists/*/config.pck files.

As far as Postfix is concerned, it should be possible to move those
queue entries too, but it's probably easier to start Postfix on the old
server and let it deliver outbound mail if any and queue inbound mail if
any in Mailman's qfiles/in before you move it.

There may also be Mailman held messages. For these, you need to move the
data/heldmsg* files to the new server and either move the
lists/*/request.pck files or use a script like
<https://www.msapiro.net/scripts/hold_again> to reprocess the
data/heldmsg* files.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From tom.browder at gmail.com  Sat Aug 15 04:07:32 2015
From: tom.browder at gmail.com (Tom Browder)
Date: Fri, 14 Aug 2015 21:07:32 -0500
Subject: [Mailman-Users] Moving MM2 to a new server (revisited).
In-Reply-To: <55CE843F.3000008@msapiro.net>
References: <CAFMGiz9jeHQzFvONv_fa4h=RtdM+w0mCQbZUkXVeW9NHDUca+w@mail.gmail.com>
 <55CE843F.3000008@msapiro.net>
Message-ID: <CAFMGiz9S_XOUEw5_t+UpodW9vkN1MucXkVaXrbP3DStwAeEusA@mail.gmail.com>

On Aug 14, 2015 7:14 PM, "Mark Sapiro" <mark at msapiro.net> wrote:
> On 08/14/2015 05:38 AM, Tom Browder wrote:
> > Is is necessary to restart Apache2, Postfix, and MM2 on the old server
...
> No. You shouldn't need to start the web server, mail server or Mailman
> on the old server.
...

Thanks so much, Mark!

Best,

-Tom

From martin at skjoldebrand.eu  Sat Aug 15 08:25:52 2015
From: martin at skjoldebrand.eu (Martin =?ISO-8859-1?Q?Skj=F6ldebrand?=)
Date: Sat, 15 Aug 2015 08:25:52 +0200
Subject: [Mailman-Users] Postfix with MySQL
Message-ID: <1439619952.1543.13.camel@skjoldebrand.eu>

Hi,

I've been looking at the mailman installation instructions and
obviously in my installation mailman isn't correctly integrated with
Postfix. Postfix is the host for 4-5 domains of which 1 (or at most 3)
would be using masiling lists. (I can run all on one domain as well).
In the instruction it says that you should do to postfix main.cf:

virtual_alias_maps = <your normal virtual alias files>,
        hash:/usr/local/mailman/data/virtual-mailman

while editing mm_cfg.py to include

POSTFIX_STYLE_VIRTUAL_DOMAINS = ['dom2.ain', 'dom3.ain']

As my Postfix installation uses MySQL as a backend it all become
slightly confusing. And in the end when trying to send mail to the
list I get 550 errors says the virtual address isn't found.

Is there a pointer to a Postfix w/ MySQL and mailman resource? Apart
from http://wiki.list.org/DOC/Mailman%20-%20Postfix%20integration%20wi
th%20Mailman%20generated%20transport%20or%20other%20maps  


For one thing I'm not sure that I understand 
"Create /path/to/data/virtual_to_transport with the following content
sed -r -e 's/(^[^#]\S+\s+).+$/\1local:/' $1 \
 > /path/to/data/transport-mailman
/usr/sbin/postmap /path/to/data/transport-mailman"

Should that create a virtual_to_transport from these statements? Or
should I first create the file and then feed it with the above? Also
I'm not altogether certain that it covers mysql-based postfix setups. 


/Martin S
--
*************************************************
-- 
*************************************************
This address is for technical mail lists only.
For all other matters, please use my main address
at the .org domain.
*************************************************

From mark at msapiro.net  Sat Aug 15 18:29:24 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 15 Aug 2015 09:29:24 -0700
Subject: [Mailman-Users] Postfix with MySQL
In-Reply-To: <1439619952.1543.13.camel@skjoldebrand.eu>
References: <1439619952.1543.13.camel@skjoldebrand.eu>
Message-ID: <55CF68E4.9050807@msapiro.net>

On 08/14/2015 11:25 PM, Martin Skj?ldebrand wrote:
> In the instruction it says that you should do to postfix main.cf:
> 
> virtual_alias_maps = <your normal virtual alias files>,
>         hash:/usr/local/mailman/data/virtual-mailman
> 
> while editing mm_cfg.py to include
> 
> POSTFIX_STYLE_VIRTUAL_DOMAINS = ['dom2.ain', 'dom3.ain']
> 
> As my Postfix installation uses MySQL as a backend it all become
> slightly confusing. And in the end when trying to send mail to the
> list I get 550 errors says the virtual address isn't found.


I'm not too well versed in mixing postfix hash: and mysql: tables, but
if you post the output from `postconf -n`, I may be able to help you get
this working in the "normal" way.

Also, what's in data/virtual-mailman?

and, if you didn't create the lists that have addresses in 'dom2.ain'
and 'dom3.ain' after adding

POSTFIX_STYLE_VIRTUAL_DOMAINS = ['dom2.ain', 'dom3.ain']

to mm_cfg.py, did you run Mailman's bin/genaliases?


> Is there a pointer to a Postfix w/ MySQL and mailman resource? Apart
> from http://wiki.list.org/DOC/Mailman%20-%20Postfix%20integration%20with%20Mailman%20generated%20transport%20or%20other%20maps  


Not that I know of.


> For one thing I'm not sure that I understand 
> "Create /path/to/data/virtual_to_transport with the following content
> sed -r -e 's/(^[^#]\S+\s+).+$/\1local:/' $1 \
>  > /path/to/data/transport-mailman
> /usr/sbin/postmap /path/to/data/transport-mailman"
> 
> Should that create a virtual_to_transport from these statements? Or
> should I first create the file and then feed it with the above? Also
> I'm not altogether certain that it covers mysql-based postfix setups. 


The method in the FAQ is an alternative that should work in your case.
In step 3, suppose that Mailman's data directory is
/var/lib/mailman/data. If not, adjust the following for the correct path.

Create the file /var/lib/mailman/data/virtual_to_transport and put the
three lines

sed -r -e 's/(^[^#]\S+\s+).+$/\1local:/' $1 \
 > /var/lib/mailman/data/transport-mailman
/usr/sbin/postmap /var/lib/mailman/data/transport-mailman

in that file. I.e. "first create the file and then feed it with the above".

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From martin at skjoldebrand.eu  Sun Aug 16 08:35:32 2015
From: martin at skjoldebrand.eu (Martin =?ISO-8859-1?Q?Skj=F6ldebrand?=)
Date: Sun, 16 Aug 2015 08:35:32 +0200
Subject: [Mailman-Users] Postfix with MySQL
In-Reply-To: <55CF68E4.9050807@msapiro.net>
References: <1439619952.1543.13.camel@skjoldebrand.eu>
 <55CF68E4.9050807@msapiro.net>
Message-ID: <1439706932.1160.4.camel@skjoldebrand.eu>

On Sat, 2015-08-15 at 09:29 -0700, Mark Sapiro wrote:
> 
> Also, what's in data/virtual-mailman?

I don't have one, maybe that's the problem =)
This is one of the confusing points, because all virtual-domain stuff
is handled by MySQL tables in my setup.


to mm_cfg.py, did you run Mailman's bin/genaliases?


Yes.



Anyway; output of postconf -n is:
alias_database = hash:/etc/aliasesalias_maps = hash:/etc/aliases, hash:/var/lib/mailman/data/aliasesappend_dot_mydomain = nobiff = nobroken_sasl_auth_clients = yesconfig_directory = /etc/postfixcontent_filter = smtp-amavis:[127.0.0.1]:10024delay_warning_time = 4hhtml_directory = /usr/share/doc/postfix/htmlinet_interfaces = allmailbox_size_limit = 0mailman_destination_recipient_limit = 1mydestination = mail.skjoldebrand.eu, localhost, localhost.localdomainmyhostname = mail.skjoldebramd.eumynetworks = 127.0.0.0/8myorigin = /etc/mailnameproxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $virtual_mailbox_limit_mapsreadme_directory = /usr/share/doc/postfixrecipient_delimiter = +relay_domains = lists.skjoldebrand.eurelayhost =smtp_tls_ciphers = mediumsmtp_tls_exclude_ciphers = MD5, SRP, PSK, aDSS, kECDH, kDH, SEED, IDEA, RC2, RC5smtp_tls_protocols = !SSLv2, !SSLv3smtp_tls_session_cache_database = btree:${data_directory}/smtp_scachesmtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)smtpd_helo_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_hostname, reject_invalid_hostname, permitsmtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_pipelining, reject_non_fqdn_recipient, reject_unknown_recipient_domain, reject_unauth_destination, reject_rbl_client sbl-xbl.spamhaus.org, reject_rbl_client cbl.abuseat.org, reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net, reject_rbl_client cbl.abuseat.org, reject_rbl_client dnsbl.sorbs.net, permitsmtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destinationsmtpd_sasl_auth_enable = yessmtpd_sasl_authenticated_header = yessmtpd_sender_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_non_fqdn_sender, reject_unknown_sender_domain, permitsmtpd_tls_auth_only = yessmtpd_tls_cert_file = /etc/postfix/smtpd.certsmtpd_tls_ciphers = mediumsmtpd_tls_key_file = /etc/postfix/smtpd.keysmtpd_tls_protocols = !SSLv2, !SSLv3smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scachesmtpd_use_tls = yestransport_maps = proxy:mysql:/etc/postfix/mysql-virtual_transports.cf hash:/var/lib/mailman/data/transport-mailmanunknown_local_recipient_reject_code = 550virtual_alias_domains =virtual_alias_maps = proxy:mysql:/etc/postfix/mysql-virtual_forwardings.cf, mysql:/etc/postfix/mysql-virtual_email2email.cfvirtual_gid_maps = static:5000virtual_mailbox_base = /home/vmailvirtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql-virtual_domains.cfvirtual_mailbox_limit_maps = proxy:mysql:/etc/postfix/mysql-virtual_mailbox_limit_maps.cfvirtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql-virtual_mailboxes.cfvirtual_uid_maps = static:5000
/Martin S

-- 
*************************************************
This address is for technical mail lists only.
For all other matters, please use my main address
at the .org domain.
*************************************************

From 1nickt at gmail.com  Sun Aug 16 02:57:25 2015
From: 1nickt at gmail.com (Nick Tonkin)
Date: Sat, 15 Aug 2015 17:57:25 -0700
Subject: [Mailman-Users] Attachments sent with Apple Mail being stripped
Message-ID: <CAEqhxnA7oWoVyg87d7Wh38MFd9gccKH5r1SrYzKE8=L4d+5Hvg@mail.gmail.com>

Hi all, I'm having a problem with mailman stripping attachments from
messages sent with Apple Mail. It looks like the content-type is
multipart/related, but I have added that to my list of allowed
content-types and the jpeg still gets removed. Any ideas please?

- nick

From mark at msapiro.net  Sun Aug 16 16:18:22 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 16 Aug 2015 07:18:22 -0700
Subject: [Mailman-Users] Attachments sent with Apple Mail being stripped
In-Reply-To: <CAEqhxnA7oWoVyg87d7Wh38MFd9gccKH5r1SrYzKE8=L4d+5Hvg@mail.gmail.com>
References: <CAEqhxnA7oWoVyg87d7Wh38MFd9gccKH5r1SrYzKE8=L4d+5Hvg@mail.gmail.com>
Message-ID: <55D09BAE.8090808@msapiro.net>

On 08/15/2015 05:57 PM, Nick Tonkin wrote:
> Hi all, I'm having a problem with mailman stripping attachments from
> messages sent with Apple Mail. It looks like the content-type is
> multipart/related, but I have added that to my list of allowed
> content-types and the jpeg still gets removed. Any ideas please?


I recommend that instead of putting 'multipart/this' and
'multipart/that' in pass_mime_types, that you put just 'multipart' to
allow all multipart parts and also put those elemental types you want.

Do you have collapse_alternatives = Yes? If so, and the images are in an
alternative part, that would explain it. E.g., if the message structure is

multipart/alternative
    text/plain
    multipart/related
        text/html
        image/jpeg

the multipart/alternative part will be replaced by the first alternative
leaving only that text/plain part.

Also, Apple Mail tends to create messages that don't render well with
some non-Apple MUAs. You might try setting filter_content = No to see if
the problem is content filtering or the viewing MUA.

If that doesn't explain your issue, post all your content filtering
settings and a raw example message. You can strip the body parts from
the message, but we'd want to see at least all the Content-Type headers
and boundaries.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Aug 16 16:54:09 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 16 Aug 2015 07:54:09 -0700
Subject: [Mailman-Users] Postfix with MySQL
In-Reply-To: <1439706932.1160.4.camel@skjoldebrand.eu>
References: <1439619952.1543.13.camel@skjoldebrand.eu>
 <55CF68E4.9050807@msapiro.net> <1439706932.1160.4.camel@skjoldebrand.eu>
Message-ID: <55D0A411.1000301@msapiro.net>

On 08/15/2015 11:35 PM, Martin Skj?ldebrand wrote:
> On Sat, 2015-08-15 at 09:29 -0700, Mark Sapiro wrote:
>>
>> Also, what's in data/virtual-mailman?
> 
> I don't have one, maybe that's the problem =)


Yes, that is the problem. You posted that you had in mm_cfg.py


POSTFIX_STYLE_VIRTUAL_DOMAINS = ['dom2.ain', 'dom3.ain']

I assumed that 'dom2.ain' and 'dom3.ain' were just generic replacements
for what you actually had, but perhaps not.

What you need is

POSTFIX_STYLE_VIRTUAL_DOMAINS = ['lists.skjoldebrand.eu']

or possibly a longer list including the host_name attributes other than
'mail.skjoldebrand.eu' of all your lists.


...
> Anyway; output of postconf -n is:

It would have been more helpful if you had posted this in a way that
didn't strip the line breaks.


> alias_database = hash:/etc/aliasesalias_maps = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases

alias_maps is good.

append_dot_mydomain = nobiff = nobroken_sasl_auth_clients =
yesconfig_directory = /etc/postfixcontent_filter =
smtp-amavis:[127.0.0.1]:10024delay_warning_time = 4hhtml_directory =
/usr/share/doc/postfix/htmlinet_interfaces = allmailbox_size_limit =
0mailman_destination_recipient_limit = 1

mailman_destination_recipient_limit is unnecessary. It is for the highly
discouraged postfix_to_mailman.py delivery method.

mydestination = mail.skjoldebrand.eu, localhost,
localhost.localdomainmyhostname = mail.skjoldebramd.eumynetworks =
127.0.0.0/8myorigin = /etc/mailnameproxy_read_maps =
$local_recipient_maps $mydestination $virtual_alias_maps
$virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains
$relay_recipient_maps $relay_domains $canonical_maps
$sender_canonical_maps $recipient_canonical_maps $relocated_maps
$transport_maps $mynetworks $virtual_mailbox_limit_mapsreadme_directory
= /usr/share/doc/postfixrecipient_delimiter = +relay_domains =
lists.skjoldebrand.eu

relay_domains is probably unnecessary.

relayhost =smtp_tls_ciphers = mediumsmt
>  p_tls_exclude_ciphers = MD5, SRP, PSK, aDSS, kECDH, kDH, SEED, IDEA, RC2, RC5smtp_tls_protocols = !SSLv2, !SSLv3smtp_tls_session_cache_database = btree:${data_directory}/smtp_scachesmtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)smtpd_helo_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_hostname, reject_invalid_hostname, permitsmtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_pipelining, reject_non_fqdn_recipient, reject_unknown_recipient_domain, reject_unauth_destination, reject_rbl_client sbl-xbl.spamhaus.org, reject_rbl_client cbl.abuseat.org, reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net, reject_rbl_client cbl.abuseat.org, reject_rbl_client dnsbl.sorbs.net, permitsmtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destinationsmtpd_sasl_auth_enable = yessmtpd_sasl_authenticated_header = yessmtpd_sender_restrictions = permit_sasl_authenticated, per
 m
i
>  t_mynetworks, reject_non_fqdn_sender, reject_unknown_sender_domain, permitsmtpd_tls_auth_only = yessmtpd_tls_cert_file = /etc/postfix/smtpd.certsmtpd_tls_ciphers = mediumsmtpd_tls_key_file = /etc/postfix/smtpd.keysmtpd_tls_protocols = !SSLv2, !SSLv3smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scachesmtpd_use_tls = yestransport_maps = proxy:mysql:/etc/postfix/mysql-virtual_transports.cf hash:/var/lib/mailman/data/transport-mailman

remove hash:/var/lib/mailman/data/transport-mailman from transport_maps

unknown_local_recipient_reject_code = 550virtual_alias_domains
=virtual_alias_maps =
proxy:mysql:/etc/postfix/mysql-virtual_forwardings.cf,
mysql:/etc/postfix/mysql-virtual_email2email.cf

add lists.skjoldebrand.eu to virtual_alias_domains and add
hash:/var/lib/mailman/data/virtual-mailman to virtual_alias_maps

virtual_gid_maps = static:5000virtual_mailbox_base =
/home/vmailvirtual_mailbox_domains =
proxy:mysql:/etc/postfix/mysql-virtual_domains.cfvirtual_mailbox_limit_maps
=
proxy:mysql:/etc/postfix/mysql-virtual_mailbox_limit_maps.cfvirtual_mailbox_maps
= proxy:mysql:/etc/postfix/mysql-virtual_mailboxes.cfvirtual_uid_maps =
static:5000


Make the above changes to main.cf, fix POSTFIX_STYLE_VIRTUAL_DOMAINS in
mm_cfg.py and remove "POSTFIX_MAP_CMD =
'/path/to/data/virtual_to_transport'" from mm_cfg.py.

Then run Mailman's bin/genaliases and see if it now works.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From martin at skjoldebrand.eu  Sun Aug 16 18:31:13 2015
From: martin at skjoldebrand.eu (Martin =?utf-8?b?U2tqw7ZsZGVicmFuZA==?=)
Date: Sun, 16 Aug 2015 18:31:13 +0200
Subject: [Mailman-Users] Postfix with MySQL
In-Reply-To: <55D0A411.1000301@msapiro.net>
References: <1439619952.1543.13.camel@skjoldebrand.eu>
 <55CF68E4.9050807@msapiro.net> <1439706932.1160.4.camel@skjoldebrand.eu>
 <55D0A411.1000301@msapiro.net>
Message-ID: <20150816183113.Horde.JHbBLxNynTYsL2tzmeCds5Z@mail.skjoldebrand.eu>


Quoting Mark Sapiro <mark at msapiro.net>:

> On 08/15/2015 11:35 PM, Martin Skj?ldebrand wrote:
>> On Sat, 2015-08-15 at 09:29 -0700, Mark Sapiro wrote:
>>>
>>> Also, what's in data/virtual-mailman?
>>
>> I don't have one, maybe that's the problem =)
>
>
> Yes, that is the problem. You posted that you had in mm_cfg.py

>
>
> ...
>> Anyway; output of postconf -n is:
>
> It would have been more helpful if you had posted this in a way that
> didn't strip the line breaks.

Apologies, I thought it looked OK when I sent - maybe I didn't look  
close enough or I still need to do battle with my email client.  
Sending this from webmail instead.

>
> Make the above changes to main.cf, fix POSTFIX_STYLE_VIRTUAL_DOMAINS in
> mm_cfg.py and remove "POSTFIX_MAP_CMD =
> '/path/to/data/virtual_to_transport'" from mm_cfg.py.
>
> Then run Mailman's bin/genaliases and see if it now works.

Yes!
Now it works. Many thanks, I obviously need to go back and try to  
understand the docs better =(

/Martin S

From akostocker at gmail.com  Wed Aug 19 20:31:53 2015
From: akostocker at gmail.com (Tony Stocker)
Date: Wed, 19 Aug 2015 14:31:53 -0400
Subject: [Mailman-Users] Altering 'From' address for a list
Message-ID: <CACLi31V_-gAdvTOkuJBaQGcB-iBHQaP4bQn8MM=vOs_FXk6G6A@mail.gmail.com>

I've been directed by management to change/munge the 'From' address of a
mailing list that is used for announcements to a non-list address.  This is
in addition to providing the specific 'Reply-to' address via configuration
which has already been done.  In other words instead of seeing an email
with a header like so:

Date: Wed, 19 Aug 2015 13:00:46 -0400
From: Administrators <admin at mail.pos.net>
Reply-To: helpdesk at mail.pos.net
To: : , :
Subject: [admin] Test email


to something like this:

Date: Wed, 19 Aug 2015 13:00:46 -0400
From: helpdesk at mail.pos.net
Reply-To: helpdesk at mail.pos.net
To: : , :
Subject: [admin] Test email


I'm using Mailman version 2.1.12 with Postfix (version 2.6.6-6.el6_5
provided by RHEL/CentOS package) as the MTA.

I have tried setting up /etc/postfix/generic file to munge the address, but
no setting that I make has made any difference, things continue to show a
'From:' address as admin at mail.pos.net and inside the headers I see that the
'Sender:' line looks like this:
Sender: admin-bounces at mail.pos.net

I've tried the following entries in the /etc/postfix/generic file:

admin at mail.pos.net          helpdesk at mail.pos.net
admin-bounces at mail.pos.net          helpdesk at mail.pos.net
admin-owner at mail.pos.net          helpdesk at mail.pos.net
admin at localhost.pos.net          helpdesk at mail.pos.net
admin-bounces at localhost.pos.net          helpdesk at mail.pos.net
admin-owner at localhost.pos.net          helpdesk at mail.pos.net
admin          helpdesk at mail.pos.net

I *have* added 'smtp_generic_maps = hash:/etc/postfix/generic' to the
/etc/postfix/main.cf file and I *have* done 'postmap
hash:/etc/postfix/generic' after adding/changing entries and I *have* done
/etc/init.d/postfix restart after making the changes.

However, the 'From:' and 'Sender:' remain the same when I receive an email
from the list, no matter what entries are added to to the generic table.


This is all being done so that even if someone chooses to ignore the
'Reply-to' address if they reply to a posted message, it will go to this
help desk email address and not to the group.

Thanks in advance for your help.

Tony

From mark at msapiro.net  Thu Aug 20 01:27:42 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 19 Aug 2015 16:27:42 -0700
Subject: [Mailman-Users] Altering 'From' address for a list
In-Reply-To: <CACLi31V_-gAdvTOkuJBaQGcB-iBHQaP4bQn8MM=vOs_FXk6G6A@mail.gmail.com>
References: <CACLi31V_-gAdvTOkuJBaQGcB-iBHQaP4bQn8MM=vOs_FXk6G6A@mail.gmail.com>
Message-ID: <55D510EE.60600@msapiro.net>

On 08/19/2015 11:31 AM, Tony Stocker wrote:
> 
> I have tried setting up /etc/postfix/generic file to munge the address, but
> no setting that I make has made any difference, things continue to show a
> 'From:' address as admin at mail.pos.net and inside the headers I see that the
> 'Sender:' line looks like this:
> Sender: admin-bounces at mail.pos.net
> 
> I've tried the following entries in the /etc/postfix/generic file:
> 
> admin at mail.pos.net          helpdesk at mail.pos.net
> admin-bounces at mail.pos.net          helpdesk at mail.pos.net
> admin-owner at mail.pos.net          helpdesk at mail.pos.net
> admin at localhost.pos.net          helpdesk at mail.pos.net
> admin-bounces at localhost.pos.net          helpdesk at mail.pos.net
> admin-owner at localhost.pos.net          helpdesk at mail.pos.net
> admin          helpdesk at mail.pos.net


mailman-users at python.org is not the best resource for generic Postfix
questions. I suggest you join the postfix-users at postfix.org list and
post there. See <http://www.postfix.org/lists.html>.

That said, I think you can accomplish what you're trying to do with
sender_canonical_maps instead of smtp_generic_maps.

Also note, if you rewrite the -bounces address, you will break Mailman's
automated bounce processing and the helpdesk will have to deal with
bouncing addresses manually.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From billycrook at gmail.com  Thu Aug 20 08:43:27 2015
From: billycrook at gmail.com (Billy Crook)
Date: Thu, 20 Aug 2015 01:43:27 -0500
Subject: [Mailman-Users] Restricting access to the pipermail archives
Message-ID: <CANZ96zVQe7UX5z0-RG_-Jt=2k87kkQNhXRmaoJvcifMvu7GMLw@mail.gmail.com>

I want to give my users archives, but require authentication to access the
archives.  They should not be accessible to non-members.

How can I tie mailman's existing authentication (which can for example
conceal the member list from non-members), into pipermail?

From paul at tokyoprogressive.org  Thu Aug 20 13:37:10 2015
From: paul at tokyoprogressive.org (Paul Arenson/tokyoprogressive)
Date: Thu, 20 Aug 2015 20:37:10 +0900
Subject: [Mailman-Users] Why do my posts to my own mailman list disappear.
Message-ID: <E181B566-F7C7-40D6-97DA-175ECA7F1F99@tokyoprogressive.org>

I have had my own mailman list for two years. Recently, messages I sent to the list neither appeared (when sent from my regular email account), nor generated a message that the message was being held for moderation, when sent from another email account.

I have no clue. Well, just one really. Recently I have had so many spam messages, dealt with by banning or automatically rejecting, am am wondering if I somehow automatically deleted my own info. 

If so, is there any way to check and remedy?

Or could there be another reason?

I am on a Mac, for what it is worth.

Thanks,
paul

Sent from my iPad

From mark at msapiro.net  Thu Aug 20 18:08:43 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 20 Aug 2015 09:08:43 -0700
Subject: [Mailman-Users] Restricting access to the pipermail archives
In-Reply-To: <CANZ96zVQe7UX5z0-RG_-Jt=2k87kkQNhXRmaoJvcifMvu7GMLw@mail.gmail.com>
References: <CANZ96zVQe7UX5z0-RG_-Jt=2k87kkQNhXRmaoJvcifMvu7GMLw@mail.gmail.com>
Message-ID: <55D5FB8B.7090007@msapiro.net>

On 08/19/2015 11:43 PM, Billy Crook wrote:
> I want to give my users archives, but require authentication to access the
> archives.  They should not be accessible to non-members.
> 
> How can I tie mailman's existing authentication (which can for example
> conceal the member list from non-members), into pipermail?


On the web admin Archiving Options page for the list, set
archive_private to 'private'.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From adam-mailman at amyl.org.uk  Thu Aug 20 18:13:46 2015
From: adam-mailman at amyl.org.uk (Adam McGreggor)
Date: Thu, 20 Aug 2015 17:13:46 +0100
Subject: [Mailman-Users] Restricting access to the pipermail archives
In-Reply-To: <CANZ96zVQe7UX5z0-RG_-Jt=2k87kkQNhXRmaoJvcifMvu7GMLw@mail.gmail.com>
References: <CANZ96zVQe7UX5z0-RG_-Jt=2k87kkQNhXRmaoJvcifMvu7GMLw@mail.gmail.com>
Message-ID: <20150820161345.GD9573@hendricks.amyl.org.uk>

On Thu, Aug 20, 2015 at 01:43:27AM -0500, Billy Crook wrote:
> I want to give my users archives, but require authentication to access the
> archives.  They should not be accessible to non-members.
> 
> How can I tie mailman's existing authentication (which can for example
> conceal the member list from non-members), into pipermail?

How about something that steps through each list's list_members,
extracts their Mailman password, runs that (their Mailman password)
through htpasswd, appends to an htaccess file, which you'd then use
for authorization against a <LocationMatch> rule in Apache (or the
equivalent in nginx) -- LocationMatch, so you can use regexp for the
filename to correspond to the listname.

Cron it, and problem "solved"?

A


-- 
"I am invariably puzzled by the attitude of Oxbridge to its 
 ecclesiastical past. Hearing grace recited before dinner by
 people you know don?t believe a word of it seems to be 
 considered quite normal."
   -- David Colquhoun

From adam-mailman at amyl.org.uk  Thu Aug 20 18:15:05 2015
From: adam-mailman at amyl.org.uk (Adam McGreggor)
Date: Thu, 20 Aug 2015 17:15:05 +0100
Subject: [Mailman-Users] Restricting access to the pipermail archives
In-Reply-To: <55D5FB8B.7090007@msapiro.net>
References: <CANZ96zVQe7UX5z0-RG_-Jt=2k87kkQNhXRmaoJvcifMvu7GMLw@mail.gmail.com>
 <55D5FB8B.7090007@msapiro.net>
Message-ID: <20150820161505.GE9573@hendricks.amyl.org.uk>

On Thu, Aug 20, 2015 at 09:08:43AM -0700, Mark Sapiro wrote:
> On 08/19/2015 11:43 PM, Billy Crook wrote:
> > I want to give my users archives, but require authentication to access the
> > archives.  They should not be accessible to non-members.
> > 
> > How can I tie mailman's existing authentication (which can for example
> > conceal the member list from non-members), into pipermail?
> 
> 
> On the web admin Archiving Options page for the list, set
> archive_private to 'private'.

Which is (much) simpler than the way I suggested, thinking you were
after something for all list members, regardless of which list.


-- 
"If more of us valued food and cheer and song above hoarded gold, it
 would be a merrier world"
    -- J. R. R. Tolkien

From mark at msapiro.net  Thu Aug 20 18:43:28 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 20 Aug 2015 09:43:28 -0700
Subject: [Mailman-Users] Why do my posts to my own mailman list
	disappear.
In-Reply-To: <E181B566-F7C7-40D6-97DA-175ECA7F1F99@tokyoprogressive.org>
References: <E181B566-F7C7-40D6-97DA-175ECA7F1F99@tokyoprogressive.org>
Message-ID: <55D603B0.70604@msapiro.net>

On 08/20/2015 04:37 AM, Paul Arenson/tokyoprogressive wrote:
> I have had my own mailman list for two years. Recently, messages I sent to the list neither appeared (when sent from my regular email account), nor generated a message that the message was being held for moderation, when sent from another email account.


If the messages are reaching Mailman, they are being discarded.


> I have no clue. Well, just one really. Recently I have had so many spam messages, dealt with by banning or automatically rejecting, am am wondering if I somehow automatically deleted my own info. 
> 
> If so, is there any way to check and remedy?


Discards are logged in Mailman's vette log with the Message-ID of the
discarded message.

Discards can occur for these reasons:
1)Message From: a domain with a DMARC reject or quarantine policy and
dmarc_moderation_action = Discard.
2)Message matches a header_filter_rules rule with discard action.
3)Message from a moderated member and member_moderation_action = Discard.
4)Message from a non-member in discard_these_nonmembers or with
generic_nonmember_action = Discard.
5)Message empty after content filtering and filter_action = Discard.
6)If scrub_nondigest is Yes and mm_cfg.ARCHIVE_HTML_SANITIZER = 0 and
the message's Content-Type is text/html.

In your case, 2) seems most likely.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From billycrook at gmail.com  Thu Aug 20 20:26:56 2015
From: billycrook at gmail.com (Billy Crook)
Date: Thu, 20 Aug 2015 13:26:56 -0500
Subject: [Mailman-Users] Restricting access to the pipermail archives
In-Reply-To: <20150820161505.GE9573@hendricks.amyl.org.uk>
References: <CANZ96zVQe7UX5z0-RG_-Jt=2k87kkQNhXRmaoJvcifMvu7GMLw@mail.gmail.com>
 <55D5FB8B.7090007@msapiro.net> <20150820161505.GE9573@hendricks.amyl.org.uk>
Message-ID: <CANZ96zV4Bp8fVmiXCt+sYSeD-39ceC-5-87xGjWSr6368Jh+2w@mail.gmail.com>

It is not clear to me that archive_private controls access to the archive.

My understanding is that archive_private is used to inform mailman that the
email addresses visible in the archive should be presented differently
depending if the archive will be made publicly accessible or not.  i.e.
munge the email addresses if there's a chance spambots can find them.

I would like to NOT munge the addresses, so they are useful to click on for
my users.  But I want all of the text of each archived message to be kept
in secret on the server, viewable only by members of the list.

I only really have one list given the kind of organization this is for. (a
neighborhood of about 35 homeowners)  My reason for offering mailman to
them in the first place is so I don't have to see a big list of email
addresses every time we send or receive something, and so that the list can
be kept in a central place (on the HOA web server).

I want all members and no non-members to be able to see who else is on the
list (that part already works), and for all members and no non-members to
be able to browse the archives.

To that end Adam's original suggestion of iterating over list_members seems
plausible.  I would hope there was a more "supported" way to do this, like
having a setting in postfix that causes it to create and maintain htaccess
files in each list's private directory.  But I am up to the challenge of
writing it myself.  I already made minor sourcecode modification to my
install to tweak how the names in from addresses are calculated.

On Thu, Aug 20, 2015 at 11:15 AM, Adam McGreggor <adam-mailman at amyl.org.uk>
wrote:

> On Thu, Aug 20, 2015 at 09:08:43AM -0700, Mark Sapiro wrote:
> > On 08/19/2015 11:43 PM, Billy Crook wrote:
> > > I want to give my users archives, but require authentication to access
> the
> > > archives.  They should not be accessible to non-members.
> > >
> > > How can I tie mailman's existing authentication (which can for example
> > > conceal the member list from non-members), into pipermail?
> >
> >
> > On the web admin Archiving Options page for the list, set
> > archive_private to 'private'.
>
> Which is (much) simpler than the way I suggested, thinking you were
> after something for all list members, regardless of which list.
>
>
> --
> "If more of us valued food and cheer and song above hoarded gold, it
>  would be a merrier world"
>     -- J. R. R. Tolkien
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/billycrook%40gmail.com
>

From mark at msapiro.net  Thu Aug 20 20:42:12 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 20 Aug 2015 11:42:12 -0700
Subject: [Mailman-Users] Restricting access to the pipermail archives
In-Reply-To: <CANZ96zV4Bp8fVmiXCt+sYSeD-39ceC-5-87xGjWSr6368Jh+2w@mail.gmail.com>
References: <CANZ96zVQe7UX5z0-RG_-Jt=2k87kkQNhXRmaoJvcifMvu7GMLw@mail.gmail.com>
 <55D5FB8B.7090007@msapiro.net> <20150820161505.GE9573@hendricks.amyl.org.uk>
 <CANZ96zV4Bp8fVmiXCt+sYSeD-39ceC-5-87xGjWSr6368Jh+2w@mail.gmail.com>
Message-ID: <55D61F84.1070407@msapiro.net>

On 08/20/2015 11:26 AM, Billy Crook wrote:
> It is not clear to me that archive_private controls access to the archive.


It does. Try it.  If archive_private is 'public' there is a symlink from
archives/public/listname to archives/private/listname and a URL like
http://example.com/pipermail/listname will work without authentication.
If you set archive_private to 'private', that symlink is removed, the
pipermail URL won't work and the only access to the archive is via a URL
like http://example.com/mailman/private/listname which requires
authentication and which will be displayed the archive URL on the
listinfo page and in List-Archive: headers.


> My understanding is that archive_private is used to inform mailman that the
> email addresses visible in the archive should be presented differently
> depending if the archive will be made publicly accessible or not.  i.e.
> munge the email addresses if there's a chance spambots can find them.


That is the Defaults.py/mm_cfg.py setting ARCHIVER_OBSCURES_EMAIL_ADDRESS.


> I would like to NOT munge the addresses, so they are useful to click on for
> my users.  But I want all of the text of each archived message to be kept
> in secret on the server, viewable only by members of the list.


Set

ARCHIVER_OBSCURES_EMAIL_ADDRESS = No

in mm_cfg.py and set the list's archive_private to 'private.


> I would hope there was a more "supported" way to do this,...


Just do the above. That will do it.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From paul at tokyoprogressive.org  Fri Aug 21 03:24:22 2015
From: paul at tokyoprogressive.org (Paul Arenson/tokyoprogressive)
Date: Fri, 21 Aug 2015 10:24:22 +0900
Subject: [Mailman-Users] Why do my posts to own mailman list disappear.
In-Reply-To: <mailman.528.1440095263.28099.mailman-users@python.org>
References: <mailman.528.1440095263.28099.mailman-users@python.org>
Message-ID: <3E918CBF-C737-42E7-9285-D5C97E4E4275@tokyoprogressive.org>

Thank you Mark.  About Vette list, I don't know if I have shell access.  I have CPanel, and that is all I have used.  Any way to check?  

I will, in the meantime, try another subject, though somehow I don't think that is it, as I havenever set anything to discard. Anyway to check what my settings are within mailman, such as if I marked my name for automatic deletion by mistake?

paul

From mark at msapiro.net  Fri Aug 21 03:57:37 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 20 Aug 2015 18:57:37 -0700
Subject: [Mailman-Users] Why do my posts to own mailman list disappear.
In-Reply-To: <3E918CBF-C737-42E7-9285-D5C97E4E4275@tokyoprogressive.org>
References: <mailman.528.1440095263.28099.mailman-users@python.org>
 <3E918CBF-C737-42E7-9285-D5C97E4E4275@tokyoprogressive.org>
Message-ID: <55D68591.6020003@msapiro.net>

On 08/20/2015 06:24 PM, Paul Arenson/tokyoprogressive wrote:
> Thank you Mark.  About Vette list, I don't know if I have shell access.  I have CPanel, and that is all I have used.  Any way to check?  


On cPanel, if you can access the file system, you'll find Mailman's logs
in /usr/local/cpanel/3rdparty/mailman/logs/.

See <http://wiki.list.org/DOC/Mailman%20and%20CPanel>.

Look at all the following settings in the list admin UI for Discard actions

Privacy options... -> Sender filters -> member_moderation_action
Privacy options... -> Sender filters -> dmarc_moderation_action
Privacy options... -> Sender filters -> discard_these_nonmembers
Privacy options... -> Sender filters -> generic_nonmember_action
Privacy options... -> Spam filters -> header_filter_rules
Content filtering -> filter_action

If there are no Discard actions in those, look at

Non-digest options -> scrub_nondigest

If this is Yes, it is possible that your posts are HTML only and being
discarded by the scrubber.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From dunkel at calamus.net  Fri Aug 21 10:28:24 2015
From: dunkel at calamus.net (Ulf Dunkel)
Date: Fri, 21 Aug 2015 10:28:24 +0200
Subject: [Mailman-Users] How to discard messages from just a few list
	members?
Message-ID: <55D6E128.3020307@calamus.net>

I am running some moderated mailing lists (for newsletters) where only I
myself should be able to send messages. Every now and then some of my
list members get trapped in a spam bulk mail and repeat this spam even
to my lists.

In "Privacy options > Sender filters", I see many options to treat
postings from non-member addresses. But I don't find a way to discard
posting from defined list members automatically, without dropping them
from the list.

Is there any such option that I have overseen?

I am currently running Mailman 2.1.16.

---Ulf Dunkel

From Richard at Damon-Family.org  Fri Aug 21 14:38:04 2015
From: Richard at Damon-Family.org (Richard Damon)
Date: Fri, 21 Aug 2015 08:38:04 -0400
Subject: [Mailman-Users] How to discard messages from just a few list
 members?
In-Reply-To: <55D6E128.3020307@calamus.net>
References: <55D6E128.3020307@calamus.net>
Message-ID: <55D71BAC.7040701@Damon-Family.org>

On 8/21/15 4:28 AM, Ulf Dunkel wrote:
> I am running some moderated mailing lists (for newsletters) where only I
> myself should be able to send messages. Every now and then some of my
> list members get trapped in a spam bulk mail and repeat this spam even
> to my lists.
>
> In "Privacy options > Sender filters", I see many options to treat
> postings from non-member addresses. But I don't find a way to discard
> posting from defined list members automatically, without dropping them
> from the list.
>
> Is there any such option that I have overseen?
>
> I am currently running Mailman 2.1.16.
>
> ---Ulf Dunkel
>
Just setup a filter ( Privacy  > SPAM Filters ) to detect messages From: 
them and set the disposition to discard.

-- 
Richard Damon


From wetz.peter at gmail.com  Fri Aug 21 11:29:07 2015
From: wetz.peter at gmail.com (Peter Wetz)
Date: Fri, 21 Aug 2015 11:29:07 +0200
Subject: [Mailman-Users] Archive page not showing dates properly
Message-ID: <CAGmyAkAB7h3YSVVm4LSpHt-7mQ8tFQD5N-Ok=3R8BQuJfVSNEQ@mail.gmail.com>

On the archive page of my mailman list the dates are not showing up
properly. It looks like this: http://i.imgur.com/IrmWnie.png

As you can see, instead of the date a string "%(date)s" is shown. It is
also worth noting, that it already worked properly. However, suddenly (at
least I cannot state when exactly) it stopped working and since then dates
are shown as seen in the screenshot.

The archives are generated with Pipermail 0.09 (Mailman edition).

How can I fix this problem?

Thanks and best,
Peter

From legalize at xmission.com  Thu Aug 20 19:13:45 2015
From: legalize at xmission.com (Richard)
Date: Thu, 20 Aug 2015 11:13:45 -0600
Subject: [Mailman-Users] Mailman's solution to DMARC makes List-Id useless
Message-ID: <E1ZSTPN-0005qu-VB@shell.xmission.com>

Hi,

I'm on a mailing list that recently switched the mailman DMARC setting
to "Munge from".  IMO, the munging of the From: line is fine as far as
I'm concerned and I see how that fixes the DMARC problem.

However, what is really annoying is that it takes the original From:
line and puts it on the Reply-To: line and there's no way to turn this
off.

I can't seem to find any explanation of why anyone thinks this is a
good idea.  Maybe someone here can explain it to me.

Below is why I think it's a bad idea.  Why can't we encode the original
email address in a comment or quoted token on the From: line instead of
jamming it onto Reply-To?

This is how I'm seeing mailing list messages now:

      To: Hal Finkel <hfinkel at anl.gov>
      cc: cfe-dev at lists.llvm.org,
          Commit Messages and Patches for LLVM <llvm-commits at lists.llvm.org>
    From: Lang Hames via cfe-dev <cfe-dev at lists.llvm.org>
Reply-To: Lang Hames <lhames at gmail.com>

The reply-to is going to the sender instead of the list, which is
making people cc the mailing list in order to get things to go back to
the list.  From what I read of DMARC, it's the munging of the From:
line that is needed in order to have messages pass the DMARC checks.
To me this makes sense -- the mailing list domain is sending the message
to the list and the appropriate domain checks need to be made against
the mailing list's domain, not the original author of the mailing list
message.

What I'm not understanding is how DMARC is mandating that Reply-To: go
back to the original author, and not the mailing list, as is the
usualy convention: public conversations from a mailing list cycle back
to the mailing list by default and only fork into a private
conversation when specifically requested.

What's happening now is that people are doing "reply to all" in order
to get the mailing list included, which makes *me* the recipient of
their reply and the mailing list cc'ed.  Then the mailing list
software notices that the message was sent *to* an address already on
the mailing list, so it doesn't send me a second copy of the message.

This means that every time someone replies to my messages on the
mailing list, and all subsequent replies in the thread because
everyone else will do reply-to-all as well, I'll be getting all these
private messages that are actually copies of the public messages but
I won't be getting the public messages.

This makes the entire List-Id field useless because none of these
replies to threads in which I participate will come back to me through
the mailing list, but instead as private copies of public messages
sent to the mailing list.  As a result, it basically screws up all
mailing list filtering -- which was the whole point of the list-id
field.
-- 
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
     The Computer Graphics Museum <http://ComputerGraphicsMuseum.org>
         The Terminals Wiki <http://terminals.classiccmp.org>
  Legalize Adulthood! (my blog) <http://LegalizeAdulthood.wordpress.com>

From mark at msapiro.net  Fri Aug 21 16:09:24 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 21 Aug 2015 07:09:24 -0700
Subject: [Mailman-Users] How to discard messages from just a few list
 members?
In-Reply-To: <55D6E128.3020307@calamus.net>
References: <55D6E128.3020307@calamus.net>
Message-ID: <55D73114.9020301@msapiro.net>

On 08/21/2015 01:28 AM, Ulf Dunkel wrote:
> 
> In "Privacy options > Sender filters", I see many options to treat
> postings from non-member addresses. But I don't find a way to discard
> posting from defined list members automatically, without dropping them
> from the list.
> 
> Is there any such option that I have overseen?


Set the members moderated and set member_moderation_action to Discard.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Fri Aug 21 16:37:52 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 21 Aug 2015 07:37:52 -0700
Subject: [Mailman-Users] Mailman's solution to DMARC makes List-Id
	useless
In-Reply-To: <E1ZSTPN-0005qu-VB@shell.xmission.com>
References: <E1ZSTPN-0005qu-VB@shell.xmission.com>
Message-ID: <55D737C0.6080701@msapiro.net>

On 08/20/2015 10:13 AM, Richard wrote:
> 
> However, what is really annoying is that it takes the original From:
> line and puts it on the Reply-To: line and there's no way to turn this
> off.
> 
> I can't seem to find any explanation of why anyone thinks this is a
> good idea.  Maybe someone here can explain it to me.


It is there to make a "reply" and "reply-all" actions on posts with
munged From: be as consistent as possible with the same action on a
non-munged post, and to expose the poster's address in a header which is
normally displayed by MUAs.


> Below is why I think it's a bad idea.  Why can't we encode the original
> email address in a comment or quoted token on the From: line instead of
> jamming it onto Reply-To?
> 
> This is how I'm seeing mailing list messages now:
> 
>       To: Hal Finkel <hfinkel at anl.gov>
>       cc: cfe-dev at lists.llvm.org,
>           Commit Messages and Patches for LLVM <llvm-commits at lists.llvm.org>
>     From: Lang Hames via cfe-dev <cfe-dev at lists.llvm.org>
> Reply-To: Lang Hames <lhames at gmail.com>
> 
> The reply-to is going to the sender instead of the list, which is
> making people cc the mailing list in order to get things to go back to
> the list.


And without munging, the same post would be

>       To: Hal Finkel <hfinkel at anl.gov>
>       cc: cfe-dev at lists.llvm.org,
>           Commit Messages and Patches for LLVM
<llvm-commits at lists.llvm.org>
>     From: Lang Hames <lhames at gmail.com>

and reply would still go to the sender in From:


> From what I read of DMARC, it's the munging of the From:
> line that is needed in order to have messages pass the DMARC checks.
> To me this makes sense -- the mailing list domain is sending the message
> to the list and the appropriate domain checks need to be made against
> the mailing list's domain, not the original author of the mailing list
> message.


And if you used dmarc_moderation_action instead of from_is_list to munge
the from, only posts From: domains which publish DMARC reject (or
optionally, quarantine) policies would be munged. The policy for
gmail.com in 'none'.


> What I'm not understanding is how DMARC is mandating that Reply-To: go
> back to the original author, and not the mailing list, as is the
> usualy convention: public conversations from a mailing list cycle back
> to the mailing list by default and only fork into a private
> conversation when specifically requested.


If you want this behavior, set the list's reply_goes_to_list to "This
list", then with current Mailman, the above becomes

>       To: Hal Finkel <hfinkel at anl.gov>
>       cc: cfe-dev at lists.llvm.org,
>           Commit Messages and Patches for LLVM
<llvm-commits at lists.llvm.org>,
>           Lang Hames <lhames at gmail.com>
>     From: Lang Hames via cfe-dev <cfe-dev at lists.llvm.org>
> Reply-To: <cfe-dev at lists.llvm.org>

Whether or not to munge Reply-To: to the list address is controversial
and has been argued and flamed multiple times for years. See
<http://marc.merlins.org/netrants/listreplyto.html>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Fri Aug 21 16:59:21 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 21 Aug 2015 07:59:21 -0700
Subject: [Mailman-Users] Archive page not showing dates properly
In-Reply-To: <CAGmyAkAB7h3YSVVm4LSpHt-7mQ8tFQD5N-Ok=3R8BQuJfVSNEQ@mail.gmail.com>
References: <CAGmyAkAB7h3YSVVm4LSpHt-7mQ8tFQD5N-Ok=3R8BQuJfVSNEQ@mail.gmail.com>
Message-ID: <55D73CC9.1050303@msapiro.net>

On 08/21/2015 02:29 AM, Peter Wetz wrote:
> On the archive page of my mailman list the dates are not showing up
> properly. It looks like this: http://i.imgur.com/IrmWnie.png
> 
> As you can see, instead of the date a string "%(date)s" is shown. It is
> also worth noting, that it already worked properly. However, suddenly (at
> least I cannot state when exactly) it stopped working and since then dates
> are shown as seen in the screenshot.


The standard Mailman archidxentry.html template does not have a %(date)s
substitution in it. Presumably, your installation/list has a custom
version of this template (<http://wiki.list.org/x/4030605>) which adds
the %(date)s substitution and along with this, modifications to
Mailman/Archiver/HyperArch.py to add a 'date' entry to the interpolation
dictionary for this template, and also presumably, some Mailman upgrade
removed the modification to HyperArch.py.


> How can I fix this problem?


Replace the current Mailman/Archiver/HyperArch.py with the one that was
there when it worked, or in the definition of write_index_entry at about
line 1073 where you see

        d = {
            'filename': urllib.quote(article.filename),
            'subject':  subject,
            'sequence': article.sequence,
            'author':   author
        }

change it to

        d = {
            'filename': urllib.quote(article.filename),
            'subject':  subject,
            'sequence': article.sequence,
            'date':     article.datestr,
            'author':   author
        }

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From stephen at xemacs.org  Fri Aug 21 20:26:01 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 22 Aug 2015 03:26:01 +0900
Subject: [Mailman-Users] Mailman's solution to DMARC makes List-Id useless
In-Reply-To: <E1ZSTPN-0005qu-VB@shell.xmission.com>
References: <E1ZSTPN-0005qu-VB@shell.xmission.com>
Message-ID: <877foo8pmu.fsf@uwakimon.sk.tsukuba.ac.jp>

Richard writes:

 > Below is why I think it's a bad idea.  Why can't we encode the
 > original email address in a comment or quoted token on the From:
 > line instead of jamming it onto Reply-To?

Because that makes it very inconvenient to reply to author.  On some
lists, that's a crucial feature.  For example, blind people require
excessive effort for cutting and pasting, but typically have access to
features that make selectively including addresses already present in
the headers very simple.  So it *must* be at least an option to
include the author as a real address in From or Reply-To, and I
believe it should be the default (see below).

 > the list.  From what I read of DMARC, it's the munging of the From:
 > line that is needed in order to have messages pass the DMARC
 > checks.

Actually, what happens is that the message *fails* the DMARC checks in
such a way that DMARC specifies the failure should be ignored, and the
message handled as though DMARC didn't even exist.

 > To me this makes sense -- the mailing list domain is sending the
 > message to the list and the appropriate domain checks need to be
 > made against the mailing list's domain, not the original author of
 > the mailing list message.

That's not a tenable interpretation.  There is a header for the
purpose of identifying the sender in your sense, its name is "Sender",
and the designers of DMARC deliberately rejected its use for this
purpose (for good reason).

As a pragmatic matter, mailing list domains rarely even have a _dmarc
DNS record, so such checks cannot be made effectively.

In any case, "From" is *defined* by RFC 5322 to be the *content
author*'s address for various purposes (such as identification and
reply).  It is also the preferred address for automatic reply to
author unless Reply-To is set.  The designers of Internet mail made
these decisions based on hard thought and long practical experience
because they make it possible for an MUA to handle both the common
cases and the edge cases smoothly.

The email RFCs do not envision *anybody but the original sender*
setting the From header, so one can't say anything with authority, but
my take is that if you insist on breaking the From header, you should
put the author in Reply-To so that receiving MUAs can find her address
and automatically reply to it.

 > What I'm not understanding is how DMARC is mandating that Reply-To:
 > go back to the original author,

It doesn't.  It's the basic Internet message standard (currently RFC
5322) that governs From and Reply-To.  As Mark explained, Mailman's
current behavior when From-munging is a delicate balancing act to
preserve as much of the "normal" operation of MUAs as possible without
triggering DMARC rejects.

DMARC p=reject gives list admins an unpleasant choice: (1) violate the
mail standards and suffer various degradations of service because
others in the mail system assume conformance (eg, your "wrong
duplicate" problem), (2) tell your p=reject users that their posts are
going to be rejected or discarded by many subscribers, or (3) stop
decorating posts with [List] tags or material prefixed and affixed to
the message body (so that the originator's DKIM signature will remain
valid and the DMARC checks will pass).

N.B. The tech staff from Yahoo! and AOL have acknowledged (on the
ietf-dmarc mailing list) that their employers are knowingly breaking
mailing lists (and other services) to address their security fiascos.
The designers of DMARC have always maintained that the Yahoo!/AOL use
case is abusive -- DMARC was designed to protect official mail to
customers sent on behalf of corporations by their employees, not the
general use mail of users with addresses at freemail providers.  In
other words, mailing lists just shouldn't receive mail from p=reject
domains, ever.  No problem -- until Yahoo! and AOL decided to *create*
one.

IMO, given those facts, posting from a Yahoo! or AOL address is just
plain rude.  (I can and do get away with banning their posts.  I wish
everybody could do that.)

 > and not the mailing list, as is the usualy convention: public
 > conversations from a mailing list cycle back to the mailing list by
 > default and only fork into a private conversation when specifically
 > requested.

The "usual convention" (of munging Reply-To) violates the mail RFCs
and breaks interoperability, and should be entirely unnecessary now
that we've had List-Post for more than a decade.  Consider an MUA
whose default reply function looks in Reply-To first, then in
List-Post, then in From.  Why doesn't everybody's do that?  It's
trivial to implement. :-(

 > What's happening now is that people are doing "reply to all" in
 > order to get the mailing list included, which makes *me* the
 > recipient of their reply and the mailing list cc'ed.

As Mark points out, the first thing to do is to make sure you set
dmarc_moderation_action, not from_is_list.  Then only Yahoo! and AOL
posters are likely to cause pain.  Hopefully they are few....


From paul at tokyoprogressive.org  Sat Aug 22 07:26:22 2015
From: paul at tokyoprogressive.org (Paul Arenson/tokyoprogressive)
Date: Sat, 22 Aug 2015 14:26:22 +0900
Subject: [Mailman-Users] Why do my posts to own mailman list disappear.
In-Reply-To: <3E918CBF-C737-42E7-9285-D5C97E4E4275@tokyoprogressive.org>
References: <mailman.528.1440095263.28099.mailman-users@python.org>
 <3E918CBF-C737-42E7-9285-D5C97E4E4275@tokyoprogressive.org>
Message-ID: <138DBE3F-0BB0-4474-AD34-0366C0582176@tokyoprogressive.org>

Thank you, Mark.

I looked in Cpanel and could see no directory starting with /usr/.  I have a reseller account and nangoku-jiyu-jin.net <http://nangoku-jiyu-jin.net/> is a subset of turnlefthostimg.org <http://turnlefthostimg.org/>, but for now I am looking in the cpanel for nangoku-jiyu-jin.net <http://nangoku-jiyu-jin.net/>. Again, no such directory in that CPanel.  I tried looking in root, etc. but do not see such a name as /usr/.

Then I figured out my FTP login and got in using the Transmit program.  I see a /usr/ folder at the end of a very long list of folders and files:
.bash.logout 
?
---
--
cpanel folder



public_ftp
public_html
ssl
tmp
usr
www

I click on usr and get Could not change directory to /usr
Server said can?t change directory to /usr: No such directory.
Error -125: remote chdir failed.

There is a cpanel  folder but that doesn?t seem to help. 

Am I looking in the wrong place?

Thanks, Paul









From paul at tokyoprogressive.org  Sat Aug 22 09:01:02 2015
From: paul at tokyoprogressive.org (Paul Arenson/tokyoprogressive)
Date: Sat, 22 Aug 2015 16:01:02 +0900
Subject: [Mailman-Users] Fwd: Why do my posts to own mailman list disappear.
References: <138DBE3F-0BB0-4474-AD34-0366C0582176@tokyoprogressive.org>
Message-ID: <D25E945B-3465-4C3A-8221-965F632BBD4D@tokyoprogressive.org>

Further to the above, I have tested and found NOTHING goes through?I can?t even get a request to subscribe through. It is effectively dead!!!

What to do?

Paul Arenson/tokyoprogressive
paul at tokyoprogressive.org



> Begin forwarded message:
> 
> From: Paul Arenson/tokyoprogressive <paul at tokyoprogressive.org>
> Subject: Re: Why do my posts to own mailman list disappear.
> Date: August 22, 2015 at 2:26:22 PM GMT+9
> To: "mailman-users at python.org" <mailman-users at python.org>
> 
> Thank you, Mark.
> 
> I looked in Cpanel and could see no directory starting with /usr/.  I have a reseller account and nangoku-jiyu-jin.net <http://nangoku-jiyu-jin.net/> is a subset of turnlefthostimg.org <http://turnlefthostimg.org/>, but for now I am looking in the cpanel for nangoku-jiyu-jin.net <http://nangoku-jiyu-jin.net/>. Again, no such directory in that CPanel.  I tried looking in root, etc. but do not see such a name as /usr/.
> 
> Then I figured out my FTP login and got in using the Transmit program.  I see a /usr/ folder at the end of a very long list of folders and files:
> .bash.logout 
> ?
> ---
> --
> cpanel folder
> 
> 
> 
> public_ftp
> public_html
> ssl
> tmp
> usr
> www
> 
> I click on usr and get Could not change directory to /usr
> Server said can?t change directory to /usr: No such directory.
> Error -125: remote chdir failed.
> 
> There is a cpanel  folder but that doesn?t seem to help. 
> 
> Am I looking in the wrong place?
> 
> Thanks, Paul
> 
> 
> 
> 
> 
> 
> 
> 


From stephen at xemacs.org  Sat Aug 22 13:22:30 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 22 Aug 2015 20:22:30 +0900
Subject: [Mailman-Users] Why do my posts to own mailman list disappear.
In-Reply-To: <138DBE3F-0BB0-4474-AD34-0366C0582176@tokyoprogressive.org>
References: <mailman.528.1440095263.28099.mailman-users@python.org>
 <3E918CBF-C737-42E7-9285-D5C97E4E4275@tokyoprogressive.org>
 <138DBE3F-0BB0-4474-AD34-0366C0582176@tokyoprogressive.org>
Message-ID: <8761478t55.fsf@uwakimon.sk.tsukuba.ac.jp>

Paul Arenson/tokyoprogressive writes:

 > I looked in Cpanel and could see no directory starting with /usr/.

cPanel won't give you access to that, I'm pretty sure.  You need to
have shell access to the host or virtual machine.  If you don't know
if you have shell access, you probably don't.  Large hosting services
tend to be quite stingy about handing out shell access.

 > I have a reseller account

What does that mean?  Aha, I see.  *You* are turnlefthosting.org, but
the PTR says your IP is assigned to 100tb.com, so you're the reseller
for somebody who actually runs the hardware.

A little bit of browsing the related DNS shows quite a bit of a mess.
None of the published domains properly roundtrip A -> PTR -> A, they
just lead into a maze of little twisty passages all alike.  That
doesn't necessarily cause a problem by itself, but it often makes
problems harder to diagnose.

 > and nangoku-jiyu-jin.net <http://nangoku-jiyu-jin.net/> is a subset
 > of turnlefthostimg.org <http://turnlefthostimg.org/>,

I think you're going to have to ask your upstream for help on this.

That said there are a few long shots you could try to diagnose it
yourself.  One is a manual transaction with the mailserver and see if
it's being rejected at the SMTP level.

Another thing I noticed is that you typoed "turnlefthosting.org" as
"turnlefthostimg.org" in the above.  Is it possible that you typoed a
host or list name somewhere in a recent configuration ichange?

Do you have a personal mailbox on nangoku-jiyu-jin.net?  If so, does
that work?

I suppose you are the owner of your lists.  Have you tried writing to
<listname>-owner at nangoku-jiyu-jin.net?  It should forward to you.

According to the page at
http://www.nangoku-jiyu-jin.net/mailman/listinfo/mailman, the mailman@
list is hosted by dionysus.thewebhostserver.com, and the owner is
root at dionysus.thewebhostserver.com.  This is probably typical of
cPanel (which has its own way of handling virtual hosting), but you
should check your own listinfo and admin pages to be sure the virtual
hosting is working correctly and everything points to
nangoku-jiyu-jin.net.  (You may have trouble finding the proper
Mailman interface, though, because cPanel has its own idiosyncratic
management interface, and may even make the original inaccessible.)

Are your lists archived?

 > Then I figured out my FTP login and got in using the Transmit
 > program.

I don't think this will help.  Most FTP server software is hard to
secure, and standard practice of hosting systems is to chroot them
(that means that only a part of the filesystem is visible to the FTP
server, and in fact to any programs run by the FTP server).  chroots
are almost impossible to break unless you have shell access (which is
one reason why hosting services are kechi about handing out shell).

 > I click on usr and get Could not change directory to /usr
 > Server said can?t change directory to /usr: No such directory.
 > Error -125: remote chdir failed.

That's bizarre.  I wonder if your host knows what it's doing.

 > There is a cpanel folder but that doesn?t seem to help.

I'm not surprised.

Good luck,

Steve

From dunkel at calamus.net  Sat Aug 22 14:19:13 2015
From: dunkel at calamus.net (Ulf Dunkel)
Date: Sat, 22 Aug 2015 14:19:13 +0200
Subject: [Mailman-Users] How to discard messages from just a few list
 members?
In-Reply-To: <55D73114.9020301@msapiro.net>
References: <55D6E128.3020307@calamus.net> <55D73114.9020301@msapiro.net>
Message-ID: <55D868C1.8080606@calamus.net>

>> In "Privacy options > Sender filters", I see many options to treat
>> postings from non-member addresses. But I don't find a way to discard
>> posting from defined list members automatically, without dropping them
>> from the list.
>>
>> Is there any such option that I have overseen?
> 
> 
> Set the members moderated and set member_moderation_action to Discard.

I guess if I would do so, this could mean that I myself won't be able to
send postings to the list, too, right?

---Ulf Dunkel

From Richard at Damon-Family.org  Sat Aug 22 15:03:47 2015
From: Richard at Damon-Family.org (Richard Damon)
Date: Sat, 22 Aug 2015 09:03:47 -0400
Subject: [Mailman-Users] How to discard messages from just a few list
 members?
In-Reply-To: <55D868C1.8080606@calamus.net>
References: <55D6E128.3020307@calamus.net> <55D73114.9020301@msapiro.net>
 <55D868C1.8080606@calamus.net>
Message-ID: <55D87333.3060501@Damon-Family.org>

On 8/22/15 8:19 AM, Ulf Dunkel wrote:
>>> In "Privacy options > Sender filters", I see many options to treat
>>> postings from non-member addresses. But I don't find a way to discard
>>> posting from defined list members automatically, without dropping them
>>> from the list.
>>>
>>> Is there any such option that I have overseen?
>>
>> Set the members moderated and set member_moderation_action to Discard.
> I guess if I would do so, this could mean that I myself won't be able to
> send postings to the list, too, right?
>
> ---Ulf Dunkel
You would be able to post if you make yourself not moderated.

-- 
Richard Damon


From mark at msapiro.net  Sat Aug 22 17:13:49 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 22 Aug 2015 08:13:49 -0700
Subject: [Mailman-Users] Fwd: Why do my posts to own mailman list
	disappear.
In-Reply-To: <D25E945B-3465-4C3A-8221-965F632BBD4D@tokyoprogressive.org>
References: <138DBE3F-0BB0-4474-AD34-0366C0582176@tokyoprogressive.org>
 <D25E945B-3465-4C3A-8221-965F632BBD4D@tokyoprogressive.org>
Message-ID: <55D891AD.30206@msapiro.net>

On 08/22/2015 12:01 AM, Paul Arenson/tokyoprogressive wrote:
> Further to the above, I have tested and found NOTHING goes through?I can?t even get a request to subscribe through. It is effectively dead!!!
> 
> What to do?


You can see the FAQ at <http://wiki.list.org/x/4030723>, but given your
lack of access to the underlying Mailman installation, that may not help.

As discussed in the FAQ at <http://wiki.list.org/x/4030641> (aka
<http://wiki.list.org/DOC/Mailman%20and%20CPanel>), we probably can't
help either.

As Steve suggests, you need to take this up with the people who support
the physical machine that hosts your Mailman installation.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From dunkel at calamus.net  Sat Aug 22 17:26:50 2015
From: dunkel at calamus.net (Ulf Dunkel)
Date: Sat, 22 Aug 2015 17:26:50 +0200
Subject: [Mailman-Users] How to discard messages from just a few list
 members?
In-Reply-To: <55D87333.3060501@Damon-Family.org>
References: <55D6E128.3020307@calamus.net> <55D73114.9020301@msapiro.net>
 <55D868C1.8080606@calamus.net> <55D87333.3060501@Damon-Family.org>
Message-ID: <55D894BA.4020607@calamus.net>

>>>> In "Privacy options > Sender filters", I see many options to treat
>>>> postings from non-member addresses. But I don't find a way to discard
>>>> posting from defined list members automatically, without dropping them
>>>> from the list.
>>>>
>>>> Is there any such option that I have overseen?
>>>
>>> Set the members moderated and set member_moderation_action to Discard.
>> I guess if I would do so, this could mean that I myself won't be able to
>> send postings to the list, too, right?
>>
> You would be able to post if you make yourself not moderated.

Thank you, but I won't send unmoderated news to any newsletter. I am
using Mailman for felt decades and I am so happy that the moderated flag
always kept me from sending postings too early. Chances are that you
post some stuff with unfixed errors and the moderated mode helps you to
re-check your posting before you confirm that it should be spread.

I have now set up a regEx command and will see if it works.

---Ulf Dunkel

From mark at msapiro.net  Sat Aug 22 17:28:02 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 22 Aug 2015 08:28:02 -0700
Subject: [Mailman-Users] How to discard messages from just a few list
 members?
In-Reply-To: <55D87333.3060501@Damon-Family.org>
References: <55D6E128.3020307@calamus.net> <55D73114.9020301@msapiro.net>
 <55D868C1.8080606@calamus.net> <55D87333.3060501@Damon-Family.org>
Message-ID: <55D89502.6000708@msapiro.net>

On 08/22/2015 06:03 AM, Richard Damon wrote:
> On 8/22/15 8:19 AM, Ulf Dunkel wrote:
>>>
>>> Set the members moderated and set member_moderation_action to Discard.
>> I guess if I would do so, this could mean that I myself won't be able to
>> send postings to the list, too, right?
>>
>> ---Ulf Dunkel
> You would be able to post if you make yourself not moderated.


To be more specific, set Privacy options... -> Sender filters ->
default_member_moderation to Yes to automatically moderate new members
and set Privacy options... -> Sender filters -> member_moderation_action
to Discard and Submit Your Changes. Then on the Membership Management...
-> Membership List page under Additional Member Tasks, set "Set
everyone's moderation bit, including those members not currently
visible" to On and click "Set".

Then, in order to be able to post yourself, uncheck your "mod" box on
the Membership Management... -> Membership List pages and Submit Your
Changes, or better still, leave yourself moderated and post with an
Approved: <password> header. This latter method prevents others from
posting by spoofing your address. See the FAQs at
<http://wiki.list.org/x/4030685> and <http://wiki.list.org/x/4030556>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From stephen at xemacs.org  Sun Aug 23 02:15:12 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 23 Aug 2015 09:15:12 +0900
Subject: [Mailman-Users] How to discard messages from just a few list
 members?
In-Reply-To: <55D894BA.4020607@calamus.net>
References: <55D6E128.3020307@calamus.net> <55D73114.9020301@msapiro.net>
 <55D868C1.8080606@calamus.net> <55D87333.3060501@Damon-Family.org>
 <55D894BA.4020607@calamus.net>
Message-ID: <87zj1i7tdb.fsf@uwakimon.sk.tsukuba.ac.jp>

Ulf Dunkel writes:

 > Thank you, but I won't send unmoderated news to any newsletter. I am
 > using Mailman for felt decades and I am so happy that the moderated flag
 > always kept me from sending postings too early.

Unmoderate yourself in the member view, and then use a privacy filter
with your address, and action set to Hold.

The same strategy should work for any number of authorized posters by
adding their addresses to the regexp with "|".

Note: Untested, but AFAIK the several moderation methods are
independent in this way.

Steve

From paul at tokyoprogressive.org  Sun Aug 23 08:12:43 2015
From: paul at tokyoprogressive.org (Paul Arenson/tokyoprogressive)
Date: Sun, 23 Aug 2015 15:12:43 +0900
Subject: [Mailman-Users] Why do my posts to own mailman list disappear.
In-Reply-To: <8761478t55.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <mailman.528.1440095263.28099.mailman-users@python.org>
 <3E918CBF-C737-42E7-9285-D5C97E4E4275@tokyoprogressive.org>
 <138DBE3F-0BB0-4474-AD34-0366C0582176@tokyoprogressive.org>
 <8761478t55.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <F31F58EC-6058-4A4F-A8F6-8550AEE7E98E@tokyoprogressive.org>

Thank you Stephen?..


> On Aug 22, 2015, at 8:22 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> 
> Paul Arenson/tokyoprogressive writes:
> 
>> I looked in Cpanel and could see no directory starting with /usr/.

Stephen:

> 
> cPanel won't give you access to that, I'm pretty sure.  You need to
> have shell access to the host or virtual machine.  If you don't know
> if you have shell access, you probably don't.  Large hosting services
> tend to be quite stingy about handing out shell access.

I guess maybe I don?t.  Just normal FTP. 



turnlefthosting.org <http://turnlefthosting.org/> was my idea to run a simple hosting service. With reseller hosting, in my case I don?t sell anything, but donate webspace
to NPOs. Because it is a resller account with what I think it is called WHM, I create space for their domains. They get their own CPanel.

So turnlefthosting.org <http://turnlefthosting.org/> also hosts nangoku-jiyu-jin.net <http://nangoku-jiyu-jin.net/>, gale-sig.org <http://gale-sig.org/>, welljapan.org <http://welljapan.org/>, tokyoprogressive.org.uk <http://tokyoprogressive.org.uk/>.

In fact, I realize now I don?t need all this complexity. I could do the same thing with a regular account and add-on domains.

But as I have it, I can create lists for the different domains using their version of CPanel and Softaculous.




> 
>> I have a reseller account
> 
> What does that mean?  Aha, I see.  *You* are turnlefthosting.org, but
> the PTR says your IP is assigned to 100tb.com, so you're the reseller
> for somebody who actually runs the hardware.

Actually I am not sure what PTR is?..but my reseller hosting is just that, I pay for space from EZPZ and
then can do as described above.
> 
> A little bit of browsing the related DNS shows quite a bit of a mess.
> None of the published domains properly roundtrip A -> PTR -> A, they
> just lead into a maze of little twisty passages all alike.  That
> doesn't necessarily cause a problem by itself, but it often makes
> problems harder to diagnose.

I see. Not sure exactly what that means?.but thanks...
> 
>> and nangoku-jiyu-jin.net <http://nangoku-jiyu-jin.net/> is a subset
>> of turnlefthostimg.org <http://turnlefthostimg.org/>,
> 
> I think you're going to have to ask your upstream for help on this.

Ok...
> 
> That said there are a few long shots you could try to diagnose it
> yourself.  One is a manual transaction with the mailserver and see if
> it's being rejected at the SMTP level.

What would I need to do?


> 
> Another thing I noticed is that you typoed "turnlefthosting.org" as
> "turnlefthostimg.org" in the above.  Is it possible that you typoed a
> host or list name somewhere in a recent configuration ichange?

Nothing changed at all in my list.  It was working with no problems for 
2 years.  Recently I have been getting a lot of spam, so when a sufficient number had built
up I went in and got rid of it using BAN so-and-so, DISCARD, so and so?

I am suspicious that I did something at this level, but can?t see anything that would cause 
that problem?.
> 
> Do you have a personal mailbox on nangoku-jiyu-jin.net?  If so, does
> that work?

Yes, both paul at nangoku-jiyu-jin.net <mailto:paul at nangoku-jiyu-jin.net> and paul at turnlefthosting.org <mailto:paul at turnlefthosting.org> work when sent mail from another address.

BUT it is going on 5 minutes that I sent mail FROM each of those addresses to themselves and they did NOT arrive.

An SMTP level error?

They show up in their respective SENT folders and I get no error message.

This sounds like NOT a MAILMAN problem?

The last time I used the nangoku-jiyu-jin account to send was actually to mailman in July for the newsletter, which went through, so 
something has happened in the last month to affect SENDING from my domains it seems.


> 
> I suppose you are the owner of your lists.  Have you tried writing to
> <listname>-owner at nangoku-jiyu-jin.net?  It should forward to you.

Just tried, from a different domain, and am still waiting for a response.  Suggests it is not ONLY an SMPT problem?

> 
> According to the page at
> http://www.nangoku-jiyu-jin.net/mailman/listinfo/mailman, the mailman@
> list is hosted by dionysus.thewebhostserver.com, and the owner is
> root at dionysus.thewebhostserver.com.  This is probably typical of
> cPanel (which has its own way of handling virtual hosting), but you
> should check your own listinfo and admin pages to be sure the virtual
> hosting is working correctly and everything points to



> nangoku-jiyu-jin.net.  (You may have trouble finding the proper
> Mailman interface, though, because cPanel has its own idiosyncratic
> management interface, and may even make the original inaccessible.)


This is it, right?


http://nangoku-jiyu-jin.net/mailman/listinfo/tokyoprogressive_nangoku-jiyu-jin.net <http://nangoku-jiyu-jin.net/mailman/listinfo/tokyoprogressive_nangoku-jiyu-jin.net>


webhostserver is the name EZPZ (my webhosting provider) assigns.  I am not sure about the dionysus?.

> 
> Are your lists archived?
> 

yes, here:

http://nangoku-jiyu-jin.net/pipermail/tokyoprogressive_nangoku-jiyu-jin.net/ <http://nangoku-jiyu-jin.net/pipermail/tokyoprogressive_nangoku-jiyu-jin.net/>





>> Then I figured out my FTP login and got in using the Transmit
>> program.
> 
> I don't think this will help.  Most FTP server software is hard to
> secure, and standard practice of hosting systems is to chroot them
> (that means that only a part of the filesystem is visible to the FTP
> server, and in fact to any programs run by the FTP server).  chroots
> are almost impossible to break unless you have shell access (which is
> one reason why hosting services are kechi about handing out shell).

You are in Japan by any chance? You know the term ?kechi?.
There is an historian on Japanese history with your name, but wait, there is an academic at Tsukuba
with the same name.  (I was there for two years starting in 2009 or 10).

> 
>> I click on usr and get Could not change directory to /usr
>> Server said can?t change directory to /usr: No such directory.
>> Error -125: remote chdir failed.
> 
> That's bizarre.  I wonder if your host knows what it's doing.

I wonder?.EZPZ is pretty big, but i guess that doesn?t necessarily mean anything?.Also, this is an installation from the Softaculous package, so
I doubt they even support it.

> 
>> There is a cpanel folder but that doesn?t seem to help.
> 
> I'm not surprised.
> 
> Good luck,
> 
> Steve


Thanks for all the brainstorming.

paul



From dennisthetiger at chez-vrolet.net  Sun Aug 23 08:42:13 2015
From: dennisthetiger at chez-vrolet.net (Dennis Carr)
Date: Sat, 22 Aug 2015 23:42:13 -0700
Subject: [Mailman-Users] SPF best practices?
Message-ID: <20150822234213.cfead8879db5006f095af5bf@chez-vrolet.net>

After a rash of bounces that brought to my attention that my SPF
record was not properly published, I rewrote my SPF record a bit.  What
I'm looking for is some insight on best practices and a second set of
eyes, as my list hosts for addresses all over the internet. 

The record is such:

v=spf1 ip4:206.225.172.6 a:bast.chez-vrolet.net a:chez-vrolet.net
a:smtp.comcast.net ~all

This is published as a TXT record as per specification.

The a:smtp.comcast.net is necessary so I can send email remotely
through my ISP and clear out successfully. 

I'm a bit bothered by the '~all', however.  I really don't want to do
'-all' as I'm concerned that anybody who posts to the list would cause
anybody on Yahoo or the MSFT owned domains (hotmail, live, etc.) to
bounce again.  

Anybody have any suggestions on how to make this happen *correctly*, or
if there are any revisions I should make to my record?

-Dennis Carr

From jimpop at gmail.com  Sun Aug 23 12:24:40 2015
From: jimpop at gmail.com (Jim Popovitch)
Date: Sun, 23 Aug 2015 06:24:40 -0400
Subject: [Mailman-Users] SPF best practices?
In-Reply-To: <20150822234213.cfead8879db5006f095af5bf@chez-vrolet.net>
References: <20150822234213.cfead8879db5006f095af5bf@chez-vrolet.net>
Message-ID: <CAGfsgR0QUwCsAgVNcNxVPrR8WSneEz9r3BEGgsHbRtXCP6MZRg@mail.gmail.com>

On Sun, Aug 23, 2015 at 2:42 AM, Dennis Carr
<dennisthetiger at chez-vrolet.net> wrote:
> After a rash of bounces that brought to my attention that my SPF
> record was not properly published, I rewrote my SPF record a bit.  What
> I'm looking for is some insight on best practices and a second set of
> eyes, as my list hosts for addresses all over the internet.
>
> The record is such:
>
> v=spf1 ip4:206.225.172.6 a:bast.chez-vrolet.net a:chez-vrolet.net
> a:smtp.comcast.net ~all
>
> This is published as a TXT record as per specification.
>
> The a:smtp.comcast.net is necessary so I can send email remotely
> through my ISP and clear out successfully.


Try this:

   "v=spf1 mx a:comcast.net -all"

Or this:

    "v=spf1 ip4:206.225.172.6 a:comcast.net -all"

The first one is flexible over time, the 2nd one is more strict.

> I'm a bit bothered by the '~all', however.  I really don't want to do
> '-all' as I'm concerned that anybody who posts to the list would cause
> anybody on Yahoo or the MSFT owned domains (hotmail, live, etc.) to
> bounce again.

You do want -all, as Mailman sends from $list-bounces at domain.tld, not
user at gmail.com

-Jim P.

From stephen at xemacs.org  Sun Aug 23 17:13:14 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 24 Aug 2015 00:13:14 +0900
Subject: [Mailman-Users]  SPF best practices?
In-Reply-To: <20150822234213.cfead8879db5006f095af5bf@chez-vrolet.net>
References: <20150822234213.cfead8879db5006f095af5bf@chez-vrolet.net>
Message-ID: <87y4h26nsl.fsf@uwakimon.sk.tsukuba.ac.jp>

Dennis Carr writes:

 > The a:smtp.comcast.net is necessary so I can send email remotely
 > through my ISP and clear out successfully. 

That does mean that anybody who can send through smtp.comcast.net can
send as a mailbox from your domain and pass DMARC, most likely.  I
don't see a way to profitably exploit that offhand, though (unless
you're a bank).

 > I'm a bit bothered by the '~all', however.  I really don't want to do
 > '-all' as I'm concerned that anybody who posts to the list would cause
 > anybody on Yahoo or the MSFT owned domains (hotmail, live, etc.) to
 > bounce again.

Executive summary: if you're sure you've got all your hosts covered by
the SPF record, use -all as Jim P says.

Explanation: If you've got the SPF right, you *do* know all of the
relevant hosts, and you've got them covered.  Anybody else is spoofing
your host at the transport level (*not* the From header), so deny
them.

OTOH, your SPF has nothing to do with authentication of list posts
from other domains.  If your MTA and Mailman are configured correctly,
both HELO and MAIL FROM defined by RFC 5321 will contain one of your
domains (bast.chez-vrolet.net or chez-vrolet.net), and the last hop
will be verified as coming from your domain using your SPF.  This is
regardless of the identity in From.

If the recipient participates in DMARC, and the message is From you,
it will also pass DMARC.  (Effectively; the details are nitpicky.)

If the recipient participates in the DMARC protocol, and you resend a
post from a third party, the recipient will *also* check the SPF for
the domain in the RFC 5322 From field, and it will fail.  There is no
change you can make to your SPF record that can change this; it's the
remote domain's SPF record that matters.

This is why DMARC specifies that a valid DKIM signature by the domain
in From is also a pass.  SPF is absolutely useless except for "direct
to recipient" messages (strictly speaking, sender's MX to recipient's
MX, it might bounce around a bit inside each domain).  Of course such
direct mail is a large fraction of mail on the Internet nowadays, so
it's a very useful exception in general.

Unfortunately, public discussion mailing lists can't take advantage of
that exception.

HTH,

Steve

From mark at msapiro.net  Sun Aug 23 18:03:46 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 23 Aug 2015 09:03:46 -0700
Subject: [Mailman-Users] SPF best practices?
In-Reply-To: <87y4h26nsl.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20150822234213.cfead8879db5006f095af5bf@chez-vrolet.net>
 <87y4h26nsl.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <55D9EEE2.8000301@msapiro.net>

On 08/23/2015 08:13 AM, Stephen J. Turnbull wrote:
> 
> Executive summary: if you're sure you've got all your hosts covered by
> the SPF record, use -all as Jim P says.


There is an issue with -all. SPF does not work with .forwards or other
relaying of that nature. If you can be certain that every recipient's
final MX is the one your server sends to, then -all is OK, but you can't.

The scenario is your list member is user at example.com. user at example.com
is set to forward all mail to example_user at yahoo.com. Yahoo receives the
list post with envelope from listname-bounces at your.list.server (or
listname-bounces+user=example.com at your.list.server if VERPed) for
example_user at yahoo.com from the server at example.com. The SPF for
your.list.server doesn't allow example.com as a relay, so SPF fails at
yahoo.com.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From stephen at xemacs.org  Sun Aug 23 18:44:01 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 24 Aug 2015 01:44:01 +0900
Subject: [Mailman-Users] Why do my posts to own mailman list disappear.
In-Reply-To: <F31F58EC-6058-4A4F-A8F6-8550AEE7E98E@tokyoprogressive.org>
References: <mailman.528.1440095263.28099.mailman-users@python.org>
 <3E918CBF-C737-42E7-9285-D5C97E4E4275@tokyoprogressive.org>
 <138DBE3F-0BB0-4474-AD34-0366C0582176@tokyoprogressive.org>
 <8761478t55.fsf@uwakimon.sk.tsukuba.ac.jp>
 <F31F58EC-6058-4A4F-A8F6-8550AEE7E98E@tokyoprogressive.org>
Message-ID: <87wpwm6jla.fsf@uwakimon.sk.tsukuba.ac.jp>

Paul Arenson/tokyoprogressive writes:

 > Actually I am not sure what PTR is?..but my reseller hosting is
 > just that, I pay for space from EZPZ and then can do as described
 > above.

Ah, sorry.  PTR is the pointer record from a numeric IP address to a
human-readable domain name.

 > I see. Not sure exactly what that means?.but thanks...

If you don't know, and don't care, no problem.  If you want the long
story, just ask. :-)

 > > That said there are a few long shots you could try to diagnose it
 > > yourself.  One is a manual transaction with the mailserver and see if
 > > it's being rejected at the SMTP level.
 > 
 > What would I need to do?

Give me an existing address, which you already did.  :-)  I just sent
you a test message from steve at turnbull.sk.tsukuba.ac.jp, and it was
accepted "OK" by nangoku-jiyu-jin-net (which again identified itself
as dionysus).  So it's not a problem at the incoming SMTP level, mail
is going in to the host.

 > Nothing changed at all in my list.

No changes to the host configuration (general spam filters, etc) that
you know of, either, right?

 > It was working with no problems for 2 years.  Recently I have been
 > getting a lot of spam, so when a sufficient number had built up I
 > went in and got rid of it using BAN so-and-so, DISCARD, so and so?

You mean in the Mailman moderation interface for the list, right?
Theoretically you could have sent mail, it got spamtrapped but you
didn't notice the your post never appeared on the list, you banned the
author and didn't notice it was you, etc, etc.  Doesn't seem likely.

It would be much more likely if you entered patterns by hand in the
Privacy -> {Sender,Recipient,Spam} Filter screens.  (I don't know if
cPanel has those, that's what they are called in unmodified Mailman.)
But I guess you didn't do that?

 > Yes, both paul at nangoku-jiyu-jin.net
 > <mailto:paul at nangoku-jiyu-jin.net> and paul at turnlefthosting.org
 > <mailto:paul at turnlefthosting.org> work when sent mail from another
 > address.
 > 
 > BUT it is going on 5 minutes that I sent mail FROM each of those
 > addresses to themselves and they did NOT arrive.

How do you do that?  By setting the From and To addresses in your
local mail client, and sending via smtp.comcast.net?  Or do you have
webmail running on nangoku-jiyu-jin.net, or ... ?

 > They show up in their respective SENT folders and I get no error
 > message.

Do you have a copy of an old message sent through your list, with all
headers intact (I want to look at the Received headers)?  I don't need
the content, just the headers, but if the content is innocuous, just
forward the whole message to me.

 > This sounds like NOT a MAILMAN problem?

I agree.

 > http://nangoku-jiyu-jin.net/mailman/listinfo/tokyoprogressive_nangoku-jiyu-jin.net

Everything looks good there.  Not that there's all that much, but it
looks like the admin addresses etc in the mailto links are right and
the archive link works, and the admin link works (although as expected
it's not very helpful since I can't log in).

 > > Are your lists archived?

Thanks, I found them.  Unfortunately, they don't have the trace
headers (Received) in them (and they wouldn't have the outgoing hops
anyway).  So they weren't much help.

 > You are in Japan by any chance? You know the term ?kechi?.  There
 > is an historian on Japanese history with your name, but wait, there
 > is an academic at Tsukuba with the same name.  (I was there for two
 > years starting in 2009 or 10).

I'm the latter, in the Policy and Planning Sciences department.

 > I wonder?.EZPZ is pretty big, but i guess that doesn?t necessarily
 > mean anything?.Also, this is an installation from the Softaculous
 > package, so I doubt they even support it.

How do you install something unsupported if you don't have a shell
account?  I'm curious because maybe you do have a shell account but
don't know it. :-)

Steve

From stephen at xemacs.org  Sun Aug 23 19:59:16 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 24 Aug 2015 02:59:16 +0900
Subject: [Mailman-Users] SPF best practices?
In-Reply-To: <55D9EEE2.8000301@msapiro.net>
References: <20150822234213.cfead8879db5006f095af5bf@chez-vrolet.net>
 <87y4h26nsl.fsf@uwakimon.sk.tsukuba.ac.jp>
 <55D9EEE2.8000301@msapiro.net>
Message-ID: <87twrp7uob.fsf@uwakimon.sk.tsukuba.ac.jp>

Mark Sapiro writes:
 > On 08/23/2015 08:13 AM, Stephen J. Turnbull wrote:
 > > 
 > > Executive summary: if you're sure you've got all your hosts covered by
 > > the SPF record, use -all as Jim P says.
 > 
 > There is an issue with -all. SPF does not work with .forwards or other
 > relaying of that nature. If you can be certain that every recipient's
 > final MX is the one your server sends to, then -all is OK, but you
 > can't.

True enough.

Note: If I took that argument seriously, I'd use ?all, not ~all,
though.  According to RFC 4408, you shouldn't reject a message only
because of an SPF softfail, but it's not neutral, either.  Mail will
be lost if you use ~all, just not as much.

 > The scenario is your list member is user at example.com.
 > user at example.com is set to forward all mail to example_user at yahoo.com.

Heh.  This user is screwed if you use dmarc_moderation_action too.

Bottom line: Friends don't let friends use Yahoo! or AOL.


From mark at msapiro.net  Sun Aug 23 20:27:57 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 23 Aug 2015 11:27:57 -0700
Subject: [Mailman-Users] SPF best practices?
In-Reply-To: <87twrp7uob.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20150822234213.cfead8879db5006f095af5bf@chez-vrolet.net>	<87y4h26nsl.fsf@uwakimon.sk.tsukuba.ac.jp>	<55D9EEE2.8000301@msapiro.net>
 <87twrp7uob.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <55DA10AD.40507@msapiro.net>

On 08/23/2015 10:59 AM, Stephen J. Turnbull wrote:
> Mark Sapiro writes:
> 
>  > The scenario is your list member is user at example.com.
>  > user at example.com is set to forward all mail to example_user at yahoo.com.
> 
> Heh.  This user is screwed if you use dmarc_moderation_action too.


I don't think so. The munged From: will be from the list's domain which
probably doesn't publish a DMARC policy, but even if it does, it
*should* also be DKIM signing the outgoing mail.

The forward shouldn't alter the message in ways that break the list
server's DKIM sig so at the ultimate receiving end the message has a
valid DKIM sig that aligns with the From: domain.


> Bottom line: Friends don't let friends use Yahoo! or AOL.


+1

As an aside, perhaps a more telling example of how SPF is broken is the
following. example.com publishes an SPF with '-all'. user at example.com
sends a message to postmaster at python.org which is an alias for a few
people's addresses on other hosts. If all those end recipients check SPF
they may reject the message with envelope from user at example.com because
it comes from a server at mail.python.org which isn't allowed to send
mail with envelope from the example.com domain.

You can never know if any of your intended recipient addresses pass
through such a relay, thus my opinion is if you're concerned about your
mail being delivered, you can't use SPF -all.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From nnicholson at dioceseofnewark.org  Mon Aug 24 01:45:57 2015
From: nnicholson at dioceseofnewark.org (Nina Nicholson)
Date: Sun, 23 Aug 2015 23:45:57 +0000
Subject: [Mailman-Users] need assistance upgrading Mailman
Message-ID: <628EB8DB28F42B4AB423051B8A25BC6201344DED5E@EDNW2008.dioceseofnewark.local>

I manage 70+ Mailman listservs for the Episcopal Diocese of Newark, and we are having terrible problems with emails not being delivered because they fail DMARC Evaluation, to the point that it's disrupting communications in the diocese.

Our listservs are hosted by MediaTemple which provides Mailman ver. 2.1.9. I understand that if I were to upgrade to the latest version there are new features that would solve this problem. I've found the upgrade instructions (http://wiki.list.org/DOC/4.80%20How%20do%20I%20upgrade%20from%20Mailman%202.1.x%20to%20a%20later%20Mailman%202.1.y%3F) but they are beyond my limited Unix skills. MediaTemple refuses to do the upgrade.

I'm desperately looking for some guidance on how to get this taken care of. I would be willing to pay someone to do this as a freelance project.

Any help is appreciated. Thank you.

Nina Nicholson
Director of Communications & Technology
The Episcopal Diocese of Newark
973-430-9907
http://dioceseofnewark.org
http://facebook.com/dionewark
http://twitter.com/dionewark

From mark at msapiro.net  Mon Aug 24 03:02:08 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 23 Aug 2015 18:02:08 -0700
Subject: [Mailman-Users] need assistance upgrading Mailman
In-Reply-To: <628EB8DB28F42B4AB423051B8A25BC6201344DED5E@EDNW2008.dioceseofnewark.local>
References: <628EB8DB28F42B4AB423051B8A25BC6201344DED5E@EDNW2008.dioceseofnewark.local>
Message-ID: <55DA6D10.9070700@msapiro.net>

On 08/23/2015 04:45 PM, Nina Nicholson wrote:
> I manage 70+ Mailman listservs for the Episcopal Diocese of Newark, and we are having terrible problems with emails not being delivered because they fail DMARC Evaluation, to the point that it's disrupting communications in the diocese.


Please see <http://wiki.list.org/DOC/Mailman%20is%20not%20Listserv>


> Our listservs are hosted by MediaTemple which provides Mailman ver. 2.1.9. I understand that if I were to upgrade to the latest version there are new features that would solve this problem. I've found the upgrade instructions (http://wiki.list.org/DOC/4.80%20How%20do%20I%20upgrade%20from%20Mailman%202.1.x%20to%20a%20later%20Mailman%202.1.y%3F) but they are beyond my limited Unix skills. MediaTemple refuses to do the upgrade.


Do you have a VPS with Mailman installed or are you on a shared host? If
the latter, it is unlikely that you have the access necessary to upgrade
Mailman on the host and it is unlikely that MediaTemple would be happy
with your doing it even if you could.

You may wish to see
<http://wiki.list.org/COM/Mailman%20hosting%20services>. I have no
information on most of these, but based on interactions with the
proprietor of EMWD and dotList, I can recommend them. This should not be
construed as a negative review of anyone else on that list. It's just
that I have no basis to evaluate most of them.

You may also find
<http://wiki.list.org/COM/Mailman%20consulting%20services> of interest.
Particularly the link to the Python jobs board.

Finally, you may wish to try one of the suggestions at
<http://wiki.list.org/x/17891458> as an interim stopgap measure.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From stephen at xemacs.org  Mon Aug 24 03:23:56 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 24 Aug 2015 10:23:56 +0900
Subject: [Mailman-Users] SPF best practices?
In-Reply-To: <55DA10AD.40507@msapiro.net>
References: <20150822234213.cfead8879db5006f095af5bf@chez-vrolet.net>
 <87y4h26nsl.fsf@uwakimon.sk.tsukuba.ac.jp>
 <55D9EEE2.8000301@msapiro.net>
 <87twrp7uob.fsf@uwakimon.sk.tsukuba.ac.jp>
 <55DA10AD.40507@msapiro.net>
Message-ID: <87si797a37.fsf@uwakimon.sk.tsukuba.ac.jp>

Mark Sapiro writes:
 > On 08/23/2015 10:59 AM, Stephen J. Turnbull wrote:
 > > Mark Sapiro writes:
 > > 
 > >  > The scenario is your list member is user at example.com.
 > >  > user at example.com is set to forward all mail to example_user at yahoo.com.
 > > 
 > > Heh.  This user is screwed if you use dmarc_moderation_action too.
 > 
 > 
 > I don't think so. The munged From:

I was thinking about the opposite case, where it *doesn't* get munged.
Temporary brain fart -- that is OK because the *From* domain can't be
p=reject, so no problem with DMARC even though Yahoo! has draconian
policies.  (Of course the user must deal with rejects due to Yahoo!'s
AUP, but that doesn't have anything to do with DMARC.)

This stuff is just hard to think about. :-(



From paul at tokyoprogressive.org  Mon Aug 24 02:25:12 2015
From: paul at tokyoprogressive.org (Paul Arenson/tokyoprogressive)
Date: Mon, 24 Aug 2015 09:25:12 +0900
Subject: [Mailman-Users] Why do my posts to own mailman list disappear.
In-Reply-To: <87wpwm6jla.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <mailman.528.1440095263.28099.mailman-users@python.org>
 <3E918CBF-C737-42E7-9285-D5C97E4E4275@tokyoprogressive.org>
 <138DBE3F-0BB0-4474-AD34-0366C0582176@tokyoprogressive.org>
 <8761478t55.fsf@uwakimon.sk.tsukuba.ac.jp>
 <F31F58EC-6058-4A4F-A8F6-8550AEE7E98E@tokyoprogressive.org>
 <87wpwm6jla.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <7F23B7BE-6646-4E4B-A80C-ADB47F5A4216@tokyoprogressive.org>

Thanks for your message, Steven.



> On 2015/08/24, at 1:44, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> 
> Paul Arenson/tokyoprogressive writes:
> 
>> Actually I am not sure what PTR is?..but my reseller hosting is
>> just that, I pay for space from EZPZ and then can do as described
>> above.
> 
> Ah, sorry.  PTR is the pointer record from a numeric IP address to a
> human-readable domain name.
> 
>> I see. Not sure exactly what that means?.but thanks...
> 
> If you don't know, and don't care, no problem.  If you want the long
> story, just ask. :-)
> 


Thanks....

>>> That said there are a few long shots you could try to diagnose it
>>> yourself.  One is a manual transaction with the mailserver and see if
>>> it's being rejected at the SMTP level.
>> 
>> What would I need to do?
> 
> Give me an existing address, which you already did.  :-)  I just sent
> you a test message from steve at turnbull.sk.tsukuba.ac.jp, and it was
> accepted "OK" by nangoku-jiyu-jin-net (which again identified itself
> as dionysus).  So it's not a problem at the incoming SMTP level, mail
> is going in to the host.


Right. I received it. I responded once from the same account. And once from the Tokyoprogressive account.  I believe ONLY the one I sent from Tokyoprogressive went through, pointing to a SENDING problem from nangoku-jiyu-jin and maybe also turnlefthosting.org.  Tokyoprogressive would be unaffected because it is a totally different provider.
> 
>> Nothing changed at all in my list.
> 
> No changes to the host configuration (general spam filters, etc) that
> you know of, either, right?
> 
>> It was working with no problems for 2 years.  Recently I have been
>> getting a lot of spam, so when a sufficient number had built up I
>> went in and got rid of it using BAN so-and-so, DISCARD, so and so?
> 
> You mean in the Mailman moderation interface for the list, right?
> Theoretically you could have sent mail, it got spamtrapped but you
> didn't notice the your post never appeared on the list, you banned the
> author and didn't notice it was you, etc, etc.  Doesn't seem likely.
> 
> It would be much more likely if you entered patterns by hand in the
> Privacy -> {Sender,Recipient,Spam} Filter screens.  (I don't know if
> cPanel has those, that's what they are called in unmodified Mailman.)
> But I guess you didn't do that?
> 

No, and everything I do though in mailman is in the mailman program.  Almost never touch cpanel.  
>> Yes, both paul at nangoku-jiyu-jin.net
>> <mailto:paul at nangoku-jiyu-jin.net> and paul at turnlefthosting.org
>> <mailto:paul at turnlefthosting.org> work when sent mail from another
>> address.
>> 
>> BUT it is going on 5 minutes that I sent mail FROM each of those
>> addresses to themselves and they did NOT arrive.
> 
> How do you do that?  By setting the From and To addresses in your
> local mail client, and sending via smtp.comcast.net?  Or do you have
> webmail running on nangoku-jiyu-jin.net, or ... ?

On my Mac in my mail client. There is a web interface in cpanel, but I have rarely used it.
> 
>> They show up in their respective SENT folders and I get no error
>> message.
> 
> Do you have a copy of an old message sent through your list, with all
> headers intact (I want to look at the Received headers)?  I don't need
> the content, just the headers, but if the content is innocuous, just
> forward the whole message to me.

Yes. When I get home I can send you the July message, which went through.  In mailman's archives it shows up with the html unreadable, but in  a mail client it works.
> 
>> This sounds like NOT a MAILMAN problem?
> 
> I agree.
> 
>> http://nangoku-jiyu-jin.net/mailman/listinfo/tokyoprogressive_nangoku-jiyu-jin.net
> 
> Everything looks good there.  Not that there's all that much, but it
> looks like the admin addresses etc in the mailto links are right and
> the archive link works, and the admin link works (although as expected
> it's not very helpful since I can't log in).

I could enable that if need be.
> 
>>> Are your lists archived?
> 
> Thanks, I found them.  Unfortunately, they don't have the trace
> headers (Received) in them (and they wouldn't have the outgoing hops
> anyway).  So they weren't much help.
> 
>> You are in Japan by any chance? You know the term ?kechi?.  There
>> is an historian on Japanese history with your name, but wait, there
>> is an academic at Tsukuba with the same name.  (I was there for two
>> years starting in 2009 or 10).
> 
> I'm the latter, in the Policy and Planning Sciences department.

Dozo,yoroshiku. I was at the language center and also taught the head of the nursing dept privately as my wife is also a nurse.
> 
>> I wonder?.EZPZ is pretty big, but i guess that doesn?t necessarily
>> mean anything?.Also, this is an installation from the Softaculous
>> package, so I doubt they even support it.
> 
> How do you install something unsupported if you don't have a shell
> account?  I'm curious because maybe you do have a shell account but
> don't know it. :-)

In Cpanel they have something called Softaculous. Previously they used something called Fantastico. I guess they are a collection of auto installers. So you can find Joomla, Mailman, etc. all located there.


Thanks,

paul
> 
> Steve

From stephen at xemacs.org  Mon Aug 24 04:00:35 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 24 Aug 2015 11:00:35 +0900
Subject: [Mailman-Users]  need assistance upgrading Mailman
In-Reply-To: <628EB8DB28F42B4AB423051B8A25BC6201344DED5E@EDNW2008.dioceseofnewark.local>
References: <628EB8DB28F42B4AB423051B8A25BC6201344DED5E@EDNW2008.dioceseofnewark.local>
Message-ID: <87r3mt78e4.fsf@uwakimon.sk.tsukuba.ac.jp>

Nina Nicholson writes:

 > Our listservs are hosted by MediaTemple which provides Mailman
 > ver. 2.1.9. I understand that if I were to upgrade to the latest
 > version there are new features that would solve this problem.

Yes.  They don't completely solve the problem, but they provide a
level of mitigation that is satisfactory for most lists.

 > I've found the upgrade instructions, but they are beyond my limited
 > Unix skills.  MediaTemple refuses to do the upgrade.
 >
 > I'm desperately looking for some guidance on how to get this taken
 > care of.

I assume you have the necessary access (a shell login account, and
permissions to install software) to the server to do the upgrade.  It
should be a straightforward operation for an experienced Mailman
admin, so hiring a consultant should be an inexpensive and quick way
to improve your service.

For the long run, I would suggest considering a specialist host if you
don't want to take on a permanent Unix admin or put a consultant on
retainer.  Mail services require quite a bit of finicky configuration,
more so than pretty much any other Internet service.  There is active
research on protecting mail service from spam and phishing going on,
and no guarantee that there won't be further adjustments needed in the
future.  There are plenty of inexpensive hosts out there, and you
don't necessarily need to move your other services.

Steve

From stephen at xemacs.org  Mon Aug 24 06:34:18 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 24 Aug 2015 13:34:18 +0900
Subject: [Mailman-Users] Why do my posts to own mailman list disappear.
In-Reply-To: <7F23B7BE-6646-4E4B-A80C-ADB47F5A4216@tokyoprogressive.org>
References: <mailman.528.1440095263.28099.mailman-users@python.org>
 <3E918CBF-C737-42E7-9285-D5C97E4E4275@tokyoprogressive.org>
 <138DBE3F-0BB0-4474-AD34-0366C0582176@tokyoprogressive.org>
 <8761478t55.fsf@uwakimon.sk.tsukuba.ac.jp>
 <F31F58EC-6058-4A4F-A8F6-8550AEE7E98E@tokyoprogressive.org>
 <87wpwm6jla.fsf@uwakimon.sk.tsukuba.ac.jp>
 <7F23B7BE-6646-4E4B-A80C-ADB47F5A4216@tokyoprogressive.org>
Message-ID: <87k2sl719x.fsf@uwakimon.sk.tsukuba.ac.jp>

Paul Arenson/tokyoprogressive writes:

 > Right. I received it.

 > I responded once from the same account.  And once from the
 > Tokyoprogressive account.

To "steve at turnbull.sk.tsukuba.ac.jp"?

I don't see either one yet, but that could be my network.  The
university just upgraded the network and past experience suggests
things are likely to be unreliable here until New Year's.

 > I believe ONLY the one I sent from Tokyoprogressive went through,
 > pointing to a SENDING problem from nangoku-jiyu-jin and maybe also
 > turnlefthosting.org.  Tokyoprogressive would be unaffected because
 > it is a totally different provider.

If you got (temporary) bounces sending to "stephen at xemacs.org", that
is probably due to "greylisting" (an antispam measure) at xemacs.org.

 > No, and everything I do though in mailman is in the mailman
 > program.  Almost never touch cpanel.

OK, that's helpful to know in general, although it doesn't change any
immediate diagnosis.

 > On my Mac in my mail client. There is a web interface in cpanel,
 > but I have rarely used it.

Is your mail client configured to use those hosts (that would be very
unusual), or just your local ISP connection?

 > In Cpanel they have something called Softaculous. Previously they
 > used something called Fantastico. I guess they are a collection of
 > auto installers. So you can find Joomla, Mailman, etc. all located
 > there.

I see.  Love people who care more about names than functionality, but
I guess that's the marketing department.  I'll see if your mail comes
through in the next few hours.

Steve

From dunkel at calamus.net  Mon Aug 24 09:10:46 2015
From: dunkel at calamus.net (Ulf Dunkel)
Date: Mon, 24 Aug 2015 09:10:46 +0200
Subject: [Mailman-Users] How to discard messages from just a few list
 members?
In-Reply-To: <55D89502.6000708@msapiro.net>
References: <55D6E128.3020307@calamus.net> <55D73114.9020301@msapiro.net>
 <55D868C1.8080606@calamus.net> <55D87333.3060501@Damon-Family.org>
 <55D89502.6000708@msapiro.net>
Message-ID: <55DAC376.7080905@calamus.net>

Dear Mark.

> To be more specific, set Privacy options... -> Sender filters ->
> default_member_moderation to Yes to automatically moderate new members
> and set Privacy options... -> Sender filters -> member_moderation_action
> to Discard and Submit Your Changes. Then on the Membership Management...
> -> Membership List page under Additional Member Tasks, set "Set
> everyone's moderation bit, including those members not currently
> visible" to On and click "Set".
> 
> Then, in order to be able to post yourself, uncheck your "mod" box on
> the Membership Management... -> Membership List pages and Submit Your
> Changes, or better still, leave yourself moderated and post with an
> Approved: <password> header. This latter method prevents others from
> posting by spoofing your address. See the FAQs at
> <http://wiki.list.org/x/4030685> and <http://wiki.list.org/x/4030556>.

Thank you for your detailed cookbook, but I would lose moderation of my
own postings to this newsletter mailing list if I would do either way as
you suggested. I know that the basic intention of Mailman mailing lists
is to let all members of a list communicate with others and I am kind of
"misusing" it as a newsletter mailing list.

---Ulf Dunkel

From dunkel at calamus.net  Mon Aug 24 09:23:26 2015
From: dunkel at calamus.net (Ulf Dunkel)
Date: Mon, 24 Aug 2015 09:23:26 +0200
Subject: [Mailman-Users] How to discard messages from just a few list
 members?
In-Reply-To: <87zj1i7tdb.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <55D6E128.3020307@calamus.net> <55D73114.9020301@msapiro.net>
 <55D868C1.8080606@calamus.net> <55D87333.3060501@Damon-Family.org>
 <55D894BA.4020607@calamus.net> <87zj1i7tdb.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <55DAC66E.1050000@calamus.net>

Hi Stephen.

> > Thank you, but I won't send unmoderated news to any newsletter. I am
> > using Mailman for felt decades and I am so happy that the moderated flag
> > always kept me from sending postings too early.
>
> Unmoderate yourself in the member view, and then use a privacy filter
> with your address, and action set to Hold.
>
> The same strategy should work for any number of authorized posters by
> adding their addresses to the regexp with "|".
>
> Note: Untested, but AFAIK the several moderation methods are
> independent in this way.

I now have set these settings:

Privacy Options > Subscription rules > Sender filters:
- default_member_moderation: Yes (was already set)
- member_moderation_action: Discard (was: Hold)
- member_moderation_notice:
"Sorry. This is a moderated mailing list. You don't have write access here."
(without quotation marks)

Privacy Options > Subscription rules > Spam filters:
- header_filter_rules:
Spam Filter Regexp:
"From: Ulf Dunkel <dunkel at calamus.net>" (without quotation marks) *)
Action: Hold

*) I firstly had to check which exact header phrase was used from my
email client.

This seems to be the setting that works for me.

---Ulf Dunkel

From lac at openend.se  Mon Aug 24 09:49:14 2015
From: lac at openend.se (Laura Creighton)
Date: Mon, 24 Aug 2015 09:49:14 +0200
Subject: [Mailman-Users] How to discard messages from just a few list
	members?
In-Reply-To: <55DAC376.7080905@calamus.net>
References: <55D6E128.3020307@calamus.net> <55D73114.9020301@msapiro.net>
 <55D868C1.8080606@calamus.net> <55D87333.3060501@Damon-Family.org>
 <55D89502.6000708@msapiro.net> <55DAC376.7080905@calamus.net>
Message-ID: <201508240749.t7O7nEYm023394@fido.openend.se>

In a message of Mon, 24 Aug 2015 09:10:46 +0200, Ulf Dunkel writes:

>Thank you for your detailed cookbook, but I would lose moderation of my
>own postings to this newsletter mailing list if I would do either way as
>you suggested. I know that the basic intention of Mailman mailing lists
>is to let all members of a list communicate with others and I am kind of
>"misusing" it as a newsletter mailing list.
>
>---Ulf Dunkel

I had an idea for a work-around for you.

Get yourself a second account.  Subscribe both of them to the mailing list.
Moderate one, not the other.  Disable mail delivery for one of the
accounts -- don't unsubscribe, just turn off delivery.  You can set both
of them up as list moderators, list owners if you like, or just the one
account if that is better for you.

Laura



From dunkel at calamus.net  Mon Aug 24 11:18:39 2015
From: dunkel at calamus.net (Ulf Dunkel)
Date: Mon, 24 Aug 2015 11:18:39 +0200
Subject: [Mailman-Users] How to discard messages from just a few list
 members?
In-Reply-To: <201508240749.t7O7nEYm023394@fido.openend.se>
References: <55D6E128.3020307@calamus.net> <55D73114.9020301@msapiro.net>
 <55D868C1.8080606@calamus.net> <55D87333.3060501@Damon-Family.org>
 <55D89502.6000708@msapiro.net> <55DAC376.7080905@calamus.net>
 <201508240749.t7O7nEYm023394@fido.openend.se>
Message-ID: <55DAE16F.10006@calamus.net>

> Get yourself a second account.  Subscribe both of them to the mailing list.
> Moderate one, not the other.  Disable mail delivery for one of the
> accounts -- don't unsubscribe, just turn off delivery.  You can set both
> of them up as list moderators, list owners if you like, or just the one
> account if that is better for you.

Thank you for that suggestion, Laura, but it would also remove the
basically wanted moderation of my own postings to my newsletter mailing
list (I explained earlier why it is required).

I really seem to have found a working setup using the Spam filter now,
as I posted about 40 minutes ago.

---Ulf Dunkel

From wetz.peter at gmail.com  Mon Aug 24 14:09:56 2015
From: wetz.peter at gmail.com (Peter Wetz)
Date: Mon, 24 Aug 2015 14:09:56 +0200
Subject: [Mailman-Users] Archive page not showing dates properly
In-Reply-To: <55D73CC9.1050303@msapiro.net>
References: <CAGmyAkAB7h3YSVVm4LSpHt-7mQ8tFQD5N-Ok=3R8BQuJfVSNEQ@mail.gmail.com>
 <55D73CC9.1050303@msapiro.net>
Message-ID: <CAGmyAkD86FTBfiGpKDHM+nGsMPe9gS_h8Uwov0Lx_XMS=sAgVw@mail.gmail.com>

thanks for the help.
what do I have to do after editing Mailman/Archiver/HyperArch.py as
suggested by you? Or: How can I test, if the changes actually changed
something?

Because right now, I added your changes and sent a testmail to the list.
The mail gets added to the archive, however, the date still looks as in the
screenshot posted in my initial posting.



On Fri, Aug 21, 2015 at 4:59 PM, Mark Sapiro <mark at msapiro.net> wrote:

> On 08/21/2015 02:29 AM, Peter Wetz wrote:
> > On the archive page of my mailman list the dates are not showing up
> > properly. It looks like this: http://i.imgur.com/IrmWnie.png
> >
> > As you can see, instead of the date a string "%(date)s" is shown. It is
> > also worth noting, that it already worked properly. However, suddenly (at
> > least I cannot state when exactly) it stopped working and since then
> dates
> > are shown as seen in the screenshot.
>
>
> The standard Mailman archidxentry.html template does not have a %(date)s
> substitution in it. Presumably, your installation/list has a custom
> version of this template (<http://wiki.list.org/x/4030605>) which adds
> the %(date)s substitution and along with this, modifications to
> Mailman/Archiver/HyperArch.py to add a 'date' entry to the interpolation
> dictionary for this template, and also presumably, some Mailman upgrade
> removed the modification to HyperArch.py.
>
>
> > How can I fix this problem?
>
>
> Replace the current Mailman/Archiver/HyperArch.py with the one that was
> there when it worked, or in the definition of write_index_entry at about
> line 1073 where you see
>
>         d = {
>             'filename': urllib.quote(article.filename),
>             'subject':  subject,
>             'sequence': article.sequence,
>             'author':   author
>         }
>
> change it to
>
>         d = {
>             'filename': urllib.quote(article.filename),
>             'subject':  subject,
>             'sequence': article.sequence,
>             'date':     article.datestr,
>             'author':   author
>         }
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/wetz.peter%40gmail.com
>

From wetz.peter at gmail.com  Mon Aug 24 14:20:01 2015
From: wetz.peter at gmail.com (Peter Wetz)
Date: Mon, 24 Aug 2015 14:20:01 +0200
Subject: [Mailman-Users] Archive page not showing dates properly
In-Reply-To: <55D73CC9.1050303@msapiro.net>
References: <CAGmyAkAB7h3YSVVm4LSpHt-7mQ8tFQD5N-Ok=3R8BQuJfVSNEQ@mail.gmail.com>
 <55D73CC9.1050303@msapiro.net>
Message-ID: <CAGmyAkDG6XRY_X0nSwkVAJ=tSRbBZimrvwJqozdYzxxAuMD8zw@mail.gmail.com>

okay, my admin forgot to restart the service. the dates are now working
again!

one more question: how/where can I format the string?
right now it looks like this: "Mon, 24 Aug 2015 14:18:21 +0200"
I would like the following: "24 Aug 2015 14:18:21"

Thanks and best,
Peter

On Fri, Aug 21, 2015 at 4:59 PM, Mark Sapiro <mark at msapiro.net> wrote:

> On 08/21/2015 02:29 AM, Peter Wetz wrote:
> > On the archive page of my mailman list the dates are not showing up
> > properly. It looks like this: http://i.imgur.com/IrmWnie.png
> >
> > As you can see, instead of the date a string "%(date)s" is shown. It is
> > also worth noting, that it already worked properly. However, suddenly (at
> > least I cannot state when exactly) it stopped working and since then
> dates
> > are shown as seen in the screenshot.
>
>
> The standard Mailman archidxentry.html template does not have a %(date)s
> substitution in it. Presumably, your installation/list has a custom
> version of this template (<http://wiki.list.org/x/4030605>) which adds
> the %(date)s substitution and along with this, modifications to
> Mailman/Archiver/HyperArch.py to add a 'date' entry to the interpolation
> dictionary for this template, and also presumably, some Mailman upgrade
> removed the modification to HyperArch.py.
>
>
> > How can I fix this problem?
>
>
> Replace the current Mailman/Archiver/HyperArch.py with the one that was
> there when it worked, or in the definition of write_index_entry at about
> line 1073 where you see
>
>         d = {
>             'filename': urllib.quote(article.filename),
>             'subject':  subject,
>             'sequence': article.sequence,
>             'author':   author
>         }
>
> change it to
>
>         d = {
>             'filename': urllib.quote(article.filename),
>             'subject':  subject,
>             'sequence': article.sequence,
>             'date':     article.datestr,
>             'author':   author
>         }
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/wetz.peter%40gmail.com
>

From wetz.peter at gmail.com  Mon Aug 24 16:34:44 2015
From: wetz.peter at gmail.com (Peter Wetz)
Date: Mon, 24 Aug 2015 16:34:44 +0200
Subject: [Mailman-Users] Archive page not showing dates properly
In-Reply-To: <55D73CC9.1050303@msapiro.net>
References: <CAGmyAkAB7h3YSVVm4LSpHt-7mQ8tFQD5N-Ok=3R8BQuJfVSNEQ@mail.gmail.com>
 <55D73CC9.1050303@msapiro.net>
Message-ID: <CAGmyAkCGsQrt3o++3+_2t-tnhsq-Sm6A=UtH0m_EDPzi28gD+w@mail.gmail.com>

okay, sorry for the spam, but I already wrapped my head around how to
change the date format.

just use python's functionality to parse it into a different string.
for instance: 'date': datetime.datetime.strptime(article.datestr, "%a, %d
%b %Y %H:%M:%S %z").strftime("%d %b %Y %H:%M:%S"),

best,
Peter

On Fri, Aug 21, 2015 at 4:59 PM, Mark Sapiro <mark at msapiro.net> wrote:

> On 08/21/2015 02:29 AM, Peter Wetz wrote:
> > On the archive page of my mailman list the dates are not showing up
> > properly. It looks like this: http://i.imgur.com/IrmWnie.png
> >
> > As you can see, instead of the date a string "%(date)s" is shown. It is
> > also worth noting, that it already worked properly. However, suddenly (at
> > least I cannot state when exactly) it stopped working and since then
> dates
> > are shown as seen in the screenshot.
>
>
> The standard Mailman archidxentry.html template does not have a %(date)s
> substitution in it. Presumably, your installation/list has a custom
> version of this template (<http://wiki.list.org/x/4030605>) which adds
> the %(date)s substitution and along with this, modifications to
> Mailman/Archiver/HyperArch.py to add a 'date' entry to the interpolation
> dictionary for this template, and also presumably, some Mailman upgrade
> removed the modification to HyperArch.py.
>
>
> > How can I fix this problem?
>
>
> Replace the current Mailman/Archiver/HyperArch.py with the one that was
> there when it worked, or in the definition of write_index_entry at about
> line 1073 where you see
>
>         d = {
>             'filename': urllib.quote(article.filename),
>             'subject':  subject,
>             'sequence': article.sequence,
>             'author':   author
>         }
>
> change it to
>
>         d = {
>             'filename': urllib.quote(article.filename),
>             'subject':  subject,
>             'sequence': article.sequence,
>             'date':     article.datestr,
>             'author':   author
>         }
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/wetz.peter%40gmail.com
>

From mark at msapiro.net  Mon Aug 24 16:59:57 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 24 Aug 2015 07:59:57 -0700
Subject: [Mailman-Users] {Disarmed} Re: Archive page not showing dates
 properly
In-Reply-To: <CAGmyAkCGsQrt3o++3+_2t-tnhsq-Sm6A=UtH0m_EDPzi28gD+w@mail.gmail.com>
References: <CAGmyAkAB7h3YSVVm4LSpHt-7mQ8tFQD5N-Ok=3R8BQuJfVSNEQ@mail.gmail.com>	<55D73CC9.1050303@msapiro.net>
 <CAGmyAkCGsQrt3o++3+_2t-tnhsq-Sm6A=UtH0m_EDPzi28gD+w@mail.gmail.com>
Message-ID: <55DB316D.9080905@msapiro.net>

On 08/24/2015 07:34 AM, Peter Wetz wrote:
> okay, sorry for the spam, but I already wrapped my head around how to
> change the date format.
> 
> just use python's functionality to parse it into a different string.
> for instance: 'date': datetime.datetime.strptime(article.datestr, "%a,
> %d %b %Y %H:%M:%S %z").strftime("%d %b %Y %H:%M:%S"),

Or, perhaps a bit simpler:

'date': datetime.datetime.fromtimestamp(article.date).strftime("%d %b %Y
%H:%M:%S")

or simpler still and not requiring import of datetime (HyperArch.py
already imports time

'date': time.strftime("%d %b %Y %H:%M:%S", time.localtime(article.date))

Note that the article instance has attributes

#  datestr  : The posting date, in human-readable format
#  date     : The posting date, in purely numeric format

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From wetz.peter at gmail.com  Mon Aug 24 17:06:36 2015
From: wetz.peter at gmail.com (Peter Wetz)
Date: Mon, 24 Aug 2015 17:06:36 +0200
Subject: [Mailman-Users] Archive page not showing dates properly
In-Reply-To: <55D73CC9.1050303@msapiro.net>
References: <CAGmyAkAB7h3YSVVm4LSpHt-7mQ8tFQD5N-Ok=3R8BQuJfVSNEQ@mail.gmail.com>
 <55D73CC9.1050303@msapiro.net>
Message-ID: <CAGmyAkBH-e6=+wHOQvdvYOs2qDh6msoyEwvcbx5qizVFb4dU1A@mail.gmail.com>

> presumably, some Mailman upgrade removed the modification to HyperArch.py.

Is there some sort of best practice on how to avoid this situation? That
is, how to avoid modification of HyperArch.py through some Mailman upgrade?

best,
Peter

On Fri, Aug 21, 2015 at 4:59 PM, Mark Sapiro <mark at msapiro.net> wrote:

> On 08/21/2015 02:29 AM, Peter Wetz wrote:
> > On the archive page of my mailman list the dates are not showing up
> > properly. It looks like this: http://i.imgur.com/IrmWnie.png
> >
> > As you can see, instead of the date a string "%(date)s" is shown. It is
> > also worth noting, that it already worked properly. However, suddenly (at
> > least I cannot state when exactly) it stopped working and since then
> dates
> > are shown as seen in the screenshot.
>
>
> The standard Mailman archidxentry.html template does not have a %(date)s
> substitution in it. Presumably, your installation/list has a custom
> version of this template (<http://wiki.list.org/x/4030605>) which adds
> the %(date)s substitution and along with this, modifications to
> Mailman/Archiver/HyperArch.py to add a 'date' entry to the interpolation
> dictionary for this template, and also presumably, some Mailman upgrade
> removed the modification to HyperArch.py.
>
>
> > How can I fix this problem?
>
>
> Replace the current Mailman/Archiver/HyperArch.py with the one that was
> there when it worked, or in the definition of write_index_entry at about
> line 1073 where you see
>
>         d = {
>             'filename': urllib.quote(article.filename),
>             'subject':  subject,
>             'sequence': article.sequence,
>             'author':   author
>         }
>
> change it to
>
>         d = {
>             'filename': urllib.quote(article.filename),
>             'subject':  subject,
>             'sequence': article.sequence,
>             'date':     article.datestr,
>             'author':   author
>         }
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/wetz.peter%40gmail.com
>

From mark at msapiro.net  Mon Aug 24 17:08:46 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 24 Aug 2015 08:08:46 -0700
Subject: [Mailman-Users] How to discard messages from just a few list
 members?
In-Reply-To: <55DAC66E.1050000@calamus.net>
References: <55D6E128.3020307@calamus.net> <55D73114.9020301@msapiro.net>
 <55D868C1.8080606@calamus.net> <55D87333.3060501@Damon-Family.org>
 <55D894BA.4020607@calamus.net> <87zj1i7tdb.fsf@uwakimon.sk.tsukuba.ac.jp>
 <55DAC66E.1050000@calamus.net>
Message-ID: <55DB337E.5070204@msapiro.net>

On 08/24/2015 12:23 AM, Ulf Dunkel wrote:
> 
> I now have set these settings:
> 
> Privacy Options > Subscription rules > Sender filters:
> - default_member_moderation: Yes (was already set)
> - member_moderation_action: Discard (was: Hold)
> - member_moderation_notice:
> "Sorry. This is a moderated mailing list. You don't have write access here."
> (without quotation marks)


The above is only used if member_moderation_action is Reject.


> Privacy Options > Subscription rules > Spam filters:
> - header_filter_rules:
> Spam Filter Regexp:
> "From: Ulf Dunkel <dunkel at calamus.net>" (without quotation marks) *)
> Action: Hold
> 
> *) I firstly had to check which exact header phrase was used from my
> email client.


For the regexp, I would use

^From:.*dunkel at calamus.net

The ^ ensures that you will match an actual From: header and not just
'from:' anywhere in the headers, and then just look for your email
address anywhere in that line. That way, it is not sensitive to how your
MUA formats it. Also, since the action is Hold and you want to discard
the rest, a false positive is not a problem as you can just discard the
held message.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Mon Aug 24 17:16:25 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 24 Aug 2015 08:16:25 -0700
Subject: [Mailman-Users] {Disarmed} Re: Archive page not showing dates
 properly
In-Reply-To: <CAGmyAkBH-e6=+wHOQvdvYOs2qDh6msoyEwvcbx5qizVFb4dU1A@mail.gmail.com>
References: <CAGmyAkAB7h3YSVVm4LSpHt-7mQ8tFQD5N-Ok=3R8BQuJfVSNEQ@mail.gmail.com>	<55D73CC9.1050303@msapiro.net>
 <CAGmyAkBH-e6=+wHOQvdvYOs2qDh6msoyEwvcbx5qizVFb4dU1A@mail.gmail.com>
Message-ID: <55DB3549.30005@msapiro.net>

On 08/24/2015 08:06 AM, Peter Wetz wrote:
>> presumably, some Mailman upgrade removed the modification to HyperArch.py.
> 
> Is there some sort of best practice on how to avoid this situation? That
> is, how to avoid modification of HyperArch.py through some Mailman upgrade?


A few things survive upgrades. mm_cfg.py is not changed in an upgrade
and list specific templates in lists/LISTNAME/LC/, domain specific
templates in templates/DOMAIN/ and sitewide templates in templates/site/
are not changed.

Any modified modules in the Mailman/ directory will be replaced in an
upgrade. Therefore, best practice is to keep all your local mods in
patch files and reapply the patches following an upgrade.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From nnicholson at dioceseofnewark.org  Mon Aug 24 18:51:19 2015
From: nnicholson at dioceseofnewark.org (Nina Nicholson)
Date: Mon, 24 Aug 2015 16:51:19 +0000
Subject: [Mailman-Users] need assistance upgrading Mailman
In-Reply-To: <55DA6D10.9070700@msapiro.net>
References: <628EB8DB28F42B4AB423051B8A25BC6201344DED5E@EDNW2008.dioceseofnewark.local>
 <55DA6D10.9070700@msapiro.net>
Message-ID: <628EB8DB28F42B4AB423051B8A25BC6201344DF486@EDNW2008.dioceseofnewark.local>

Thanks all for your quick replies.

Duly noted: Mailman lists are not Listservs. Also, not all tissues are Kleenex and not all adhesive tape is Scotch Tape -- but these copyrighted names all tend to get used as generic terms. :)

I have a dedicated virtual server with a shell login account and am allowed to update it. Unfortunately, my Unix skills are minimal and not up to attempting this on my own. I have managed other installations on my server when being walked through it, remotely, by an experienced person.

I looked at pricing for http://www.emwd.com, and at $4.50 per month per list, with 70+ lists, this would cost us in excess of $3780/year. We're a religious organization with the budget to match -- before I can get Diocesan Council to approve that type of budget increase for something we've had for free up until now, I first have to convince them I've exhausted every other option for fixing our current lists -- which means at least trying the update to see if that helps.

I'll look at the list of Mailman hosts to see if I can find something more affordable -- I would certainly love to outsource this task if I could -- but I'm still hoping to connect with someone who can assist me with the upgrade.

Many thanks,
Nina

Nina Nicholson
Director of Communications & Technology
The Episcopal Diocese of Newark
973-430-9907
http://dioceseofnewark.org
http://facebook.com/dionewark
http://twitter.com/dionewark

-----Original Message-----
From: Mailman-Users [mailto:mailman-users-bounces+nnicholson=dioceseofnewark.org at python.org] On Behalf Of Mark Sapiro
Sent: Sunday, August 23, 2015 9:02 PM
To: mailman-users at python.org
Subject: Re: [Mailman-Users] need assistance upgrading Mailman

On 08/23/2015 04:45 PM, Nina Nicholson wrote:
> I manage 70+ Mailman listservs for the Episcopal Diocese of Newark, and we are having terrible problems with emails not being delivered because they fail DMARC Evaluation, to the point that it's disrupting communications in the diocese.


Please see <http://wiki.list.org/DOC/Mailman%20is%20not%20Listserv>


> Our listservs are hosted by MediaTemple which provides Mailman ver. 2.1.9. I understand that if I were to upgrade to the latest version there are new features that would solve this problem. I've found the upgrade instructions (http://wiki.list.org/DOC/4.80%20How%20do%20I%20upgrade%20from%20Mailman%202.1.x%20to%20a%20later%20Mailman%202.1.y%3F) but they are beyond my limited Unix skills. MediaTemple refuses to do the upgrade.


Do you have a VPS with Mailman installed or are you on a shared host? If the latter, it is unlikely that you have the access necessary to upgrade Mailman on the host and it is unlikely that MediaTemple would be happy with your doing it even if you could.

You may wish to see
<http://wiki.list.org/COM/Mailman%20hosting%20services>. I have no information on most of these, but based on interactions with the proprietor of EMWD and dotList, I can recommend them. This should not be construed as a negative review of anyone else on that list. It's just that I have no basis to evaluate most of them.

You may also find
<http://wiki.list.org/COM/Mailman%20consulting%20services> of interest.
Particularly the link to the Python jobs board.

Finally, you may wish to try one of the suggestions at <http://wiki.list.org/x/17891458> as an interim stopgap measure.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan
------------------------------------------------------
Mailman-Users mailing list Mailman-Users at python.org https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: https://mail.python.org/mailman/options/mailman-users/nnicholson%40dioceseofnewark.org

From brian at emwd.com  Mon Aug 24 19:11:17 2015
From: brian at emwd.com (Brian Carpenter)
Date: Mon, 24 Aug 2015 13:11:17 -0400
Subject: [Mailman-Users] need assistance upgrading Mailman
In-Reply-To: <628EB8DB28F42B4AB423051B8A25BC6201344DF486@EDNW2008.dioceseofnewark.local>
References: <628EB8DB28F42B4AB423051B8A25BC6201344DED5E@EDNW2008.dioceseofnewark.local>
 <55DA6D10.9070700@msapiro.net>
 <628EB8DB28F42B4AB423051B8A25BC6201344DF486@EDNW2008.dioceseofnewark.local>
Message-ID: <04d101d0de8f$e418e230$ac4aa690$@emwd.com>

Hi Nina:

I actually pointed you to our Cloud VPS service where we can setup Mailman
on it and assist you with the migration of your lists. With as many lists as
you have, our VPS service is for more affordable an option.



Brian Carpenter
Owner

Providing Cloud Services and more for over 15 years.

T: 336.755.0685
E: brian at emwd.com
www.emwd.com

?



> -----Original Message-----
> From: Mailman-Users [mailto:mailman-users-
> bounces+brian=emwd.com at python.org] On Behalf Of Nina Nicholson
> Sent: Monday, August 24, 2015 12:51 PM
> To: mailman-users at python.org
> Subject: Re: [Mailman-Users] need assistance upgrading Mailman
> 
> Thanks all for your quick replies.
> 
> Duly noted: Mailman lists are not Listservs. Also, not all tissues are
Kleenex
> and not all adhesive tape is Scotch Tape -- but these copyrighted names
all
> tend to get used as generic terms. :)
> 
> I have a dedicated virtual server with a shell login account and am
allowed to
> update it. Unfortunately, my Unix skills are minimal and not up to
attempting
> this on my own. I have managed other installations on my server when being
> walked through it, remotely, by an experienced person.
> 
> I looked at pricing for http://www.emwd.com, and at $4.50 per month per
> list, with 70+ lists, this would cost us in excess of $3780/year. We're a
religious
> organization with the budget to match -- before I can get Diocesan Council
to
> approve that type of budget increase for something we've had for free up
> until now, I first have to convince them I've exhausted every other option
for
> fixing our current lists -- which means at least trying the update to see
if that
> helps.
> 
> I'll look at the list of Mailman hosts to see if I can find something more
> affordable -- I would certainly love to outsource this task if I could --
but I'm
> still hoping to connect with someone who can assist me with the upgrade.
> 
> Many thanks,
> Nina
> 
> Nina Nicholson
> Director of Communications & Technology
> The Episcopal Diocese of Newark
> 973-430-9907
> http://dioceseofnewark.org
> http://facebook.com/dionewark
> http://twitter.com/dionewark
> 
> -----Original Message-----
> From: Mailman-Users [mailto:mailman-users-
> bounces+nnicholson=dioceseofnewark.org at python.org] On Behalf Of Mark
> Sapiro
> Sent: Sunday, August 23, 2015 9:02 PM
> To: mailman-users at python.org
> Subject: Re: [Mailman-Users] need assistance upgrading Mailman
> 
> On 08/23/2015 04:45 PM, Nina Nicholson wrote:
> > I manage 70+ Mailman listservs for the Episcopal Diocese of Newark, and
> we are having terrible problems with emails not being delivered because
> they fail DMARC Evaluation, to the point that it's disrupting
communications
> in the diocese.
> 
> 
> Please see <http://wiki.list.org/DOC/Mailman%20is%20not%20Listserv>
> 
> 
> > Our listservs are hosted by MediaTemple which provides Mailman ver.
> 2.1.9. I understand that if I were to upgrade to the latest version there
are
> new features that would solve this problem. I've found the upgrade
> instructions
> (http://wiki.list.org/DOC/4.80%20How%20do%20I%20upgrade%20from%20
> Mailman%202.1.x%20to%20a%20later%20Mailman%202.1.y%3F) but they are
> beyond my limited Unix skills. MediaTemple refuses to do the upgrade.
> 
> 
> Do you have a VPS with Mailman installed or are you on a shared host? If
the
> latter, it is unlikely that you have the access necessary to upgrade
Mailman
> on the host and it is unlikely that MediaTemple would be happy with your
> doing it even if you could.
> 
> You may wish to see
> <http://wiki.list.org/COM/Mailman%20hosting%20services>. I have no
> information on most of these, but based on interactions with the
proprietor
> of EMWD and dotList, I can recommend them. This should not be construed
> as a negative review of anyone else on that list. It's just that I have no
basis
> to evaluate most of them.
> 
> You may also find
> <http://wiki.list.org/COM/Mailman%20consulting%20services> of interest.
> Particularly the link to the Python jobs board.
> 
> Finally, you may wish to try one of the suggestions at
> <http://wiki.list.org/x/17891458> as an interim stopgap measure.
> 
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy:
> http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-
> archive.com/mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-
> users/nnicholson%40dioceseofnewark.org
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/mailman-
> users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-
> users/brian%40emwd.com


From andrew at hodgsonfamily.org  Mon Aug 24 19:48:21 2015
From: andrew at hodgsonfamily.org (Andrew Hodgson)
Date: Mon, 24 Aug 2015 17:48:21 +0000
Subject: [Mailman-Users] need assistance upgrading Mailman
In-Reply-To: <628EB8DB28F42B4AB423051B8A25BC6201344DF486@EDNW2008.dioceseofnewark.local>
References: <628EB8DB28F42B4AB423051B8A25BC6201344DED5E@EDNW2008.dioceseofnewark.local>
 <55DA6D10.9070700@msapiro.net>
 <628EB8DB28F42B4AB423051B8A25BC6201344DF486@EDNW2008.dioceseofnewark.local>
Message-ID: <DB5PR04MB1478B4E71121749017EACCBFC5620@DB5PR04MB1478.eurprd04.prod.outlook.com>

Nina Nicholson wrote:

>I have a dedicated virtual server with a shell login account and am allowed to update it. Unfortunately, my Unix skills are minimal and not up to attempting this on my own. I have managed other installations on my server when >being walked through it, remotely, by an experienced person.

Looking at your host (lists.dionewark.org) seems to suggest you are using Plesk as the control panel, are you using Mailman as part of that?  If so, you will need to discuss this with Plesk, as the Mailman installation is customised as part of the control panel and is not from the stock Mailman distribution.  The CPanel control panel system which is one of Plesk's competitors ships with the latest version of Mailman, so that could be an option if you want to stick to using the control panel, but migration between the control panels may be slightly complex.

Look here:
http://wiki.list.org/DOC/Mailman%20and%20Plesk

Is that host doing anything else or is it hosting lists only?

Thanks.
Andrew. 

From nnicholson at dioceseofnewark.org  Mon Aug 24 21:44:02 2015
From: nnicholson at dioceseofnewark.org (Nina Nicholson)
Date: Mon, 24 Aug 2015 19:44:02 +0000
Subject: [Mailman-Users] need assistance upgrading Mailman
In-Reply-To: <DB5PR04MB1478B4E71121749017EACCBFC5620@DB5PR04MB1478.eurprd04.prod.outlook.com>
References: <628EB8DB28F42B4AB423051B8A25BC6201344DED5E@EDNW2008.dioceseofnewark.local>
 <55DA6D10.9070700@msapiro.net>
 <628EB8DB28F42B4AB423051B8A25BC6201344DF486@EDNW2008.dioceseofnewark.local>
 <DB5PR04MB1478B4E71121749017EACCBFC5620@DB5PR04MB1478.eurprd04.prod.outlook.com>
Message-ID: <628EB8DB28F42B4AB423051B8A25BC6201344DF821@EDNW2008.dioceseofnewark.local>

Andrew - I am using Plesk, and Mailman does interface with it. I didn't realize this is a customized installation -- good to know. That would certainly change things. Thanks for the suggestion of CPanel, I've worked with it elsewhere and can look into it.

Nina Nicholson
Director of Communications & Technology
The Episcopal Diocese of Newark
973-430-9907
http://dioceseofnewark.org
http://facebook.com/dionewark
http://twitter.com/dionewark

-----Original Message-----
From: Andrew Hodgson [mailto:andrew at hodgsonfamily.org] 
Sent: Monday, August 24, 2015 1:48 PM
To: Nina Nicholson; mailman-users at python.org
Subject: RE: [Mailman-Users] need assistance upgrading Mailman

Nina Nicholson wrote:

>I have a dedicated virtual server with a shell login account and am allowed to update it. Unfortunately, my Unix skills are minimal and not up to attempting this on my own. I have managed other installations on my server when >being walked through it, remotely, by an experienced person.

Looking at your host (lists.dionewark.org) seems to suggest you are using Plesk as the control panel, are you using Mailman as part of that?  If so, you will need to discuss this with Plesk, as the Mailman installation is customised as part of the control panel and is not from the stock Mailman distribution.  The CPanel control panel system which is one of Plesk's competitors ships with the latest version of Mailman, so that could be an option if you want to stick to using the control panel, but migration between the control panels may be slightly complex.

Look here:
http://wiki.list.org/DOC/Mailman%20and%20Plesk

Is that host doing anything else or is it hosting lists only?

Thanks.
Andrew. 

From mark at msapiro.net  Mon Aug 24 23:20:54 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 24 Aug 2015 14:20:54 -0700
Subject: [Mailman-Users] need assistance upgrading Mailman
In-Reply-To: <628EB8DB28F42B4AB423051B8A25BC6201344DF821@EDNW2008.dioceseofnewark.local>
References: <628EB8DB28F42B4AB423051B8A25BC6201344DED5E@EDNW2008.dioceseofnewark.local>
 <55DA6D10.9070700@msapiro.net>
 <628EB8DB28F42B4AB423051B8A25BC6201344DF486@EDNW2008.dioceseofnewark.local>
 <DB5PR04MB1478B4E71121749017EACCBFC5620@DB5PR04MB1478.eurprd04.prod.outlook.com>
 <628EB8DB28F42B4AB423051B8A25BC6201344DF821@EDNW2008.dioceseofnewark.local>
Message-ID: <6CEA4C73-E8BE-48C5-A11F-F5879286BDE1@msapiro.net>



On August 24, 2015 12:44:02 PM PDT, Nina Nicholson <nnicholson at dioceseofnewark.org> wrote:
>Andrew - I am using Plesk, and Mailman does interface with it.

According to the FAQ at <http://wiki.list.org/DOC/Mailman%20and%20Plesk>, updating Plesk Mailman from source is "not for the faint of heart".

-- 
Mark Sapiro <mark at msapiro.net>
Sent from my Not_an_iThing with standards compliant, open source software.

From stephen at xemacs.org  Tue Aug 25 08:32:04 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 25 Aug 2015 15:32:04 +0900
Subject: [Mailman-Users] need assistance upgrading Mailman
In-Reply-To: <6CEA4C73-E8BE-48C5-A11F-F5879286BDE1@msapiro.net>
References: <628EB8DB28F42B4AB423051B8A25BC6201344DED5E@EDNW2008.dioceseofnewark.local>
 <55DA6D10.9070700@msapiro.net>
 <628EB8DB28F42B4AB423051B8A25BC6201344DF486@EDNW2008.dioceseofnewark.local>
 <DB5PR04MB1478B4E71121749017EACCBFC5620@DB5PR04MB1478.eurprd04.prod.outlook.com>
 <628EB8DB28F42B4AB423051B8A25BC6201344DF821@EDNW2008.dioceseofnewark.local>
 <6CEA4C73-E8BE-48C5-A11F-F5879286BDE1@msapiro.net>
Message-ID: <87bndv7uaj.fsf@uwakimon.sk.tsukuba.ac.jp>

Mark Sapiro writes:

 > On August 24, 2015 12:44:02 PM PDT, Nina Nicholson
 > <nnicholson at dioceseofnewark.org> wrote:

 > >Andrew - I am using Plesk, and Mailman does interface with it.
 > 
 > According to the FAQ at
 > <http://wiki.list.org/DOC/Mailman%20and%20Plesk>, updating Plesk
 > Mailman from source is "not for the faint of heart".

IIRC, that FAQ is quite old (like 2008?)  At least it mentions Plesk 7
(currently Plesk is at 12+ it seems, with 11.x still in support).
According to this thread:

http://forum.odin.com/threads/fix-for-mailman-posts-being-rejected-due-to-yahoo-and-aol-dmarc-policy.302641/

from a year ago, at least on Linux Plesk can use the OS distro package
for Mailman.  So it *may* be as easy as "yum update mailman; mchk" as
the thread suggests.  MediaTemple (Nina's hosting service) documents
Plesk's "mchk" command here:

https://mediatemple.net/community/products/dv/204404814/how-do-i-rebuild-my-mail-configuration-files

OTOH, if MediaTemple is still on Mailman as old as 2.1.9, there may be
a pile of dependencies (most importantly, python) that are need to be
upgraded as well.  Besides dependencies like Python, the DMARC-aware
Mailman versions provide new handlers as separate modules IIRC.  Since
Plesk's APS packaging standard includes a complete manifest of package
files (path, size, SHA1), that could upset it.

Of course all the above is guesswork; it would be nice if someone with
Plesk experience could confirm.

Steve

From wetz.peter at gmail.com  Tue Aug 25 08:35:37 2015
From: wetz.peter at gmail.com (Peter Wetz)
Date: Tue, 25 Aug 2015 08:35:37 +0200
Subject: [Mailman-Users] {Disarmed} Re: Archive page not showing dates
	properly
In-Reply-To: <55DB316D.9080905@msapiro.net>
References: <CAGmyAkAB7h3YSVVm4LSpHt-7mQ8tFQD5N-Ok=3R8BQuJfVSNEQ@mail.gmail.com>
 <55D73CC9.1050303@msapiro.net>
 <CAGmyAkCGsQrt3o++3+_2t-tnhsq-Sm6A=UtH0m_EDPzi28gD+w@mail.gmail.com>
 <55DB316D.9080905@msapiro.net>
Message-ID: <CAGmyAkAWMCezhu4VjzamcTch3yjApDsiH9pN8GEi_EVjuodApg@mail.gmail.com>

>
> > okay, sorry for the spam, but I already wrapped my head around how to
> > change the date format.
> >
> > just use python's functionality to parse it into a different string.
> > for instance: 'date': datetime.datetime.strptime(article.datestr, "%a,
> > %d %b %Y %H:%M:%S %z").strftime("%d %b %Y %H:%M:%S"),
>
> Or, perhaps a bit simpler:
>
> 'date': datetime.datetime.fromtimestamp(article.date).strftime("%d %b %Y
> %H:%M:%S")
>
> or simpler still and not requiring import of datetime (HyperArch.py
> already imports time
>
> 'date': time.strftime("%d %b %Y %H:%M:%S", time.localtime(article.date))
>

hm, I get this error though:

Aug 25 07:53:42 2015 (2883) Uncaught runner exception: a float is required
Aug 25 07:53:42 2015 (2883) Traceback (most recent call last):
File "/var/lib/mailman/Mailman/Queue/Runner.py", line 120, in _oneloop
self._onefile(msg, msgdata)
File "/var/lib/mailman/Mailman/Queue/Runner.py", line 191, in _onefile
keepqueued = self._dispose(mlist, msg, msgdata)
File "/var/lib/mailman/Mailman/Queue/ArchRunner.py", line 73, in _dispose
mlist.ArchiveMail(msg)
File "/var/lib/mailman/Mailman/Archiver/Archiver.py", line 215, in
ArchiveMail
h.close()
File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 323, in close
self.update_dirty_archives()
File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 540, in
update_dirty_archives
self.update_archive(i)
File "/var/lib/mailman/Mailman/Archiver/HyperArch.py", line 1135, in
update_archive
self.__super_update_archive(archive)
File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 444, in
update_archive
self._update_simple_index(hdr, archive, arcdir)
File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 465, in
_update_simple_index
self.write_index_entry(article)
File "/var/lib/mailman/Mailman/Archiver/HyperArch.py", line 1072, in
write_index_entry
'date': time.strftime("%d %b %Y %H:%M:%S", time.localtime(article.date)),
TypeError: a float is required

"a float is required"
this implies that "article.date" is not a float in my case. maybe it needs
to be converted to a float explicitly?
any other suggestions?


> Note that the article instance has attributes
>
> #  datestr  : The posting date, in human-readable format
> #  date     : The posting date, in purely numeric format
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
>

From stephen at xemacs.org  Tue Aug 25 13:49:32 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 25 Aug 2015 20:49:32 +0900
Subject: [Mailman-Users] need assistance upgrading Mailman
In-Reply-To: <87bndv7uaj.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <628EB8DB28F42B4AB423051B8A25BC6201344DED5E@EDNW2008.dioceseofnewark.local>
 <55DA6D10.9070700@msapiro.net>
 <628EB8DB28F42B4AB423051B8A25BC6201344DF486@EDNW2008.dioceseofnewark.local>
 <DB5PR04MB1478B4E71121749017EACCBFC5620@DB5PR04MB1478.eurprd04.prod.outlook.com>
 <628EB8DB28F42B4AB423051B8A25BC6201344DF821@EDNW2008.dioceseofnewark.local>
 <6CEA4C73-E8BE-48C5-A11F-F5879286BDE1@msapiro.net>
 <87bndv7uaj.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <878u8z7flf.fsf@uwakimon.sk.tsukuba.ac.jp>

FTR, I've updated the FAQ 6.15 to refer to this thread:

 > http://forum.odin.com/threads/fix-for-mailman-posts-being-rejected-due-to-yahoo-and-aol-dmarc-policy.302641/

Steve

From dunkel at calamus.net  Tue Aug 25 16:37:25 2015
From: dunkel at calamus.net (Ulf Dunkel)
Date: Tue, 25 Aug 2015 16:37:25 +0200
Subject: [Mailman-Users] How to discard messages from just a few list
 members?
In-Reply-To: <55DB337E.5070204@msapiro.net>
References: <55D6E128.3020307@calamus.net> <55D73114.9020301@msapiro.net>
 <55D868C1.8080606@calamus.net> <55D87333.3060501@Damon-Family.org>
 <55D894BA.4020607@calamus.net> <87zj1i7tdb.fsf@uwakimon.sk.tsukuba.ac.jp>
 <55DAC66E.1050000@calamus.net> <55DB337E.5070204@msapiro.net>
Message-ID: <55DC7DA5.9070109@calamus.net>

Thank you for your explanations, Mark!

(Mission accomplished)

---Ulf Dunkel

From mailman at veggiechinese.net  Tue Aug 25 19:12:10 2015
From: mailman at veggiechinese.net (Will Yardley)
Date: Tue, 25 Aug 2015 10:12:10 -0700
Subject: [Mailman-Users] list subscription spammers
Message-ID: <20150825171210.GM60769@aura.veggiechinese.net>

I'm seeing massive numbers of subscription lists to various lists we
host (including multiple requests to the same list).

These are submitted via a distributed network of hosts, presumably
botnet victims / open proxies.

The requests are from
foo+[0-9]{9}@gmail.com

e.g.,

foo+55216394@

where foo is one of several dropboxes.

Obviously, even after being banned, they're still getting a 200 response
code, so they keep spamming away, but what's their goal here? Trying to
get access to a list so that they can get archives / memberlists (or
post to them)? Or just trying to create a nuissance? If the former, why
do they submit so many requests for each list?

If anyone has seen similar behavior and has more info, please feel free
to share on or off list.

w


From mailman at veggiechinese.net  Tue Aug 25 20:25:10 2015
From: mailman at veggiechinese.net (Will Yardley)
Date: Tue, 25 Aug 2015 11:25:10 -0700
Subject: [Mailman-Users] list subscription spammers
In-Reply-To: <20150825171210.GM60769@aura.veggiechinese.net>
References: <20150825171210.GM60769@aura.veggiechinese.net>
Message-ID: <20150825182510.GP60769@aura.veggiechinese.net>

And, self-followup, I had looked on list.org before, but hadn't checked
the archives, so seems clear that the behavior is common. Running the
RHEL5 Mailman currently (2.1.9) so still don't have the
SUBSCRIBE_FORM_SECRET feature yet.

Mostly curious about the motivation behind this - what do these bots do
if / when they are actually able to confirm or get confirmed?

w


From mailman at veggiechinese.net  Tue Aug 25 20:40:24 2015
From: mailman at veggiechinese.net (Will Yardley)
Date: Tue, 25 Aug 2015 11:40:24 -0700
Subject: [Mailman-Users] need assistance upgrading Mailman
In-Reply-To: <87bndv7uaj.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <628EB8DB28F42B4AB423051B8A25BC6201344DED5E@EDNW2008.dioceseofnewark.local>
 <55DA6D10.9070700@msapiro.net>
 <628EB8DB28F42B4AB423051B8A25BC6201344DF486@EDNW2008.dioceseofnewark.local>
 <DB5PR04MB1478B4E71121749017EACCBFC5620@DB5PR04MB1478.eurprd04.prod.outlook.com>
 <628EB8DB28F42B4AB423051B8A25BC6201344DF821@EDNW2008.dioceseofnewark.local>
 <6CEA4C73-E8BE-48C5-A11F-F5879286BDE1@msapiro.net>
 <87bndv7uaj.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20150825184024.GQ60769@aura.veggiechinese.net>

On Tue, Aug 25, 2015 at 03:32:04PM +0900, Stephen J. Turnbull wrote:
> Mark Sapiro writes:
>  > On August 24, 2015 12:44:02 PM PDT, Nina Nicholson
>  > <nnicholson at dioceseofnewark.org> wrote:

>  > >Andrew - I am using Plesk, and Mailman does interface with it.
>  > 
>  > According to the FAQ at
>  > <http://wiki.list.org/DOC/Mailman%20and%20Plesk>, updating Plesk
>  > Mailman from source is "not for the faint of heart".
> 
> from a year ago, at least on Linux Plesk can use the OS distro package
> for Mailman.  So it *may* be as easy as "yum update mailman; mchk" as
> the thread suggests.  MediaTemple (Nina's hosting service) documents
> Plesk's "mchk" command here:

FWIW, if it's RHEL 5 or 6, I got them to backport the DMARC fixes. So,
if it's 2.1.9 because it's the RH version, you might be in luck.

https://bugzilla.redhat.com/show_bug.cgi?id=1095359
EL6:
https://rhn.redhat.com/errata/RHSA-2015-1417.html
EL5:
https://rhn.redhat.com/errata/RHBA-2015-0706.html

w


From skip.montanaro at gmail.com  Tue Aug 25 21:58:11 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 25 Aug 2015 14:58:11 -0500
Subject: [Mailman-Users] list subscription spammers
In-Reply-To: <20150825182510.GP60769@aura.veggiechinese.net>
References: <20150825171210.GM60769@aura.veggiechinese.net>
 <20150825182510.GP60769@aura.veggiechinese.net>
Message-ID: <CANc-5Ux3-H+Vc2=2u60gCYaNcFSKwiKb7HM0mWQgchP_Xf3J_A@mail.gmail.com>

On Tue, Aug 25, 2015 at 1:25 PM, Will Yardley <mailman at veggiechinese.net>
wrote:

> Mostly curious about the motivation behind this - what do these bots do
> if / when they are actually able to confirm or get confirmed?
>

They probably try to read the subscription list to get more email
addresses. Of course, they can also post spam to the lists. Perhaps both,
but I suspect the email addresses are of more value.

Skip

From billycrook at gmail.com  Wed Aug 26 00:59:59 2015
From: billycrook at gmail.com (Billy Crook)
Date: Tue, 25 Aug 2015 17:59:59 -0500
Subject: [Mailman-Users] Restricting access to the pipermail archives
In-Reply-To: <55D61F84.1070407@msapiro.net>
References: <CANZ96zVQe7UX5z0-RG_-Jt=2k87kkQNhXRmaoJvcifMvu7GMLw@mail.gmail.com>
 <55D5FB8B.7090007@msapiro.net> <20150820161505.GE9573@hendricks.amyl.org.uk>
 <CANZ96zV4Bp8fVmiXCt+sYSeD-39ceC-5-87xGjWSr6368Jh+2w@mail.gmail.com>
 <55D61F84.1070407@msapiro.net>
Message-ID: <CANZ96zWnWe45t0SytqDbOp584R-68vE7eXP3cpr6argiVZ05Dg@mail.gmail.com>

Awesome!  Thanks Mark!  That worked exactly as you described!

Any reason I should keep pipermail's config in apache around if I don't
want the list archives viewable anonymously?

On Thu, Aug 20, 2015 at 1:42 PM, Mark Sapiro <mark at msapiro.net> wrote:

> On 08/20/2015 11:26 AM, Billy Crook wrote:
> > It is not clear to me that archive_private controls access to the
> archive.
>
>
> It does. Try it.  If archive_private is 'public' there is a symlink from
> archives/public/listname to archives/private/listname and a URL like
> http://example.com/pipermail/listname will work without authentication.
> If you set archive_private to 'private', that symlink is removed, the
> pipermail URL won't work and the only access to the archive is via a URL
> like http://example.com/mailman/private/listname which requires
> authentication and which will be displayed the archive URL on the
> listinfo page and in List-Archive: headers.
>
>
> > My understanding is that archive_private is used to inform mailman that
> the
> > email addresses visible in the archive should be presented differently
> > depending if the archive will be made publicly accessible or not.  i.e.
> > munge the email addresses if there's a chance spambots can find them.
>
>
> That is the Defaults.py/mm_cfg.py setting ARCHIVER_OBSCURES_EMAIL_ADDRESS.
>
>
> > I would like to NOT munge the addresses, so they are useful to click on
> for
> > my users.  But I want all of the text of each archived message to be kept
> > in secret on the server, viewable only by members of the list.
>
>
> Set
>
> ARCHIVER_OBSCURES_EMAIL_ADDRESS = No
>
> in mm_cfg.py and set the list's archive_private to 'private.
>
>
> > I would hope there was a more "supported" way to do this,...
>
>
> Just do the above. That will do it.
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/billycrook%40gmail.com
>

From stephen at xemacs.org  Wed Aug 26 04:28:33 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 26 Aug 2015 11:28:33 +0900
Subject: [Mailman-Users] Restricting access to the pipermail archives
In-Reply-To: <CANZ96zWnWe45t0SytqDbOp584R-68vE7eXP3cpr6argiVZ05Dg@mail.gmail.com>
References: <CANZ96zVQe7UX5z0-RG_-Jt=2k87kkQNhXRmaoJvcifMvu7GMLw@mail.gmail.com>
 <55D5FB8B.7090007@msapiro.net>
 <20150820161505.GE9573@hendricks.amyl.org.uk>
 <CANZ96zV4Bp8fVmiXCt+sYSeD-39ceC-5-87xGjWSr6368Jh+2w@mail.gmail.com>
 <55D61F84.1070407@msapiro.net>
 <CANZ96zWnWe45t0SytqDbOp584R-68vE7eXP3cpr6argiVZ05Dg@mail.gmail.com>
Message-ID: <871teq7pgu.fsf@uwakimon.sk.tsukuba.ac.jp>

Billy Crook writes:

 > Any reason I should keep pipermail's config in apache around if I don't
 > want the list archives viewable anonymously?

Mailman doesn't serve HTTP, Apache does.  The Apache config is what
makes them viewable on the web.  If you expect them to be viewable to
subscribers, you need the config.  If you don't want *anybody* viewing
them, no need for the config.  (I wonder why you keep archives in
Mailman at all in that case, though.)


From billycrook at gmail.com  Wed Aug 26 23:42:20 2015
From: billycrook at gmail.com (Billy Crook)
Date: Wed, 26 Aug 2015 16:42:20 -0500
Subject: [Mailman-Users] Restricting access to the pipermail archives
In-Reply-To: <871teq7pgu.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CANZ96zVQe7UX5z0-RG_-Jt=2k87kkQNhXRmaoJvcifMvu7GMLw@mail.gmail.com>
 <55D5FB8B.7090007@msapiro.net> <20150820161505.GE9573@hendricks.amyl.org.uk>
 <CANZ96zV4Bp8fVmiXCt+sYSeD-39ceC-5-87xGjWSr6368Jh+2w@mail.gmail.com>
 <55D61F84.1070407@msapiro.net>
 <CANZ96zWnWe45t0SytqDbOp584R-68vE7eXP3cpr6argiVZ05Dg@mail.gmail.com>
 <871teq7pgu.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CANZ96zX=awPd=WHf2bPYRjEZJZSKzeFmQ3wOv81hENdgGs0W+w@mail.gmail.com>

On Tue, Aug 25, 2015 at 9:28 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
>
> Billy Crook writes:
>
>  > Any reason I should keep pipermail's config in apache around if I don't
>  > want the list archives viewable anonymously?
>
> Mailman doesn't serve HTTP, Apache does.  The Apache config is what
> makes them viewable on the web.  If you expect them to be viewable to
> subscribers, you need the config.  If you don't want *anybody* viewing
> them, no need for the config.  (I wonder why you keep archives in
> Mailman at all in that case, though.)


There were two parts that I see in mailman's apache config:
        ScriptAlias /mailman/       /usr/lib/mailman/cgi-bin/
        <Directory /usr/lib/mailman/cgi-bin/>
                AllowOverride None
                Options ExecCGI
                Order allow,deny
                Allow from all
        </Directory>

The part above, I expect to need to keep.
The part below here, is what I think I can just remove since I don't
want public archives provided via pipermail.  (Just private archives
via the mailman path.)

        Alias /pipermail/ /var/lib/mailman/archives/public/
        <Directory /var/lib/mailman/archives/public>
                Options Indexes MultiViews FollowSymLinks
                AllowOverride None
                Order allow,deny
                Allow from all
                AddDefaultCharset Off
        </Directory>

From stephen at xemacs.org  Thu Aug 27 03:19:58 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 27 Aug 2015 10:19:58 +0900
Subject: [Mailman-Users] Restricting access to the pipermail archives
In-Reply-To: <CANZ96zX=awPd=WHf2bPYRjEZJZSKzeFmQ3wOv81hENdgGs0W+w@mail.gmail.com>
References: <CANZ96zVQe7UX5z0-RG_-Jt=2k87kkQNhXRmaoJvcifMvu7GMLw@mail.gmail.com>
 <55D5FB8B.7090007@msapiro.net>
 <20150820161505.GE9573@hendricks.amyl.org.uk>
 <CANZ96zV4Bp8fVmiXCt+sYSeD-39ceC-5-87xGjWSr6368Jh+2w@mail.gmail.com>
 <55D61F84.1070407@msapiro.net>
 <CANZ96zWnWe45t0SytqDbOp584R-68vE7eXP3cpr6argiVZ05Dg@mail.gmail.com>
 <871teq7pgu.fsf@uwakimon.sk.tsukuba.ac.jp>
 <CANZ96zX=awPd=WHf2bPYRjEZJZSKzeFmQ3wOv81hENdgGs0W+w@mail.gmail.com>
Message-ID: <87h9nl5xz5.fsf@uwakimon.sk.tsukuba.ac.jp>

Billy Crook writes:

 > The part below here, is what I think I can just remove since I don't
 > want public archives provided via pipermail.  (Just private archives
 > via the mailman path.)
 > 
 >         Alias /pipermail/ /var/lib/mailman/archives/public/

Ah, OK, yes, you can remove that part.  The confusion is because we
generally refer to the bundled archiver as "pipermail".

Steve

From mailman at veggiechinese.net  Thu Aug 27 04:14:35 2015
From: mailman at veggiechinese.net (Will Yardley)
Date: Wed, 26 Aug 2015 19:14:35 -0700
Subject: [Mailman-Users] list subscription spammers
In-Reply-To: <20150825171210.GM60769@aura.veggiechinese.net>
References: <20150825171210.GM60769@aura.veggiechinese.net>
Message-ID: <20150827021435.GA79882@aura.veggiechinese.net>

ps - Here are a couple of quick throwaway 'withlist' scripts I threw
together to add a regexp matching the spammers' email addresses, and to
purge pending subscription requests matching a similar regexp. Posting
here in case someone finds them useful for this, or some other, purpose.

% cat purge_requests.py 
# quick "withlist" script.
# call via withlist -r purge_requests LISTNAME
# or via loop:
# for LIST in `./list_lists -b` ; do ./withlist -r purge_requests $LIST ; done

import re
from Mailman import mm_cfg
from Mailman.mm_cfg import DISCARD

email_re = re.compile('^(foo|bar.baz|blah)\+[0-9]+ at example\.com')

def purge_requests(m):
    print "** Checking %s" % m.internal_name()
    m.Lock()
    for item in m.GetSubscriptionIds():
        email_addr = m.GetRecord(item)[1]
        if email_re.match(email_addr):
            print ">> Discarding %s from %s" % (email_addr,
                                                m.internal_name())
            m.HandleRequest(item, DISCARD)
    m.Save()
    m.Unlock()

% cat set_banlist.py 
# quick "withlist" script.
# call via withlist -r set_banlist LISTNAME
# or via loop:
# for LIST in `./list_lists -b` ; do ./withlist -r set_banlist $LIST ; done

# Obviously, the problem becomes a bit more complex if we want to adjust
# the existing rule, vs. adding a new one; then we'd have to track the
# item # within the list, and modify that item (this also shouldn't be
# too hard).

from Mailman import mm_cfg

# This will append the item below to the ban_list for $LISTNAME
# Note additional pattern below; that's just a check to see if the
# pattern already exists.
ban_item = '^(foo|bar.baz|blah)\\+[0-9]+ at example\.com'

def set_banlist(m):
    # Rather than use regex, use this to see if substring is in list
    if not rule_exists(m.ban_list):
	print "**** Updating %s" % m.internal_name()
        m.Lock()
        m.ban_list.append(ban_item)
        m.Save()


# Could probably do this a little more tersely with 'lambda'
def rule_exists(list):
    for item in list:
        if '\(foo' in item:
            return 1


From billycrook at gmail.com  Thu Aug 27 19:49:24 2015
From: billycrook at gmail.com (Billy Crook)
Date: Thu, 27 Aug 2015 12:49:24 -0500
Subject: [Mailman-Users] Footer Bloat
Message-ID: <CANZ96zUruNB54Q2ED4_YeZux-gDH3-O=abMfX3h-k0aB+kDoqg@mail.gmail.com>

I've noticed that as people reply to threads on my mailman instance, I see
copies of the footer pile up at the bottom of each message.

Rants about top/inline/bottom posting aside, I thought it would be a smart
feature of mailman to look for a copy of the footer in any incoming message
before adding the footer to the bottom.

Or perhaps easier to implement, instead of "apply a footer" or "don't apply
a footer", add the option "apply footer to the first message of each thread
and no other messages of the same thread"

Am I just missing where this setting is, or has this not been thought of
yet.

"apply footer only if footer text not already in message" would be my logic
of preference.

If that's not already something mailman can do, where would I modify the
code to add it?

From seun.ojedeji at gmail.com  Thu Aug 27 20:02:43 2015
From: seun.ojedeji at gmail.com (Seun Ojedeji)
Date: Thu, 27 Aug 2015 19:02:43 +0100
Subject: [Mailman-Users] Footer Bloat
In-Reply-To: <CANZ96zUruNB54Q2ED4_YeZux-gDH3-O=abMfX3h-k0aB+kDoqg@mail.gmail.com>
References: <CANZ96zUruNB54Q2ED4_YeZux-gDH3-O=abMfX3h-k0aB+kDoqg@mail.gmail.com>
Message-ID: <CAD_dc6jYq17jrHaTRMFSRkY7=5gO41X06Uy7=yDJvMzDcgKF9A@mail.gmail.com>

Hi,

I think it will be a good feature to have. Thinking about it's
implementation however assumes that every mail clients uses the thread view
mail option. Also those who subscribe in the middle of a conversion will
most likely not get the footer.

Nerveless it's good to have it for use according to our environment.

Cheers!

Sent from my Asus Zenfone2
Kindly excuse brevity and typos.
On 27 Aug 2015 18:50, "Billy Crook" <billycrook at gmail.com> wrote:

> I've noticed that as people reply to threads on my mailman instance, I see
> copies of the footer pile up at the bottom of each message.
>
> Rants about top/inline/bottom posting aside, I thought it would be a smart
> feature of mailman to look for a copy of the footer in any incoming message
> before adding the footer to the bottom.
>
> Or perhaps easier to implement, instead of "apply a footer" or "don't apply
> a footer", add the option "apply footer to the first message of each thread
> and no other messages of the same thread"
>
> Am I just missing where this setting is, or has this not been thought of
> yet.
>
> "apply footer only if footer text not already in message" would be my logic
> of preference.
>
> If that's not already something mailman can do, where would I modify the
> code to add it?
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/seun.ojedeji%40gmail.com
>

From billycrook at gmail.com  Thu Aug 27 20:21:49 2015
From: billycrook at gmail.com (Billy Crook)
Date: Thu, 27 Aug 2015 13:21:49 -0500
Subject: [Mailman-Users] Footer Bloat
In-Reply-To: <CAD_dc6jYq17jrHaTRMFSRkY7=5gO41X06Uy7=yDJvMzDcgKF9A@mail.gmail.com>
References: <CANZ96zUruNB54Q2ED4_YeZux-gDH3-O=abMfX3h-k0aB+kDoqg@mail.gmail.com>
 <CAD_dc6jYq17jrHaTRMFSRkY7=5gO41X06Uy7=yDJvMzDcgKF9A@mail.gmail.com>
Message-ID: <CANZ96zV1U=gcep49AygKmw=6NK=Hc7KGOo=pC_Pqejk8tumveQ@mail.gmail.com>

Where the current behavior is 'always append footer'. I think these options
are worth implementing:
'Append footer for just the first message'
'Append footer if footer not already present in quoted text'

On Thu, Aug 27, 2015 at 1:02 PM, Seun Ojedeji <seun.ojedeji at gmail.com>
wrote:

> Hi,
>
> I think it will be a good feature to have. Thinking about it's
> implementation however assumes that every mail clients uses the thread view
> mail option. Also those who subscribe in the middle of a conversion will
> most likely not get the footer.
>
> Nerveless it's good to have it for use according to our environment.
>
> Cheers!
>
> Sent from my Asus Zenfone2
> Kindly excuse brevity and typos.
> On 27 Aug 2015 18:50, "Billy Crook" <billycrook at gmail.com> wrote:
>
>> I've noticed that as people reply to threads on my mailman instance, I see
>> copies of the footer pile up at the bottom of each message.
>>
>> Rants about top/inline/bottom posting aside, I thought it would be a smart
>> feature of mailman to look for a copy of the footer in any incoming
>> message
>> before adding the footer to the bottom.
>>
>> Or perhaps easier to implement, instead of "apply a footer" or "don't
>> apply
>> a footer", add the option "apply footer to the first message of each
>> thread
>> and no other messages of the same thread"
>>
>> Am I just missing where this setting is, or has this not been thought of
>> yet.
>>
>> "apply footer only if footer text not already in message" would be my
>> logic
>> of preference.
>>
>> If that's not already something mailman can do, where would I modify the
>> code to add it?
>> ------------------------------------------------------
>> Mailman-Users mailing list Mailman-Users at python.org
>> https://mail.python.org/mailman/listinfo/mailman-users
>> Mailman FAQ: http://wiki.list.org/x/AgA3
>> Security Policy: http://wiki.list.org/x/QIA9
>> Searchable Archives:
>> http://www.mail-archive.com/mailman-users%40python.org/
>> Unsubscribe:
>> https://mail.python.org/mailman/options/mailman-users/seun.ojedeji%40gmail.com
>>
>

From lac at openend.se  Thu Aug 27 20:23:56 2015
From: lac at openend.se (Laura Creighton)
Date: Thu, 27 Aug 2015 20:23:56 +0200
Subject: [Mailman-Users] Footer Bloat
In-Reply-To: <CANZ96zUruNB54Q2ED4_YeZux-gDH3-O=abMfX3h-k0aB+kDoqg@mail.gmail.com>
References: <CANZ96zUruNB54Q2ED4_YeZux-gDH3-O=abMfX3h-k0aB+kDoqg@mail.gmail.com>
Message-ID: <201508271823.t7RINujE008297@theraft.openend.se>

You may be interested in this thread:
https://mail.python.org/pipermail/mailman-users/2012-April/073181.html

Laura


From billynoah at gmail.com  Wed Aug 26 01:28:41 2015
From: billynoah at gmail.com (billy noah)
Date: Tue, 25 Aug 2015 19:28:41 -0400
Subject: [Mailman-Users] Error updating mailman Ubuntu 14.04.3
Message-ID: <CADbGg-PnKP0muXB-G=39js35QVrZDOHt1TSGG1O+OH425J5NKw@mail.gmail.com>

Tried running an update today and received this message:

Setting up mailman (1:2.1.16-2ubuntu0.1) ...
Looking for enabled languages (this may take some time) ... done.
Removing unmodified files from /etc/mailman/eu  done.
Directory /etc/mailman/eu not empty, not removed.
*... {repeat above message for each language} ...*
Removing unmodified files from /etc/mailman/sv  done.
Directory /etc/mailman/sv not empty, not removed.
Removing unmodified files from /etc/mailman/ast  done.
Directory /etc/mailman/ast not empty, not removed.
Installing site language en ............................................
done.
Downgrade detected, from version 0x20112f1 to version 0x20110f0
This is probably not safe.
Exiting.
dpkg: error processing package mailman (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 mailman
E: Sub-process /usr/bin/dpkg returned an error code (1)

Any ideas what I can do to remedy this?

From mark at incet.com  Thu Aug 27 10:58:21 2015
From: mark at incet.com (Mark)
Date: Thu, 27 Aug 2015 20:58:21 +1200
Subject: [Mailman-Users] Assistance with altering reply-to behaviours and
	DMARC
Message-ID: <55DED12D.3000300@incet.com>

Hi,

First, apologies if this has been discussed before.  I run a number of mailman lists on a Centos 6 platform and mailman 2.1.12-25. This version was updated in July as follows: 
https://rhn.redhat.com/errata/RHSA-2015-1417.html and included fixes for a number of DMARC issues.  Including "... With this update, domains with a "reject" DMARC policy are 
recognized correctly.."

Normally the lists we use are configured with reply_goes_to_list and anonymous_list is off.  We have done some testing, and we have not been able to identify settings that will 
retain reply_goes_to_list functionality  and turn on DMARC Mung.

Generally what happens is that the first message is successfully sent to a yahoo.com address and this user can reply to the message as normal.  However, mailman adds the original 
sender to the reply-to meaning that when another user replies to this message reply_goes_to_list doesn't work.

It seemed to me that a small modification to CookHeaders.py could prevent the adding of the original sender email to "Reply-To"

         # We also need to put the old From: in Reply-To: in all cases.
         if o_from:
             add(o_from)
         # Set Reply-To: header to point back to this list.  Add this last

seemed like a good place to look, but before I try to debug commenting out this didn't prevent the addition of the original sender to Reply-To I would ask for some advice !!

any assistance would be very helpful!!   (it maybe that changing the order of addresses in the Reply-To field would all reply_goes_to_list would work?)

thanks
Mark.




From mark at incet.com  Thu Aug 27 22:42:09 2015
From: mark at incet.com (Mark)
Date: Fri, 28 Aug 2015 08:42:09 +1200
Subject: [Mailman-Users] Assistance with altering reply-to behaviours and
	DMARC
Message-ID: <55DF7621.3000008@incet.com>

Hi,

First, apologies if this has been discussed before.  I run a number of mailman lists on a Centos 6 platform and mailman 2.1.12-25. This version was updated in July as follows: 
https://rhn.redhat.com/errata/RHSA-2015-1417.html and included fixes for a number of DMARC issues.  Including "... With this update, domains with a "reject" DMARC policy are 
recognized correctly.."

Normally the lists we use are configured with reply_goes_to_list and anonymous_list is off.  We have done some testing, and we have not been able to identify settings that will 
retain reply_goes_to_list functionality  and turn on DMARC Mung.

Generally what happens is that the first message is successfully sent to a yahoo.com address and this user can reply to the message as normal.  However, mailman adds the original 
sender to the reply-to meaning that when another user replies to this message reply_goes_to_list doesn't work.

It seemed to me that a small modification to CookHeaders.py could prevent the adding of the original sender email to "Reply-To"

         # We also need to put the old From: in Reply-To: in all cases.
         if o_from:
             add(o_from)
         # Set Reply-To: header to point back to this list.  Add this last

seemed like a good place to look, but before I try to debug commenting out this didn't prevent the addition of the original sender to Reply-To I would ask for some advice !!

any assistance would be very helpful!!   (it maybe that changing the order of addresses in the Reply-To field would all reply_goes_to_list would work?)

thanks
Mark.




From mediamon2003 at yahoo.com  Thu Aug 27 10:37:05 2015
From: mediamon2003 at yahoo.com (Woody Mon)
Date: Thu, 27 Aug 2015 01:37:05 -0700
Subject: [Mailman-Users] ISO RegExp example to block bot subscribe requests
Message-ID: <1440664625.11103.YahooMailBasic@web125203.mail.ne1.yahoo.com>

Hello, List is running on Mailman version 2.1.20.

Last several days began receiving hundreds of bot subscription requests (all gmail addresses).

All of format like:
blech+12345678 at gmail.com
zark+98765432 at gmail.com
etc.

I entered following regex lines in Privacy Options -> Subscription Rules -> ban_list

^.*blech
^.*zark
etc

Temporarily did the job, but the bot adapted and changed the prefix of the address string to:
crap+12345678 at gmail.com
grum+98765432 at gmail.com
etc.

Unable to keep up with the bots changing the address prefix.

Common to ALL of the bot subscribe requests is including the "+" symbol in the email address

So I tried entering RegExp ^.*+ but Mailman rejected this entry.

But was able to enter RegExp of ^.*/+
However that allowed all the bot spam requests through.

Can anyone recommend an example of a RegExp entry which would solve my dilemma?

Note: Unfortunately I do not have access to the server so unable to configure SUBSCRIBE_FORM_SECRET 

Thanks for any ideas.


From nelsonkelly446 at gmail.com  Thu Aug 27 11:49:39 2015
From: nelsonkelly446 at gmail.com (Nelson Kelly)
Date: Thu, 27 Aug 2015 02:49:39 -0700
Subject: [Mailman-Users] ISO speciific RegExp to filter/discard bot
	subscribe requests
Message-ID: <55DEDD33.5020207@gmail.com>

Hello, We have a list running on Mailman version 2.1.20.

Over the last several days began receiving hundreds of bot subscription 
requests (all gmail addresses).

All of format like:
blech+12345678 at gmail.com
zark+98765432 at gmail.com

In attempt to solve this issue, entered the following regex lines in 
Privacy Options -> Subscription Rules -> ban_list

^.*blech
^.*zark

Temporarily did the job, but the bot(s) adapted and changed the prefix 
of the address string to:

crap+12345678 at gmail.com
grum+98765432 at gmail.com

Common to ALL of the bot subscribe requests is including within the the 
email address a "+" symbol.

So I tried entering RegExp ^.*+ but Mailman rejected this entry.

But was able to enter RegExp of ^.*/+
However that allowed all the bot spam requests through.

Unable to keep up with the bots changing the address prefix.

Can anyone recommend an example of a RegExp entry which would 
prevent/solve this dilemma?

Note: I've searched the Mailman-Users list archives and noted Mr. 
Yardley's recent posts, but not able to implement his solution as I 
don't have access the server. (so also unable to configure 
SUBSCRIBE_FORM_SECRET solution).

Thank you for any ideas.



From Ralf+MailMan at bergs.biz  Tue Aug 25 13:29:54 2015
From: Ralf+MailMan at bergs.biz (Ralf G. R. Bergs)
Date: Tue, 25 Aug 2015 13:29:54 +0200
Subject: [Mailman-Users] How to prevent messages from being "held"
Message-ID: <55DC51B2.4050004@bergs.biz>

Hi guys.

I'm struggling since a while since MailMan will sometimes put messages
into the "Held" queue even though I did my very best to set all config
options so that MailMan won't... :-(

Is there a way to once and for all forbid MailMan to put messages into
the queue so that a moderator has to deal with it?

We use MailMan as a team mailing list, and it should never happen that a
message is sitting in the queue, waiting to be released...

Thanks a lot in advance for your help.

Kind regards,

Ralf


From Ralf+MailMan at bergs.biz  Thu Aug 27 20:10:26 2015
From: Ralf+MailMan at bergs.biz (Ralf G. R. Bergs)
Date: Thu, 27 Aug 2015 20:10:26 +0200
Subject: [Mailman-Users] How to prevent messages from being "held"
In-Reply-To: <55DC51B2.4050004@bergs.biz>
References: <55DC51B2.4050004@bergs.biz>
Message-ID: <55DF5292.90402@bergs.biz>

Hi guys.

I'm reposting this as I haven't received any feedback -- can anyone help
me, please?

Kind regards,

Ralf

On 2015-08-25 13:29 , Ralf G. R. Bergs wrote:
> Hi guys.
>
> I'm struggling since a while since MailMan will sometimes put messages
> into the "Held" queue even though I did my very best to set all config
> options so that MailMan won't... :-(
>
> Is there a way to once and for all forbid MailMan to put messages into
> the queue so that a moderator has to deal with it?
>
> We use MailMan as a team mailing list, and it should never happen that a
> message is sitting in the queue, waiting to be released...
>
> Thanks a lot in advance for your help.
>
> Kind regards,
>
> Ralf
>



From waldbiec at lafayette.edu  Wed Aug 26 22:08:28 2015
From: waldbiec at lafayette.edu (Waldbieser, Carl)
Date: Wed, 26 Aug 2015 16:08:28 -0400 (EDT)
Subject: [Mailman-Users] Pluggable authentication for Mailman web interface?
Message-ID: <854480980.1021293.1440619708372.JavaMail.zimbra@lafayette.edu>


Are there any guidelines for adding authentication and /or authorization mechanisms to the Mailman web user interface?
Specifically, I was wondering if there is any kind of guidance for authenticating the user via an HTTP header (e.g. HTTP_REMOTE_USER) so that an authenticating reverse proxy could be placed in front of the Mailman web interface.

If there is no such built-in mechanism or pluggable mechanism, is there any kind of guidance on how the existing authentication mechanism might be replaced from a technical standpoint?

Thanks,
Carl Waldbieser
ITS Systems Programmer
Lafayette College

From pshute at nuw.org.au  Fri Aug 28 03:22:19 2015
From: pshute at nuw.org.au (Peter Shute)
Date: Fri, 28 Aug 2015 11:22:19 +1000
Subject: [Mailman-Users] Does Mailman alter messages attached to moderation
 notification emails?
Message-ID: <C2F1879073AE7347BAB44065F143E17002AA5D94ED@nuwvicms2>

I am again trying to answer a query from a list member about spelling errors in his postings that show up in his list messages, but not in his copy of the sent message.

Initially we thought this was because he was composing in html, and that his mail client wasn't including an accurate representation of the message in the plain text section. Our list sends out only the plain text section.

In at least one case we proved that the spelling error (a missing period in a URL) was only in the plain text section, so I advised him to compose in plain text.

Now it's happened again, this time in a plain text only message. The period is there in his copy of the sent message, missing in what went to the list. I can also see that it's missing in the copy of the message attached to the moderation notification.

Does the list send out moderation notifications with the attached messages unaltered? Is there any mechanism in mailman that could have removed the period?

It also removed one from a URL sent to the list owner address, but again that message has passed through Mailman. I'm not able to easily get a sample of a message from him that hasn't been through Mailman, so I can't eliminate it as a suspect.

Looking at the raw messages, something that's common about them is that the missing period would fall exactly on the start of a line when it's wrapped, i.e after the = symbol which I assume is a soft eol, and in one of them the period is there in the plain text section, but missing in the html section.

Peter Shute

From stephen at xemacs.org  Fri Aug 28 05:25:11 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 28 Aug 2015 12:25:11 +0900
Subject: [Mailman-Users]  Error updating mailman Ubuntu 14.04.3
In-Reply-To: <CADbGg-PnKP0muXB-G=39js35QVrZDOHt1TSGG1O+OH425J5NKw@mail.gmail.com>
References: <CADbGg-PnKP0muXB-G=39js35QVrZDOHt1TSGG1O+OH425J5NKw@mail.gmail.com>
Message-ID: <87d1y85c2w.fsf@uwakimon.sk.tsukuba.ac.jp>

billy noah writes:

 > Downgrade detected, from version 0x20112f1 to version 0x20110f0
 > This is probably not safe.

The basic answer is "You're going to need help from Ubuntu experts
anyway because the package database is out of sync with reality, so go
ask them."

That said, there are three possibilities.

1. Ubuntu screwed up.  Ubuntu will fix it.  If you're in a big hurry,
   ask in Ubuntu forums how to work around until they do.

2. You installed a more recent version of Mailman yourself

   a.  From a PPA.  Talk to Ubuntu about how to manage these.

   b.  From source.  Tell us what you did.  We can help you get
       Mailman running, but your package DB is still broken, you need
       to talk to Ubuntu about how to fix that.

3. Something else.  Tell us about it.  (The package DB is still
   broken, Ubuntu forums are probably the right place to learn how to
   fix that.)

 > Exiting.
 > dpkg: error processing package mailman (--configure):

You should be able to roll back the update at this point.  Check the
docs for apt or whatever front-end you're using.

Then, until you get things figured out, you could pin Mailman to the
current version (that Ubuntu thinks is installed) and run the rest of
any pending updates.


From stephen at xemacs.org  Fri Aug 28 05:32:25 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 28 Aug 2015 12:32:25 +0900
Subject: [Mailman-Users] ISO RegExp example to block bot subscribe requests
In-Reply-To: <1440664625.11103.YahooMailBasic@web125203.mail.ne1.yahoo.com>
References: <1440664625.11103.YahooMailBasic@web125203.mail.ne1.yahoo.com>
Message-ID: <87bnds5bqu.fsf@uwakimon.sk.tsukuba.ac.jp>

Woody Mon via Mailman-Users writes:

 > So I tried entering RegExp ^.*+ but Mailman rejected this entry.

That's illegal syntax for a regexp because "+" is an operator.

The version you want is "^.*\+".  However, that is a very bad idea if
you have any posters with email skills, because embedding "+" in the
return address for outgoing mail is a common way to identify the
source of incoming replies.

A more conservative regexp would be "^[a-z]+\+[0-9]+@" which will
catch the pattern of LETTERS "+" DIGITS.


From stephen at xemacs.org  Fri Aug 28 06:07:07 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 28 Aug 2015 13:07:07 +0900
Subject: [Mailman-Users]  Footer Bloat
In-Reply-To: <CANZ96zUruNB54Q2ED4_YeZux-gDH3-O=abMfX3h-k0aB+kDoqg@mail.gmail.com>
References: <CANZ96zUruNB54Q2ED4_YeZux-gDH3-O=abMfX3h-k0aB+kDoqg@mail.gmail.com>
Message-ID: <87a8tc5a50.fsf@uwakimon.sk.tsukuba.ac.jp>

Billy Crook writes:

 > Am I just missing where this setting is, or has this not been
 > thought of yet.

It's been thought of, and it's just plain hard to do, because
automatically editing mail bodies robustly is just plain hard to do.

If you have a vast majority of users who use the same MUA, you can
probably get away with assuming that MUA's format, and will rarely, if
ever, get complaints from users whose mail has been mutilated.  Given
the variety and generally abysmal level of standard conformance of the
MUAs out there, we can't count on that. :-(

 > "apply footer only if footer text not already in message" would be
 > my logic of preference.

If there is any personalization (eg, user-specific unsubscribe link),
this logic doesn't work.  You need to remove the existing footer(s)
for other user(s), and add the correct one.  If you're not using any
personalization, this isn't a problem, of course.

It's also ugly, as in most cases the replying user's MUA will quote
the footer, often multiple times.

The "first in thread" logic has the same problems of personalization
and ugliness.

 > If that's not already something mailman can do, where would I modify the
 > code to add it?

Mailman/Handlers/Decorate.py.  Warning: Mailman 2 doesn't know
anything about threads, and Mailman 3 (unmerged development code) has
a rather different idea of threads (called "dynamic lists", "dlists"
for short).  So you'll have to use the "detect existing footer" logic,
not the "first in thread" logic.


From stephen at xemacs.org  Fri Aug 28 06:37:18 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 28 Aug 2015 13:37:18 +0900
Subject: [Mailman-Users] Assistance with altering reply-to behaviours
	and	DMARC
In-Reply-To: <55DF7621.3000008@incet.com>
References: <55DF7621.3000008@incet.com>
Message-ID: <878u8w58qp.fsf@uwakimon.sk.tsukuba.ac.jp>

Mark writes:

 > First, apologies if this has been discussed before.  I run a number
 > of mailman lists on a Centos 6 platform and mailman 2.1.12-25. This
 > version was updated in July as follows:
 > https://rhn.redhat.com/errata/RHSA-2015-1417.html and included
 > fixes for a number of DMARC issues.  Including "... With this
 > update, domains with a "reject" DMARC policy are recognized
 > correctly.."

Stock Mailman 2.1.12 doesn't do any DMARC detection.  This is quite
bizarre that they would backport such a feature rather than update to
2.1.18-1 or later.  Mailman 2.1 is hardly an unstable package.

 > Normally the lists we use are configured with reply_goes_to_list
 > and anonymous_list is off.  We have done some testing, and we have
 > not been able to identify settings that will retain
 > reply_goes_to_list functionality and turn on DMARC Mung.

You'll have to talk to the vendor who did the modifications.

 > Generally what happens is that the first message is successfully
 > sent to a yahoo.com address and this user can reply to the message
 > as normal.  However, mailman adds the original sender to the
 > reply-to meaning that when another user replies to this message
 > reply_goes_to_list doesn't work.

What do you mean by "doesn't work"?  That it doesn't go to the list at
all, or that although it goes to the list, it *also* goes (separately)
to the original poster?

Not going to the list at all means the MUA is broken IMO; multiple
addresses in Reply-To are explicitly allowed by the RFC, and MUAs
should reply to all of them.

 > It seemed to me that a small modification to CookHeaders.py could
 > prevent the adding of the original sender email to "Reply-To"
 > 
 >          # We also need to put the old From: in Reply-To: in all cases.
 >          if o_from:
 >              add(o_from)
 >          # Set Reply-To: header to point back to this list.  Add this last

After disabling the "add(o_from)" operation, if reply_goes_to_list is
set, then (1) the reply goes *only* to the list by default, and (2) it
is impossible for the user's MUA to automatically format a reply to
poster.  If reply_goes_to_list is not set, (1) reply by default will go to
list (since it's the address in From), and (2) it is impossible to
automatically format a reply to poster.

It is our opinion that (2) is a sufficiently important bug that we are
unlikely to change this, even to provide an option.  But if you want
to make that edit, it probably does what you want.  Depending on your
users, you may or may not get complaints about inability to respond
privately.


From es at fruitcom.com  Fri Aug 28 07:53:19 2015
From: es at fruitcom.com (Eric Smith)
Date: Fri, 28 Aug 2015 07:53:19 +0200
Subject: [Mailman-Users] old messages sent to list on reboot
Message-ID: <20150828055319.GA4244@biovolt.nl>

Hi

After rebooting my server, several mails from some weeks ago were
resent to lists and also a bunch of moderator requests to me.

How may I prevent this from happening again?

Eric

From mark at incet.com  Fri Aug 28 08:52:01 2015
From: mark at incet.com (Mark)
Date: Fri, 28 Aug 2015 18:52:01 +1200
Subject: [Mailman-Users] Assistance with altering reply-to behaviours
 and DMARC
In-Reply-To: <878u8w58qp.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <55DF7621.3000008@incet.com>
 <878u8w58qp.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <55E00511.7070506@incet.com>


On 28/08/15 4:37 pm, Stephen J. Turnbull wrote:
> Stock Mailman 2.1.12 doesn't do any DMARC detection.  This is quite
> bizarre that they would backport such a feature rather than update to
> 2.1.18-1 or later.  Mailman 2.1 is hardly an unstable package.

I am not sure why the changes were back-ported - maybe it had something to do with the version of Python on the platform.  I fully understand that the ability of the developers is 
limited to supporting known releases, and this release is not a supported one. So I don't expect more than some general guidance :)

>   > Normally the lists we use are configured with reply_goes_to_list
>   > and anonymous_list is off.  We have done some testing, and we have
>   > not been able to identify settings that will retain
>   > reply_goes_to_list functionality and turn on DMARC Mung.
>
> You'll have to talk to the vendor who did the modifications.
>
>   > Generally what happens is that the first message is successfully
>   > sent to a yahoo.com address and this user can reply to the message
>   > as normal.  However, mailman adds the original sender to the
>   > reply-to meaning that when another user replies to this message
>   > reply_goes_to_list doesn't work.
>
> What do you mean by "doesn't work"?  That it doesn't go to the list at
> all, or that although it goes to the list, it *also* goes (separately)
> to the original poster?

I will try to explain, but its only on subsequent replies the issue appears:-

Step 1:  email is sent to the list - the mail to DMARC protected domain is received ok by Yahoo web client.  Reply-to is set to the list.

Message source of first email is:

  From: Mark <mark at incet.com>
  Reply-To: Test <test at testdomain.org>

Step 2: Reply from Yahoo client sends an email to the list and this has the following source when viewed on Thunderbird MUA

  From: Mark H via Test <test at testdomain.org>
  Reply-To: Mark H <markhnzy at yahoo.com>, Test <test at testdomain.org>

Step 3: A reply from the TB MUA goes to markhnzy at yahoo.com and test at testdomain.org

Because mailman helpfully recognises that a copy has already been sent to the user directly, a copy is not sent to markhnzy@ from the list.

Step 4: Now, when markhnzy@ replies, the conversation is between the recipient and the sender alone.



>
> Not going to the list at all means the MUA is broken IMO; multiple
> addresses in Reply-To are explicitly allowed by the RFC, and MUAs
> should reply to all of them.
>
>   > It seemed to me that a small modification to CookHeaders.py could
>   > prevent the adding of the original sender email to "Reply-To"
>   >
>   >          # We also need to put the old From: in Reply-To: in all cases.
>   >          if o_from:
>   >              add(o_from)
>   >          # Set Reply-To: header to point back to this list.  Add this last
>
> After disabling the "add(o_from)" operation, if reply_goes_to_list is
> set, then (1) the reply goes *only* to the list by default, and (2) it
> is impossible for the user's MUA to automatically format a reply to
> poster.  If reply_goes_to_list is not set, (1) reply by default will go to
> list (since it's the address in From), and (2) it is impossible to
> automatically format a reply to poster.

I agree that this is not a bug as such; its more about the desired behaviour for a specific need of mailman and that new non-RFC compliant behaviour will result.

I tried commenting out if o_from and add(o_from), but original from address was still added to the reply-to(as well as the list address).  I did change some permissions and restart 
mailman and confirm that a new py file was created, so I don't think its that my change wasn't included.

Is there any other code in the standard mailman where the reply-to is populated with the original sender?  I realise  fully aware that the version of mailman I have here isn't 
supported, or released, and other bugs may have been introduced.

thanks again for the replies.

Mark.

> It is our opinion that (2) is a sufficiently important bug that we are
> unlikely to change this, even to provide an option.  But if you want
> to make that edit, it probably does what you want.  Depending on your
> users, you may or may not get complaints about inability to respond
> privately.


From andrew at hodgsonfamily.org  Fri Aug 28 09:40:40 2015
From: andrew at hodgsonfamily.org (Andrew Hodgson)
Date: Fri, 28 Aug 2015 07:40:40 +0000
Subject: [Mailman-Users] Assistance with altering reply-to behaviours
 and DMARC
In-Reply-To: <55E00511.7070506@incet.com>
References: <55DF7621.3000008@incet.com>
 <878u8w58qp.fsf@uwakimon.sk.tsukuba.ac.jp>,<55E00511.7070506@incet.com>
Message-ID: <DB5PR04MB14783D75138CC5B5D8E80316C56E0@DB5PR04MB1478.eurprd04.prod.outlook.com>

Mark wrote:

>Step 1:  email is sent to the list - the mail to DMARC protected domain is received ok by Yahoo web client.  Reply-to is set to the list.
>
>Message source of first email is:
>
>  From: Mark <mark at incet.com>
>  Reply-To: Test <test at testdomain.org>
>
>Step 2: Reply from Yahoo client sends an email to the list and this has the following source when viewed on Thunderbird MUA
>
>  From: Mark H via Test <test at testdomain.org>
>  Reply-To: Mark H <markhnzy at yahoo.com>, Test <test at testdomain.org>

That is interesting, and I wonder if this is something to do with the way that Redhat have implemented the new feature.

On my stock Mailman installation, the headers would look like this:

|From: Mark H via Test <test at testdomain.org>
|Reply-To: Test <test at testdomain.org>
|CC: Mark H <markhnzy at yahoo.com>

This is a much better imho way of dealing with the issue and keeps the messages flowing through the list.  I know if I had the same headers on my list that you are getting, that users would be falling into the same trap, and was something I tested hard before implementing it on my lists.

Is there any way you could just upgrade your installation to the latest 2.1.20?  There are some compiling guides out there for Redhat/Centos systems that were designed to get people through this before Redhat backported the feature.

Thanks.
Andrew.

From mailman at veggiechinese.net  Fri Aug 28 10:03:08 2015
From: mailman at veggiechinese.net (Will Yardley)
Date: Fri, 28 Aug 2015 01:03:08 -0700
Subject: [Mailman-Users] Assistance with altering reply-to behaviours
 and DMARC
In-Reply-To: <878u8w58qp.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <55DF7621.3000008@incet.com>
 <878u8w58qp.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20150828080308.GE95567@aura.veggiechinese.net>

On Fri, Aug 28, 2015 at 01:37:18PM +0900, Stephen J. Turnbull wrote:
> Mark writes:
>  > First, apologies if this has been discussed before.  I run a number
>  > of mailman lists on a Centos 6 platform and mailman 2.1.12-25. This
>  > version was updated in July as follows:
>  > https://rhn.redhat.com/errata/RHSA-2015-1417.html and included
>  > fixes for a number of DMARC issues.  Including "... With this
>  > update, domains with a "reject" DMARC policy are recognized
>  > correctly.."
> 
> Stock Mailman 2.1.12 doesn't do any DMARC detection.  This is quite
> bizarre that they would backport such a feature rather than update to
> 2.1.18-1 or later.  Mailman 2.1 is hardly an unstable package.

That's the way most "stable" Linux distributions do it. I think it has a
lot to do with keeping the complex web of interdependencies in check,
but probably also has to do with keeping features predictable. Even with
security issues, generally, they go to quite a bit of work to backport
things, rather than change versions.

Not sure about 6, but the specific way they backported the changes for
the DMARC patch on EL5 had to do partially with dependencies... either
the fact that the normal version didn't depend on dnspython and this one
did, or the version of dnspython required wasn't available in EL5 (looks
like they actually ended up vendoring dnspython inside the Mailman
package).

Folks who want more up to date versions tend to either jump major
releases more often, use third party repos, rebuild RPMs from upstream,
or just build from source. We use the vendor provided Mailman package,
and for the most part, it works for us, though in cases like this, it is
problematic.

w


From andrew at hodgsonfamily.org  Fri Aug 28 09:50:36 2015
From: andrew at hodgsonfamily.org (Andrew Hodgson)
Date: Fri, 28 Aug 2015 07:50:36 +0000
Subject: [Mailman-Users] Error updating mailman Ubuntu 14.04.3
In-Reply-To: <CADbGg-PnKP0muXB-G=39js35QVrZDOHt1TSGG1O+OH425J5NKw@mail.gmail.com>
References: <CADbGg-PnKP0muXB-G=39js35QVrZDOHt1TSGG1O+OH425J5NKw@mail.gmail.com>
Message-ID: <DB5PR04MB147822E67C76988BFDA9205EC56E0@DB5PR04MB1478.eurprd04.prod.outlook.com>

billy noah wrote:

[...]

>Looking for enabled languages (this may take some time) ... done.
>Removing unmodified files from /etc/mailman/eu  done.
>Directory /etc/mailman/eu not empty, not removed.

Is there any chance you installed a later version of Mailman from source and forgot to remove the Ubuntu package?

Thanks.
Andrew.

From bsfinkel at att.net  Fri Aug 28 17:10:53 2015
From: bsfinkel at att.net (Barry S. Finkel)
Date: Fri, 28 Aug 2015 10:10:53 -0500
Subject: [Mailman-Users] Assistance with altering reply-to behaviours
 and DMARC
In-Reply-To: <20150828080308.GE95567@aura.veggiechinese.net>
References: <55DF7621.3000008@incet.com>
 <878u8w58qp.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20150828080308.GE95567@aura.veggiechinese.net>
Message-ID: <55E079FD.6040704@att.net>

On 8/28/2015 3:03 AM, Will Yardley wrote:
> On Fri, Aug 28, 2015 at 01:37:18PM +0900, Stephen J. Turnbull wrote:
>> Mark writes:
>>   > First, apologies if this has been discussed before.  I run a number
>>   > of mailman lists on a Centos 6 platform and mailman 2.1.12-25. This
>>   > version was updated in July as follows:
>>   > https://rhn.redhat.com/errata/RHSA-2015-1417.html and included
>>   > fixes for a number of DMARC issues.  Including "... With this
>>   > update, domains with a "reject" DMARC policy are recognized
>>   > correctly.."
>>
>> Stock Mailman 2.1.12 doesn't do any DMARC detection.  This is quite
>> bizarre that they would backport such a feature rather than update to
>> 2.1.18-1 or later.  Mailman 2.1 is hardly an unstable package.
>
> That's the way most "stable" Linux distributions do it. I think it has a
> lot to do with keeping the complex web of interdependencies in check,
> but probably also has to do with keeping features predictable. Even with
> security issues, generally, they go to quite a bit of work to backport
> things, rather than change versions.
>
> Not sure about 6, but the specific way they backported the changes for
> the DMARC patch on EL5 had to do partially with dependencies... either
> the fact that the normal version didn't depend on dnspython and this one
> did, or the version of dnspython required wasn't available in EL5 (looks
> like they actually ended up vendoring dnspython inside the Mailman
> package).
>
> Folks who want more up to date versions tend to either jump major
> releases more often, use third party repos, rebuild RPMs from upstream,
> or just build from source. We use the vendor provided Mailman package,
> and for the most part, it works for us, though in cases like this, it is
> problematic.
>
> w

It appears to me that if someone were to back-port the DMARC changes
from 2.1.18 back to 2.1.12, then there is a possibility that there
will be problems with the back-port due to code changes between
.12 and .18.  When I looked at the Ubuntu changes, I saw many
changes that were not documented, and I had no idea what they did.
Part of the problem in Ubuntu (or any other packager) who wants
to create a package for a newer Mailman is that all of these
patches, whether created to fix bugs or add features, have to be
re-fitted into the code.  And that takes time.  I am not sure
which process is more prone to introduce errors - re-fitting
patches into a newer Mailman or trying to re-fit DMARC patches
into on older Mailman.

This is why I determined, when I was running a Mailman installation
on Ubuntu, that I would create my own package from the SourceForge
source instead of installing the Ubuntu Mailman package.  It took
me a while to figure out how to do it, but once I did it, I had
the instructions to re-do it for subsequent SF Mailman releases.

I wanted to know exactly what source I was running, so I could get
assistance from the authors via this list.

--Barry Finkel
--Barry Finkel


From cpz at tuunq.com  Fri Aug 28 21:05:11 2015
From: cpz at tuunq.com (Carl Zwanzig)
Date: Fri, 28 Aug 2015 12:05:11 -0700
Subject: [Mailman-Users] How to prevent messages from being "held"
In-Reply-To: <55DC51B2.4050004@bergs.biz>
References: <55DC51B2.4050004@bergs.biz>
Message-ID: <55E0B0E7.8020405@tuunq.com>

On 8/25/2015 4:29 AM, Ralf G. R. Bergs wrote:
> I'm struggling since a while since MailMan will sometimes put messages
> into the "Held" queue even though I did my very best to set all config
> options so that MailMan won't... :-(

Please tell us what mailman says about them being held- are they over-size, 
implicit destinations, from a non-subscriber on a closed list, etc? There's 
usually a reason, although it's not always clear what that reason is.

z!


From Ralf+MailMan at bergs.biz  Fri Aug 28 21:42:22 2015
From: Ralf+MailMan at bergs.biz (Ralf G. R. Bergs)
Date: Fri, 28 Aug 2015 21:42:22 +0200
Subject: [Mailman-Users] How to prevent messages from being "held"
In-Reply-To: <55E0B0E7.8020405@tuunq.com>
References: <55DC51B2.4050004@bergs.biz> <55E0B0E7.8020405@tuunq.com>
Message-ID: <55E0B99E.9050806@bergs.biz>

Hi Carl.

Thanks for your response.

On 2015-08-28 21:05 , Carl Zwanzig wrote:
> On 8/25/2015 4:29 AM, Ralf G. R. Bergs wrote:
>> I'm struggling since a while since MailMan will sometimes put messages
>> into the "Held" queue even though I did my very best to set all config
>> options so that MailMan won't... :-(
>
> Please tell us what mailman says about them being held- are they
> over-size, implicit destinations, from a non-subscriber on a closed
> list, etc? There's usually a reason, although it's not always clear
> what that reason is.
It's always the following:
> SpamAssassin identified this message as possible spam
I've checked all config options for that ML many times, everything is
set up as "openly" as possible. I've even an entry in
header_filter_rules for "Accept" for regexp "^From:.*" -- still it
happens once in a while that messages get trapped as purported "spam"...

Regardless of what SA thinks about a message, I want Mailman to
distribute it to all subs...

Kind regards,

Ralf



From mailman at veggiechinese.net  Sat Aug 29 08:53:10 2015
From: mailman at veggiechinese.net (Will Yardley)
Date: Fri, 28 Aug 2015 23:53:10 -0700
Subject: [Mailman-Users] Assistance with altering reply-to behaviours
 and DMARC
In-Reply-To: <55E079FD.6040704@att.net>
References: <55DF7621.3000008@incet.com>
 <878u8w58qp.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20150828080308.GE95567@aura.veggiechinese.net>
 <55E079FD.6040704@att.net>
Message-ID: <20150829065309.GA10668@aura.veggiechinese.net>

On Fri, Aug 28, 2015 at 10:10:53AM -0500, Barry S. Finkel wrote:
> On 8/28/2015 3:03 AM, Will Yardley wrote:
> > On Fri, Aug 28, 2015 at 01:37:18PM +0900, Stephen J. Turnbull wrote:
> >> Mark writes:

> >> > First, apologies if this has been discussed before.  I run a
> >> > number of mailman lists on a Centos 6 platform and mailman
> >> > 2.1.12-25. This version was updated in July as follows:
> >> > https://rhn.redhat.com/errata/RHSA-2015-1417.html and included
> >> > fixes for a number of DMARC issues.

> >> Stock Mailman 2.1.12 doesn't do any DMARC detection.  This is quite
> >> bizarre that they would backport such a feature rather than update
> >> to 2.1.18-1 or later.  Mailman 2.1 is hardly an unstable package.

> > That's the way most "stable" Linux distributions do it. I think it
> > has a lot to do with keeping the complex web of interdependencies in
> > check, but probably also has to do with keeping features
> > predictable. Even with security issues, generally, they go to quite
> > a bit of work to backport things, rather than change versions.

> It appears to me that if someone were to back-port the DMARC changes
> from 2.1.18 back to 2.1.12, then there is a possibility that there
> will be problems with the back-port due to code changes between
> .12 and .18.

We've been running the RHEL5 one, which is an even older Mailman version
(2.1.9), with the DMARC stuff that Red Hat merged in, and has been
working fine for us (on a machine with thousands of lists, some of which
are fairly active).

It's not usual to merge in this kind of added feature, but I think in
this case, there was enough customer demand, and it was causing enough
problems for enough people, that they made the decision to devote the
resources to implement / test.

w


From stephen at xemacs.org  Sat Aug 29 14:23:23 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 29 Aug 2015 21:23:23 +0900
Subject: [Mailman-Users] Assistance with altering reply-to behaviours
 and DMARC
In-Reply-To: <20150828080308.GE95567@aura.veggiechinese.net>
References: <55DF7621.3000008@incet.com>
 <878u8w58qp.fsf@uwakimon.sk.tsukuba.ac.jp>
 <20150828080308.GE95567@aura.veggiechinese.net>
Message-ID: <877foe5lms.fsf@uwakimon.sk.tsukuba.ac.jp>

Will Yardley writes:
 > On Fri, Aug 28, 2015 at 01:37:18PM +0900, Stephen J. Turnbull wrote:

 > > Stock Mailman 2.1.12 doesn't do any DMARC detection.  This is quite
 > > bizarre that they would backport such a feature rather than update to
 > > 2.1.18-1 or later.  Mailman 2.1 is hardly an unstable package.
 > 
 > That's the way most "stable" Linux distributions do it.

I understand that, and I agree with the logic *when you're talking
about feature upgrades*.  I'm just saying I don't think it's a good
idea when you're dealing with a very stable package and an extreme
change in the environment that effectively hamstrings the package --
they should bite the bullet and devote the resources to upgrading that
package, or alternatively declare it unsupported.

 > We use the vendor provided Mailman package, and for the most part,
 > it works for us, though in cases like this, it is problematic.

Precisely my point.  IIUC, by the time they backported, Mailman had
iterated on DMARC three times (first with an unconditional
From-munging patch, then with the original patch to conditionally
munge, and finally with the improved patch to deal with the preserving
the ability to automatically format the response to the author).  They
probably also have other backported bugfixes.  OTOH, Mailman 2 has
basically been feature-frozen throughout the period.


From stephen at xemacs.org  Sat Aug 29 15:02:53 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 29 Aug 2015 22:02:53 +0900
Subject: [Mailman-Users] Assistance with altering reply-to behaviours
 and DMARC
In-Reply-To: <55E00511.7070506@incet.com>
References: <55DF7621.3000008@incet.com>
 <878u8w58qp.fsf@uwakimon.sk.tsukuba.ac.jp>
 <55E00511.7070506@incet.com>
Message-ID: <87613y5jsy.fsf@uwakimon.sk.tsukuba.ac.jp>

Mark writes:

 > Is there any other code in the standard mailman where the reply-to
 > is populated with the original sender?  I realise fully aware that
 > the version of mailman I have here isn't supported, or released,
 > and other bugs may have been introduced.

In standard Mailman >= 2.1.18-1 (more or less) there are two places.
I guess the one you found is the unconditional "from_is_list" setting
in CookHeaders.py; sorry, I misled you because I was under them
impression that all such changes would take place in CookHeaders.  In
2.1.20, there's another, more complex path which involves checking
dmarc_moderation_action in SpamDetect.py, deciding which headers to
munge in CookHeaders.py:change_header(), and actually doing the work
in WrapMessage.py.

I don't have 2.1.18-1 to hand, and I'm not sure that is the version
that was backported anyway, so it's hard to be sure what you'd need to
change.  Probably searching for "dmarc" in SpamDetect and WrapMessage,
and "change_header" in CookHeaders, will allow you to figure out how
the backport works.


From stephen at xemacs.org  Sat Aug 29 17:16:59 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 30 Aug 2015 00:16:59 +0900
Subject: [Mailman-Users] Assistance with altering reply-to behaviours
 and DMARC
In-Reply-To: <DB5PR04MB14783D75138CC5B5D8E80316C56E0@DB5PR04MB1478.eurprd04.prod.outlook.com>
References: <55DF7621.3000008@incet.com>
 <878u8w58qp.fsf@uwakimon.sk.tsukuba.ac.jp>
 <55E00511.7070506@incet.com>
 <DB5PR04MB14783D75138CC5B5D8E80316C56E0@DB5PR04MB1478.eurprd04.prod.outlook.com>
Message-ID: <871tem5dlg.fsf@uwakimon.sk.tsukuba.ac.jp>

Sorry, serious finger fumble there.  Please ignore the messed up one.

Andrew Hodgson writes:

 > That is interesting, and I wonder if this is something to do with
 > the way that Redhat have implemented the new feature.

My guess is that Red Hat backported 2.1.17 or 2.1.18-1.  After getting
complaints, Mark S fixed things so that the From address was moved to
Cc rather than Reply-To in 2.1.19 or 2.1.20.

 > Is there any way you could just upgrade your installation to the
 > latest 2.1.20?  There are some compiling guides out there for
 > Redhat/Centos systems that were designed to get people through this
 > before Redhat backported the feature.

It's easy enough to do, but beware of installing into the same places
that Red Hat/CentOS uses, as that is quite likely to result in various
problems if you try to upgrade the Mailman package from the OS vendor.
On the other hand, using a source build installed to the default
/var/local hierarchy means you'll need to copy the site config
mm_cfg.py, the list configs, and the archives to the new location, and
teach the webserver and mailserver about the new Mailman.  In
principle it's pretty straightforward and I haven't hesitated to do so
on several occasions, but I can understand why an admin might be
nervous (or they might have a rule that all installed software must be
controlled by the system's PMS, and that's a good rule that shouldn't
be violated lightly).

From ricardo.rodriguez at idisantiago.es  Fri Aug 28 09:52:57 2015
From: ricardo.rodriguez at idisantiago.es (=?UTF-8?Q?=5BIDIS_Technical_Secretariat=5D_Ricardo_Rodr=C3=ADguez?=)
Date: Fri, 28 Aug 2015 09:52:57 +0200
Subject: [Mailman-Users] importing archived Maildir email lists into Mailman
	lists archives
Message-ID: <CAFPUu6zTDhxxqkyfvaa-DUAhxrgjzSMyM4U2cSHmsh2gAmA44A@mail.gmail.com>

Hi everybody!

I'm new to most of the fields related with this issue so, please, accept my
apologies in advance if the question or the way towards its answer is too
obvious!

I have a number of Maildir format mailing lists archives and I would like
to add them to the archives of the same lists created in a new Mailman
installation.

I find several messages/scripts doing the job the other way round
(mbox/pipermail/mailman to maildir), but nothing about Maildir to Mailman,
assumable by a "programming capabilities-less user".

Thanks for your time!!!

-- 
Ricardo Rodr?guez
Research Management and Promotion Technician
Technical Secretariat
Health Research Institute of Santiago de Compostela (IDIS)
http://www.idisantiago.es

From mark at msapiro.net  Sun Aug 30 00:09:47 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 29 Aug 2015 15:09:47 -0700
Subject: [Mailman-Users] {Disarmed} Re: Archive page not showing dates
 properly
In-Reply-To: <CAGmyAkAWMCezhu4VjzamcTch3yjApDsiH9pN8GEi_EVjuodApg@mail.gmail.com>
References: <CAGmyAkAB7h3YSVVm4LSpHt-7mQ8tFQD5N-Ok=3R8BQuJfVSNEQ@mail.gmail.com>	<55D73CC9.1050303@msapiro.net>	<CAGmyAkCGsQrt3o++3+_2t-tnhsq-Sm6A=UtH0m_EDPzi28gD+w@mail.gmail.com>	<55DB316D.9080905@msapiro.net>
 <CAGmyAkAWMCezhu4VjzamcTch3yjApDsiH9pN8GEi_EVjuodApg@mail.gmail.com>
Message-ID: <55E22DAB.1040404@msapiro.net>

On 08/24/2015 11:35 PM, Peter Wetz wrote:
> 
> hm, I get this error though:
> 
...
> File "/var/lib/mailman/Mailman/Archiver/HyperArch.py", line 1072, in
> write_index_entry
> 'date': time.strftime("%d %b %Y %H:%M:%S", time.localtime(article.date)),
> TypeError: a float is required 
> 
> "a float is required"
> this implies that "article.date" is not a float in my case. maybe it
> needs to be converted to a float explicitly?
> any other suggestions? 


No. Converting it to float is the appropriate thing. e.g.

>    'date': time.strftime("%d %b %Y %H:%M:%S",
time.localtime(float(article.date))),

The issue is that while article.date is a *nix timestamp, it is a string
rather than an int or float.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Aug 30 02:02:51 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 29 Aug 2015 17:02:51 -0700
Subject: [Mailman-Users] Pluggable authentication for Mailman web
	interface?
In-Reply-To: <854480980.1021293.1440619708372.JavaMail.zimbra@lafayette.edu>
References: <854480980.1021293.1440619708372.JavaMail.zimbra@lafayette.edu>
Message-ID: <55E2482B.8000805@msapiro.net>

On 08/26/2015 01:08 PM, Waldbieser, Carl wrote:
> 
> Are there any guidelines for adding authentication and /or authorization mechanisms to the Mailman web user interface?
> Specifically, I was wondering if there is any kind of guidance for authenticating the user via an HTTP header (e.g. HTTP_REMOTE_USER) so that an authenticating reverse proxy could be placed in front of the Mailman web interface.


There is no such for Mailman 2.1. I *think* the authentication mechanism
for MM 3 is pluggable, but you should join and ask on
mailman-developers at python.org for a more definitive answer.


> If there is no such built-in mechanism or pluggable mechanism, is there any kind of guidance on how the existing authentication mechanism might be replaced from a technical standpoint?


In MM 2.1, all the work is done by the Mailman/SecurityManager.py
module. You should be able to tweak that to suit your needs.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Aug 30 02:24:07 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 29 Aug 2015 17:24:07 -0700
Subject: [Mailman-Users] list subscription spammers
In-Reply-To: <20150825171210.GM60769@aura.veggiechinese.net>
References: <20150825171210.GM60769@aura.veggiechinese.net>
Message-ID: <55E24D27.5060409@msapiro.net>

On 08/25/2015 10:12 AM, Will Yardley wrote:
> I'm seeing massive numbers of subscription lists to various lists we
> host (including multiple requests to the same list).
> 
> These are submitted via a distributed network of hosts, presumably
> botnet victims / open proxies.
> 
> The requests are from
> foo+[0-9]{9}@gmail.com


We have seen a huge rash of these on the python.org lists. Even with a
SUBSCRIBE_FORM_SECRET set and SUBSCRIBE_FORM_MIN_TIME = seconds(8) a
small percentage succeeded and that was still a lot.

Because of this, the head of the 2.1 branch at
<https://launchpad.net/mailman/2.1/> now implements a GLOBAL_BAN_LIST
and on mail.python.org, that is set to

GLOBAL_BAN_LIST = ['^.*\+\d{3,}@']

to ban any address whose local part ends with a '+' followed by 3 or
more digits. Before doing this I checked and there was only one member
of one list out of all the python.org list subscribers that had a local
part ending with '+' and digits and it only had one digit between the
'+' and '@'.

The attempts keep coming though.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Aug 30 02:27:01 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 29 Aug 2015 17:27:01 -0700
Subject: [Mailman-Users] list subscription spammers
In-Reply-To: <20150827021435.GA79882@aura.veggiechinese.net>
References: <20150825171210.GM60769@aura.veggiechinese.net>
 <20150827021435.GA79882@aura.veggiechinese.net>
Message-ID: <55E24DD5.9080601@msapiro.net>

On 08/26/2015 07:14 PM, Will Yardley wrote:
> 
> % cat set_banlist.py 
...

Also see <https://www.msapiro.net/scripts/add_banned.py>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Aug 30 02:47:19 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 29 Aug 2015 17:47:19 -0700
Subject: [Mailman-Users] Assistance with altering reply-to behaviours
 and DMARC
In-Reply-To: <55DED12D.3000300@incet.com>
References: <55DED12D.3000300@incet.com>
Message-ID: <55E25297.4060002@msapiro.net>

On 08/27/2015 01:58 AM, Mark wrote:
> Hi,
> 
> First, apologies if this has been discussed before.  I run a number of
> mailman lists on a Centos 6 platform and mailman 2.1.12-25. This version
> was updated in July as follows:
> https://rhn.redhat.com/errata/RHSA-2015-1417.html and included fixes for
> a number of DMARC issues.  Including "... With this update, domains with
> a "reject" DMARC policy are recognized correctly.."
> 
> Normally the lists we use are configured with reply_goes_to_list and
> anonymous_list is off.  We have done some testing, and we have not been
> able to identify settings that will retain reply_goes_to_list
> functionality  and turn on DMARC Mung.
> 
> Generally what happens is that the first message is successfully sent to
> a yahoo.com address and this user can reply to the message as normal. 
> However, mailman adds the original sender to the reply-to meaning that
> when another user replies to this message reply_goes_to_list doesn't work.


It should reply to both the list and the original poster and any other
addresses in Reply-To; If it isn't, the replying MUA is at fault. RFC
5322, sec 3.6.2 says in part "When the "Reply-To:" field is present, it
indicates the address(es) to which the author of the message suggests
that replies be sent."

That said however, this is a bug. For a list with reply_goes_to_list =
This list, The original poster's address should be added to Cc:, not
Reply-To:. See <https://bugs.launchpad.net/mailman/+bug/1407098>, fixed
in Mailman 2.1.19.

If RedHat is going to backport these mitigatations into their package,
they should keep up.


> It seemed to me that a small modification to CookHeaders.py could
> prevent the adding of the original sender email to "Reply-To"
> 
>         # We also need to put the old From: in Reply-To: in all cases.
>         if o_from:
>             add(o_from)
>         # Set Reply-To: header to point back to this list.  Add this last
> 
> seemed like a good place to look, but before I try to debug commenting
> out this didn't prevent the addition of the original sender to Reply-To
> I would ask for some advice !!
> 
> any assistance would be very helpful!!   (it maybe that changing the
> order of addresses in the Reply-To field would all reply_goes_to_list
> would work?)


See the fix at
<http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1510>.
If you apply this, you need both the CookHeaders.py patch and the
WrapMessage.py patch.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Aug 30 02:53:47 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 29 Aug 2015 17:53:47 -0700
Subject: [Mailman-Users] ISO speciific RegExp to filter/discard bot
 subscribe requests
In-Reply-To: <55DEDD33.5020207@gmail.com>
References: <55DEDD33.5020207@gmail.com>
Message-ID: <55E2541B.6070808@msapiro.net>

On 08/27/2015 02:49 AM, Nelson Kelly wrote:

> 
> Common to ALL of the bot subscribe requests is including within the the
> email address a "+" symbol.
> 
> So I tried entering RegExp ^.*+ but Mailman rejected this entry.


Because '+' means 1 or more of the preceding, but the preceding is a
repeat itself so it's a bad regexp.

> But was able to enter RegExp of ^.*/+
> However that allowed all the bot spam requests through.


because that says 0 or more of anything followed by 1 or more slashes
(/) which is unlikely to match anything. (You probably wanted ^.*\+ but
see below.)


> Unable to keep up with the bots changing the address prefix.
> 
> Can anyone recommend an example of a RegExp entry which would
> prevent/solve this dilemma?


Yes.

^.*\+\d{3,}@

See
<https://mail.python.org/pipermail/mailman-users/2015-August/079668.html> for
a bit more.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Aug 30 03:28:52 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 29 Aug 2015 18:28:52 -0700
Subject: [Mailman-Users] Does Mailman alter messages attached to
 moderation notification emails?
In-Reply-To: <C2F1879073AE7347BAB44065F143E17002AA5D94ED@nuwvicms2>
References: <C2F1879073AE7347BAB44065F143E17002AA5D94ED@nuwvicms2>
Message-ID: <55E25C54.6070909@msapiro.net>

On 08/27/2015 06:22 PM, Peter Shute wrote:
> 
> Now it's happened again, this time in a plain text only message. The period is there in his copy of the sent message, missing in what went to the list. I can also see that it's missing in the copy of the message attached to the moderation notification.


Is the Message-ID: in his copy the same as in the list post?


> Does the list send out moderation notifications with the attached messages unaltered?


Yes.


> Is there any mechanism in mailman that could have removed the period?


I won't say it's not possible, but I doubt it. If so it would be in an
underlying Python library module.


> It also removed one from a URL sent to the list owner address, but again that message has passed through Mailman. I'm not able to easily get a sample of a message from him that hasn't been through Mailman, so I can't eliminate it as a suspect.
> 
> Looking at the raw messages, something that's common about them is that the missing period would fall exactly on the start of a line when it's wrapped, i.e after the = symbol which I assume is a soft eol, and in one of them the period is there in the plain text section, but missing in the html section.


It sounds as if this could be a 'format=flowed' issue. Please post one
of these raw messages as sent to mailman. Put the message in a file with
a .txt extension and attach it to your post. You can remove all the
headers with identifying info. I want to see all the MIME-Version:,
Content-Type: and Content-Transfer-Encoding: headers, the boundaries and
the sub-part headers and the part content.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Aug 30 03:32:21 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 29 Aug 2015 18:32:21 -0700
Subject: [Mailman-Users] ISO RegExp example to block bot subscribe
	requests
In-Reply-To: <87bnds5bqu.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <1440664625.11103.YahooMailBasic@web125203.mail.ne1.yahoo.com>
 <87bnds5bqu.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <55E25D25.5030106@msapiro.net>

On 08/27/2015 08:32 PM, Stephen J. Turnbull wrote:
> Woody Mon via Mailman-Users writes:
> 
>  > So I tried entering RegExp ^.*+ but Mailman rejected this entry.
> 
> That's illegal syntax for a regexp because "+" is an operator.
> 
> The version you want is "^.*\+".  However, that is a very bad idea if
> you have any posters with email skills, because embedding "+" in the
> return address for outgoing mail is a common way to identify the
> source of incoming replies.
> 
> A more conservative regexp would be "^[a-z]+\+[0-9]+@" which will
> catch the pattern of LETTERS "+" DIGITS.


Also, see my reply to Nelson Kelly at
<https://mail.python.org/pipermail/mailman-users/2015-August/079671.html>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Aug 30 03:39:49 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 29 Aug 2015 18:39:49 -0700
Subject: [Mailman-Users] Footer Bloat
In-Reply-To: <201508271823.t7RINujE008297@theraft.openend.se>
References: <CANZ96zUruNB54Q2ED4_YeZux-gDH3-O=abMfX3h-k0aB+kDoqg@mail.gmail.com>
 <201508271823.t7RINujE008297@theraft.openend.se>
Message-ID: <55E25EE5.2090705@msapiro.net>

On 08/27/2015 11:23 AM, Laura Creighton wrote:
> You may be interested in this thread:
> https://mail.python.org/pipermail/mailman-users/2012-April/073181.html


Thank you Laura.

Also note that if sending users add a standard signature (prefixed with
a '-- ' line), and repliers use an MUA that doesn't quote signatures in
replies, the problem doesn't exist, but that's probably too much to hope
for.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Aug 30 03:54:50 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 29 Aug 2015 18:54:50 -0700
Subject: [Mailman-Users] old messages sent to list on reboot
In-Reply-To: <20150828055319.GA4244@biovolt.nl>
References: <20150828055319.GA4244@biovolt.nl>
Message-ID: <55E2626A.7060904@msapiro.net>

On 08/27/2015 10:53 PM, Eric Smith wrote:
> Hi
> 
> After rebooting my server, several mails from some weeks ago were
> resent to lists and also a bunch of moderator requests to me.


There was a bug in Mailman 2.1.9 that would not unlink the
qfiles/QUEUE/*.bak files used for recovery processing under some
conditions. These files would then be 'recovered' when Mailman is restarted.


> How may I prevent this from happening again?


If the above is the reason, update to a more recent version, or possibly
run a periodic cron to remove qfiles/QUEUE/*.bak files older than some age.

It the above is not the reason, We'd need more information to diagnose
how this happened.

Perhaps id you examine your 'qrunner' log leading up to the reboot,
you'll find that some runner or runners had died and not restarted, and
then started and processed their queues after the reboot.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Aug 30 04:04:44 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 29 Aug 2015 19:04:44 -0700
Subject: [Mailman-Users] Error updating mailman Ubuntu 14.04.3
In-Reply-To: <CADbGg-PnKP0muXB-G=39js35QVrZDOHt1TSGG1O+OH425J5NKw@mail.gmail.com>
References: <CADbGg-PnKP0muXB-G=39js35QVrZDOHt1TSGG1O+OH425J5NKw@mail.gmail.com>
Message-ID: <55E264BC.5050002@msapiro.net>

On 08/25/2015 04:28 PM, billy noah wrote:
> Tried running an update today and received this message:
> 
> Setting up mailman (1:2.1.16-2ubuntu0.1) ...
...
> Downgrade detected, from version 0x20112f1 to version 0x20110f0


I'm with Steve on this. This is very wierd. The first message says you
are setting up 2.1.16-2 and the later message says you are downgrading
from 2.1.12-1 to 2.1.10.


> Any ideas what I can do to remedy this?


You can run Mailman's 'bin/update --force' manually, but I wouldn't
recommend it, because something unknown is messed up.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Aug 30 04:09:45 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 29 Aug 2015 19:09:45 -0700
Subject: [Mailman-Users] importing archived Maildir email lists into
 Mailman lists archives
In-Reply-To: <CAFPUu6zTDhxxqkyfvaa-DUAhxrgjzSMyM4U2cSHmsh2gAmA44A@mail.gmail.com>
References: <CAFPUu6zTDhxxqkyfvaa-DUAhxrgjzSMyM4U2cSHmsh2gAmA44A@mail.gmail.com>
Message-ID: <55E265E9.3040305@msapiro.net>

On 08/28/2015 12:52 AM, [IDIS Technical Secretariat] Ricardo Rodr?guez
wrote:
> 
> I have a number of Maildir format mailing lists archives and I would like
> to add them to the archives of the same lists created in a new Mailman
> installation.


This search
<https://www.google.com/?gws_rd=ssl#q=convert+maildir+to+mbox> should
return something useful.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Aug 30 04:29:04 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 29 Aug 2015 19:29:04 -0700
Subject: [Mailman-Users] How to prevent messages from being "held"
In-Reply-To: <55E0B99E.9050806@bergs.biz>
References: <55DC51B2.4050004@bergs.biz> <55E0B0E7.8020405@tuunq.com>
 <55E0B99E.9050806@bergs.biz>
Message-ID: <55E26A70.2070106@msapiro.net>

On 08/28/2015 12:42 PM, Ralf G. R. Bergs wrote:
>>
>> Please tell us what mailman says about them being held- are they
>> over-size, implicit destinations, from a non-subscriber on a closed
>> list, etc? There's usually a reason, although it's not always clear
>> what that reason is.
> It's always the following:
>> SpamAssassin identified this message as possible spam


This message comes from a non-standard handler, perhaps one of the ones
at <https://bugs.launchpad.net/mailman/+bug/558094>,
<https://bugs.launchpad.net/mailman/+bug/557991> or
<https://bugs.launchpad.net/mailman/+bug/266588>, although none of those
produce that exact reason.


> I've checked all config options for that ML many times, everything is
> set up as "openly" as possible. I've even an entry in
> header_filter_rules for "Accept" for regexp "^From:.*" -- still it
> happens once in a while that messages get trapped as purported "spam"...


This is not being done by header_filter_rules.


> Regardless of what SA thinks about a message, I want Mailman to
> distribute it to all subs...


The handler doing this may be in the GLOBAL_PIPELINE or a list specific
pipeline attribute. You need to remove it from whatever pipeline
specifies it.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From es at fruitcom.com  Sun Aug 30 08:01:13 2015
From: es at fruitcom.com (Eric Smith)
Date: Sun, 30 Aug 2015 08:01:13 +0200
Subject: [Mailman-Users] old messages sent to list on reboot
In-Reply-To: <55E2626A.7060904@msapiro.net>
References: <20150828055319.GA4244@biovolt.nl> <55E2626A.7060904@msapiro.net>
Message-ID: <mailbox-21530-1440914150-14620@pepper.0.0.200>


Thank Mark

Mark Sapiro wrote on Sun-30-Aug 15  3:54AM
> On 08/27/2015 10:53 PM, Eric Smith wrote:
> There was a bug in Mailman 2.1.9 that would not unlink the
> qfiles/QUEUE/*.bak files used for recovery processing under some
> conditions. These files would then be 'recovered' when Mailman is restarted.

% dpkg -l|grep mailman.
ii  mailman                                     1:2.1.18-1ubuntu1

> It the above is not the reason, We'd need more information to diagnose
> how this happened.
> 
> Perhaps id you examine your 'qrunner' log leading up to the reboot,
> you'll find that some runner or runners had died and not restarted, and
> then started and processed their queues after the reboot.

I started reboot on 27 August at 16:14;
# zcat  qrunner.3.gz
Aug 27 16:15:25 2015 (3092) CommandRunner qrunner started.
Aug 27 16:15:25 2015 (3094) NewsRunner qrunner started.
Aug 27 16:15:25 2015 (3090) ArchRunner qrunner started.
Aug 27 16:15:25 2015 (3091) BounceRunner qrunner started.
Aug 27 16:15:25 2015 (3093) IncomingRunner qrunner started.
Aug 27 16:15:25 2015 (3097) RetryRunner qrunner started.
Aug 27 16:15:26 2015 (3096) VirginRunner qrunner started.
Aug 27 16:15:26 2015 (3095) OutgoingRunner qrunner started.
# zcat  qrunner.2.gz
Aug 28 08:04:29 2015 (3097) RetryRunner qrunner caught SIGHUP.  Reopening logs.
Aug 28 08:04:29 2015 (3089) Master watcher caught SIGHUP.  Re-opening log files.
Aug 28 08:04:29 2015 (3092) CommandRunner qrunner caught SIGHUP.  Reopening logs.
Aug 28 08:04:29 2015 (3096) VirginRunner qrunner caught SIGHUP.  Reopening logs.
Aug 28 08:04:29 2015 (3091) BounceRunner qrunner caught SIGHUP.  Reopening logs.
Aug 28 08:04:29 2015 (3095) OutgoingRunner qrunner caught SIGHUP.  Reopening logs.
Aug 28 08:04:29 2015 (3094) NewsRunner qrunner caught SIGHUP.  Reopening logs.
Aug 28 08:04:29 2015 (3093) IncomingRunner qrunner caught SIGHUP.  Reopening logs.
Aug 28 08:04:29 2015 (3090) ArchRunner qrunner caught SIGHUP.  Reopening logs.
# zcat  qrunner.1.gz
pepper:/var/log/mailman# cat  qrunner.1
Aug 29 07:53:39 2015 (3096) VirginRunner qrunner caught SIGHUP.  Reopening logs.
Aug 29 07:53:39 2015 (3095) OutgoingRunner qrunner caught SIGHUP.  Reopening logs.
Aug 29 07:53:40 2015 (3089) Master watcher caught SIGHUP.  Re-opening log files.
Aug 29 07:53:42 2015 (3091) BounceRunner qrunner caught SIGHUP.  Reopening logs.
Aug 29 07:53:42 2015 (3094) NewsRunner qrunner caught SIGHUP.  Reopening logs.
Aug 29 07:53:43 2015 (3092) CommandRunner qrunner caught SIGHUP.  Reopening logs.
Aug 29 07:53:43 2015 (3097) RetryRunner qrunner caught SIGHUP.  Reopening logs.
Aug 29 07:53:43 2015 (3093) IncomingRunner qrunner caught SIGHUP.  Reopening logs.
Aug 29 07:53:43 2015 (3090) ArchRunner qrunner caught SIGHUP.  Reopening logs.

huh?
# zcat  qrunner.4.gz
# zcat  qrunner.5.gz
# ls -lrt qr*
-rw-rw-r-- 1 list list  20 Aug 23 07:58 qrunner.7.gz
-rw-rw-r-- 1 list list  20 Aug 24 08:06 qrunner.6.gz
-rw-rw-r-- 1 list list  20 Aug 25 07:48 qrunner.5.gz
-rw-rw-r-- 1 list list  20 Aug 26 07:39 qrunner.4.gz
-rw-rw-r-- 1 list list 151 Aug 27 16:15 qrunner.3.gz
-rw-rw-r-- 1 list list 194 Aug 28 08:04 qrunner.2.gz
-rw-rw-r-- 1 list list 729 Aug 29 07:53 qrunner.1
-rw-rw-r-- 1 list list 729 Aug 30 07:37 qrunner

Eric

From mark at msapiro.net  Sun Aug 30 08:18:49 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 29 Aug 2015 23:18:49 -0700
Subject: [Mailman-Users] old messages sent to list on reboot
In-Reply-To: <mailbox-21530-1440914150-14620@pepper.0.0.200>
References: <20150828055319.GA4244@biovolt.nl> <55E2626A.7060904@msapiro.net>
 <mailbox-21530-1440914150-14620@pepper.0.0.200>
Message-ID: <55E2A049.3090400@msapiro.net>

On 08/29/2015 11:01 PM, Eric Smith wrote:
> 
> I started reboot on 27 August at 16:14;
> # zcat  qrunner.3.gz
> Aug 27 16:15:25 2015 (3092) CommandRunner qrunner started.
> Aug 27 16:15:25 2015 (3094) NewsRunner qrunner started.
> Aug 27 16:15:25 2015 (3090) ArchRunner qrunner started.
> Aug 27 16:15:25 2015 (3091) BounceRunner qrunner started.
> Aug 27 16:15:25 2015 (3093) IncomingRunner qrunner started.
> Aug 27 16:15:25 2015 (3097) RetryRunner qrunner started.
> Aug 27 16:15:26 2015 (3096) VirginRunner qrunner started.
> Aug 27 16:15:26 2015 (3095) OutgoingRunner qrunner started.


And 8 runners were started.


> # zcat  qrunner.2.gz
> Aug 28 08:04:29 2015 (3097) RetryRunner qrunner caught SIGHUP.  Reopening logs.
> Aug 28 08:04:29 2015 (3089) Master watcher caught SIGHUP.  Re-opening log files.
> Aug 28 08:04:29 2015 (3092) CommandRunner qrunner caught SIGHUP.  Reopening logs.
> Aug 28 08:04:29 2015 (3096) VirginRunner qrunner caught SIGHUP.  Reopening logs.
> Aug 28 08:04:29 2015 (3091) BounceRunner qrunner caught SIGHUP.  Reopening logs.
> Aug 28 08:04:29 2015 (3095) OutgoingRunner qrunner caught SIGHUP.  Reopening logs.
> Aug 28 08:04:29 2015 (3094) NewsRunner qrunner caught SIGHUP.  Reopening logs.
> Aug 28 08:04:29 2015 (3093) IncomingRunner qrunner caught SIGHUP.  Reopening logs.
> Aug 28 08:04:29 2015 (3090) ArchRunner qrunner caught SIGHUP.  Reopening logs.

Then the next morning Mailman's logs were rotated.

> # zcat  qrunner.1.gz
> pepper:/var/log/mailman# cat  qrunner.1
> Aug 29 07:53:39 2015 (3096) VirginRunner qrunner caught SIGHUP.  Reopening logs.
> Aug 29 07:53:39 2015 (3095) OutgoingRunner qrunner caught SIGHUP.  Reopening logs.
> Aug 29 07:53:40 2015 (3089) Master watcher caught SIGHUP.  Re-opening log files.
> Aug 29 07:53:42 2015 (3091) BounceRunner qrunner caught SIGHUP.  Reopening logs.
> Aug 29 07:53:42 2015 (3094) NewsRunner qrunner caught SIGHUP.  Reopening logs.
> Aug 29 07:53:43 2015 (3092) CommandRunner qrunner caught SIGHUP.  Reopening logs.
> Aug 29 07:53:43 2015 (3097) RetryRunner qrunner caught SIGHUP.  Reopening logs.
> Aug 29 07:53:43 2015 (3093) IncomingRunner qrunner caught SIGHUP.  Reopening logs.
> Aug 29 07:53:43 2015 (3090) ArchRunner qrunner caught SIGHUP.  Reopening logs.


And again the following Morning.


> huh?
> # zcat  qrunner.4.gz
> # zcat  qrunner.5.gz
> # ls -lrt qr*
> -rw-rw-r-- 1 list list  20 Aug 23 07:58 qrunner.7.gz
> -rw-rw-r-- 1 list list  20 Aug 24 08:06 qrunner.6.gz
> -rw-rw-r-- 1 list list  20 Aug 25 07:48 qrunner.5.gz
> -rw-rw-r-- 1 list list  20 Aug 26 07:39 qrunner.4.gz
> -rw-rw-r-- 1 list list 151 Aug 27 16:15 qrunner.3.gz
> -rw-rw-r-- 1 list list 194 Aug 28 08:04 qrunner.2.gz
> -rw-rw-r-- 1 list list 729 Aug 29 07:53 qrunner.1
> -rw-rw-r-- 1 list list 729 Aug 30 07:37 qrunner


You haven't shown 'cat qrunner', but it presumably is just another
logrotate set like 'cat  qrunner.1' only a day later.

The strange (interesting?) thing is all of qrunner.4.gz through
qrunner.7.gz are gzips of empty files. logrotate ran because the log was
rotated, but no SIGHUP entries were written.

My best guess is this is because Mailman wasn't running between Aug 23
07:58 and the reboot time and probably for some time before that which
is why messages were just queued and not processed until you rebooted
which started Mailman.

If you have no older Mailman logs, we don't know why or exactly when
Mailman stopped, but I'm reasonably sure that's what happened.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From nelsonkelly446 at gmail.com  Sun Aug 30 11:06:26 2015
From: nelsonkelly446 at gmail.com (Nelson Kelly)
Date: Sun, 30 Aug 2015 02:06:26 -0700
Subject: [Mailman-Users] ISO speciific RegExp to filter/discard bot
 subscribe requests
In-Reply-To: <55E2541B.6070808@msapiro.net>
References: <55DEDD33.5020207@gmail.com> <55E2541B.6070808@msapiro.net>
Message-ID: <55E2C792.6050801@gmail.com>

On 8/29/2015 5:53 PM, Mark Sapiro wrote:
> On 08/27/2015 02:49 AM, Nelson Kelly wrote:
>
>> Can anyone recommend an example of a RegExp entry which would
>> prevent/solve this dilemma?
>
> Yes.
>
> ^.*\+\d{3,}@
>
> See
> <https://mail.python.org/pipermail/mailman-users/2015-August/079668.html> for
> a bit more.
>

Inserted the above recommended RegExp string into the ban_list, and 
within minutes subscribe request bot spam began showing up in the mod 
queue.

All the new spams appear to be of a slightly different format from which 
I described in the OP.

blahblah+blah-blah-blah-blah-12345678 at gmail.com
blah_12_34+blah-blah-blah-blah-12345678 at hotmail.com

Indeed I need to learn how to generate my own Regular Expressions. But 
until then I'll rely on your advice. I can post the actual list of spam 
addresses if that is preferred. Thanks.


From steve at pearwood.info  Sun Aug 30 13:43:07 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 30 Aug 2015 21:43:07 +1000
Subject: [Mailman-Users] ISO speciific RegExp to filter/discard bot
	subscribe requests
In-Reply-To: <55E2C792.6050801@gmail.com>
References: <55DEDD33.5020207@gmail.com> <55E2541B.6070808@msapiro.net>
 <55E2C792.6050801@gmail.com>
Message-ID: <20150830114307.GB19373@ando.pearwood.info>

On Sun, Aug 30, 2015 at 02:06:26AM -0700, Nelson Kelly wrote:
[...]
> Inserted the above recommended RegExp string into the ban_list, and 
> within minutes subscribe request bot spam began showing up in the mod 
> queue.
> 
> All the new spams appear to be of a slightly different format from which 
> I described in the OP.
> 
> blahblah+blah-blah-blah-blah-12345678 at gmail.com
> blah_12_34+blah-blah-blah-blah-12345678 at hotmail.com

Try this regex instead:

^.*\+.*?\d{3,}@


The meaning of it is:

^	start of string
.*	any number of characters
\+	a literal plus sign
.*?	any number of characters (non-greedy)
\d{3,}	at least three digits
@	a literal at sign


I'm not sure if the difference between "non-greedy" .*? and "greedy" .* 
is important in this case.

Good luck!



-- 
Steve

From Ralf at Bergs.biz  Sun Aug 30 10:59:19 2015
From: Ralf at Bergs.biz (Ralf G. R. Bergs)
Date: Sun, 30 Aug 2015 10:59:19 +0200
Subject: [Mailman-Users] How to prevent messages from being "held"
In-Reply-To: <55E26A70.2070106@msapiro.net>
References: <55DC51B2.4050004@bergs.biz> <55E0B0E7.8020405@tuunq.com>
 <55E0B99E.9050806@bergs.biz> <55E26A70.2070106@msapiro.net>
Message-ID: <55E2C5E7.1060200@Bergs.biz>

Hi Mark.

On 2015-08-30 04:29 , Mark Sapiro wrote:
> The handler doing this may be in the GLOBAL_PIPELINE or a list
> specific pipeline attribute. You need to remove it from whatever
> pipeline specifies it. 
You were right, thanks so much.

This was on an old Debian box, and it had the following in mm_cfg.py:
> GLOBAL_PIPELINE.insert(1, 'SpamAssassin')
I've commented that out now.

Thanks for your help.

KR,

Ralf




From mark at msapiro.net  Sun Aug 30 18:05:48 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 30 Aug 2015 09:05:48 -0700
Subject: [Mailman-Users] ISO speciific RegExp to filter/discard bot
 subscribe requests
In-Reply-To: <20150830114307.GB19373@ando.pearwood.info>
References: <55DEDD33.5020207@gmail.com> <55E2541B.6070808@msapiro.net>
 <55E2C792.6050801@gmail.com> <20150830114307.GB19373@ando.pearwood.info>
Message-ID: <55E329DC.3070901@msapiro.net>

On 08/30/2015 04:43 AM, Steven D'Aprano wrote:
> On Sun, Aug 30, 2015 at 02:06:26AM -0700, Nelson Kelly wrote:
>>
>> All the new spams appear to be of a slightly different format from which 
>> I described in the OP.
>>
>> blahblah+blah-blah-blah-blah-12345678 at gmail.com
>> blah_12_34+blah-blah-blah-blah-12345678 at hotmail.com


I'm now seeing these too.


> Try this regex instead:
> 
> ^.*\+.*?\d{3,}@
> 
> 
> The meaning of it is:
> 
> ^	start of string
> .*	any number of characters
> \+	a literal plus sign
> .*?	any number of characters (non-greedy)
> \d{3,}	at least three digits
> @	a literal at sign
> 
> 
> I'm not sure if the difference between "non-greedy" .*? and "greedy" .* 
> is important in this case.


It doesn't matter here. It would matter if there were groups. E.g.,

^.*\+(.*?)(\d{3,})@

In this case, the (.*?) group would match everything after the '+' up to
and not including the digits and the (\d{3,}) group would match all the
digits.

If the first group were greedy, i.e. (.*) without the ?, it would match
up to the last 3 digits and the (\d{3,}) group would match only the last
3 digits.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From paul at tokyoprogressive.org  Mon Aug 31 11:29:39 2015
From: paul at tokyoprogressive.org (Paul Arenson/tokyoprogressive)
Date: Mon, 31 Aug 2015 18:29:39 +0900
Subject: [Mailman-Users] Why do my posts to own mailman list disappear.
In-Reply-To: <3E918CBF-C737-42E7-9285-D5C97E4E4275@tokyoprogressive.org>
References: <mailman.528.1440095263.28099.mailman-users@python.org>
 <3E918CBF-C737-42E7-9285-D5C97E4E4275@tokyoprogressive.org>
Message-ID: <56E8D35B-DC7E-491D-B480-2A800DFDFCEA@tokyoprogressive.org>

Thank you to Steve Turnbull and Mark Shapiro for helping me brainstorm why my mailing list was not working.

It turns out it was not a Mailman problem after all. my provider sent me new server info that went straight into my spam box,
and so when it was time to update my mail server (send and receive both new), I stopped getting mail and being able to send mail.

Now that I have updated the servers, all is well.  Well, one thing is odd: no system messages came into mailman until I contacted support and got the instructions. In face, all became normal IN THE MAILMAN INTERFACE at least once I had contacted support.  I don?t see how my being able to send and receive would affect system messages just coming into Mailman itself?Although the messages started again from the time I was told the new server info by support, so until then they were being lost.

I don?t know what to make of that, but am glad it is running again.  

Thanks Mark and Steve!


paul

From ricardo.rodriguez at idisantiago.es  Mon Aug 31 14:58:51 2015
From: ricardo.rodriguez at idisantiago.es (=?UTF-8?Q?=5BIDIS_Technical_Secretariat=5D_Ricardo_Rodr=C3=ADguez?=)
Date: Mon, 31 Aug 2015 14:58:51 +0200
Subject: [Mailman-Users] importing archived Maildir email lists into
 Mailman lists archives
In-Reply-To: <55E265E9.3040305@msapiro.net>
References: <CAFPUu6zTDhxxqkyfvaa-DUAhxrgjzSMyM4U2cSHmsh2gAmA44A@mail.gmail.com>
 <55E265E9.3040305@msapiro.net>
Message-ID: <CAFPUu6wyzUa5_aGyTu3Pf4ySReJdh=97xAFi2u=Avb=zwbkLwQ@mail.gmail.com>

Thanks Mark, all,

On Sun, Aug 30, 2015 at 4:09 AM, Mark Sapiro <mark at msapiro.net> wrote:

> On 08/28/2015 12:52 AM, [IDIS Technical Secretariat] Ricardo Rodr?guez
> wrote:
> >
> > I have a number of Maildir format mailing lists archives and I would like
> > to add them to the archives of the same lists created in a new Mailman
> > installation.
>
>
> This search
> <https://www.google.com/?gws_rd=ssl#q=convert+maildir+to+mbox> should
> return something useful.
>

This has been more or less my entry point to this issue. Perhaps I've not
been able to explain the issue correctly or I'm missing something. What I'm
looking to move into mbox is a set of archives of Maildir mailing lists.
Those archives don't contain new, cur and tmp folders all the scripts I've
spotted at looked for. This is the structure of each archived email list
folder I have here...

Ricardo-Rodriguezs-Mac-Pro:r.users rrodriguez$ ls
Log bounce digissue headerremove lock mod outlocal remote
allow bouncer dignum indexed lockbounce modsub owner subscribers
archive config editor inlocal mailinglist num prefix text
archived digest headeradd key manager outhost public tstdig

Within /archive, there are two folders, 0 and 1, with a number of files,
each of them containing one message, and an index file.

Please, does this made sense for you? Am I completely lost? How could I
deal with this "archives" to move them to mbox files and get them imported
our brand new Mailman server?

Thank you very much for your help!

Ricardo


>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/ricardo.rodriguez%40idisantiago.es




-- 
Ricardo Rodr?guez
Research Management and Promotion Technician
Technical Secretariat
Health Research Institute of Santiago de Compostela (IDIS)
http://www.idisantiago.es

From wetz.peter at gmail.com  Mon Aug 31 15:03:06 2015
From: wetz.peter at gmail.com (Peter Wetz)
Date: Mon, 31 Aug 2015 15:03:06 +0200
Subject: [Mailman-Users] {Disarmed} Re: Archive page not showing dates
	properly
In-Reply-To: <55E22DAB.1040404@msapiro.net>
References: <CAGmyAkAB7h3YSVVm4LSpHt-7mQ8tFQD5N-Ok=3R8BQuJfVSNEQ@mail.gmail.com>
 <55D73CC9.1050303@msapiro.net>
 <CAGmyAkCGsQrt3o++3+_2t-tnhsq-Sm6A=UtH0m_EDPzi28gD+w@mail.gmail.com>
 <55DB316D.9080905@msapiro.net>
 <CAGmyAkAWMCezhu4VjzamcTch3yjApDsiH9pN8GEi_EVjuodApg@mail.gmail.com>
 <55E22DAB.1040404@msapiro.net>
Message-ID: <CAGmyAkAPnsyAP5BYz21oT0MqLo2SpSZCLK89AiqS+15HspadTg@mail.gmail.com>

thanks. that did the trick. works smoothly now.

On Sun, Aug 30, 2015 at 12:09 AM, Mark Sapiro <mark at msapiro.net> wrote:

> On 08/24/2015 11:35 PM, Peter Wetz wrote:
> >
> > hm, I get this error though:
> >
> ...
> > File "/var/lib/mailman/Mailman/Archiver/HyperArch.py", line 1072, in
> > write_index_entry
> > 'date': time.strftime("%d %b %Y %H:%M:%S", time.localtime(article.date)),
> > TypeError: a float is required
> >
> > "a float is required"
> > this implies that "article.date" is not a float in my case. maybe it
> > needs to be converted to a float explicitly?
> > any other suggestions?
>
>
> No. Converting it to float is the appropriate thing. e.g.
>
> >    'date': time.strftime("%d %b %Y %H:%M:%S",
> time.localtime(float(article.date))),
>
> The issue is that while article.date is a *nix timestamp, it is a string
> rather than an int or float.
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
>

From andrew.stuart at supercoders.com.au  Mon Aug 31 23:08:11 2015
From: andrew.stuart at supercoders.com.au (Andrew Stuart)
Date: Tue, 1 Sep 2015 07:08:11 +1000
Subject: [Mailman-Users] Pluggable authentication for Mailman web
	interface?
In-Reply-To: <854480980.1021293.1440619708372.JavaMail.zimbra@lafayette.edu>
References: <854480980.1021293.1440619708372.JavaMail.zimbra@lafayette.edu>
Message-ID: <FF2F6931-A530-4A78-BB73-7D6DA8353B6F@supercoders.com.au>

Can you say more about what you are trying to achieve?

There is an authenticating reverse proxy server for the Mailman REST API at https://gitlab.com/astuart/mailmania

But I don?t think anyone has run it yet - it?s pretty raw, not much more than alpha but fully functional.

I?m sorry but I?ve been dragged to other priorities so there?s no real documentation but I?m happy to answer any questions if you want to give it a try.

This thread really should like on Mailman Developers <Mailman-Developers at python.org> though.

as




On 27 Aug 2015, at 6:08 am, Waldbieser, Carl <waldbiec at lafayette.edu> wrote:


Are there any guidelines for adding authentication and /or authorization mechanisms to the Mailman web user interface?
Specifically, I was wondering if there is any kind of guidance for authenticating the user via an HTTP header (e.g. HTTP_REMOTE_USER) so that an authenticating reverse proxy could be placed in front of the Mailman web interface.

If there is no such built-in mechanism or pluggable mechanism, is there any kind of guidance on how the existing authentication mechanism might be replaced from a technical standpoint?

Thanks,
Carl Waldbieser
ITS Systems Programmer
Lafayette College
------------------------------------------------------
Mailman-Users mailing list Mailman-Users at python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: https://mail.python.org/mailman/options/mailman-users/andrew.stuart%40supercoders.com.au


From dmagda at ee.ryerson.ca  Mon Aug 31 20:09:21 2015
From: dmagda at ee.ryerson.ca (David Magda)
Date: Mon, 31 Aug 2015 14:09:21 -0400
Subject: [Mailman-Users] UnicodeDecodeError with Mailman 2.1 and Python 2.6
Message-ID: <19fb0150b67e7f3f4db974b0df615bfc.squirrel@webmail.ee.ryerson.ca>

Hello,

We recently upgrade from Debian 5 to 6, and are now having issues with
Mailman. Messages are still flowing properly, but the web archives are not
being generated since the upgrade.

When I run 'bin/arch mylistname' I get the following:

[...]
figuring article archives
2005-October
/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py:176:
UnicodeWarning: Unicode equal comparison failed to convert both arguments
to Unicode - interpreting them as being unequal
  self.dict = marshal.load(fp)
/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py:74:
UnicodeWarning: Unicode equal comparison failed to convert both arguments
to Unicode - interpreting them as being unequal
  self.sorted.sort()
Updating index files for archive [2004-December]
[...]
Updating HTML for article 214
Pickling archive state into
/usr/local/mailman-2.1.20/archives/private/mylistname/pipermail.pck
Traceback (most recent call last):
  File "bin/arch", line 201, in <module>
    main()
  File "bin/arch", line 189, in main
    archiver.processUnixMailbox(fp, start, end)
  File "/usr/local/mailman-2.1.20/Mailman/Archiver/pipermail.py", line
586, in processUnixMailbox
    self.add_article(a)
  File "/usr/local/mailman-2.1.20/Mailman/Archiver/pipermail.py", line
638, in add_article
    article.parentID = parentID = self.get_parent_info(arch, article)
  File "/usr/local/mailman-2.1.20/Mailman/Archiver/pipermail.py", line
658, in get_parent_info
    if self.database.hasArticle(archive, article.in_reply_to):
  File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line
279, in hasArticle
    self.__openIndices(archive)
  File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line
257, in __openIndices
    t = DumbBTree(os.path.join(arcdir, archive + '-' + i))
  File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line
66, in __init__
    self.load()
  File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line
185, in load
    self.__sort(dirty=1)
  File "/usr/local/mailman-2.1.20/Mailman/Archiver/HyperDatabase.py", line
74, in __sort
    self.sorted.sort()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xea in position 3:
ordinal not in range(128)

We're running 2.1.13, but as you can see, the problem persists even with
the latest release. We're using the stock Debian Python (2.6.6), but
Mailman for tarballs.

Thanks for any info.

Regards,
David