From avhernandez at uci.cu  Sun Nov  1 02:28:54 2015
From: avhernandez at uci.cu (=?UTF-8?Q?Amaury_Viera_Hern=c3=a1ndez?=)
Date: Sun, 1 Nov 2015 02:28:54 -0500
Subject: [Mailman-Users] [MASSMAIL] Re:  restrict subscription,
 unsubscription and send mails in mailman list
In-Reply-To: <5634E341.7020505@msapiro.net>
References: <296351665.3097696.1446236056480.JavaMail.zimbra@uci.cu>
 <5634E341.7020505@msapiro.net>
Message-ID: <5635BF36.307@uci.cu>

On 31/10/15 11:50, Mark Sapiro wrote:
> On 10/30/2015 01:14 PM, Amaury Viera Hern?ndez wrote:
>> Hi everyone:
>> I just arrived to this list and I'm needing your help. I have a mailman list and I need 3 things (I couldn't find this in google and in http://wiki.list.org/DOC/Frequently%20Asked%20Questions):
>> 1- I need that the users couldn't subscribe to the list
>
> In the web admin UI for the list set Privacy options... -> Subscription
> rules -> subscribe_policy to Require approval and then reject or discard
> any subscription requests.
>
>
>> 2- I need that the subscribed users can't unsubscribe
>
> In the web admin UI for the list set Privacy options... -> Subscription
> rules -> unsubscribe_policy to Yes and then reject or discard any
> unsubscription requests.
>
>
>> 3- I need that just some users can send mails to the list
>
> In the web admin UI for the list set Privacy options... -> Sender
> filters set default_member_moderation to Yes and
> member_moderation_action to Reject or Discard as desired, and if Reject,
> maybe set member_moderation_notice.
>
> Then go to Membership Management... -> Membership List and under
> Additional Member Tasks set everyone moderated and then uncheck 'mod'
> for those allowed to post.
>
> Also, see <http://wiki.list.org/x/4030685>, particularly the sections
> "How to restrict the list so only authorized persons can post:" and "How
> to post to the announcement list:".
>

Thanks for your quickly answer. Everything is fine but I have another 
problem now. I have created the mailman alias in MySQL. When a user 
request to the list administrator the subscription to the list the 
administrator, the list administrator is not receiving the mail. In the 
postfix logs appear this messages:
Nov  1 00:26:14 mailserver postfix/smtpd[7055]: connect from 
localhost[127.0.0.1]
Nov  1 00:26:15 mailserver postfix/smtpd[7055]: 30F2021402: 
client=localhost[127.0.0.1]
Nov  1 00:26:16 mailserver postfix/cleanup[7111]: 30F2021402: 
message-id=<mailman.0.1446351974.7879.mailman at dominio.pdc.cu>
Nov  1 00:26:16 mailserver postfix/qmgr[1953]: 30F2021402: 
from=<mailman-bounces at dominio.pdc.cu>, size=1200, nrcpt=2 (queue active)
Nov  1 00:26:16 mailserver postfix/smtpd[7055]: disconnect from 
localhost[127.0.0.1]
Nov  1 00:26:32 mailserver postfix/smtpd[7107]: connect from 
localhost[127.0.0.1]
Nov  1 00:26:32 mailserver postfix/smtpd[7107]: D814621432: 
client=localhost[127.0.0.1]
Nov  1 00:26:32 mailserver postfix/smtpd[7108]: connect from 
localhost[127.0.0.1]
Nov  1 00:26:32 mailserver postfix/cleanup[7061]: D814621432: 
message-id=<mailman.0.1446351974.7879.mailman at dominio.pdc.cu>
Nov  1 00:26:32 mailserver postfix/smtpd[7108]: E10EE21459: 
client=localhost[127.0.0.1]
Nov  1 00:26:32 mailserver postfix/qmgr[1953]: D814621432: 
from=<mailman-bounces at dominio.pdc.cu>, size=2649, nrcpt=1 (queue active)
Nov  1 00:26:32 mailserver postfix/smtpd[7107]: disconnect from 
localhost[127.0.0.1]
Nov  1 00:26:32 mailserver postfix/cleanup[7111]: E10EE21459: 
message-id=<mailman.0.1446351974.7879.mailman at dominio.pdc.cu>
Nov  1 00:26:32 mailserver amavis[1665]: (01665-06) Passed CLEAN 
{RelayedInternal}, MYNETS/MYUSERS LOCAL [127.0.0.1]:36203 [127.0.0.1] 
<mailman-bounces at dominio.pdc.cu> -> 
<mailman-owner at listas.dominio.pdc.cu>, Queue-ID: 30F2021402, Message-ID: 
<mailman.0.1446351974.7879.mailman at dominio.pdc.cu>, mail_id: 
d0iNH8sq7q2i, Hits: -0.621, size: 1200, queued_as: D814621432, 
dkim_new=dkim:dominio.pdc.cu, 16692 ms
Nov  1 00:26:32 mailserver postfix/qmgr[1953]: E10EE21459: 
from=<mailman-bounces at dominio.pdc.cu>, size=2623, nrcpt=1 (queue active)
Nov  1 00:26:32 mailserver postfix/smtpd[7108]: disconnect from 
localhost[127.0.0.1]
Nov  1 00:26:33 mailserver amavis[1667]: (01667-06) Passed CLEAN 
{RelayedInternal}, MYNETS/MYUSERS LOCAL [127.0.0.1]:36203 [127.0.0.1] 
<mailman-bounces at dominio.pdc.cu> -> <auditor at dominio.pdc.cu>, Queue-ID: 
30F2021402, Message-ID: 
<mailman.0.1446351974.7879.mailman at dominio.pdc.cu>, mail_id: 
8RQ7kxJdlJqn, Hits: -0.621, size: 1200, queued_as: E10EE21459, 
dkim_new=dkim:dominio.pdc.cu, 16727 ms
Nov  1 00:26:33 mailserver postfix/smtp[7064]: 30F2021402: 
to=<mailman-owner at listas.dominio.pdc.cu>, 
relay=127.0.0.1[127.0.0.1]:10024, delay=18, delays=1.3/0.01/0.02/17, 
dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 
2.0.0 Ok: queued as D814621432)
Nov  1 00:26:33 mailserver postfix/smtp[7065]: 30F2021402: 
to=<auditor at dominio.pdc.cu>, relay=127.0.0.1[127.0.0.1]:10024, delay=18, 
delays=1.3/0/0.02/17, dsn=2.0.0, status=sent (250 2.0.0 from 
MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as E10EE21459)
Nov  1 00:26:33 mailserver postfix/qmgr[1953]: 30F2021402: removed
Nov  1 00:26:33 mailserver postfix/pipe[7112]: E10EE21459: 
to=<auditor at dominio.pdc.cu>, relay=dovecot, delay=0.23, 
delays=0.07/0.04/0/0.12, dsn=2.0.0, status=sent (delivered via dovecot 
service)
Nov  1 00:26:33 mailserver postfix/qmgr[1953]: E10EE21459: removed
Nov  1 00:26:33 mailserver postfix/pipe[7917]: D814621432: 
to=<mailman-owner at listas.dominio.pdc.cu>, relay=mailman, delay=0.42, 
delays=0.06/0.12/0/0.25, dsn=2.0.0, status=sent (delivered via mailman 
service)
Nov  1 00:26:33 mailserver postfix/qmgr[1953]: D814621432: removed
Nov  1 00:26:34 mailserver postfix/smtpd[7055]: connect from 
localhost[127.0.0.1]
Nov  1 00:26:34 mailserver postfix/smtpd[7055]: NOQUEUE: reject: RCPT 
from localhost[127.0.0.1]: 550 5.1.0 
<mailman-bounces at listas.dominio.pdc.cu>: Sender address rejected: User 
unknown in virtual mailbox table; 
from=<mailman-bounces at listas.dominio.pdc.cu> 
to=<mailmanadmin at dominio.pdc.cu> proto=ESMTP 
helo=<mailserver.dominio.pdc.cu>
Nov  1 00:26:34 mailserver postfix/smtpd[7055]: disconnect from 
localhost[127.0.0.1]

my test domain is: dominio.pdc.cu

my mailman config is:

from Defaults import *
MAILMAN_SITE_LIST = 'mailman'
DEFAULT_URL_PATTERN = 'http://%s/cgi-bin/mailman/'
PRIVATE_ARCHIVE_URL = '/cgi-bin/mailman/private'
IMAGE_LOGOS         = '/images/mailman/'
DEFAULT_EMAIL_HOST = 'dominio.pdc.cu'
DEFAULT_URL_HOST   = 'listas.dominio.pdc.cu'
DEFAULT_SERVER_LANGUAGE = 'es'
USE_ENVELOPE_SENDER    = 0
DEFAULT_SEND_REMINDERS = 0
MTA='Postfix'

I have my mailman transport in mysql and the alias too

everything is fine, but the message to the list administrator is not 
sent when a user request a subscription

Thanks in advance, Amaury.

Noviembre 13-14: Final Caribe?a 2015 del Concurso de Programaci?n ACM-ICPC
https://icpc.baylor.edu/regionals/finder/cf-2015

From mark at msapiro.net  Sun Nov  1 11:31:47 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 1 Nov 2015 08:31:47 -0800
Subject: [Mailman-Users] [MASSMAIL] Re: restrict subscription,
 unsubscription and send mails in mailman list
In-Reply-To: <5635BF36.307@uci.cu>
References: <296351665.3097696.1446236056480.JavaMail.zimbra@uci.cu>
 <5634E341.7020505@msapiro.net> <5635BF36.307@uci.cu>
Message-ID: <56363E73.3050703@msapiro.net>

On 11/01/2015 12:28 AM, Amaury Viera Hern?ndez wrote:
> 
> Thanks for your quickly answer. Everything is fine but I have another
> problem now. I have created the mailman alias in MySQL.


I'm not sure what 'mailman' alias you created. This should not be
needed. See below.


> When a user
> request to the list administrator the subscription to the list the
> administrator, the list administrator is not receiving the mail. In the
> postfix logs appear this messages:
> Nov  1 00:26:14 mailserver postfix/smtpd[7055]: connect from
> localhost[127.0.0.1]
> Nov  1 00:26:15 mailserver postfix/smtpd[7055]: 30F2021402:
> client=localhost[127.0.0.1]
> Nov  1 00:26:16 mailserver postfix/cleanup[7111]: 30F2021402:
> message-id=<mailman.0.1446351974.7879.mailman at dominio.pdc.cu>
> Nov  1 00:26:16 mailserver postfix/qmgr[1953]: 30F2021402:
> from=<mailman-bounces at dominio.pdc.cu>, size=1200, nrcpt=2 (queue active)
> Nov  1 00:26:16 mailserver postfix/smtpd[7055]: disconnect from
> localhost[127.0.0.1]


Above is receipt of message-id
<mailman.0.1446351974.7879.mailman at dominio.pdc.cu> (a Mailman generated
ID) apparently from the 'mailman' list with Postfix ID 30F2021402.


> Nov  1 00:26:32 mailserver postfix/smtpd[7107]: connect from
> localhost[127.0.0.1]
> Nov  1 00:26:32 mailserver postfix/smtpd[7107]: D814621432:
> client=localhost[127.0.0.1]
> Nov  1 00:26:32 mailserver postfix/smtpd[7108]: connect from
> localhost[127.0.0.1]
> Nov  1 00:26:32 mailserver postfix/cleanup[7061]: D814621432:
> message-id=<mailman.0.1446351974.7879.mailman at dominio.pdc.cu>
> Nov  1 00:26:32 mailserver postfix/smtpd[7108]: E10EE21459:
> client=localhost[127.0.0.1]
> Nov  1 00:26:32 mailserver postfix/qmgr[1953]: D814621432:
> from=<mailman-bounces at dominio.pdc.cu>, size=2649, nrcpt=1 (queue active)
> Nov  1 00:26:32 mailserver postfix/smtpd[7107]: disconnect from
> localhost[127.0.0.1]
> Nov  1 00:26:32 mailserver postfix/cleanup[7111]: E10EE21459:
> message-id=<mailman.0.1446351974.7879.mailman at dominio.pdc.cu>

And above are two more receipts of the same message with Postfix IDs
D814621432 and E10EE21459.


> Nov  1 00:26:32 mailserver amavis[1665]: (01665-06) Passed CLEAN
> {RelayedInternal}, MYNETS/MYUSERS LOCAL [127.0.0.1]:36203 [127.0.0.1]
> <mailman-bounces at dominio.pdc.cu> ->
> <mailman-owner at listas.dominio.pdc.cu>, Queue-ID: 30F2021402, Message-ID:
> <mailman.0.1446351974.7879.mailman at dominio.pdc.cu>, mail_id:
> d0iNH8sq7q2i, Hits: -0.621, size: 1200, queued_as: D814621432,
> dkim_new=dkim:dominio.pdc.cu, 16692 ms

This is Postfix ID 30F2021402 passing through amavis and being requeued
as D814621432 (out of sequence as D814621432 was received above).


> Nov  1 00:26:32 mailserver postfix/qmgr[1953]: E10EE21459:
> from=<mailman-bounces at dominio.pdc.cu>, size=2623, nrcpt=1 (queue active)
> Nov  1 00:26:32 mailserver postfix/smtpd[7108]: disconnect from
> localhost[127.0.0.1]
> Nov  1 00:26:33 mailserver amavis[1667]: (01667-06) Passed CLEAN
> {RelayedInternal}, MYNETS/MYUSERS LOCAL [127.0.0.1]:36203 [127.0.0.1]
> <mailman-bounces at dominio.pdc.cu> -> <auditor at dominio.pdc.cu>, Queue-ID:
> 30F2021402, Message-ID:
> <mailman.0.1446351974.7879.mailman at dominio.pdc.cu>, mail_id:
> 8RQ7kxJdlJqn, Hits: -0.621, size: 1200, queued_as: E10EE21459,
> dkim_new=dkim:dominio.pdc.cu, 16727 ms

And here 30F2021402 passes through amavis and is requeued as E10EE21459.

It appears there may have been some duplication here for some reason as
the original 30F2021402 was delivered twice to amavis.


> Nov  1 00:26:33 mailserver postfix/smtp[7064]: 30F2021402:
> to=<mailman-owner at listas.dominio.pdc.cu>,
> relay=127.0.0.1[127.0.0.1]:10024, delay=18, delays=1.3/0.01/0.02/17,
> dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250
> 2.0.0 Ok: queued as D814621432)

Here (out of sequence) the original 30F2021402 is received from amavis
and requeued as D814621432 for mailman-owner at listas.dominio.pdc.cu.


> Nov  1 00:26:33 mailserver postfix/smtp[7065]: 30F2021402:
> to=<auditor at dominio.pdc.cu>, relay=127.0.0.1[127.0.0.1]:10024, delay=18,
> delays=1.3/0/0.02/17, dsn=2.0.0, status=sent (250 2.0.0 from
> MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as E10EE21459)

And similarly for 30F2021402 received from amavis and requeued as
E10EE21459 for auditor at dominio.pdc.cu.


> Nov  1 00:26:33 mailserver postfix/qmgr[1953]: 30F2021402: removed
> Nov  1 00:26:33 mailserver postfix/pipe[7112]: E10EE21459:
> to=<auditor at dominio.pdc.cu>, relay=dovecot, delay=0.23,
> delays=0.07/0.04/0/0.12, dsn=2.0.0, status=sent (delivered via dovecot
> service)

And here E10EE21459 for auditor at dominio.pdc.cu is delivered via dovecot.


> Nov  1 00:26:33 mailserver postfix/qmgr[1953]: E10EE21459: removed
> Nov  1 00:26:33 mailserver postfix/pipe[7917]: D814621432:
> to=<mailman-owner at listas.dominio.pdc.cu>, relay=mailman, delay=0.42,
> delays=0.06/0.12/0/0.25, dsn=2.0.0, status=sent (delivered via mailman
> service)

And here, D814621432 for mailman-owner at listas.dominio.pdc.cu is
delivered to Mailman by the 'mailman' transport (presumably
postfix-to-mailman.py, see <http://wiki.list.org/x/15564817>,
particularly the 3rd paragraph).


> Nov  1 00:26:33 mailserver postfix/qmgr[1953]: D814621432: removed
> Nov  1 00:26:34 mailserver postfix/smtpd[7055]: connect from
> localhost[127.0.0.1]
> Nov  1 00:26:34 mailserver postfix/smtpd[7055]: NOQUEUE: reject: RCPT
> from localhost[127.0.0.1]: 550 5.1.0
> <mailman-bounces at listas.dominio.pdc.cu>: Sender address rejected: User
> unknown in virtual mailbox table;
> from=<mailman-bounces at listas.dominio.pdc.cu>
> to=<mailmanadmin at dominio.pdc.cu> proto=ESMTP
> helo=<mailserver.dominio.pdc.cu>


And finally, we get to what I think your problem is. The message to
mailman-owner has passed through mailman and is being resent to the
owner of the list, mailmanadmin at dominio.pdc.cu, but this is rejected by
Postfix. It is rejected because the sender,
mailman-bounces at listas.dominio.pdc.cu is an 'unknown' address (it is
only known via the 'mailman' transport. `postconf -n` output would help,
but presumably this occurs because you have specified something in
Postfix smtpd_sender_restrictions that is incompatible with
postfix-to-mailman.py delivery since the 'listname-bounces' addresses
are not being allowed as senders.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From lists at damorris.com  Mon Nov  2 05:25:00 2015
From: lists at damorris.com (Adam Morris)
Date: Mon, 2 Nov 2015 21:25:00 +1100
Subject: [Mailman-Users] attachments being sent with messages.
Message-ID: <563739FC.2000006@damorris.com>

Hi all,

most messages sent to lists seem to have some sort of attachment.
If I go to non digest options and turn on the feature to scrub 
attachments if messages are sent using html there is no body to the message.
Is there a way to not have attachments in messages but still have html 
messages have their body sent to lists?


-- 


Adam Morris

email:
adam at damorris.com


		


From mark at msapiro.net  Mon Nov  2 11:15:50 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 2 Nov 2015 08:15:50 -0800
Subject: [Mailman-Users] attachments being sent with messages.
In-Reply-To: <563739FC.2000006@damorris.com>
References: <563739FC.2000006@damorris.com>
Message-ID: <56378C36.1090509@msapiro.net>

On 11/02/2015 02:25 AM, Adam Morris wrote:
> Hi all,
> 
> most messages sent to lists seem to have some sort of attachment.
> If I go to non digest options and turn on the feature to scrub
> attachments if messages are sent using html there is no body to the
> message.


See the FAQ article at <http://wiki.list.org/x/4030707>.


> Is there a way to not have attachments in messages but still have html
> messages have their body sent to lists?


Configure the list to not add msg_header or msg_footer. I.e., set those
attributes empty in the web admin UI Non-digest options section.

Or, if you just want the text and don't care about the HTML formatting,
use content filtering to accept text/html but collapse alternatives and
convert html to plain text.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From her at adm.ku.dk  Tue Nov  3 09:32:18 2015
From: her at adm.ku.dk (Henrik Rasmussen)
Date: Tue, 3 Nov 2015 14:32:18 +0000
Subject: [Mailman-Users] Part of moderated mails postponed to later digests
Message-ID: <6DCC3E5DA06FE346B4DE4876C4F2713D014D8582EF@P2KITMBX06WC03.unicph.domain>

One of my list administrators complains that digest mails now seems to be sent in fragmented chunks and instead of sending all moderated mails in a digest the same day, only some of the mails will be sent with a digest the same day, and the rest will be sent in a digest with the next moderated mails, half a week later. For instance, when moderating say 50 mails in week 1 and 50 mails in week 2, the digest mail in week 1 will contain say 30 mails in a digest, while the digest in week 2 will contain like 20 mails from week 1 and like 10 mails from week 2.

There is no information about when the admin discovered the problem, but on August 20th 2015 Mailman has been upgraded from 3:2.1.12-18.el6 to 3:2.1.12-25.el6 and I wonder if the DMARC implementation in 3:2.1.12-22.el6 has something to do with it, as this is the only changes I can think of having any effect on the mail behavior.

dmarc_quarantine_moderation_action is set to Yes on the list in particular and according to some discussions on the Internet, this strict action has been coursing bounces, so I suggested him to set this to No.

But even being strict, I don't see why this would course parts of the digest being held half a week, so something else must interfere.

Regards
Henrik Rasmussen

From mark at msapiro.net  Tue Nov  3 11:32:02 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 3 Nov 2015 08:32:02 -0800
Subject: [Mailman-Users] Part of moderated mails postponed to later
 digests
In-Reply-To: <6DCC3E5DA06FE346B4DE4876C4F2713D014D8582EF@P2KITMBX06WC03.unicph.domain>
References: <6DCC3E5DA06FE346B4DE4876C4F2713D014D8582EF@P2KITMBX06WC03.unicph.domain>
Message-ID: <5638E182.7080305@msapiro.net>

On 11/03/2015 06:32 AM, Henrik Rasmussen wrote:
> One of my list administrators complains that digest mails now seems to be sent in fragmented chunks and instead of sending all moderated mails in a digest the same day, only some of the mails will be sent with a digest the same day, and the rest will be sent in a digest with the next moderated mails, half a week later. For instance, when moderating say 50 mails in week 1 and 50 mails in week 2, the digest mail in week 1 will contain say 30 mails in a digest, while the digest in week 2 will contain like 20 mails from week 1 and like 10 mails from week 2.


My best guess is you are triggering digests on size (Digest options ->
digest_size_threshold) and either not periodically (digest_send_periodic
= No) or cron/senddigests runs only weekly.

50 messages are approved; the 30th message triggers a digest on size.
And the remaining wait until digest_size_threshold is reached again.
Then the next week a bunch more messages are approved and the 10th
message triggers a digest on size and the rest wait and until either
threshold is reached again or a periodic digest is sent.

As to why this is changed behavior, perhaps after the upgrade,
cron/senddigests is not being run for some reason to produce a daily
digest. Check Mailman's crontab, Mailman's error log and the system cron
log.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Wed Nov  4 11:06:58 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 4 Nov 2015 08:06:58 -0800
Subject: [Mailman-Users] Part of moderated mails postponed to later
 digests
In-Reply-To: <6DCC3E5DA06FE346B4DE4876C4F2713D014D858F18@P2KITMBX06WC03.unicph.domain>
References: <6DCC3E5DA06FE346B4DE4876C4F2713D014D8582EF@P2KITMBX06WC03.unicph.domain>
 <5638E182.7080305@msapiro.net>
 <6DCC3E5DA06FE346B4DE4876C4F2713D014D858F18@P2KITMBX06WC03.unicph.domain>
Message-ID: <563A2D22.3070005@msapiro.net>

On 11/04/2015 02:09 AM, Henrik Rasmussen wrote:
>> My best guess is you are triggering digests on size (Digest options -> digest_size_threshold) and either not periodically (digest_send_periodic = No) or cron/senddigests runs only weekly.
> 
> To clarify, he has set digest_size_threshold to 10000 and digest_send_periodic =Yes. /usr/lib/mailman/cron/senddigests is run by cron every day at noon.
> 
> Other digest settings is
> 
> nondigestable = True
> scrub_nondigest = False
> digestable = True
> digest_is_default = 1
> mime_is_default_digest = 0
> digest_size_threshhold = 10000
> digest_send_periodic = 1
> digest_volume_frequency = 3
> discard_these_nonmembers = (241 members)
> ban_list = (188 members)
> 
> The digest_size_threshhold = 10000 may likely course the problem, but is necessary as a digest containing a single mail is sent each time a mail is sent to the list-address as suggested by https://mail.python.org/pipermail/mailman-users/2014-August/077622.html.


That's a bug <https://bugs.launchpad.net/mailman/+bug/558274> which was
fixed in MM 2.1.16, but in any case, if you don't want digests triggered
on size and your mailman is older than 2.1.16, setting
digest_size_threshhold = 10000 (KB = 10 MB) should stop digests being
triggered on size unless your list has a lot of large graphics attached
to posts. I.e. 10MB / 100 posts is still 100KB per post.


Anyway, I understood your OP to say digests were being sent only weekly
when batches of held messages are approved. If as you say,
digest_send_periodic =Yes and /usr/lib/mailman/cron/senddigests is run
by cron every day at noon, even if 50 messages are approved and 30 of
them are sent immediately in a digest (triggered on size), the other 20
should be sent in a digest the next noon.

If this is not the case, either cron/senddigests is not running every
noon or it is somehow skipping this list or encountering errors with
this list. Is there anything in Mailman's error log?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mrbrklyn at panix.com  Tue Nov  3 21:42:09 2015
From: mrbrklyn at panix.com (Ruben Safir)
Date: Tue, 3 Nov 2015 21:42:09 -0500
Subject: [Mailman-Users] 64 bit encoding
Message-ID: <56397081.7030408@panix.com>

I'm converting my mailing lists from majordomo to mailman using postfix.
 A lot of little issues have cropped up but the biggest problem I have
currently is that it seems that mailman is converting files to 64 bit
which is screwing up my home cooked archives but good.

How you I stop it from doing this:


 <mailto:hangout-request at nylxs.com?subject=subscribe>
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
Errors-To: hangout-bounces at nylxs.com
Sender: "hangout" <hangout-bounces at nylxs.com>

T24gMTEvMDMvMjAxNSAxMTozOCBBTSwgUnViZW4gU2FmaXIgd3JvdGU6Cj4gTG93IGxldmVsIGF1
dG9tYXRpb24gdG9vbHMgZGV2ZWxvcGVyCj4g16nXnNeXINen15XXqNeV16og15fXmdeZ150KPiDX
lNeV16HXoyDXnNeh15wKPiBEZXZhbG9yZUV4cGVyaWVuY2VkIHdpdGggdG9vbHMgYW5kIGF1dG9t
YXRpb24gZnJhbWV3b3JrIGRldmVsb3BtZW50IGZvcgo+IGxvdyBsZXZlbCBzb2Z0d2FyZSBGVy9k
cml2ZXIKPiBFeGNlbGxlbnQgdW5kZXJzdGFuZGluZyBjb21wbGljYXRlZCBvZiBTVyBhcmNoaXRl
Y3R1cmUgSFcvRHJpdmVyL1NXCj4gKEV4Y2VsbGVudCBrbm93bGVkZ2UgaW4gQysrICg0KyB5ZWFy
cwo+IEdvb2Qga25vd2xlZGdlIGluIEMjIGFuZCBPT1AgLSBhZHZhbnRhZ2UgMisgeWVhcnMKPiBE
ZWVwIHVuZGVyc3RhbmRpbmcgaW4gTGludXggaW50ZXJuYWxzCj4gSGFuZHMgb24gSmF2YSBwcm9n
cmFtbWluZyBleHBlcmllbmNlCj4gQWR2YW50YWdlIDpGYW1pbGlhcml0eSB3aXRoIG5ldHdvcmtp
bmcgc3lzdGVtL3Byb3RvY29scwo+IEFkdmFudGFnZSA6RXhwZXJpZW5jZSB3aXRoIE9iamVjdCBP
cmllbnRlZCBtZXRob2RvbG9naWVzCj4gKEJpZyBBZHZhbnRhZ2UgOkZhbWlsaWFyaXR5IHdpdGgg
V2ktRmkgdGVjaG5vbG9naWVzICg4MDIuMTEKPiBHb29kIGRlYnVnZ2luZyBhbmQgcHJvYmxlbSBz
b2x2aW5nIHNraWxscwo+IEdvb2QgY29tbXVuaWNhdGlvbiBza2lsbHMgYW5kIGFiaWxpdHkgJiBk
ZXNpcmUgdG8gd29yayBhcyBhIHRlYW0gcGxheWVyCj4gYXJlIGEgbXVzdAo+IFRoZXNlIHBvc2l0

From mark at msapiro.net  Wed Nov  4 12:10:04 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 4 Nov 2015 09:10:04 -0800
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <56397081.7030408@panix.com>
References: <56397081.7030408@panix.com>
Message-ID: <563A3BEC.9090700@msapiro.net>

On 11/03/2015 06:42 PM, Ruben Safir wrote:
> I'm converting my mailing lists from majordomo to mailman using postfix.
>  A lot of little issues have cropped up but the biggest problem I have
> currently is that it seems that mailman is converting files to 64 bit
> which is screwing up my home cooked archives but good.


Actually, Mailman is sending messages in UTF-8 character set as base64
encoded. Actually, many MUAs will also base64 encode UTF-8 message bodies.

This is actually a python email library thing. You can change the
encoding for UTF-8 message bodies with the following patch to
Mailman/Message.py, but if your messages are not predominately ascii
text, you won't like the result any better.

diff -u 2.1/Mailman/Message.py 21/Mailman/Message.py
--- 2.1/Mailman/Message.py	2014-05-17 18:34:20.240159000 -0700
+++ 21/Mailman/Message.py	2015-11-04 08:50:31.772899979 -0800
@@ -36,6 +36,10 @@
 from Mailman import mm_cfg
 from Mailman import Utils

+email.Charset.add_charset('utf-8',
+                          email.Charset.SHORTEST,
+                          email.Charset.QP,
+                          'utf-8')
 COMMASPACE = ', '

 mo = re.match(r'([\d.]+)', email.__version__)

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mrbrklyn at panix.com  Wed Nov  4 12:16:48 2015
From: mrbrklyn at panix.com (Ruben Safir)
Date: Wed, 4 Nov 2015 12:16:48 -0500
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563A3BEC.9090700@msapiro.net>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
Message-ID: <563A3D80.4020704@panix.com>

On 11/04/2015 12:10 PM, Mark Sapiro wrote:
>  You can change the
> encoding for UTF-8 message bodies with the following patch to
> Mailman/Message.py, but if your messages are not predominate


Oh thank you!  Because I was reading that it is triggered by header
msg_header or msg_footer, but nobody really know what that was exactly.
 It didn't match any settings in the interface.  I greatly appreciate this

Ruben

From mark at msapiro.net  Wed Nov  4 12:27:47 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 4 Nov 2015 09:27:47 -0800
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563A3D80.4020704@panix.com>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <563A3D80.4020704@panix.com>
Message-ID: <563A4013.8060807@msapiro.net>

On 11/04/2015 09:16 AM, Ruben Safir wrote:
> 
> Oh thank you!  Because I was reading that it is triggered by header
> msg_header or msg_footer, but nobody really know what that was exactly.
>  It didn't match any settings in the interface.  I greatly appreciate this


To be more specific, if a list specifies a msg_header and/or a
msg_footer (in the list admin Non-digest options), Mailman will decode
the message body of a text/plain message in order to prepend the
msg_header and/or append the msg_footer. Then the message body is
re-encoded by the email library when Mailman sets the new body.

Thus, if msg_header and msg_footer are both empty, the recoding won't
happen, but when it does, it is controlled by the
email.Charset.add_charset() settings for the character set of the body,
and the patch I gave you changes the body encoding from base64 to
quoted-printable.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From her at adm.ku.dk  Wed Nov  4 05:09:32 2015
From: her at adm.ku.dk (Henrik Rasmussen)
Date: Wed, 4 Nov 2015 10:09:32 +0000
Subject: [Mailman-Users] Part of moderated mails postponed to later
 digests
In-Reply-To: <5638E182.7080305@msapiro.net>
References: <6DCC3E5DA06FE346B4DE4876C4F2713D014D8582EF@P2KITMBX06WC03.unicph.domain>
 <5638E182.7080305@msapiro.net>
Message-ID: <6DCC3E5DA06FE346B4DE4876C4F2713D014D858F18@P2KITMBX06WC03.unicph.domain>

> My best guess is you are triggering digests on size (Digest options -> digest_size_threshold) and either not periodically (digest_send_periodic = No) or cron/senddigests runs only weekly.

To clarify, he has set digest_size_threshold to 10000 and digest_send_periodic =Yes. /usr/lib/mailman/cron/senddigests is run by cron every day at noon.

Other digest settings is

nondigestable = True
scrub_nondigest = False
digestable = True
digest_is_default = 1
mime_is_default_digest = 0
digest_size_threshhold = 10000
digest_send_periodic = 1
digest_volume_frequency = 3
discard_these_nonmembers = (241 members)
ban_list = (188 members)

The digest_size_threshhold = 10000 may likely course the problem, but is necessary as a digest containing a single mail is sent each time a mail is sent to the list-address as suggested by https://mail.python.org/pipermail/mailman-users/2014-August/077622.html.

Regards Henrik

From stephen at xemacs.org  Wed Nov  4 19:26:35 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 5 Nov 2015 09:26:35 +0900
Subject: [Mailman-Users] [Mailman-Developers] Mailman and Box
In-Reply-To: <alpine.LRH.2.03.1511041419100.24833@conundrum.unh.edu>
References: <alpine.LRH.2.03.1511041419100.24833@conundrum.unh.edu>
Message-ID: <22074.41531.560633.920628@turnbull.sk.tsukuba.ac.jp>

Moving to mailman-users, be a better place to post a request for
experience.

Please respect cross-posting restrictions.  If you're just reporting
experience, reply-to is set to Mailman-Users which is appropriate (at
least, IIRC the Mailman Users/Developers lists don't mess with
preexisting Reply-To).  If you have patched Mailman to make it work
better in this kind of application, you *may* wish to move the
conversation back to Mailman Developers to propose inclusin in a
future version of Mailman.  (If you just want to say "hey I have a
patch", I would leave that on -Users, FWIW YMMV.)

Bill.Costa at unh.edu writes:
 > We will be retiring our Blackboard system (an e-learning portal)
 > which offered users 'organizations'; basically a combination of
 > group sharing of documents and collaboration via email.
 > 
 > I can't begin to duplicate all of the functionality of this part
 > of Blackboard, but I figured if I could link our school's Box
 > cloud storage with our new Mailman v2 installation, that would go
 > a long ways to provide a similar service.
 > 
 > The idea is that for every Mailman list created, a new Box shared
 > folder would also be created and associated with that list.  The
 > list owner and subscriber information would live in Mailman.  A
 > nightly reconciliation job would maintain the
 > list-owner/folder-owner roster and subscriber list between the
 > two systems.  Subscribers to the list would be granted, as a
 > whole, either reader or contributor access to the Box folder
 > (whatever the owner chooses).  The list-owner would be able to do
 > things like put the associated Box folder's URL into the list's
 > message footer.
 > 
 > Box provides a RESTful API, so I think I just might be able to
 > pull this off.
 > 
 > My question is, is there anybody else already doing this or
 > something similar?

From stephen at xemacs.org  Wed Nov  4 19:27:26 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 5 Nov 2015 09:27:26 +0900
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563A3BEC.9090700@msapiro.net>
References: <56397081.7030408@panix.com>
	<563A3BEC.9090700@msapiro.net>
Message-ID: <22074.41582.78231.830496@turnbull.sk.tsukuba.ac.jp>

Mark Sapiro writes:

 > +email.Charset.add_charset('utf-8',
 > +                          email.Charset.SHORTEST,
 > +                          email.Charset.QP,
 > +                          'utf-8')

I don't know if this actually is available, let alone whether it would
work (might affect headers, too, and of course elderly recipient
systems might throw a tantrum), but how about Content-Transfer-Encoding
of 8bit?


From mark at msapiro.net  Wed Nov  4 19:55:39 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 4 Nov 2015 16:55:39 -0800
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <22074.41582.78231.830496@turnbull.sk.tsukuba.ac.jp>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <22074.41582.78231.830496@turnbull.sk.tsukuba.ac.jp>
Message-ID: <563AA90B.7090307@msapiro.net>

On 11/04/2015 04:27 PM, Stephen J. Turnbull wrote:
> Mark Sapiro writes:
> 
>  > +email.Charset.add_charset('utf-8',
>  > +                          email.Charset.SHORTEST,
>  > +                          email.Charset.QP,
>  > +                          'utf-8')
> 
> I don't know if this actually is available, let alone whether it would
> work (might affect headers, too, and of course elderly recipient
> systems might throw a tantrum), but how about Content-Transfer-Encoding
> of 8bit?


I did test it and it does work. I didn't think of it at the time, but
replacing the body encoding (email.Charset.QP in the above patch) with
None, i.e.

email.Charset.add_charset('utf-8',
                          email.Charset.SHORTEST,
                          None,
                          'utf-8')

also works and produces a message with

Content-Transfer-Encoding: 8bit

It doesn't affect headers. The arguments are character set, header
encoding, body encoding, output codec. The only thing the add_charset in
the patch changes from the default is the body encoding argument.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From chris.puchalski at raytheon.com  Wed Nov  4 17:05:41 2015
From: chris.puchalski at raytheon.com (Christopher A Puchalski)
Date: Wed, 4 Nov 2015 16:05:41 -0600
Subject: [Mailman-Users] List Owner Password Reminder
Message-ID: <OFFC10EB62.E30273E1-ON86257EF3.0078831C-86257EF3.00795EBB@raytheon.com>



Is there a "self service" password reminder for list owners?

I am in the process of trying to deploy mailman within my company to
replace a old majordomo instance. Majordomo access/authentication was
managed by local LDAP credentials so users had no real excuse not to know
their passwords. This was setup long ago and predates my joining the
company by 15+ years. In the case of mailman it looks like passwords for
list owners are tied to the software. We have many lists (1000+) so was
hoping I could avoid getting into the business of password resets if
possible. My goal would be to add a "I forgot my password" button to the
admin login page that would kind of force that monthly password reminder
right then.  I know list members have several options but from what I have
seen owners do not.

Is that even possible?


Chris P

From mark at msapiro.net  Thu Nov  5 11:29:54 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 5 Nov 2015 08:29:54 -0800
Subject: [Mailman-Users] List Owner Password Reminder
In-Reply-To: <OFFC10EB62.E30273E1-ON86257EF3.0078831C-86257EF3.00795EBB@raytheon.com>
References: <OFFC10EB62.E30273E1-ON86257EF3.0078831C-86257EF3.00795EBB@raytheon.com>
Message-ID: <563B8402.2030300@msapiro.net>

On 11/04/2015 02:05 PM, Christopher A Puchalski wrote:
> 
> Is there a "self service" password reminder for list owners?


No.


> I am in the process of trying to deploy mailman within my company to
> replace a old majordomo instance. Majordomo access/authentication was
> managed by local LDAP credentials so users had no real excuse not to know
> their passwords. This was setup long ago and predates my joining the
> company by 15+ years. In the case of mailman it looks like passwords for
> list owners are tied to the software. We have many lists (1000+) so was
> hoping I could avoid getting into the business of password resets if
> possible. My goal would be to add a "I forgot my password" button to the
> admin login page that would kind of force that monthly password reminder
> right then.  I know list members have several options but from what I have
> seen owners do not.
> 
> Is that even possible?


In MM 2.1.x list owner/moderator passwords, as opposed to the list
member password for the owner if the owner is a list member (not
required), are kept in encrypted form and are not retrievable as plain
text, thus a reminder is not possible. A 'reset' function with a dialog
with the owner/moderator email address(es) is possible, but such a thing
has not been implemented.

See the FAQs at <http://wiki.list.org/x/4030543> and
<http://wiki.list.org/x/4030693>, and Mailman's 'bin/change_pw --help'.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Nov  5 12:31:13 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 5 Nov 2015 09:31:13 -0800
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563B86C2.7060604@panix.com>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <22074.41582.78231.830496@turnbull.sk.tsukuba.ac.jp>
 <563AA90B.7090307@msapiro.net> <563B86C2.7060604@panix.com>
Message-ID: <563B9261.3040108@msapiro.net>

On 11/05/2015 08:41 AM, Ruben Safir wrote:
> 
> Which file am I patching?  Is this a normal setup?  I pulled it from
> opensuse.
> 
> 
> ruben at www:~> locate Mailman/Message.py
> /usr/lib/mailman/Mailman/Message.py
> /usr/lib/mailman/Mailman/Message.pyc

The above two are probably what you want.
/usr/lib/mailman/Mailman/Message.py is the source to patch. After you
apply the patch and restart Mailman, Python will automatically detect
that the .py is newer and recompile it into the .pyc.


> /usr/local/apache/conf/mailman/Mailman/Message.py
> /usr/local/apache/conf/mailman/Mailman/Message.pyc

This is very strange. I've never seen anything like it. My best guess is
that for some  reason, the openSUSE package installs a separate Mailman/
directory for use by the web server. If that is the case, this one
doesn't need to be patched because the patch doesn't affect anything in
the web UI.

To be sure, I'd need to see the apache configuration, the MTA
configuration and the contents of the directories.

I did quickly look at
<https://build.opensuse.org/package/view_file/openSUSE:Leap:42.1/mailman/mailman-apache2.conf?expand=1>,
<https://build.opensuse.org/package/view_file/server:mail/mailman/mailman-apache2.conf?expand=1>,
etc. and they all refer to /usr/lib/mailman so I have no idea what
/usr/local/apache/conf/mailman/ is about.

The bottom line is the Message.py you need to patch is the one that's
imported by the qrunners, IncomingRunner and OutgoingRunner in
particular.  Is there even a
/usr/local/apache/conf/mailman/Mailman/Queue/ directory at all?

If the two Message.py files are the same, it wouldn't hurt to patch both.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mrbrklyn at panix.com  Thu Nov  5 12:48:32 2015
From: mrbrklyn at panix.com (Ruben Safir)
Date: Thu, 5 Nov 2015 12:48:32 -0500
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563B90BF.4060708@msapiro.net>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <22074.41582.78231.830496@turnbull.sk.tsukuba.ac.jp>
 <563AA90B.7090307@msapiro.net> <563B86C2.7060604@panix.com>
 <563B90BF.4060708@msapiro.net>
Message-ID: <563B9670.90601@panix.com>

On 11/05/2015 12:24 PM, Mark Sapiro wrote:
> On 11/05/2015 08:41 AM, Ruben Safir wrote:
>>
>> Which file am I patching?  Is this a normal setup?  I pulled it from
>> opensuse.
>>
>>
>> ruben at www:~> locate Mailman/Message.py
>> /usr/lib/mailman/Mailman/Message.py
>> /usr/lib/mailman/Mailman/Message.pyc
> 
> The above two are probably what you want.
> /usr/lib/mailman/Mailman/Message.py is the source to patch. After you
> apply the patch and restart Mailman, Python will automatically detect
> that the .py is newer and recompile it into the .pyc.
> 
> 
>> /usr/local/apache/conf/mailman/Mailman/Message.py
>> /usr/local/apache/conf/mailman/Mailman/Message.pyc
> 

ohh.

Maybe I did that because I didn't want my cgi-bin directory outside of
my apache trunk.  It is odd habit I picked up in the last 30 years is
that I want my webservers restricted to the apache directory tree which
is /usr/local/apache/

It didn't occur to be that mailing system binary and script libraries
would also end up in the apache trunk.


<VirtualHost *:80>
    ServerAdmin ruben at mrbrklyn.com
    DocumentRoot "/usr/local/apache/htdocs/nylxs"
    ScriptAlias     /mailman/
/usr/local/apache/htdocs/mailman/cgi-bin/
    Alias           /mailmanicons/  /usr/local/apache/htdocs/mailman/icons/
    Alias           /pipermail/
/usr/local/apache/htdocs/mailman/archives/public/
    ServerName www.nylxs.com
    DirectoryIndex index.html
    PerlModule Embperl
    Embperl_UseEnv        on
    ErrorLog logs/nylxs_error_log
    CustomLog logs/nylxs_access_log common
    <Directory /usr/local/apache/htdocs/nylxs>
           Options     Indexes ExecCGI
           DirectoryIndex index.html
           <Files *.html>
           SetHandler perl-script
#          PerlResponseHandler URL_BLOCK
           PerlHandler Embperl
           </Files>
           AllowOverride None
           Order allow,deny
           Allow from all
    </Directory>
</VirtualHost>

<VirtualHost *:80>
        ServerName   lists.mrbrklyn.com
        DocumentRoot "/usr/local/apache/htdocs/mailman/"
        ScriptAlias     /mailman/
/usr/local/apache/htdocs/mailman/cgi-bin/
        Alias           /mailmanicons/
/usr/local/apache/htdocs/mailman/icons/
        Alias           /pipermail/
/usr/local/apache/htdocs/mailman/archives/public/


"
        ErrorLog     logs/mailman_error_log
        TransferLog  logs/mailman_access_log
        <Directory /usr/local/apache/htdocs/mailman/cgi-bin>
         order allow,deny
         allow from all
        </Directory>

        <Directory /usr/local/apache/htdocs/mailman/icons>
                order allow,deny
                allow from all
        </Directory>

        <Directory /usr/local/apache/htdocs/mailman/archives/>
                Options +FollowSymLinks
                order allow,deny
                allow from all
        </Directory>

</VirtualHost>


> This is very strange. I've never seen anything like it. My best guess is
> that for some  reason, the openSUSE package installs a separate Mailman/
> directory for use by the web server. If that is the case, this one
> doesn't need to be patched because the patch doesn't affect anything in
> the web UI.
> 
> To be sure, I'd need to see the apache configuration, the MTA
> configuration and the contents of the directories.
> 
> I did quickly look at
> <https://build.opensuse.org/package/view_file/openSUSE:Leap:42.1/mailman/mailman-apache2.conf?expand=1>,
> <https://build.opensuse.org/package/view_file/server:mail/mailman/mailman-apache2.conf?expand=1>,
> etc. and they all refer to /usr/lib/mailman so I have no idea what
> /usr/local/apache/conf/mailman/ is about.
> 
> The bottom line is the Message.py you need to patch is the one that's
> imported by the qrunners, IncomingRunner and OutgoingRunner in
> particular.  Is there even a
> /usr/local/apache/conf/mailman/Mailman/Queue/ directory at all?
> 
> If the two Message.py files are the same, it wouldn't hurt to patch both.
> 


From mrbrklyn at panix.com  Thu Nov  5 12:55:10 2015
From: mrbrklyn at panix.com (Ruben Safir)
Date: Thu, 5 Nov 2015 12:55:10 -0500
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563B9261.3040108@msapiro.net>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <22074.41582.78231.830496@turnbull.sk.tsukuba.ac.jp>
 <563AA90B.7090307@msapiro.net> <563B86C2.7060604@panix.com>
 <563B9261.3040108@msapiro.net>
Message-ID: <563B97FE.6000700@panix.com>

On 11/05/2015 12:31 PM, Mark Sapiro wrote:
> On 11/05/2015 08:41 AM, Ruben Safir wrote:
>>
>> Which file am I patching?  Is this a normal setup?  I pulled it from
>> opensuse.
>>
>>
>> ruben at www:~> locate Mailman/Message.py
>> /usr/lib/mailman/Mailman/Message.py
>> /usr/lib/mailman/Mailman/Message.pyc
> 
> The above two are probably what you want.
> /usr/lib/mailman/Mailman/Message.py is the source to patch. After you
> apply the patch and restart Mailman, Python will automatically detect
> that the .py is newer and recompile it into the .pyc.
> 
> 
>> /usr/local/apache/conf/mailman/Mailman/Message.py
>> /usr/local/apache/conf/mailman/Mailman/Message.pyc
> 
> This is very strange. I've never seen anything like it. My best guess is
> that for some  reason, the openSUSE package installs a separate Mailman/
> directory for use by the web server. If that is the case, this one
> doesn't need to be patched because the patch doesn't affect anything in
> the web UI.
> 
> To be sure, I'd need to see the apache configuration, the MTA
> configuration and the contents of the directories.
> 
> I did quickly look at
> <https://build.opensuse.org/package/view_file/openSUSE:Leap:42.1/mailman/mailman-apache2.conf?expand=1>,
> <https://build.opensuse.org/package/view_file/server:mail/mailman/mailman-apache2.conf?expand=1>,
> etc. and they all refer to /usr/lib/mailman so I have no idea what
> /usr/local/apache/conf/mailman/ is about.
> 
> The bottom line is the Message.py you need to patch is the one that's
> imported by the qrunners, IncomingRunner and OutgoingRunner in
> particular.  Is there even a
> /usr/local/apache/conf/mailman/Mailman/Queue/ directory at all?
> 
> If the two Message.py files are the same, it wouldn't hurt to patch both.
> 
www:/etc/postfix # grep mailman main.cf
#alias_maps = "hash:/etc/aliases, hash:/var/lib/mailman/data/aliases"
alias_maps = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases

www:/etc/postfix # grep  mailman ./*
./main.cf:#alias_maps = "hash:/etc/aliases, hash:/var/lib/mailman/data/aliases"
./main.cf:alias_maps = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases
./master.cf:#mailman   unix  -       n       n       -       -       pipe
./master.cf:#  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
./master.cf.rpmnew:#mailman   unix  -       n       n       -       -       pipe
./master.cf.rpmnew:#  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py


evidently it is under /usr/lib/mailman/bin/ ?


From mark at msapiro.net  Thu Nov  5 13:05:32 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 5 Nov 2015 10:05:32 -0800
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563B9670.90601@panix.com>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <22074.41582.78231.830496@turnbull.sk.tsukuba.ac.jp>
 <563AA90B.7090307@msapiro.net> <563B86C2.7060604@panix.com>
 <563B90BF.4060708@msapiro.net> <563B9670.90601@panix.com>
Message-ID: <563B9A6C.1090109@msapiro.net>

On 11/05/2015 09:48 AM, Ruben Safir wrote:

>>> /usr/local/apache/conf/mailman/Mailman/Message.py
>>> /usr/local/apache/conf/mailman/Mailman/Message.pyc
>>
> 
> ohh.
> 
> Maybe I did that because I didn't want my cgi-bin directory outside of
> my apache trunk.  It is odd habit I picked up in the last 30 years is
> that I want my webservers restricted to the apache directory tree which
> is /usr/local/apache/
> 
> It didn't occur to be that mailing system binary and script libraries
> would also end up in the apache trunk.


They don't have to, but the real question now is do you have one Mailman
installation in /usr/local/apache/conf/mailman, or do you have a split
installation with some things there and some things in /usr/lib/mailman.


> <VirtualHost *:80>
>     ServerAdmin ruben at mrbrklyn.com
>     DocumentRoot "/usr/local/apache/htdocs/nylxs"
>     ScriptAlias     /mailman/
> /usr/local/apache/htdocs/mailman/cgi-bin/
>     Alias           /mailmanicons/  /usr/local/apache/htdocs/mailman/icons/
>     Alias           /pipermail/
> /usr/local/apache/htdocs/mailman/archives/public/
...
> <VirtualHost *:80>
>         ServerName   lists.mrbrklyn.com
>         DocumentRoot "/usr/local/apache/htdocs/mailman/"
>         ScriptAlias     /mailman/
> /usr/local/apache/htdocs/mailman/cgi-bin/
>         Alias           /mailmanicons/
> /usr/local/apache/htdocs/mailman/icons/
>         Alias           /pipermail/
> /usr/local/apache/htdocs/mailman/archives/public/


Now it seems that apache is pointed at Mailman stuff in yes a third
place, /usr/local/apache/htdocs/mailman.

How do you start Mailman? Where is the bin/mailmanctl that starts it?
what is in paths.py in the same bin/ directory? Are there any lists with
public archives and if so, are there symlinks in
/usr/local/apache/htdocs/mailman/archives/public/ and where do they point?

Do the web server and the qrunners actually reference the same archives,
data, lists, locks, logs, qfiles and spam directories? Of these, some
are only used by one or the other, but it's absolutely critical that
they reference the same archives, data, lists and locks directories.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Nov  5 13:18:38 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 5 Nov 2015 10:18:38 -0800
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563B97FE.6000700@panix.com>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <22074.41582.78231.830496@turnbull.sk.tsukuba.ac.jp>
 <563AA90B.7090307@msapiro.net> <563B86C2.7060604@panix.com>
 <563B9261.3040108@msapiro.net> <563B97FE.6000700@panix.com>
Message-ID: <563B9D7E.2080209@msapiro.net>

On 11/05/2015 09:55 AM, Ruben Safir wrote:

> www:/etc/postfix # grep  mailman ./*
> ./main.cf:#alias_maps = "hash:/etc/aliases, hash:/var/lib/mailman/data/aliases"
> ./main.cf:alias_maps = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases
> ./master.cf:#mailman   unix  -       n       n       -       -       pipe
> ./master.cf:#  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
> ./master.cf.rpmnew:#mailman   unix  -       n       n       -       -       pipe
> ./master.cf.rpmnew:#  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
> 
> 
> evidently it is under /usr/lib/mailman/bin/ ?


All I see there is a pointer to /var/lib/mailman/data/aliases. What's in
that file?

I'm glad to see the postfix-to-mailman.py trensport is commented out.
See the FAQ at <http://wiki.list.org/x/15564817>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mrbrklyn at panix.com  Thu Nov  5 13:33:20 2015
From: mrbrklyn at panix.com (Ruben Safir)
Date: Thu, 5 Nov 2015 13:33:20 -0500
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563B9A6C.1090109@msapiro.net>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <22074.41582.78231.830496@turnbull.sk.tsukuba.ac.jp>
 <563AA90B.7090307@msapiro.net> <563B86C2.7060604@panix.com>
 <563B90BF.4060708@msapiro.net> <563B9670.90601@panix.com>
 <563B9A6C.1090109@msapiro.net>
Message-ID: <563BA0F0.60905@panix.com>

On 11/05/2015 01:05 PM, Mark Sapiro wrote:
> On 11/05/2015 09:48 AM, Ruben Safir wrote:
> 
>>>> /usr/local/apache/conf/mailman/Mailman/Message.py
>>>> /usr/local/apache/conf/mailman/Mailman/Message.pyc
>>>
>>
>> ohh.
>>
>> Maybe I did that because I didn't want my cgi-bin directory outside of
>> my apache trunk.  It is odd habit I picked up in the last 30 years is
>> that I want my webservers restricted to the apache directory tree which
>> is /usr/local/apache/
>>
>> It didn't occur to be that mailing system binary and script libraries
>> would also end up in the apache trunk.
> 
> 
> They don't have to, but the real question now is do you have one Mailman
> installation in /usr/local/apache/conf/mailman, or do you have a split
> installation with some things there and some things in /usr/lib/mailman.
> 

The idea is to have it split, and let the mail stuff do its thing in /usr/lib/ or /var/lib,
and to integrate the admin interface to my existing apache tree minus the mailman archive 
which I don't currently have use for.

> 
>> <VirtualHost *:80>
>>     ServerAdmin ruben at mrbrklyn.com
>>     DocumentRoot "/usr/local/apache/htdocs/nylxs"
>>     ScriptAlias     /mailman/
>> /usr/local/apache/htdocs/mailman/cgi-bin/
>>     Alias           /mailmanicons/  /usr/local/apache/htdocs/mailman/icons/
>>     Alias           /pipermail/
>> /usr/local/apache/htdocs/mailman/archives/public/
> ...
>> <VirtualHost *:80>
>>         ServerName   lists.mrbrklyn.com
>>         DocumentRoot "/usr/local/apache/htdocs/mailman/"
>>         ScriptAlias     /mailman/
>> /usr/local/apache/htdocs/mailman/cgi-bin/
>>         Alias           /mailmanicons/
>> /usr/local/apache/htdocs/mailman/icons/
>>         Alias           /pipermail/
>> /usr/local/apache/htdocs/mailman/archives/public/
> 
> 
> Now it seems that apache is pointed at Mailman stuff in yes a third
> place, /usr/local/apache/htdocs/mailman.
>

well, that is just the public side of the webserver so anything that is displayed to the world is under /usr/local/apache/htdocs/ and for mailman

/usr/local/apache/htdocs/mailman
the configuration files for the web sever should be nicely tucked into /usr/local/apache/conf and /usr/local/apache/conf/mailman/

normal I would put cgi-bins under /usr/local/apache/cgi-bin/ but for mailman I was going tuck them under the mailman config section for the webserver under /usr/local/apache/conf/mailman


 
> How do you start Mailman? Where is the bin/mailmanctl that starts it?

/etc/rc.d/mailman which unfortunately is piped through systemd.

#! /bin/sh
# Copyright (c) 2000-2001 SuSE GmbH Nuernberg, Germany.
# Copyright (c) 2002 SuSE Linux AG
#
# Author: Heiko Rommel <feedback at suse.de>
#
# /etc/init.d/mailman
#
### BEGIN INIT INFO
# Provides: mailman
# Required-Start: $remote_fs $syslog cron sendmail $named
# Required-Stop:  $remote_fs $syslog
# Default-Start:  3 5
# Default-Stop:   0 1 2 6
# Description:    Starts the mailinglist manager Mailman
### END INIT INFO

# Check for missing binaries (stale symlinks should not happen)
ETC_CT=/etc/cron.d/mailman
MM_CT=/usr/lib/mailman/cron/crontab
MM_CTRL=/usr/lib/mailman/bin/mailmanctl
MM_PID=/var/lib/mailman/data/master-qrunner.pid
test -f $MM_CT || exit 5
test -x $MM_CTRL || exit 5


> what is in  in the same bin/ directory? 

Excellent question...

import os
import sys

# some scripts expect this attribute to be in this module
prefix = '/usr/lib/mailman'
exec_prefix = '${prefix}'

# work around a bogus autoconf 2.12 bug
if exec_prefix == '${prefix}':
    exec_prefix = prefix


> Are there any lists with
> public archives and if so, are there symlinks in
> /usr/local/apache/htdocs/mailman/archives/public/ and where do they point?

I have done the archives using the same software I wrote in 1996 but the mbox et al seems to be in 

www:/etc/postfix # ls -al  /var/lib/mailman/archives/private/hangout.mbox/hangout.mbox 
-rw-rw-r-- 1 mailman mailman 486238 Nov  5 11:32 /var/lib/mailman/archives/private/hangout.mbox/hangout.mbox

www:/etc/postfix # ls -al  /var/lib/mailman/archives/public/
total 8
drwxrwsr-x 2 root   mailman 4096 Oct 20 23:43 .
drwxrwsr-x 4 root   mailman 4096 Oct  8  2014 ..
lrwxrwxrwx 1 daemon mailman   41 Oct 20 23:43 hangout -> /var/lib/mailman/archives/private/hangout
lrwxrwxrwx 1 daemon mailman   39 Oct 20 16:08 learn -> /var/lib/mailman/archives/private/learn
lrwxrwxrwx 1 root   mailman   41 Oct 20 11:14 mailman -> /var/lib/mailman/archives/private/mailman



> 
> Do the web server and the qrunners actually reference the same archives,

I would prefer the webserver not to reach the archives that mailman archives at all, 
since my real archive is sitting on a remote postgres server.


> data, lists, locks, logs, qfiles and spam directories? Of these, some
> are only used by one or the other, but it's absolutely critical that
> they reference the same archives, data, lists and locks directories.
> 


From mrbrklyn at panix.com  Thu Nov  5 13:38:04 2015
From: mrbrklyn at panix.com (Ruben Safir)
Date: Thu, 5 Nov 2015 13:38:04 -0500
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563B9D7E.2080209@msapiro.net>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <22074.41582.78231.830496@turnbull.sk.tsukuba.ac.jp>
 <563AA90B.7090307@msapiro.net> <563B86C2.7060604@panix.com>
 <563B9261.3040108@msapiro.net> <563B97FE.6000700@panix.com>
 <563B9D7E.2080209@msapiro.net>
Message-ID: <563BA20C.2020105@panix.com>

On 11/05/2015 01:18 PM, Mark Sapiro wrote:
> On 11/05/2015 09:55 AM, Ruben Safir wrote:
> 
>> www:/etc/postfix # grep  mailman ./*
>> ./main.cf:#alias_maps = "hash:/etc/aliases, hash:/var/lib/mailman/data/aliases"
>> ./main.cf:alias_maps = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases
>> ./master.cf:#mailman   unix  -       n       n       -       -       pipe
>> ./master.cf:#  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
>> ./master.cf.rpmnew:#mailman   unix  -       n       n       -       -       pipe
>> ./master.cf.rpmnew:#  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
>>
>>
>> evidently it is under /usr/lib/mailman/bin/ ?
> 
> 
> All I see there is a pointer to /var/lib/mailman/data/aliases. What's in
> that file?
> 

# This file is generated by Mailman, and is kept in sync with the
# binary hash file aliases.db.  YOU SHOULD NOT MANUALLY EDIT THIS FILE
# unless you know what you're doing, and can keep the two files properly
# in sync.  If you screw it up, you're on your own.

# The ultimate loop stopper address
mailman-loop: /var/lib/mailman/data/owner-bounces.mbox

# STANZA START: mailman
# CREATED: Tue Oct 20 11:14:22 2015
mailman:             "|/usr/lib/mailman/mail/mailman post mailman"
mailman-admin:       "|/usr/lib/mailman/mail/mailman admin mailman"
mailman-bounces:     "|/usr/lib/mailman/mail/mailman bounces mailman"
mailman-confirm:     "|/usr/lib/mailman/mail/mailman confirm mailman"
mailman-join:        "|/usr/lib/mailman/mail/mailman join mailman"
mailman-leave:       "|/usr/lib/mailman/mail/mailman leave mailman"
mailman-owner:       "|/usr/lib/mailman/mail/mailman owner mailman"
mailman-request:     "|/usr/lib/mailman/mail/mailman request mailman"
mailman-subscribe:   "|/usr/lib/mailman/mail/mailman subscribe mailman"
mailman-unsubscribe: "|/usr/lib/mailman/mail/mailman unsubscribe mailman"
# STANZA END: mailman

# STANZA START: hangout
# CREATED: Tue Oct 20 11:23:34 2015
hangout:             "|/usr/lib/mailman/mail/mailman post hangout"
hangout-admin:       "|/usr/lib/mailman/mail/mailman admin hangout"
hangout-bounces:     "|/usr/lib/mailman/mail/mailman bounces hangout"
hangout-confirm:     "|/usr/lib/mailman/mail/mailman confirm hangout"
hangout-join:        "|/usr/lib/mailman/mail/mailman join hangout"
hangout-leave:       "|/usr/lib/mailman/mail/mailman leave hangout"
hangout-owner:       "|/usr/lib/mailman/mail/mailman owner hangout"
hangout-request:     "|/usr/lib/mailman/mail/mailman request hangout"
hangout-subscribe:   "|/usr/lib/mailman/mail/mailman subscribe hangout"
hangout-unsubscribe: "|/usr/lib/mailman/mail/mailman unsubscribe hangout"
# STANZA END: hangout


> I'm glad to see the postfix-to-mailman.py trensport is commented out.
> See the FAQ at <http://wiki.list.org/x/15564817>.
> 


From mark at msapiro.net  Thu Nov  5 15:21:35 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 5 Nov 2015 12:21:35 -0800
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563BA20C.2020105@panix.com>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <22074.41582.78231.830496@turnbull.sk.tsukuba.ac.jp>
 <563AA90B.7090307@msapiro.net> <563B86C2.7060604@panix.com>
 <563B9261.3040108@msapiro.net> <563B97FE.6000700@panix.com>
 <563B9D7E.2080209@msapiro.net> <563BA20C.2020105@panix.com>
Message-ID: <563BBA4F.3030508@msapiro.net>

On 11/05/2015 10:38 AM, Ruben Safir wrote:
> 
> # STANZA START: mailman
> # CREATED: Tue Oct 20 11:14:22 2015
> mailman:             "|/usr/lib/mailman/mail/mailman post mailman"
...


So list related email goes to the mailman in /usr/lib/mailman/, so
/usr/lib/mailman/Mailman/Message.py is the file to patch.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Nov  5 15:50:18 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 5 Nov 2015 12:50:18 -0800
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563BA0F0.60905@panix.com>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <22074.41582.78231.830496@turnbull.sk.tsukuba.ac.jp>
 <563AA90B.7090307@msapiro.net> <563B86C2.7060604@panix.com>
 <563B90BF.4060708@msapiro.net> <563B9670.90601@panix.com>
 <563B9A6C.1090109@msapiro.net> <563BA0F0.60905@panix.com>
Message-ID: <563BC10A.4070005@msapiro.net>

On 11/05/2015 10:33 AM, Ruben Safir wrote:
> On 11/05/2015 01:05 PM, Mark Sapiro wrote:
>  
>> How do you start Mailman? Where is the bin/mailmanctl that starts it?
> 
> /etc/rc.d/mailman which unfortunately is piped through systemd.
> 
...
> MM_CT=/usr/lib/mailman/cron/crontab
> MM_CTRL=/usr/lib/mailman/bin/mailmanctl
> MM_PID=/var/lib/mailman/data/master-qrunner.pid


So starting Mailman runs the Mailman in /usr/lib/mailman/


>> what is in paths.py in the same bin/ directory? 
> 
> Excellent question...
> 
> import os
> import sys
> 
> # some scripts expect this attribute to be in this module
> prefix = '/usr/lib/mailman'


And paths.py is consistent with this.


>> Are there any lists with
>> public archives and if so, are there symlinks in
>> /usr/local/apache/htdocs/mailman/archives/public/ and where do they point?
> 
> I have done the archives using the same software I wrote in 1996 but the mbox et al seems to be in 
> 
> www:/etc/postfix # ls -al  /var/lib/mailman/archives/private/hangout.mbox/hangout.mbox 
> -rw-rw-r-- 1 mailman mailman 486238 Nov  5 11:32 /var/lib/mailman/archives/private/hangout.mbox/hangout.mbox
> 
> www:/etc/postfix # ls -al  /var/lib/mailman/archives/public/
> total 8
> drwxrwsr-x 2 root   mailman 4096 Oct 20 23:43 .
> drwxrwsr-x 4 root   mailman 4096 Oct  8  2014 ..
> lrwxrwxrwx 1 daemon mailman   41 Oct 20 23:43 hangout -> /var/lib/mailman/archives/private/hangout
> lrwxrwxrwx 1 daemon mailman   39 Oct 20 16:08 learn -> /var/lib/mailman/archives/private/learn
> lrwxrwxrwx 1 root   mailman   41 Oct 20 11:14 mailman -> /var/lib/mailman/archives/private/mailman


This is all consistent with an instalation with prefix =
/usr/lib/mailman and var_prefix = /var/lib/mailman


>>
>> Do the web server and the qrunners actually reference the same archives,
> 
> I would prefer the webserver not to reach the archives that mailman archives at all, 
> since my real archive is sitting on a remote postgres server.


And you've basically arranged that by making the archive public as far
as mailman is concerned and telling apache

    Alias /pipermail/ /usr/local/apache/htdocs/mailman/archives/public/

so at least the default public archive URLs point somewhere where the
archives presumably aren't.

You might consider setting PUBLIC_ARCHIVE_URL in mm_cfg.py to something like

PUBLIC_ARCHIVE_URL = 'http://remote.server/whatever/%(listname)s'

You might also avoid this whole patching issue by teaching your archiver
to decode base64 and quoted-printable encoded message bodies.

I still wonder about

    ScriptAlias /mailman/ /usr/local/apache/htdocs/mailman/cgi-bin/

If you do for example

strings /usr/local/apache/htdocs/mailman/cgi-bin/admin | grep mailman

do you see paths like /usr/lib/mailman and /usr/lib/mailman/scripts?

If so, you are probably OK. If they point at some /usr/local/apache
directory, you may have issues with the web UI manipulating totally
different list objects than the real ones.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From doar9999 at yahoo.com  Thu Nov  5 14:11:23 2015
From: doar9999 at yahoo.com (am)
Date: Thu, 5 Nov 2015 13:11:23 -0600
Subject: [Mailman-Users] Prefix in Subject line anomaly??
Message-ID: <CAG+EZgGuBRPF5nBvherWL5DXmh40Su=0H2fqQUA8xuyT6atJhQ@mail.gmail.com>

Prefix for Subject Line not showing in email subject line, but it DOES show
in the archive.
?
 The prefix is  [CaucusPlanners]

?Mailman ?
version 2.1.20...
?

help please...

C-L

From doar9999 at yahoo.com  Thu Nov  5 14:36:46 2015
From: doar9999 at yahoo.com (am)
Date: Thu, 5 Nov 2015 13:36:46 -0600
Subject: [Mailman-Users] Another anomaly? Header is not showing
Message-ID: <CAG+EZgFPa8gYKJTrNjXsVYU-_sWVcVrVhF1zxGEN0re1=Dz8fw@mail.gmail.com>

In the mailing list setup (v.2.1.20) there is a section in non-digest to
add Header info: msg_header Option

I used it in the hopes that I can put something at the top of EVERY
message.

But nothing appears in the messages - neither header nor does the footer
show up.

Help, please...

C-L

From mark at msapiro.net  Thu Nov  5 16:35:58 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 5 Nov 2015 13:35:58 -0800
Subject: [Mailman-Users] Prefix in Subject line anomaly??
In-Reply-To: <CAG+EZgGuBRPF5nBvherWL5DXmh40Su=0H2fqQUA8xuyT6atJhQ@mail.gmail.com>
References: <CAG+EZgGuBRPF5nBvherWL5DXmh40Su=0H2fqQUA8xuyT6atJhQ@mail.gmail.com>
Message-ID: <563BCBBE.1060005@msapiro.net>

On 11/05/2015 11:11 AM, am via Mailman-Users wrote:
> Prefix for Subject Line not showing in email subject line, but it DOES show
> in the archive.


Are you looking at a message actually received from the list and not one
received by direct To:, Cc: or Bcc:?

Examine the complete headers of the message.

Does it have headers like

X-BeenThere: list at example.com
X-Mailman-Version: 2.1.20

If not, it didn't come from Mailman.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Nov  5 16:41:22 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 5 Nov 2015 13:41:22 -0800
Subject: [Mailman-Users] Another anomaly? Header is not showing
In-Reply-To: <CAG+EZgFPa8gYKJTrNjXsVYU-_sWVcVrVhF1zxGEN0re1=Dz8fw@mail.gmail.com>
References: <CAG+EZgFPa8gYKJTrNjXsVYU-_sWVcVrVhF1zxGEN0re1=Dz8fw@mail.gmail.com>
Message-ID: <563BCD02.4080002@msapiro.net>

On 11/05/2015 11:36 AM, am via Mailman-Users wrote:
> In the mailing list setup (v.2.1.20) there is a section in non-digest to
> add Header info: msg_header Option
> 
> I used it in the hopes that I can put something at the top of EVERY
> message.
> 
> But nothing appears in the messages - neither header nor does the footer
> show up.


There are two possibilities.

1) You are looking at a message that didn't come from the list. See my
reply in another thread at
<https://mail.python.org/pipermail/mailman-users/2015-November/080068.html>.

2) The msg_header and msg_footer are there as separate MIME parts
because the post after any content filtering was not a simple text/plain
message and your MUA is not showing them to you. See the FAQ at
<http://wiki.list.org/x/4030707>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Nov  5 16:57:19 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 5 Nov 2015 13:57:19 -0800
Subject: [Mailman-Users] Prefix in Subject line anomaly??
In-Reply-To: <CAG+EZgG2L1E-FWQf51LLhPniBjZE10nMDNygXEiDq0hseTmkDg@mail.gmail.com>
References: <CAG+EZgGuBRPF5nBvherWL5DXmh40Su=0H2fqQUA8xuyT6atJhQ@mail.gmail.com>
 <563BCBBE.1060005@msapiro.net>
 <CAG+EZgG2L1E-FWQf51LLhPniBjZE10nMDNygXEiDq0hseTmkDg@mail.gmail.com>
Message-ID: <563BD0BF.1060701@msapiro.net>

On 11/05/2015 01:53 PM, am wrote:
> Yes, I am looking at the Welcome to the list message. I see: 


Only list posts have the subject_prefix added. Welcome and other
administrative messages do not.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mrbrklyn at panix.com  Fri Nov  6 00:05:06 2015
From: mrbrklyn at panix.com (Ruben Safir)
Date: Fri, 6 Nov 2015 00:05:06 -0500
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563BC10A.4070005@msapiro.net>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <22074.41582.78231.830496@turnbull.sk.tsukuba.ac.jp>
 <563AA90B.7090307@msapiro.net> <563B86C2.7060604@panix.com>
 <563B90BF.4060708@msapiro.net> <563B9670.90601@panix.com>
 <563B9A6C.1090109@msapiro.net> <563BA0F0.60905@panix.com>
 <563BC10A.4070005@msapiro.net>
Message-ID: <563C3502.6080702@panix.com>

On 11/05/2015 03:50 PM, Mark Sapiro wrote:
> And you've basically arranged that by making the archive public as far
> as mailman is concerned and telling apache
> 
>     Alias /pipermail/ /usr/local/apache/htdocs/mailman/archives/public/
> 
> so at least the default public archive URLs point somewhere where the
> archives presumably aren't.


Yeah - I should take it out.  I didn't know what it was when I installed
it so I just followed directions and inserted it.  I really don't need
though, do I, since it is about the archives and archives and I'm not
using mailman's archives.



From doar9999 at yahoo.com  Fri Nov  6 09:50:05 2015
From: doar9999 at yahoo.com (am)
Date: Fri, 6 Nov 2015 08:50:05 -0600
Subject: [Mailman-Users] Delete archive?
Message-ID: <CAG+EZgGLGK6ZMFjBB9A5edpWLxdHRU-oL-2Pby0aGCbqmpUNpw@mail.gmail.com>

Is there a way to wipe out an archive and restart it from scratch?

Mailman v2.1.20

C-L

From chris.puchalski at raytheon.com  Fri Nov  6 10:37:56 2015
From: chris.puchalski at raytheon.com (Christopher A Puchalski)
Date: Fri, 6 Nov 2015 09:37:56 -0600
Subject: [Mailman-Users] List Owner Password Reminder
In-Reply-To: <563B8402.2030300@msapiro.net>
References: <OFFC10EB62.E30273E1-ON86257EF3.0078831C-86257EF3.00795EBB@raytheon.com>
 <563B8402.2030300@msapiro.net>
Message-ID: <OFDA320143.69999F73-ON86257EF5.00557217-86257EF5.0055DEF7@raytheon.com>


So in the future that may be implemented in the v2.1.x family? If so that
would be a good thing and I may be able to live with a few requests until
then.

Will that be a situation like send a message with a certain "code" in it
and owners get a confirmation email with a link to follow? Similar to self
service joining of a list to prevent abuse.


Chris P



From:	Mark Sapiro <mark at msapiro.net>
To:	mailman-users at python.org
Date:	11/05/2015 10:30 AM
Subject:	Re: [Mailman-Users] List Owner Password Reminder
Sent by:	"Mailman-Users" <mailman-users-bounces
            +chris.puchalski=raytheon.com at python.org>



On 11/04/2015 02:05 PM, Christopher A Puchalski wrote:
>
> Is there a "self service" password reminder for list owners?


No.


> I am in the process of trying to deploy mailman within my company to
> replace a old majordomo instance. Majordomo access/authentication was
> managed by local LDAP credentials so users had no real excuse not to know
> their passwords. This was setup long ago and predates my joining the
> company by 15+ years. In the case of mailman it looks like passwords for
> list owners are tied to the software. We have many lists (1000+) so was
> hoping I could avoid getting into the business of password resets if
> possible. My goal would be to add a "I forgot my password" button to the
> admin login page that would kind of force that monthly password reminder
> right then.  I know list members have several options but from what I
have
> seen owners do not.
>
> Is that even possible?


In MM 2.1.x list owner/moderator passwords, as opposed to the list
member password for the owner if the owner is a list member (not
required), are kept in encrypted form and are not retrievable as plain
text, thus a reminder is not possible. A 'reset' function with a dialog
with the owner/moderator email address(es) is possible, but such a thing
has not been implemented.

See the FAQs at <http://wiki.list.org/x/4030543> and
<http://wiki.list.org/x/4030693>, and Mailman's 'bin/change_pw --help'.

--
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan
------------------------------------------------------
Mailman-Users mailing list Mailman-Users at python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives:
http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe:
https://mail.python.org/mailman/options/mailman-users/chris.puchalski%40raytheon.com


From mark at msapiro.net  Fri Nov  6 11:55:52 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 6 Nov 2015 08:55:52 -0800
Subject: [Mailman-Users] Delete archive?
In-Reply-To: <CAG+EZgGLGK6ZMFjBB9A5edpWLxdHRU-oL-2Pby0aGCbqmpUNpw@mail.gmail.com>
References: <CAG+EZgGLGK6ZMFjBB9A5edpWLxdHRU-oL-2Pby0aGCbqmpUNpw@mail.gmail.com>
Message-ID: <563CDB98.5010107@msapiro.net>

On 11/06/2015 06:50 AM, am via Mailman-Users wrote:
> Is there a way to wipe out an archive and restart it from scratch?


To delete the archive completely including the cumulative mbox file, do

echo > /path/to/mailman/archives/private/LISTNAME.mbox/LISTNAME.mbox
/path/to/mailman/bin/arch --wipe LISTNAME

To delete just the web archive without deleting messages from the
cumulative mbox file, just do

/path/to/mailman/bin/arch --wipe LISTNAME /dev/null

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Fri Nov  6 12:28:32 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 6 Nov 2015 09:28:32 -0800
Subject: [Mailman-Users] List Owner Password Reminder
In-Reply-To: <OFDA320143.69999F73-ON86257EF5.00557217-86257EF5.0055DEF7@raytheon.com>
References: <OFFC10EB62.E30273E1-ON86257EF3.0078831C-86257EF3.00795EBB@raytheon.com>
 <563B8402.2030300@msapiro.net>
 <OFDA320143.69999F73-ON86257EF5.00557217-86257EF5.0055DEF7@raytheon.com>
Message-ID: <563CE340.5010106@msapiro.net>

On 11/06/2015 07:37 AM, Christopher A Puchalski wrote:
> So in the future that may be implemented in the v2.1.x family? If so
> that would be a good thing and I may be able to live with a few requests
> until then.


There are no plans to implement this in any future release of Mailman 2.1.x.


> Will that be a situation like send a message with a certain "code" in it
> and owners get a confirmation email with a link to follow? Similar to
> self service joining of a list to prevent abuse.


I can envision implementing this with a reminder button on the admin
login page which would result in a confirmation request being sent to
all 'owner' addresses. Then if any recipient confirmed, a new password
would be randomly generated and mailed to all the owner addresses.
Basically, the confirmation would result in running change_pw for the
list, but I see potential problems with this, and I have no plans to
actually implement it.

Mailman 3 works differently and already has mechanisms for users to
reset their passwords, and there are no 'owners' in the MM 2.1 sense,
only users with multiple roles such as digest member of list1, regular
member of list2, owner of list2, moderator of list3, etc.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Fri Nov  6 14:10:01 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 6 Nov 2015 11:10:01 -0800
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563C3502.6080702@panix.com>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <22074.41582.78231.830496@turnbull.sk.tsukuba.ac.jp>
 <563AA90B.7090307@msapiro.net> <563B86C2.7060604@panix.com>
 <563B90BF.4060708@msapiro.net> <563B9670.90601@panix.com>
 <563B9A6C.1090109@msapiro.net> <563BA0F0.60905@panix.com>
 <563BC10A.4070005@msapiro.net> <563C3502.6080702@panix.com>
Message-ID: <563CFB09.2080503@msapiro.net>

On 11/5/15 9:05 PM, Ruben Safir wrote:
> On 11/05/2015 03:50 PM, Mark Sapiro wrote:
>> And you've basically arranged that by making the archive public as far
>> as mailman is concerned and telling apache
>>
>>     Alias /pipermail/ /usr/local/apache/htdocs/mailman/archives/public/
>>
>> so at least the default public archive URLs point somewhere where the
>> archives presumably aren't.
> 
> 
> Yeah - I should take it out.  I didn't know what it was when I installed
> it so I just followed directions and inserted it.  I really don't need
> though, do I, since it is about the archives and archives and I'm not
> using mailman's archives.


There are links to the archive on the list's listinfo pages (if you
haven't edited them).

If you don't override PUBLIC_ARCHIVE_URL as I suggested in the post at
<https://mail.python.org/pipermail/mailman-users/2015-November/080065.html>,
these links will 404 in any case.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Mon Nov  9 09:51:31 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 9 Nov 2015 06:51:31 -0800
Subject: [Mailman-Users] Part of moderated mails postponed to later
 digests
In-Reply-To: <6DCC3E5DA06FE346B4DE4876C4F2713D014D85D5B7@P2KITMBX06WC03.unicph.domain>
References: <6DCC3E5DA06FE346B4DE4876C4F2713D014D8582EF@P2KITMBX06WC03.unicph.domain>
 <5638E182.7080305@msapiro.net>
 <6DCC3E5DA06FE346B4DE4876C4F2713D014D858F18@P2KITMBX06WC03.unicph.domain>
 <563A2D22.3070005@msapiro.net>
 <6DCC3E5DA06FE346B4DE4876C4F2713D014D85D5B7@P2KITMBX06WC03.unicph.domain>
Message-ID: <5640B2F3.6010508@msapiro.net>

On 11/9/15 4:50 AM, Henrik Rasmussen wrote:

>> Mark Sapiro [mailto:mark at msapiro.net]
>> Sent: 4. november 2015 17:07

>> Anyway, I understood your OP to say digests were being sent only weekly when batches of held messages are approved.
> That was an example of the frequency, held mails is being approved by the moderator. When the moderator approves a bunch of mails in one week, the list admin claims that only some of the mails are posted to the list, but that some of the mails will not be sent in a digest until the next moderation a week later (example).


This is the first time (I think) you mentioned 'posted to the list' Are
the messages missing from the digest not posted to the list at all until
the next week? I.e. are they not in the list archives and not delivered
to any non-digest members?

If in fact the missing messages don't appear in the archive until the
next week, this is a different issue which has nothing to do with
digests per se.


>> if you don't want digests triggered on size
> The list admin doesn't need digests sent according to size so a digest_size_threshhold = 10000 should be fine. The moderator wants to have the mails included in a digest the next noon after approving the mails.


Are the messages which are sent, sent in an immediate digest at the time
of approval or just the next noon?

If the former, that digest is triggered on size. If the latter, those
are the only messages that actually got approved at that time.


>> and your mailman is older than 2.1.16
> I run Mailman 2.1.12 which is the latest available for yum update on RHEL6. I think the last Mailman version on RHEL7 is 2.1.15, but we're not there yet.


So you can't use digest_size_threshhold = 0 for unlimited because of the
bug, but if digests are triggered on size and you don't want that, you
can set digest_size_threshhold to a bigger number.


> According to the vette log, some are discarded by moderator actions, :
>    Oct 25 10:20:42 2015 (16801) Precedence: bulk message discarded by: LISTNAME-request at list.example.com
> And of cause a lot of "Reason: Non-members" and spam-like messages "Reason: Your message was deemed inappropriate by the moderator.".


What does the vette log say about the messages that don't appear in a
digest until the following week? Do they show as approved at the same
time as the others that are in the first digest or not until later.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From kapuzino at web.de  Mon Nov  9 10:00:04 2015
From: kapuzino at web.de (kapuzino at web.de)
Date: Mon, 9 Nov 2015 16:00:04 +0100
Subject: [Mailman-Users] Mailman only delivers inside server
Message-ID: <3nvbML0hlMzFpWN@mail.python.org>

Hello

>> Mailman had worked fine primarily.
>> After a server-update it deliveres messages only to email-adresses hostet on the same server (my own adresses connected to two different tld's).

(Gao)
>I am having the same problem  ...
>...  So I changed my main.cf:
>mynetworks = 192.168.1.0/24, 127.0.0.0/8 localhost

It is done, without bettering.

(Mark Sapiro)
>This is almost certainly a Postfix issue, not a Mailman issue.

But what is the diffrence between sending with postfix using a distributor generated in plesk-email-panel and the connection of mailman to postfix ?
If it is no internal mailman issue, though a special configuration issue needed for mailman, when postfix is sending everything well, but mailman tasks.

What information could help to localize the failure ?

regards 
Kapuzino





From doar9999 at yahoo.com  Mon Nov  9 10:30:33 2015
From: doar9999 at yahoo.com (am)
Date: Mon, 9 Nov 2015 09:30:33 -0600
Subject: [Mailman-Users] Delete Archive
In-Reply-To: <CAG+EZgH4DpJQ5xiS_0_2RMqpuZjrd89psoe6F=dCXH9tbtTzSQ@mail.gmail.com>
References: <CAG+EZgH4DpJQ5xiS_0_2RMqpuZjrd89psoe6F=dCXH9tbtTzSQ@mail.gmail.com>
Message-ID: <CAG+EZgFoXbKBrvMFbPGQN0DV9XAej1ux6Z-+F623QxLNBwtKGw@mail.gmail.com>

I posted a similar q on 6th but did not see the post nor any responses. (I
may be still in the newbie doghouse :-) ...)

My question: Is there a way to reset the archive, or delete / restart it?

Mailman 2.1.20

Thank you -

C-L

From mark at msapiro.net  Mon Nov  9 10:37:04 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 9 Nov 2015 07:37:04 -0800
Subject: [Mailman-Users] Delete Archive
In-Reply-To: <CAG+EZgFoXbKBrvMFbPGQN0DV9XAej1ux6Z-+F623QxLNBwtKGw@mail.gmail.com>
References: <CAG+EZgH4DpJQ5xiS_0_2RMqpuZjrd89psoe6F=dCXH9tbtTzSQ@mail.gmail.com>
 <CAG+EZgFoXbKBrvMFbPGQN0DV9XAej1ux6Z-+F623QxLNBwtKGw@mail.gmail.com>
Message-ID: <5640BDA0.9060209@msapiro.net>

On 11/9/15 7:30 AM, am wrote:
> I posted a similar q on 6th but did not see the post nor any responses.
> (I may be still in the newbie doghouse :-) ...)


Your post was received. it is answered at
<https://mail.python.org/pipermail/mailman-users/2015-November/080074.html>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Mon Nov  9 10:51:04 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 9 Nov 2015 07:51:04 -0800
Subject: [Mailman-Users] Mailman only delivers inside server
In-Reply-To: <3nvbML0hlMzFpWN@mail.python.org>
References: <3nvbML0hlMzFpWN@mail.python.org>
Message-ID: <5640C0E8.6040302@msapiro.net>

On 11/9/15 7:00 AM, kapuzino at web.de wrote:
> 
> But what is the diffrence between sending with postfix using a distributor generated in plesk-email-panel and the connection of mailman to postfix ?
> If it is no internal mailman issue, though a special configuration issue needed for mailman, when postfix is sending everything well, but mailman tasks.


The primary sources of Postfix support can be found at
<http://www.postfix.org/>.


> What information could help to localize the failure ?


The output of 'postconf -n' and all of the postfix log entries relating
to delivery of a message from Mailman to Postfix.

Possibly Mailman's 'smtp' and 'smtp-failure' log entries and the
contents of Postfix master.cf and other files referenced from main.cf.

If I had to guess, my guess would be that 'localhost' or it's equivalent
IP (possibly IPv6 - see /etc/hosts) is not in Postfix mynetworks in
main.cf and the resultant issue is 'relay access denied'.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From her at adm.ku.dk  Mon Nov  9 07:50:20 2015
From: her at adm.ku.dk (Henrik Rasmussen)
Date: Mon, 9 Nov 2015 12:50:20 +0000
Subject: [Mailman-Users] Part of moderated mails postponed to later
 digests
In-Reply-To: <563A2D22.3070005@msapiro.net>
References: <6DCC3E5DA06FE346B4DE4876C4F2713D014D8582EF@P2KITMBX06WC03.unicph.domain>
 <5638E182.7080305@msapiro.net>
 <6DCC3E5DA06FE346B4DE4876C4F2713D014D858F18@P2KITMBX06WC03.unicph.domain>
 <563A2D22.3070005@msapiro.net>
Message-ID: <6DCC3E5DA06FE346B4DE4876C4F2713D014D85D5B7@P2KITMBX06WC03.unicph.domain>

Sorry for the late reply.

> Mark Sapiro [mailto:mark at msapiro.net]
> Sent: 4. november 2015 17:07

> Anyway, I understood your OP to say digests were being sent only weekly when batches of held messages are approved.
That was an example of the frequency, held mails is being approved by the moderator. When the moderator approves a bunch of mails in one week, the list admin claims that only some of the mails are posted to the list, but that some of the mails will not be sent in a digest until the next moderation a week later (example).

> if you don't want digests triggered on size
The list admin doesn't need digests sent according to size so a digest_size_threshhold = 10000 should be fine. The moderator wants to have the mails included in a digest the next noon after approving the mails.

> and your mailman is older than 2.1.16
I run Mailman 2.1.12 which is the latest available for yum update on RHEL6. I think the last Mailman version on RHEL7 is 2.1.15, but we're not there yet.

I just confirmed that senddigest is run every noon:
   /var/log/cron:Nov  8 12:00:01 p1kitlst01l CROND[18196]: (mailman) CMD (/usr/lib/mailman/cron/senddigests)
   /var/log/cron:Nov  9 12:00:01 p1kitlst01l CROND[27052]: (mailman) CMD (/usr/lib/mailman/cron/senddigests)

> If this is not the case, either cron/senddigests is not running every noon or it is somehow skipping this list or encountering errors with this list. Is there anything in Mailman's error log?

The error log doesn't show anything in association with the list in question. 

According to the vette log, some are discarded by moderator actions, :
   Oct 25 10:20:42 2015 (16801) Precedence: bulk message discarded by: LISTNAME-request at list.example.com
And of cause a lot of "Reason: Non-members" and spam-like messages "Reason: Your message was deemed inappropriate by the moderator.".

Regards, Henrik Rasmussen

-----Oprindelig meddelelse-----
Fra: Mark Sapiro [mailto:mark at msapiro.net] 
Sendt: 4. november 2015 17:07
Til: Henrik Rasmussen; mailman-users at python.org
Emne: Re: SV: [Mailman-Users] Part of moderated mails postponed to later digests

On 11/04/2015 02:09 AM, Henrik Rasmussen wrote:
>> My best guess is you are triggering digests on size (Digest options -> digest_size_threshold) and either not periodically (digest_send_periodic = No) or cron/senddigests runs only weekly.
> 
> To clarify, he has set digest_size_threshold to 10000 and digest_send_periodic =Yes. /usr/lib/mailman/cron/senddigests is run by cron every day at noon.
> 
> Other digest settings is
> 
> nondigestable = True
> scrub_nondigest = False
> digestable = True
> digest_is_default = 1
> mime_is_default_digest = 0
> digest_size_threshhold = 10000
> digest_send_periodic = 1
> digest_volume_frequency = 3
> discard_these_nonmembers = (241 members)
> ban_list = (188 members)
> 
> The digest_size_threshhold = 10000 may likely course the problem, but is necessary as a digest containing a single mail is sent each time a mail is sent to the list-address as suggested by https://mail.python.org/pipermail/mailman-users/2014-August/077622.html.


That's a bug <https://bugs.launchpad.net/mailman/+bug/558274> which was
fixed in MM 2.1.16, but in any case, if you don't want digests triggered
on size and your mailman is older than 2.1.16, setting
digest_size_threshhold = 10000 (KB = 10 MB) should stop digests being
triggered on size unless your list has a lot of large graphics attached
to posts. I.e. 10MB / 100 posts is still 100KB per post.


Anyway, I understood your OP to say digests were being sent only weekly
when batches of held messages are approved. If as you say,
digest_send_periodic =Yes and /usr/lib/mailman/cron/senddigests is run
by cron every day at noon, even if 50 messages are approved and 30 of
them are sent immediately in a digest (triggered on size), the other 20
should be sent in a digest the next noon.

If this is not the case, either cron/senddigests is not running every
noon or it is somehow skipping this list or encountering errors with
this list. Is there anything in Mailman's error log?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mrbrklyn at panix.com  Mon Nov  9 18:49:53 2015
From: mrbrklyn at panix.com (Ruben)
Date: Mon, 9 Nov 2015 18:49:53 -0500
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <563A3BEC.9090700@msapiro.net>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
Message-ID: <56413121.3040904@panix.com>



On 11/04/2015 12:10 PM, Mark Sapiro wrote:
> On 11/03/2015 06:42 PM, Ruben Safir wrote:
>> I'm converting my mailing lists from majordomo to mailman using postfix.
>>   A lot of little issues have cropped up but the biggest problem I have
>> currently is that it seems that mailman is converting files to 64 bit
>> which is screwing up my home cooked archives but good.
>
>
> Actually, Mailman is sending messages in UTF-8 character set as base64
> encoded. Actually, many MUAs will also base64 encode UTF-8 message bodies.
>
> This is actually a python email library thing. You can change the
> encoding for UTF-8 message bodies with the following patch to
> Mailman/Message.py, but if your messages are not predominately ascii
> text, you won't like the result any better.
>
> diff -u 2.1/Mailman/Message.py 21/Mailman/Message.py
> --- 2.1/Mailman/Message.py	2014-05-17 18:34:20.240159000 -0700
> +++ 21/Mailman/Message.py	2015-11-04 08:50:31.772899979 -0800
> @@ -36,6 +36,10 @@
>   from Mailman import mm_cfg
>   from Mailman import Utils
>
> +email.Charset.add_charset('utf-8',
> +                          email.Charset.SHORTEST,
> +                          email.Charset.QP,
> +                          'utf-8')
>   COMMASPACE = ', '
>
>   mo = re.match(r'([\d.]+)', email.__version__)
>

I made this change and I think I'm getting no throughput through the 
list altogether :(

I have the mail logs being tailed and all the mailman/logs but I'm not 
spotting the problem


==> /var/lib/mailman/logs/post <==
Nov 09 18:34:12 2015 (28269) post to hangout from mrbrklyn at panix.com, 
size=6608, message-id=<563AE93A.2000208 at panix.com>, 11 failures


/usr/lib/mailman/Mailman/Message.py
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
import email.Message
import email.Utils
from email.Charset import Charset
from email.Header import Header

from types import ListType, StringType

from Mailman import mm_cfg
from Mailman import Utils
email.Charset.add_charset('utf-8',
                           email.Charset.SHORTEST,
                           email.Charset.QP,
                           'utf-8')

COMMASPACE = ', '

mo = re.match(r'([\d.]+)', email.__version__)
VERSION = tuple([int(s) for s in mo.group().split('.')])


From mark at msapiro.net  Mon Nov  9 19:12:27 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 09 Nov 2015 19:12:27 -0500
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <8B6BC652-7EE2-48D0-98FA-B5C9B49602DA@msapiro.net>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <56413121.3040904@panix.com>
 <8B6BC652-7EE2-48D0-98FA-B5C9B49602DA@msapiro.net>
Message-ID: <7704D2BD-638F-4204-9E79-1D9CAD59CFF3@msapiro.net>



On November 9, 2015 6:56:56 PM EST, Mark Sapiro <mark at msapiro.net> wrote:
>
>
>On November 9, 2015 6:49:53 PM EST, Ruben <mrbrklyn at panix.com> wrote:
>>
>>
>
>>I made this change and I think I'm getting no throughput through the 
>>list altogether :(
>>
>>I have the mail logs being tailed and all the mailman/logs but I'm not
>
>>spotting the problem
>>
>>
>>==> /var/lib/mailman/logs/post <==
>>Nov 09 18:34:12 2015 (28269) post to hangout from mrbrklyn at panix.com, 
>>size=6608, message-id=<563AE93A.2000208 at panix.com>, 11 failures
>
>
>What's in Mailman's smtp-failure log for these?
>
>Anything in Mailman's error log?
>
>What's in the mail log for these?


-- 
Mark Sapiro <mark at msapiro.net>
Sent from my Not_an_iThing with standards compliant, open source software.

From mrbrklyn at panix.com  Mon Nov  9 20:11:35 2015
From: mrbrklyn at panix.com (Ruben)
Date: Mon, 9 Nov 2015 20:11:35 -0500
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <7704D2BD-638F-4204-9E79-1D9CAD59CFF3@msapiro.net>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <56413121.3040904@panix.com>
 <8B6BC652-7EE2-48D0-98FA-B5C9B49602DA@msapiro.net>
 <7704D2BD-638F-4204-9E79-1D9CAD59CFF3@msapiro.net>
Message-ID: <56414447.9050003@panix.com>



On 11/09/2015 07:12 PM, Mark Sapiro wrote:
>
>
> On November 9, 2015 6:56:56 PM EST, Mark Sapiro <mark at msapiro.net> wrote:
>>
>>
>> On November 9, 2015 6:49:53 PM EST, Ruben <mrbrklyn at panix.com> wrote:
>>>
>>>
>>
>>> I made this change and I think I'm getting no throughput through the
>>> list altogether :(
>>>
>>> I have the mail logs being tailed and all the mailman/logs but I'm not
>>
>>> spotting the problem
>>>
>>>
>>> ==> /var/lib/mailman/logs/post <==
>>> Nov 09 18:34:12 2015 (28269) post to hangout from mrbrklyn at panix.com,
>>> size=6608, message-id=<563AE93A.2000208 at panix.com>, 11 failures
>>
>>
>> What's in Mailman's smtp-failure log for these?

what would I be looking for here?  there is nothing but failed domain 
lookups, none of then from working email lists, that I see

>>
>> Anything in Mailman's error log?


nothing since Oct 25

www:~ # tail -f /var/lib/mailman/logs/error
[Errno 2] No such file or directory: 
'/var/lib/mailman/qfiles/out/1445396648.951926+5b9ff5008329d53f6f74502963da062941598f03.bak'
Oct 25 01:32:55 2015 mailmanctl(4827): No child with pid: 30374
Oct 25 01:32:55 2015 mailmanctl(4827): [Errno 3] No such process
Oct 25 01:32:55 2015 mailmanctl(4827): Stale pid file removed.
Oct 25 01:32:58 2015 mailmanctl(4830): PID unreadable in: 
/var/lib/mailman/data/master-qrunner.pid
Oct 25 01:32:58 2015 mailmanctl(4830): [Errno 2] No such file or 
directory: '/var/lib/mailman/data/master-qrunner.pid'
Oct 25 01:32:58 2015 mailmanctl(4830): Is qrunner even running?
Oct 25 01:33:01 2015 mailmanctl(4833): PID unreadable in: 
/var/lib/mailman/data/master-qrunner.pid
Oct 25 01:33:01 2015 mailmanctl(4833): [Errno 2] No such file or 
directory: '/var/lib/mailman/data/master-qrunner.pid'
Oct 25 01:33:01 2015 mailmanctl(4833): Is qrunner even running?

>>
>> What's in the mail log for these?
>
>

2015-11-09T19:58:56.574951-05:00 www postfix/smtpd[28205]: NOQUEUE: 
reject: RCPT from www.mrbrklyn.com[96.57.23.82]: 450 4.1.2 
<lai at council.nyc>: Recipient address rejected: Domain not found; 
from=<hangout-bounces at nylxs.com> to=<lai at council.nyc> proto=ESMTP 
helo=<www.mrbrklyn.com>
2015-11-09T19:58:56.582345-05:00 www postfix/smtpd[28205]: NOQUEUE: 
reject: RCPT from www.mrbrklyn.com[96.57.23.82]: 450 4.1.2 
<ksnyder at stclair.k12>: Recipient address rejected: Domain not found; 
from=<hangout-bounces at nylxs.com> to=<ksnyder at stclair.k12> proto=ESMTP 
helo=<www.mrbrklyn.com>
2015-11-09T19:58:56.590302-05:00 www postfix/smtpd[28205]: NOQUEUE: 
reject: RCPT from www.mrbrklyn.com[96.57.23.82]: 450 4.1.2 
<matt at ele.uri>: Recipient address rejected: Domain not found; 
from=<hangout-bounces at nylxs.com> to=<matt at ele.uri> proto=ESMTP 
helo=<www.mrbrklyn.com>
2015-11-09T19:58:56.598704-05:00 www postfix/smtpd[28205]: NOQUEUE: 
reject: RCPT from www.mrbrklyn.com[96.57.23.82]: 450 4.1.2 
<sbrown at emoglen.law>: Recipient address rejected: Domain not found; 
from=<hangout-bounces at nylxs.com> to=<sbrown at emoglen.law> proto=ESMTP 
helo=<www.mrbrklyn.com>
2015-11-09T19:58:56.606356-05:00 www postfix/smtpd[28205]: NOQUEUE: 
reject: RCPT from www.mrbrklyn.com[96.57.23.82]: 450 4.1.2 
<mbmiller at taxa.epi>: Recipient address rejected: Domain not found; 
from=<hangout-bounces at nylxs.com> to=<mbmiller at taxa.epi> proto=ESMTP 
helo=<www.mrbrklyn.com>
2015-11-09T19:58:56.615825-05:00 www postfix/smtpd[28205]: disconnect 
from www.mrbrklyn.com[96.57.23.82]
2015-11-09T19:58:56.747491-05:00 www postfix/smtpd[28737]: connect from 
www.mrbrklyn.com[96.57.23.82]
2015-11-09T19:58:56.757163-05:00 www postfix/smtpd[28737]: NOQUEUE: 
reject: RCPT from www.mrbrklyn.com[96.57.23.82]: 450 4.1.2 
<zbrown at tumblerings.org>: Recipient address rejected: Domain not found; 
from=<hangout-bounces at nylxs.com> to=<zbrown at tumblerings.org> proto=ESMTP 
helo=<www.mrbrklyn.com>

From mrbrklyn at panix.com  Mon Nov  9 20:15:49 2015
From: mrbrklyn at panix.com (Ruben)
Date: Mon, 9 Nov 2015 20:15:49 -0500
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <7704D2BD-638F-4204-9E79-1D9CAD59CFF3@msapiro.net>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <56413121.3040904@panix.com>
 <8B6BC652-7EE2-48D0-98FA-B5C9B49602DA@msapiro.net>
 <7704D2BD-638F-4204-9E79-1D9CAD59CFF3@msapiro.net>
Message-ID: <56414545.2030502@panix.com>



On 11/09/2015 07:12 PM, Mark Sapiro wrote:
>
>
> On November 9, 2015 6:56:56 PM EST, Mark Sapiro <mark at msapiro.net> wrote:
>>
>>
>> On November 9, 2015 6:49:53 PM EST, Ruben <mrbrklyn at panix.com> wrote:
>>>
>>>
>>
>>> I made this change and I think I'm getting no throughput through the
>>> list altogether :(
>>>
>>> I have the mail logs being tailed and all the mailman/logs but I'm not
>>
>>> spotting the problem
>>>
>>>
>>> ==> /var/lib/mailman/logs/post <==
>>> Nov 09 18:34:12 2015 (28269) post to hangout from mrbrklyn at panix.com,
>>> size=6608, message-id=<563AE93A.2000208 at panix.com>, 11 failures
>>
>>
>> What's in Mailman's smtp-failure log for these?
>>
>> Anything in Mailman's error log?
>>
>> What's in the mail log for these?
>
>
FWIW, I greps one of the message ids for a submited message

www:~ # grep 563F4BDF.8010808 at panix.com  /var/log/mail
2015-11-08T08:19:52.449351-05:00 www postfix/cleanup[21142]: 
48B12161158: message-id=<563F4BDF.8010808 at panix.com>
2015-11-08T08:55:41.806591-05:00 www postfix/cleanup[21512]: 
B7863161158: message-id=<563F4BDF.8010808 at panix.com>
2015-11-08T08:56:51.592725-05:00 www postfix/cleanup[21512]: 
EE83216249E: message-id=<563F4BDF.8010808 at panix.com>
2015-11-08T08:56:53.680355-05:00 www postfix/cleanup[21512]: 
0AF2B161158: message-id=<563F4BDF.8010808 at panix.com>
2015-11-08T08:56:54.786350-05:00 www postfix/smtp[21541]: 0AF2B161158: 
to=<juanag at msn.com>, relay=mx1.hotmail.com[134.170.2.199]:25, delay=2.8, 
delays=1.7/0.6/0.16/0.27, dsn=2.0.0, status=sent (250 
<563F4BDF.8010808 at panix.com> Queued mail for delivery)
2015-11-08T08:56:56.824374-05:00 www postfix/smtp[21527]: 0AF2B161158: 
to=<jordan at jordanloyd.com>, 
relay=jordanloyd-com.mail.protection.outlook.com[216.32.181.170]:25, 
delay=4.8, delays=1.7/0.08/0.82/2.2, dsn=2.6.0, status=sent (250 2.6.0 
563F4BDF.8010808 at panix.com> [InternalId=28059021348382, 
Hostname=BY1PR02MB1274.namprd02.prod.outlook.com] Queued mail for delivery)

From mark at msapiro.net  Mon Nov  9 20:30:16 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 09 Nov 2015 20:30:16 -0500
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <56414545.2030502@panix.com>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <56413121.3040904@panix.com>
 <8B6BC652-7EE2-48D0-98FA-B5C9B49602DA@msapiro.net>
 <7704D2BD-638F-4204-9E79-1D9CAD59CFF3@msapiro.net>
 <56414545.2030502@panix.com>
Message-ID: <364FD51E-154D-43DE-BD6D-B1D132F77FB7@msapiro.net>



On November 9, 2015 8:15:49 PM EST, Ruben <mrbrklyn at panix.com> wrote:

>2015-11-08T08:56:54.786350-05:00 www postfix/smtp[21541]: 0AF2B161158: 
>to=<juanag at msn.com>, relay=mx1.hotmail.com[134.170.2.199]:25,
>delay=2.8, 
>delays=1.7/0.6/0.16/0.27, dsn=2.0.0, status=sent (250 
><563F4BDF.8010808 at panix.com> Queued mail for delivery)
>2015-11-08T08:56:56.824374-05:00 www postfix/smtp[21527]: 0AF2B161158: 
>to=<jordan at jordanloyd.com>, 
>relay=jordanloyd-com.mail.protection.outlook.com[216.32.181.170]:25, 
>delay=4.8, delays=1.7/0.08/0.82/2.2, dsn=2.6.0, status=sent (250 2.6.0 
>563F4BDF.8010808 at panix.com> [InternalId=28059021348382, 
>Hostname=BY1PR02MB1274.namprd02.prod.outlook.com] Queued mail for
>delivery)


So the messages were delivered.  What's the problem?


-- 
Mark Sapiro <mark at msapiro.net>
Sent from my Not_an_iThing with standards compliant, open source software.

From mrbrklyn at panix.com  Mon Nov  9 22:18:53 2015
From: mrbrklyn at panix.com (Ruben Safir)
Date: Mon, 9 Nov 2015 22:18:53 -0500
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <364FD51E-154D-43DE-BD6D-B1D132F77FB7@msapiro.net>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <56413121.3040904@panix.com>
 <8B6BC652-7EE2-48D0-98FA-B5C9B49602DA@msapiro.net>
 <7704D2BD-638F-4204-9E79-1D9CAD59CFF3@msapiro.net>
 <56414545.2030502@panix.com>
 <364FD51E-154D-43DE-BD6D-B1D132F77FB7@msapiro.net>
Message-ID: <5641621D.9010502@panix.com>

On 11/09/2015 08:30 PM, Mark Sapiro wrote:
> 
> 
> On November 9, 2015 8:15:49 PM EST, Ruben <mrbrklyn at panix.com> wrote:
> 
>> 2015-11-08T08:56:54.786350-05:00 www postfix/smtp[21541]: 0AF2B161158: 
>> to=<juanag at msn.com>, relay=mx1.hotmail.com[134.170.2.199]:25,
>> delay=2.8, 
>> delays=1.7/0.6/0.16/0.27, dsn=2.0.0, status=sent (250 
>> <563F4BDF.8010808 at panix.com> Queued mail for delivery)
>> 2015-11-08T08:56:56.824374-05:00 www postfix/smtp[21527]: 0AF2B161158: 
>> to=<jordan at jordanloyd.com>, 
>> relay=jordanloyd-com.mail.protection.outlook.com[216.32.181.170]:25, 
>> delay=4.8, delays=1.7/0.08/0.82/2.2, dsn=2.6.0, status=sent (250 2.6.0 
>> 563F4BDF.8010808 at panix.com> [InternalId=28059021348382, 
>> Hostname=BY1PR02MB1274.namprd02.prod.outlook.com] Queued mail for
>> delivery)
> 
> 
> So the messages were delivered.  What's the problem?
> 
> 


that is 2 of 108 :)

From mark at msapiro.net  Tue Nov 10 08:43:08 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 10 Nov 2015 05:43:08 -0800
Subject: [Mailman-Users] 64 bit encoding
In-Reply-To: <5641621D.9010502@panix.com>
References: <56397081.7030408@panix.com> <563A3BEC.9090700@msapiro.net>
 <56413121.3040904@panix.com>
 <8B6BC652-7EE2-48D0-98FA-B5C9B49602DA@msapiro.net>
 <7704D2BD-638F-4204-9E79-1D9CAD59CFF3@msapiro.net>
 <56414545.2030502@panix.com>
 <364FD51E-154D-43DE-BD6D-B1D132F77FB7@msapiro.net>
 <5641621D.9010502@panix.com>
Message-ID: <5641F46C.1010607@msapiro.net>

On 11/9/15 7:18 PM, Ruben Safir wrote:
> On 11/09/2015 08:30 PM, Mark Sapiro wrote:
>>
>> So the messages were delivered.  What's the problem?
>>
>>
> 
> 
> that is 2 of 108 :)


So are the other 106 eligible to receive non-digest messages; i.e.
regular, not digest members with delivery enabled?  If so, what does
postfix say in the mail log about those 106 recipients?

Please note that I am in an airport after traveling for 10 hours and
still have over 7 hours to my final destination, so don't expect quick
replies from me. I will have little time to reply for some while. It
will help if you can actually find some log messages that indicate a
problem.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From pawses at gmail.com  Wed Nov 11 01:16:37 2015
From: pawses at gmail.com (Paw)
Date: Wed, 11 Nov 2015 07:16:37 +0100
Subject: [Mailman-Users] listname@lists.domain does not work,
 listname@domain does
Message-ID: <CAP53jTxJS23na8TYfXH-yY5g2hgbGYt4xivuReH0=ePFpCw1FA@mail.gmail.com>

Hi Mailman community,

I am using postfix with dovecot on a Debian server. I followed <
http://wiki.list.org/x/ZoCj> for setting up mailman, but something is wrong
with my postfix configuration.

I can create new lists on http://lists.mydomain and when subscribing I get
the subscription notification from mailman. But for sending to the list, I
have to use listname at domain instead of listname at lists.domain. Mails sent to
listname at lists.domain just seems to get lost(don't bounce or end up in
/var/log/mail.log)
I am sorry so disturb with such a question, but I am really stuck.

This is the full log when sending a mail to test at mydomain. The mail is
being delivered to local user tester's inbox.
Nov 11 06:41:59 loki postfix/smtpd[4543]: connect from
mail-lf0-f49.google.com[209.85.215.49]
Nov 11 06:41:59 loki postfix/smtpd[4543]: 7F4E724C09B: client=
mail-lf0-f49.google.com[209.85.215.49]
Nov 11 06:41:59 loki postfix/cleanup[4914]: 7F4E724C09B:
message-id=<CAP53jTxugaP79R=Uqr_1zipb2z_EVQRno=
LXJJ7KUJAaOBgxZA at mail.gmail.com>
Nov 11 06:41:59 loki postfix/qmgr[31085]: 7F4E724C09B: from=<
pawses at gmail.com>, size=1680, nrcpt=1 (queue active)
Nov 11 06:41:59 loki postfix/smtpd[4543]: disconnect from
mail-lf0-f49.google.com[209.85.215.49]
Nov 11 06:41:59 loki postfix/local[4915]: 7F4E724C09B: to=<
test at studentergaarden.dk>, relay=local, delay=0.63,
delays=0.49/0.01/0/0.13, dsn=2.0.0, status=sent (delivered to command:
/var/lib/mailman/mail/mailman post test)
Nov 11 06:41:59 loki postfix/qmgr[31085]: 7F4E724C09B: removed
Nov 11 06:42:01 loki postfix/smtpd[4543]: connect from localhost[127.0.0.1]
Nov 11 06:42:01 loki postfix/smtpd[4543]: 44A6324C0A7:
client=localhost[127.0.0.1]
Nov 11 06:42:01 loki postfix/cleanup[4914]: 44A6324C0A7:
message-id=<CAP53jTxugaP79R=Uqr_1zipb2z_EVQRno=
LXJJ7KUJAaOBgxZA at mail.gmail.com>
Nov 11 06:42:01 loki postfix/qmgr[31085]: 44A6324C0A7: from=<
test-bounces at lists.studentergaarden.dk>, size=3127, nrcpt=1 (queue active)
Nov 11 06:42:01 loki postfix/smtpd[4543]: 5D5F124C0E1:
client=localhost[127.0.0.1]
Nov 11 06:42:01 loki postfix/cleanup[4914]: 5D5F124C0E1:
message-id=<CAP53jTxugaP79R=Uqr_1zipb2z_EVQRno=
LXJJ7KUJAaOBgxZA at mail.gmail.com>
Nov 11 06:42:01 loki postfix/qmgr[31085]: 5D5F124C0E1: from=<
test-bounces at lists.studentergaarden.dk>, size=3137, nrcpt=1 (queue active)
Nov 11 06:42:01 loki postfix/smtpd[4543]: disconnect from
localhost[127.0.0.1]
Nov 11 06:42:01 loki dovecot: lda(tester):
msgid=<CAP53jTxugaP79R=Uqr_1zipb2z_EVQRno=LXJJ7KUJAaOBgxZA at mail.gmail.com>:
saved mail to INBOX
Nov 11 06:42:01 loki postfix/local[4915]: 5D5F124C0E1: to=<
tester at studentergaarden.dk>, relay=local, delay=0.19, delays=0.05/0/0/0.13,
dsn=2.0.0, status=sent (delivered to command: /usr/lib/dovecot/deliver)
Nov 11 06:42:01 loki postfix/qmgr[31085]: 5D5F124C0E1: removed
Nov 11 06:42:01 loki postfix/smtp[4944]: 44A6324C0A7: to=<pawses at gmail.com>,
relay=gmail-smtp-in.l.google.com[173.194.222.27]:25, delay=0.49,
delays=0.1/0.01/0.26/0.12, dsn=2.0.0, status=sent (250 2.0.0 OK 1447220691
38si4916356lfq.17 - gsmtp)
Nov 11 06:42:01 loki postfix/qmgr[31085]: 44A6324C0A7: removed


My config is as follow:
-----------------------------------------------
Dumps of postconf, mm_cfg.py virtual-mailman,
virtual_to_transport and transport-mailman
-----------------------------------------------
$ postconf -n
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases hash:/var/lib/mailman/data/aliases
append_dot_mydomain = no
biff = no
body_checks = pcre:/etc/postfix/bodychecks
config_directory = /etc/postfix
dovecot_destination_recipient_limit = 1
header_checks = pcre:/etc/postfix/headerchecks
inet_interfaces = mail.studentergaarden.dk, localhost
inet_protocols = ipv4
mailbox_command = /usr/lib/dovecot/deliver
mailbox_size_limit = 2048000000
message_size_limit = 40480000
mydestination = $mydomain, $myhostname, localhost.$mydomain, localhost
myhostname = mail.studentergaarden.dk
mynetworks = 130.226.169.128/26, 172.16.0.0/16, 127.0.0.0/8
myorigin = /etc/mailname
recipient_delimiter = +
relayhost =
smtpd_banner = $myhostname ESMTP $mail_name
smtpd_client_restrictions = permit_mynetworks, reject_unknown_client
smtpd_hard_error_limit = 10
smtpd_helo_required = yes
smtpd_helo_restrictions = reject_invalid_hostname
smtpd_recipient_restrictions = permit_mynetworks, reject_rbl_client
zen.spamhaus.org, reject_unauth_destination, reject_unauth_pipelining,
smtpd_relay_restrictions = permit_mynetworks, reject_rbl_client
zen.spamhaus.org, reject_unauth_destination, reject_unauth_pipelining,
smtpd_sender_restrictions = reject_non_fqdn_sender,
reject_unknown_sender_domain
smtpd_soft_error_limit = 8
transport_maps = hash:/var/lib/mailman/data/transport-mailman
virtual_mailbox_domains = mail.studentergaarden.dk
virtual_transport = dovecot

$ cat  /etc/mailman/mm_cfg.py
... few irrelevant lines truncated ...
DEFAULT_EMAIL_HOST = 'lists.studentergaarden.dk'
DEFAULT_URL_HOST   = 'lists.studentergaarden.dk'
MTA='Postfix'
POSTFIX_STYLE_VIRTUAL_DOMAINS = ['lists.studentergaarden.dk']
POSTFIX_MAP_CMD = '/var/lib/mailman/data/transport-mailman'

---
Whether POSTFIX_MAP_CMD is commented out or not, does not seems to make a
difference

$ ll /var/lib/mailman/data
total 80K
-rw-r----- 1 root     list   41 Oct 22 05:15 adm.pw
-rw-rw---- 1 root     list 5,6K Nov 11 06:13 aliases
-rw-rw---- 1 list     list  12K Nov 11 06:14 aliases.db
-rw-r----- 1 root     list   41 Nov 11 04:40 creator.pw
-rw-rw-r-- 1 root     list   10 Oct 22 03:11 last_mailman_version
-rw-rw-r-- 1 root     list  14K Apr  6  2015 sitelist.cfg
-rwxrwxr-x 1 root     list    0 Nov 11 03:53 transport-mailman
-rw-r--r-- 1 root     list  12K Nov 11 04:10 transport-mailman.db
-rw-rw---- 1 www-data list 5,3K Nov 11 06:13 virtual-mailman
-rw-rw---- 1 list     list  12K Nov 11 06:14 virtual-mailman.db
-rwxr-x--- 1 root     list  153 Nov 11 04:00 virtual_to_transport

$ cat /var/lib/mailman/data/virtual-mailman
# STANZA START: test
# CREATED: Wed Nov 11 05:55:30 2015
test at lists.studentergaarden.dk              test
test-admin at lists.studentergaarden.dk        test-admin
test-bounces at lists.studentergaarden.dk      test-bounces
test-confirm at lists.studentergaarden.dk      test-confirm
test-join at lists.studentergaarden.dk         test-join
test-leave at lists.studentergaarden.dk        test-leave
test-owner at lists.studentergaarden.dk        test-owner
test-request at lists.studentergaarden.dk      test-request
test-subscribe at lists.studentergaarden.dk    test-subscribe
test-unsubscribe at lists.studentergaarden.dk  test-unsubscribe
# STANZA END: test
(the Mailman list is also included in the above file)

$ cat /var/lib/mailman/data/virtual_to_transport
sed -r -e 's/(^[^#]\S+\s+).+$/\1local:/' $1 >
/var/lib/mailman/data/transport-mailman
/usr/sbin/postmap /var/lib/mailman/data/transport-mailman

$ cat /var/lib/mailman/data/transport-mailman
<<<EOF (empty)

There are no errors in relevant postfix/mailman logs.
I do not use postfix-to-mailman.py.

What have I done wrong with the postfix configuration? And where does the
listname at list.domain maills go?

$ uname -a
Linux loki 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u5 (2015-10-09)
x86_64 GNU/Linux
$ /usr/sbin/check_perms
No problems found
$ /usr/lib/mailman/bin/version
Using Mailman version: 2.1.18


  Thanks in advance!
/Paw

From mark at msapiro.net  Wed Nov 11 12:31:31 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 11 Nov 2015 09:31:31 -0800
Subject: [Mailman-Users] listname@lists.domain does not work,
 listname@domain does
In-Reply-To: <CAP53jTxJS23na8TYfXH-yY5g2hgbGYt4xivuReH0=ePFpCw1FA@mail.gmail.com>
References: <CAP53jTxJS23na8TYfXH-yY5g2hgbGYt4xivuReH0=ePFpCw1FA@mail.gmail.com>
Message-ID: <56437B73.8050104@msapiro.net>

On 11/10/15 10:16 PM, Paw wrote:
> Hi Mailman community,
> 
> I am using postfix with dovecot on a Debian server. I followed <
> http://wiki.list.org/x/ZoCj> for setting up mailman, but something is wrong
> with my postfix configuration.


You may not have needed to follow that guide. See below.


> I can create new lists on http://lists.mydomain and when subscribing I get
> the subscription notification from mailman. But for sending to the list, I
> have to use listname at domain instead of listname at lists.domain. Mails sent to
> listname at lists.domain just seems to get lost(don't bounce or end up in
> /var/log/mail.log)
> I am sorry so disturb with such a question, but I am really stuck.
> 
> This is the full log when sending a mail to test at mydomain. The mail is
> being delivered to local user tester's inbox.
> Nov 11 06:41:59 loki postfix/smtpd[4543]: connect from
> mail-lf0-f49.google.com[209.85.215.49]
> Nov 11 06:41:59 loki postfix/smtpd[4543]: 7F4E724C09B: client=
> mail-lf0-f49.google.com[209.85.215.49]
> Nov 11 06:41:59 loki postfix/cleanup[4914]: 7F4E724C09B:
> message-id=<CAP53jTxugaP79R=Uqr_1zipb2z_EVQRno=
> LXJJ7KUJAaOBgxZA at mail.gmail.com>
> Nov 11 06:41:59 loki postfix/qmgr[31085]: 7F4E724C09B: from=<
> pawses at gmail.com>, size=1680, nrcpt=1 (queue active)
> Nov 11 06:41:59 loki postfix/smtpd[4543]: disconnect from
> mail-lf0-f49.google.com[209.85.215.49]
> Nov 11 06:41:59 loki postfix/local[4915]: 7F4E724C09B: to=<
> test at studentergaarden.dk>, relay=local, delay=0.63,
> delays=0.49/0.01/0/0.13, dsn=2.0.0, status=sent (delivered to command:
> /var/lib/mailman/mail/mailman post test)


Mail is received from google and delivered via the local transport and
Mailman aliases to the appropriate pipe. It appears that
'studentergaarden.dk' is local so the local transport is used. Dovecot
is not involved because it is a virtual transport and not a local one.
Transport maps is not involved either because it only mentions
'lists.studentergaarden.dk' and not 'studentergaarden.dk' addresses.


> Nov 11 06:41:59 loki postfix/qmgr[31085]: 7F4E724C09B: removed
> Nov 11 06:42:01 loki postfix/smtpd[4543]: connect from localhost[127.0.0.1]
> Nov 11 06:42:01 loki postfix/smtpd[4543]: 44A6324C0A7:
> client=localhost[127.0.0.1]
> Nov 11 06:42:01 loki postfix/cleanup[4914]: 44A6324C0A7:
> message-id=<CAP53jTxugaP79R=Uqr_1zipb2z_EVQRno=
> LXJJ7KUJAaOBgxZA at mail.gmail.com>
> Nov 11 06:42:01 loki postfix/qmgr[31085]: 44A6324C0A7: from=<
> test-bounces at lists.studentergaarden.dk>, size=3127, nrcpt=1 (queue active)
> Nov 11 06:42:01 loki postfix/smtpd[4543]: 5D5F124C0E1:
> client=localhost[127.0.0.1]
> Nov 11 06:42:01 loki postfix/cleanup[4914]: 5D5F124C0E1:
> message-id=<CAP53jTxugaP79R=Uqr_1zipb2z_EVQRno=
> LXJJ7KUJAaOBgxZA at mail.gmail.com>
> Nov 11 06:42:01 loki postfix/qmgr[31085]: 5D5F124C0E1: from=<
> test-bounces at lists.studentergaarden.dk>, size=3137, nrcpt=1 (queue active)
> Nov 11 06:42:01 loki postfix/smtpd[4543]: disconnect from
> localhost[127.0.0.1]
> Nov 11 06:42:01 loki dovecot: lda(tester):
> msgid=<CAP53jTxugaP79R=Uqr_1zipb2z_EVQRno=LXJJ7KUJAaOBgxZA at mail.gmail.com>:
> saved mail to INBOX
> Nov 11 06:42:01 loki postfix/local[4915]: 5D5F124C0E1: to=<
> tester at studentergaarden.dk>, relay=local, delay=0.19, delays=0.05/0/0/0.13,
> dsn=2.0.0, status=sent (delivered to command: /usr/lib/dovecot/deliver)


Here mail to tester is delivered to dovecot via the 'local' transport,
presumably because of an alias for 'tester' in /etc/aliases.


> Nov 11 06:42:01 loki postfix/qmgr[31085]: 5D5F124C0E1: removed
> Nov 11 06:42:01 loki postfix/smtp[4944]: 44A6324C0A7: to=<pawses at gmail.com>,
> relay=gmail-smtp-in.l.google.com[173.194.222.27]:25, delay=0.49,
> delays=0.1/0.01/0.26/0.12, dsn=2.0.0, status=sent (250 2.0.0 OK 1447220691
> 38si4916356lfq.17 - gsmtp)
> Nov 11 06:42:01 loki postfix/qmgr[31085]: 44A6324C0A7: removed
> 
> 
> My config is as follow:
> -----------------------------------------------
> Dumps of postconf, mm_cfg.py virtual-mailman,
> virtual_to_transport and transport-mailman
> -----------------------------------------------
> $ postconf -n
> alias_database = hash:/etc/aliases
> alias_maps = hash:/etc/aliases hash:/var/lib/mailman/data/aliases
> append_dot_mydomain = no
> biff = no
> body_checks = pcre:/etc/postfix/bodychecks
> config_directory = /etc/postfix
> dovecot_destination_recipient_limit = 1
> header_checks = pcre:/etc/postfix/headerchecks
> inet_interfaces = mail.studentergaarden.dk, localhost
> inet_protocols = ipv4
> mailbox_command = /usr/lib/dovecot/deliver
> mailbox_size_limit = 2048000000
> message_size_limit = 40480000
> mydestination = $mydomain, $myhostname, localhost.$mydomain, localhost
> myhostname = mail.studentergaarden.dk
> mynetworks = 130.226.169.128/26, 172.16.0.0/16, 127.0.0.0/8
> myorigin = /etc/mailname
> recipient_delimiter = +
> relayhost =
> smtpd_banner = $myhostname ESMTP $mail_name
> smtpd_client_restrictions = permit_mynetworks, reject_unknown_client
> smtpd_hard_error_limit = 10
> smtpd_helo_required = yes
> smtpd_helo_restrictions = reject_invalid_hostname
> smtpd_recipient_restrictions = permit_mynetworks, reject_rbl_client
> zen.spamhaus.org, reject_unauth_destination, reject_unauth_pipelining,
> smtpd_relay_restrictions = permit_mynetworks, reject_rbl_client
> zen.spamhaus.org, reject_unauth_destination, reject_unauth_pipelining,
> smtpd_sender_restrictions = reject_non_fqdn_sender,
> reject_unknown_sender_domain
> smtpd_soft_error_limit = 8
> transport_maps = hash:/var/lib/mailman/data/transport-mailman
> virtual_mailbox_domains = mail.studentergaarden.dk
> virtual_transport = dovecot


OK.


> $ cat  /etc/mailman/mm_cfg.py
> ... few irrelevant lines truncated ...
> DEFAULT_EMAIL_HOST = 'lists.studentergaarden.dk'
> DEFAULT_URL_HOST   = 'lists.studentergaarden.dk'
> MTA='Postfix'
> POSTFIX_STYLE_VIRTUAL_DOMAINS = ['lists.studentergaarden.dk']
> POSTFIX_MAP_CMD = '/var/lib/mailman/data/transport-mailman'


See below.


> ---
> Whether POSTFIX_MAP_CMD is commented out or not, does not seems to make a
> difference


POSTFIX_MAP_CMD should be


POSTFIX_MAP_CMD = '/var/lib/mailman/data/virtual_to_transport'

not what you have.


> $ ll /var/lib/mailman/data
> total 80K
> -rw-r----- 1 root     list   41 Oct 22 05:15 adm.pw
> -rw-rw---- 1 root     list 5,6K Nov 11 06:13 aliases
> -rw-rw---- 1 list     list  12K Nov 11 06:14 aliases.db
> -rw-r----- 1 root     list   41 Nov 11 04:40 creator.pw
> -rw-rw-r-- 1 root     list   10 Oct 22 03:11 last_mailman_version
> -rw-rw-r-- 1 root     list  14K Apr  6  2015 sitelist.cfg
> -rwxrwxr-x 1 root     list    0 Nov 11 03:53 transport-mailman
> -rw-r--r-- 1 root     list  12K Nov 11 04:10 transport-mailman.db
> -rw-rw---- 1 www-data list 5,3K Nov 11 06:13 virtual-mailman
> -rw-rw---- 1 list     list  12K Nov 11 06:14 virtual-mailman.db
> -rwxr-x--- 1 root     list  153 Nov 11 04:00 virtual_to_transport


OK.


> $ cat /var/lib/mailman/data/virtual-mailman
> # STANZA START: test
> # CREATED: Wed Nov 11 05:55:30 2015
> test at lists.studentergaarden.dk              test
> test-admin at lists.studentergaarden.dk        test-admin
> test-bounces at lists.studentergaarden.dk      test-bounces
> test-confirm at lists.studentergaarden.dk      test-confirm
> test-join at lists.studentergaarden.dk         test-join
> test-leave at lists.studentergaarden.dk        test-leave
> test-owner at lists.studentergaarden.dk        test-owner
> test-request at lists.studentergaarden.dk      test-request
> test-subscribe at lists.studentergaarden.dk    test-subscribe
> test-unsubscribe at lists.studentergaarden.dk  test-unsubscribe
> # STANZA END: test
> (the Mailman list is also included in the above file)


OK.


> $ cat /var/lib/mailman/data/virtual_to_transport
> sed -r -e 's/(^[^#]\S+\s+).+$/\1local:/' $1 >
> /var/lib/mailman/data/transport-mailman
> /usr/sbin/postmap /var/lib/mailman/data/transport-mailman
> 
> $ cat /var/lib/mailman/data/transport-mailman
> <<<EOF (empty)


This shouldn't be empty.  It should have entries like

test at lists.studentergaarden.dk local:

corresponding to all the entries in virtual-mailman.

POSTFIX_MAP_CMD = '/var/lib/mailman/data/transport-mailman'

is wrong. See above.


> There are no errors in relevant postfix/mailman logs.
> I do not use postfix-to-mailman.py.
> 
> What have I done wrong with the postfix configuration? And where does the
> listname at list.domain maills go?


A mail domain must have an MX record pointing to a domain with an A
record or if no MX record must have an A record. See, e.g.,
<http://wiki.list.org/x/4030649>.

lists.studentergaarden.dk has neither. It has only a CNAME to
loki.studentergaarden.dk. This should net be a CNAME. It should be
either an A pointing to 130.226.169.164 or an MX pointing to
loki.studentergaarden.dk or maybe mail.studentergaarden.dk.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From her at adm.ku.dk  Thu Nov 12 08:06:26 2015
From: her at adm.ku.dk (Henrik Rasmussen)
Date: Thu, 12 Nov 2015 13:06:26 +0000
Subject: [Mailman-Users] Part of moderated mails postponed to later
 digests
In-Reply-To: <5640B2F3.6010508@msapiro.net>
References: <6DCC3E5DA06FE346B4DE4876C4F2713D014D8582EF@P2KITMBX06WC03.unicph.domain>
 <5638E182.7080305@msapiro.net>
 <6DCC3E5DA06FE346B4DE4876C4F2713D014D858F18@P2KITMBX06WC03.unicph.domain>
 <563A2D22.3070005@msapiro.net>
 <6DCC3E5DA06FE346B4DE4876C4F2713D014D85D5B7@P2KITMBX06WC03.unicph.domain>
 <5640B2F3.6010508@msapiro.net>
Message-ID: <6DCC3E5DA06FE346B4DE4876C4F2713D014D861243@P2KITMBX06WC03.unicph.domain>

> This is the first time (I think) you mentioned 'posted to the list'
This may be something with lost in translation. What I meant by "posted to the list" is "forwarded to the members" after approval. 

> Are the messages missing from the digest not posted to the list at all until the next week? I.e. are they not in the list archives and not delivered to any non-digest members?
There are only a few non-digest members and the list admin is not aware whether they receive the mails or not.

> If in fact the missing messages don't appear in the archive until the next week, this is a different issue which has nothing to do with digests per se.
I will look closely at that, and get info from the list admin on when he approves mails.

>> if you don't want digests triggered on size
> The list admin doesn't need digests sent according to size so a digest_size_threshhold = 10000 should be fine. The moderator wants to have the mails included in a digest the next noon after approving the mails.

> Are the messages which are sent, sent in an immediate digest at the time of approval or just the next noon?
> If the former, that digest is triggered on size. If the latter, those are the only messages that actually got approved at that time.
Next noon.

> What does the vette log say about the messages that don't appear in a digest until the following week? Do they show as approved at the same time as the others that are in the first digest or not until later.
I am still trying to figure that out, as I still need to know exactly when the list admin has approved messages. The vette log shows a lot of  Refused posting labeled with the list name, but all the held message approved messages does not contain information on which list the message was approved, so I can't use the vette log to determine when messages on the list was approved.

But I wonder why the smtp log contains so many entries postings to exactly 1209 recipients on the list, even the list has 1222 normal members and 6493 digest members. Why only 6 times 1209. And are the 1209 digest members while the 6 recips could be regular members or something?

   /var/log/mailman/smtp Nov 09 09 06 44 2015 (16804) <mailman.577.1447056402.16802.LISTNAME at LISTSERVER.dk> smtp to LISTNAME for 1 recips, completed in 0.098 seconds
   /var/log/mailman/smtp Nov 09 09 06 45 2015 (16804) <mailman.577.1447056402.16802.LISTNAME at LISTSERVER.dk> smtp to LISTNAME for 6 recips, completed in 0.200 seconds
   /var/log/mailman/smtp Nov 09 09 06 45 2015 (16804) <mailman.577.1447056402.16802.LISTNAME at LISTSERVER.dk> smtp to LISTNAME for 6 recips, completed in 0.200 seconds
   /var/log/mailman/smtp Nov 09 09 06 45 2015 (16804) <mailman.577.1447056402.16802.LISTNAME at LISTSERVER.dk> smtp to LISTNAME for 6 recips, completed in 0.200 seconds
   /var/log/mailman/smtp Nov 09 09 06 45 2015 (16804) <mailman.577.1447056402.16802.LISTNAME at LISTSERVER.dk> smtp to LISTNAME for 6 recips, completed in 0.200 seconds
   /var/log/mailman/smtp Nov 09 09 06 45 2015 (16804) <mailman.577.1447056402.16802.LISTNAME at LISTSERVER.dk> smtp to LISTNAME for 6 recips, completed in 0.200 seconds
   /var/log/mailman/smtp Nov 09 09 06 45 2015 (16804) <mailman.577.1447056402.16802.LISTNAME at LISTSERVER.dk> smtp to LISTNAME for 6 recips, completed in 0.200 seconds
   /var/log/mailman/smtp Nov 09 13 28 33 2015 (16804) <6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local> smtp to LISTNAME for 1209 recips, completed in 110.189 seconds
   /var/log/mailman/smtp Nov 09 13 28 33 2015 (16804) <6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local> smtp to LISTNAME for 1209 recips, completed in 110.189 seconds
   /var/log/mailman/smtp Nov 09 13 28 33 2015 (16804) <6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local> smtp to LISTNAME for 1209 recips, completed in 110.189 seconds
   /var/log/mailman/smtp Nov 09 13 28 33 2015 (16804) <6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local> smtp to LISTNAME for 1209 recips, completed in 110.189 seconds
   /var/log/mailman/smtp Nov 09 13 28 33 2015 (16804) <6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local> smtp to LISTNAME for 1209 recips, completed in 110.189 seconds
   /var/log/mailman/smtp Nov 09 13 28 33 2015 (16804) <6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local> smtp to LISTNAME for 1209 recips, completed in 110.189 seconds
   /var/log/mailman/post Nov 09 13 28 33 2015 (16804) post to LISTNAME from someone at example.com, size=5675, message-id=<6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local>, success

Where can I see which messages was contained in which digests and when? My problem is that I don't seem to be able to tell whether the list admin is right or not. Not that I doubt his statement, but I really like to be able to tell when what happens.

Regards, Henrik Rasmussen

From alexander at neilson.net.nz  Wed Nov 11 11:31:39 2015
From: alexander at neilson.net.nz (Alexander Neilson)
Date: Thu, 12 Nov 2015 05:31:39 +1300
Subject: [Mailman-Users] listname@lists.domain does not work,
 listname@domain does
In-Reply-To: <CAP53jTxJS23na8TYfXH-yY5g2hgbGYt4xivuReH0=ePFpCw1FA@mail.gmail.com>
References: <CAP53jTxJS23na8TYfXH-yY5g2hgbGYt4xivuReH0=ePFpCw1FA@mail.gmail.com>
Message-ID: <AE3C6EA6-717A-4EC3-8690-6AC2A03B2799@neilson.net.nz>

Hi paw

It looks like maybe incorrect DNS. 

studentergaarden.dk MX record points at mail.

However lists. MX record points to Loki.

I think the mail is being tried against Loki. Try changing that MX record to point to mail. And see if this helps. 

Regards
Alexander

> On 11/11/2015, at 7:16 PM, Paw <pawses at gmail.com> wrote:
> 
> Hi Mailman community,
> 
> I am using postfix with dovecot on a Debian server. I followed <
> http://wiki.list.org/x/ZoCj> for setting up mailman, but something is wrong
> with my postfix configuration.
> 
> I can create new lists on http://lists.mydomain and when subscribing I get
> the subscription notification from mailman. But for sending to the list, I
> have to use listname at domain instead of listname at lists.domain. Mails sent to
> listname at lists.domain just seems to get lost(don't bounce or end up in
> /var/log/mail.log)
> I am sorry so disturb with such a question, but I am really stuck.
> 
> This is the full log when sending a mail to test at mydomain. The mail is
> being delivered to local user tester's inbox.
> Nov 11 06:41:59 loki postfix/smtpd[4543]: connect from
> mail-lf0-f49.google.com[209.85.215.49]
> Nov 11 06:41:59 loki postfix/smtpd[4543]: 7F4E724C09B: client=
> mail-lf0-f49.google.com[209.85.215.49]
> Nov 11 06:41:59 loki postfix/cleanup[4914]: 7F4E724C09B:
> message-id=<CAP53jTxugaP79R=Uqr_1zipb2z_EVQRno=
> LXJJ7KUJAaOBgxZA at mail.gmail.com>
> Nov 11 06:41:59 loki postfix/qmgr[31085]: 7F4E724C09B: from=<
> pawses at gmail.com>, size=1680, nrcpt=1 (queue active)
> Nov 11 06:41:59 loki postfix/smtpd[4543]: disconnect from
> mail-lf0-f49.google.com[209.85.215.49]
> Nov 11 06:41:59 loki postfix/local[4915]: 7F4E724C09B: to=<
> test at studentergaarden.dk>, relay=local, delay=0.63,
> delays=0.49/0.01/0/0.13, dsn=2.0.0, status=sent (delivered to command:
> /var/lib/mailman/mail/mailman post test)
> Nov 11 06:41:59 loki postfix/qmgr[31085]: 7F4E724C09B: removed
> Nov 11 06:42:01 loki postfix/smtpd[4543]: connect from localhost[127.0.0.1]
> Nov 11 06:42:01 loki postfix/smtpd[4543]: 44A6324C0A7:
> client=localhost[127.0.0.1]
> Nov 11 06:42:01 loki postfix/cleanup[4914]: 44A6324C0A7:
> message-id=<CAP53jTxugaP79R=Uqr_1zipb2z_EVQRno=
> LXJJ7KUJAaOBgxZA at mail.gmail.com>
> Nov 11 06:42:01 loki postfix/qmgr[31085]: 44A6324C0A7: from=<
> test-bounces at lists.studentergaarden.dk>, size=3127, nrcpt=1 (queue active)
> Nov 11 06:42:01 loki postfix/smtpd[4543]: 5D5F124C0E1:
> client=localhost[127.0.0.1]
> Nov 11 06:42:01 loki postfix/cleanup[4914]: 5D5F124C0E1:
> message-id=<CAP53jTxugaP79R=Uqr_1zipb2z_EVQRno=
> LXJJ7KUJAaOBgxZA at mail.gmail.com>
> Nov 11 06:42:01 loki postfix/qmgr[31085]: 5D5F124C0E1: from=<
> test-bounces at lists.studentergaarden.dk>, size=3137, nrcpt=1 (queue active)
> Nov 11 06:42:01 loki postfix/smtpd[4543]: disconnect from
> localhost[127.0.0.1]
> Nov 11 06:42:01 loki dovecot: lda(tester):
> msgid=<CAP53jTxugaP79R=Uqr_1zipb2z_EVQRno=LXJJ7KUJAaOBgxZA at mail.gmail.com>:
> saved mail to INBOX
> Nov 11 06:42:01 loki postfix/local[4915]: 5D5F124C0E1: to=<
> tester at studentergaarden.dk>, relay=local, delay=0.19, delays=0.05/0/0/0.13,
> dsn=2.0.0, status=sent (delivered to command: /usr/lib/dovecot/deliver)
> Nov 11 06:42:01 loki postfix/qmgr[31085]: 5D5F124C0E1: removed
> Nov 11 06:42:01 loki postfix/smtp[4944]: 44A6324C0A7: to=<pawses at gmail.com>,
> relay=gmail-smtp-in.l.google.com[173.194.222.27]:25, delay=0.49,
> delays=0.1/0.01/0.26/0.12, dsn=2.0.0, status=sent (250 2.0.0 OK 1447220691
> 38si4916356lfq.17 - gsmtp)
> Nov 11 06:42:01 loki postfix/qmgr[31085]: 44A6324C0A7: removed
> 
> 
> My config is as follow:
> -----------------------------------------------
> Dumps of postconf, mm_cfg.py virtual-mailman,
> virtual_to_transport and transport-mailman
> -----------------------------------------------
> $ postconf -n
> alias_database = hash:/etc/aliases
> alias_maps = hash:/etc/aliases hash:/var/lib/mailman/data/aliases
> append_dot_mydomain = no
> biff = no
> body_checks = pcre:/etc/postfix/bodychecks
> config_directory = /etc/postfix
> dovecot_destination_recipient_limit = 1
> header_checks = pcre:/etc/postfix/headerchecks
> inet_interfaces = mail.studentergaarden.dk, localhost
> inet_protocols = ipv4
> mailbox_command = /usr/lib/dovecot/deliver
> mailbox_size_limit = 2048000000
> message_size_limit = 40480000
> mydestination = $mydomain, $myhostname, localhost.$mydomain, localhost
> myhostname = mail.studentergaarden.dk
> mynetworks = 130.226.169.128/26, 172.16.0.0/16, 127.0.0.0/8
> myorigin = /etc/mailname
> recipient_delimiter = +
> relayhost =
> smtpd_banner = $myhostname ESMTP $mail_name
> smtpd_client_restrictions = permit_mynetworks, reject_unknown_client
> smtpd_hard_error_limit = 10
> smtpd_helo_required = yes
> smtpd_helo_restrictions = reject_invalid_hostname
> smtpd_recipient_restrictions = permit_mynetworks, reject_rbl_client
> zen.spamhaus.org, reject_unauth_destination, reject_unauth_pipelining,
> smtpd_relay_restrictions = permit_mynetworks, reject_rbl_client
> zen.spamhaus.org, reject_unauth_destination, reject_unauth_pipelining,
> smtpd_sender_restrictions = reject_non_fqdn_sender,
> reject_unknown_sender_domain
> smtpd_soft_error_limit = 8
> transport_maps = hash:/var/lib/mailman/data/transport-mailman
> virtual_mailbox_domains = mail.studentergaarden.dk
> virtual_transport = dovecot
> 
> $ cat  /etc/mailman/mm_cfg.py
> ... few irrelevant lines truncated ...
> DEFAULT_EMAIL_HOST = 'lists.studentergaarden.dk'
> DEFAULT_URL_HOST   = 'lists.studentergaarden.dk'
> MTA='Postfix'
> POSTFIX_STYLE_VIRTUAL_DOMAINS = ['lists.studentergaarden.dk']
> POSTFIX_MAP_CMD = '/var/lib/mailman/data/transport-mailman'
> 
> ---
> Whether POSTFIX_MAP_CMD is commented out or not, does not seems to make a
> difference
> 
> $ ll /var/lib/mailman/data
> total 80K
> -rw-r----- 1 root     list   41 Oct 22 05:15 adm.pw
> -rw-rw---- 1 root     list 5,6K Nov 11 06:13 aliases
> -rw-rw---- 1 list     list  12K Nov 11 06:14 aliases.db
> -rw-r----- 1 root     list   41 Nov 11 04:40 creator.pw
> -rw-rw-r-- 1 root     list   10 Oct 22 03:11 last_mailman_version
> -rw-rw-r-- 1 root     list  14K Apr  6  2015 sitelist.cfg
> -rwxrwxr-x 1 root     list    0 Nov 11 03:53 transport-mailman
> -rw-r--r-- 1 root     list  12K Nov 11 04:10 transport-mailman.db
> -rw-rw---- 1 www-data list 5,3K Nov 11 06:13 virtual-mailman
> -rw-rw---- 1 list     list  12K Nov 11 06:14 virtual-mailman.db
> -rwxr-x--- 1 root     list  153 Nov 11 04:00 virtual_to_transport
> 
> $ cat /var/lib/mailman/data/virtual-mailman
> # STANZA START: test
> # CREATED: Wed Nov 11 05:55:30 2015
> test at lists.studentergaarden.dk              test
> test-admin at lists.studentergaarden.dk        test-admin
> test-bounces at lists.studentergaarden.dk      test-bounces
> test-confirm at lists.studentergaarden.dk      test-confirm
> test-join at lists.studentergaarden.dk         test-join
> test-leave at lists.studentergaarden.dk        test-leave
> test-owner at lists.studentergaarden.dk        test-owner
> test-request at lists.studentergaarden.dk      test-request
> test-subscribe at lists.studentergaarden.dk    test-subscribe
> test-unsubscribe at lists.studentergaarden.dk  test-unsubscribe
> # STANZA END: test
> (the Mailman list is also included in the above file)
> 
> $ cat /var/lib/mailman/data/virtual_to_transport
> sed -r -e 's/(^[^#]\S+\s+).+$/\1local:/' $1 >
> /var/lib/mailman/data/transport-mailman
> /usr/sbin/postmap /var/lib/mailman/data/transport-mailman
> 
> $ cat /var/lib/mailman/data/transport-mailman
> <<<EOF (empty)
> 
> There are no errors in relevant postfix/mailman logs.
> I do not use postfix-to-mailman.py.
> 
> What have I done wrong with the postfix configuration? And where does the
> listname at list.domain maills go?
> 
> $ uname -a
> Linux loki 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u5 (2015-10-09)
> x86_64 GNU/Linux
> $ /usr/sbin/check_perms
> No problems found
> $ /usr/lib/mailman/bin/version
> Using Mailman version: 2.1.18
> 
> 
>  Thanks in advance!
> /Paw
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/alexander%40neilson.net.nz

From mark at msapiro.net  Thu Nov 12 12:20:51 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 12 Nov 2015 09:20:51 -0800
Subject: [Mailman-Users] Part of moderated mails postponed to later
 digests
In-Reply-To: <6DCC3E5DA06FE346B4DE4876C4F2713D014D861243@P2KITMBX06WC03.unicph.domain>
References: <6DCC3E5DA06FE346B4DE4876C4F2713D014D8582EF@P2KITMBX06WC03.unicph.domain>
 <5638E182.7080305@msapiro.net>
 <6DCC3E5DA06FE346B4DE4876C4F2713D014D858F18@P2KITMBX06WC03.unicph.domain>
 <563A2D22.3070005@msapiro.net>
 <6DCC3E5DA06FE346B4DE4876C4F2713D014D85D5B7@P2KITMBX06WC03.unicph.domain>
 <5640B2F3.6010508@msapiro.net>
 <6DCC3E5DA06FE346B4DE4876C4F2713D014D861243@P2KITMBX06WC03.unicph.domain>
Message-ID: <5644CA73.3010907@msapiro.net>

On 11/12/15 5:06 AM, Henrik Rasmussen wrote:
> 
>> What does the vette log say about the messages that don't appear in a digest until the following week? Do they show as approved at the same time as the others that are in the first digest or not until later.
> I am still trying to figure that out, as I still need to know exactly when the list admin has approved messages. The vette log shows a lot of  Refused posting labeled with the list name, but all the held message approved messages does not contain information on which list the message was approved, so I can't use the vette log to determine when messages on the list was approved.


Yes, that is a bug, <https://bugs.launchpad.net/mailman/+bug/1295875>,
fixed in MM 2.1.18.


> But I wonder why the smtp log contains so many entries postings to exactly 1209 recipients on the list, even the list has 1222 normal members and 6493 digest members. Why only 6 times 1209. And are the 1209 digest members while the 6 recips could be regular members or something?
> 
>    /var/log/mailman/smtp Nov 09 09 06 44 2015 (16804) <mailman.577.1447056402.16802.LISTNAME at LISTSERVER.dk> smtp to LISTNAME for 1 recips, completed in 0.098 seconds
>    /var/log/mailman/smtp Nov 09 09 06 45 2015 (16804) <mailman.577.1447056402.16802.LISTNAME at LISTSERVER.dk> smtp to LISTNAME for 6 recips, completed in 0.200 seconds
>    /var/log/mailman/smtp Nov 09 09 06 45 2015 (16804) <mailman.577.1447056402.16802.LISTNAME at LISTSERVER.dk> smtp to LISTNAME for 6 recips, completed in 0.200 seconds
>    /var/log/mailman/smtp Nov 09 09 06 45 2015 (16804) <mailman.577.1447056402.16802.LISTNAME at LISTSERVER.dk> smtp to LISTNAME for 6 recips, completed in 0.200 seconds
>    /var/log/mailman/smtp Nov 09 09 06 45 2015 (16804) <mailman.577.1447056402.16802.LISTNAME at LISTSERVER.dk> smtp to LISTNAME for 6 recips, completed in 0.200 seconds
>    /var/log/mailman/smtp Nov 09 09 06 45 2015 (16804) <mailman.577.1447056402.16802.LISTNAME at LISTSERVER.dk> smtp to LISTNAME for 6 recips, completed in 0.200 seconds
>    /var/log/mailman/smtp Nov 09 09 06 45 2015 (16804) <mailman.577.1447056402.16802.LISTNAME at LISTSERVER.dk> smtp to LISTNAME for 6 recips, completed in 0.200 seconds


The above are a Mailman notice. It is unclear to me exactly what is
happening here? The first log message is the message to LISTNAME-owner
and the next is the resending of that message to the 6 owner/moderator
addresses. The next 5 appear to be duplicates because the 0.200 seconds
are identical and even if there were 6 separate sends, that totals 1.2
seconds, yet the timestamp never advanced even 1 second.


>    /var/log/mailman/smtp Nov 09 13 28 33 2015 (16804) <6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local> smtp to LISTNAME for 1209 recips, completed in 110.189 seconds
>    /var/log/mailman/smtp Nov 09 13 28 33 2015 (16804) <6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local> smtp to LISTNAME for 1209 recips, completed in 110.189 seconds
>    /var/log/mailman/smtp Nov 09 13 28 33 2015 (16804) <6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local> smtp to LISTNAME for 1209 recips, completed in 110.189 seconds
>    /var/log/mailman/smtp Nov 09 13 28 33 2015 (16804) <6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local> smtp to LISTNAME for 1209 recips, completed in 110.189 seconds
>    /var/log/mailman/smtp Nov 09 13 28 33 2015 (16804) <6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local> smtp to LISTNAME for 1209 recips, completed in 110.189 seconds
>    /var/log/mailman/smtp Nov 09 13 28 33 2015 (16804) <6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local> smtp to LISTNAME for 1209 recips, completed in 110.189 seconds


Here again this appears to be 6 duplicate copies of one message. It
could be 6 approved posts each delivered to 1209 eligible non-digest
members upon approval (the other 13 non-digest members could be delivery
disabled or direct addressees with 'no dups' or a combination), but it
is not 6 posts because the log messages are all for the same Message-ID
and all from the same (pid = 16804) OutgoingRunner and delivery takes
over 110 seconds, but the timestamp doesn't advance. Also, 6 separate
sends to 1209 recips would be very unlikely to be completed in the exact
same time of 110.189 seconds.


>    /var/log/mailman/post Nov 09 13 28 33 2015 (16804) post to LISTNAME from someone at example.com, size=5675, message-id=<6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local>, success


And the post log shows only one post.


> Where can I see which messages was contained in which digests and when? My problem is that I don't seem to be able to tell whether the list admin is right or not. Not that I doubt his statement, but I really like to be able to tell when what happens.


The send of the digests would be logged as above in the smtp log at 'Nov
09 12 xx xx'. To see which messages it contained, you'd have to see the
digest itself. That info isn't logged.

You could subscribe yourself to the MIME digest. Then you'd see the
messages in it and their Message-IDs. The Message-IDs should be in the
vette log. Also, for the smtp log entry above like

Nov 09 13 28 33 2015 (16804)
<6f7a7eb8e5e442668ea1a864e84c2530 at pobox.engr.local> smtp to LISTNAME for
1209 recips, completed in 110.189 seconds

there should be a vette log entry with the same Message-ID and a
timestamp about 110-115 seconds earlier.

The six apparent duplications of certain smtp log entries is very
puzzling. I don't know what would cause that.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From VxMr at pge.com  Thu Nov 12 13:14:09 2015
From: VxMr at pge.com (Montanez, Victor)
Date: Thu, 12 Nov 2015 18:14:09 +0000
Subject: [Mailman-Users] Downgrade options/alternatives
Message-ID: <C3B68975750F324FB948A2E44C95021F8938FE4D@EXCHMBSF324.Utility.pge.com>


Hello,

I have done a successful upgrade from Mailman version 2.1.16 to 2.1.20 successfully. I am now testing the downgrade (In case some issue comes up and I have to rollback) and it seemed to have gone okay for the most part.

At the of the end of the downgrade I get the following message(which I have googled around but no real alternative given).


Error:
Compiling  i18n.py ...

Compiling versions.py ...

Downgrade detected, from version 0x20114f0 to version 0x20110f0

This is probably not safe.

Exiting.

make: *** [update] Error 1



I understand is a downgrade, that is what I want to do. But it looks like "downgrading" isn't really supported in Mailman. What is a recommended alternative for having a way to go back to previous Mailman version?

Thank you

From mark at msapiro.net  Thu Nov 12 15:00:35 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 12 Nov 2015 12:00:35 -0800
Subject: [Mailman-Users] Downgrade options/alternatives
In-Reply-To: <C3B68975750F324FB948A2E44C95021F8938FE4D@EXCHMBSF324.Utility.pge.com>
References: <C3B68975750F324FB948A2E44C95021F8938FE4D@EXCHMBSF324.Utility.pge.com>
Message-ID: <5644EFE3.2020201@msapiro.net>

On 11/12/15 10:14 AM, Montanez, Victor wrote:
> 
> Hello,
> 
> I have done a successful upgrade from Mailman version 2.1.16 to 2.1.20 successfully. I am now testing the downgrade (In case some issue comes up and I have to rollback) and it seemed to have gone okay for the most part.
> 
> At the of the end of the downgrade I get the following message(which I have googled around but no real alternative given).
> 
> 
> Error:
> Compiling  i18n.py ...
> 
> Compiling versions.py ...
> 
> Downgrade detected, from version 0x20114f0 to version 0x20110f0
> 
> This is probably not safe.
> 
> Exiting.
> 
> make: *** [update] Error 1
> 
> 
> 
> I understand is a downgrade, that is what I want to do. But it looks like "downgrading" isn't really supported in Mailman. What is a recommended alternative for having a way to go back to previous Mailman version?


Actually, you did do the downgrade. Only the final running of Mailman's
bin/upgrade did not succeed. You can run it with the -f or --force
option to make it succeed, but it won't do anything anyway except maybe
update Mailman's data/last_mailman_version. See 'bin/upgrade --help'.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From VxMr at pge.com  Thu Nov 12 16:28:49 2015
From: VxMr at pge.com (Montanez, Victor)
Date: Thu, 12 Nov 2015 21:28:49 +0000
Subject: [Mailman-Users] Downgrade options/alternatives
In-Reply-To: <5644EFE3.2020201@msapiro.net>
References: <C3B68975750F324FB948A2E44C95021F8938FE4D@EXCHMBSF324.Utility.pge.com>
 <5644EFE3.2020201@msapiro.net>
Message-ID: <C3B68975750F324FB948A2E44C95021F89390372@EXCHMBSF324.Utility.pge.com>

Hello Mark,

Thank you for the information/confirmation.  Knowing that the only thing left to do is to update the file on the last mailman version makes me feel more confident that all functional pieces were successfully rolled back. And that will make me feel better about upgrading soon. 

I have checked the update script but I get some errors when trying to run the bin/update --force . I think this has to do with the import statements specified on the script. But I can say I do see the version I rolled back to through the web interface (which does appear to have rolled back successfully).

Thank you

$ bin/update --help
Traceback (most recent call last):
  File "bin/update", line 46, in <module>
    import paths
ImportError: No module named paths


>Actually, you did do the downgrade. Only the final running of Mailman's
>bin/upgrade did not succeed. You can run it with the -f or --force
>option to make it succeed, but it won't do anything anyway except maybe
>update Mailman's data/last_mailman_version. See 'bin/upgrade --help'.




-----Original Message-----
From: Mailman-Users [mailto:mailman-users-bounces+vxmr=pge.com at python.org] On Behalf Of Mark Sapiro
Sent: Thursday, November 12, 2015 12:01 PM
To: mailman-users at python.org
Subject: Re: [Mailman-Users] Downgrade options/alternatives

On 11/12/15 10:14 AM, Montanez, Victor wrote:
>
> Hello,
>
> I have done a successful upgrade from Mailman version 2.1.16 to 2.1.20 successfully. I am now testing the downgrade (In case some issue comes up and I have to rollback) and it seemed to have gone okay for the most part.
>
> At the of the end of the downgrade I get the following message(which I have googled around but no real alternative given).
>
>
> Downgrade detected, from version 0x20114f0 to version 0x20110f0
>
> This is probably not safe.
>
> Exiting.
>
> make: *** [update] Error 1
>
>
>
> I understand is a downgrade, that is what I want to do. But it looks like "downgrading" isn't really supported in Mailman. What is a recommended alternative for having a way to go back to previous Mailman version?


Actually, you did do the downgrade. Only the final running of Mailman's
bin/upgrade did not succeed. You can run it with the -f or --force
option to make it succeed, but it won't do anything anyway except maybe
update Mailman's data/last_mailman_version. See 'bin/upgrade --help'.

--
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan
------------------------------------------------------
Mailman-Users mailing list Mailman-Users at python.org


From pshute at nuw.org.au  Thu Nov 12 19:00:27 2015
From: pshute at nuw.org.au (Peter Shute)
Date: Fri, 13 Nov 2015 11:00:27 +1100
Subject: [Mailman-Users] Readonly admin access
Message-ID: <C2F1879073AE7347BAB44065F143E17002C7DDD594@nuwvicms2>

On our list, several of the moderators have admin access so that we can add and remove members. Has it ever been considered to add a third type of user who can perform those tasks without being able to modify the list configuration? I assume there isn't already a way to achieve this?

A related idea is to have mailman keep a log of configuration changes so that's it's easy to see when and what changes have been made.

Peter Shute

From mark at msapiro.net  Fri Nov 13 01:53:44 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 13 Nov 2015 07:53:44 +0100
Subject: [Mailman-Users] Readonly admin access
In-Reply-To: <C2F1879073AE7347BAB44065F143E17002C7DDD594@nuwvicms2>
References: <C2F1879073AE7347BAB44065F143E17002C7DDD594@nuwvicms2>
Message-ID: <0A8197CB-CE1C-4B10-9BF3-A901BC0C973B@msapiro.net>



On November 13, 2015 1:00:27 AM GMT+01:00, Peter Shute <pshute at nuw.org.au> wrote:
>On our list, several of the moderators have admin access so that we can
>add and remove members. Has it ever been considered to add a third type
>of user who can perform those tasks without being able to modify the
>list configuration? I assume there isn't already a way to achieve this?


Correct. You could develop your own web page to invoke Mailman's add_members and remove_members scripts and authenticate however you like, but the standard admin ui is all or nothing.


>A related idea is to have mailman keep a log of configuration changes
>so that's it's easy to see when and what changes have been made.


That won't happen in MM 2.1.x.


-- 
Mark Sapiro <mark at msapiro.net>
Sent from my Not_an_iThing with standards compliant, open source software.

From mark at msapiro.net  Fri Nov 13 12:02:02 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 13 Nov 2015 09:02:02 -0800
Subject: [Mailman-Users] Downgrade options/alternatives
In-Reply-To: <C3B68975750F324FB948A2E44C95021F89390372@EXCHMBSF324.Utility.pge.com>
References: <C3B68975750F324FB948A2E44C95021F8938FE4D@EXCHMBSF324.Utility.pge.com>
 <5644EFE3.2020201@msapiro.net>
 <C3B68975750F324FB948A2E44C95021F89390372@EXCHMBSF324.Utility.pge.com>
Message-ID: <5646178A.5030605@msapiro.net>

On 11/12/15 1:28 PM, Montanez, Victor wrote:
> 
> I have checked the update script but I get some errors when trying to run the bin/update --force . I think this has to do with the import statements specified on the script. But I can say I do see the version I rolled back to through the web interface (which does appear to have rolled back successfully).
> 
> Thank you
> 
> $ bin/update --help
> Traceback (most recent call last):
>   File "bin/update", line 46, in <module>
>     import paths
> ImportError: No module named paths


This is very strange. There must be a paths.py module in Mailman's bin/
directory or pretty much non of Mailman's bin/ commands will run.
Possibly the user you ran it as does not have permission to read
bin/paths.py.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From VxMr at pge.com  Fri Nov 13 18:41:55 2015
From: VxMr at pge.com (Montanez, Victor)
Date: Fri, 13 Nov 2015 23:41:55 +0000
Subject: [Mailman-Users] Downgrade options/alternatives
In-Reply-To: <5646178A.5030605@msapiro.net>
References: <C3B68975750F324FB948A2E44C95021F8938FE4D@EXCHMBSF324.Utility.pge.com>
 <5644EFE3.2020201@msapiro.net>
 <C3B68975750F324FB948A2E44C95021F89390372@EXCHMBSF324.Utility.pge.com>
 <5646178A.5030605@msapiro.net>
Message-ID: <C3B68975750F324FB948A2E44C95021F89391E57@EXCHMBSF324.Utility.pge.com>

Ah got it. Thank you. I was looking on the wrong location, using the wrong "update" script. I ran it and it updated properly.

Thank you



-----Original Message-----
From: Mark Sapiro [mailto:mark at msapiro.net] 
Sent: Friday, November 13, 2015 9:02 AM
To: Montanez, Victor; mailman-users at python.org
Subject: Re: [Mailman-Users] Downgrade options/alternatives


On 11/12/15 1:28 PM, Montanez, Victor wrote:
>
> I have checked the update script but I get some errors when trying to run the bin/update --force . I think this has to do with the import statements specified on the script. But I can say I do see the version I rolled back to through the web interface (which does appear to have rolled back successfully).
>
> Thank you
>
> $ bin/update --help
> Traceback (most recent call last):
>   File "bin/update", line 46, in <module>
>     import paths
> ImportError: No module named paths


This is very strange. There must be a https://urldefense.proofpoint.com/v2/url?u=http-3A__paths.py&d=BQIC-g&c=hLS_V_MyRCwXDjNCFvC1XhVzdhW2dOtrP9xQj43rEYI&r=CNBJ5WQxVRyGeDTLL7hLtA&m=wvZdR3Nfu5vXQHaIJcr6TuCU3HyNyUUPmHSryKuhFXQ&s=AscK60oiU0TF9Sl8VpEEnuzis07VcF3zmfm6YFLM26U&e=  module in Mailman's bin/
directory or pretty much non of Mailman's bin/ commands will run.
Possibly the user you ran it as does not have permission to read
bin/paths.py.

--
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From kapuzino at web.de  Sun Nov 15 13:14:00 2015
From: kapuzino at web.de (kapuzino at web.de)
Date: Sun, 15 Nov 2015 19:14:00 +0100
Subject: [Mailman-Users] Mailman only delivers inside server
Message-ID: <3nzM9J5cmdzFpZK@mail.python.org>

(Mark Sapiro)
>If I had to guess, my guess would be that 'localhost' or it's equivalent
>IP (possibly IPv6 - see /etc/hosts) is not in Postfix mynetworks in
>main.cf 

Following the advice of "Gao" and netfound tips it was defined already:
mynetworks = 127.0.0.0/8, 89.163.212.5/32
but no change of behaviour.

Below the complete main.cf and master.cf.
Maybe, someone can see abscent or unmatching entries.

Kapuzino

______________________________________________________________
main.cf

# See /usr/share/postfix/main.cf.dist for a commented, more complete version


# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# TLS parameters
smtpd_tls_cert_file = /etc/postfix/postfix_default.pem
smtpd_tls_key_file = $smtpd_tls_cert_file
smtpd_use_tls = yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = f632.fuchsia.scp-manager.com
alias_maps = hash:/etc/aliases, hash:/var/spool/postfix/plesk/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = localhost.fuchsia.scp-manager.com, localhost, localhost.localdomain
relayhost = 
mynetworks = 127.0.0.0/8, 89.163.212.5/32
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
virtual_mailbox_domains = $virtual_mailbox_maps, hash:/var/spool/postfix/plesk/virtual_domains
virtual_alias_maps = $virtual_maps, hash:/var/spool/postfix/plesk/virtual
virtual_mailbox_maps = , hash:/var/spool/postfix/plesk/vmailbox
transport_maps = , hash:/var/spool/postfix/plesk/transport
smtpd_tls_security_level = may
smtp_tls_security_level = may
smtp_use_tls = no
smtpd_timeout = 3600s
smtpd_proxy_timeout = 3600s
disable_vrfy_command = yes
smtpd_sender_restrictions = check_sender_access hash:/var/spool/postfix/plesk/blacklists, permit_sasl_authenticated
smtpd_client_restrictions = permit_mynetworks, permit_sasl_authenticated
smtp_send_xforward_command = yes
smtpd_authorized_xforward_hosts = 127.0.0.0/8 [::1]/128
smtpd_sasl_auth_enable = yes
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination
virtual_mailbox_base = /var/qmail/mailnames
virtual_uid_maps = static:30
virtual_gid_maps = static:31
smtpd_milters = , inet:127.0.0.1:12768 inet:127.0.0.1:12345
sender_dependent_default_transport_maps = hash:/var/spool/postfix/plesk/sdd_transport_maps
virtual_transport = plesk_virtual
plesk_virtual_destination_recipient_limit = 1
mailman_destination_recipient_limit = 1
virtual_mailbox_limit = 0
smtpd_tls_mandatory_protocols = TLSv1 TLSv1.1 TLSv1.2
smtpd_tls_protocols = TLSv1 TLSv1.1 TLSv1.2
smtpd_tls_ciphers = medium
smtpd_tls_mandatory_ciphers = medium
tls_medium_cipherlist = HIGH:!aNULL:!MD5
message_size_limit = 102400000
milter_connect_macros = j {daemon_name} {client_connections} {client_addr} {client_ptr} v
milter_default_action = accept

______________________________________________________________
master.cf

#
# Postfix master process configuration file.  For details on the format
# of the file, see the master(5) manual page (command: "man 5 master" or
# on-line: http://www.postfix.org/master.5.html).
#
# Do not forget to execute "postfix reload" after editing this file.
#
# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
smtp      inet  n       -       -       -       -       smtpd
#smtp      inet  n       -       -       -       1       postscreen
#smtpd     pass  -       -       -       -       -       smtpd
#dnsblog   unix  -       -       -       -       0       dnsblog
#tlsproxy  unix  -       -       -       -       0       tlsproxy
#submission inet n       -       -       -       -       smtpd
#  -o syslog_name=postfix/submission
#  -o smtpd_tls_security_level=encrypt
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
#  -o smtpd_recipient_restrictions=
#  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING
#smtps     inet  n       -       -       -       -       smtpd
#  -o syslog_name=postfix/smtps
#  -o smtpd_tls_wrappermode=yes
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
#  -o smtpd_recipient_restrictions=
#  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING
#628       inet  n       -       -       -       -       qmqpd
cleanup   unix  n       -       -       -       0       cleanup
#qmgr     unix  n       -       n       300     1       oqmgr
tlsmgr    unix  -       -       -       1000?   1       tlsmgr
rewrite   unix  -       -       -       -       -       trivial-rewrite
bounce    unix  -       -       -       -       0       bounce
defer     unix  -       -       -       -       0       bounce
trace     unix  -       -       -       -       0       bounce
verify    unix  -       -       -       -       1       verify
flush     unix  n       -       -       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       -       -       -       smtp
relay     unix  -       -       -       -       -       smtp
#       -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq     unix  n       -       -       -       -       showq
error     unix  -       -       -       -       -       error
retry     unix  -       -       -       -       -       error
discard   unix  -       -       -       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       -       -       -       lmtp
anvil     unix  -       -       -       -       1       anvil
scache    unix  -       -       -       -       1       scache
#
# ====================================================================
# Interfaces to non-Postfix software. Be sure to examine the manual
# pages of the non-Postfix software to find out what options it wants.
#
# Many of the following services use the Postfix pipe(8) delivery
# agent.  See the pipe(8) man page for information about ${recipient}
# and other message envelope options.
# ====================================================================
#
# maildrop. See the Postfix MAILDROP_README file for details.
# Also specify in main.cf: maildrop_destination_recipient_limit=1
#
maildrop  unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
#
# ====================================================================
#
# Recent Cyrus versions can use the existing "lmtp" master.cf entry.
#
# Specify in cyrus.conf:
#   lmtp    cmd="lmtpd -a" listen="localhost:lmtp" proto=tcp4
#
# Specify in main.cf one or more of the following:
#  mailbox_transport = lmtp:inet:localhost
#  virtual_transport = lmtp:inet:localhost
#
# ====================================================================
#
# Cyrus 2.1.5 (Amos Gouaux)
# Also specify in main.cf: cyrus_destination_recipient_limit=1
#
#cyrus     unix  -       n       n       -       -       pipe
#  user=cyrus argv=/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user}
#
# ====================================================================
# Old example of delivery via Cyrus.
#
#old-cyrus unix  -       n       n       -       -       pipe
#  flags=R user=cyrus argv=/cyrus/bin/deliver -e -m ${extension} ${user}
#
# ====================================================================
#
# See the Postfix UUCP_README file for configuration details.
#
uucp      unix  -       n       n       -       -       pipe
  flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
#
# Other external delivery methods.
#
ifmail    unix  -       n       n       -       -       pipe
  flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp     unix  -       n       n       -       -       pipe
  flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
scalemail-backend unix	-	n	n	-	2	pipe
  flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
mailman unix - n n - - pipe flags=R user=list:list argv=/usr/lib/plesk-9.0/postfix-mailman ${nexthop} ${user} ${recipient}


plesk_virtual unix - n n - - pipe flags=DORhu user=popuser:popuser argv=/usr/lib/plesk-9.0/postfix-local -f ${sender} -d ${recipient} -p /var/qmail/mailnames
pickup fifo n - - 60 1 pickup
plesk_saslauthd unix y y y - 1 plesk_saslauthd status=5 listen=6 dbpath=/plesk/passwd.db
qmgr fifo n - n 1 1 qmgr
smtps inet n - - - - smtpd -o smtpd_tls_wrappermode=yes

plesk-89.163.212.5- unix - n n - - smtp -o smtp_bind_address=89.163.212.5 -o smtp_bind_address6= -o smtp_address_preference=ipv4

plesk-89.163.142.29- unix - n n - - smtp -o smtp_bind_address=89.163.142.29 -o smtp_bind_address6= -o smtp_address_preference=ipv4

plesk-89.163.142.28- unix - n n - - smtp -o smtp_bind_address=89.163.142.28 -o smtp_bind_address6= -o smtp_address_preference=ipv4








From barry at list.org  Sun Nov 15 14:42:19 2015
From: barry at list.org (Barry Warsaw)
Date: Sun, 15 Nov 2015 14:42:19 -0500
Subject: [Mailman-Users] ANNOUNCING: GNU Mailman 3.0.1
Message-ID: <20151115144219.5ec1c4d6@anarchist.wooz.org>

I'm happy to announce the first maintenance release of GNU Mailman 3.  This
version 3.0.1 release includes updates to the core, Postorius, HyperKitty, and
mailman.client.  The release matrix of versions is available here:

    http://wiki.list.org/Mailman3

You can find more information about the changes in each component by following
the version links in that overview page.

Enjoy,
-Barry (on behalf of the entire Mailman development team)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20151115/9e592500/attachment.sig>

From mark at msapiro.net  Sun Nov 15 16:10:31 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 15 Nov 2015 13:10:31 -0800
Subject: [Mailman-Users] Mailman only delivers inside server
In-Reply-To: <3nzM9J5cmdzFpZK@mail.python.org>
References: <3nzM9J5cmdzFpZK@mail.python.org>
Message-ID: <5648F4C7.1050508@msapiro.net>

On 11/15/15 10:14 AM, kapuzino at web.de wrote:
> (Mark Sapiro)
>> If I had to guess, my guess would be that 'localhost' or it's equivalent
>> IP (possibly IPv6 - see /etc/hosts) is not in Postfix mynetworks in
>> main.cf 
> 
> Following the advice of "Gao" and netfound tips it was defined already:
> mynetworks = 127.0.0.0/8, 89.163.212.5/32
> but no change of behaviour.


What is the definition of localhost in /etc/hosts? If it is an IPv6
address like

::1  localhost

you need to either add ::1 to mynetworks in main.cf or change localhost to

127.0.0.1  localhost

in /etc/hosts.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From listgnome at protonmail.ch  Tue Nov 17 22:46:43 2015
From: listgnome at protonmail.ch (ListGnome ProtonMail)
Date: Tue, 17 Nov 2015 22:46:43 -0500
Subject: [Mailman-Users] Reverse chronological date order for archives
Message-ID: <8rdWEkySZzy6A1MW3GSxtc1hF6pLu3wj0gQNooASyDfnxl2B2Fcu03e9vPz6REyuQsEG6b7l6l__KdbkVNH95Q==@protonmail.ch>

Hi,

Has anyone been able to order their archive messages in reverse chronological date order - so the most recent post is at the top of the page. (Version 2.1.20)

I found a related thread in the Mailman-Users archive (dated 2002) but it didn't explain how it was done.

https://mail.python.org/pipermail/mailman-users/2002-February/017906.html

Any clues greatly appreciated.

Cheers,
Mark

From mark at msapiro.net  Wed Nov 18 15:16:34 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 18 Nov 2015 12:16:34 -0800
Subject: [Mailman-Users] Reverse chronological date order for archives
In-Reply-To: <8rdWEkySZzy6A1MW3GSxtc1hF6pLu3wj0gQNooASyDfnxl2B2Fcu03e9vPz6REyuQsEG6b7l6l__KdbkVNH95Q==@protonmail.ch>
References: <8rdWEkySZzy6A1MW3GSxtc1hF6pLu3wj0gQNooASyDfnxl2B2Fcu03e9vPz6REyuQsEG6b7l6l__KdbkVNH95Q==@protonmail.ch>
Message-ID: <564CDCA2.3080703@msapiro.net>

On 11/17/15 7:46 PM, ListGnome ProtonMail via Mailman-Users wrote:
> Hi,
> 
> Has anyone been able to order their archive messages in reverse chronological date order - so the most recent post is at the top of the page. (Version 2.1.20)
> 
> I found a related thread in the Mailman-Users archive (dated 2002) but it didn't explain how it was done.
> 
> https://mail.python.org/pipermail/mailman-users/2002-February/017906.html


I looked at the posts in that thread, and as you, I see nothing about
how it was done. Further, I don't see how this can be done in
Defaults.py/mm_cfg.py short of some very complex 'patch the archiver on
the fly' kind of coding if it can be done at all.

Are you talking about just the various date sequence index (date.html)
files or do you also want the messages for each author in the author
index, the messages for each subject in the subject index and the
threads in the thread index also in reverse? For threads, you would
presumably want them ordered by the date of the most recent post in the
thread.

The pipermail archiver is the most difficult code in all of Mailman 2.1
to understand and follow. It may be possible to accomplish this, and the
ultimate patch might be simple, but I don't think it would be too easy
to figure it out.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Nov 19 01:43:44 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 18 Nov 2015 22:43:44 -0800
Subject: [Mailman-Users] Reverse chronological date order for archives
In-Reply-To: <UjBmIUf27jBsnl3sYj4fAGQ9R26Vf_AI899K30r-MEMjwiQR_z5feAYKLc2p7_0puhU00CryKBN0R-xjb0puXg==@protonmail.ch>
References: <8rdWEkySZzy6A1MW3GSxtc1hF6pLu3wj0gQNooASyDfnxl2B2Fcu03e9vPz6REyuQsEG6b7l6l__KdbkVNH95Q==@protonmail.ch>
 <564CDCA2.3080703@msapiro.net>
 <UjBmIUf27jBsnl3sYj4fAGQ9R26Vf_AI899K30r-MEMjwiQR_z5feAYKLc2p7_0puhU00CryKBN0R-xjb0puXg==@protonmail.ch>
Message-ID: <564D6FA0.7050805@msapiro.net>

On 11/18/15 7:03 PM, ListGnome ProtonMail wrote:
> Hi Mark,
> 
> I was referring to the date.html page - but it would also apply to all
> the other views - whereby the most recent post is always at the top
> rather than the bottom.
> 
> I found another post (from 2004) where the author modified the
> HyperDatabase.py file and says that managed to reverse the chronological
> date order.
> 
> https://www.mail-archive.com/mailman-users%40python.org/msg28944.html
> 
> I had a look at the HyperDatabase.py file in my version (2.1.20) and the
> code that he shows doesn't exist in my file.
...


The patch in that post is against Mailman 2.1.5. The reason you can't
find that code in any HyperDatabase.py is the patch is 'reversed' and
the lines that show as deleted were actually added. A proper context
diff for that patch is

$ diff -u HyperDatabase.py.old HyperDatabase.py
--- HyperDatabase.py.old	2015-11-18 22:15:25.000000000 -0800
+++ HyperDatabase.py	2015-11-18 22:20:43.000000000 -0800
@@ -291,9 +291,14 @@

     def first(self, archive, index):
         self.__openIndices(archive)
+        if index == 'date': dateIndex = 1
+        else: dateIndex = 0
         index = getattr(self, index + 'Index')
         try:
             key, msgid = index.first()
+            if dateIndex == 1:
+                index.sorted.reverse()
+                key, msgid = index.first()
             return msgid
         except KeyError:
             return None

And the patch refactored for style against 2.1.20 is

$ diff -u 2.1/Mailman/Archiver/HyperDatabase.py
21/Mailman/Archiver/HyperDatabase.py
--- 2.1/Mailman/Archiver/HyperDatabase.py	2013-05-31 14:44:32.000000000
-0700
+++ 21/Mailman/Archiver/HyperDatabase.py	2015-11-18 22:32:14.000000000 -0800
@@ -297,8 +297,14 @@

     def first(self, archive, index):
         self.__openIndices(archive)
+        if index == 'date':
+            dateIndex = 1
+        else:
+            dateIndex = 0
         index = getattr(self, index + 'Index')
         try:
+            if dateIndex == 1:
+                index.sorted.reverse()
             key, msgid = index.first()
             return msgid
         except KeyError:


I note three things.
1) I have not tested the patch. It looks like it might work, but no
guarantees.
2) The patch only affects the date index.
3) After installing the patch and restarting Mailman, the current period
date index for a list should be reordered with the next post to that
list, but older date index files will not be unless you rebuild the
archive with bin/arch --wipe. See the section "NOTE ON MBOX ANOMALIES:"
in the FAQ at <http://wiki.list.org/x/4030681> for caveats before doing
this.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From dmarino at law.fsu.edu  Thu Nov 19 12:07:17 2015
From: dmarino at law.fsu.edu (Marino, Dominic)
Date: Thu, 19 Nov 2015 17:07:17 +0000
Subject: [Mailman-Users] Files able to remove
Message-ID: <BY2PR0201MB1848954D3AC356EF2D837493E61B0@BY2PR0201MB1848.namprd02.prod.outlook.com>

I need to remove some files to free up some space on my server and I am new to Mailman aspects. So my question is what files are save to remove where I can get the most bang from my buck so to say.

________________________________
Dominic Marino


From nicola.caon at iac.es  Thu Nov 19 12:25:10 2015
From: nicola.caon at iac.es (Nicola Caon)
Date: Thu, 19 Nov 2015 17:25:10 +0000
Subject: [Mailman-Users] Can the list prefix placed at the end of the
 subject line?
Message-ID: <CACQKn0edJGi0VZqJ-rnGXTmABmfNPQ8b1gZVnEs8dVfEh7fk2g@mail.gmail.com>

Hi,

here at my workplace several users have suggested that the list prefix be
removed from the email subject, as the list prefix takes up precious space
in the subject line and makes it more difficult to see what the email is
about without opening it (especially on mobile or small screens).

In our case the problem is magnified by the fact that our main lists are
actually made up of sublists, so that the email subject might go like
[postdocs][researchers] Seminar tomorrow on ...

Other users strongly object to removing the list prefix altogether from the
email subject.

Perhaps the best compromise would be moving the list prefix at the end of
the subject line, however it seems that there is no such option in mailman
(we are running version 2.1.15).

Are there any hidden settings or some simple change to the code that can
enable this option? Or could it be implemented in a future release?

Thanks!

Nicola Caon

From mark at msapiro.net  Thu Nov 19 15:45:24 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 19 Nov 2015 12:45:24 -0800
Subject: [Mailman-Users] Can the list prefix placed at the end of the
 subject line?
In-Reply-To: <CACQKn0edJGi0VZqJ-rnGXTmABmfNPQ8b1gZVnEs8dVfEh7fk2g@mail.gmail.com>
References: <CACQKn0edJGi0VZqJ-rnGXTmABmfNPQ8b1gZVnEs8dVfEh7fk2g@mail.gmail.com>
Message-ID: <564E34E4.8030507@msapiro.net>

On 11/19/15 9:25 AM, Nicola Caon wrote:
> 
> Perhaps the best compromise would be moving the list prefix at the end of
> the subject line, however it seems that there is no such option in mailman
> (we are running version 2.1.15).
> 
> Are there any hidden settings or some simple change to the code that can
> enable this option? Or could it be implemented in a future release?


No, there is no option/setting for this and it is unlikely that there
will be one.

You could look at the code in the prefix_subject() function in
Mailman/Handlers/CookHeaders.py if you're interested in implementing it.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Nov 19 15:55:06 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 19 Nov 2015 12:55:06 -0800
Subject: [Mailman-Users] Files able to remove
In-Reply-To: <BY2PR0201MB1848954D3AC356EF2D837493E61B0@BY2PR0201MB1848.namprd02.prod.outlook.com>
References: <BY2PR0201MB1848954D3AC356EF2D837493E61B0@BY2PR0201MB1848.namprd02.prod.outlook.com>
Message-ID: <564E372A.6020902@msapiro.net>

On 11/19/15 9:07 AM, Marino, Dominic wrote:
> I need to remove some files to free up some space on my server and I am new to Mailman aspects. So my question is what files are save to remove where I can get the most bang from my buck so to say.


The easiest thing, with no adverse effects is to

1) make sure you are not overriding the default

GZIP_ARCHIVE_TXT_FILES = No

2) comment out or remove the 'nightly-gzip' job from Mailman's crontab.

3) remove all the archives/private/*/*.txt.gz files.

Beyond that you can turn off archiving for selected lists and then
remove the archives/private/LISTNAME and archives/private/LISTNAME.mbox
directories.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From dmarino at law.fsu.edu  Thu Nov 19 15:29:46 2015
From: dmarino at law.fsu.edu (Marino, Dominic)
Date: Thu, 19 Nov 2015 20:29:46 +0000
Subject: [Mailman-Users] FW: Can the list prefix placed at the end of the
 subject line?
References: <CACQKn0edJGi0VZqJ-rnGXTmABmfNPQ8b1gZVnEs8dVfEh7fk2g@mail.gmail.com> 
Message-ID: <BY2PR0201MB18480EB8B61659245DF6FD12E61B0@BY2PR0201MB1848.namprd02.prod.outlook.com>

I just need to know which files on the apache/unix server can be deleted to free some space(ie log files) being we are 98% used.  

________________________________
Dominic Marino

-----Original Message-----
From: Mailman-Users [mailto:mailman-users-bounces+dmarino=law.fsu.edu at python.org] On Behalf Of Nicola Caon
Sent: Thursday, November 19, 2015 12:25 PM
To: mailman-users at python.org
Subject: [Mailman-Users] Can the list prefix placed at the end of the subject line?

Hi,

here at my workplace several users have suggested that the list prefix be removed from the email subject, as the list prefix takes up precious space in the subject line and makes it more difficult to see what the email is about without opening it (especially on mobile or small screens).

In our case the problem is magnified by the fact that our main lists are actually made up of sublists, so that the email subject might go like [postdocs][researchers] Seminar tomorrow on ...

Other users strongly object to removing the list prefix altogether from the email subject.

Perhaps the best compromise would be moving the list prefix at the end of the subject line, however it seems that there is no such option in mailman (we are running version 2.1.15).

Are there any hidden settings or some simple change to the code that can enable this option? Or could it be implemented in a future release?

Thanks!

Nicola Caon
------------------------------------------------------
Mailman-Users mailing list Mailman-Users at python.org https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: https://mail.python.org/mailman/options/mailman-users/dmarino%40law.fsu.edu

From arpepper at uwaterloo.ca  Thu Nov 19 17:36:22 2015
From: arpepper at uwaterloo.ca (Adrian Pepper)
Date: Thu, 19 Nov 2015 17:36:22 -0500 (EST)
Subject: [Mailman-Users] Can the list prefix placed at the end of the
 subject line?
Message-ID: <20151119223622.E451B164A151@ubuntu1204-102.cs.uwaterloo.ca>

On 11/19/15 9:25 AM, Nicola Caon wrote:
> 
> Hi,
> 
> here at my workplace several users have suggested that the list prefix be
> removed from the email subject, as the list prefix takes up precious space
> in the subject line and makes it more difficult to see what the email is
> about without opening it (especially on mobile or small screens).
> 
> In our case the problem is magnified by the fact that our main lists are
> actually made up of sublists, so that the email subject might go like
> [postdocs][researchers] Seminar tomorrow on ...
> 
> Other users strongly object to removing the list prefix altogether from the
> email subject.
> 
> Perhaps the best compromise would be moving the list prefix at the end of
> the subject line, however it seems that there is no such option in mailman
> (we are running version 2.1.15).
> 
> Are there any hidden settings or some simple change to the code that can
> enable this option? Or could it be implemented in a future release?
> 
> Thanks!
> 
> Nicola Caon
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/arpepper%40uwaterloo.ca
> 

I see, you are talking about your own prefixes, not [Mailman-Users]
specifically.

What is in [] doesn't need to match the list name exactly (or at all,
although obviously it is better if it matches somewhat).

So you could shorten to e.g. "[pdoc][rsrch] Seminar tomorrow on ..."
Perhaps some of the top lists don't need a tag?


Adrian Pepper
arpepper at uwaterloo.ca


From mark at msapiro.net  Fri Nov 20 11:27:07 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 20 Nov 2015 08:27:07 -0800
Subject: [Mailman-Users] Reducing disk footprint was: FW: Can the list
 prefix placed at the end of the subject line?
In-Reply-To: <BY2PR0201MB18480EB8B61659245DF6FD12E61B0@BY2PR0201MB1848.namprd02.prod.outlook.com>
References: <CACQKn0edJGi0VZqJ-rnGXTmABmfNPQ8b1gZVnEs8dVfEh7fk2g@mail.gmail.com>
 <BY2PR0201MB18480EB8B61659245DF6FD12E61B0@BY2PR0201MB1848.namprd02.prod.outlook.com>
Message-ID: <564F49DB.9050207@msapiro.net>

On 11/19/15 12:29 PM, Marino, Dominic wrote:
> I just need to know which files on the apache/unix server can be deleted to free some space(ie log files) being we are 98% used.  


Please don't hijack existing threads. To start a new topic create a new
post which is not a reply. Do not reply to an existing post even if you
change the subject.

For your question, there are several options.

1) You can remove all the log files in mailman/logs and then run
Mailman's 'bin/mailmanctl reopen'. You can also set up a reasonable
logrotate script to rotate the logs and not keep too much.

2) You can do things to reduce the archives footprint. You can remove
and stop creating the .txt.gz files as they are redundant and not needed.
a) Make sure you are not overriding the Defaults.py setting
'GZIP_ARCHIVE_TXT_FILES = No' in mm_cfg.py.
b) Remove or comment out the nightly_gzip job in Mailman's crontab.
c) Then you can remove all the archives/private/*/*.txt.gz files and
they won't come back.
The links on archive TOC pages may still point to the .txt.gz files, but
this should self correct with the first post to the list.

You can also turn off archiving for selected lists and remove the
archives/private/LISTNAME* directories for that list.

3) You can remove the messages/LC/ and templates/LC/ directories for the
languages (Code LC) you don't want to support.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From addw at phcomp.co.uk  Sat Nov 21 06:22:42 2015
From: addw at phcomp.co.uk (Alain Williams)
Date: Sat, 21 Nov 2015 11:22:42 +0000
Subject: [Mailman-Users] Problem with this list archive
Message-ID: <20151121112242.GM9602@phcomp.co.uk>

It seems to have recently gone to HTTPS and is generating a 404 for HTTP URLs.

This causes a problem with archives email that contains a link to old messages. Eg:

    https://mail.python.org/pipermail/mailman-users/2005-January/042243.html

contains a link to:

    http://mail.python.org/pipermail/mailman-users/2005-January/041697.html

which comes up 404. However, if you manually convert ''http:'' to ''https:'' the
URL then works. Could you persuade Apache to do that redirect ?

Regards

-- 
Alain Williams
Linux/GNU Consultant - Mail systems, Web sites, Networking, Programmer, IT Lecturer.
+44 (0) 787 668 0256  http://www.phcomp.co.uk/
Parliament Hill Computers Ltd. Registration Information: http://www.phcomp.co.uk/contact.php
#include <std_disclaimer.h>

From addw at phcomp.co.uk  Sat Nov 21 06:39:35 2015
From: addw at phcomp.co.uk (Alain Williams)
Date: Sat, 21 Nov 2015 11:39:35 +0000
Subject: [Mailman-Users] Problem pgp signed messages
Message-ID: <20151121113935.GN9602@phcomp.co.uk>

I am trying to allow signed messages on a mail list.

I added multipart/signed & application/pgp-signature to pass_mime_types which
stopped signed emails being rejected by mailman.

My problem is that the signature attachment is being stripped from the message.
What should I be doing ?

Thanks in advance.

At the location: /mailman/admin/ListName/contentfilter

I have:
    filter_content	yes

    filter_mime_types	empty

    pass_mime_types	multipart/mixed
			multipart/alternative
			text/plain
			multipart/signed
			application/pgp-signature
			message/rfc822

    filter_filename_extensions	exe
				bat
				cmd
				com
				pif
				scr
				vbs
				cpl

    pass_filename_extensions	empty

    collapse_alternatives	yes

    convert_html_to_plaintext	yes

    filter_action		reject

I am running mailman 2.1.18-1 on CentOS 6.7.

-- 
Alain Williams
Linux/GNU Consultant - Mail systems, Web sites, Networking, Programmer, IT Lecturer.
+44 (0) 787 668 0256  http://www.phcomp.co.uk/
Parliament Hill Computers Ltd. Registration Information: http://www.phcomp.co.uk/contact.php
#include <std_disclaimer.h>

From mark at msapiro.net  Sat Nov 21 10:59:08 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 21 Nov 2015 07:59:08 -0800
Subject: [Mailman-Users] Problem with this list archive
In-Reply-To: <20151121112242.GM9602@phcomp.co.uk>
References: <20151121112242.GM9602@phcomp.co.uk>
Message-ID: <565094CC.5030902@msapiro.net>

On 11/21/15 3:22 AM, Alain Williams wrote:
> It seems to have recently gone to HTTPS and is generating a 404 for HTTP URLs.
> 
> This causes a problem with archives email that contains a link to old messages. Eg:
> 
>     https://mail.python.org/pipermail/mailman-users/2005-January/042243.html
> 
> contains a link to:
> 
>     http://mail.python.org/pipermail/mailman-users/2005-January/041697.html
> 
> which comes up 404. However, if you manually convert ''http:'' to ''https:'' the
> URL then works. Could you persuade Apache to do that redirect ?



There is a rewrite from http to https in the apache config for
mail.python.org. It was working.

The server was rebooted at about 20/Nov/2015:09:50 (-0500) and since
then the rewrite is apparently not working. I can't see that anything in
the apache configs has changed, and I have tried both reloading and
restarting apache to no avail.

Apparently something changed when the server was rebooted yesterday, but
I don't know what.

Perhaps another postmaster or 'rooter' has an idea.

(I'm on vacation in Croatia at the moment and don't have a lot of time
to devote to this)

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sat Nov 21 11:04:56 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 21 Nov 2015 08:04:56 -0800
Subject: [Mailman-Users] Problem pgp signed messages
In-Reply-To: <20151121113935.GN9602@phcomp.co.uk>
References: <20151121113935.GN9602@phcomp.co.uk>
Message-ID: <56509628.1090508@msapiro.net>

On 11/21/15 3:39 AM, Alain Williams wrote:
> I am trying to allow signed messages on a mail list.
> 
> I added multipart/signed & application/pgp-signature to pass_mime_types which
> stopped signed emails being rejected by mailman.
> 
> My problem is that the signature attachment is being stripped from the message.
> What should I be doing ?


What you have done should be working.

This list for example has in pass_mime_types

multipart
text/plain
text/x-diff
application/pgp-signature

and I'm sure my sig will come through.

Perhaps the sigs being stripped are another protocol, e.g.
application/pkcs7-signature or ??

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20151121/befd3d21/attachment.sig>

From mark at msapiro.net  Sun Nov 22 07:56:06 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 22 Nov 2015 04:56:06 -0800
Subject: [Mailman-Users] Problem with this list archive
In-Reply-To: <565094CC.5030902@msapiro.net>
References: <20151121112242.GM9602@phcomp.co.uk> <565094CC.5030902@msapiro.net>
Message-ID: <5651BB66.5060105@msapiro.net>

On 11/21/15 7:59 AM, Mark Sapiro wrote:
> On 11/21/15 3:22 AM, Alain Williams wrote:
>> It seems to have recently gone to HTTPS and is generating a 404 for HTTP URLs.
...
> There is a rewrite from http to https in the apache config for
> mail.python.org. It was working.
> 
> The server was rebooted at about 20/Nov/2015:09:50 (-0500) and since
> then the rewrite is apparently not working....


Thanks to Ernest W. Durbin III, this is now fixed. At some point an
Apache update added a symlink

/etc/apache2/sites-enabled/000-default.conf ->
/etc/apache2/sites-available/000-default.conf

and when the server was rebooted, this overrode our config for the port
80 listener.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From fil at rezo.net  Sun Nov 22 12:49:09 2015
From: fil at rezo.net (Fil)
Date: Sun, 22 Nov 2015 18:49:09 +0100
Subject: [Mailman-Users] Fwd: Fwd: [Spip-Ann] Discover UAE National Day Deals
In-Reply-To: <CAERh8-nuxd3J335=LQ4BK7gYNgXJsgjXwEi3V6B4ZcUe4CQBTg@mail.gmail.com>
References: <c09699a085e20cbca361b1230d0d3738@zongo.be>
 <5651F47D.3010100@gmail.com>
 <CAERh8-nuxd3J335=LQ4BK7gYNgXJsgjXwEi3V6B4ZcUe4CQBTg@mail.gmail.com>
Message-ID: <CAERh8-kyUusdr_0rwEcC1SK9BP4EHQfsf+TT_yhUoiGuQJD=Dg@mail.gmail.com>

Hello,

a spam went through a moderated list today, and the story is weird. Here's
what I understand/ (mailman version 2.1.20)


First, the message was received and rejected by the list moderators

logs/vette.log:Nov 22 14:56:45 2015 (10257) Message rejected, msgid: <
20151122135632.C128E412040 at mst135.mdreg-mstnew.qlc.co.in>



Then, the (rejected!) message was sent to the list members;


logs/post.log:Nov 22 17:05:41 2015 (15369) post to spip-ann from
mails at clovertourism.com, size=4374, message-id=<
20151122135632.C128E412040 at mst135.mdreg-mstnew.qlc.co.in>, success
logs/smtp.log:Nov 22 17:05:41 2015 (15369) <
20151122135632.C128E412040 at mst135.mdreg-mstnew.qlc.co.in> smtp to spip-ann
for 2880 recips, completed in 2.228 seconds


The "reason" why this went through at 17+ is that I have a cron job that
tries to unshunt messages each day at that hour. So the (discarded then
shunted) message was unshunt and sent to the list.

logs/error.log:Nov 22 14:56:44 2015 (10257) IOError reading list extension:
[Errno 12] Cannot allocate memory:
'/var/local/mailman/lists/spip-ann/extend.py'
logs/error.log:Nov 22 14:56:45 2015 (10257) Uncaught runner exception:
[Errno 12] Cannot allocate memory: '/var/local/mailman/qfiles/virgin'
logs/error.log:Nov 22 14:56:45 2015 (10257) Traceback (most recent call
last):
logs/error.log:Nov 22 14:56:45 2015 (10257) SHUNTING:
1448200600.413774+c9883eaaeaec5a744cfe6a49237d51c2c0373b98


-- Fil

From fil at rezo.net  Sun Nov 22 13:22:11 2015
From: fil at rezo.net (Fil)
Date: Sun, 22 Nov 2015 19:22:11 +0100
Subject: [Mailman-Users] spam coming through a moderated list after a shunt
Message-ID: <CAERh8-me=HAZW_150_tWtHR-qj+Ry_1hnDbJSMCLNtr-VyhbHg@mail.gmail.com>

Hello,

a spam went through a moderated list today, and the story is weird. Here's
what I understand/ (mailman version 2.1.20)


First, the message was received and rejected by the list moderators

logs/vette.log:Nov 22 14:56:45 2015 (10257) Message rejected, msgid: <
20151122135632.C128E412040 at mst135.mdreg-mstnew.qlc.co.in>



Then, the (rejected!) message was sent to the list members;


logs/post.log:Nov 22 17:05:41 2015 (15369) post to spip-ann from
mails at clovertourism.com, size=4374, message-id=<
20151122135632.C128E412040 at mst135.mdreg-mstnew.qlc.co.in>, success
logs/smtp.log:Nov 22 17:05:41 2015 (15369) <
20151122135632.C128E412040 at mst135.mdreg-mstnew.qlc.co.in> smtp to
spip-ann
for 2880 recips, completed in 2.228 seconds


The "reason" why this went through at 17+ is that I have a cron job that
tries to unshunt messages each day at that hour. So the (discarded then
shunted) message was unshunt and sent to the list.

logs/error.log:Nov 22 14:56:44 2015 (10257) IOError reading list extension:
[Errno 12] Cannot allocate memory:
'/var/local/mailman/lists/spip-ann/extend.py'
logs/error.log:Nov 22 14:56:45 2015 (10257) Uncaught runner exception:
[Errno 12] Cannot allocate memory: '/var/local/mailman/qfiles/virgin'
logs/error.log:Nov 22 14:56:45 2015 (10257) Traceback (most recent call
last):
logs/error.log:Nov 22 14:56:45 2015 (10257) SHUNTING:
1448200600.413774+c9883eaaeaec5a744cfe6a49237d51c2c0373b98


-- Fil

(sorry for the initial message with wrong subject line)

From mark at msapiro.net  Sun Nov 22 13:50:30 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 22 Nov 2015 10:50:30 -0800
Subject: [Mailman-Users] Fwd: Fwd: [Spip-Ann] Discover UAE National Day
 Deals
In-Reply-To: <CAERh8-kyUusdr_0rwEcC1SK9BP4EHQfsf+TT_yhUoiGuQJD=Dg@mail.gmail.com>
References: <c09699a085e20cbca361b1230d0d3738@zongo.be>
 <5651F47D.3010100@gmail.com>
 <CAERh8-nuxd3J335=LQ4BK7gYNgXJsgjXwEi3V6B4ZcUe4CQBTg@mail.gmail.com>
 <CAERh8-kyUusdr_0rwEcC1SK9BP4EHQfsf+TT_yhUoiGuQJD=Dg@mail.gmail.com>
Message-ID: <56520E76.3020703@msapiro.net>

On 11/22/15 9:49 AM, Fil wrote:
> Hello,
> 
> a spam went through a moderated list today, and the story is weird. Here's
> what I understand/ (mailman version 2.1.20)


Is there a question in this or is it just an informational report?


> First, the message was received and rejected by the list moderators
> 
> logs/vette.log:Nov 22 14:56:45 2015 (10257) Message rejected, msgid: <
> 20151122135632.C128E412040 at mst135.mdreg-mstnew.qlc.co.in>


This is not a moderator reject. This message was logged by
IncomingRunner after some handler in the pipeline raised
Errors.RejectMessage. This could have happened for example because it
was a non-member post and the list is configured to reject a non-member
post.

In 2.1.20, there should be another 3 lines in this log message giving
the list name, the handler and  the reject reason.


> Then, the (rejected!) message was sent to the list members;
> 
> 
> logs/post.log:Nov 22 17:05:41 2015 (15369) post to spip-ann from
> mails at clovertourism.com, size=4374, message-id=<
> 20151122135632.C128E412040 at mst135.mdreg-mstnew.qlc.co.in>, success
> logs/smtp.log:Nov 22 17:05:41 2015 (15369) <
> 20151122135632.C128E412040 at mst135.mdreg-mstnew.qlc.co.in> smtp to spip-ann
> for 2880 recips, completed in 2.228 seconds
> 
> 
> The "reason" why this went through at 17+ is that I have a cron job that
> tries to unshunt messages each day at that hour. So the (discarded then
> shunted) message was unshunt and sent to the list.
> 
> logs/error.log:Nov 22 14:56:44 2015 (10257) IOError reading list extension:
> [Errno 12] Cannot allocate memory:
> '/var/local/mailman/lists/spip-ann/extend.py'
> logs/error.log:Nov 22 14:56:45 2015 (10257) Uncaught runner exception:
> [Errno 12] Cannot allocate memory: '/var/local/mailman/qfiles/virgin'
> logs/error.log:Nov 22 14:56:45 2015 (10257) Traceback (most recent call
> last):
> logs/error.log:Nov 22 14:56:45 2015 (10257) SHUNTING:
> 1448200600.413774+c9883eaaeaec5a744cfe6a49237d51c2c0373b98>


So a handler said to reject the message. The reject was logged and then
the BounceMessage method was called to send a reject notice. The
BounceMessage method encountered the above exception when trying to
queue the reject notice in the virgin queue, and because of this
exception, the original message was shunted.

Unfortunately, when the message is shunted, the handler that rejected
the message has already been popped from the pipeline so the unshunted
message begins processing with the subsequent handler and is not rejected.

I have a few observations.

1) The skipping of the handler throwing the exception upon unshunting
seems to be a bug.

2) Rejecting rather than discarding unwanted posts produces backscatter
and is not a good idea in general.

3) Normally, I think it is rare that unshunting a shunted post without
analyzing and fixing the underlying exception will succeed. In any case,
blindly running unshunt via cron is probably not a good idea. Is there
some unusual situation that makes you want to do this?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Nov 22 13:58:01 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 22 Nov 2015 10:58:01 -0800
Subject: [Mailman-Users] spam coming through a moderated list after a
 shunt
In-Reply-To: <CAERh8-me=HAZW_150_tWtHR-qj+Ry_1hnDbJSMCLNtr-VyhbHg@mail.gmail.com>
References: <CAERh8-me=HAZW_150_tWtHR-qj+Ry_1hnDbJSMCLNtr-VyhbHg@mail.gmail.com>
Message-ID: <56521039.1070207@msapiro.net>

On 11/22/15 10:22 AM, Fil wrote:

> (sorry for the initial message with wrong subject line)

Copying my reply in the new subject thread ...


On 11/22/15 9:49 AM, Fil wrote:
> Hello,
>
> a spam went through a moderated list today, and the story is weird. Here's
> what I understand/ (mailman version 2.1.20)


Is there a question in this or is it just an informational report?


> First, the message was received and rejected by the list moderators
>
> logs/vette.log:Nov 22 14:56:45 2015 (10257) Message rejected, msgid: <
> 20151122135632.C128E412040 at mst135.mdreg-mstnew.qlc.co.in>


This is not a moderator reject. This message was logged by
IncomingRunner after some handler in the pipeline raised
Errors.RejectMessage. This could have happened for example because it
was a non-member post and the list is configured to reject a non-member
post.

In 2.1.20, there should be another 3 lines in this log message giving
the list name, the handler and  the reject reason.


> Then, the (rejected!) message was sent to the list members;
>
>
> logs/post.log:Nov 22 17:05:41 2015 (15369) post to spip-ann from
> mails at clovertourism.com, size=4374, message-id=<
> 20151122135632.C128E412040 at mst135.mdreg-mstnew.qlc.co.in>, success
> logs/smtp.log:Nov 22 17:05:41 2015 (15369) <
> 20151122135632.C128E412040 at mst135.mdreg-mstnew.qlc.co.in> smtp to spip-ann
> for 2880 recips, completed in 2.228 seconds
>
>
> The "reason" why this went through at 17+ is that I have a cron job that
> tries to unshunt messages each day at that hour. So the (discarded then
> shunted) message was unshunt and sent to the list.
>
> logs/error.log:Nov 22 14:56:44 2015 (10257) IOError reading list
extension:
> [Errno 12] Cannot allocate memory:
> '/var/local/mailman/lists/spip-ann/extend.py'
> logs/error.log:Nov 22 14:56:45 2015 (10257) Uncaught runner exception:
> [Errno 12] Cannot allocate memory: '/var/local/mailman/qfiles/virgin'
> logs/error.log:Nov 22 14:56:45 2015 (10257) Traceback (most recent call
> last):
> logs/error.log:Nov 22 14:56:45 2015 (10257) SHUNTING:
> 1448200600.413774+c9883eaaeaec5a744cfe6a49237d51c2c0373b98>


So a handler said to reject the message. The reject was logged and then
the BounceMessage method was called to send a reject notice. The
BounceMessage method encountered the above exception when trying to
queue the reject notice in the virgin queue, and because of this
exception, the original message was shunted.

Unfortunately, when the message is shunted, the handler that rejected
the message has already been popped from the pipeline so the unshunted
message begins processing with the subsequent handler and is not rejected.

I have a few observations.

1) The skipping of the handler throwing the exception upon unshunting
seems to be a bug.

2) Rejecting rather than discarding unwanted posts produces backscatter
and is not a good idea in general.

3) Normally, I think it is rare that unshunting a shunted post without
analyzing and fixing the underlying exception will succeed. In any case,
blindly running unshunt via cron is probably not a good idea. Is there
some unusual situation that makes you want to do this?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From andrew at hodgsonfamily.org  Sun Nov 22 10:07:33 2015
From: andrew at hodgsonfamily.org (Andrew Hodgson)
Date: Sun, 22 Nov 2015 15:07:33 +0000
Subject: [Mailman-Users] Old chestnut: List integration with forum
Message-ID: <HE1PR04MB1244F37995E061DFC99992FAC5180@HE1PR04MB1244.eurprd04.prod.outlook.com>

Hi,

I have a list which is working well at the moment, but want to introduce a forum that gates to and from the list.

Which forum software currently in people's opinion has the best integration with Mailman 2.x?

Thanks.
Andrew.

From fil at rezo.net  Sun Nov 22 14:43:31 2015
From: fil at rezo.net (Fil)
Date: Sun, 22 Nov 2015 20:43:31 +0100
Subject: [Mailman-Users] Fwd: Fwd: [Spip-Ann] Discover UAE National Day
 Deals
In-Reply-To: <56520E76.3020703@msapiro.net>
References: <c09699a085e20cbca361b1230d0d3738@zongo.be>
 <5651F47D.3010100@gmail.com>
 <CAERh8-nuxd3J335=LQ4BK7gYNgXJsgjXwEi3V6B4ZcUe4CQBTg@mail.gmail.com>
 <CAERh8-kyUusdr_0rwEcC1SK9BP4EHQfsf+TT_yhUoiGuQJD=Dg@mail.gmail.com>
 <56520E76.3020703@msapiro.net>
Message-ID: <CAERh8-=_rWcHT1Oe26TKq1VS12eon=_AjGHHhKp8zNYV29VMbA@mail.gmail.com>

> Is there a question in this or is it just an informational report?
>

Hi Mark,

it's mainly a report, the underlying question being "what happened? is
there a problem somewhere?".


> > First, the message was received and rejected by the list moderators
> >
> > logs/vette.log:Nov 22 14:56:45 2015 (10257) Message rejected, msgid: <
> > 20151122135632.C128E412040 at mst135.mdreg-mstnew.qlc.co.in>
>
>
> This is not a moderator reject. This message was logged by
> IncomingRunner after some handler in the pipeline raised
> Errors.RejectMessage. This could have happened for example because it
> was a non-member post and the list is configured to reject a non-member
> post.
>
In 2.1.20, there should be another 3 lines in this log message giving
> the list name, the handler and  the reject reason.
>


You are right: it was discarded automatically

Nov 22 14:56:45 2015 (10257) Message rejected, msgid:
<20151122135632.C128E412040 at mst135.mdreg-mstnew.qlc
        list: spip-ann,
        handler: Moderate,
        reason: (blah blah blah)


So a handler said to reject the message. The reject was logged and then
> the BounceMessage method was called to send a reject notice. The
> BounceMessage method encountered the above exception when trying to
> queue the reject notice in the virgin queue, and because of this
> exception, the original message was shunted.
>

Secondary question: how comes there can be a memory exception on trying to
load spip-ann/extend.py (which is non-existent).


> Unfortunately, when the message is shunted, the handler that rejected
> the message has already been popped from the pipeline so the unshunted
> message begins processing with the subsequent handler and is not rejected.
>

Thanks for the explanations.


> I have a few observations.
>
> 1) The skipping of the handler throwing the exception upon unshunting
> seems to be a bug.
>

Agreed.



> 2) Rejecting rather than discarding unwanted posts produces backscatter
> and is not a good idea in general.
>

OK. I have configured that list to instead discard and send to the
moderators.



> 3) Normally, I think it is rare that unshunting a shunted post without
> analyzing and fixing the underlying exception will succeed. In any case,
> blindly running unshunt via cron is probably not a good idea. Is there
> some unusual situation that makes you want to do this?
>


I think I added this cron job a few years ago after I found many legitimate
emails had ended in the shunt directory. I run about 800 lists on the
server which helps explain why some choices are sometimes gross :)

Thanks a lot Mark.

From mark at msapiro.net  Sun Nov 22 15:15:05 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 22 Nov 2015 12:15:05 -0800
Subject: [Mailman-Users] spam coming through a moderated list after a
 shunt
In-Reply-To: <CAERh8-=_rWcHT1Oe26TKq1VS12eon=_AjGHHhKp8zNYV29VMbA@mail.gmail.com>
References: <c09699a085e20cbca361b1230d0d3738@zongo.be>
 <5651F47D.3010100@gmail.com>
 <CAERh8-nuxd3J335=LQ4BK7gYNgXJsgjXwEi3V6B4ZcUe4CQBTg@mail.gmail.com>
 <CAERh8-kyUusdr_0rwEcC1SK9BP4EHQfsf+TT_yhUoiGuQJD=Dg@mail.gmail.com>
 <56520E76.3020703@msapiro.net>
 <CAERh8-=_rWcHT1Oe26TKq1VS12eon=_AjGHHhKp8zNYV29VMbA@mail.gmail.com>
Message-ID: <56522249.4070607@msapiro.net>

On 11/22/15 11:43 AM, Fil wrote:
> 
> Secondary question: how comes there can be a memory exception on trying
> to load spip-ann/extend.py (which is non-existent).


This is a bit of a puzzle. Whenever a MailList object is instantiated
(Mailman.MailList.MailList.__init__) there is an attempt to execfile the
list's extend.py whether or not it exists. ENOENT (non-existent file) is
ignored and other errors log the 'IOError reading list extension:'
message, but this exception is caught and other than the logging, is
ignored. I think that log message actually occurred early in the
processing of the message.

The error that actually caused the shunting was the

Uncaught runner exception: [Errno 12] Cannot allocate memory:
'/var/local/mailman/qfiles/virgin'

log message that occurred when trying to queue the rejection notice.

As to why these errors occurred, this is at the interface between Python
and the OS and there was apparently some transient memory issue on the
server that lasted for a bit of time since quite a bit of processing
occurred between instantiating the list and queueing the rejection
notice, but unless there is some clue in other system logs, we don't
know what the issue was.

I will look into the issue of handler skipping with shunted/unshunted
messages and see about making that more robust.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From dave at 3rdValve.net  Mon Nov 23 16:34:04 2015
From: dave at 3rdValve.net (Dave Arndt)
Date: Mon, 23 Nov 2015 16:34:04 -0500
Subject: [Mailman-Users] Installing Mailman
Message-ID: <CAAfcYTibY5z7RSK7urHHRag0N=bHQZWvsOfSeyeR6HS9GCPqcw@mail.gmail.com>

I'm trying to install Mailman 3.0 on CentOS 7.1

checking Mailman status, it shows that its "running".

To test the installation, I'm trying to access

http://domain.com/mailman/listinfo

I receive a "500 Internal Server Error".

How do I determine what's wrong?  Which logs should I look at?

Where can I find truly comprehensive documentation?  All the installation
docs  I've found online seem to be just a little bit different - and none
of them provide insight on troubleshooting problems like the one above.

Thanks in advance for your insight.

- da

From terry.lund at gmail.com  Mon Nov 23 13:53:58 2015
From: terry.lund at gmail.com (Terry Lund)
Date: Mon, 23 Nov 2015 13:53:58 -0500
Subject: [Mailman-Users] Can't handle Administrative requests for mailing
 list
Message-ID: <CA++scQD7Cv5jjuAv0G1T7jP9uq36obCG78aN7Jv+iCY7fkecTQ@mail.gmail.com>

While doing testing for some new mailing lists, I've got a couple of Held
Messages, waiting for administrator handling, but when I go to the admin
screen for the list, none of the available optons will work, the screen
just reloads and displays the same message. I've tried the individual
options for Defer, Accept, Reject, Discard and they don't do anything, and
I have also tried the check box "Discard all messages marked Defer" to no
avail.

My hosting company, Bluehost has tried checking permissions etc or
something and said it's not an issue they can resolve since technically
they don't "support" Mailman. I've used the lists on other accounts with
Bluehost and they work just fine, as expected. This is the first time for
these lists on this account.

What might cause this issue?

-- 
Terry
terry.lund at gmail.com

From mark at msapiro.net  Tue Nov 24 02:53:29 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 23 Nov 2015 23:53:29 -0800
Subject: [Mailman-Users] Can't handle Administrative requests for
 mailing list
In-Reply-To: <CA++scQD7Cv5jjuAv0G1T7jP9uq36obCG78aN7Jv+iCY7fkecTQ@mail.gmail.com>
References: <CA++scQD7Cv5jjuAv0G1T7jP9uq36obCG78aN7Jv+iCY7fkecTQ@mail.gmail.com>
Message-ID: <56541779.4070903@msapiro.net>

On 11/23/15 10:53 AM, Terry Lund wrote:
> While doing testing for some new mailing lists, I've got a couple of Held
> Messages, waiting for administrator handling, but when I go to the admin
> screen for the list, none of the available optons will work, the screen
> just reloads and displays the same message. I've tried the individual
> options for Defer, Accept, Reject, Discard and they don't do anything, and
> I have also tried the check box "Discard all messages marked Defer" to no
> avail.


See the FAQ article at <http://wiki.list.org/x/4030602>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Tue Nov 24 02:59:06 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 23 Nov 2015 23:59:06 -0800
Subject: [Mailman-Users] Installing Mailman
In-Reply-To: <CAAfcYTibY5z7RSK7urHHRag0N=bHQZWvsOfSeyeR6HS9GCPqcw@mail.gmail.com>
References: <CAAfcYTibY5z7RSK7urHHRag0N=bHQZWvsOfSeyeR6HS9GCPqcw@mail.gmail.com>
Message-ID: <565418CA.6020301@msapiro.net>

On 11/23/15 1:34 PM, Dave Arndt wrote:
> I'm trying to install Mailman 3.0 on CentOS 7.1


Questions about Mailman 3 and particularly Postorius and Hyperkitty are
better posted to the mailman-developers at python.org list
<http://mail.python.org/mailman/listinfo/mailman-developers>. Please
join that list if you are not already a member and post there.

The people working most closely with MM 3 are much more likely to read
your post on mailman-developers. We will soon have a
mailman3-users at mailman3.org list (running on MM 3 of course), but there
are still infrastructure issues to work out before that list will be
operational.


> checking Mailman status, it shows that its "running".
> 
> To test the installation, I'm trying to access
> 
> http://domain.com/mailman/listinfo
> 
> I receive a "500 Internal Server Error".
> 
> How do I determine what's wrong?  Which logs should I look at?


/var/log/http/error_log or whatever the relevant Apache error log is on
your server. Also possibly Mailman's Error log.


> Where can I find truly comprehensive documentation?  All the installation
> docs  I've found online seem to be just a little bit different - and none
> of them provide insight on troubleshooting problems like the one above.


MM 3 docs are at <http://docs.list.org/>

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From stephen at xemacs.org  Tue Nov 24 04:11:16 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 24 Nov 2015 18:11:16 +0900
Subject: [Mailman-Users] Problem pgp signed messages
In-Reply-To: <56509628.1090508@msapiro.net>
References: <20151121113935.GN9602@phcomp.co.uk> <56509628.1090508@msapiro.net>
Message-ID: <22100.10676.829650.257884@turnbull.sk.tsukuba.ac.jp>

Mark Sapiro writes:

 > Perhaps the sigs being stripped are another protocol, e.g.
 > application/pkcs7-signature or ??

That was the first thing that occurred to me, although I was thinking
S-MIME.


From dave at 3rdValve.net  Tue Nov 24 08:24:40 2015
From: dave at 3rdValve.net (Dave Arndt)
Date: Tue, 24 Nov 2015 08:24:40 -0500
Subject: [Mailman-Users] Installing Mailman
In-Reply-To: <565418CA.6020301@msapiro.net>
References: <CAAfcYTibY5z7RSK7urHHRag0N=bHQZWvsOfSeyeR6HS9GCPqcw@mail.gmail.com>
 <565418CA.6020301@msapiro.net>
Message-ID: <CAAfcYTjeSWix4_pO9Y6+Bh99BiE+ArNthDox8LVxjd0E=Vwieg@mail.gmail.com>

On Tue, Nov 24, 2015 at 2:59 AM, Mark Sapiro <mark at msapiro.net> wrote:

> On 11/23/15 1:34 PM, Dave Arndt wrote:
> > I'm trying to install Mailman 3.0 on CentOS 7.1
>
>
> Questions about Mailman 3 and particularly Postorius and Hyperkitty are
> better posted to the mailman-developers at python.org list
> <http://mail.python.org/mailman/listinfo/mailman-developers>. Please
> join that list if you are not already a member and post there.
>
> The people working most closely with MM 3 are much more likely to read
> your post on mailman-developers. We will soon have a
> mailman3-users at mailman3.org list (running on MM 3 of course), but there
> are still infrastructure issues to work out before that list will be
> operational.
>
>
So this list is version specific?  I though it was a mailman users list (in
general).

Thanks for the info.  I'll look elsewhere.

- da

From mark at msapiro.net  Tue Nov 24 10:15:51 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 24 Nov 2015 07:15:51 -0800
Subject: [Mailman-Users] spam coming through a moderated list after a
 shunt
In-Reply-To: <56521039.1070207@msapiro.net>
References: <56521039.1070207@msapiro.net>
Message-ID: <56547F27.8010609@msapiro.net>

Mark Sapiro wrote:

> 1) The skipping of the handler throwing the exception upon unshunting
> seems to be a bug.


This bug has been reported at
<https://bugs.launchpad.net/mailman/+bug/1519062> and a fix committed
for the next release.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From stephen at xemacs.org  Tue Nov 24 12:41:03 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 25 Nov 2015 02:41:03 +0900
Subject: [Mailman-Users] Installing Mailman
In-Reply-To: <CAAfcYTjeSWix4_pO9Y6+Bh99BiE+ArNthDox8LVxjd0E=Vwieg@mail.gmail.com>
References: <CAAfcYTibY5z7RSK7urHHRag0N=bHQZWvsOfSeyeR6HS9GCPqcw@mail.gmail.com>
 <565418CA.6020301@msapiro.net>
 <CAAfcYTjeSWix4_pO9Y6+Bh99BiE+ArNthDox8LVxjd0E=Vwieg@mail.gmail.com>
Message-ID: <22100.41263.873936.491863@turnbull.sk.tsukuba.ac.jp>

Dave Arndt writes:

 > So this list is version specific?  I though it was a mailman users
 > list (in general).

No, and yes.  People are welcome to discuss Mailman 3 here.  But
Mailman 3 is not yet widely deployed, so there is little experience
with it on this list.  Eventually Mailman 3 will be user-supported, as
Mailman 2 is, but for now almost all expertise is in the developers,
and they don't hang out here (with the exception of myself and Mark,
and neither of us has global knowledge of Mailman 3).  So Mark is
saying, if you've got a specific question, you're *welcome* to ask the
developers directly.  Or you can ask here and hope to get lucky. ;-)


From dave at 3rdValve.net  Tue Nov 24 12:44:26 2015
From: dave at 3rdValve.net (Dave Arndt)
Date: Tue, 24 Nov 2015 12:44:26 -0500
Subject: [Mailman-Users] Installing Mailman
In-Reply-To: <22100.41263.873936.491863@turnbull.sk.tsukuba.ac.jp>
References: <CAAfcYTibY5z7RSK7urHHRag0N=bHQZWvsOfSeyeR6HS9GCPqcw@mail.gmail.com>
 <565418CA.6020301@msapiro.net>
 <CAAfcYTjeSWix4_pO9Y6+Bh99BiE+ArNthDox8LVxjd0E=Vwieg@mail.gmail.com>
 <22100.41263.873936.491863@turnbull.sk.tsukuba.ac.jp>
Message-ID: <CAAfcYThOajizVHxuge9N5k4D2Y==0cwxMxrJ62Ky1bBFaPL66A@mail.gmail.com>

On Tue, Nov 24, 2015 at 12:41 PM, Stephen J. Turnbull <stephen at xemacs.org>
wrote:

> Dave Arndt writes:
>
>  > So this list is version specific?  I though it was a mailman users
>  > list (in general).
>
> No, and yes.  People are welcome to discuss Mailman 3 here.  But
> Mailman 3 is not yet widely deployed, so there is little experience
> with it on this list.  Eventually Mailman 3 will be user-supported...
>

Then... would the install and the "basics" be that different?

I had tried installing a different version of Mailman earlier (2) and had
the same issues I'm having now.

From stephen at xemacs.org  Tue Nov 24 13:16:07 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 25 Nov 2015 03:16:07 +0900
Subject: [Mailman-Users] Installing Mailman
In-Reply-To: <CAAfcYThOajizVHxuge9N5k4D2Y==0cwxMxrJ62Ky1bBFaPL66A@mail.gmail.com>
References: <CAAfcYTibY5z7RSK7urHHRag0N=bHQZWvsOfSeyeR6HS9GCPqcw@mail.gmail.com>
 <565418CA.6020301@msapiro.net>
 <CAAfcYTjeSWix4_pO9Y6+Bh99BiE+ArNthDox8LVxjd0E=Vwieg@mail.gmail.com>
 <22100.41263.873936.491863@turnbull.sk.tsukuba.ac.jp>
 <CAAfcYThOajizVHxuge9N5k4D2Y==0cwxMxrJ62Ky1bBFaPL66A@mail.gmail.com>
Message-ID: <22100.43367.508964.753357@turnbull.sk.tsukuba.ac.jp>

Dave Arndt writes:

 > Then... would the install and the "basics" be that different?

In a word ... "yes".  Mailman 2 is a monolithic system, with a fair
amount of attention given to "pluggability" of many features.
However, in practice, the vast majority of Mailman 2 systems just use
the default, bundled implementations of things like the archiver.

In Mailman 3, the mail reception and distribution system is one
subproject (dubbed "core"), the archiver ("HyperKitty") is another
subproject, and the web application ("Postorius") that users and list
owners use to configure their accounts and lists yet a third.  The
members overlap somewhat, but really, these are separate projects in
practice.  This isn't a problem for interoperability because the
design is good, but it does mean that installation questions have to
be directed to the right subgroup.  Pretty good progress has been made
on installing all subprojects as a bundle [1], but it's not anywhere
near as easy as "apt-get install mailman3" yet.

 > I had tried installing a different version of Mailman earlier (2)
 > and had the same issues I'm having now.

Ah, in that case maybe your issue is not in Mailman, but rather in
your system.  Then the generic experience with various OSes and MTAs
might be valuable to you.

Exactly what steps did you take to install Mailman 3 on your Centos
host?

FWIW, this:

 >>> http://domain.com/mailman/listinfo

 >>> I receive a "500 Internal Server Error".

looks like an Apache error rather than a Postorius error to me.  That
suggests that Apache is not configured to delegate to Postorius, or
perhaps Postorius isn't installed at all.  AFAICS, the typical install
instructions[1] have you set up Postorius and HyperKitty as standalone
Django apps (not recommended for production, and the instructions are
often not very specific about how to integrate these apps with "real"
webservers).

Footnotes: 
[1]  Eg, http://mailman-bundler.readthedocs.org/en/latest/.


From arpepper at uwaterloo.ca  Tue Nov 24 16:37:26 2015
From: arpepper at uwaterloo.ca (Adrian Pepper)
Date: Tue, 24 Nov 2015 16:37:26 -0500 (EST)
Subject: [Mailman-Users] utf-8 subjects;
 extended "." regexp really necessary?
Message-ID: <20151124213726.48CD7165D890@ubuntu1204-102.cs.uwaterloo.ca>

(Mailman 2.1.12, some local mods, but not around topics...)

 I had a utf-8 subject I was having difficulty matching with a topic regexp.

 Eventually I concluded the subject still had newlines in it when it was
 matched against the regexp.  (That is the continuation lines were not
 joined before matching).  And "." would not match the newline character(s)).

 So, for test purposes...

     Farmers[_ ]Weekly[\s\S\n\r]*Ac

 seemed to match my particular test subject.

 While the following did not.

     Farmers[_ ]Weekly.*Ac

 Am I correct in my conclusion that .* won't match newline characters,
 but <space-chars><not-space-chars><linefeed><carriage-return> will ?
 (And also, that that is the character class I created).

 For production I might need to put [\s\S\n\r]* between every pair of
 characters after a reasonable point in the expression.  Unless I can
 enumerate the possibilities more precisely.  (Which will probably
 result in an even longer looking character class).

 Empirically I see  ?=\n =?utf-8?q?_ after "Weekly" and before "Ac".
 (And it seems the matching is done on the incoming subject, not the
 one formatted for resending, which, with my tag, and the utf-8
 of an incoming tag pushes the expression entirely onto the second
 line where I think the ".*" variant (or even [_ ]) would match.

 More generally, my question applies to any potentially long subject,
 but utf-8 subjects seem to get longer more easily.

 There is no header-equivalent line in the body (it's mime anyway).


Adrian Pepper


From stephen at xemacs.org  Tue Nov 24 21:53:10 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 25 Nov 2015 11:53:10 +0900
Subject: [Mailman-Users]  utf-8 subjects;
 extended "." regexp really necessary?
In-Reply-To: <20151124213726.48CD7165D890@ubuntu1204-102.cs.uwaterloo.ca>
References: <20151124213726.48CD7165D890@ubuntu1204-102.cs.uwaterloo.ca>
Message-ID: <22101.8854.418388.552807@turnbull.sk.tsukuba.ac.jp>

Adrian Pepper writes:
 > (Mailman 2.1.12, some local mods, but not around topics...)
 > 
 >  I had a utf-8 subject I was having difficulty matching with a topic regexp.
 > 
 >  Eventually I concluded the subject still had newlines in it when it was
 >  matched against the regexp.  (That is the continuation lines were not
 >  joined before matching).  And "." would not match the newline character(s)).

 >  Am I correct in my conclusion that .* won't match newline characters,
 >  but <space-chars><not-space-chars><linefeed><carriage-return> will ?
 >  (And also, that that is the character class I created).

Yes.  Here are the docs for Python regular expressions as used in
Mailman: https://docs.python.org/2.7/library/re.html.

In general this problem would be addressed with the DOTALL flag:

    The special characters are:

    '.'
    (Dot.) In the default mode, this matches any character except a
    newline. If the DOTALL flag has been specified, this matches any
    character including a newline.

Note that the definition of "newline" here is exactly "\n".

However, in your case I think there's a simpler method.

 >  For production I might need to put [\s\S\n\r]* between every pair of
 >  characters after a reasonable point in the expression.  Unless I can
 >  enumerate the possibilities more precisely.  (Which will probably
 >  result in an even longer looking character class).

Well, actually what you need is just "\s*" (or perhaps "\s+" or
"(\s|_)+") wherever a space might occur in the topic regexp, I think.
Line folding can only occur at whitespace (breaking this rule would be
noticed by everybody, and so is not likely to go unfixed), and "\s"
already includes "\n".

 >  Empirically I see  ?=\n =?utf-8?q?_ after "Weekly" and before "Ac".
 >  (And it seems the matching is done on the incoming subject, not the
 >  one formatted for resending, which, with my tag, and the utf-8
 >  of an incoming tag pushes the expression entirely onto the second
 >  line where I think the ".*" variant (or even [_ ]) would match.

That would explain your observations, but I am not familiar with the
topic code.  I don't have time to address that until the weekend, and
maybe not then as $DAYJOB is piling up work on me, and Mark is on
vacation in Croatia, so you may have to wait a bit for a final answer
on that.  I'm sorry about that, but I think at least for now the "\s*"
bandaid will get you most of the way to where you want to go.


From lucio at lambrate.inaf.it  Wed Nov 25 07:52:24 2015
From: lucio at lambrate.inaf.it (Lucio Chiappetti)
Date: Wed, 25 Nov 2015 13:52:24 +0100 (CET)
Subject: [Mailman-Users] using topics
Message-ID: <alpine.LSU.2.00.1511251231290.3316@cbfrvqba.ynzoengr.vans.vg>

I am one of the co-moderators of a restricted discussion list within our 
organization (a nationwide research institution). The list is restricted 
in the sense posting is allowed to subscribers only, and subscription is 
allowed only to staff, associates, and former members.

So far we allowed also communications from the trade unions, in the sense 
they post a (signed) message from a non member account, it stops into 
moderation and one of the co-moderator lets it pass. We could have used 
also accept_these_nonmembers (as we do for recognized subscribers who post 
from alternate addresses) but since the various trade unions 
correspondents did not use a fixed address we did not do it.

Now the trade unions requested to be allowed to subscribe as regular users 
from a fixed address, which would relieve us moderators from the need of 
manual acceptance.

I hope this won't cause an excess of trade union related messages with 
respect to other messages, which can be resented by some of our users.

I thought one possibility was to enable "topics", to allow users to decide 
whether they want to see such messages or not.

I have some questions about topics.

- can a user select to receive all messages EXCLUDING a particular
   topic ? It looks like the interface allows to subscribe to one topic,
   and also to receive topic-less messages ... but what about one
   wants to receive ONLY topic-less messages ?

   Assuming messages are classified as :

    - topic trade union
    - no topic

   Shall one create a dummy placeholder topic which will be never
   populated ? or how can one assign a default topic ?

- can the mailman administrator assign topics based on the originator
   address instead of Subject or Keyword or regexp in the first n lines ?

   This may allow to tag the topics automatically (instead of relying on
   the originator using special tags, which is sort of hopeless)


If it helps, the list is managed by mailman 2.1.12 and is not on a 
computer in my institute (I have 2.1.14 on my machine which I use for 
other lists). Neither me nor the administrators of the central site are 
keen to change. but if a more recent version is more suited for this sort 
of filtering we might consider it.

Thanks

-- 
------------------------------------------------------------------------
Lucio Chiappetti - INAF/IASF - via Bassini 15 - I-20133 Milano (Italy)
For more info : http://www.iasf-milano.inaf.it/~lucio/personal.html
------------------------------------------------------------------------
Do not like Firefox >=29 ?  Get Pale Moon !  http://www.palemoon.org

From listgnome at protonmail.ch  Thu Nov 26 00:17:02 2015
From: listgnome at protonmail.ch (ListGnome ProtonMail)
Date: Thu, 26 Nov 2015 00:17:02 -0500
Subject: [Mailman-Users] Reverse chronological date order for archives
Message-ID: <58GV4gFRB7acZM1MQbwdgRDVh2DsVTvoqCNW82_lY4_nJNQiipU7an1LcooNpUKEyfY9mL5HQ4_QAq9Zfo6NvA==@protonmail.ch>

Hi Mark,


I've tested the modified HyperDatabase.py file and it does indeed reverse the chronological order of the archives (but only in date view).
And also as you said, it was necessary to "/bin/arch --wipe LISTNAME" to reverse any existing archives.

Thanks again,
Mark

***
And the patch refactored for style against 2.1.20 is

$ diff -u 2.1/Mailman/Archiver/HyperDatabase.py

def first(self, archive, index):
self.__openIndices(archive)
+ if index == 'date':
+ dateIndex = 1
+ else:
+ dateIndex = 0
index = getattr(self, index + 'Index')
try:
+ if dateIndex == 1:
+ index.sorted.reverse()
key, msgid = index.first()
return msgid
except KeyError:

From Steven.Jones at vuw.ac.nz  Wed Nov 25 14:54:53 2015
From: Steven.Jones at vuw.ac.nz (Steven Jones)
Date: Wed, 25 Nov 2015 19:54:53 +0000
Subject: [Mailman-Users] corrupt list file?
Message-ID: <HK2PR04MB1186F5451842C8D86A40B746D4050@HK2PR04MB1186.apcprd04.prod.outlook.com>

Hi,


We lost our San and on bringing back our mailman server this alert happens every minute,  the filesystems are read/write, but it had gone ro, so I rebooted the server, most lists seem to be working fine,


==========

abrt_version:   2.0.8
cmdline:        /usr/bin/python /usr/lib/mailman/bin/list_members isc
event_log:
executable:     /usr/lib/mailman/bin/list_members
hostname:       vuwunicomailmn1.vuw.ac.nz
kernel:         2.6.32-573.7.1.el6.x86_64
last_occurrence: 1448480447
machineid:      sosreport_uploader-dmidecode=932d92f15b5e914127493c105c3a0fb10af13399d0e4f82ddc20a6fc387628b8
pkg_arch:       x86_64
pkg_epoch:      3
pkg_name:       mailman
pkg_release:    25.el6
pkg_version:    2.1.12
time:           Wed 25 Nov 2015 10:00:56 PM NZDT
uid:            99
username:       nobody

sosreport.tar.xz: Binary file, 1747868 bytes

backtrace:
:MailList.py:610:__load:IOError: [Errno 13] Permission denied: '/var/lib/mailman/lists/isc/config.pck'
:
:Traceback (most recent call last):
:  File "/usr/lib/mailman/bin/list_members", line 286, in <module>
:    main()
:  File "/usr/lib/mailman/bin/list_members", line 230, in main
:    mlist = MailList.MailList(listname, lock=False)
:  File "/usr/lib/mailman/Mailman/MailList.py", line 130, in __init__
:    self.Load()
:  File "/usr/lib/mailman/Mailman/MailList.py", line 644, in Load
:    dict, e = self.__load(file)
:  File "/usr/lib/mailman/Mailman/MailList.py", line 610, in __load
:    fp = open(dbfile)
:IOError: [Errno 13] Permission denied: '/var/lib/mailman/lists/isc/config.pck'
:
:Local variables in innermost frame:
:dbfile: '/var/lib/mailman/lists/isc/config.pck'
:e: IOError(13, 'Permission denied')
:self: <mailing list "isc" (unlocked) at 189a908>
:loadfunc: <built-in function load>
:mtime: 1448406079.5505853

==========


I dont quite understand why I am having this alert "permission denied"  the filesystem is rw,


==========

[root at vuwunicomailmn1 isc]# touch file
[root at vuwunicomailmn1 isc]# ls -l
total 908
-rw-rw----. 1 mailman mailman 459163 Nov 26 08:00 config.pck
-rw-rw----. 1 mailman mailman 458617 Nov 26 08:00 config.pck.last
-rw-r--r--. 1 root    mailman      0 Nov 26 08:52 file
-rw-rw----. 1 mailman mailman    133 Nov 20 10:19 pending.pck
-rw-rw-r--. 1 mailman mailman    615 Nov 20 10:19 request.pck
[root at vuwunicomailmn1 isc]#

==========


is config.pck corrupt?


regards

Steven


From mark at msapiro.net  Thu Nov 26 14:53:51 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 26 Nov 2015 11:53:51 -0800
Subject: [Mailman-Users] corrupt list file?
In-Reply-To: <HK2PR04MB1186F5451842C8D86A40B746D4050@HK2PR04MB1186.apcprd04.prod.outlook.com>
References: <HK2PR04MB1186F5451842C8D86A40B746D4050@HK2PR04MB1186.apcprd04.prod.outlook.com>
Message-ID: <5657634F.5090405@msapiro.net>

On 11/25/2015 11:54 AM, Steven Jones wrote:
=========
> 
> abrt_version:   2.0.8
> cmdline:        /usr/bin/python /usr/lib/mailman/bin/list_members isc
> event_log:
> executable:     /usr/lib/mailman/bin/list_members
> hostname:       vuwunicomailmn1.vuw.ac.nz
> kernel:         2.6.32-573.7.1.el6.x86_64
> last_occurrence: 1448480447
> machineid:      sosreport_uploader-dmidecode=932d92f15b5e914127493c105c3a0fb10af13399d0e4f82ddc20a6fc387628b8
> pkg_arch:       x86_64
> pkg_epoch:      3
> pkg_name:       mailman
> pkg_release:    25.el6
> pkg_version:    2.1.12
> time:           Wed 25 Nov 2015 10:00:56 PM NZDT
> uid:            99
> username:       nobody
                  ^^^^^^

This is the issue. See below.


> sosreport.tar.xz: Binary file, 1747868 bytes
> 
> backtrace:
> :MailList.py:610:__load:IOError: [Errno 13] Permission denied: '/var/lib/mailman/lists/isc/config.pck'
> :
> :Traceback (most recent call last):
> :  File "/usr/lib/mailman/bin/list_members", line 286, in <module>
> :    main()
> :  File "/usr/lib/mailman/bin/list_members", line 230, in main
> :    mlist = MailList.MailList(listname, lock=False)
> :  File "/usr/lib/mailman/Mailman/MailList.py", line 130, in __init__
> :    self.Load()
> :  File "/usr/lib/mailman/Mailman/MailList.py", line 644, in Load
> :    dict, e = self.__load(file)
> :  File "/usr/lib/mailman/Mailman/MailList.py", line 610, in __load
> :    fp = open(dbfile)
> :IOError: [Errno 13] Permission denied: '/var/lib/mailman/lists/isc/config.pck'
> :
> :Local variables in innermost frame:
> :dbfile: '/var/lib/mailman/lists/isc/config.pck'
> :e: IOError(13, 'Permission denied')
> :self: <mailing list "isc" (unlocked) at 189a908>
> :loadfunc: <built-in function load>
> :mtime: 1448406079.5505853
> 
> ==========
> 
> 
> I dont quite understand why I am having this alert "permission denied"  the filesystem is rw,


User nobody does not have permission to read the list's config.pck.

How is this bin/list_members being run? It neesds to be run by root or a
user in the 'mailman' group.

> 
> [root at vuwunicomailmn1 isc]# touch file
> [root at vuwunicomailmn1 isc]# ls -l
> total 908
> -rw-rw----. 1 mailman mailman 459163 Nov 26 08:00 config.pck
> -rw-rw----. 1 mailman mailman 458617 Nov 26 08:00 config.pck.last
> -rw-r--r--. 1 root    mailman      0 Nov 26 08:52 file
> -rw-rw----. 1 mailman mailman    133 Nov 20 10:19 pending.pck
> -rw-rw-r--. 1 mailman mailman    615 Nov 20 10:19 request.pck
> [root at vuwunicomailmn1 isc]#
> 
> ==========
> 
> 
> is config.pck corrupt?


No.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Nov 26 16:10:50 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 26 Nov 2015 13:10:50 -0800
Subject: [Mailman-Users] using topics
In-Reply-To: <alpine.LSU.2.00.1511251231290.3316@cbfrvqba.ynzoengr.vans.vg>
References: <alpine.LSU.2.00.1511251231290.3316@cbfrvqba.ynzoengr.vans.vg>
Message-ID: <5657755A.9000502@msapiro.net>

On 11/25/2015 04:52 AM, Lucio Chiappetti wrote:
> 
> I have some questions about topics.
> 
> - can a user select to receive all messages EXCLUDING a particular
>   topic ? It looks like the interface allows to subscribe to one topic,
>   and also to receive topic-less messages ... but what about one
>   wants to receive ONLY topic-less messages ?


There is an out of date branch at
<https://code.launchpad.net/~msapiro/mailman/topics> that implements the
ability for users to choose to receive only messages that match no
topic, but there are other ways to accomplish this. E.g., you can define
the topic 'no_topic' with a regexp which won't match any message. E.g.
'(?!)' - this is a negative lookahead which only matches something which
doesn't begin with a null string (every string has a null string at
every position). Then users who want all but the trade union messages
can subscribe to no_topic and also elect to receive messages which match
no topic.


>   Assuming messages are classified as :
> 
>    - topic trade union
>    - no topic
> 
>   Shall one create a dummy placeholder topic which will be never
>   populated ? or how can one assign a default topic ?


As suggested above, just create a regular topic with a regexp that
matches nothing.


> - can the mailman administrator assign topics based on the originator
>   address instead of Subject or Keyword or regexp in the first n lines ?


No.


>   This may allow to tag the topics automatically (instead of relying on
>   the originator using special tags, which is sort of hopeless)


You could implement a custom handler (see
<http://wiki.list.org/x/4030615> to be in the pipeline before Tagger
which would add a Keywords: header based on sender.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From Steven.Jones at vuw.ac.nz  Thu Nov 26 14:41:35 2015
From: Steven.Jones at vuw.ac.nz (Steven Jones)
Date: Thu, 26 Nov 2015 19:41:35 +0000
Subject: [Mailman-Users] corrupt list file?
In-Reply-To: <HK2PR04MB1186F5451842C8D86A40B746D4050@HK2PR04MB1186.apcprd04.prod.outlook.com>
References: <HK2PR04MB1186F5451842C8D86A40B746D4050@HK2PR04MB1186.apcprd04.prod.outlook.com>
Message-ID: <HK2PR04MB11865BB195BCB1F351FD99BAD4040@HK2PR04MB1186.apcprd04.prod.outlook.com>

The list was obsolete so I have deleted, and rebooted the server.  The list no longer exists but I am still getting  this alert 1 to several times a minute.

So what could be calling "/usr/bin/python /usr/lib/mailman/bin/list_members isc"   ?

regards

Steven


________________________________________
From: Mailman-Users <mailman-users-bounces+steven.jones=vuw.ac.nz at python.org> on behalf of Steven Jones <Steven.Jones at vuw.ac.nz>
Sent: Thursday, 26 November 2015 8:54 a.m.
To: Mailman-Users at python.org
Subject: [Mailman-Users] corrupt list file?

Hi,


We lost our San and on bringing back our mailman server this alert happens every minute,  the filesystems are read/write, but it had gone ro, so I rebooted the server, most lists seem to be working fine,




==========

abrt_version:   2.0.8
cmdline:        /usr/bin/python /usr/lib/mailman/bin/list_members isc
event_log:
executable:     /usr/lib/mailman/bin/list_members
hostname:       vuwunicomailmn1.vuw.ac.nz
kernel:         2.6.32-573.7.1.el6.x86_64
last_occurrence: 1448480447
machineid:      sosreport_uploader-dmidecode=932d92f15b5e914127493c105c3a0fb10af13399d0e4f82ddc20a6fc387628b8
pkg_arch:       x86_64
pkg_epoch:      3
pkg_name:       mailman
pkg_release:    25.el6
pkg_version:    2.1.12
time:           Wed 25 Nov 2015 10:00:56 PM NZDT
uid:            99
username:       nobody

sosreport.tar.xz: Binary file, 1747868 bytes

backtrace:
:MailList.py:610:__load:IOError: [Errno 13] Permission denied: '/var/lib/mailman/lists/isc/config.pck'
:
:Traceback (most recent call last):
:  File "/usr/lib/mailman/bin/list_members", line 286, in <module>
:    main()
:  File "/usr/lib/mailman/bin/list_members", line 230, in main
:    mlist = MailList.MailList(listname, lock=False)
:  File "/usr/lib/mailman/Mailman/MailList.py", line 130, in __init__
:    self.Load()
:  File "/usr/lib/mailman/Mailman/MailList.py", line 644, in Load
:    dict, e = self.__load(file)
:  File "/usr/lib/mailman/Mailman/MailList.py", line 610, in __load
:    fp = open(dbfile)
:IOError: [Errno 13] Permission denied: '/var/lib/mailman/lists/isc/config.pck'
:
:Local variables in innermost frame:
:dbfile: '/var/lib/mailman/lists/isc/config.pck'
:e: IOError(13, 'Permission denied')
:self: <mailing list "isc" (unlocked) at 189a908>
:loadfunc: <built-in function load>
:mtime: 1448406079.5505853

==========


I dont quite understand why I am having this alert "permission denied"  the filesystem is rw,


==========

[root at vuwunicomailmn1 isc]# touch file
[root at vuwunicomailmn1 isc]# ls -l
total 908
-rw-rw----. 1 mailman mailman 459163 Nov 26 08:00 config.pck
-rw-rw----. 1 mailman mailman 458617 Nov 26 08:00 config.pck.last
-rw-r--r--. 1 root    mailman      0 Nov 26 08:52 file
-rw-rw----. 1 mailman mailman    133 Nov 20 10:19 pending.pck
-rw-rw-r--. 1 mailman mailman    615 Nov 20 10:19 request.pck
[root at vuwunicomailmn1 isc]#

==========


is config.pck corrupt?


regards

Steven

------------------------------------------------------
Mailman-Users mailing list Mailman-Users at python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: https://mail.python.org/mailman/options/mailman-users/steven.jones%40vuw.ac.nz

From mark at msapiro.net  Thu Nov 26 16:34:01 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 26 Nov 2015 13:34:01 -0800
Subject: [Mailman-Users] corrupt list file?
In-Reply-To: <HK2PR04MB11865BB195BCB1F351FD99BAD4040@HK2PR04MB1186.apcprd04.prod.outlook.com>
References: <HK2PR04MB1186F5451842C8D86A40B746D4050@HK2PR04MB1186.apcprd04.prod.outlook.com>
 <HK2PR04MB11865BB195BCB1F351FD99BAD4040@HK2PR04MB1186.apcprd04.prod.outlook.com>
Message-ID: <56577AC9.3080304@msapiro.net>

On 11/26/2015 11:41 AM, Steven Jones wrote:
> The list was obsolete so I have deleted, and rebooted the server.  The list no longer exists but I am still getting  this alert 1 to several times a minute.


See my prior reply at
<https://mail.python.org/pipermail/mailman-users/2015-November/080142.html>.


> So what could be calling "/usr/bin/python /usr/lib/mailman/bin/list_members isc"   ?


That's a question only you can answer. I would start with

sudo grep -r '/usr/lib/mailman/bin/list_members' /etc

If that doesn't find it, maybe

sudo grep -r 'list_members' /etc

or

sudo grep -r '/usr/lib/mailman/bin/list_members' /

or

sudo grep -r 'list_members' /

will.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From Steven.Jones at vuw.ac.nz  Thu Nov 26 17:11:42 2015
From: Steven.Jones at vuw.ac.nz (Steven Jones)
Date: Thu, 26 Nov 2015 22:11:42 +0000
Subject: [Mailman-Users] corrupt list file?
In-Reply-To: <56577AC9.3080304@msapiro.net>
References: <HK2PR04MB1186F5451842C8D86A40B746D4050@HK2PR04MB1186.apcprd04.prod.outlook.com>
 <HK2PR04MB11865BB195BCB1F351FD99BAD4040@HK2PR04MB1186.apcprd04.prod.outlook.com>,
 <56577AC9.3080304@msapiro.net>
Message-ID: <HK2PR04MB1186AE489CE27E8FE214D8DCD4040@HK2PR04MB1186.apcprd04.prod.outlook.com>

Thanks,

Mail seems extremely slow for us right now.

I tried running,

==========
[root at vuwunicomailmn1 ~]# /usr/lib/mailman/bin/check_perms -f

Traceback (most recent call last):
  File "/usr/lib/mailman/bin/check_perms", line 395, in <module>
    checkdata()
  File "/usr/lib/mailman/bin/check_perms", line 349, in checkdata
    mode = statmode(path)
  File "/usr/lib/mailman/bin/check_perms", line 82, in statmode
    return os.stat(path)[ST_MODE]
OSError: [Errno 20] Not a directory: '/var/lib/mailman/lists/owners-final/config.pck'
[root at vuwunicomailmn1 ~]# 
===========

It could be I have a trashed OS/box after losing the disk?

===========
[root at vuwunicomailmn1 ~]# sudo grep -r '/usr/lib/mailman/bin/list_members' /etc
Binary file /etc/prelink.cache matches
[root at vuwunicomailmn1 ~]# 
===========

===========
[root at vuwunicomailmn1 ~]# sudo grep -r 'list_members' /etc
Binary file /etc/prelink.cache matches
[root at vuwunicomailmn1 ~]# 
===========

===========
sudo grep -r 'list_members' /

8><----
===========

has a lot of output.

regards

Steven 
________________________________________
From: Mailman-Users <mailman-users-bounces+steven.jones=vuw.ac.nz at python.org> on behalf of Mark Sapiro <mark at msapiro.net>
Sent: Friday, 27 November 2015 10:34 a.m.
To: mailman-users at python.org
Subject: Re: [Mailman-Users] corrupt list file?

On 11/26/2015 11:41 AM, Steven Jones wrote:
> The list was obsolete so I have deleted, and rebooted the server.  The list no longer exists but I am still getting  this alert 1 to several times a minute.


See my prior reply at
<https://mail.python.org/pipermail/mailman-users/2015-November/080142.html>.


> So what could be calling "/usr/bin/python /usr/lib/mailman/bin/list_members isc"   ?


That's a question only you can answer. I would start with

sudo grep -r '/usr/lib/mailman/bin/list_members' /etc

If that doesn't find it, maybe

sudo grep -r 'list_members' /etc

or

sudo grep -r '/usr/lib/mailman/bin/list_members' /

or

sudo grep -r 'list_members' /

will.

--
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan
------------------------------------------------------
Mailman-Users mailing list Mailman-Users at python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: https://mail.python.org/mailman/options/mailman-users/steven.jones%40vuw.ac.nz

From mark at msapiro.net  Thu Nov 26 22:05:54 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 26 Nov 2015 19:05:54 -0800
Subject: [Mailman-Users] corrupt list file?
In-Reply-To: <HK2PR04MB1186AE489CE27E8FE214D8DCD4040@HK2PR04MB1186.apcprd04.prod.outlook.com>
References: <HK2PR04MB1186F5451842C8D86A40B746D4050@HK2PR04MB1186.apcprd04.prod.outlook.com>
 <HK2PR04MB11865BB195BCB1F351FD99BAD4040@HK2PR04MB1186.apcprd04.prod.outlook.com>
 <56577AC9.3080304@msapiro.net>
 <HK2PR04MB1186AE489CE27E8FE214D8DCD4040@HK2PR04MB1186.apcprd04.prod.outlook.com>
Message-ID: <5657C892.6010102@msapiro.net>

On 11/26/2015 02:11 PM, Steven Jones wrote:
> 
> Mail seems extremely slow for us right now.


This is consistent with my guess below.


> I tried running,
> 
> ==========
> [root at vuwunicomailmn1 ~]# /usr/lib/mailman/bin/check_perms -f
> 
> Traceback (most recent call last):
>   File "/usr/lib/mailman/bin/check_perms", line 395, in <module>
>     checkdata()
>   File "/usr/lib/mailman/bin/check_perms", line 349, in checkdata
>     mode = statmode(path)
>   File "/usr/lib/mailman/bin/check_perms", line 82, in statmode
>     return os.stat(path)[ST_MODE]
> OSError: [Errno 20] Not a directory: '/var/lib/mailman/lists/owners-final/config.pck'
> [root at vuwunicomailmn1 ~]# 
> ===========


1) Your issue won't be fixed by check_perms because the permissions you
posted are OK. The issue is that the list_members script is being run by
a user (nobody) that isn't supposed to have permission.

Is there a list named owners-final and a
/var/lib/mailman/lists/owners-final/ directory without a
/var/lib/mailman/lists/owners-final/config.pck file. If so, the missing
config.pck file is an issue. On the other hand, if you just happen to
have created a /var/lib/mailman/lists/owners-final/ directory for some
other purpose, that's something that check_perms can't handle.


> It could be I have a trashed OS/box after losing the disk?


It could be. Something is clearly not the same.


> ===========
> [root at vuwunicomailmn1 ~]# sudo grep -r '/usr/lib/mailman/bin/list_members' /etc
> Binary file /etc/prelink.cache matches
> [root at vuwunicomailmn1 ~]# 
> ===========
> 
> ===========
> [root at vuwunicomailmn1 ~]# sudo grep -r 'list_members' /etc
> Binary file /etc/prelink.cache matches
> [root at vuwunicomailmn1 ~]# 
> ===========
> 
> ===========
> sudo grep -r 'list_members' /
> 
> 8><----
> ===========
> 
> has a lot of output.

And is there anything in that output that doesn't look like a Mailman file?


My best guess at this point is your MTA has some sort of milter or proxy
that is run on incoming mail to try to determine if mail is a list post
and if so, if the sender is a list member, and this milter/proxy/???
tries to do bin/list_members to see if the sender is a member, and the
process is running as user 'nobody' and doesn't have permission to do
list_members.

Try looking at your MTA configs and logs.

Also is there any clue in

> sosreport.tar.xz: Binary file, 1747868 bytes

that might indicate what process is actually issuing the

> /usr/bin/python /usr/lib/mailman/bin/list_members isc

command?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From dave at 3rdValve.net  Sat Nov 28 11:30:36 2015
From: dave at 3rdValve.net (Dave Arndt)
Date: Sat, 28 Nov 2015 11:30:36 -0500
Subject: [Mailman-Users] command listinfo not in docroot (10005)
Message-ID: <CAAfcYTh2ySD98DkSm4cCpshhgOAzReT7zJfN5NRRSvWyVr2cug@mail.gmail.com>

After installing mailman (with yum), I get an error when attempting to
access the listinfo page:

"Internal Server Error"


Inspecting the logs in httpd, I see this in suexec-log:

command listinfo not in docroot (10005)

I do NOT have the option of disabling suexec.

After considerable time spent looking into this, it seems that I need to
have mailman installed under "DocumentRoot", which on my host appears to be
/var/www/html

Further reading indicates that i cannot use the "installroot" option on yum
to change the installation root directory, UNLESS I've created my own
custom RPM.  That rules that out.

So now, I'm resorting to 100% manual installation...

My questions are:

1) Am I taking the right approach in solving this problem?  That is, by
installing under docroot, will it address the Apache suexec problem we're
seeing?

2) If docroot is /var/www/html, do i create the mailman directory
*directly* beneath the /var/www.html folder?  i.e. - /var/www.html/mailman
(that just doesn't seem right...?)

Is there some other way to solve this problem?  (Very difficult package to
get installed and running, unless your a huge linux/mailman guru, I
guess...)

Thanks,

- da

From dave at 3rdValve.net  Sat Nov 28 12:35:45 2015
From: dave at 3rdValve.net (Dave Arndt)
Date: Sat, 28 Nov 2015 12:35:45 -0500
Subject: [Mailman-Users] command listinfo not in docroot (10005)
In-Reply-To: <CAAfcYTh2ySD98DkSm4cCpshhgOAzReT7zJfN5NRRSvWyVr2cug@mail.gmail.com>
References: <CAAfcYTh2ySD98DkSm4cCpshhgOAzReT7zJfN5NRRSvWyVr2cug@mail.gmail.com>
Message-ID: <CAAfcYTj6rMQRBxPQdvwjwXchLR38WVX=NZmw5kcnzjxuuGJ8+w@mail.gmail.com>

More info on this:

I added a symlink under "DocumentRoot" (which is specified as /var/www/html
on my system), so the link is  "/var/www/html/*mailman-cgi-bin*" , and
added a ScriptAlias in httpd.conf to point there:

ScriptAlias /*mailman*/ "/var/www/html/*mailman-cgi-bin*/"

After adding the alias, I restarted apache: "apachectl restart"

Still no luck.

When I attempt to access the ~/*mailman*/listinfo page, I still get the
"InternalAccessError" and this line in suexec_log:

"command listinfo not in docroot (10005)"

What could I being doing wrong here?







On Sat, Nov 28, 2015 at 11:30 AM, Dave Arndt <dave at 3rdvalve.net> wrote:

> After installing mailman (with yum), I get an error when attempting to
> access the listinfo page:
>
> "Internal Server Error"
>
>
> Inspecting the logs in httpd, I see this in suexec-log:
>
> command listinfo not in docroot (10005)
>
> I do NOT have the option of disabling suexec.
>
> After considerable time spent looking into this, it seems that I need to
> have mailman installed under "DocumentRoot", which on my host appears to be
> /var/www/html
>
> Further reading indicates that i cannot use the "installroot" option on
> yum to change the installation root directory, UNLESS I've created my own
> custom RPM.  That rules that out.
>
> So now, I'm resorting to 100% manual installation...
>
> My questions are:
>
> 1) Am I taking the right approach in solving this problem?  That is, by
> installing under docroot, will it address the Apache suexec problem we're
> seeing?
>
> 2) If docroot is /var/www/html, do i create the mailman directory
> *directly* beneath the /var/www.html folder?  i.e. - /var/www.html/mailman
> (that just doesn't seem right...?)
>
> Is there some other way to solve this problem?  (Very difficult package to
> get installed and running, unless your a huge linux/mailman guru, I
> guess...)
>
> Thanks,
>
> - da
>
>
>
>

From dave at 3rdValve.net  Sat Nov 28 12:38:54 2015
From: dave at 3rdValve.net (Dave Arndt)
Date: Sat, 28 Nov 2015 12:38:54 -0500
Subject: [Mailman-Users] command listinfo not in docroot (10005)
In-Reply-To: <CAAfcYTj6rMQRBxPQdvwjwXchLR38WVX=NZmw5kcnzjxuuGJ8+w@mail.gmail.com>
References: <CAAfcYTh2ySD98DkSm4cCpshhgOAzReT7zJfN5NRRSvWyVr2cug@mail.gmail.com>
 <CAAfcYTj6rMQRBxPQdvwjwXchLR38WVX=NZmw5kcnzjxuuGJ8+w@mail.gmail.com>
Message-ID: <CAAfcYTh23FfmDuQsOKSE=kU6T4_VcXE4SmC7A0akZBkrjaMouA@mail.gmail.com>

PS: The symlink does indeed point to the correct mailman cgi folder:

/usr/lib/mailman/cgi-bin

/var/www.html/mailman-cgi-bin

Both of these folders have:

admin admindb confim create edithtml listinfo options private rmlist roster
subscribe



On Sat, Nov 28, 2015 at 12:35 PM, Dave Arndt <dave at 3rdvalve.net> wrote:

> More info on this:
>
> I added a symlink under "DocumentRoot" (which is specified as
> /var/www/html on my system), so the link is  "/var/www/html/
> *mailman-cgi-bin*" , and added a ScriptAlias in httpd.conf to point there:
>
> ScriptAlias /*mailman*/ "/var/www/html/*mailman-cgi-bin*/"
>
> After adding the alias, I restarted apache: "apachectl restart"
>
> Still no luck.
>
> When I attempt to access the ~/*mailman*/listinfo page, I still get the
> "InternalAccessError" and this line in suexec_log:
>
> "command listinfo not in docroot (10005)"
>
> What could I being doing wrong here?
>
>
>
>
>
>
>
> On Sat, Nov 28, 2015 at 11:30 AM, Dave Arndt <dave at 3rdvalve.net> wrote:
>
>> After installing mailman (with yum), I get an error when attempting to
>> access the listinfo page:
>>
>> "Internal Server Error"
>>
>>
>> Inspecting the logs in httpd, I see this in suexec-log:
>>
>> command listinfo not in docroot (10005)
>>
>> I do NOT have the option of disabling suexec.
>>
>> After considerable time spent looking into this, it seems that I need to
>> have mailman installed under "DocumentRoot", which on my host appears to be
>> /var/www/html
>>
>> Further reading indicates that i cannot use the "installroot" option on
>> yum to change the installation root directory, UNLESS I've created my own
>> custom RPM.  That rules that out.
>>
>> So now, I'm resorting to 100% manual installation...
>>
>> My questions are:
>>
>> 1) Am I taking the right approach in solving this problem?  That is, by
>> installing under docroot, will it address the Apache suexec problem we're
>> seeing?
>>
>> 2) If docroot is /var/www/html, do i create the mailman directory
>> *directly* beneath the /var/www.html folder?  i.e. - /var/www.html/mailman
>> (that just doesn't seem right...?)
>>
>> Is there some other way to solve this problem?  (Very difficult package
>> to get installed and running, unless your a huge linux/mailman guru, I
>> guess...)
>>
>> Thanks,
>>
>> - da
>>
>>
>>
>>
>

From lac at openend.se  Sat Nov 28 12:43:17 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 28 Nov 2015 18:43:17 +0100
Subject: [Mailman-Users] command listinfo not in docroot (10005)
In-Reply-To: <CAAfcYTh2ySD98DkSm4cCpshhgOAzReT7zJfN5NRRSvWyVr2cug@mail.gmail.com>
References: <CAAfcYTh2ySD98DkSm4cCpshhgOAzReT7zJfN5NRRSvWyVr2cug@mail.gmail.com>
Message-ID: <201511281743.tASHhHfe022142@fido.openend.se>

Do you have selinux installed?  It can cause a gazillion problems.

Laura

From dave at 3rdValve.net  Sat Nov 28 12:48:52 2015
From: dave at 3rdValve.net (Dave Arndt)
Date: Sat, 28 Nov 2015 12:48:52 -0500
Subject: [Mailman-Users] command listinfo not in docroot (10005)
In-Reply-To: <201511281743.tASHhHfe022142@fido.openend.se>
References: <CAAfcYTh2ySD98DkSm4cCpshhgOAzReT7zJfN5NRRSvWyVr2cug@mail.gmail.com>
 <201511281743.tASHhHfe022142@fido.openend.se>
Message-ID: <CAAfcYTjokvn4BGFhz+FZa+AT9vCgbsA2XgRxhpYch2A9+x+3wQ@mail.gmail.com>

On Sat, Nov 28, 2015 at 12:43 PM, Laura Creighton <lac at openend.se> wrote:

> selinux


Running "getenforce" returns "Disabled"

From lac at openend.se  Sat Nov 28 13:02:22 2015
From: lac at openend.se (Laura Creighton)
Date: Sat, 28 Nov 2015 19:02:22 +0100
Subject: [Mailman-Users] command listinfo not in docroot (10005)
In-Reply-To: <CAAfcYTjokvn4BGFhz+FZa+AT9vCgbsA2XgRxhpYch2A9+x+3wQ@mail.gmail.com>
References: <CAAfcYTh2ySD98DkSm4cCpshhgOAzReT7zJfN5NRRSvWyVr2cug@mail.gmail.com>
 <201511281743.tASHhHfe022142@fido.openend.se>
 <CAAfcYTjokvn4BGFhz+FZa+AT9vCgbsA2XgRxhpYch2A9+x+3wQ@mail.gmail.com>
Message-ID: <201511281802.tASI2MJo022398@fido.openend.se>

In a message of Sat, 28 Nov 2015 12:48:52 -0500, Dave Arndt writes:
>On Sat, Nov 28, 2015 at 12:43 PM, Laura Creighton <lac at openend.se> wrote:
>
>> selinux

That eliminates a lot of hell.  Good.  I wish I could be of more help,
but that much I do know.

Laura

From mark at msapiro.net  Sat Nov 28 14:08:35 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 28 Nov 2015 11:08:35 -0800
Subject: [Mailman-Users] command listinfo not in docroot (10005)
In-Reply-To: <CAAfcYTh2ySD98DkSm4cCpshhgOAzReT7zJfN5NRRSvWyVr2cug@mail.gmail.com>
References: <CAAfcYTh2ySD98DkSm4cCpshhgOAzReT7zJfN5NRRSvWyVr2cug@mail.gmail.com>
Message-ID: <5659FBB3.9000006@msapiro.net>

On 11/28/2015 08:30 AM, Dave Arndt wrote:
> 
> My questions are:
> 
> 1) Am I taking the right approach in solving this problem?  That is, by
> installing under docroot, will it address the Apache suexec problem we're
> seeing?


Maybe. See <http://wiki.list.org/DOC/Apache%2BSuexec>


> 2) If docroot is /var/www/html, do i create the mailman directory
> *directly* beneath the /var/www.html folder?  i.e. - /var/www.html/mailman
> (that just doesn't seem right...?)


The above FAQ seems to indicate that /var/www/html/mailman is correct.

Also, have you looked at all 20 steps under "suEXEC Security Model" at
<http://httpd.apache.org/docs/2.4/suexec.html>?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From stephen at xemacs.org  Sat Nov 28 18:49:10 2015
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 29 Nov 2015 08:49:10 +0900
Subject: [Mailman-Users] command listinfo not in docroot (10005)
In-Reply-To: <5659FBB3.9000006@msapiro.net>
References: <CAAfcYTh2ySD98DkSm4cCpshhgOAzReT7zJfN5NRRSvWyVr2cug@mail.gmail.com>
 <5659FBB3.9000006@msapiro.net>
Message-ID: <22106.15734.812335.331735@turnbull.sk.tsukuba.ac.jp>

Mark Sapiro writes:
 > On 11/28/2015 08:30 AM, Dave Arndt wrote:

 > > My questions are:
 > > 
 > > 1) Am I taking the right approach in solving this problem?  That is, by
 > > installing under docroot, will it address the Apache suexec problem we're
 > > seeing?

Which docroot?  suEXEC has its own docroot configured, which may be
different from Apache's (or maybe not, the docs are unclear on this).

 > Maybe. See <http://wiki.list.org/DOC/Apache%2BSuexec>
 > 
 > 
 > > 2) If docroot is /var/www/html, do i create the mailman directory
 > > *directly* beneath the /var/www.html folder?  i.e. -
 > > /var/www.html/mailman (that just doesn't seem right...?)

As you say, that just doesn't feel right; usual security models say
keep your executables out of the document tree.  Whoever configured
your suEXEC may have felt the same way.  Also, according to the docs
referenced below, --suexec-docroot defaults to $(DATADIR)/htdocs.  I
suspect your $(DATADIR) is indeed /var/www, and you evidently have the
"root" for Apache itself set to /var/www/html, but if that value was
defaulted for suEXEC, suEXEC may think --suexec-docroot=/var/www/htdocs.

You may also be running into one of the other restrictions, such as
uidmin or gidmin.  Eg, on my Debian system Mailman's uid = gid = 38,
but in suEXEC's default, uidmin = gidmin = 100, so I'd lose.  (In that
case suEXEC's error log is confusing, but that kind of thing happens.)

 > Also, have you looked at all 20 steps under "suEXEC Security Model" at
 > <http://httpd.apache.org/docs/2.4/suexec.html>?

You're right, suEXEC is very difficult software to work with.  You
can't blame that on Mailman, though, and if you got suEXEC from your
distro, you should file a bug with them (probably on the Mailman
package).  If it's locally built by your Information Thuggery
department, they should get the "glory".

BTW, I disagree with Mark's wording on "different security models".  I
would say that the two security models are the same, except for who
keeps the keys to the bus.  Apache expects that suEXEC will be
driving, while Mailman is designed as the designated driver.  (Cue
"Three Stooges" schtick with Larry, Curly, and Moe all trying to get
into the driver's seat simultaneously.)

From dave at 3rdValve.net  Mon Nov 30 16:09:46 2015
From: dave at 3rdValve.net (Dave Arndt)
Date: Mon, 30 Nov 2015 16:09:46 -0500
Subject: [Mailman-Users] Text "Reappears" when MBox Archive Rebuilt
Message-ID: <CAAfcYTgZDJSD_be_epXeAfByDBugkN0gKHVpRt2Dyo17ZGAZTQ@mail.gmail.com>

This is an odd one. I'm hoping there's a straight forward answer:

1) I edited our mbox archive (with vi) to remove some offensive content.
(x, xx)

2) Saved the file.

3) Did a *grep* on the text that was removed - not found.

4) Uploaded the mbox file to a brand new server

5) Rebuilt the html archives using that mbox file.

The text that was deleted in step #1 appears in the newly generated HTML
archives!!!

How could this be possible?

Out of paranoia I went back to the mbox file that I uploaded, grep'd again
- and also loaded the file into vi and searched for any of the text that
was removed.  Not found.

Any ideas?  This seems really, really strange.

- Caught in a Parallel Universe

From mark at msapiro.net  Mon Nov 30 16:15:40 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 30 Nov 2015 13:15:40 -0800
Subject: [Mailman-Users] Text "Reappears" when MBox Archive Rebuilt
In-Reply-To: <CAAfcYTgZDJSD_be_epXeAfByDBugkN0gKHVpRt2Dyo17ZGAZTQ@mail.gmail.com>
References: <CAAfcYTgZDJSD_be_epXeAfByDBugkN0gKHVpRt2Dyo17ZGAZTQ@mail.gmail.com>
Message-ID: <565CBC7C.2080809@msapiro.net>

On 11/30/2015 01:09 PM, Dave Arndt wrote:
> This is an odd one. I'm hoping there's a straight forward answer:
> 
> 1) I edited our mbox archive (with vi) to remove some offensive content.
> (x, xx)
> 
> 2) Saved the file.
> 
> 3) Did a *grep* on the text that was removed - not found.
> 
> 4) Uploaded the mbox file to a brand new server
> 
> 5) Rebuilt the html archives using that mbox file.


Exactly what did you do at this step. If you used bin/arch did you use
the --wipe option. If not, you didn't remove anything from an existing
archive

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From dave at 3rdValve.net  Mon Nov 30 16:22:10 2015
From: dave at 3rdValve.net (Dave Arndt)
Date: Mon, 30 Nov 2015 16:22:10 -0500
Subject: [Mailman-Users] Text "Reappears" when MBox Archive Rebuilt
In-Reply-To: <565CBC7C.2080809@msapiro.net>
References: <CAAfcYTgZDJSD_be_epXeAfByDBugkN0gKHVpRt2Dyo17ZGAZTQ@mail.gmail.com>
 <565CBC7C.2080809@msapiro.net>
Message-ID: <CAAfcYTiK83TK4tfzsHWL5thP1wzk8h8Fu5JpQ=7155SRtynbcA@mail.gmail.com>

On Mon, Nov 30, 2015 at 4:15 PM, Mark Sapiro <mark at msapiro.net> wrote:

>
> Exactly what did you do at this step. If you used bin/arch did you use
> the --wipe option. If not, you didn't remove anything from an existing
> archive
>

Hi Mark,

Thanks for the speedy reply.  The steps were as outlined.  As for the last
step...

I did not do the rebuild myself - but I would assume they just ran bin/arch

How would the text re-appear if it was removed, as per step #1?

in other words, How would the text still be in the file after removing it,
and it doesn't appear with grep?

From mark at msapiro.net  Mon Nov 30 17:03:18 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 30 Nov 2015 14:03:18 -0800
Subject: [Mailman-Users] Text "Reappears" when MBox Archive Rebuilt
In-Reply-To: <CAAfcYTiK83TK4tfzsHWL5thP1wzk8h8Fu5JpQ=7155SRtynbcA@mail.gmail.com>
References: <CAAfcYTgZDJSD_be_epXeAfByDBugkN0gKHVpRt2Dyo17ZGAZTQ@mail.gmail.com>
 <565CBC7C.2080809@msapiro.net>
 <CAAfcYTiK83TK4tfzsHWL5thP1wzk8h8Fu5JpQ=7155SRtynbcA@mail.gmail.com>
Message-ID: <565CC7A6.5090200@msapiro.net>

On 11/30/2015 01:22 PM, Dave Arndt wrote:
> 
> I did not do the rebuild myself - but I would assume they just ran bin/arch
> 
> How would the text re-appear if it was removed, as per step #1?
> 
> in other words, How would the text still be in the file after removing
> it, and it doesn't appear with grep?


If you have an existing HTML archive and the corresponding .mbox, and
you run bin/arch without --wipe, every message in the mbox will be added
to the HTML archive, but they won't be indexed because the Message-IDs
are duplicates.

For example if there are a total of 10 messages in the archive, the HTML
messages will have names like 000000.html, 000001.html, ...,
000009.html. If you then run bin/arch without --wipe, you will add files
000010.html, 000011,html, ..., 000019.html which may or may not be a bit
different if you modified the mbox. Now, when the archiver added say
000010.html, its Message-ID is the same as that of 000000.html, so it
won't be indexed and the index will still point to 000000.html.

The answer is if you want to rebuild and archive and not just add to it,
you have to use --wipe to remove the existing HTML archive before adding.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From dave at 3rdValve.net  Mon Nov 30 17:37:15 2015
From: dave at 3rdValve.net (Dave Arndt)
Date: Mon, 30 Nov 2015 17:37:15 -0500
Subject: [Mailman-Users] Text "Reappears" when MBox Archive Rebuilt
In-Reply-To: <565CC7A6.5090200@msapiro.net>
References: <CAAfcYTgZDJSD_be_epXeAfByDBugkN0gKHVpRt2Dyo17ZGAZTQ@mail.gmail.com>
 <565CBC7C.2080809@msapiro.net>
 <CAAfcYTiK83TK4tfzsHWL5thP1wzk8h8Fu5JpQ=7155SRtynbcA@mail.gmail.com>
 <565CC7A6.5090200@msapiro.net>
Message-ID: <CAAfcYTjHv_KAtxgYf0RziHxUXSS+CuXGbx6Kh5YyHZvTUrpEyw@mail.gmail.com>

In this case, there are NO existing HTML files.  It's being rebuilt on a
brand new installation.

I also just heard from the person managing the site that they did use the
--wipe option (so I guess that's all moot).

The mystery is this:  How is it possible to edit out text from an mbox
file, verify that it is NOT there with grep, then see it reappear in the
resulting html file when bin/arch is run?

It's almost as if editing the file with VI left the original text and only
hid it with escape sequences or something.

Whatever it is... the mbox file that got uploaded to the new site HAD to
have had the original text, even though that text does not show up with
when running grep against that same mbox file (and is also not visible when
editing the same file with VI)...

Strange.






On Mon, Nov 30, 2015 at 5:03 PM, Mark Sapiro <mark at msapiro.net> wrote:

> On 11/30/2015 01:22 PM, Dave Arndt wrote:
> >
> > I did not do the rebuild myself - but I would assume they just ran
> bin/arch
> >
> > How would the text re-appear if it was removed, as per step #1?
> >
> > in other words, How would the text still be in the file after removing
> > it, and it doesn't appear with grep?
>
>
> If you have an existing HTML archive and the corresponding .mbox, and
> you run bin/arch without --wipe, every message in the mbox will be added
> to the HTML archive, but they won't be indexed because the Message-IDs
> are duplicates.
>
> For example if there are a total of 10 messages in the archive, the HTML
> messages will have names like 000000.html, 000001.html, ...,
> 000009.html. If you then run bin/arch without --wipe, you will add files
> 000010.html, 000011,html, ..., 000019.html which may or may not be a bit
> different if you modified the mbox. Now, when the archiver added say
> 000010.html, its Message-ID is the same as that of 000000.html, so it
> won't be indexed and the index will still point to 000000.html.
>
> The answer is if you want to rebuild and archive and not just add to it,
> you have to use --wipe to remove the existing HTML archive before adding.
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
>

From mark at msapiro.net  Mon Nov 30 17:51:39 2015
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 30 Nov 2015 14:51:39 -0800
Subject: [Mailman-Users] Text "Reappears" when MBox Archive Rebuilt
In-Reply-To: <CAAfcYTjHv_KAtxgYf0RziHxUXSS+CuXGbx6Kh5YyHZvTUrpEyw@mail.gmail.com>
References: <CAAfcYTgZDJSD_be_epXeAfByDBugkN0gKHVpRt2Dyo17ZGAZTQ@mail.gmail.com>
 <565CBC7C.2080809@msapiro.net>
 <CAAfcYTiK83TK4tfzsHWL5thP1wzk8h8Fu5JpQ=7155SRtynbcA@mail.gmail.com>
 <565CC7A6.5090200@msapiro.net>
 <CAAfcYTjHv_KAtxgYf0RziHxUXSS+CuXGbx6Kh5YyHZvTUrpEyw@mail.gmail.com>
Message-ID: <565CD2FB.1060804@msapiro.net>

On 11/30/2015 02:37 PM, Dave Arndt wrote:
> 
> The mystery is this:  How is it possible to edit out text from an mbox
> file, verify that it is NOT there with grep, then see it reappear in the
> resulting html file when bin/arch is run?
> 
> It's almost as if editing the file with VI left the original text and
> only hid it with escape sequences or something.


But vi doesn't do that.


> Whatever it is... the mbox file that got uploaded to the new site HAD to
> have had the original text, even though that text does not show up with
> when running grep against that same mbox file (and is also not visible
> when editing the same file with VI)... 
> 
> Strange.


Yes it's strange, and if everything is as you say, I can't explain it.

If you want to investigate further, create a new list on your local
installation and then run

bin/arch --wipe NEW_LISTNAME /path/to/edited/mbox

and see what appears in NEW_LISTNAME's archive. If the elided text is
not there, then I suggest that the archive wasn't built on the new
server with the same mbox. If the edlided text is there, it must be in
the mbox. berhaps grep doesn't find it because it is split across lines
or some other reason.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From LDGraham at iinet.net.au  Mon Nov 30 17:21:20 2015
From: LDGraham at iinet.net.au (Lindsay Graham)
Date: Tue, 01 Dec 2015 09:21:20 +1100
Subject: [Mailman-Users] List messages showing as attachments
Message-ID: <565CCBE0.4030506@iinet.net.au>

I hope this is the right place to ask this question -- if not, please 
direct me to the right place.

I'm a subscriber to 2 Mailman lists, Ger-Poland-Volhynia list at 
sggee.org and TSS list at surname-society.org.  I joined the former 
(with Individual messages) several years ago and it has always worked as 
expected.

The TSS list is new and, as a member of TSS (The Surname Society), I was 
subscribed automatically to Digest mode, but very soon switched to 
Individual mode.  All email messages that come from the list (both 
Digest and Individual) are blank except for the list footer and the real 
message is an attachment to that blank message.  I always have to open 
that attachment to read the message but to reply, I have to reply to the 
blank message.  It's my impression that very few other TSS list 
subscribers receive their messages in this way, and I've not so far been 
able to get any explanation form the TSS webmaster.

I've checked the Mailman subscription options for both lists and the 
settings appear to be identical.  My email client is Thunderbird 31.7 
(not quite the latest version) running under Windows 8.1 64-bit.  The 
problem with the TSS list exists whether I view the email in Thunderbird 
or my ISP's webmail.

If anybody has any clues why the TSS messages should appear in this way, 
I'd be very grateful for your help/advice.

Lindsay Graham
Canberra, Australia



From ottosonwebmaster at arlington.k12.ma.us  Mon Nov 30 17:48:00 2015
From: ottosonwebmaster at arlington.k12.ma.us (Ottoson Webmaster)
Date: Mon, 30 Nov 2015 17:48:00 -0500
Subject: [Mailman-Users] strange interaction between mailman and google docs
Message-ID: <CAHww1eX63wrS61xJzWnKO7up-pHDR-+huVMs7yjYZPBswyh77Q@mail.gmail.com>

I am seeing a strange problem and I am wondering if someone has any idea.
I am using google docs to create a newsletter which I then send to a
mailman mailing list.

Within google docs I use the feature "Email as attachment" to send to the
list.
The mailman list has about 1000 addresses some of which are gmail addresses
and some are not (yahoo, hotmail, etc.)

Well, the email from google docs is *only* received by the gmail addresses
in the mailing list but *not* by any of the other addresses.

If I send the email with the attachment from google docs to a *single*
non-google email, then I see no problems.
If I send the email with the attachment from gmail  (outside of google
docs) to the entire list I see no problems.

The problem is only *from google docs to non-google email in mailman list*

Ideas?

Thank you!
Antonella