From eggert.ehmke at berlin.de  Tue Jan  5 14:30:36 2016
From: eggert.ehmke at berlin.de (Eggert Ehmke)
Date: Tue, 05 Jan 2016 20:30:36 +0100
Subject: [Mailman-Users] transport-mailman and virtual-mailman not updated
 with new list
Message-ID: <3405446.DpzdzYycM5@wookie>

Hello,

I have added some new lists. The transport-mailman and virtual-mailman files in 
the data directory still only contain the old list. What must be done to 
update them? bin/genaliases does not do it.

Eggert

From mark at msapiro.net  Wed Jan  6 15:10:48 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 6 Jan 2016 12:10:48 -0800
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <3405446.DpzdzYycM5@wookie>
References: <3405446.DpzdzYycM5@wookie>
Message-ID: <568D74C8.5040608@msapiro.net>

On 01/05/2016 11:30 AM, Eggert Ehmke wrote:
> 
> I have added some new lists. The transport-mailman and virtual-mailman files in 
> the data directory still only contain the old list. What must be done to 
> update them? bin/genaliases does not do it.


I am guessing at one time you followed the FAQ at
<http://wiki.list.org/x/10715238>

You need to verify all of the following.

1) Mailman/mm_cfg.py includes
MTA = 'Postfix'
POSTFIX_MAP_CMD = '/path/to/data/virtual_to_transport'
POSTFIX_STYLE_VIRTUAL_DOMAINS = [...]

where '/path/to/data/virtual_to_transport' is the actual path to the
virtual_to_transport file ant the value of POSTFIX_STYLE_VIRTUAL_DOMAINS
is a list of the host names (email hosts) of the lists to which this
process applies. In particular, if the new lists are in a new domain,
that domain needs to be added to POSTFIX_STYLE_VIRTUAL_DOMAINS.

2) /path/to/data/virtual_to_transport exists and has content, group and
mode as in items 3) and 4) of the FAQ.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From eggert.ehmke at berlin.de  Wed Jan  6 15:55:38 2016
From: eggert.ehmke at berlin.de (Eggert Ehmke)
Date: Wed, 06 Jan 2016 21:55:38 +0100
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <568D74C8.5040608@msapiro.net>
References: <3405446.DpzdzYycM5@wookie> <568D74C8.5040608@msapiro.net>
Message-ID: <7413613.CiiH4N9rKV@wookie>

Hello Mark,

Am Mittwoch, 6. Januar 2016, 12:10:48 schrieb Mark Sapiro:
> On 01/05/2016 11:30 AM, Eggert Ehmke wrote:
> > I have added some new lists. The transport-mailman and virtual-mailman
> > files in the data directory still only contain the old list. What must be
> > done to update them? bin/genaliases does not do it.
> 
> I am guessing at one time you followed the FAQ at
> <http://wiki.list.org/x/10715238>

Yes

> You need to verify all of the following.
> 
> 1) Mailman/mm_cfg.py includes
> MTA = 'Postfix'
> POSTFIX_MAP_CMD = '/path/to/data/virtual_to_transport'
> POSTFIX_STYLE_VIRTUAL_DOMAINS = [...]

I have these entries in /etc/mailman/mm_cfg.py. Should I copy the file 
someplace else?

> where '/path/to/data/virtual_to_transport' is the actual path to the
> virtual_to_transport file ant the value of POSTFIX_STYLE_VIRTUAL_DOMAINS
> is a list of the host names (email hosts) of the lists to which this
> process applies. In particular, if the new lists are in a new domain,
> that domain needs to be added to POSTFIX_STYLE_VIRTUAL_DOMAINS.

Yes. All domains are on the same server where also the mailman installation 
runs.

> 2) /path/to/data/virtual_to_transport exists and has content, group and
> mode as in items 3) and 4) of the FAQ.

Yes. Any more hints?


From mark at msapiro.net  Wed Jan  6 16:28:12 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 6 Jan 2016 13:28:12 -0800
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <7413613.CiiH4N9rKV@wookie>
References: <3405446.DpzdzYycM5@wookie> <568D74C8.5040608@msapiro.net>
 <7413613.CiiH4N9rKV@wookie>
Message-ID: <568D86EC.2030707@msapiro.net>

On 01/06/2016 12:55 PM, Eggert Ehmke wrote:
> Hello Mark,
> 
> Am Mittwoch, 6. Januar 2016, 12:10:48 schrieb Mark Sapiro:

>> You need to verify all of the following.
>>
>> 1) Mailman/mm_cfg.py includes
>> MTA = 'Postfix'
>> POSTFIX_MAP_CMD = '/path/to/data/virtual_to_transport'
>> POSTFIX_STYLE_VIRTUAL_DOMAINS = [...]
> 
> I have these entries in /etc/mailman/mm_cfg.py. Should I copy the file 
> someplace else?


No. Presumably you have some package which puts things in various
places. You may see that /etc/mailman/mm_cfg.py is actually a symlink to
some Mailman/mm_cfg.py file or vice versa, but /etc/mailman/mm_cfg.py is
probably a correct path.


>> where '/path/to/data/virtual_to_transport' is the actual path to the
>> virtual_to_transport file ant the value of POSTFIX_STYLE_VIRTUAL_DOMAINS
>> is a list of the host names (email hosts) of the lists to which this
>> process applies. In particular, if the new lists are in a new domain,
>> that domain needs to be added to POSTFIX_STYLE_VIRTUAL_DOMAINS.
> 
> Yes. All domains are on the same server where also the mailman installation 
> runs.


It doesn't matter that they are on the same server/mailman installation.
What is important is that if in mailman's directory, you run

for list in `bin/list_lists -b`; do
 bin/dumpdb lists/$list/config.pck | grep host_name.:
done

That every listed host_name be in POSTFIX_STYLE_VIRTUAL_DOMAINS

Is that the case?

When you run bin/genaliases, is there any output to the terminal? If so,
what?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From eggert.ehmke at berlin.de  Wed Jan  6 17:24:30 2016
From: eggert.ehmke at berlin.de (Eggert Ehmke)
Date: Wed, 06 Jan 2016 23:24:30 +0100
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <568D86EC.2030707@msapiro.net>
References: <3405446.DpzdzYycM5@wookie> <7413613.CiiH4N9rKV@wookie>
 <568D86EC.2030707@msapiro.net>
Message-ID: <20749428.9t78aWb0DX@wookie>

Am Mittwoch, 6. Januar 2016, 13:28:12 schrieb Mark Sapiro:

> What is important is that if in mailman's directory, you run
> 
> for list in `bin/list_lists -b`; do
>  bin/dumpdb lists/$list/config.pck | grep host_name.:
> done
> That every listed host_name be in POSTFIX_STYLE_VIRTUAL_DOMAINS
> 
> Is that the case?

When I run this command, it first only showed me the domain of my first mailing 
list. I then discovered in the web gui, that the hostname for the new list was 
set to the domain of the old list. I changed that in the web gui (where is the 
corresponding entry in the mailman config?) After that run above command again, 
and now both domains are listed.

> When you run bin/genaliases, is there any output to the terminal? If so,
> what?

No output. After running genaliases, data/aliases and data/virtual-mailman 
have new time stamps. However, only data/aliases has all mailing lists, 
data/virtual-mailman has only the old list.

From mark at msapiro.net  Wed Jan  6 17:42:25 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 6 Jan 2016 14:42:25 -0800
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <20749428.9t78aWb0DX@wookie>
References: <3405446.DpzdzYycM5@wookie> <7413613.CiiH4N9rKV@wookie>
 <568D86EC.2030707@msapiro.net> <20749428.9t78aWb0DX@wookie>
Message-ID: <568D9851.1060609@msapiro.net>

On 01/06/2016 02:24 PM, Eggert Ehmke wrote:
> Am Mittwoch, 6. Januar 2016, 13:28:12 schrieb Mark Sapiro:
> 
>> What is important is that if in mailman's directory, you run
>>
>> for list in `bin/list_lists -b`; do
>>  bin/dumpdb lists/$list/config.pck | grep host_name.:
>> done
>> That every listed host_name be in POSTFIX_STYLE_VIRTUAL_DOMAINS
>>
>> Is that the case?
> 
> When I run this command, it first only showed me the domain of my first mailing 
> list. I then discovered in the web gui, that the hostname for the new list was 
> set to the domain of the old list. I changed that in the web gui (where is the 
> corresponding entry in the mailman config?) After that run above command again, 
> and now both domains are listed.


The web admin General Options setting

Host name this list prefers for email.
(Details for host_name)

Is the list's host_name attribute.

The output from the above command should be one line for each list of
the form

    'host_name': 'example.com',

It doesn't identify the list, but each entry corresponds to the
host_name in the web UI.


>> When you run bin/genaliases, is there any output to the terminal? If so,
>> what?
> 
> No output. After running genaliases, data/aliases and data/virtual-mailman 
> have new time stamps. However, only data/aliases has all mailing lists, 
> data/virtual-mailman has only the old list.


OK. Everything seems to be working correctly. It appears that the issue
is with POSTFIX_STYLE_VIRTUAL_DOMAINS in mm_cfg.py. In order for entries
to be generated in data/virtual-mailman (and data/transport-mailman) for
a list, that list's host_name must be in the
POSTFIX_STYLE_VIRTUAL_DOMAINS list in mm_cfg.py.

Is that the case?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From eggert.ehmke at berlin.de  Wed Jan  6 18:33:59 2016
From: eggert.ehmke at berlin.de (Eggert Ehmke)
Date: Thu, 07 Jan 2016 00:33:59 +0100
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <568D9851.1060609@msapiro.net>
References: <3405446.DpzdzYycM5@wookie> <20749428.9t78aWb0DX@wookie>
 <568D9851.1060609@msapiro.net>
Message-ID: <6210524.JK4zRcbA6F@wookie>

Am Mittwoch, 6. Januar 2016, 14:42:25 schrieb Mark Sapiro:
> On 01/06/2016 02:24 PM, Eggert Ehmke wrote:
> > Am Mittwoch, 6. Januar 2016, 13:28:12 schrieb Mark Sapiro:
> >> What is important is that if in mailman's directory, you run
> >> 
> >> for list in `bin/list_lists -b`; do
> >> 
> >>  bin/dumpdb lists/$list/config.pck | grep host_name.:
> >> done
> >> That every listed host_name be in POSTFIX_STYLE_VIRTUAL_DOMAINS
> >> 
> >> Is that the case?
> > 
> > When I run this command, it first only showed me the domain of my first
> > mailing list. I then discovered in the web gui, that the hostname for the
> > new list was set to the domain of the old list. I changed that in the web
> > gui (where is the corresponding entry in the mailman config?) After that
> > run above command again, and now both domains are listed.
> 
> The web admin General Options setting
> 
> Host name this list prefers for email.
> (Details for host_name)
> 
> Is the list's host_name attribute.
> 
> The output from the above command should be one line for each list of
> the form
> 
>     'host_name': 'example.com',
> 
> It doesn't identify the list, but each entry corresponds to the
> host_name in the web UI.
> 
> >> When you run bin/genaliases, is there any output to the terminal? If so,
> >> what?
> > 
> > No output. After running genaliases, data/aliases and data/virtual-mailman
> > have new time stamps. However, only data/aliases has all mailing lists,
> > data/virtual-mailman has only the old list.
> 
> OK. Everything seems to be working correctly. It appears that the issue
> is with POSTFIX_STYLE_VIRTUAL_DOMAINS in mm_cfg.py. In order for entries
> to be generated in data/virtual-mailman (and data/transport-mailman) for
> a list, that list's host_name must be in the
> POSTFIX_STYLE_VIRTUAL_DOMAINS list in mm_cfg.py.
> 
> Is that the case?

Yes. And data/virtual-mailman and data/transport-mailman still don't get the 
new list entries, even when they are updated after running genaliases.



From mark at msapiro.net  Wed Jan  6 18:41:42 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 6 Jan 2016 15:41:42 -0800
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <6210524.JK4zRcbA6F@wookie>
References: <3405446.DpzdzYycM5@wookie> <20749428.9t78aWb0DX@wookie>
 <568D9851.1060609@msapiro.net> <6210524.JK4zRcbA6F@wookie>
Message-ID: <568DA636.9070603@msapiro.net>

On 01/06/2016 03:33 PM, Eggert Ehmke wrote:
> Am Mittwoch, 6. Januar 2016, 14:42:25 schrieb Mark Sapiro:
>>
>> OK. Everything seems to be working correctly. It appears that the issue
>> is with POSTFIX_STYLE_VIRTUAL_DOMAINS in mm_cfg.py. In order for entries
>> to be generated in data/virtual-mailman (and data/transport-mailman) for
>> a list, that list's host_name must be in the
>> POSTFIX_STYLE_VIRTUAL_DOMAINS list in mm_cfg.py.
>>
>> Is that the case?
> 
> Yes. And data/virtual-mailman and data/transport-mailman still don't get the 
> new list entries, even when they are updated after running genaliases.


What exactly is the POSTFIX_STYLE_VIRTUAL_DOMAINS entry in mm_cfg.py and
what exactly is the output from

for list in `bin/list_lists -b`; do
 bin/dumpdb lists/$list/config.pck | grep host_name.:
done

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From eggert.ehmke at berlin.de  Wed Jan  6 18:56:33 2016
From: eggert.ehmke at berlin.de (Eggert Ehmke)
Date: Thu, 07 Jan 2016 00:56:33 +0100
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <568DA636.9070603@msapiro.net>
References: <3405446.DpzdzYycM5@wookie> <6210524.JK4zRcbA6F@wookie>
 <568DA636.9070603@msapiro.net>
Message-ID: <1458448.DKuyAGZ0b3@wookie>

Am Mittwoch, 6. Januar 2016, 15:41:42 schrieb Mark Sapiro:
> On 01/06/2016 03:33 PM, Eggert Ehmke wrote:
> > Am Mittwoch, 6. Januar 2016, 14:42:25 schrieb Mark Sapiro:
> >> OK. Everything seems to be working correctly. It appears that the issue
> >> is with POSTFIX_STYLE_VIRTUAL_DOMAINS in mm_cfg.py. In order for entries
> >> to be generated in data/virtual-mailman (and data/transport-mailman) for
> >> a list, that list's host_name must be in the
> >> POSTFIX_STYLE_VIRTUAL_DOMAINS list in mm_cfg.py.
> >> 
> >> Is that the case?
> > 
> > Yes. And data/virtual-mailman and data/transport-mailman still don't get
> > the new list entries, even when they are updated after running
> > genaliases.
> What exactly is the POSTFIX_STYLE_VIRTUAL_DOMAINS entry in mm_cfg.py and
> what exactly is the output from
> 
> for list in `bin/list_lists -b`; do
>  bin/dumpdb lists/$list/config.pck | grep host_name.:
> done

POSTFIX_STYLE_VIRTUAL_DOMAINS = ['domain1.de', 'domain2.de']

# for list in `bin/list_lists -b`; do  bin/dumpdb lists/$list/config.pck | grep 
host_name.:; done
    'host_name': 'lists.domain2.de',
    'host_name': 'lists.domain1.de',
    'host_name': 'vxxxxxxxxxxxx.yourvserver.net',
    'host_name': 'domain1.de',
    'host_name': 'lists.domain1.de',

Where domain1 is the domain of the old list, domain2 the domain of the new 
list. Vxxxxxx is the hostname that my provider has assigned to the server.



From mark at msapiro.net  Wed Jan  6 19:23:52 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 6 Jan 2016 16:23:52 -0800
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <1458448.DKuyAGZ0b3@wookie>
References: <3405446.DpzdzYycM5@wookie> <6210524.JK4zRcbA6F@wookie>
 <568DA636.9070603@msapiro.net> <1458448.DKuyAGZ0b3@wookie>
Message-ID: <568DB018.8090105@msapiro.net>

On 01/06/2016 03:56 PM, Eggert Ehmke wrote:
> 
> POSTFIX_STYLE_VIRTUAL_DOMAINS = ['domain1.de', 'domain2.de']
> 
> # for list in `bin/list_lists -b`; do  bin/dumpdb lists/$list/config.pck | grep 
> host_name.:; done
>     'host_name': 'lists.domain2.de',
>     'host_name': 'lists.domain1.de',
>     'host_name': 'vxxxxxxxxxxxx.yourvserver.net',
>     'host_name': 'domain1.de',
>     'host_name': 'lists.domain1.de',
> 
> Where domain1 is the domain of the old list, domain2 the domain of the new 
> list. Vxxxxxx is the hostname that my provider has assigned to the server.


The names in POSTFIX_STYLE_VIRTUAL_DOMAINS must exactly match the
host_names. I.e. for the above, the only list that will get
data/virtual-mailman and data/transport-mailman entries is the one with
host_name = 'domain1.de'.

Since host_name is intended to be the email domain of the list, not the
web domain, I'm guessing that the host_name values for the three lists
that currently begin with 'lists.' should actually be just 'domain1.de'
or 'domain2.de', but if you really mail to these lists
@lists.domainx.de, you need to add 'lists.domain2.de' and
'lists.domain1.de' to POSTFIX_STYLE_VIRTUAL_DOMAINS.

Also, if 'vxxxxxxxxxxxx.yourvserver.net' is a virtual domain as opposed
to a Postfix mydestination (local) domain, that needs to be in
POSTFIX_STYLE_VIRTUAL_DOMAINS too.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From eisaksso at gmail.com  Thu Jan  7 07:40:26 2016
From: eisaksso at gmail.com (Eva Isaksson)
Date: Thu, 7 Jan 2016 14:40:26 +0200
Subject: [Mailman-Users] Encoding problem with 2.15 to 2.18 upgrade with
 Finnish
In-Reply-To: <568C1994.7050403@msapiro.net>
References: <CAEMzDVPAOw=jOK6NpruVh0N5UzyVGDhiLuS4TLOjDSzAzxt_Yg@mail.gmail.com>
 <566F15FD.5060102@msapiro.net>
 <CAEMzDVPXa+6Y83s5jZBmGGmF8d46Bogua_BeqtTqdkcngSm7HA@mail.gmail.com>
 <568C0E8A.5030907@msapiro.net> <568C1994.7050403@msapiro.net>
Message-ID: <CAEMzDVPsj_cK_qSDQWqoqHLyc_Gix=554oV0uS45tBBYAujkrw@mail.gmail.com>

I tried to approach this problem by getting the source package from the
debian repository.
We are still running wheezy, so we need to stick to 2.1.18 for now.

When I built the package, the resulting mailman.mo file for Finnish ends up
utf-8 encoded
(which I deduced by converting it back to a .po file by msgunfmt.
The encoding seems to happen in debian/patches/91_utf8.patch
which does have a list of correct encodings, but this is what seems to
happen:
(not only to Finnish, but to all listed languages):

msgconv -t utf-8 fi/LC_MESSAGES/mailman.po | tee
fi/LC_MESSAGES/mailman.po.utf-8 | msgfmt -o fi/LC_MESSAGES/mailman.mo -

Also:
http://sources.debian.net/patches/patch/mailman/1:2.1.18-2/91_utf8.patch/
<https://mail.google.com/>

If this means that debian is forcing everything to utf-8, what should I do,
if every package comes with this forced utf-8 encoding?

Here is the trackback we got in October:

Oct 28 10:04:28 2015 (22144) Uncaught runner exception: 'utf8' codec
can't decode byte 0xe4 in position 11: invalid continuation byt
e
Oct 28 10:04:28 2015 (22144) Traceback (most recent call last):
  File "/var/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop
    self._onefile(msg, msgdata)
  File "/var/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/var/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in
_dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
  File "/var/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in
_dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
  File "/var/lib/mailman/Mailman/Handlers/CookHeaders.py", line 239, in
process
    i18ndesc = uheader(mlist, mlist.description, 'List-Id', maxlinelen=998)
  File "/var/lib/mailman/Mailman/Handlers/CookHeaders.py", line 65, in
uheader
    return Header(s, charset, maxlinelen, header_name, continuation_ws)
  File "/usr/lib/python2.7/email/header.py", line 183, in __init__
    self.append(s, charset, errors)
  File "/usr/lib/python2.7/email/header.py", line 267, in append
    ustr = unicode(s, incodec, errors)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe4 in position 11:
invalid continuation byte


- Eva Isaksson




2016-01-05 21:29 GMT+02:00 Mark Sapiro <mark at msapiro.net>:

> On 01/05/2016 10:42 AM, Mark Sapiro wrote:
> > On 01/05/2016 04:43 AM, Eva Isaksson wrote:
> >>
> >> This is what we see after installing the .deb:
> >>
> >> ```Uncaught runner exception: 'utf8' codec can't decode byte 0xe4 in
> >> position 11: invalid continuation byte```
> >
> >
> > Please post the entire traceback from Mailman's error log.
> >
> > Please keep this thread on the list.
>
>
> Although I can't say for sure without seeing the traceback, this may be
> similar to the situation described at
> <https://bugs.launchpad.net/mailman/+bug/1462755>.
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
>

From eggert.ehmke at berlin.de  Thu Jan  7 11:05:34 2016
From: eggert.ehmke at berlin.de (Eggert Ehmke)
Date: Thu, 07 Jan 2016 17:05:34 +0100
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <568DB018.8090105@msapiro.net>
References: <3405446.DpzdzYycM5@wookie> <1458448.DKuyAGZ0b3@wookie>
 <568DB018.8090105@msapiro.net>
Message-ID: <1800110.WseoqeGRRj@wookie>

Am Mittwoch, 6. Januar 2016, 16:23:52 schrieb Mark Sapiro:
> On 01/06/2016 03:56 PM, Eggert Ehmke wrote:
> > POSTFIX_STYLE_VIRTUAL_DOMAINS = ['domain1.de', 'domain2.de']
> > 
> > # for list in `bin/list_lists -b`; do  bin/dumpdb lists/$list/config.pck |
> > grep host_name.:; done
> > 
> >     'host_name': 'lists.domain2.de',
> >     'host_name': 'lists.domain1.de',
> >     'host_name': 'vxxxxxxxxxxxx.yourvserver.net',
> >     'host_name': 'domain1.de',
> >     'host_name': 'lists.domain1.de',
> > 
> > Where domain1 is the domain of the old list, domain2 the domain of the new
> > list. Vxxxxxx is the hostname that my provider has assigned to the server.
> 
> The names in POSTFIX_STYLE_VIRTUAL_DOMAINS must exactly match the
> host_names. I.e. for the above, the only list that will get
> data/virtual-mailman and data/transport-mailman entries is the one with
> host_name = 'domain1.de'.

That is indeed so.

> Since host_name is intended to be the email domain of the list, not the
> web domain, I'm guessing that the host_name values for the three lists
> that currently begin with 'lists.' should actually be just 'domain1.de'
> or 'domain2.de', but if you really mail to these lists
> @lists.domainx.de, you need to add 'lists.domain2.de' and
> 'lists.domain1.de' to POSTFIX_STYLE_VIRTUAL_DOMAINS.
> 
> Also, if 'vxxxxxxxxxxxx.yourvserver.net' is a virtual domain as opposed
> to a Postfix mydestination (local) domain, that needs to be in
> POSTFIX_STYLE_VIRTUAL_DOMAINS too.

The Vxxxxx is not a virtual domain and will not be used.

Ok I changed /etc/mailman/mm_cfg.py and removed the "lists" from the 
hostnames. It lookes like this now:
DEFAULT_EMAIL_HOST = 'domain1.de'
DEFAULT_URL_HOST   = 'domain1.de'
add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)
add_virtualhost('domain2.de', 'domain2.de')

Now the above command produces this list:
    'host_name': 'domain2.de',
    'host_name': 'lists.domain1.de',
    'host_name': 'vxxxxxx.yourvserver.net',
    'host_name': 'domain1.de',
    'host_name': 'lists.domain1.de',

When running genaliases again, 
data/aliases contains all lists in both domains, db is updated
data/transport-mailman contains only one list of both domains, 
data/virtual-mailman contains only one list of both domains, db is not updated

Anyway, the most importand list is now there. I run postmap on the virtual-
mailman manually. Mailinglist still not working, that may be a postfix problem. 






From cwieland at uci.edu  Thu Jan  7 14:32:47 2016
From: cwieland at uci.edu (Con Wieland)
Date: Thu, 7 Jan 2016 11:32:47 -0800
Subject: [Mailman-Users] remove pending subscribers
Message-ID: <385520B4-5544-48A7-982F-2FC1D25327B9@uci.edu>

It appears my pending subscriptions file is corrupt.  I have several ?very? long bogus addresses and the only buttons on the admindb page is to ?Submit All Data? The individual buttons to accept, defer, ?. are not there.

I tried to remove the subscription requests with:

https://mail.python.org/pipermail/mailman-users/2011-October/072336.html

but when I run the dumpdb script on the pending requests:

./dumpdb ~mailman/lists/socaliug/pending.pc

alll I get is:

[----- start pickle file -----]
<----- start object 1 ----->
{   '44c0eed8b26bce3337d68fb198f549c78446a1c7': ('H', 1461),
    'evictions': {   '44c0eed8b26bce3337d68fb198f549c78446a1c7': 1402604941.357022},
    'version': 2}
[----- end pickle file ??]

even though there are 9 pending requests 2 of which are just very long strings

Con Wieland
UC Irvine

From mark at msapiro.net  Thu Jan  7 14:44:52 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 7 Jan 2016 11:44:52 -0800
Subject: [Mailman-Users] Encoding problem with 2.15 to 2.18 upgrade with
 Finnish
In-Reply-To: <CAEMzDVPsj_cK_qSDQWqoqHLyc_Gix=554oV0uS45tBBYAujkrw@mail.gmail.com>
References: <CAEMzDVPAOw=jOK6NpruVh0N5UzyVGDhiLuS4TLOjDSzAzxt_Yg@mail.gmail.com>
 <566F15FD.5060102@msapiro.net>
 <CAEMzDVPXa+6Y83s5jZBmGGmF8d46Bogua_BeqtTqdkcngSm7HA@mail.gmail.com>
 <568C0E8A.5030907@msapiro.net> <568C1994.7050403@msapiro.net>
 <CAEMzDVPsj_cK_qSDQWqoqHLyc_Gix=554oV0uS45tBBYAujkrw@mail.gmail.com>
Message-ID: <568EC034.7070706@msapiro.net>

On 01/07/2016 04:40 AM, Eva Isaksson wrote:
> 
> Also:
> http://sources.debian.net/patches/patch/mailman/1:2.1.18-2/91_utf8.patch/
> <https://mail.google.com/>
> 
> If this means that debian is forcing everything to utf-8, what should I do,
> if every package comes with this forced utf-8 encoding?


This is a Debian issue. The FAQ at <http://wiki.list.org/x/12812344>
addresses that, but probably won't be much comfort to you.

You could install Mailman from source. See the FAQ at
<http://wiki.list.org/x/17891606> for info.

More below ...


> Here is the trackback we got in October:
> 
> Oct 28 10:04:28 2015 (22144) Uncaught runner exception: 'utf8' codec
> can't decode byte 0xe4 in position 11: invalid continuation byt
> e
> Oct 28 10:04:28 2015 (22144) Traceback (most recent call last):
>   File "/var/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop
>     self._onefile(msg, msgdata)
>   File "/var/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile
>     keepqueued = self._dispose(mlist, msg, msgdata)
>   File "/var/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in
> _dispose
>     more = self._dopipeline(mlist, msg, msgdata, pipeline)
>   File "/var/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in
> _dopipeline
>     sys.modules[modname].process(mlist, msg, msgdata)
>   File "/var/lib/mailman/Mailman/Handlers/CookHeaders.py", line 239, in
> process
>     i18ndesc = uheader(mlist, mlist.description, 'List-Id', maxlinelen=998)
>   File "/var/lib/mailman/Mailman/Handlers/CookHeaders.py", line 65, in
> uheader
>     return Header(s, charset, maxlinelen, header_name, continuation_ws)
>   File "/usr/lib/python2.7/email/header.py", line 183, in __init__
>     self.append(s, charset, errors)
>   File "/usr/lib/python2.7/email/header.py", line 267, in append
>     ustr = unicode(s, incodec, errors)
> UnicodeDecodeError: 'utf8' codec can't decode byte 0xe4 in position 11:
> invalid continuation byte


This is exactly the issue described at
<https://bugs.launchpad.net/mailman/+bug/1462755>. Note that the person
who reported that is Thijs Kinkhorst who is the Debian maintainer of
Mailman, so they are well aware of this issue yet they continue to
distribute a package which causes this issue for their users. See in
particular <https://bugs.launchpad.net/mailman/+bug/1462755/comments/3>.
Had I realized when I wrote that that it was Debian and not the user
that had changed the encoding for the language, that reply would have
been much stronger.

The problem in your case is that even though the character set for
Finnish in your Mailman is UTF-8, your list(s) still have strings in
their attributes which are ISO-8859-1 encoded. In this particular case
the list's General Options -> description contains one or more
characters encoded in ISO-8859-1.

I think simply re-entering the description and saving changes will fix
this, but you should probably go through the entire web admin UI and do
the same for any other non-ascii strings.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Jan  7 15:02:48 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 7 Jan 2016 12:02:48 -0800
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <1800110.WseoqeGRRj@wookie>
References: <3405446.DpzdzYycM5@wookie> <1458448.DKuyAGZ0b3@wookie>
 <568DB018.8090105@msapiro.net> <1800110.WseoqeGRRj@wookie>
Message-ID: <568EC468.3050005@msapiro.net>

On 01/07/2016 08:05 AM, Eggert Ehmke wrote:

> 
> Ok I changed /etc/mailman/mm_cfg.py and removed the "lists" from the 
> hostnames. It lookes like this now:
> DEFAULT_EMAIL_HOST = 'domain1.de'
> DEFAULT_URL_HOST   = 'domain1.de'
> add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)
> add_virtualhost('domain2.de', 'domain2.de')


Actually, what you probably want is

DEFAULT_EMAIL_HOST = 'domain1.de'
DEFAULT_URL_HOST   = 'lists.domain1.de'
add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)
add_virtualhost('lists.domain2.de', 'domain2.de')

Because at least in a normal Debian/Ubuntu package web URLs are
http://lists.domainx.de/... and list mailing addresses are list at domainx.de.

> Now the above command produces this list:
>     'host_name': 'domain2.de',
>     'host_name': 'lists.domain1.de',
>     'host_name': 'vxxxxxx.yourvserver.net',
>     'host_name': 'domain1.de',
>     'host_name': 'lists.domain1.de',
> 
> When running genaliases again, 
> data/aliases contains all lists in both domains, db is updated
> data/transport-mailman contains only one list of both domains, 
> data/virtual-mailman contains only one list of both domains, db is not updated
> 
> Anyway, the most importand list is now there. I run postmap on the virtual-
> mailman manually. Mailinglist still not working, that may be a postfix problem. 


The changes you made to mm_cfg.py do not affect the host_name for any
existing lists. I suggest you do the following:

1) make the changes I suggest above in mm_cfg.py

2) for each list in domain1 run

bin/withlist -l -r fix_url LISTNAME

and for each list in domain2 run

bin/withlist -l -r fix_url LISTNAME -u lists.domain2.de

This will fix the list's web_page_url and host_name attributes
appropriately.

3) run bin/genaliases

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Jan  7 15:12:37 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 7 Jan 2016 12:12:37 -0800
Subject: [Mailman-Users] Encoding problem with 2.15 to 2.18 upgrade with
 Finnish
In-Reply-To: <568EC034.7070706@msapiro.net>
References: <CAEMzDVPAOw=jOK6NpruVh0N5UzyVGDhiLuS4TLOjDSzAzxt_Yg@mail.gmail.com>
 <566F15FD.5060102@msapiro.net>
 <CAEMzDVPXa+6Y83s5jZBmGGmF8d46Bogua_BeqtTqdkcngSm7HA@mail.gmail.com>
 <568C0E8A.5030907@msapiro.net> <568C1994.7050403@msapiro.net>
 <CAEMzDVPsj_cK_qSDQWqoqHLyc_Gix=554oV0uS45tBBYAujkrw@mail.gmail.com>
 <568EC034.7070706@msapiro.net>
Message-ID: <568EC6B5.6060507@msapiro.net>

On 01/07/2016 11:44 AM, Mark Sapiro wrote:
> 
> You could install Mailman from source. See the FAQ at
> <http://wiki.list.org/x/17891606> for info.


Just to be clear, by install from source I mean our source at e.g.
<https://launchpad.net/mailman/+download>, not some packager's source
package.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Jan  7 15:36:29 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 7 Jan 2016 12:36:29 -0800
Subject: [Mailman-Users] remove pending subscribers
In-Reply-To: <385520B4-5544-48A7-982F-2FC1D25327B9@uci.edu>
References: <385520B4-5544-48A7-982F-2FC1D25327B9@uci.edu>
Message-ID: <568ECC4D.4050504@msapiro.net>

On 01/07/2016 11:32 AM, Con Wieland wrote:
> It appears my pending subscriptions file is corrupt.  I have several ?very? long bogus addresses and the only buttons on the admindb page is to ?Submit All Data? The individual buttons to accept, defer, ?. are not there.


Have you tried scrolling the display left and right? The long addresses
make the page wide.

If this is not the issue, please send me off list an intact copy (before
you do anything else) of ~mailman/lists/socaliug/request.pck


> I tried to remove the subscription requests with:
> 
> https://mail.python.org/pipermail/mailman-users/2011-October/072336.html
> 
> but when I run the dumpdb script on the pending requests:
> 
> ./dumpdb ~mailman/lists/socaliug/pending.pc
> 
> alll I get is:
> 
> [----- start pickle file -----]
> <----- start object 1 ----->
> {   '44c0eed8b26bce3337d68fb198f549c78446a1c7': ('H', 1461),
>     'evictions': {   '44c0eed8b26bce3337d68fb198f549c78446a1c7': 1402604941.357022},
>     'version': 2}
> [----- end pickle file ??]
> 
> even though there are 9 pending requests 2 of which are just very long strings


That post is incorrect. It should reference
/path/to/mailman/lists/LISTNAME/request.pck, not
/path/to/mailman/lists/LISTNAME/pending.pck.

However, it is also obsolete. See the script at
<https://www.msapiro.net/scripts/list_requests>. This script will list
all the outstanding moderator requests for a list or lists and has an
option to accept an "action" to be applied to each request as it is listed.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From eisaksso at gmail.com  Thu Jan  7 15:40:20 2016
From: eisaksso at gmail.com (Eva Isaksson)
Date: Thu, 7 Jan 2016 22:40:20 +0200
Subject: [Mailman-Users] Encoding problem with 2.15 to 2.18 upgrade with
 Finnish
In-Reply-To: <568EC6B5.6060507@msapiro.net>
References: <CAEMzDVPAOw=jOK6NpruVh0N5UzyVGDhiLuS4TLOjDSzAzxt_Yg@mail.gmail.com>
 <566F15FD.5060102@msapiro.net>
 <CAEMzDVPXa+6Y83s5jZBmGGmF8d46Bogua_BeqtTqdkcngSm7HA@mail.gmail.com>
 <568C0E8A.5030907@msapiro.net> <568C1994.7050403@msapiro.net>
 <CAEMzDVPsj_cK_qSDQWqoqHLyc_Gix=554oV0uS45tBBYAujkrw@mail.gmail.com>
 <568EC034.7070706@msapiro.net> <568EC6B5.6060507@msapiro.net>
Message-ID: <CAEMzDVPGnXVuN3zNNFaknm_aW9LdMuAvJft3SOThj57Wwv0KhQ@mail.gmail.com>

2016-01-07 22:12 GMT+02:00 Mark Sapiro <mark at msapiro.net>:
On 01/07/2016 11:44 AM, Mark Sapiro wrote:

> I think simply re-entering the description and saving changes will fix
> this, but you should probably go through the entire web admin UI and do
> the same for any other non-ascii strings.

Am I right in thinking that this option would mean choosing the debian utf-8
alternative? We have some 750 lists, so checking all of them would have to
happen outside the web admin UI, i.e. from command line.

> You could install Mailman from source. See the FAQ at
> <http://wiki.list.org/x/17891606> for info.

Does this allow for keeping all the settings and passwords from the old
package
installation intact? The number of lists is so big that I want to be on the
safe
side.

- Eva Isaksson

From mark at msapiro.net  Thu Jan  7 16:20:03 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 7 Jan 2016 13:20:03 -0800
Subject: [Mailman-Users] Encoding problem with 2.15 to 2.18 upgrade with
 Finnish
In-Reply-To: <CAEMzDVPGnXVuN3zNNFaknm_aW9LdMuAvJft3SOThj57Wwv0KhQ@mail.gmail.com>
References: <CAEMzDVPAOw=jOK6NpruVh0N5UzyVGDhiLuS4TLOjDSzAzxt_Yg@mail.gmail.com>
 <566F15FD.5060102@msapiro.net>
 <CAEMzDVPXa+6Y83s5jZBmGGmF8d46Bogua_BeqtTqdkcngSm7HA@mail.gmail.com>
 <568C0E8A.5030907@msapiro.net> <568C1994.7050403@msapiro.net>
 <CAEMzDVPsj_cK_qSDQWqoqHLyc_Gix=554oV0uS45tBBYAujkrw@mail.gmail.com>
 <568EC034.7070706@msapiro.net> <568EC6B5.6060507@msapiro.net>
 <CAEMzDVPGnXVuN3zNNFaknm_aW9LdMuAvJft3SOThj57Wwv0KhQ@mail.gmail.com>
Message-ID: <568ED683.5010300@msapiro.net>

On 01/07/2016 12:40 PM, Eva Isaksson wrote:
> 2016-01-07 22:12 GMT+02:00 Mark Sapiro <mark at msapiro.net
> <mailto:mark at msapiro.net>>:
> On 01/07/2016 11:44 AM, Mark Sapiro wrote:
> 
>> I think simply re-entering the description and saving changes will fix
>> this, but you should probably go through the entire web admin UI and do
>> the same for any other non-ascii strings.
> 
> Am I right in thinking that this option would mean choosing the debian utf-8
> alternative? We have some 750 lists, so checking all of them would have to
> happen outside the web admin UI, i.e. from command line.


You can keep the Debian package with it's utf-8 character sets and
programmatically change all your lists so they work. In MM 2.1.19 I
changed the character sets for Russian and Romanian from koi8-r and
iso-8859-2 respectively to utf-8 because I was convinced that the former
character sets were not appropriate for those languages. In so doing, I
augmented Mailman's version updater to identify strings in list
attributes that were not valid utf-8 and convert them from the former
character set.

This is an imperfect process. It does guarantee that all the strings in
list attributes are valid utf-8 encodings, but not necessarily that they
are 'correct'.

I am working on a stand-alone script to do this. I will post it later today.


>> You could install Mailman from source. See the FAQ at
>> <http://wiki.list.org/x/17891606> for info.
> 
> Does this allow for keeping all the settings and passwords from the old
> package
> installation intact? The number of lists is so big that I want to be on
> the safe
> side.


Yes. Installing the latest GNU Mailman source over the Debian package
following <http://wiki.list.org/x/17891606> will not change any list
data, but there may be other issues depending on what if any Debian
patches you choose to install.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From bsfinkel at att.net  Thu Jan  7 16:39:59 2016
From: bsfinkel at att.net (Barry S. Finkel)
Date: Thu, 7 Jan 2016 15:39:59 -0600
Subject: [Mailman-Users] Encoding problem with 2.15 to 2.18 upgrade with
 Finnish
In-Reply-To: <CAEMzDVPGnXVuN3zNNFaknm_aW9LdMuAvJft3SOThj57Wwv0KhQ@mail.gmail.com>
References: <CAEMzDVPAOw=jOK6NpruVh0N5UzyVGDhiLuS4TLOjDSzAzxt_Yg@mail.gmail.com>
 <566F15FD.5060102@msapiro.net>
 <CAEMzDVPXa+6Y83s5jZBmGGmF8d46Bogua_BeqtTqdkcngSm7HA@mail.gmail.com>
 <568C0E8A.5030907@msapiro.net> <568C1994.7050403@msapiro.net>
 <CAEMzDVPsj_cK_qSDQWqoqHLyc_Gix=554oV0uS45tBBYAujkrw@mail.gmail.com>
 <568EC034.7070706@msapiro.net> <568EC6B5.6060507@msapiro.net>
 <CAEMzDVPGnXVuN3zNNFaknm_aW9LdMuAvJft3SOThj57Wwv0KhQ@mail.gmail.com>
Message-ID: <568EDB2F.5000505@att.net>

On 1/7/2016 2:40 PM, Eva Isaksson wrote:
> 2016-01-07 22:12 GMT+02:00 Mark Sapiro <mark at msapiro.net>:
> On 01/07/2016 11:44 AM, Mark Sapiro wrote:
>
>> I think simply re-entering the description and saving changes will fix
>> this, but you should probably go through the entire web admin UI and do
>> the same for any other non-ascii strings.
>
> Am I right in thinking that this option would mean choosing the debian utf-8
> alternative? We have some 750 lists, so checking all of them would have to
> happen outside the web admin UI, i.e. from command line.
>
>> You could install Mailman from source. See the FAQ at
>> <http://wiki.list.org/x/17891606> for info.
>
> Does this allow for keeping all the settings and passwords from the old
> package
> installation intact? The number of lists is so big that I want to be on the
> safe
> side.
>
> - Eva Isaksson


I have posted this before, and I will post it again.  When I was an
administrator for a Mailman system running on Ubuntu, I decided
that the Debian package had problems.  Besides having patches that
were undocumented, there was one patch that reomoved a library that
sometimes was needed.  So I figured out how to create a Mailman
package from the SourceForge source.  The last Mailman package
I built was 2.1.15; I assume that the same process would work
for the latest Mailman.  What I did is available to anyone who
wants it; send me a private e-mail to get the output from
"script" of the session when I built the last package.  The only
Debian patch I kept was one that placed libraries in the proper
directories for Debian/Ubuntu.

--Barry Finkel


From mark at msapiro.net  Thu Jan  7 19:55:09 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 7 Jan 2016 16:55:09 -0800
Subject: [Mailman-Users] Encoding problem with 2.15 to 2.18 upgrade with
 Finnish
In-Reply-To: <568ED683.5010300@msapiro.net>
References: <CAEMzDVPAOw=jOK6NpruVh0N5UzyVGDhiLuS4TLOjDSzAzxt_Yg@mail.gmail.com>
 <566F15FD.5060102@msapiro.net>
 <CAEMzDVPXa+6Y83s5jZBmGGmF8d46Bogua_BeqtTqdkcngSm7HA@mail.gmail.com>
 <568C0E8A.5030907@msapiro.net> <568C1994.7050403@msapiro.net>
 <CAEMzDVPsj_cK_qSDQWqoqHLyc_Gix=554oV0uS45tBBYAujkrw@mail.gmail.com>
 <568EC034.7070706@msapiro.net> <568EC6B5.6060507@msapiro.net>
 <CAEMzDVPGnXVuN3zNNFaknm_aW9LdMuAvJft3SOThj57Wwv0KhQ@mail.gmail.com>
 <568ED683.5010300@msapiro.net>
Message-ID: <568F08ED.60609@msapiro.net>

On 01/07/2016 01:20 PM, Mark Sapiro wrote:
> 
> You can keep the Debian package with it's utf-8 character sets and
> programmatically change all your lists so they work. In MM 2.1.19 I
> changed the character sets for Russian and Romanian from koi8-r and
> iso-8859-2 respectively to utf-8 because I was convinced that the former
> character sets were not appropriate for those languages. In so doing, I
> augmented Mailman's version updater to identify strings in list
> attributes that were not valid utf-8 and convert them from the former
> character set.
> 
> This is an imperfect process. It does guarantee that all the strings in
> list attributes are valid utf-8 encodings, but not necessarily that they
> are 'correct'.
> 
> I am working on a stand-alone script to do this. I will post it later today.


There is now a first cut at this script. The functions that do the
recoding are lifted straight from Mailman/versions.py so they have been
fairly well exercised. I didn't just import them because I wanted the
caller to know if they had changed anything and that required a minor
addition to the top level function to return a flag.

The script is at <https://www.msapiro.net/scripts/recode_list> (mirrored
at <http://fog.ccsf.edu/~msapiro/scripts/recode_list>)

Install the script in Mailman's bin/ directory and run it with the -h
option for full info. It uses Python's argparse function and as such
requires python 2.7.

In your case, to convert all Finnish preferred_language lists strings
from iso-8859-1 encoding to utf-8

recode_list --language=fi

would suffice. There are options to specify other than iso-8859-1 to
utf-8 and to produce output about skipped or unchanged lists, but the
defaults should be good for you and will report which lists were
changed. In no case does it report the actual changes, just what lists
were changed.

I recommend making backups of your lists/*/config.pck files before
running this in case something goes wrong.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From eggert.ehmke at berlin.de  Fri Jan  8 08:05:49 2016
From: eggert.ehmke at berlin.de (Eggert Ehmke)
Date: Fri, 08 Jan 2016 14:05:49 +0100
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <568EC468.3050005@msapiro.net>
References: <3405446.DpzdzYycM5@wookie> <1800110.WseoqeGRRj@wookie>
 <568EC468.3050005@msapiro.net>
Message-ID: <16507882.Y2xJ67ms3h@wookie>

Hello Mark,

thank you, that looks much better now! The alias, transport and virtual tables 
are now correct.  The problem that remains may be postfix or dovecot related, 
but perhaps you have an idea too:

When I send a mail to list at domain2.de, it is forwarded to list at domain1.de, 
which does not work of course.

Jan  8 13:56:05 domain1 postfix/pipe[2066]: 9EE1C1003F1: to=<list at domain1.de>, 
orig_to=<list at domain2.de>, relay=dovecot, delay=0.03, delays=0.02/0/0/0.02, 
dsn=5.1.1, status=bounced (user unknown)

Eggert


Am Donnerstag, 7. Januar 2016, 12:02:48 schrieb Mark Sapiro:
> On 01/07/2016 08:05 AM, Eggert Ehmke wrote:
> > Ok I changed /etc/mailman/mm_cfg.py and removed the "lists" from the
> > hostnames. It lookes like this now:
> > DEFAULT_EMAIL_HOST = 'domain1.de'
> > DEFAULT_URL_HOST   = 'domain1.de'
> > add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)
> > add_virtualhost('domain2.de', 'domain2.de')
> 
> Actually, what you probably want is
> 
> DEFAULT_EMAIL_HOST = 'domain1.de'
> DEFAULT_URL_HOST   = 'lists.domain1.de'
> add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)
> add_virtualhost('lists.domain2.de', 'domain2.de')
> 
> Because at least in a normal Debian/Ubuntu package web URLs are
> http://lists.domainx.de/... and list mailing addresses are list at domainx.de.
> 
> > Now the above command produces this list:
> >     'host_name': 'domain2.de',
> >     'host_name': 'lists.domain1.de',
> >     'host_name': 'vxxxxxx.yourvserver.net',
> >     'host_name': 'domain1.de',
> >     'host_name': 'lists.domain1.de',
> > 
> > When running genaliases again,
> > data/aliases contains all lists in both domains, db is updated
> > data/transport-mailman contains only one list of both domains,
> > data/virtual-mailman contains only one list of both domains, db is not
> > updated
> > 
> > Anyway, the most importand list is now there. I run postmap on the
> > virtual-
> > mailman manually. Mailinglist still not working, that may be a postfix
> > problem.
> The changes you made to mm_cfg.py do not affect the host_name for any
> existing lists. I suggest you do the following:
> 
> 1) make the changes I suggest above in mm_cfg.py
> 
> 2) for each list in domain1 run
> 
> bin/withlist -l -r fix_url LISTNAME
> 
> and for each list in domain2 run
> 
> bin/withlist -l -r fix_url LISTNAME -u lists.domain2.de
> 
> This will fix the list's web_page_url and host_name attributes
> appropriately.
> 
> 3) run bin/genaliases

From eggert.ehmke at berlin.de  Fri Jan  8 11:42:04 2016
From: eggert.ehmke at berlin.de (Eggert Ehmke)
Date: Fri, 08 Jan 2016 17:42:04 +0100
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <16507882.Y2xJ67ms3h@wookie>
References: <3405446.DpzdzYycM5@wookie> <568EC468.3050005@msapiro.net>
 <16507882.Y2xJ67ms3h@wookie>
Message-ID: <1568209.V5CsxmUuzM@wookie>

Am Freitag, 8. Januar 2016, 14:05:49 schrieb Eggert Ehmke:
> Hello Mark,
> 
> thank you, that looks much better now! The alias, transport and virtual
> tables are now correct.  The problem that remains may be postfix or dovecot
> related, but perhaps you have an idea too:
> 
> When I send a mail to list at domain2.de, it is forwarded to list at domain1.de,
> which does not work of course.
> 
> Jan  8 13:56:05 domain1 postfix/pipe[2066]: 9EE1C1003F1:
> to=<list at domain1.de>, orig_to=<list at domain2.de>, relay=dovecot, delay=0.03,
> delays=0.02/0/0/0.02, dsn=5.1.1, status=bounced (user unknown)

I think this is because in the virtual_mailman all list addresses are mapped 
to the naked list name, and postfix then appends the default domain. I thought 
this would be prevented by the mailman/bin/postfix-to-mailman.py script, but it 
does not work. I also found a comment of you stating you don't see the reason 
for this script. However, what can be done to make it work?

> 
> Eggert
> 
> Am Donnerstag, 7. Januar 2016, 12:02:48 schrieb Mark Sapiro:
> > On 01/07/2016 08:05 AM, Eggert Ehmke wrote:
> > > Ok I changed /etc/mailman/mm_cfg.py and removed the "lists" from the
> > > hostnames. It lookes like this now:
> > > DEFAULT_EMAIL_HOST = 'domain1.de'
> > > DEFAULT_URL_HOST   = 'domain1.de'
> > > add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)
> > > add_virtualhost('domain2.de', 'domain2.de')
> > 
> > Actually, what you probably want is
> > 
> > DEFAULT_EMAIL_HOST = 'domain1.de'
> > DEFAULT_URL_HOST   = 'lists.domain1.de'
> > add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)
> > add_virtualhost('lists.domain2.de', 'domain2.de')
> > 
> > Because at least in a normal Debian/Ubuntu package web URLs are
> > http://lists.domainx.de/... and list mailing addresses are
> > list at domainx.de.
> > 
> > > Now the above command produces this list:
> > >     'host_name': 'domain2.de',
> > >     'host_name': 'lists.domain1.de',
> > >     'host_name': 'vxxxxxx.yourvserver.net',
> > >     'host_name': 'domain1.de',
> > >     'host_name': 'lists.domain1.de',
> > > 
> > > When running genaliases again,
> > > data/aliases contains all lists in both domains, db is updated
> > > data/transport-mailman contains only one list of both domains,
> > > data/virtual-mailman contains only one list of both domains, db is not
> > > updated
> > > 
> > > Anyway, the most importand list is now there. I run postmap on the
> > > virtual-
> > > mailman manually. Mailinglist still not working, that may be a postfix
> > > problem.
> > 
> > The changes you made to mm_cfg.py do not affect the host_name for any
> > existing lists. I suggest you do the following:
> > 
> > 1) make the changes I suggest above in mm_cfg.py
> > 
> > 2) for each list in domain1 run
> > 
> > bin/withlist -l -r fix_url LISTNAME
> > 
> > and for each list in domain2 run
> > 
> > bin/withlist -l -r fix_url LISTNAME -u lists.domain2.de
> > 
> > This will fix the list's web_page_url and host_name attributes
> > appropriately.
> > 
> > 3) run bin/genaliases
> 
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/eggert.ehmke%40berlin
> .de


From mark at msapiro.net  Fri Jan  8 13:08:04 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 8 Jan 2016 10:08:04 -0800
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <1568209.V5CsxmUuzM@wookie>
References: <3405446.DpzdzYycM5@wookie> <568EC468.3050005@msapiro.net>
 <16507882.Y2xJ67ms3h@wookie> <1568209.V5CsxmUuzM@wookie>
Message-ID: <568FFB04.9090701@msapiro.net>

On 01/08/2016 08:42 AM, Eggert Ehmke wrote:
> Am Freitag, 8. Januar 2016, 14:05:49 schrieb Eggert Ehmke:
>>
>> When I send a mail to list at domain2.de, it is forwarded to list at domain1.de,
>> which does not work of course.
>>
>> Jan  8 13:56:05 domain1 postfix/pipe[2066]: 9EE1C1003F1:
>> to=<list at domain1.de>, orig_to=<list at domain2.de>, relay=dovecot, delay=0.03,
>> delays=0.02/0/0/0.02, dsn=5.1.1, status=bounced (user unknown)


Something is wrong here because if everything is properly configured per
the FAQ at <http://wiki.list.org/x/10715238> the transport (relay)
should be 'local' not 'dovecot'
.

> I think this is because in the virtual_mailman all list addresses are mapped 
> to the naked list name, and postfix then appends the default domain.


Again, if things are properly configured per the FAQ at
<http://wiki.list.org/x/10715238>, virtual-mailman is only used to build
transport-mailman and only aliases and transport-mailman are used by
postfix for Mailman mail.


> I thought 
> this would be prevented by the mailman/bin/postfix-to-mailman.py script, but it 
> does not work.


postfix-to-mailman.py is incompatible with this and definitely won't
work well in a multi-domain environment. See the FAQ at
<http://wiki.list.org/x/15564817>.


> I also found a comment of you stating you don't see the reason 
> for this script. However, what can be done to make it work?


You probably can't make it work in your environment.

For further help, please post the output from 'postconf -n' and the
contents of /etc/postfix/master.cf.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From eggert.ehmke at berlin.de  Fri Jan  8 14:34:43 2016
From: eggert.ehmke at berlin.de (Eggert Ehmke)
Date: Fri, 08 Jan 2016 20:34:43 +0100
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <568FFB04.9090701@msapiro.net>
References: <3405446.DpzdzYycM5@wookie> <1568209.V5CsxmUuzM@wookie>
 <568FFB04.9090701@msapiro.net>
Message-ID: <4166212.6XQ5Nomdn5@wookie>

Hello Mark,

finally it works! The clue was that the 
VIRTUAL_MAILMAN_LOCAL_DOMAIN = 'localhost'
was missing in mm_cfg.py. After that the mail was delivered locally, and 
mailman could send it to the list members. The /usr/lib/mailman/bin/postfix-to-
mailman.py line in master.cf is commented out, it indeed works fine following 
your hints.

Thank you again a lot for your patience! You will agree there are a lot of 
documents on the net leading in the wrong direction.

Eggert


Am Freitag, 8. Januar 2016, 10:08:04 schrieb Mark Sapiro:
> On 01/08/2016 08:42 AM, Eggert Ehmke wrote:
> > Am Freitag, 8. Januar 2016, 14:05:49 schrieb Eggert Ehmke:
> >> When I send a mail to list at domain2.de, it is forwarded to
> >> list at domain1.de,
> >> which does not work of course.
> >> 
> >> Jan  8 13:56:05 domain1 postfix/pipe[2066]: 9EE1C1003F1:
> >> to=<list at domain1.de>, orig_to=<list at domain2.de>, relay=dovecot,
> >> delay=0.03,
> >> delays=0.02/0/0/0.02, dsn=5.1.1, status=bounced (user unknown)
> 
> Something is wrong here because if everything is properly configured per
> the FAQ at <http://wiki.list.org/x/10715238> the transport (relay)
> should be 'local' not 'dovecot'
> .
> 
> > I think this is because in the virtual_mailman all list addresses are
> > mapped to the naked list name, and postfix then appends the default
> > domain.
> Again, if things are properly configured per the FAQ at
> <http://wiki.list.org/x/10715238>, virtual-mailman is only used to build
> transport-mailman and only aliases and transport-mailman are used by
> postfix for Mailman mail.
> 
> > I thought
> > this would be prevented by the mailman/bin/postfix-to-mailman.py script,
> > but it does not work.
> 
> postfix-to-mailman.py is incompatible with this and definitely won't
> work well in a multi-domain environment. See the FAQ at
> <http://wiki.list.org/x/15564817>.
> 
> > I also found a comment of you stating you don't see the reason
> > for this script. However, what can be done to make it work?
> 
> You probably can't make it work in your environment.
> 
> For further help, please post the output from 'postconf -n' and the
> contents of /etc/postfix/master.cf.

From mark at msapiro.net  Fri Jan  8 15:00:24 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 8 Jan 2016 12:00:24 -0800
Subject: [Mailman-Users] transport-mailman and virtual-mailman not
 updated with new list
In-Reply-To: <4166212.6XQ5Nomdn5@wookie>
References: <3405446.DpzdzYycM5@wookie> <1568209.V5CsxmUuzM@wookie>
 <568FFB04.9090701@msapiro.net> <4166212.6XQ5Nomdn5@wookie>
Message-ID: <56901558.1000500@msapiro.net>

On 01/08/2016 11:34 AM, Eggert Ehmke wrote:
> 
> finally it works! The clue was that the 
> VIRTUAL_MAILMAN_LOCAL_DOMAIN = 'localhost'
> was missing in mm_cfg.py. After that the mail was delivered locally, and 
> mailman could send it to the list members. The /usr/lib/mailman/bin/postfix-to-
> mailman.py line in master.cf is commented out, it indeed works fine following 
> your hints.


I'm glad it's working, however something is still not right with your
implementation of the things in the FAQ at
<http://wiki.list.org/x/10715238>. If you have implemented the things in
that FAQ completely and correctly, the setting of
VIRTUAL_MAILMAN_LOCAL_DOMAIN wouldn't change anything because appending
@localhost to the destinations in virtual-mailman wouldn't affect the
content of transport-mailman and further POSTFIX_MAP_CMD would not
update virtual-mailman.db.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From dap1 at bellsouth.net  Sat Jan  9 09:14:42 2016
From: dap1 at bellsouth.net (Dennis Putnam)
Date: Sat, 9 Jan 2016 09:14:42 -0500
Subject: [Mailman-Users] Reply-To-List Header
Message-ID: <569115D2.4000603@bellsouth.net>

On some mailman lists I receive messages in which Thunderbird displays a
"Reply to List" button. However, on my own lists that does not happen.
What header is TB relying on to get that and what setting do I need to
change my mailman for that to happen? TIA.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160109/47063ca6/attachment.sig>

From mark at msapiro.net  Sat Jan  9 13:36:37 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 9 Jan 2016 10:36:37 -0800
Subject: [Mailman-Users] Reply-To-List Header
In-Reply-To: <569115D2.4000603@bellsouth.net>
References: <569115D2.4000603@bellsouth.net>
Message-ID: <56915335.2000202@msapiro.net>

On 01/09/2016 06:14 AM, Dennis Putnam wrote:
> On some mailman lists I receive messages in which Thunderbird displays a
> "Reply to List" button. However, on my own lists that does not happen.
> What header is TB relying on to get that and what setting do I need to
> change my mailman for that to happen? TIA.


Thunderbird and many other MUAs rely on the RFC 2369 List-Post: header
for "reply to list". There are six RFC 2369 List-* headers; List-Id:,
List-Unsubscribe:, List-Archive:, List-Post:, List-Help: and
List-Subscribe:. Mailman's adding of these headers is controlled by the
lists General Options include_rfc2369_headers and
include_list_post_header settings.

If include_rfc2369_headers is No, none of these headers will be added.
If include_rfc2369_headers is Yes, List-Post: will be added only if
include_list_post_header is Yes, List-Archive will be added only if the
list is archiving and the other four will be added.

So to ensure posts received from the list have a List-Post header, both
include_rfc2369_headers and include_list_post_header must be Yes.

Also note that if your "Avoid duplicate copies of messages?" setting on
a list is Yes and you are directly addressed in a reply, the copy you
receive doesn't come from the list and won't have any Mailman headers.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160109/20f2f407/attachment.sig>

From dap1 at bellsouth.net  Sat Jan  9 16:10:17 2016
From: dap1 at bellsouth.net (Dennis Putnam)
Date: Sat, 9 Jan 2016 16:10:17 -0500
Subject: [Mailman-Users] Reply-To-List Header
In-Reply-To: <56915335.2000202@msapiro.net>
References: <569115D2.4000603@bellsouth.net> <56915335.2000202@msapiro.net>
Message-ID: <56917739.8060207@bellsouth.net>

On 1/9/2016 1:36 PM, Mark Sapiro wrote:
> On 01/09/2016 06:14 AM, Dennis Putnam wrote:
>> On some mailman lists I receive messages in which Thunderbird displays a
>> "Reply to List" button. However, on my own lists that does not happen.
>> What header is TB relying on to get that and what setting do I need to
>> change my mailman for that to happen? TIA.
>
> Thunderbird and many other MUAs rely on the RFC 2369 List-Post: header
> for "reply to list". There are six RFC 2369 List-* headers; List-Id:,
> List-Unsubscribe:, List-Archive:, List-Post:, List-Help: and
> List-Subscribe:. Mailman's adding of these headers is controlled by the
> lists General Options include_rfc2369_headers and
> include_list_post_header settings.
>
> If include_rfc2369_headers is No, none of these headers will be added.
> If include_rfc2369_headers is Yes, List-Post: will be added only if
> include_list_post_header is Yes, List-Archive will be added only if the
> list is archiving and the other four will be added.
>
> So to ensure posts received from the list have a List-Post header, both
> include_rfc2369_headers and include_list_post_header must be Yes.
>
> Also note that if your "Avoid duplicate copies of messages?" setting on
> a list is Yes and you are directly addressed in a reply, the copy you
> receive doesn't come from the list and won't have any Mailman headers.
>
>
Thanks for the reply. Those options are and apparently always have been
on. But none of those headers show up. I wonder if an SMTP server
somewhere along the way is stripping them out for some reason.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160109/5e561338/attachment.sig>

From mark at msapiro.net  Sat Jan  9 16:43:35 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 9 Jan 2016 13:43:35 -0800
Subject: [Mailman-Users] Reply-To-List Header
In-Reply-To: <56917739.8060207@bellsouth.net>
References: <569115D2.4000603@bellsouth.net> <56915335.2000202@msapiro.net>
 <56917739.8060207@bellsouth.net>
Message-ID: <56917F07.9020502@msapiro.net>

On 01/09/2016 01:10 PM, Dennis Putnam wrote:
>>
> Thanks for the reply. Those options are and apparently always have been
> on. But none of those headers show up. I wonder if an SMTP server
> somewhere along the way is stripping them out for some reason.


Are there other Mailman headers such as X-BeenThere: and
X-Mailman-Version: in the messages?

Do you have any local modifications to Mailman/Handlers/CookHeaders.py?

How do messages get from Mailman to you?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160109/314054be/attachment.sig>

From dap1 at bellsouth.net  Sun Jan 10 09:29:16 2016
From: dap1 at bellsouth.net (Dennis Putnam)
Date: Sun, 10 Jan 2016 09:29:16 -0500
Subject: [Mailman-Users] Reply-To-List Header
In-Reply-To: <56917F07.9020502@msapiro.net>
References: <569115D2.4000603@bellsouth.net> <56915335.2000202@msapiro.net>
 <56917739.8060207@bellsouth.net> <56917F07.9020502@msapiro.net>
Message-ID: <56926ABC.1060603@bellsouth.net>

On 1/9/2016 4:43 PM, Mark Sapiro wrote:
> On 01/09/2016 01:10 PM, Dennis Putnam wrote:
>> Thanks for the reply. Those options are and apparently always have been
>> on. But none of those headers show up. I wonder if an SMTP server
>> somewhere along the way is stripping them out for some reason.
>
> Are there other Mailman headers such as X-BeenThere: and
> X-Mailman-Version: in the messages?
>
> Do you have any local modifications to Mailman/Handlers/CookHeaders.py?
>
> How do messages get from Mailman to you?
>
>
Thanks for the reply. Unfortunately I don't recall but I don't think so.
I am currently unable to test it to see as imap.gmail.com seems to be
down and has been for a while. The messages go from mailman on my
Postfix server to JetMail, to the subscriber's ISP (AT&T in my case).

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160110/8267d321/attachment.sig>

From mark at msapiro.net  Sun Jan 10 13:57:26 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 10 Jan 2016 10:57:26 -0800
Subject: [Mailman-Users] Reply-To-List Header
In-Reply-To: <56926ABC.1060603@bellsouth.net>
References: <569115D2.4000603@bellsouth.net> <56915335.2000202@msapiro.net>
 <56917739.8060207@bellsouth.net> <56917F07.9020502@msapiro.net>
 <56926ABC.1060603@bellsouth.net>
Message-ID: <5692A996.2030707@msapiro.net>

On 01/10/2016 06:29 AM, Dennis Putnam wrote:
> On 1/9/2016 4:43 PM, Mark Sapiro wrote:
>>
>> Are there other Mailman headers such as X-BeenThere: and
>> X-Mailman-Version: in the messages?
>>
>> Do you have any local modifications to Mailman/Handlers/CookHeaders.py?
>>
>> How do messages get from Mailman to you?
>>
>>
> Thanks for the reply. Unfortunately I don't recall but I don't think so.
> I am currently unable to test it to see as imap.gmail.com seems to be
> down and has been for a while.


I use imap.googlemail.com for my gmail imap server, but that won't make
any difference. I've just tested, and it appears that gmail/googlemail
is now forcing SSL. I.e., neither imap.gmail.com nor imap.googlemail.com
is answering port 143 connects, but both work with SSL on port 993.


> The messages go from mailman on my
> Postfix server to JetMail, to the subscriber's ISP (AT&T in my case).


The next time you receive a post from the list in question, view the
source (More -> View Source in Thunderbird). If you don't see the List-*
headers, copy ALL of the headers and post them here. You can munge any
personal info if you like.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160110/d61bd9c7/attachment.sig>

From dap1 at bellsouth.net  Sun Jan 10 18:07:30 2016
From: dap1 at bellsouth.net (Dennis Putnam)
Date: Sun, 10 Jan 2016 18:07:30 -0500
Subject: [Mailman-Users] Reply-To-List Header
In-Reply-To: <5692A996.2030707@msapiro.net>
References: <569115D2.4000603@bellsouth.net> <56915335.2000202@msapiro.net>
 <56917739.8060207@bellsouth.net> <56917F07.9020502@msapiro.net>
 <56926ABC.1060603@bellsouth.net> <5692A996.2030707@msapiro.net>
Message-ID: <5692E432.4070500@bellsouth.net>

On 1/10/2016 1:57 PM, Mark Sapiro wrote:
> On 01/10/2016 06:29 AM, Dennis Putnam wrote:
>> On 1/9/2016 4:43 PM, Mark Sapiro wrote:
>>> Are there other Mailman headers such as X-BeenThere: and
>>> X-Mailman-Version: in the messages?
>>>
>>> Do you have any local modifications to Mailman/Handlers/CookHeaders.py?
>>>
>>> How do messages get from Mailman to you?
>>>
>>>
>> Thanks for the reply. Unfortunately I don't recall but I don't think so.
>> I am currently unable to test it to see as imap.gmail.com seems to be
>> down and has been for a while.
>
> I use imap.googlemail.com for my gmail imap server, but that won't make
> any difference. I've just tested, and it appears that gmail/googlemail
> is now forcing SSL. I.e., neither imap.gmail.com nor imap.googlemail.com
> is answering port 143 connects, but both work with SSL on port 993.
>
>
>> The messages go from mailman on my
>> Postfix server to JetMail, to the subscriber's ISP (AT&T in my case).
>
> The next time you receive a post from the list in question, view the
> source (More -> View Source in Thunderbird). If you don't see the List-*
> headers, copy ALL of the headers and post them here. You can munge any
> personal info if you like.
>
>
Google is back up. I guess I was wrong in my original assessment. The
headers are indeed there including List-Post. The reply to list button
is displayed (I don't know why I sometimes don't get that but I will
have to look closer next time it happens). Anyway, I now see the
problem, but I don't know how/where to fix it. The address in the
List-Post is wrong. I can see how to turn it off/on in the documentation
but not how to customize it?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160110/5744b1ea/attachment.sig>

From mark at msapiro.net  Sun Jan 10 18:34:47 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 10 Jan 2016 15:34:47 -0800
Subject: [Mailman-Users] Reply-To-List Header
In-Reply-To: <5692E432.4070500@bellsouth.net>
References: <569115D2.4000603@bellsouth.net> <56915335.2000202@msapiro.net>
 <56917739.8060207@bellsouth.net> <56917F07.9020502@msapiro.net>
 <56926ABC.1060603@bellsouth.net> <5692A996.2030707@msapiro.net>
 <5692E432.4070500@bellsouth.net>
Message-ID: <5692EA97.3040506@msapiro.net>

On 01/10/2016 03:07 PM, Dennis Putnam wrote:
>>
> Google is back up. I guess I was wrong in my original assessment. The
> headers are indeed there including List-Post. The reply to list button
> is displayed (I don't know why I sometimes don't get that but I will
> have to look closer next time it happens). Anyway, I now see the
> problem, but I don't know how/where to fix it. The address in the
> List-Post is wrong. I can see how to turn it off/on in the documentation
> but not how to customize it?


It is the list posting address, i.e. listname at host_name, the same as the
address in "To post a message to all the list members, send email to
..." on the list's listinfo page.

If the local part in the List-Post: header is not the list name,
something is seriously wrong, but I don't know what. If the domain is
not what you think it should be, the list's host_name (on the General
Options page) is wrong. You can change it there, although that may
indicate issues with add_virtualhost and DEFAULT_*_HOST in mm_cfg.py.

As I said before, if you sometimes don't see it, it might be because you
are looking at a reply sent to you directly (To: or Cc:) and not from
the list.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160110/bf1ee6ff/attachment.sig>

From greg at headingup.net  Mon Jan 11 13:47:59 2016
From: greg at headingup.net (Greg Sims)
Date: Mon, 11 Jan 2016 10:47:59 -0800
Subject: [Mailman-Users] Daily Restarts Needed
Message-ID: <CAOXbGWFGUsv2K-y_H_15ectSMrp7ERJnFqjCAZP2zUhHo=FCdA@mail.gmail.com>

We are having problems with mailman needing to be restarted to complete
sending email to our lists.  The problem has been getting worse over time
and is now almost a daily requirement.  I spent some time debugging this
morning and found some issues.

I used top to look at the number of active processes that are owned by user
mailman.  I found 35 processes including 14 command "mailmanctl".  This
looks like it might be the source (at least one of them) to our problems.
Mailman is currently delivering email to our users after a restart.

I would like to clear this up without rebooting our server.  Please give me
some input on a plan to clean this up.  Here is a start:

   - wait for the need for mailman to be low -- we send email three times
   per day to newsletters & and have no forums/blogs
   - service mailman stop
   - delete all processes running under mailman
   - ?? should I clear lock files or other actions ??
   - service mailman start
   - verify the proper number of process are running -- ?? there should be
   eight processes ??

I would also like input on additional debug techniques to find the root
cause of this issue.

Thanks, Greg

From mark at msapiro.net  Mon Jan 11 20:29:26 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 11 Jan 2016 17:29:26 -0800
Subject: [Mailman-Users] Daily Restarts Needed
In-Reply-To: <CAOXbGWFGUsv2K-y_H_15ectSMrp7ERJnFqjCAZP2zUhHo=FCdA@mail.gmail.com>
References: <CAOXbGWFGUsv2K-y_H_15ectSMrp7ERJnFqjCAZP2zUhHo=FCdA@mail.gmail.com>
Message-ID: <569456F6.1080101@msapiro.net>

On 01/11/2016 10:47 AM, Greg Sims wrote:
> We are having problems with mailman needing to be restarted to complete
> sending email to our lists.  The problem has been getting worse over time
> and is now almost a daily requirement.  I spent some time debugging this
> morning and found some issues.
> 
> I used top to look at the number of active processes that are owned by user
> mailman.  I found 35 processes including 14 command "mailmanctl".  This
> looks like it might be the source (at least one of them) to our problems.
> Mailman is currently delivering email to our users after a restart.


This is a big problem. See the FAQ at <http://wiki.list.org/x/4030715>
for instructions on stopping Mailman completely and ythen start it only
once.

Note that several packages and even our source provide an init.d script
that runs mailmanctl with the -s option, however versions prior to
2.1.16 could start additional instances when this option was used. See
<https://bugs.launchpad.net/mailman/+bug/1189558>.


> I would like to clear this up without rebooting our server.  Please give me
> some input on a plan to clean this up.  Here is a start:
> 
>    - wait for the need for mailman to be low -- we send email three times
>    per day to newsletters & and have no forums/blogs
>    - service mailman stop
>    - delete all processes running under mailman
>    - ?? should I clear lock files or other actions ??
>    - service mailman start
>    - verify the proper number of process are running -- ?? there should be
>    eight processes ??


Your plan looks good. Once Mailman and all its qrunners are stopped,
there should be lothing in Mailman's locks/ directory. If there are any
files there at that point, remove them.

There should be nine processes, eight qrunners and mailmanctl as in

> mark at sbh16:~$ ps -fwwU mailman
> UID        PID  PPID  C STIME TTY          TIME CMD
> mailman  20219     1  0 Jan09 ?        00:00:00 /usr/bin/python /usr/local/mailman/bin/mailmanctl -s -q start
> mailman  20221 20219  0 Jan09 ?        00:00:14 /usr/bin/python /usr/local/mailman/bin/qrunner --runner=ArchRunner:0:1 -s
> mailman  20222 20219  0 Jan09 ?        00:00:14 /usr/bin/python /usr/local/mailman/bin/qrunner --runner=BounceRunner:0:1 -s
> mailman  20223 20219  0 Jan09 ?        00:00:13 /usr/bin/python /usr/local/mailman/bin/qrunner --runner=CommandRunner:0:1 -s
> mailman  20224 20219  0 Jan09 ?        00:00:13 /usr/bin/python /usr/local/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s
> mailman  20225 20219  0 Jan09 ?        00:00:13 /usr/bin/python /usr/local/mailman/bin/qrunner --runner=NewsRunner:0:1 -s
> mailman  20226 20219  0 Jan09 ?        00:00:22 /usr/bin/python /usr/local/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s
> mailman  20227 20219  0 Jan09 ?        00:00:18 /usr/bin/python /usr/local/mailman/bin/qrunner --runner=VirginRunner:0:1 -s
> mailman  20228 20219  0 Jan09 ?        00:00:00 /usr/bin/python /usr/local/mailman/bin/qrunner --runner=RetryRunner:0:1 -s
> mark at sbh16:~$ 


> I would also like input on additional debug techniques to find the root
> cause of this issue.


Start by looking at Mailman's error log. Also look at Mailman's qrunner
smtp and smtp-failure logs.

If the problem recurs it's almost certainly OutgoingRunner. At that
point, look at Mailman's qfiles/out/ directory.

If OutgoingRunner is 'stuck' there will be one file in that queue with a
.bak extension (at least in MM versions since 2.1.9). Other files if any
will have .pck extension. The file with the .bak extension is the one
being processed. You can look at it with 'bin/dumpdb -p', but it would
probably look OK.

If OutgoingRunner seems stuck, sending it a SIGHUP may unstick it
(SIGHUP normally says reopen your log files). Also look at locks (see
<http://wiki.list.org/x/17891756> for info on lock files).

If it is stuck, there are ways to get a stack trace from it, but I think
they normally require some modification to the code before it starts.

There is another possibility that the hung process is IncomingRunner. In
this case, the out/ queue would be empty and the in/ queue would have
the one .bak file. The rest of the above applies.

My guess though is OutgoingRunner is stuck. You stop Mailman and it
mostly stops, but not OutgoingRunner. You then start it again and this
starts a new OutgoingRunner which sees the old one 'crashed' and left
the .bak behind and starts by processing that. Do this a few times and
you have a number of stuck OutgoingRunner processes hanging around.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From advax at triumf.ca  Tue Jan 12 04:18:23 2016
From: advax at triumf.ca (Andrew Daviel)
Date: Tue, 12 Jan 2016 01:18:23 -0800 (PST)
Subject: [Mailman-Users] Handling bogus subscribe requests
Message-ID: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>


In the last few days we've seen several thousand bogus subscription 
requests for various lists we host, send through the web interface. They 
seem to mostly originate in China.

We see log entries such as /var/log/mailman/subscribe
Jan 11 20:50:30 2016 (27666) grsi-users: pending hellocatboots+80339132 at gmail.com  221.178.182.31
and in the webserver logs
221.178.182.31 - - [10/Jan/2016:03:27:18 -0800] "POST /mailman/subscribe/grsi-users HTTP/1.1" 200

I'm not sure what the point is - a DoS attack on a few users, perhaps. I 
see that gmail gives you infinite aliases, so that hellocatboots+80339132 
is the same as hellocatboots+96529823 at gmail.com

Since most of these seem to originate with one netblock where we have, I 
believe, no legitimate users, I've added a Deny rule in httpd.conf.

I was wondering if other admins had seen this, and if there was a better 
way to control it than blocking an ip range.

Apart from all the variants of hellocatboots, we've seen a lot of posts 
for one unique user at kezukaya.com. The subscribe log shows hundreds of 
pending requests, from which I infer that mailman has no mechanism to 
track the fact that it already sent a "please confirm" message (we have 
mailman-2.1.18 on Centos 5).

-- 
Andrew Daviel, TRIUMF, Canada

From rosenbaumlm at ornl.gov  Tue Jan 12 11:18:59 2016
From: rosenbaumlm at ornl.gov (Rosenbaum, Larry M.)
Date: Tue, 12 Jan 2016 16:18:59 +0000
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
Message-ID: <BN3PR09MB054854A9A49C1A8215B0BF3DCACA0@BN3PR09MB0548.namprd09.prod.outlook.com>

>From the "NEWS" file:

    - There is a new mm_cfg.py setting SUBSCRIBE_FORM_SECRET which will put
      a dynamically generated, hidden hash in the listinfo subscribe form and
      check it upon submission.  Setting this will prevent automated processes
      (bots) from successfully POSTing web subscribes without first retrieving
      and parsing the form from the listinfo page.  The form must also be
      submitted no later than FORM_LIFETIME nor no earlier than
      SUBSCRIBE_FORM_MIN_TIME after retrieval.  Note that enabling this will
      break any static subscribe forms on your site.  See the description in
      Defaults.py for more info.  (LP: #1082746)

> -----Original Message-----
> From: Mailman-Users [mailto:mailman-users-
> bounces+rosenbaumlm=ornl.gov at python.org] On Behalf Of Andrew Daviel
> Sent: Tuesday, January 12, 2016 4:18 AM
> To: mailman-users at python.org
> Subject: [Mailman-Users] Handling bogus subscribe requests
> 
> 
> In the last few days we've seen several thousand bogus subscription
> requests for various lists we host, send through the web interface. They
> seem to mostly originate in China.
> 
> We see log entries such as /var/log/mailman/subscribe
> Jan 11 20:50:30 2016 (27666) grsi-users: pending
> hellocatboots+80339132 at gmail.com  221.178.182.31
> and in the webserver logs
> 221.178.182.31 - - [10/Jan/2016:03:27:18 -0800] "POST
> /mailman/subscribe/grsi-users HTTP/1.1" 200
> 
> I'm not sure what the point is - a DoS attack on a few users, perhaps. I
> see that gmail gives you infinite aliases, so that hellocatboots+80339132
> is the same as hellocatboots+96529823 at gmail.com
> 
> Since most of these seem to originate with one netblock where we have, I
> believe, no legitimate users, I've added a Deny rule in httpd.conf.
> 
> I was wondering if other admins had seen this, and if there was a better
> way to control it than blocking an ip range.
> 
> Apart from all the variants of hellocatboots, we've seen a lot of posts
> for one unique user at kezukaya.com. The subscribe log shows hundreds of
> pending requests, from which I infer that mailman has no mechanism to
> track the fact that it already sent a "please confirm" message (we have
> mailman-2.1.18 on Centos 5).
> 
> --
> Andrew Daviel, TRIUMF, Canada
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/mailman-
> users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-
> users/rosenbaumlm%40ornl.gov


From mark at msapiro.net  Tue Jan 12 12:54:06 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 12 Jan 2016 09:54:06 -0800
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
Message-ID: <56953DBE.5060708@msapiro.net>

On 01/12/2016 01:18 AM, Andrew Daviel wrote:
> 
> In the last few days we've seen several thousand bogus subscription
> requests for various lists we host, send through the web interface. They
> seem to mostly originate in China.
> 
> We see log entries such as /var/log/mailman/subscribe
> Jan 11 20:50:30 2016 (27666) grsi-users: pending
> hellocatboots+80339132 at gmail.com  221.178.182.31
> and in the webserver logs
> 221.178.182.31 - - [10/Jan/2016:03:27:18 -0800] "POST
> /mailman/subscribe/grsi-users HTTP/1.1" 200
> 
> I'm not sure what the point is - a DoS attack on a few users, perhaps. I
> see that gmail gives you infinite aliases, so that
> hellocatboots+80339132 is the same as hellocatboots+96529823 at gmail.com


There are threads on this in the archives of this list. See threads
containing the posts
<https://mail.python.org/pipermail/mailman-users/2015-September/079829.html>
and
<https://mail.python.org/pipermail/mailman-users/2015-September/079844.html>
and perhaps the thread starting at
https://mail.python.org/pipermail/mailman-users/2015-September/079855.html.

For the @python.org lists, we use the regexp '^.*\+.*\d{3,}@' in the
newly implemented, not yet released GLOBAL_BAN_LIST to ban all addresses
with a '+' followed by anything followed by at least 3 digits up to the '@'.

Read
<https://mail.python.org/pipermail/mailman-users/2015-September/079844.html>
for more on that.

It's been effective so far.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Tue Jan 12 12:54:26 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 12 Jan 2016 09:54:26 -0800
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <BN3PR09MB054854A9A49C1A8215B0BF3DCACA0@BN3PR09MB0548.namprd09.prod.outlook.com>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <BN3PR09MB054854A9A49C1A8215B0BF3DCACA0@BN3PR09MB0548.namprd09.prod.outlook.com>
Message-ID: <56953DD2.3060906@msapiro.net>

On 01/12/2016 08:18 AM, Rosenbaum, Larry M. wrote:
>>From the "NEWS" file:
> 
>     - There is a new mm_cfg.py setting SUBSCRIBE_FORM_SECRET ...


This is only partially effective against this attack.

On the @python.org lists we see this attack come and go and even with
SUBSCRIBE_FORM_MIN_TIME = 10 seconds, enough of the bot subscribes delay
long enough to get through.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From advax at triumf.ca  Tue Jan 12 23:03:03 2016
From: advax at triumf.ca (Andrew Daviel)
Date: Tue, 12 Jan 2016 20:03:03 -0800 (PST)
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <56953DD2.3060906@msapiro.net>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <BN3PR09MB054854A9A49C1A8215B0BF3DCACA0@BN3PR09MB0548.namprd09.prod.outlook.com>
 <56953DD2.3060906@msapiro.net>
Message-ID: <alpine.LRH.2.03.1601121959120.1007@andrew.triumf.ca>

On Tue, 12 Jan 2016, Mark Sapiro wrote:

> On 01/12/2016 08:18 AM, Rosenbaum, Larry M. wrote:
>>> From the "NEWS" file:
>>
>>     - There is a new mm_cfg.py setting SUBSCRIBE_FORM_SECRET ...
>
>
> This is only partially effective against this attack.


Thanks for the info.

Typical of me, I kept looking for a workaround after posting and didn't 
see this straight away. I will look into SUBSCRIBE_FORM_SECRET.

Meanwhile, I found the bot was evading the address block ban by using 
other hosts, and have tried adding a simple CAPTCHA based on Apache anonymous 
authentication. If a user tries to access a mailman script from offsite, 
they get a 401 rejection and a prompt to login with a simple username 
(which changes every hour)



-- 
Andrew Daviel, TRIUMF, Canada
Tel. +1 (604) 222-7376  (Pacific Time)
Network Security Manager

From mark at tafn.org.uk  Wed Jan 13 14:52:40 2016
From: mark at tafn.org.uk (mark)
Date: Wed, 13 Jan 2016 19:52:40 +0000
Subject: [Mailman-Users] mailman help
Message-ID: <5696AB08.2050200@tafn.org.uk>

Hi all,
is there a way of setting some one subscription for a month then it be 
deleted after that time unless told not to
mark.

From mark at msapiro.net  Wed Jan 13 14:59:36 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 13 Jan 2016 11:59:36 -0800
Subject: [Mailman-Users] mailman help
In-Reply-To: <5696AB08.2050200@tafn.org.uk>
References: <5696AB08.2050200@tafn.org.uk>
Message-ID: <5696ACA8.1020305@msapiro.net>

On 01/13/2016 11:52 AM, mark wrote:
> Hi all,
> is there a way of setting some one subscription for a month then it be
> deleted after that time unless told not to
> mark.


Not within Mailman, but if you have server access you could set a cron
to run bin/remove_members.

That might be practical for a one time thing, but it's probably too
cumbersome for a continuous stream of "trial subscriptions".

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From collaborista at gmail.com  Wed Jan 13 12:25:46 2016
From: collaborista at gmail.com (Karma Kolabor)
Date: Wed, 13 Jan 2016 18:25:46 +0100
Subject: [Mailman-Users] THIS list is for mailman 3 or mailman 2 or BOTH?
Message-ID: <CAPjHDBT5kEi-2HgJ=wPX2McR-yVh82G=9o8fvs+QdyzN-ALkNw@mail.gmail.com>

Hi,

this list - is it for mailman 3 or mailman 2 or are you planning on
using ONE mailing list for BOTH versions of mailman?

I did not find any information about separated lists, so I assume that
here both versions are discussed - what looks like the worst case
scenario to me.

If my apprehension is right, then I would like to suggest to maintain
one mailing list for mailman 3  and one for mailman 2.

Thanks for your attention,
Karma

From mark at msapiro.net  Wed Jan 13 23:19:53 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 13 Jan 2016 20:19:53 -0800
Subject: [Mailman-Users] THIS list is for mailman 3 or mailman 2 or BOTH?
In-Reply-To: <CAPjHDBT5kEi-2HgJ=wPX2McR-yVh82G=9o8fvs+QdyzN-ALkNw@mail.gmail.com>
References: <CAPjHDBT5kEi-2HgJ=wPX2McR-yVh82G=9o8fvs+QdyzN-ALkNw@mail.gmail.com>
Message-ID: <569721E9.9040401@msapiro.net>

On 01/13/2016 09:25 AM, Karma Kolabor wrote:
> Hi,
> 
> this list - is it for mailman 3 or mailman 2 or are you planning on
> using ONE mailing list for BOTH versions of mailman?


This list is primarily for Mailman 2, although Mailman 3 questions will
be answered to the best of our ability.

We are working on setting up mailman3-users at mailman3.org (running on MM
3 of course), but there are still infrastructure issues to work out
before that list will be operational.

In the mean time the best list for Mailman 3 issues is
mailman-developers at python.org.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From dap1 at bellsouth.net  Thu Jan 14 07:02:03 2016
From: dap1 at bellsouth.net (Dennis Putnam)
Date: Thu, 14 Jan 2016 07:02:03 -0500
Subject: [Mailman-Users] Reply-To-List Header
In-Reply-To: <5692EA97.3040506@msapiro.net>
References: <569115D2.4000603@bellsouth.net> <56915335.2000202@msapiro.net>
 <56917739.8060207@bellsouth.net> <56917F07.9020502@msapiro.net>
 <56926ABC.1060603@bellsouth.net> <5692A996.2030707@msapiro.net>
 <5692E432.4070500@bellsouth.net> <5692EA97.3040506@msapiro.net>
Message-ID: <56978E3B.7090704@bellsouth.net>

On 1/10/2016 6:34 PM, Mark Sapiro wrote:
> On 01/10/2016 03:07 PM, Dennis Putnam wrote:
>> Google is back up. I guess I was wrong in my original assessment. The
>> headers are indeed there including List-Post. The reply to list button
>> is displayed (I don't know why I sometimes don't get that but I will
>> have to look closer next time it happens). Anyway, I now see the
>> problem, but I don't know how/where to fix it. The address in the
>> List-Post is wrong. I can see how to turn it off/on in the documentation
>> but not how to customize it?
>
> It is the list posting address, i.e. listname at host_name, the same as the
> address in "To post a message to all the list members, send email to
> ..." on the list's listinfo page.
>
> If the local part in the List-Post: header is not the list name,
> something is seriously wrong, but I don't know what. If the domain is
> not what you think it should be, the list's host_name (on the General
> Options page) is wrong. You can change it there, although that may
> indicate issues with add_virtualhost and DEFAULT_*_HOST in mm_cfg.py.
>
> As I said before, if you sometimes don't see it, it might be because you
> are looking at a reply sent to you directly (To: or Cc:) and not from
> the list.
>
>
Got it. I guess the "Preferred Hostname" in the settings is somewhat of
a misnomer. It is really the preferred email hostname.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160114/4717249f/attachment.sig>

From david at midrange.com  Thu Jan 14 09:55:21 2016
From: david at midrange.com (Gibbs, David)
Date: Thu, 14 Jan 2016 08:55:21 -0600
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <56953DBE.5060708@msapiro.net>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <56953DBE.5060708@msapiro.net>
Message-ID: <n78csq$ssq$1@ger.gmane.org>

On 1/12/2016 11:54 AM, Mark Sapiro wrote:

> There are threads on this in the archives of this list. See threads
> containing the posts
> <https://mail.python.org/pipermail/mailman-users/2015-September/079829.html>

I can confirm that the technique used in this post work great.

david

-- 
IBM i on Power Systems: For when you can't afford to be out of business!

I'm riding 100 miles (a full century) in the American Diabetes Association's Tour de Cure to raise money for diabetes research, education, advocacy, and awareness.  You can make a tax deductible donation to my ride by visiting http://gmanesig.diabetessucks.net.  My goal is $6000 but any amount is appreciated.

See where I get my donations from ... visit http://gmanesig.diabetessucks.net/map for an interactive map (it's a geeky thing).


From bryant at sandiego.edu  Thu Jan 14 12:45:45 2016
From: bryant at sandiego.edu (Bryan Teague)
Date: Thu, 14 Jan 2016 17:45:45 +0000
Subject: [Mailman-Users] known_spammers issue
Message-ID: <F31AA82F-42EA-408B-A0DD-6A6BE54A47CD@sandiego.edu>

Hi,

I have updated the KNOWN_SPAMMERS array in the mm_cfg.py  and then bounced the Qrunner.   I am using Mailman 2.1.15 with postfix.

My known_spammer declaration is:

KNOWN_SPAMMERS =[('from', 'hellocatboots.*?)]

Is there something I am missing on blocking the subscription requests now?  These are all coming in via email.

Thanks,
Bryan



Bryan Teague | Senior Web Administrator  | University Web Services | ITS | Maher Hall, Room 198
5998 Alcal? Park | San Diego, CA 92110-2492 | P: (619) 260-7842 | C: (619) 321-7288
bryant at sandiego.edu<mailto:bryant at sandiego.edu>


From mark at msapiro.net  Thu Jan 14 13:25:50 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 14 Jan 2016 10:25:50 -0800
Subject: [Mailman-Users] known_spammers issue
In-Reply-To: <F31AA82F-42EA-408B-A0DD-6A6BE54A47CD@sandiego.edu>
References: <F31AA82F-42EA-408B-A0DD-6A6BE54A47CD@sandiego.edu>
Message-ID: <5697E82E.3060203@msapiro.net>

On 01/14/2016 09:45 AM, Bryan Teague wrote:
> Hi,
> 
> I have updated the KNOWN_SPAMMERS array in the mm_cfg.py  and then bounced the Qrunner.   I am using Mailman 2.1.15 with postfix.
> 
> My known_spammer declaration is:
> 
> KNOWN_SPAMMERS =[('from', 'hellocatboots.*?)]


KNOWN_SPAMMERS is an old setting. The list's Privacy options... -> Spam
filters -> header_filter_rules is much more powerful and flexible.


> Is there something I am missing on blocking the subscription requests now?  These are all coming in via email.


What makes you think they are all coming by email? All the ones I see
come via the web, but in any case, neither KNOWN_SPAMMERS nor
header_filter_rules is applied to mail to the
listname-{join|subscribe|request} addresses. They only apply to mail to
the list and to listname-owner.

You need to use the ban_list to stop these. See this post
<https://mail.python.org/pipermail/mailman-users/2016-January/080303.html>
and the posts and threads linked from that.

Do not be misled by the fact that that post refers to "the
newly implemented, not yet released GLOBAL_BAN_LIST". Each list has it's
own Privacy options... -> Subscription rules -> ban_list and there is a
withlist script at <https://www.msapiro.net/scripts/add_banned.py>
(mirrored at <http://fog.ccsf.edu/~msapiro/scripts/add_banned.py>) which
can add the regexp to the ban_list for all lists.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Jan 14 13:38:50 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 14 Jan 2016 10:38:50 -0800
Subject: [Mailman-Users] Reply-To-List Header
In-Reply-To: <56978E3B.7090704@bellsouth.net>
References: <569115D2.4000603@bellsouth.net> <56915335.2000202@msapiro.net>
 <56917739.8060207@bellsouth.net> <56917F07.9020502@msapiro.net>
 <56926ABC.1060603@bellsouth.net> <5692A996.2030707@msapiro.net>
 <5692E432.4070500@bellsouth.net> <5692EA97.3040506@msapiro.net>
 <56978E3B.7090704@bellsouth.net>
Message-ID: <5697EB3A.60002@msapiro.net>

On 01/14/2016 04:02 AM, Dennis Putnam wrote:
>
> Got it. I guess the "Preferred Hostname" in the settings is somewhat of
> a misnomer. It is really the preferred email hostname.


Where are you seeing "Preferred Hostname"?

Mailman's General Options page says

Host name this list prefers for email.
(Details for host_name)
	
which seems clear enough. I can't find the regexp 'preferred.?host' case
insensitively anywhere in the GNU Mailman distribution.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160114/c7db5f3d/attachment.sig>

From dap1 at bellsouth.net  Thu Jan 14 14:58:45 2016
From: dap1 at bellsouth.net (Dennis Putnam)
Date: Thu, 14 Jan 2016 14:58:45 -0500
Subject: [Mailman-Users] Reply-To-List Header
In-Reply-To: <5697EB3A.60002@msapiro.net>
References: <569115D2.4000603@bellsouth.net> <56915335.2000202@msapiro.net>
 <56917739.8060207@bellsouth.net> <56917F07.9020502@msapiro.net>
 <56926ABC.1060603@bellsouth.net> <5692A996.2030707@msapiro.net>
 <5692E432.4070500@bellsouth.net> <5692EA97.3040506@msapiro.net>
 <56978E3B.7090704@bellsouth.net> <5697EB3A.60002@msapiro.net>
Message-ID: <5697FDF5.3010001@bellsouth.net>

On 1/14/2016 1:38 PM, Mark Sapiro wrote:
> On 01/14/2016 04:02 AM, Dennis Putnam wrote:
>> Got it. I guess the "Preferred Hostname" in the settings is somewhat of
>> a misnomer. It is really the preferred email hostname.
>
> Where are you seeing "Preferred Hostname"?
>
> Mailman's General Options page says
>
> Host name this list prefers for email.
> (Details for host_name)
> 	
> which seems clear enough. I can't find the regexp 'preferred.?host' case
> insensitively anywhere in the GNU Mailman distribution.
>
>
Sorry, I guess I should not have put it in quotes. That title is not at
all clear to me that it means the domain name (not host name) for
replying email. I had that set to my email domain (bellsouth.net).
Changing it to the reply email domain (gmail.com) worked. I tried the
host name 'pop.gmail.com' and that did not.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160114/29366ba4/attachment.sig>

From bryant at sandiego.edu  Thu Jan 14 15:26:01 2016
From: bryant at sandiego.edu (Bryan Teague)
Date: Thu, 14 Jan 2016 20:26:01 +0000
Subject: [Mailman-Users] known_spammers issue
In-Reply-To: <5697E82E.3060203@msapiro.net>
References: <F31AA82F-42EA-408B-A0DD-6A6BE54A47CD@sandiego.edu>
 <5697E82E.3060203@msapiro.net>
Message-ID: <38ED9164-953B-47F0-B8A2-FADB837ECED8@sandiego.edu>

Hi Mark,

Thank you very much.  You were right,  I was looking at the wrong log file.  As soon as I identified the proper log file,  I confirmed the subscriptions through the web page.

Using the with list and the add_banned.py as explained in your email and the other links  was what finally solved it for me.

Each list has it's
own Privacy options... -> Subscription rules -> ban_list and there is a
withlist script at <https://www.msapiro.net/scripts/add_banned.py>
(mirrored at <http://fog.ccsf.edu/~msapiro/scripts/add_banned.py>) which
can add the regexp to the ban_list for all lists.


Thanks,
Bryan

Bryan Teague | Senior Web Administrator  | University Web Services | ITS | Maher Hall, Room 198
5998 Alcal? Park | San Diego, CA 92110-2492 | P: (619) 260-7842 | C: (619) 321-7288
bryant at sandiego.edu<mailto:bryant at sandiego.edu>

On Jan 14, 2016, at 10:25 AM, Mark Sapiro <mark at msapiro.net<mailto:mark at msapiro.net>> wrote:

On 01/14/2016 09:45 AM, Bryan Teague wrote:
Hi,

I have updated the KNOWN_SPAMMERS array in the mm_cfg.py  and then bounced the Qrunner.   I am using Mailman 2.1.15 with postfix.

My known_spammer declaration is:

KNOWN_SPAMMERS =[('from', 'hellocatboots.*?)]


KNOWN_SPAMMERS is an old setting. The list's Privacy options... -> Spam
filters -> header_filter_rules is much more powerful and flexible.


Is there something I am missing on blocking the subscription requests now?  These are all coming in via email.


What makes you think they are all coming by email? All the ones I see
come via the web, but in any case, neither KNOWN_SPAMMERS nor
header_filter_rules is applied to mail to the
listname-{join|subscribe|request} addresses. They only apply to mail to
the list and to listname-owner.

You need to use the ban_list to stop these. See this post
<https://mail.python.org/pipermail/mailman-users/2016-January/080303.html>
and the posts and threads linked from that.

Do not be misled by the fact that that post refers to "the
newly implemented, not yet released GLOBAL_BAN_LIST". Each list has it's
own Privacy options... -> Subscription rules -> ban_list and there is a
withlist script at <https://www.msapiro.net/scripts/add_banned.py>
(mirrored at <http://fog.ccsf.edu/~msapiro/scripts/add_banned.py>) which
can add the regexp to the ban_list for all lists.

--
Mark Sapiro <mark at msapiro.net<mailto:mark at msapiro.net>>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan
------------------------------------------------------
Mailman-Users mailing list Mailman-Users at python.org<mailto:Mailman-Users at python.org>
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: https://mail.python.org/mailman/options/mailman-users/bryant%40sandiego.edu


From mark at msapiro.net  Thu Jan 14 16:28:01 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 14 Jan 2016 13:28:01 -0800
Subject: [Mailman-Users] Reply-To-List Header
In-Reply-To: <5697FDF5.3010001@bellsouth.net>
References: <569115D2.4000603@bellsouth.net> <56915335.2000202@msapiro.net>
 <56917739.8060207@bellsouth.net> <56917F07.9020502@msapiro.net>
 <56926ABC.1060603@bellsouth.net> <5692A996.2030707@msapiro.net>
 <5692E432.4070500@bellsouth.net> <5692EA97.3040506@msapiro.net>
 <56978E3B.7090704@bellsouth.net> <5697EB3A.60002@msapiro.net>
 <5697FDF5.3010001@bellsouth.net>
Message-ID: <569812E1.4070109@msapiro.net>

On 01/14/2016 11:58 AM, Dennis Putnam wrote:
>
> Sorry, I guess I should not have put it in quotes. That title is not at
> all clear to me that it means the domain name (not host name) for
> replying email. I had that set to my email domain (bellsouth.net).
> Changing it to the reply email domain (gmail.com) worked. I tried the
> host name 'pop.gmail.com' and that did not.


OK. Now I understand the confusion. We use 'host' in a number of places
where we really mean 'domain' and it can be particularly confusing for
email addresses where the 'domain' may just have an MX record and not be
a 'machine' at all.

However, this terminology is deeply embedded and affects translations if
it were to be changed, so I don't propose to actually change it at this
point.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160114/241f7dbe/attachment-0001.sig>

From newville at cars.uchicago.edu  Thu Jan 14 15:06:36 2016
From: newville at cars.uchicago.edu (Matt Newville)
Date: Thu, 14 Jan 2016 14:06:36 -0600
Subject: [Mailman-Users] permission issue
Message-ID: <CA+7ESboS8mDtp_WXBH53p3Afm9Uz2X4ZUF717A_ueEz367wwLg@mail.gmail.com>

Hi,

Sorry if this is actually a FAQ entry -- I looked at many FAQs and on-line
resources and haven't found a working solution yet.

I've been using Mailman for many years, very appreciative of the work.  I
am trying to move some lists to a machine running Fedora 22, with mailman
2.1.20 and apache 2.4.17, using the fedora-supplied packages.

Mailman is installed to /usr/lib/mailman, with list data in
/var/lib/mailman, and there is a mailman user and group (uid = gid = 41).
I migrated several lists, wiped and successfully re-made their Archives,
and created a fresh list for testing called test1.

I followed the migration directions as best as possible, run check_perms
-f, and  the files in /usr/lib/mailman/cgi-bin are set g+s.  The files in
/var/lib/mailman/lists/test1 have group=mailman.

I can subscribe and manage all the groups through the web interface, and
mail sent to to 'test1' group works, but the archives are not created.
/var/log/mailman/error shows

Jan 14 11:35:01 2016 (9055) Archive file access failure:
        /var/lib/mailman/archives/private/test1.mbox/test1.mbox [Errno 13]
Permission denied: '/var/lib/mailman/archives/private/test1.mbox/test1.mbox'
Jan 14 11:35:01 2016 (9055) Uncaught runner exception: [Errno 13]
Permission denied: '/var/lib/mailman/archives/private/test1.mbox/test1.mbox'
Jan 14 11:35:01 2016 (9055) Traceback (most recent call last):
  File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop
    self._onefile(msg, msgdata)
  File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/usr/lib/mailman/Mailman/Queue/ArchRunner.py", line 73, in _dispose
    mlist.ArchiveMail(msg)
  File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 200, in
ArchiveMail
    self.__archive_to_mbox(msg)
  File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 169, in
__archive_to_mbox
    mbox = self.__archive_file(afn)
  File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 157, in
__archive_file
    return Mailbox.Mailbox(open(afn, 'a+'))
IOError: [Errno 13] Permission denied:
'/var/lib/mailman/archives/private/test1.mbox/test1.mbox'

Jan 14 11:35:01 2016 (9055) SHUNTING:
1452792899.410995+4ddf5a7718ff13c171a78e27a2caaf5429312510

I get similar errors if I manually run mailman/bin/unshunt.

Apache does load mod_suexec.so, but (as far as I can tell) it's not used
anywhere.

I've tried making the permissions on the files and directories
/var/lib/mailman/  very liberal, but still see problems either with the
mbox file or (from other, existing lists that have been migrated) with
'config.pck.tmp.SERVERNAME...', and the mail is not even sent.

Any suggestions on what to try or how to better diagnose the problem?   Is
there a known issue with the fedora packages that suggest using a source
build of Mailman or Apache?

Thanks very much for any suggestions,

--Matt Newville

From mark at msapiro.net  Fri Jan 15 01:01:30 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 14 Jan 2016 22:01:30 -0800
Subject: [Mailman-Users] permission issue
In-Reply-To: <CA+7ESboS8mDtp_WXBH53p3Afm9Uz2X4ZUF717A_ueEz367wwLg@mail.gmail.com>
References: <CA+7ESboS8mDtp_WXBH53p3Afm9Uz2X4ZUF717A_ueEz367wwLg@mail.gmail.com>
Message-ID: <56988B3A.80904@msapiro.net>

On 01/14/2016 12:06 PM, Matt Newville wrote:

> 
> Jan 14 11:35:01 2016 (9055) Archive file access failure:
>         /var/lib/mailman/archives/private/test1.mbox/test1.mbox [Errno 13]
> Permission denied: '/var/lib/mailman/archives/private/test1.mbox/test1.mbox'
> Jan 14 11:35:01 2016 (9055) Uncaught runner exception: [Errno 13]
> Permission denied: '/var/lib/mailman/archives/private/test1.mbox/test1.mbox'
> Jan 14 11:35:01 2016 (9055) Traceback (most recent call last):
>   File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop
>     self._onefile(msg, msgdata)
>   File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile
>     keepqueued = self._dispose(mlist, msg, msgdata)
>   File "/usr/lib/mailman/Mailman/Queue/ArchRunner.py", line 73, in _dispose
>     mlist.ArchiveMail(msg)
>   File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 200, in
> ArchiveMail
>     self.__archive_to_mbox(msg)
>   File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 169, in
> __archive_to_mbox
>     mbox = self.__archive_file(afn)
>   File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 157, in
> __archive_file
>     return Mailbox.Mailbox(open(afn, 'a+'))
> IOError: [Errno 13] Permission denied:
> '/var/lib/mailman/archives/private/test1.mbox/test1.mbox'
> 
> Jan 14 11:35:01 2016 (9055) SHUNTING:
> 1452792899.410995+4ddf5a7718ff13c171a78e27a2caaf5429312510


So ArchRunner can't write to
/var/lib/mailman/archives/private/test1.mbox/test1.mbox.


> Apache does load mod_suexec.so, but (as far as I can tell) it's not used
> anywhere.


This would only affect web accesses in any case.


> I've tried making the permissions on the files and directories
> /var/lib/mailman/  very liberal, but still see problems either with the
> mbox file or (from other, existing lists that have been migrated) with
> 'config.pck.tmp.SERVERNAME...', and the mail is not even sent.


What are the owner, group and mode on every directory and the file in
the path /var/lib/mailman/archives/private/test1.mbox/test1.mbox and
every directory in the path /var/lib/mailman/lists/listname and the
config.pck* files for the other list names with config.pck.* problems?

And is ArchRunner and the other runners running as the 'mailman' user.


> Any suggestions on what to try or how to better diagnose the problem?   Is
> there a known issue with the fedora packages that suggest using a source
> build of Mailman or Apache?


I'm not aware of anything specific.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From newville at cars.uchicago.edu  Fri Jan 15 11:36:08 2016
From: newville at cars.uchicago.edu (Matt Newville)
Date: Fri, 15 Jan 2016 10:36:08 -0600
Subject: [Mailman-Users] permission issue
In-Reply-To: <56988B3A.80904@msapiro.net>
References: <CA+7ESboS8mDtp_WXBH53p3Afm9Uz2X4ZUF717A_ueEz367wwLg@mail.gmail.com>
 <56988B3A.80904@msapiro.net>
Message-ID: <CA+7ESbpkyrr_U=uHk--1xBoXLZknAck4QuzJwNATLiKhrEisDQ@mail.gmail.com>

Hi Mark,

Thanks very much for the reply.  The Runner programs did have mailman as
their user and group.  Following the clue given by the quote in your
signature, what I "gathered from coincidence" was that the system had
SELinux set to enforcing, but that there was no SELinux rule to allow the
mailman scripts to be run from httpd process.

Archiving is now working happily.  Thanks again!

--Matt



On Fri, Jan 15, 2016 at 12:01 AM, Mark Sapiro <mark at msapiro.net> wrote:

> On 01/14/2016 12:06 PM, Matt Newville wrote:
>
> >
> > Jan 14 11:35:01 2016 (9055) Archive file access failure:
> >         /var/lib/mailman/archives/private/test1.mbox/test1.mbox [Errno
> 13]
> > Permission denied:
> '/var/lib/mailman/archives/private/test1.mbox/test1.mbox'
> > Jan 14 11:35:01 2016 (9055) Uncaught runner exception: [Errno 13]
> > Permission denied:
> '/var/lib/mailman/archives/private/test1.mbox/test1.mbox'
> > Jan 14 11:35:01 2016 (9055) Traceback (most recent call last):
> >   File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop
> >     self._onefile(msg, msgdata)
> >   File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile
> >     keepqueued = self._dispose(mlist, msg, msgdata)
> >   File "/usr/lib/mailman/Mailman/Queue/ArchRunner.py", line 73, in
> _dispose
> >     mlist.ArchiveMail(msg)
> >   File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 200, in
> > ArchiveMail
> >     self.__archive_to_mbox(msg)
> >   File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 169, in
> > __archive_to_mbox
> >     mbox = self.__archive_file(afn)
> >   File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 157, in
> > __archive_file
> >     return Mailbox.Mailbox(open(afn, 'a+'))
> > IOError: [Errno 13] Permission denied:
> > '/var/lib/mailman/archives/private/test1.mbox/test1.mbox'
> >
> > Jan 14 11:35:01 2016 (9055) SHUNTING:
> > 1452792899.410995+4ddf5a7718ff13c171a78e27a2caaf5429312510
>
>
> So ArchRunner can't write to
> /var/lib/mailman/archives/private/test1.mbox/test1.mbox.
>
>
> > Apache does load mod_suexec.so, but (as far as I can tell) it's not used
> > anywhere.
>
>
> This would only affect web accesses in any case.
>
>
> > I've tried making the permissions on the files and directories
> > /var/lib/mailman/  very liberal, but still see problems either with the
> > mbox file or (from other, existing lists that have been migrated) with
> > 'config.pck.tmp.SERVERNAME...', and the mail is not even sent.
>
>
> What are the owner, group and mode on every directory and the file in
> the path /var/lib/mailman/archives/private/test1.mbox/test1.mbox and
> every directory in the path /var/lib/mailman/lists/listname and the
> config.pck* files for the other list names with config.pck.* problems?
>
> And is ArchRunner and the other runners running as the 'mailman' user.
>
>
> > Any suggestions on what to try or how to better diagnose the problem?
>  Is
> > there a known issue with the fedora packages that suggest using a source
> > build of Mailman or Apache?
>
>
> I'm not aware of anything specific.
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/matt.newville%40gmail.com
>

From mark at msapiro.net  Fri Jan 15 13:06:22 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 15 Jan 2016 10:06:22 -0800
Subject: [Mailman-Users] permission issue
In-Reply-To: <CA+7ESbpkyrr_U=uHk--1xBoXLZknAck4QuzJwNATLiKhrEisDQ@mail.gmail.com>
References: <CA+7ESboS8mDtp_WXBH53p3Afm9Uz2X4ZUF717A_ueEz367wwLg@mail.gmail.com>
 <56988B3A.80904@msapiro.net>
 <CA+7ESbpkyrr_U=uHk--1xBoXLZknAck4QuzJwNATLiKhrEisDQ@mail.gmail.com>
Message-ID: <5699351E.7020103@msapiro.net>

On 01/15/2016 08:36 AM, Matt Newville wrote:
> 
> Following the clue given by the quote in your
> signature, what I "gathered from coincidence" was that the system had
> SELinux set to enforcing, but that there was no SELinux rule to allow
> the mailman scripts to be run from httpd process.


I tend to forget about that, but yes, SELinux is often the cause of
mysterious lack of permissions. I have added this brief FAQ
<http://wiki.list.org/x/17891944>.


> Archiving is now working happily.  Thanks again!


Good!

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From eisaksso at gmail.com  Sat Jan 16 09:24:46 2016
From: eisaksso at gmail.com (Eva Isaksson)
Date: Sat, 16 Jan 2016 16:24:46 +0200
Subject: [Mailman-Users] Encoding problem with 2.15 to 2.18 upgrade with
 Finnish
In-Reply-To: <568F08ED.60609@msapiro.net>
References: <CAEMzDVPAOw=jOK6NpruVh0N5UzyVGDhiLuS4TLOjDSzAzxt_Yg@mail.gmail.com>
 <566F15FD.5060102@msapiro.net>
 <CAEMzDVPXa+6Y83s5jZBmGGmF8d46Bogua_BeqtTqdkcngSm7HA@mail.gmail.com>
 <568C0E8A.5030907@msapiro.net> <568C1994.7050403@msapiro.net>
 <CAEMzDVPsj_cK_qSDQWqoqHLyc_Gix=554oV0uS45tBBYAujkrw@mail.gmail.com>
 <568EC034.7070706@msapiro.net> <568EC6B5.6060507@msapiro.net>
 <CAEMzDVPGnXVuN3zNNFaknm_aW9LdMuAvJft3SOThj57Wwv0KhQ@mail.gmail.com>
 <568ED683.5010300@msapiro.net> <568F08ED.60609@msapiro.net>
Message-ID: <CAEMzDVP7KFL21LnQt7Td-=yJndu7G7youQTK7xKBoDNuECVRow@mail.gmail.com>

I did the 2.1.18 upgrade this weekend, after some weighing of the options.
In the long run, using the Debian package is less work intensive, so
in the end my choice was the utf-8 conversion.

> The script is at <https://www.msapiro.net/scripts/recode_list> (mirrored
> at <http://fog.ccsf.edu/~msapiro/scripts/recode_list>)

To cut the long story short, the script worked without any problems.
Taking backups took more time than running the script and doing the
package upgrade.

Such a relief - thank you.

- Eva Isaksson



2016-01-08 2:55 GMT+02:00 Mark Sapiro <mark at msapiro.net>:

> On 01/07/2016 01:20 PM, Mark Sapiro wrote:
> >
> > You can keep the Debian package with it's utf-8 character sets and
> > programmatically change all your lists so they work. In MM 2.1.19 I
> > changed the character sets for Russian and Romanian from koi8-r and
> > iso-8859-2 respectively to utf-8 because I was convinced that the former
> > character sets were not appropriate for those languages. In so doing, I
> > augmented Mailman's version updater to identify strings in list
> > attributes that were not valid utf-8 and convert them from the former
> > character set.
> >
> > This is an imperfect process. It does guarantee that all the strings in
> > list attributes are valid utf-8 encodings, but not necessarily that they
> > are 'correct'.
> >
> > I am working on a stand-alone script to do this. I will post it later
> today.
>
>
> There is now a first cut at this script. The functions that do the
> recoding are lifted straight from Mailman/versions.py so they have been
> fairly well exercised. I didn't just import them because I wanted the
> caller to know if they had changed anything and that required a minor
> addition to the top level function to return a flag.
>
> The script is at <https://www.msapiro.net/scripts/recode_list> (mirrored
> at <http://fog.ccsf.edu/~msapiro/scripts/recode_list>)
>
> Install the script in Mailman's bin/ directory and run it with the -h
> option for full info. It uses Python's argparse function and as such
> requires python 2.7.
>
> In your case, to convert all Finnish preferred_language lists strings
> from iso-8859-1 encoding to utf-8
>
> recode_list --language=fi
>
> would suffice. There are options to specify other than iso-8859-1 to
> utf-8 and to produce output about skipped or unchanged lists, but the
> defaults should be good for you and will report which lists were
> changed. In no case does it report the actual changes, just what lists
> were changed.
>
> I recommend making backups of your lists/*/config.pck files before
> running this in case something goes wrong.
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
>

From christian at web-analysts.net  Fri Jan 15 11:13:22 2016
From: christian at web-analysts.net (Web Analysts)
Date: Fri, 15 Jan 2016 08:13:22 -0800
Subject: [Mailman-Users] IOError reading list extension: [Errno 20] Not a
 directory
Message-ID: <010201d14faf$acbe9fa0$063bdee0$@web-analysts.net>

My mail server is delivering a post to mailman but it is not being
distributed. The mailman error log shows

 

IOError reading list extension: [Errno 20] Not a directory:
'/usr/lib/mailman/archives/private/listname.mbox/listname.mbox/extend.py'

 

I have checked perms, the db and restarted mailman, all to no avail.

 

Help please.

 

 

Regards,

_

Christian Stalberg

Web Analysts

http://web-analysts.net <http://web-analysts.net/> 

Web_Analysts_signature

This communication may be unlawfully collected and stored by the National
Security Agency (NSA) in secret. The parties to this email do not consent to
the retrieving or storing of this communication and any related metadata, as
well as printing, copying, re-transmitting, disseminating, or otherwise
using it. If you believe you have received this communication in error,
please delete it immediately.

 


From mark at msapiro.net  Sat Jan 16 14:24:10 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 16 Jan 2016 11:24:10 -0800
Subject: [Mailman-Users] Encoding problem with 2.15 to 2.18 upgrade with
 Finnish
In-Reply-To: <CAEMzDVP7KFL21LnQt7Td-=yJndu7G7youQTK7xKBoDNuECVRow@mail.gmail.com>
References: <CAEMzDVPAOw=jOK6NpruVh0N5UzyVGDhiLuS4TLOjDSzAzxt_Yg@mail.gmail.com>
 <566F15FD.5060102@msapiro.net>
 <CAEMzDVPXa+6Y83s5jZBmGGmF8d46Bogua_BeqtTqdkcngSm7HA@mail.gmail.com>
 <568C0E8A.5030907@msapiro.net> <568C1994.7050403@msapiro.net>
 <CAEMzDVPsj_cK_qSDQWqoqHLyc_Gix=554oV0uS45tBBYAujkrw@mail.gmail.com>
 <568EC034.7070706@msapiro.net> <568EC6B5.6060507@msapiro.net>
 <CAEMzDVPGnXVuN3zNNFaknm_aW9LdMuAvJft3SOThj57Wwv0KhQ@mail.gmail.com>
 <568ED683.5010300@msapiro.net> <568F08ED.60609@msapiro.net>
 <CAEMzDVP7KFL21LnQt7Td-=yJndu7G7youQTK7xKBoDNuECVRow@mail.gmail.com>
Message-ID: <569A98DA.8010801@msapiro.net>

On 01/16/2016 06:24 AM, Eva Isaksson wrote:
> 
> To cut the long story short, the script worked without any problems.
> Taking backups took more time than running the script and doing the
> package upgrade.


I'm glad the script worked for you.

I have mentioned this issue to the Debian people who are responsible for
the Mailman package, but it might be helpful if you report your
experiences to Debian. They don't seem to realize that they are
potentially causing this pain for large numbers of users of their
Mailman package.

While I have "fixed" <https://bugs.launchpad.net/mailman/+bug/1462755>,
that fix will not be released until 2.1.21 (soon I hope) and it only
addresses the symptom and only in one place. Potentially, other strings
can cause the same issue.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sat Jan 16 17:07:09 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 16 Jan 2016 14:07:09 -0800
Subject: [Mailman-Users] IOError reading list extension: [Errno 20] Not
 a directory
In-Reply-To: <010201d14faf$acbe9fa0$063bdee0$@web-analysts.net>
References: <010201d14faf$acbe9fa0$063bdee0$@web-analysts.net>
Message-ID: <569ABF0D.2050606@msapiro.net>

On 01/15/2016 08:13 AM, Web Analysts wrote:
> My mail server is delivering a post to mailman but it is not being
> distributed. The mailman error log shows
> 
>  
> 
> IOError reading list extension: [Errno 20] Not a directory:
> '/usr/lib/mailman/archives/private/listname.mbox/listname.mbox/extend.py'


This is strange. Please post the full traceback, not just the error message.

MailList should not ever be looking for
'/usr/lib/mailman/archives/private/listname.mbox/listname.mbox/extend.py'.
It only looks in the listname directory in the lists/ directory defined
in Defaults.py by LIST_DATA_DIR which in turn is $prefix/lists. I.e.it
should only be looking for '/usr/lib/mailman/lists/listname/extend.py'
which normally won't exist.

Have you set LIST_DATA_DIR in Mailman/mm_cfg.py or made any changes to
PREFIX, VAR_PREFIX or LIST_DATA_DIR in Mailman/Defaults.py or made any
changes to Mailman/Site.py?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From perry at piermont.com  Sat Jan 16 19:02:58 2016
From: perry at piermont.com (Perry E. Metzger)
Date: Sat, 16 Jan 2016 19:02:58 -0500
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <n78csq$ssq$1@ger.gmane.org>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <56953DBE.5060708@msapiro.net> <n78csq$ssq$1@ger.gmane.org>
Message-ID: <20160116190258.6b935766@jabberwock.cb.piermont.com>

On Thu, 14 Jan 2016 08:55:21 -0600 "Gibbs, David"
<david at midrange.com> wrote:
> On 1/12/2016 11:54 AM, Mark Sapiro wrote:
> 
> > There are threads on this in the archives of this list. See
> > threads containing the posts
> > <https://mail.python.org/pipermail/mailman-users/2015-September/079829.html>
> 
> I can confirm that the technique used in this post work great.

I have direct evidence that the asshats are now using "+" strings
after the main address that are not strictly numeric. They seem to
have responded to the simple ways of stopping them.

What's a good technique at this point to slow them down besides
regexps? I'm stuck on 2.1.18 at the moment as a debian user.

Perry
-- 
Perry E. Metzger		perry at piermont.com

From perry at piermont.com  Sat Jan 16 19:51:05 2016
From: perry at piermont.com (Perry E. Metzger)
Date: Sat, 16 Jan 2016 19:51:05 -0500
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <20160116190258.6b935766@jabberwock.cb.piermont.com>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <56953DBE.5060708@msapiro.net> <n78csq$ssq$1@ger.gmane.org>
 <20160116190258.6b935766@jabberwock.cb.piermont.com>
Message-ID: <20160116195105.0cc8346e@jabberwock.cb.piermont.com>

On Sat, 16 Jan 2016 19:02:58 -0500 "Perry E. Metzger"
<perry at piermont.com> wrote:
> On Thu, 14 Jan 2016 08:55:21 -0600 "Gibbs, David"
> <david at midrange.com> wrote:
> > On 1/12/2016 11:54 AM, Mark Sapiro wrote:
> > 
> > > There are threads on this in the archives of this list. See
> > > threads containing the posts
> > > <https://mail.python.org/pipermail/mailman-users/2015-September/079829.html>
> > 
> > I can confirm that the technique used in this post work great.
> 
> I have direct evidence that the asshats are now using "+" strings
> after the main address that are not strictly numeric. They seem to
> have responded to the simple ways of stopping them.
> 
> What's a good technique at this point to slow them down besides
> regexps? I'm stuck on 2.1.18 at the moment as a debian user.

Oh, and by the way, the documentation for SUBSCRIBE_FORM_SECRET (such
as it is) does not mention that it needs to be set to a string. I
only figured that out when setting it to True and 1 both failed
spectacularly and reading the traceback.

Perry
-- 
Perry E. Metzger		perry at piermont.com

From mark at msapiro.net  Sat Jan 16 19:52:29 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 16 Jan 2016 16:52:29 -0800
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <20160116190258.6b935766@jabberwock.cb.piermont.com>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <56953DBE.5060708@msapiro.net> <n78csq$ssq$1@ger.gmane.org>
 <20160116190258.6b935766@jabberwock.cb.piermont.com>
Message-ID: <569AE5CD.1090209@msapiro.net>

On 01/16/2016 04:02 PM, Perry E. Metzger wrote:
> 
> I have direct evidence that the asshats are now using "+" strings
> after the main address that are not strictly numeric. They seem to
> have responded to the simple ways of stopping them.


I haven't seen any like that yet. The regexp I use is ^.*\+.*\d{3,}@
which will block anything between '+' and '@' as long as it ends with 3
digits.

Please provide some examples. If there is any discernable pattern, it
might be blockable without impacting real subscribers.

I haven't actually seen kemo.mart or kezukaya + digits @gmail for 5
days. The last five days have been all hellocatboots+<eight
digits>@gmail.com, although a few days ago a bunch of 'ichiba at
kezukaya.com' actually got through. (I'm now banning that one too)


> What's a good technique at this point to slow them down besides
> regexps? I'm stuck on 2.1.18 at the moment as a debian user.


Based on the one above getting through, I wrote the script at
<https://www.msapiro.net/scripts/erase> (mirrored at
<http://fog.ccsf.edu/~msapiro/scripts/erase>) to 'erase' an address from
all lists.  I.e. for every list, if the address is a member, it is
removed. If there are any posts or (un)subscription requests from the
address awaiting moderator action, they are removed and optionally any
subscripions for the address awaiting user confirmation are discarded too.

As far as I can tell, even if the address isn't banned, unless you allow
open subscribe which is a bad idea for many reasons, the subscription
will require user confirmation, moderator approval, or both.

Those that require confirmation (or both) will never be confirmed
because the bots don't get the confirmation request email, but they do
create back-scatter (the confirmation request email) to the address.
Those that require moderator approval will presumably be discarded, but
are a moderator burden. The script can deal with all that at once.

Other possibilities are disabling web subscribe all together or
installing some kind of captcha (ugh) in the page. Experience shows that
SUBSCRIBE_FORM_SECRET doesn't stop them unless perhaps
SUBSCRIBE_FORM_MIN_TIME is set long enough that it becomes a problem for
real subscribers.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sat Jan 16 19:59:11 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 16 Jan 2016 16:59:11 -0800
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <20160116195105.0cc8346e@jabberwock.cb.piermont.com>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <56953DBE.5060708@msapiro.net> <n78csq$ssq$1@ger.gmane.org>
 <20160116190258.6b935766@jabberwock.cb.piermont.com>
 <20160116195105.0cc8346e@jabberwock.cb.piermont.com>
Message-ID: <569AE75F.7040902@msapiro.net>

On 01/16/2016 04:51 PM, Perry E. Metzger wrote:
> 
> Oh, and by the way, the documentation for SUBSCRIBE_FORM_SECRET (such
> as it is) does not mention that it needs to be set to a string. I
> only figured that out when setting it to True and 1 both failed
> spectacularly and reading the traceback.


It seems clear to me. The paragraph in Defaults.py begins

> # If the following is set to a non-empty string, this string in combination
> # with the time, list name and the IP address of the requestor is used to
> # create a hidden hash as part of the subscribe form on the listinfo page.

Perhaps you were only looking at the NEWS file?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sat Jan 16 20:11:44 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 16 Jan 2016 17:11:44 -0800
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <569AE5CD.1090209@msapiro.net>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <56953DBE.5060708@msapiro.net> <n78csq$ssq$1@ger.gmane.org>
 <20160116190258.6b935766@jabberwock.cb.piermont.com>
 <569AE5CD.1090209@msapiro.net>
Message-ID: <569AEA50.4070906@msapiro.net>

On 01/16/2016 04:52 PM, Mark Sapiro wrote:
> 
> Based on the one above getting through, I wrote the script at
> <https://www.msapiro.net/scripts/erase> (mirrored at
> <http://fog.ccsf.edu/~msapiro/scripts/erase>) to 'erase' an address from
> all lists.


Note, this script uses the Python argparse module to parse the command
line. As such it requires Python 2.7. Converting it to optparse is left
as an exercise for the reader.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From perry at piermont.com  Sat Jan 16 20:13:08 2016
From: perry at piermont.com (Perry E. Metzger)
Date: Sat, 16 Jan 2016 20:13:08 -0500
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <569AE5CD.1090209@msapiro.net>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <56953DBE.5060708@msapiro.net> <n78csq$ssq$1@ger.gmane.org>
 <20160116190258.6b935766@jabberwock.cb.piermont.com>
 <569AE5CD.1090209@msapiro.net>
Message-ID: <20160116201308.4afc99f4@jabberwock.cb.piermont.com>

On Sat, 16 Jan 2016 16:52:29 -0800 Mark Sapiro <mark at msapiro.net>
wrote:
> On 01/16/2016 04:02 PM, Perry E. Metzger wrote:
> > 
> > I have direct evidence that the asshats are now using "+" strings
> > after the main address that are not strictly numeric. They seem to
> > have responded to the simple ways of stopping them.
> 
> 
> I haven't seen any like that yet. The regexp I use is ^.*\+.*\d{3,}@
> which will block anything between '+' and '@' as long as it ends
> with 3 digits.
> 
> Please provide some examples. If there is any discernable pattern,
> it might be blockable without impacting real subscribers.

I don't have a lot of examples (haven't been saving them as I nuke
stuff out of the postfix queue) but I just nuked one aimed at what
I could characterize as user\+[a-z]+[0-9]@gmail.com

In each case, the instances in the queue had this or equivalent back
from google:

(host gmail-smtp-in.l.google.com[74.125.29.26] said: 450-4.2.1 The
user you are trying to contact is receiving mail at a rate that
450-4.2.1 prevents additional messages from being delivered. Please
resend your 450-4.2.1 message at a later time. If the user is able to
receive mail at that 450-4.2.1 time, your message will be delivered.
For more information, please 450-4.2.1 visit 450 4.2.1
https://support.google.com/mail/answer/6592 75si22222236qgm.43 -
gsmtp (in reply to RCPT TO command))

I already had a regexp in to nuke everything aimed at a post-+
section with just digits. I'm reluctant to go further than that
immediately, although I suspect trailing digits after alphabeticals
are also unlikely to be real submailboxes.

> Other possibilities are disabling web subscribe all together or
> installing some kind of captcha (ugh) in the page. Experience shows
> that SUBSCRIBE_FORM_SECRET doesn't stop them unless perhaps
> SUBSCRIBE_FORM_MIN_TIME is set long enough that it becomes a
> problem for real subscribers.

Would it be hard to add optional recaptcha support for the pages with
forms in a future release? That would probably prevent most such
games and it doesn't seem so bad.

Perry
-- 
Perry E. Metzger		perry at piermont.com

From mark at msapiro.net  Sat Jan 16 21:27:00 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 16 Jan 2016 18:27:00 -0800
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <20160116201308.4afc99f4@jabberwock.cb.piermont.com>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <56953DBE.5060708@msapiro.net> <n78csq$ssq$1@ger.gmane.org>
 <20160116190258.6b935766@jabberwock.cb.piermont.com>
 <569AE5CD.1090209@msapiro.net>
 <20160116201308.4afc99f4@jabberwock.cb.piermont.com>
Message-ID: <569AFBF4.3040206@msapiro.net>

On 01/16/2016 05:13 PM, Perry E. Metzger wrote:
> On Sat, 16 Jan 2016 16:52:29 -0800 Mark Sapiro <mark at msapiro.net>
> wrote:
>>
>> Please provide some examples. If there is any discernable pattern,
>> it might be blockable without impacting real subscribers.
> 
> I don't have a lot of examples (haven't been saving them as I nuke
> stuff out of the postfix queue) but I just nuked one aimed at what
> I could characterize as user\+[a-z]+[0-9]@gmail.com


Is that just one digit? If so, that's a tough one to separate from legit
ones, but in what I've seen the 'user' part doesn't vary all that much
so you could focus on that.



> I already had a regexp in to nuke everything aimed at a post-+
> section with just digits. I'm reluctant to go further than that
> immediately, although I suspect trailing digits after alphabeticals
> are also unlikely to be real submailboxes.


I certainly agree that 3 or more and perhaps even 2 trailing digits is
unlikely, but a single trailing digit is likely too agressive.

Just for info, there are a total of 60580 subscriptions with 41715
unique email addresses to lists @python.org. Of the 41745, 620 have a
'+' in the address and of those, 24 have at least one digit immediately
before the @. of the 25, 15 are subscriptions for the nabble.com
archiving service and 2 are subscriptions for a googlegroups archiving
service. Of the remaining 7, there are 2 +python3000, 1 +python-3000 and
1 +py3000, and the other 3 have only one digit including one +mm3.

So, in this installation at least, \+.*[0-9]@ is rare and most of even
those are archivers with multiple digits or are probably motivated by
the nature of the installation.


> Would it be hard to add optional recaptcha support for the pages with
> forms in a future release? That would probably prevent most such
> games and it doesn't seem so bad.


I hate them. I'd really have to be convinced of the need.

In any case, you already have that ability. The only place I think you
need it is the subscribe form on the listinfo page. It is a simple
matter to install a sitewide custom listinfo template in (for example
for english) $prefix/templates/site/en/listinfo.html or to make a list
specific one through the web admin UI.

Since reCAPTCHA in particular requires registration and perhaps other
steps outside of Mailman, I don't think it's too burdensome that the
Mailman part requires making a custom template.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From perry at piermont.com  Sun Jan 17 09:34:20 2016
From: perry at piermont.com (Perry E. Metzger)
Date: Sun, 17 Jan 2016 09:34:20 -0500
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <569AFBF4.3040206@msapiro.net>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <56953DBE.5060708@msapiro.net> <n78csq$ssq$1@ger.gmane.org>
 <20160116190258.6b935766@jabberwock.cb.piermont.com>
 <569AE5CD.1090209@msapiro.net>
 <20160116201308.4afc99f4@jabberwock.cb.piermont.com>
 <569AFBF4.3040206@msapiro.net>
Message-ID: <20160117093420.34066bdd@jabberwock.cb.piermont.com>

On Sat, 16 Jan 2016 18:27:00 -0800 Mark Sapiro <mark at msapiro.net>
wrote:
> > Would it be hard to add optional recaptcha support for the pages
> > with forms in a future release? That would probably prevent most
> > such games and it doesn't seem so bad.
> 
> I hate them. I'd really have to be convinced of the need.
> 
> In any case, you already have that ability. The only place I think
> you need it is the subscribe form on the listinfo page. It is a
> simple matter to install a sitewide custom listinfo template in
> (for example for english) $prefix/templates/site/en/listinfo.html
> or to make a list specific one through the web admin UI.
> 
> Since reCAPTCHA in particular requires registration and perhaps
> other steps outside of Mailman, I don't think it's too burdensome
> that the Mailman part requires making a custom template.

Mostly it just requires registration. Doing a custom template is
probably fine for someone like me who is able to deal with the
technical steps involved but it might be too much of a burden for
many users. If such code was contributed would you be willing to
include it? You don't have to use it yourself of course if you find
it overly vile.

Perry
-- 
Perry E. Metzger		perry at piermont.com

From jaybird at bluegrasspals.com  Sun Jan 17 11:20:57 2016
From: jaybird at bluegrasspals.com (Jayson Smith)
Date: Sun, 17 Jan 2016 11:20:57 -0500
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <20160117093420.34066bdd@jabberwock.cb.piermont.com>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <56953DBE.5060708@msapiro.net> <n78csq$ssq$1@ger.gmane.org>
 <20160116190258.6b935766@jabberwock.cb.piermont.com>
 <569AE5CD.1090209@msapiro.net>
 <20160116201308.4afc99f4@jabberwock.cb.piermont.com>
 <569AFBF4.3040206@msapiro.net>
 <20160117093420.34066bdd@jabberwock.cb.piermont.com>
Message-ID: <569BBF69.40803@bluegrasspals.com>

Hi,

Just my $0.02 worth, but as a blind person, were I going to implement a 
CAPTCHA, I would probably write some sort of simple math problem system 
or something. Besides the benefit of not needing registration, this 
CAPTCHA can be solved by anybody who can solve simple problems. 
ReCaptcha has a visual challenge and an audio challenge. That's great 
for those of us who are blind. However, as far as I know, they offer no 
alternative for people who are deaf blind.

Jayson

On 1/17/2016 9:34 AM, Perry E. Metzger wrote:
> On Sat, 16 Jan 2016 18:27:00 -0800 Mark Sapiro <mark at msapiro.net>
> wrote:
>>> Would it be hard to add optional recaptcha support for the pages
>>> with forms in a future release? That would probably prevent most
>>> such games and it doesn't seem so bad.
>> I hate them. I'd really have to be convinced of the need.
>>
>> In any case, you already have that ability. The only place I think
>> you need it is the subscribe form on the listinfo page. It is a
>> simple matter to install a sitewide custom listinfo template in
>> (for example for english) $prefix/templates/site/en/listinfo.html
>> or to make a list specific one through the web admin UI.
>>
>> Since reCAPTCHA in particular requires registration and perhaps
>> other steps outside of Mailman, I don't think it's too burdensome
>> that the Mailman part requires making a custom template.
> Mostly it just requires registration. Doing a custom template is
> probably fine for someone like me who is able to deal with the
> technical steps involved but it might be too much of a burden for
> many users. If such code was contributed would you be willing to
> include it? You don't have to use it yourself of course if you find
> it overly vile.
>
> Perry


From mailman at ravexdata.com  Sat Jan 16 14:58:22 2016
From: mailman at ravexdata.com (Paul Stuffins)
Date: Sat, 16 Jan 2016 19:58:22 +0000
Subject: [Mailman-Users] Mailman with Gmail.
Message-ID: <CAO-kBwftivwtQCprrQsLcZ2x9O57484tME7H89++HaqJTeqJEg@mail.gmail.com>

Hi Everybody,

We use Google Apps as our email platform but would like to install and run
our own mailing list software as Google's offering does not really suit us.

Can we install mailman on a VPS, and use it with Google Apps? I have tried
googling and not got a definitive answer, hence emailing the mailing list.

Many Thanks
Paul

From mark at msapiro.net  Sun Jan 17 12:34:35 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 17 Jan 2016 09:34:35 -0800
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <20160117093420.34066bdd@jabberwock.cb.piermont.com>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <56953DBE.5060708@msapiro.net> <n78csq$ssq$1@ger.gmane.org>
 <20160116190258.6b935766@jabberwock.cb.piermont.com>
 <569AE5CD.1090209@msapiro.net>
 <20160116201308.4afc99f4@jabberwock.cb.piermont.com>
 <569AFBF4.3040206@msapiro.net>
 <20160117093420.34066bdd@jabberwock.cb.piermont.com>
Message-ID: <569BD0AB.4090404@msapiro.net>

On 01/17/2016 06:34 AM, Perry E. Metzger wrote:
> 
> Mostly it just requires registration. Doing a custom template is
> probably fine for someone like me who is able to deal with the
> technical steps involved but it might be too much of a burden for
> many users. If such code was contributed would you be willing to
> include it? You don't have to use it yourself of course if you find
> it overly vile.


I would look at it, but it's not likely I will include it. Keep in mind
that MM 2.1 is end of life. By the time such code would be released, MM
3.1 with a migration path for MM 2.1 users would probably be available

I think a more reasonable approach would be to develop a listinfo.html
template and documentation on how to use it, and I would put that in the
contrib directory of the release, or you could put it on the wiki
<http://wiki.list.org/>.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Sun Jan 17 14:53:38 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 17 Jan 2016 11:53:38 -0800
Subject: [Mailman-Users] Mailman with Gmail.
In-Reply-To: <CAO-kBwftivwtQCprrQsLcZ2x9O57484tME7H89++HaqJTeqJEg@mail.gmail.com>
References: <CAO-kBwftivwtQCprrQsLcZ2x9O57484tME7H89++HaqJTeqJEg@mail.gmail.com>
Message-ID: <569BF142.4040408@msapiro.net>

On 01/16/2016 11:58 AM, Paul Stuffins wrote:
> 
> Can we install mailman on a VPS, and use it with Google Apps? I have tried
> googling and not got a definitive answer, hence emailing the mailing list.


You should be able to install Mailman on a VPS, either from source or,
depending on the particular OS on the VPS, from a distribution package
(at least for MM 2.1).

I don't understand what you mean by "use it with Google Apps". Mailman
is normally installed on a server along with an MTA and a web server,
see <http://www.list.org/mailman-install/> for more info on MM 2.1 or
<http://wiki.list.org/DOC/Mailman3> for MM 3.

Can you provide more specifics as to what you are looking for?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From benfell at parts-unknown.org  Sun Jan 17 15:15:53 2016
From: benfell at parts-unknown.org (David Benfell)
Date: Sun, 17 Jan 2016 12:15:53 -0800
Subject: [Mailman-Users] Mailman with Gmail.
In-Reply-To: <569BF142.4040408@msapiro.net>
References: <CAO-kBwftivwtQCprrQsLcZ2x9O57484tME7H89++HaqJTeqJEg@mail.gmail.com>
 <569BF142.4040408@msapiro.net>
Message-ID: <569BF679.4050005@parts-unknown.org>



On 01/17/2016 11:53 AM, Mark Sapiro wrote:
> On 01/16/2016 11:58 AM, Paul Stuffins wrote:
>> Can we install mailman on a VPS, and use it with Google Apps? I have tried
>> googling and not got a definitive answer, hence emailing the mailing list.
> I don't understand what you mean by "use it with Google Apps". Mailman
> is normally installed on a server along with an MTA and a web server,
> see <http://www.list.org/mailman-install/> for more info on MM 2.1 or
> <http://wiki.list.org/DOC/Mailman3> for MM 3.
>
> Can you provide more specifics as to what you are looking for?
I think what's going on here is that the OP wants to use gmail as their
MTA (honestly, I don't see the attraction). I assume this *might* be
possible by using a local postfix on the VPS as a secondary MX server to
the gmail primary.

-- 
David Benfell, Ph.D.
benfell at parts-unknown.org


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160117/a7a4d6cb/attachment.sig>

From mailman at ravexdata.com  Sun Jan 17 16:02:10 2016
From: mailman at ravexdata.com (Paul Stuffins)
Date: Sun, 17 Jan 2016 21:02:10 +0000
Subject: [Mailman-Users] Mailman with Gmail.
In-Reply-To: <569BF679.4050005@parts-unknown.org>
References: <CAO-kBwftivwtQCprrQsLcZ2x9O57484tME7H89++HaqJTeqJEg@mail.gmail.com>
 <569BF142.4040408@msapiro.net> <569BF679.4050005@parts-unknown.org>
Message-ID: <569C0152.3040406@ravexdata.com>

On 17/01/2016 20:15, David Benfell wrote:
>
> On 01/17/2016 11:53 AM, Mark Sapiro wrote:
>> On 01/16/2016 11:58 AM, Paul Stuffins wrote:
>>> Can we install mailman on a VPS, and use it with Google Apps? I have tried
>>> googling and not got a definitive answer, hence emailing the mailing list.
>> I don't understand what you mean by "use it with Google Apps". Mailman
>> is normally installed on a server along with an MTA and a web server,
>> see <http://www.list.org/mailman-install/> for more info on MM 2.1 or
>> <http://wiki.list.org/DOC/Mailman3> for MM 3.
>>
>> Can you provide more specifics as to what you are looking for?
> I think what's going on here is that the OP wants to use gmail as their
> MTA (honestly, I don't see the attraction). I assume this *might* be
> possible by using a local postfix on the VPS as a secondary MX server to
> the gmail primary.

This is what I was trying to say. I would prefer to use Google as the 
MTA for mailman as I do not really want to run and manage my own MTA.

Many Thanks
Paul

From mark at msapiro.net  Sun Jan 17 16:22:08 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 17 Jan 2016 13:22:08 -0800
Subject: [Mailman-Users] Mailman with Gmail.
In-Reply-To: <569BF679.4050005@parts-unknown.org>
References: <CAO-kBwftivwtQCprrQsLcZ2x9O57484tME7H89++HaqJTeqJEg@mail.gmail.com>
 <569BF142.4040408@msapiro.net> <569BF679.4050005@parts-unknown.org>
Message-ID: <569C0600.7030701@msapiro.net>

On 01/17/2016 12:15 PM, David Benfell wrote:

> I think what's going on here is that the OP wants to use gmail as their
> MTA (honestly, I don't see the attraction). I assume this *might* be
> possible by using a local postfix on the VPS as a secondary MX server to
> the gmail primary.


There are various ways to do this. See, e.g., the FAQ at
<http://wiki.list.org/x/17891892> about using fetchmail to get incoming
mail from and ISP to mailman.

For outbound mail, the outbound server and port are already configurable
via the SMTPHOST and SMTPPORT configuration settings, and if the server
requires authentication, there's a simple patch at
<https://bugs.launchpad.net/mailman/+bug/558281>.


-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160117/c05c9480/attachment.sig>

From perry at piermont.com  Sun Jan 17 20:50:19 2016
From: perry at piermont.com (Perry E. Metzger)
Date: Sun, 17 Jan 2016 20:50:19 -0500
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <569BD0AB.4090404@msapiro.net>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <56953DBE.5060708@msapiro.net> <n78csq$ssq$1@ger.gmane.org>
 <20160116190258.6b935766@jabberwock.cb.piermont.com>
 <569AE5CD.1090209@msapiro.net>
 <20160116201308.4afc99f4@jabberwock.cb.piermont.com>
 <569AFBF4.3040206@msapiro.net>
 <20160117093420.34066bdd@jabberwock.cb.piermont.com>
 <569BD0AB.4090404@msapiro.net>
Message-ID: <20160117205019.08d10c4e@jabberwock.cb.piermont.com>

On Sun, 17 Jan 2016 09:34:35 -0800 Mark Sapiro <mark at msapiro.net>
wrote:
> On 01/17/2016 06:34 AM, Perry E. Metzger wrote:
> > 
> > Mostly it just requires registration. Doing a custom template is
> > probably fine for someone like me who is able to deal with the
> > technical steps involved but it might be too much of a burden for
> > many users. If such code was contributed would you be willing to
> > include it? You don't have to use it yourself of course if you
> > find it overly vile.
> 
> I would look at it, but it's not likely I will include it. Keep in
> mind that MM 2.1 is end of life. By the time such code would be
> released, MM 3.1 with a migration path for MM 2.1 users would
> probably be available

What if the patches targeted MM 3?

Perry
-- 
Perry E. Metzger		perry at piermont.com

From mark at msapiro.net  Sun Jan 17 23:29:59 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 17 Jan 2016 20:29:59 -0800
Subject: [Mailman-Users] Handling bogus subscribe requests
In-Reply-To: <20160117205019.08d10c4e@jabberwock.cb.piermont.com>
References: <alpine.LRH.2.03.1601120053320.31359@andrew.triumf.ca>
 <56953DBE.5060708@msapiro.net> <n78csq$ssq$1@ger.gmane.org>
 <20160116190258.6b935766@jabberwock.cb.piermont.com>
 <569AE5CD.1090209@msapiro.net>
 <20160116201308.4afc99f4@jabberwock.cb.piermont.com>
 <569AFBF4.3040206@msapiro.net>
 <20160117093420.34066bdd@jabberwock.cb.piermont.com>
 <569BD0AB.4090404@msapiro.net>
 <20160117205019.08d10c4e@jabberwock.cb.piermont.com>
Message-ID: <569C6A47.6060608@msapiro.net>

On 01/17/2016 05:50 PM, Perry E. Metzger wrote:
> On Sun, 17 Jan 2016 09:34:35 -0800 Mark Sapiro <mark at msapiro.net>
> wrote:
>>
>> I would look at it, but it's not likely I will include it. Keep in
>> mind that MM 2.1 is end of life. By the time such code would be
>> released, MM 3.1 with a migration path for MM 2.1 users would
>> probably be available
> 
> What if the patches targeted MM 3?


In that case, they would be against Postorius. See
<http://postorius.readthedocs.org/en/latest/development.html> although
that is out of date as it refers to Postorius being hosted on Launchpad,
and it is now on gitlab <https://gitlab.com/mailman/postorius>.

The best way to contact Postorius developers is either the
mailman-developers at python.org list or the freenode irc channel #mailman.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From odhiambo at gmail.com  Mon Jan 18 06:30:42 2016
From: odhiambo at gmail.com (Odhiambo Washington)
Date: Mon, 18 Jan 2016 14:30:42 +0300
Subject: [Mailman-Users] Installing Mailman3 on FreeBSD
Message-ID: <CAAdA2WMBhWLo2XNC6=3+z+-Ra=B4agAMZZB=JJaFOHL6eddZTA@mail.gmail.com>

My intention is to install MM3 on FreeBSD and integrate it with Exim as the
MTA. Now, it looks like FreeBSD isn't the OS of choice for many Mailman
developers and users. This has been my wish since the beta versions but I
have never succeeded, either because of getting frustrated midway and
putting the effort in abbeyance or just because there are no resources to
help guide me.

Today I googled and found http://mailman-bundler.readthedocs.org/en/latest/

So I am trying that out and I get stuck once again!




(venv)(14:06:31 </opt/mailman3/mailman-bundler>) 0 # pip install zc.buildout
You are using pip version 7.0.3, however version 7.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting zc.buildout
  Using cached zc.buildout-2.5.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools>=8.0
in /opt/mailman3/venv/lib/python2.7/site-packages (from zc.buildout)
Installing collected packages: zc.buildout
Successfully installed zc.buildout-2.5.0
(venv)(14:06:44 </opt/mailman3/mailman-bundler>) 0 # buildout
Creating directory '/opt/mailman3/mailman-bundler/eggs'.
Creating directory '/opt/mailman3/mailman-bundler/bin'.
Creating directory '/opt/mailman3/mailman-bundler/parts'.
Creating directory '/opt/mailman3/mailman-bundler/develop-eggs'.
Develop: '/opt/mailman3/mailman-bundler/.'
warning: no files found matching '*.in' under directory 'mailman_bundler'
warning: no files found matching '*.in' under directory 'deployment'
warning: no files found matching 'deployment/mailman-web.logrotate.conf'
While:
  Installing.
  Getting section main.
  Initializing section main.
  Getting option main:eggs.
  Getting section mailman-web.
  Initializing section mailman-web.
  Installing recipe djangorecipe.
  Getting distribution for 'djangorecipe'.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/buildout.py",
line 1992, in main
    getattr(buildout, command)(args)
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/buildout.py",
line 550, in install
    [self[part]['recipe'] for part in install_parts]
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/buildout.py",
line 1135, in __getitem__
    options._initialize()
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/buildout.py",
line 1233, in _initialize
    self._dosub(k, v)
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/buildout.py",
line 1281, in _dosub
    v = '$$'.join([self._sub(s, seen) for s in v.split('$$')])
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/buildout.py",
line 1345, in _sub
    v = self.buildout[section].get(option, None, seen)
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/buildout.py",
line 1135, in __getitem__
    options._initialize()
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/buildout.py",
line 1239, in _initialize
    self.initialize()
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/buildout.py",
line 1245, in initialize
    recipe_class = _install_and_load(reqs, 'zc.buildout', entry, buildout)
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/buildout.py",
line 1199, in _install_and_load
    allow_hosts=buildout._allow_hosts
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/easy_install.py",
line 883, in install
    return installer.install(specs, working_set)
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/easy_install.py",
line 664, in install
    for_buildout_run=for_buildout_run):
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/easy_install.py",
line 496, in _get_dist
    dist, avail = self._satisfied(requirement)
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/easy_install.py",
line 253, in _satisfied
    return None, self._obtain(req, source)
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/zc/buildout/easy_install.py",
line 421, in _obtain
    if index.obtain(requirement) is None:
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/setuptools/package_index.py",
line 443, in obtain
    self.find_packages(requirement)
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/setuptools/package_index.py",
line 427, in find_packages
    self.scan_url(self.index_url + requirement.unsafe_name+'/')
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/setuptools/package_index.py",
line 761, in scan_url
    self.process_url(url, True)
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/setuptools/package_index.py",
line 304, in process_url
    f = self.open_url(url, "Download error on %s: %%s -- Some packages may
not be found!" % url)
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/setuptools/package_index.py",
line 700, in open_url
    return open_with_auth(url, self.opener)
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/setuptools/package_index.py",
line 894, in _socket_timeout
    return func(*args, **kwargs)
  File
"/opt/mailman3/venv/lib/python2.7/site-packages/setuptools/package_index.py",
line 1007, in open_with_auth
    fp = opener(request)
  File "/usr/local/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/local/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/local/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python2.7/urllib2.py", line 1227, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/local/lib/python2.7/urllib2.py", line 1200, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/local/lib/python2.7/httplib.py", line 1132, in getresponse
    response.begin()
  File "/usr/local/lib/python2.7/httplib.py", line 453, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python2.7/httplib.py", line 409, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/local/lib/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
error: [Errno 54] Connection reset by peer
(venv)(14:06:57 </opt/mailman3/mailman-bundler>) 0 #



-- 
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft."

From mark at msapiro.net  Mon Jan 18 12:22:23 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 18 Jan 2016 09:22:23 -0800
Subject: [Mailman-Users] Installing Mailman3 on FreeBSD
In-Reply-To: <CAAdA2WMBhWLo2XNC6=3+z+-Ra=B4agAMZZB=JJaFOHL6eddZTA@mail.gmail.com>
References: <CAAdA2WMBhWLo2XNC6=3+z+-Ra=B4agAMZZB=JJaFOHL6eddZTA@mail.gmail.com>
Message-ID: <569D1F4F.8000509@msapiro.net>

On 01/18/2016 03:30 AM, Odhiambo Washington wrote:
> My intention is to install MM3 on FreeBSD and integrate it with Exim as the
> MTA. Now, it looks like FreeBSD isn't the OS of choice for many Mailman
> developers and users. This has been my wish since the beta versions but I
> have never succeeded, either because of getting frustrated midway and
> putting the effort in abbeyance or just because there are no resources to
> help guide me.
> 
> Today I googled and found http://mailman-bundler.readthedocs.org/en/latest/
> 
> So I am trying that out and I get stuck once again!


There are people who've installed MM 3 on Darwin, and I think there's no
reason why it shouldn't install on FreeBSD. Exim is supported.

That said, please join mailman-developers at python.org
<http://mail.python.org/mailman/listinfo/mailman-developers> and post
your specific issue there.

We are working on setting up mailman3-users at mailman3.org (running on
MM 3 of course), but there are still infrastructure issues to work out
before that list will be operational.

In the mean time the best list for Mailman 3 and particularly
HyperKitty, Postorius and MailmanBundler issues is
mailman-developers at python.org or you can try the #mailman IRC channel on
freenode.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From stephen at xemacs.org  Tue Jan 19 00:36:00 2016
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 19 Jan 2016 14:36:00 +0900
Subject: [Mailman-Users]  Installing Mailman3 on FreeBSD
In-Reply-To: <CAAdA2WMBhWLo2XNC6=3+z+-Ra=B4agAMZZB=JJaFOHL6eddZTA@mail.gmail.com>
References: <CAAdA2WMBhWLo2XNC6=3+z+-Ra=B4agAMZZB=JJaFOHL6eddZTA@mail.gmail.com>
Message-ID: <22173.52032.911780.831578@turnbull.sk.tsukuba.ac.jp>

Odhiambo Washington writes:

 > So I am trying that out and I get stuck once again!

 > in /opt/mailman3/venv/lib/python2.7/site-packages (from
 > zc.buildout)

Mailman 3 runs only on Python3 as far as I know.  That's definitely
going to be true for 3.1 and later.  Since the output below refers to
Django, I guess it's Postorius or HyperKitty that is being installed
here, so 2.7 is OK.  However, you should make sure that the
appropriate interpreter and stdlib are being used.

 > You are using pip version 7.0.3, however version 7.1.2 is available.
 > You should consider upgrading via the 'pip install --upgrade pip' command.

Following the recommendation here might be helpfu since you
encountered a network error.

[expected output omitted]

 > While:
 >   Installing.
 >   Getting section main.
 >   Initializing section main.
 >   Getting option main:eggs.
 >   Getting section mailman-web.
 >   Initializing section mailman-web.
 >   Installing recipe djangorecipe.
 >   Getting distribution for 'djangorecipe'.

[traceback omitted]

 > error: [Errno 54] Connection reset by peer

That's a network problem of some kind.  It may be an error in the
bundler (trying to contact kgbvax or something), but more likely it's
a hiccup in the Internet, or a firewall issue.


From bcanty at apsa.org  Tue Jan 19 13:54:45 2016
From: bcanty at apsa.org (Brian Canty)
Date: Tue, 19 Jan 2016 18:54:45 +0000
Subject: [Mailman-Users] Configuration
Message-ID: <6CCC345339CE2D4085B5D3D82F7B3A9F0D51A48A@APSASBS2K11.APSA.local>

Hello Everyone -

It seems that a lot of my emails are being converted to attachments and I was wondering if there is a setting that I am missing.

Any help is appreciated.

Sincerely,


Brian Canty
Computer Information Systems
The American Psychoanalytic Association
309 East 49th St., New York, NY 10017
212-752-0450 ext. 17 | apsa.org<http://www.apsa.org/>
[https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcQQ7Kib3E-SNRpX6f3JkeBuASADejyfQMFOfbJnoHFXYBApgbDJ]<https://twitter.com/psychoanalysis_>     [https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcRk4KOTDFQ-zOC0x0HiCqzRIVhsZaxApOlF_zcpoW9sI6d4AQ16] <https://www.facebook.com/American.psychoanalysis>
[cid:image003.png at 01D152C0.F460FB50]<http://www.apsa.org/>


From gabriel at unseen.is  Wed Jan 20 14:38:34 2016
From: gabriel at unseen.is (gabriel)
Date: Wed, 20 Jan 2016 20:38:34 +0100
Subject: [Mailman-Users] diacritics in from header with from_is_list set to
 munge
Message-ID: <20160120193601.GB24047@imaginationland>



hello,

i just noticed several times now that a bunch of users (most of them 
subscribed with their yahoo email address) get bounced off my list 
when the senders address contains diacritical marks in their 
additional text field describing the name e.g 

  From: "val?rie" <valerie at somesomain.com>


the problem here occurs when setting /munge from/ in from_is_list. the 
text between the quotes is merged with the string "via " and then the 
name of the list e.g.

  From: "val?rie via mylist <this at is.my.list>"


but in mailman 2.1.20 this merging breaks presumably because of the 
character encoding.

what happens is that the From string in the mailman list header is no 
longer a valid email address but two totally mixed up. the then have 
the list name split accross both addresses and the domain is not 
correct either.

this obviously violates yahoo's dmarc policy and therefore users start 
to bounce.



what can i do?


thanks in advance,
gabriel



From mark at msapiro.net  Wed Jan 20 17:10:09 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 20 Jan 2016 14:10:09 -0800
Subject: [Mailman-Users] Configuration
In-Reply-To: <6CCC345339CE2D4085B5D3D82F7B3A9F0D51A48A@APSASBS2K11.APSA.local>
References: <6CCC345339CE2D4085B5D3D82F7B3A9F0D51A48A@APSASBS2K11.APSA.local>
Message-ID: <56A005C1.8070609@msapiro.net>

On 01/19/2016 10:54 AM, Brian Canty wrote:
> 
> It seems that a lot of my emails are being converted to attachments and I was wondering if there is a setting that I am missing.


See the FAQ at <http://wiki.list.org/x/4030707>.

In addition to the information there, Mailman will convert every message
to an attachment if the General Options -> from_is_list setting is Wrap
Message and will convert messages From: domains with DMARC policy of
reject and possibly quarantine if Privacy options... -> Sender filters
-> dmarc_moderation_action is Wrap Message.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Wed Jan 20 18:05:58 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 20 Jan 2016 15:05:58 -0800
Subject: [Mailman-Users] diacritics in from header with from_is_list set
 to munge
In-Reply-To: <20160120193601.GB24047@imaginationland>
References: <20160120193601.GB24047@imaginationland>
Message-ID: <56A012D6.6030107@msapiro.net>

On 01/20/2016 11:38 AM, gabriel wrote:
> 
> i just noticed several times now that a bunch of users (most of them 
> subscribed with their yahoo email address) get bounced off my list 
> when the senders address contains diacritical marks in their 
> additional text field describing the name e.g 
> 
>   From: "val?rie" <valerie at somesomain.com>
> 
> 
> the problem here occurs when setting /munge from/ in from_is_list. the 
> text between the quotes is merged with the string "via " and then the 
> name of the list e.g.
> 
>   From: "val?rie via mylist <this at is.my.list>"


I am unable to duplicate this in my development version which I think is
no different from 2.1.20 in this regard.

I have tried posting a message with the exact From: header

From: "val?rie" <valerie at somesomain.com>

with the ? encoded as utf-8 and in another post encoded as iso-8859-1
and in both cases the message delivered from Mailman has

From: val?rie via List1 <list1 at msapiro.net>
Reply-To: val?rie <valerie at somesomain.com>

with no quotes and the ? encoded as the same encoding as the posted message.

This could be a difference in the underlying Python email package. Mine
is Python 2.7.9 with email 4.0.3. What is yours? (do the following)

$ python
Python 2.7.9 (default, Apr  2 2015, 15:33:21)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import email
>>> email.__version__
'4.0.3'

Also, it might help if you post the exact From: and Reply-To: headers in
the garbled message.


> but in mailman 2.1.20 this merging breaks presumably because of the 
> character encoding.
> 
> what happens is that the From string in the mailman list header is no 
> longer a valid email address but two totally mixed up. the then have 
> the list name split accross both addresses and the domain is not 
> correct either.


I'm confused as to what you are saying here and how it relates to the

  From: "val?rie via mylist <this at is.my.list>"

line in your post. Is that what the From: in the delivered message looks
like or just some representation of what you think it should look like?

In any case, the From: I see is

From: val?rie via List1 <list1 at msapiro.net>

which is technically not valid because it contains a non-ascii
character, but even so, the message with that exact From: header is
accepted by Yahoo and delivered to my Yahoo address.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Wed Jan 20 20:24:21 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 20 Jan 2016 17:24:21 -0800
Subject: [Mailman-Users] diacritics in from header with from_is_list set
 to munge
In-Reply-To: <22176.9477.411589.213445@turnbull.sk.tsukuba.ac.jp>
References: <20160120193601.GB24047@imaginationland>
 <56A012D6.6030107@msapiro.net>
 <22176.9477.411589.213445@turnbull.sk.tsukuba.ac.jp>
Message-ID: <56A03345.4040805@msapiro.net>

On 01/20/2016 04:23 PM, Stephen J. Turnbull wrote:
> Mark Sapiro writes:
> 
>  > In any case, the From: I see is
>  > 
>  > From: val?rie via List1 <list1 at msapiro.net>
>  > 
>  > which is technically not valid because it contains a non-ascii
>  > character, but even so, the message with that exact From: header is
>  > accepted by Yahoo and delivered to my Yahoo address.
> 
> Really?  It's not that you're looking at this in your MUA and it's
> being silently MIME-encoded and -decoded at the MTA/MUA boundary?
> ISTR that Yahoo! is quite sensitive to non-ASCII headers.


Really.  In my initial test, I sent from mutt and mutt kindly RFC2047
encoded the name for me, so I switched to testing using Mailman's
bin/inject to post the raw message from a file, and I looked at the raw
message source of the message I received in my Yahoo inbox and the header is

From: val??rie via List1 <list1 at msapiro.net>

where ?? is the utf-8 encoding of e-acute.


> One thing I've seen is this:
> 
>     in MUA UI:    From: auser at example.com (non-ASCII display name)
> 
>     on the wire:  From: auser at example.com =?utf-8?Q?(non-ASCII display name)?=
> 
> which gets bounced at many sites because the MIME-word hides the
> comment delimiters from the receiver's parser, which typically rejects
> with "= not allowed here" or something like that.  Is it possible that
> Mailman is doing something like that?


Yes, I see where that would be an issue, but as far as Mailman is
concerned, this would only be an issue if the incoming post had that
From: header.

With a message containing

From: auser at example.com =?utf-8?Q?(non-ASCII_display_name)?=

the code Mailman uses to parse the From: will return
'non-ASCII_display_name' as the display name and
'auser at example.com=?utf-8?Q??=' as the address. This is clearly wrong,
but then the the encoded header is not RFC2047 compliant. RFC 2047, sec
5(2) is clear that an encoded-word in a comment does not include the
parentheses. It defines comment as

comment = "(" *(ctext / quoted-pair / comment / encoded-word) ")"

Mailman would never create a header like that.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From lists at boywithwings.co.uk  Wed Jan 20 17:16:57 2016
From: lists at boywithwings.co.uk (Leo Francisco)
Date: Wed, 20 Jan 2016 22:16:57 +0000
Subject: [Mailman-Users] Themeing/Skining Mailman
Message-ID: <56A00759.2060205@boywithwings.co.uk>

Hi all,

I was just wondering how easy it is to theme Mailman? I haven?t access
to an installation that I can poke around in myself.  Is it simple
HTML/CSS or is it a more complicated prospect?

Thanks in advance
Leo




From mark at msapiro.net  Wed Jan 20 20:35:49 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 20 Jan 2016 17:35:49 -0800
Subject: [Mailman-Users] Themeing/Skining Mailman
In-Reply-To: <56A00759.2060205@boywithwings.co.uk>
References: <56A00759.2060205@boywithwings.co.uk>
Message-ID: <56A035F5.6020106@msapiro.net>

On 01/20/2016 02:16 PM, Leo Francisco wrote:
> 
> I was just wondering how easy it is to theme Mailman? I haven?t access
> to an installation that I can poke around in myself.  Is it simple
> HTML/CSS or is it a more complicated prospect?


For Mailman 2.1.x (x >=19), see
<https://bugs.launchpad.net/mailman/+bug/1409396>.

For Mailman 3, the default web UI uses Django and I think there are
standard ways to theme it, but I'm not certain about how. Also, in MM 3,
the web UI is separate from the core mail list engine, and it is
possible to build your own web UI that communicates with the core via a
RESTful API.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From stephen at xemacs.org  Wed Jan 20 21:44:26 2016
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 21 Jan 2016 11:44:26 +0900
Subject: [Mailman-Users] diacritics in from header with from_is_list set
 to munge
In-Reply-To: <56A03345.4040805@msapiro.net>
References: <20160120193601.GB24047@imaginationland>
 <56A012D6.6030107@msapiro.net>
 <22176.9477.411589.213445@turnbull.sk.tsukuba.ac.jp>
 <56A03345.4040805@msapiro.net>
Message-ID: <22176.17930.403438.521009@turnbull.sk.tsukuba.ac.jp>

[Aside: I sent my previous message from the wrong address, and it was
rejected.  I am *not* resending it, since Mark quoted everything at
some point.  Nobody except me lost mail, and I deserved to! :-]

Mark Sapiro writes:
 > On 01/20/2016 04:23 PM, Stephen J. Turnbull wrote:

 > > Really?  It's not that you're looking at this in your MUA and it's
 > > being silently MIME-encoded and -decoded at the MTA/MUA boundary?
 > 
 > Really.

OK, just checking.

 > > One thing I've seen is this:
 > > 
 > >     in MUA UI:    From: auser at example.com (non-ASCII display name)
 > > 
 > >     on the wire:  From: auser at example.com (non-ASCII display name)
 >
 > Yes, I see where that would be an issue, but as far as Mailman is
 > concerned, this would only be an issue if the incoming post had that
 > From: header.

 > Mailman would never create a header like that.

Absent a bug in the email package.  But yes, I'm suggesting exactly
that there's a broken MUA out there sending something that doesn't
parse correctly, and the email package is failing to respect the "In
the face of ambiguity, refuse to guess" Zen.

We really need to see both the input and the output headers that
Mailman sends and receives.

Steve


From gabriel at unseen.is  Thu Jan 21 07:43:48 2016
From: gabriel at unseen.is (gabriel)
Date: Thu, 21 Jan 2016 13:43:48 +0100
Subject: [Mailman-Users] diacritics in from header with from_is_list set
 to munge
In-Reply-To: <22176.17930.403438.521009@turnbull.sk.tsukuba.ac.jp>
References: <20160120193601.GB24047@imaginationland>
 <56A012D6.6030107@msapiro.net>
 <22176.9477.411589.213445@turnbull.sk.tsukuba.ac.jp>
 <56A03345.4040805@msapiro.net>
 <22176.17930.403438.521009@turnbull.sk.tsukuba.ac.jp>
Message-ID: <20160121124348.GA30232@imaginationland>

On Thu, Jan 21, 2016 at 11:44:26AM +0900, Stephen J. Turnbull wrote:
> 
> Absent a bug in the email package.  But yes, I'm suggesting exactly
> that there's a broken MUA out there sending something that doesn't
> parse correctly, and the email package is failing to respect the "In
> the face of ambiguity, refuse to guess" Zen.

unfortunately i cannot check for the python version as i don't have 
commandline access to the server.

> 
> We really need to see both the input and the output headers that
> Mailman sends and receives.
> 

i can only send the headers that i receive over the list and those of the
messages that get bounced. i don't want to bother the whole list with debugging
messages.

so the message of users getting bounced look like (abbreviated):


--===============8546344873151602248==
Content-Type: message/rfc822
MIME-Version: 1.0

Delivered-To: mailman-mylist-bounces at some.server.org
Return-Path: <>
Received: from localhost (localhost [127.0.0.1])
  (ftp://ftp.isi.edu/in-notes/rfc1894.txt)
  by some.server.org with dsn; Wed, 20 Jan 2016 16:27:08 +0100
  id 000000000000002C.00000000569FA74C.00002C9F
From: "server.org postmaster" <postmaster at server.org>
To: mylist-bounces at lists.mydomain.org
Subject: NOTICE: mail delivery status.
Mime-Version: 1.0
Content-Type: multipart/report; report-type=delivery-status;
    boundary="=_courier_0"
Content-Transfer-Encoding: 7bit
Message-ID: <courier.00000000569FA74C.00002C9F at some.server.org>
Date: Wed, 20 Jan 2016 16:27:08 +0100

This is a MIME-formatted message.  If you see this text it means that your
E-mail software does not support MIME-formatted messages.

--=_courier_0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=us-ascii


This is a delivery status notification from some.server.org,
running the Courier mail server, version 0.75.0.

The original message was received on Wed, 20 Jan 2016 16:26:48 +0100
from some.server.org ([::1])

---------------------------------------------------------------------------

                           UNDELIVERABLE MAIL

Your message to the following recipients cannot be delivered:

 <xxxxxxxxxxxxx at yahoo.fr>:
     mx-eu.mail.am0.yahoodns.net [188.125.69.79]:
 >>> DATA
 <<< 554 Message not allowed - [299]

[...]


The original message follows as a separate attachment.


Received: from some.server.org ([::1])
  by some.server.org with ESMTP; Wed, 20 Jan 2016 16:26:48 +0100
  id 0000000000000066.00000000569FA738.000029C1
Delivered-To: mailman-mylist at some.server.org
Old-Return-Path: <valerie at mydomain.org>
MIME-Version: 1.0
Date: Wed, 20 Jan 2016 16:26:44 +0100
To: mylist at lists.mydomain.org
In-Reply-To: <7e498749e02d18656fd14393b90cdc38 at mydomain.org>
References: <7e498749e02d18656fd14393b90cdc38 at mydomain.org>
Message-ID: <a74fc9f373a741532c5f3d25404435ba at mydomain.org>
X-Sender: valerie at mydomain.org
Subject: [mylist] some subject
X-BeenThere: mylist at lists.mydomain.org
X-Mailman-Version: 2.1.20
Precedence: list
From: =?utf-8?q?Val=C3=A9rie/Something_via_mylist_=3Cmyli?=,
      =?utf-8?b?ZW5AbGlzdHMubXRtZWRpYS5vcmc+?=
Reply-To: valerie at mydomain.org
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"
Errors-To: mylist-bounces at lists.mydomain.org
Sender: "mylist" <mylist-bounces at lists.mydomain.org>




so the message on the list finally looks like this:


X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
X-Mozilla-Keys:                                                                                 
Return-Path: <mylist-bounces at lists.mydomain.org>
Delivered-To: mailman-mylist at some.server.org
Old-Return-Path: <valerie at mydomain.org>
MIME-Version: 1.0
Date: Wed, 20 Jan 2016 16:26:44 +0100
To: mylist at lists.mydomain.org
In-Reply-To: <7e498749e02d18656fd14393b90cdc38 at mydomain.org>
References: <7e498749e02d18656fd14393b90cdc38 at mydomain.org>
Message-ID: <a74fc9f373a741532c5f3d25404435ba at mydomain.org>
X-Sender: valerie at mydomain.org
Subject: [mylist] some subject
X-BeenThere: mylist at lists.mydomain.org
X-Mailman-Version: 2.1.20
Precedence: list
From:
  =?utf-8?q?Val=C3=A9rie/Something_via_mylist_=3Cmyli?=@some.server.org,
  =?utf-8?b?ZW5AbGlzdHMubXRtZWRpYS5vcmc+?=@some.server.org
Reply-To: valerie at mydomain.org
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"
Errors-To: mylist-bounces at lists.mydomain.org
Sender: "mylist" <mylist-bounces at lists.mydomain.org>





From stephen at xemacs.org  Thu Jan 21 11:06:29 2016
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 22 Jan 2016 01:06:29 +0900
Subject: [Mailman-Users] diacritics in from header with from_is_list set
 to munge
In-Reply-To: <20160121124348.GA30232@imaginationland>
References: <20160120193601.GB24047@imaginationland>
 <56A012D6.6030107@msapiro.net>
 <22176.9477.411589.213445@turnbull.sk.tsukuba.ac.jp>
 <56A03345.4040805@msapiro.net>
 <22176.17930.403438.521009@turnbull.sk.tsukuba.ac.jp>
 <20160121124348.GA30232@imaginationland>
Message-ID: <22177.517.393657.987628@turnbull.sk.tsukuba.ac.jp>

gabriel writes:
 > On Thu, Jan 21, 2016 at 11:44:26AM +0900, Stephen J. Turnbull wrote:
 > > 
 > > Absent a bug in the email package.  But yes, I'm suggesting exactly
 > > that there's a broken MUA out there sending something that doesn't
 > > parse correctly, and the email package is failing to respect the "In
 > > the face of ambiguity, refuse to guess" Zen.
 > 
 > unfortunately i cannot check for the python version as i don't have 
 > commandline access to the server.

Ah, that probably doesn't matter.  That remark was really directed to
Mark.

 > so the message of users getting bounced look like (abbreviated):

 > This is a delivery status notification from some.server.org,
 > running the Courier mail server, version 0.75.0.

FYI, bounce messages may or may not be useful, as some bounce programs
do mess with the mail they forward.  I know you probably can't do
anything about this, this is the best you can do.

 > From: =?utf-8?q?Val=C3=A9rie/Something_via_mylist_=3Cmyli?=,
 >       =?utf-8?b?ZW5AbGlzdHMubXRtZWRpYS5vcmc+?=

 > Sender: "mylist" <mylist-bounces at lists.mydomain.org>

So this has already been through Mailman.  We really really need to
see the mail as it was *before* Mailman handled it (possibly in the
mbox file in the archive, if you have it).

And then you've redacted stuff, and that may matter.  If you don't
want to send unredacted headers to a list with public archives, we
understand, but in that case you can and should send them to Mark (and
possibly me, but Mark is the real expert if you really want to send it
to the fewest people) privately.

I don't think this is a Mailman bug.  Mailman would not choose to send
using two different transfer encodings (Q in the first line, B in the
second).  So I suspect Mailman is just forwarding the garbage it
receives, or something downstream of Mailman is doing it.


From lists at boywithwings.co.uk  Thu Jan 21 12:02:13 2016
From: lists at boywithwings.co.uk (Leo Francisco)
Date: Thu, 21 Jan 2016 17:02:13 +0000
Subject: [Mailman-Users] Themeing/Skining Mailman
In-Reply-To: <56A035F5.6020106@msapiro.net>
References: <56A00759.2060205@boywithwings.co.uk>
 <56A035F5.6020106@msapiro.net>
Message-ID: <56A10F15.8030104@boywithwings.co.uk>

Thanks for the info, I will look into what version the group are
running. So it isn't really simple essentially?


On 21/01/16 01:35, Mark Sapiro wrote:
> On 01/20/2016 02:16 PM, Leo Francisco wrote:
>> I was just wondering how easy it is to theme Mailman? I haven?t access
>> to an installation that I can poke around in myself.  Is it simple
>> HTML/CSS or is it a more complicated prospect?
>
> For Mailman 2.1.x (x >=19), see
> <https://bugs.launchpad.net/mailman/+bug/1409396>.
>
> For Mailman 3, the default web UI uses Django and I think there are
> standard ways to theme it, but I'm not certain about how. Also, in MM 3,
> the web UI is separate from the core mail list engine, and it is
> possible to build your own web UI that communicates with the core via a
> RESTful API.
>


From gabriel at unseen.is  Thu Jan 21 12:27:42 2016
From: gabriel at unseen.is (gabriel)
Date: Thu, 21 Jan 2016 18:27:42 +0100
Subject: [Mailman-Users] diacritics in from header with from_is_list set
 to munge
In-Reply-To: <22177.517.393657.987628@turnbull.sk.tsukuba.ac.jp>
References: <20160120193601.GB24047@imaginationland>
 <56A012D6.6030107@msapiro.net>
 <22176.9477.411589.213445@turnbull.sk.tsukuba.ac.jp>
 <56A03345.4040805@msapiro.net>
 <22176.17930.403438.521009@turnbull.sk.tsukuba.ac.jp>
 <20160121124348.GA30232@imaginationland>
 <22177.517.393657.987628@turnbull.sk.tsukuba.ac.jp>
Message-ID: <20160121172740.GA20179@imaginationland>

On Fri, Jan 22, 2016 at 01:06:29AM +0900, Stephen J. Turnbull wrote:
> 
>  > From: =?utf-8?q?Val=C3=A9rie/Something_via_mylist_=3Cmyli?=,
>  >       =?utf-8?b?ZW5AbGlzdHMubXRtZWRpYS5vcmc+?=
> 
>  > Sender: "mylist" <mylist-bounces at lists.mydomain.org>
> 
> So this has already been through Mailman.  We really really need to
> see the mail as it was *before* Mailman handled it (possibly in the
> mbox file in the archive, if you have it).


That may not be so easy. in fact there is no archive of that list, but haven't
archived messages not been processed by mailman?

luckily exactly this message has been posted on one other list i have access to:

Return-Path: <otherlist-bounces at someotherserver.org>
X-Original-To: otherlist at someotherserver.org
Delivered-To: otherlist at someotherserver.org
Received: from onedomain.de (mail.onedomain.de
 [IPv6:2a01:4f8:140:4063::6])
 by someotherserver.org (Postfix) with ESMTP id DA99150142
 for <otherlist at someotherserver.org>; Wed, 20 Jan 2016 16:24:07 +0100 (CET)
Received: from eggmann.mydomain.org (eggmann.mydomain.org [178.63.68.97])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by onedomain.de (Postfix) with ESMTPS id E51E740036
 for <otherlist at onedomain.de>; Wed, 20 Jan 2016 16:24:07 +0100 (CET)
Received: from roundcube.mydomain.org ([::1])
 by eggmann.mydomain.org with ESMTP; Wed, 20 Jan 2016 16:24:05 +0100
 id 000000000002002A.00000000569FA695.00001D40
MIME-Version: 1.0
Date: Wed, 20 Jan 2016 16:24:05 +0100
From: valerie at mydomain.org
To: otherlist <otherlist at onedomain.de>
Message-ID: <7e498749e02d18656fd14393b90cdc38 at mydomain.org>
X-Sender: valerie at mydomain.org
Subject: [otherlist] some subject
X-BeenThere: otherlist at someotherserver.org
X-Mailman-Version: 2.1.15
Precedence: list
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"
Errors-To: otherlist-bounces at someotherserver.org
Sender: "otherlist" <otherlist-bounces at someotherserver.org>


what is grabbing my attention here is that there is no text in the From: field,
only the blank email address. digging little further reveals that the text that
gets merged originates from the members name field with which that address is
associated in mailman.

> 
> And then you've redacted stuff, and that may matter.  If you don't
> want to send unredacted headers to a list with public archives, we
> understand, but in that case you can and should send them to Mark (and
> possibly me, but Mark is the real expert if you really want to send it
> to the fewest people) privately.

thank you very much. i appreciate it. i will send the originals to you when
this email doesn't help.

> 
> I don't think this is a Mailman bug.  Mailman would not choose to send
> using two different transfer encodings (Q in the first line, B in the
> second).  So I suspect Mailman is just forwarding the garbage it
> receives, or something downstream of Mailman is doing it.
> 

From mark at msapiro.net  Thu Jan 21 12:59:40 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 21 Jan 2016 09:59:40 -0800
Subject: [Mailman-Users] diacritics in from header with from_is_list set
 to munge
In-Reply-To: <22177.517.393657.987628@turnbull.sk.tsukuba.ac.jp>
References: <20160120193601.GB24047@imaginationland>
 <56A012D6.6030107@msapiro.net>
 <22176.9477.411589.213445@turnbull.sk.tsukuba.ac.jp>
 <56A03345.4040805@msapiro.net>
 <22176.17930.403438.521009@turnbull.sk.tsukuba.ac.jp>
 <20160121124348.GA30232@imaginationland>
 <22177.517.393657.987628@turnbull.sk.tsukuba.ac.jp>
Message-ID: <56A11C8C.1050508@msapiro.net>

On 01/21/2016 08:06 AM, Stephen J. Turnbull wrote:
> gabriel writes:
> 
>  > so the message of users getting bounced look like (abbreviated):
> 
>  > This is a delivery status notification from some.server.org,
>  > running the Courier mail server, version 0.75.0.
> 
> FYI, bounce messages may or may not be useful, as some bounce programs
> do mess with the mail they forward.  I know you probably can't do
> anything about this, this is the best you can do.


Agreed. I'm not interested in the bounce at all.


>  > From: =?utf-8?q?Val=C3=A9rie/Something_via_mylist_=3Cmyli?=,
>  >       =?utf-8?b?ZW5AbGlzdHMubXRtZWRpYS5vcmc+?=


This is an absolute, non-compliant mess.

The first encoded word, if I ignore the comma which is non-compliant,
decodes to "Val?rie/Something via mylist <myli" and the second encoded
word decodes to "st at lists.xxxxx.org>". Thus, if I put them together, I
get "Val?rie/Something via mylist <mylist at lists.xxxxx.org>"

(I've replaced the actual last bit of the list name list and part of the
domain with st at lists.xxxxx since you seem to not want to reveal it, even
though you have as anyone can decode the RFC2047 encoding.)

The comma at the end of the first line is wrong because of RFC2047, sec
5(1):

    Ordinary ASCII text and 'encoded-word's may appear together in the
    same header field.  However, an 'encoded-word' that appears in a
    header field defined as '*text' MUST be separated from any adjacent
    'encoded-word' or 'text' by 'linear-white-space'.

More importantly, RFC2047, sec 5(3) says in part:

   + An 'encoded-word' MUST NOT appear in any portion of an 'addr-spec'.

>  > Sender: "mylist" <mylist-bounces at lists.mydomain.org>
> 
> So this has already been through Mailman.  We really really need to
> see the mail as it was *before* Mailman handled it (possibly in the
> mbox file in the archive, if you have it).
>
> And then you've redacted stuff, and that may matter.  If you don't
> want to send unredacted headers to a list with public archives, we
> understand, but in that case you can and should send them to Mark (and
> possibly me, but Mark is the real expert if you really want to send it
> to the fewest people) privately.


What I would like to see, unmunged, sent directly to me off list if you
don't want to post it, is
1) The complete, raw headers from the message as received from the list, and
2) Either the complete raw headers of the message from the archive
listname.mbox/listname.mbox file[1] or if that's not possible, from the
archive "Downloadable version .txt (or .txt.gz) file.


> I don't think this is a Mailman bug.  Mailman would not choose to send
> using two different transfer encodings (Q in the first line, B in the
> second).  So I suspect Mailman is just forwarding the garbage it
> receives, or something downstream of Mailman is doing it.


I'm certain Mailman did not create that encoded header. I suspect the
outgoing MTA. This might in fact be precipitated by a Mailman bug; i.e.,
the fact I noted earlier in this thread that the header created by
Mailman can contain a non-ascii character. This might be what triggers
the outgoing MTA to arbitrarily encode the header without actually
parsing it and encoding it correctly, but I'll know more after I see
what I've asked for.

[1] You can get the listname.mbox/listname.mbox file via the web UI.
There may be a link on the archive table of contents page, but usually
there isn't. If there isn't a link, go to the private archive URL (even
if the archive is public) - something like
http://www.example.com/mailman/private/listname - and log in. Then
retrieve http://www.example.com/mailman/private/listname.mbox/listname.mbox

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Jan 21 13:15:33 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 21 Jan 2016 10:15:33 -0800
Subject: [Mailman-Users] diacritics in from header with from_is_list set
 to munge
In-Reply-To: <20160121172740.GA20179@imaginationland>
References: <20160120193601.GB24047@imaginationland>
 <56A012D6.6030107@msapiro.net>
 <22176.9477.411589.213445@turnbull.sk.tsukuba.ac.jp>
 <56A03345.4040805@msapiro.net>
 <22176.17930.403438.521009@turnbull.sk.tsukuba.ac.jp>
 <20160121124348.GA30232@imaginationland>
 <22177.517.393657.987628@turnbull.sk.tsukuba.ac.jp>
 <20160121172740.GA20179@imaginationland>
Message-ID: <56A12045.9090304@msapiro.net>

On 01/21/2016 09:27 AM, gabriel wrote:
> 
> That may not be so easy. in fact there is no archive of that list, but haven't
> archived messages not been processed by mailman?


OK. I sent my last reply before I saw this.

So there are no archives from which to retrieve what I asked for. I'm
still interested in seeing the unmunged, raw headers from the message
delivered by Mailman, but I'm sure I know what's happening.


> luckily exactly this message has been posted on one other list i have access to:
> ...
> what is grabbing my attention here is that there is no text in the From: field,
> only the blank email address. digging little further reveals that the text that
> gets merged originates from the members name field with which that address is
> associated in mailman.


Yes, Mailman does that.

Here's what I think is happening.

Mailman is munging the From: header from

From: valerie at mydomain.org

to

From: val?rie via List <list at example.com>

where val?rie comes from the member's name field in the list membership.
This is done to try to identify the member without actually including an
email address in the display name as that is said to be a red flag to
some ISPs.

Mailman should really RFC 2047 encode the resultant display name or at
least the non-ascii part of it, but it doesn't. I accept this as a
Mailman bug and will work on fixing it.

Anyway, Mailman sends the message with that non-ascii From: header and
the outgoing MTA attempts to fix it and in the process makes it even worse.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Jan 21 13:24:56 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 21 Jan 2016 10:24:56 -0800
Subject: [Mailman-Users] Themeing/Skining Mailman
In-Reply-To: <56A10F15.8030104@boywithwings.co.uk>
References: <56A00759.2060205@boywithwings.co.uk>
 <56A035F5.6020106@msapiro.net> <56A10F15.8030104@boywithwings.co.uk>
Message-ID: <56A12278.4050206@msapiro.net>

On 01/21/2016 09:02 AM, Leo Francisco wrote:
> Thanks for the info, I will look into what version the group are
> running. So it isn't really simple essentially?


I suspect that an existing list is Mailman 2.1. Mailman 3 is still
pretty new.

The theming itself is not too difficult depending on what you want to
do. Theming the entire UI requires server access, but a few of the user
visible pages, i.e. the list's listinfo page, the 'subscribe results'
page for a list and the user options page for a list are customizable
through the web admin UI.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From gabriel at unseen.is  Thu Jan 21 13:40:04 2016
From: gabriel at unseen.is (gabriel)
Date: Thu, 21 Jan 2016 19:40:04 +0100
Subject: [Mailman-Users] diacritics in from header with from_is_list set
 to munge
In-Reply-To: <56A12045.9090304@msapiro.net>
References: <20160120193601.GB24047@imaginationland>
 <56A012D6.6030107@msapiro.net>
 <22176.9477.411589.213445@turnbull.sk.tsukuba.ac.jp>
 <56A03345.4040805@msapiro.net>
 <22176.17930.403438.521009@turnbull.sk.tsukuba.ac.jp>
 <20160121124348.GA30232@imaginationland>
 <22177.517.393657.987628@turnbull.sk.tsukuba.ac.jp>
 <20160121172740.GA20179@imaginationland>
 <56A12045.9090304@msapiro.net>
Message-ID: <20160121184003.GA21782@imaginationland>

On Thu, Jan 21, 2016 at 10:15:33AM -0800, Mark Sapiro wrote:
> 
> So there are no archives from which to retrieve what I asked for. I'm
> still interested in seeing the unmunged, raw headers from the message
> delivered by Mailman, but I'm sure I know what's happening.

i'm sorry there's really no easy way for me to get the raw headers before they 
are processed by mailman. but i guess you've already identified the problem 
correctly anyway.
 

> Mailman should really RFC 2047 encode the resultant display name or at
> least the non-ascii part of it, but it doesn't. I accept this as a
> Mailman bug and will work on fixing it.


great, thank you and steven very much for your help on this issue. i'm looking 
forward to see it fixed.

cheers,
gabriel


From mark at msapiro.net  Thu Jan 21 13:50:55 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 21 Jan 2016 10:50:55 -0800
Subject: [Mailman-Users] diacritics in from header with from_is_list set
 to munge
In-Reply-To: <20160121184003.GA21782@imaginationland>
References: <20160120193601.GB24047@imaginationland>
 <56A012D6.6030107@msapiro.net>
 <22176.9477.411589.213445@turnbull.sk.tsukuba.ac.jp>
 <56A03345.4040805@msapiro.net>
 <22176.17930.403438.521009@turnbull.sk.tsukuba.ac.jp>
 <20160121124348.GA30232@imaginationland>
 <22177.517.393657.987628@turnbull.sk.tsukuba.ac.jp>
 <20160121172740.GA20179@imaginationland> <56A12045.9090304@msapiro.net>
 <20160121184003.GA21782@imaginationland>
Message-ID: <56A1288F.2060909@msapiro.net>

On 01/21/2016 10:40 AM, gabriel wrote:
> On Thu, Jan 21, 2016 at 10:15:33AM -0800, Mark Sapiro wrote:
>>
>> So there are no archives from which to retrieve what I asked for. I'm
>> still interested in seeing the unmunged, raw headers from the message
>> delivered by Mailman, but I'm sure I know what's happening.
> 
> i'm sorry there's really no easy way for me to get the raw headers before they 
> are processed by mailman. but i guess you've already identified the problem 
> correctly anyway.


I said "I'm still interested in seeing the unmunged, raw headers from
the message delivered by Mailman" I.e., the message FROM Mailman, not
the message TO Mailman.

The problem is really an MTA bug just triggered by a Mailman bug. I'd
like to see that message so I can see what MTA(s) it passed through
after Mailman and possibly file a bug report with the MTA.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From gabriel at unseen.is  Thu Jan 21 14:08:11 2016
From: gabriel at unseen.is (gabriel)
Date: Thu, 21 Jan 2016 20:08:11 +0100
Subject: [Mailman-Users] diacritics in from header with from_is_list set
 to munge
In-Reply-To: <56A1288F.2060909@msapiro.net>
References: <56A012D6.6030107@msapiro.net>
 <22176.9477.411589.213445@turnbull.sk.tsukuba.ac.jp>
 <56A03345.4040805@msapiro.net>
 <22176.17930.403438.521009@turnbull.sk.tsukuba.ac.jp>
 <20160121124348.GA30232@imaginationland>
 <22177.517.393657.987628@turnbull.sk.tsukuba.ac.jp>
 <20160121172740.GA20179@imaginationland>
 <56A12045.9090304@msapiro.net>
 <20160121184003.GA21782@imaginationland>
 <56A1288F.2060909@msapiro.net>
Message-ID: <20160121190810.GB21782@imaginationland>

On Thu, Jan 21, 2016 at 10:50:55AM -0800, Mark Sapiro wrote:
> On 01/21/2016 10:40 AM, gabriel wrote:
> > On Thu, Jan 21, 2016 at 10:15:33AM -0800, Mark Sapiro wrote:
> >>
> >> So there are no archives from which to retrieve what I asked for. I'm
> >> still interested in seeing the unmunged, raw headers from the message
> >> delivered by Mailman, but I'm sure I know what's happening.
> > 
> > i'm sorry there's really no easy way for me to get the raw headers before they 
> > are processed by mailman. but i guess you've already identified the problem 
> > correctly anyway.
> 
> 
> I said "I'm still interested in seeing the unmunged, raw headers from
> the message delivered by Mailman" I.e., the message FROM Mailman, not
> the message TO Mailman.

I don't quite understand. Mailman is configured to munge the From header. When  
a message passes Mailman, headers will be munged. how could i get a unmunged, 
raw header message delivered by Mailman? If i set the list to from_is_list = 
off all my users will get bounced by yahoo. do you want me to enable archives 
and wait for another incident?

> 
> The problem is really an MTA bug just triggered by a Mailman bug. I'd
> like to see that message so I can see what MTA(s) it passed through
> after Mailman and possibly file a bug report with the MTA.

yes sure that makes sense, too.


From mark at msapiro.net  Thu Jan 21 16:45:17 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 21 Jan 2016 13:45:17 -0800
Subject: [Mailman-Users] diacritics in from header with from_is_list set
 to munge
In-Reply-To: <20160121190810.GB21782@imaginationland>
References: <56A012D6.6030107@msapiro.net>
 <22176.9477.411589.213445@turnbull.sk.tsukuba.ac.jp>
 <56A03345.4040805@msapiro.net>
 <22176.17930.403438.521009@turnbull.sk.tsukuba.ac.jp>
 <20160121124348.GA30232@imaginationland>
 <22177.517.393657.987628@turnbull.sk.tsukuba.ac.jp>
 <20160121172740.GA20179@imaginationland> <56A12045.9090304@msapiro.net>
 <20160121184003.GA21782@imaginationland> <56A1288F.2060909@msapiro.net>
 <20160121190810.GB21782@imaginationland>
Message-ID: <56A1516D.3040409@msapiro.net>

On 01/21/2016 11:08 AM, gabriel wrote:
> On Thu, Jan 21, 2016 at 10:50:55AM -0800, Mark Sapiro wrote:
>>
>> I said "I'm still interested in seeing the unmunged, raw headers from
>> the message delivered by Mailman" I.e., the message FROM Mailman, not
>> the message TO Mailman.
> 
> I don't quite understand. Mailman is configured to munge the From header. When  
> a message passes Mailman, headers will be munged. how could i get a unmunged, 
> raw header message delivered by Mailman?


Sorry for the confusion.

By unmunged, I'm not referring to the munging done by Mailman's Munge
From. I'm referring to the munging done by you when you replace actual
names, list names and domains by things like Something, mylist and mydomain.

What I would like to see is the raw message headers as received by you
from Mailman without alteration or editing by you. If you don't want to
post that to the list, please send it off list to me.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Thu Jan 21 17:37:12 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Thu, 21 Jan 2016 14:37:12 -0800
Subject: [Mailman-Users] diacritics in from header with from_is_list set
 to munge
In-Reply-To: <56A12045.9090304@msapiro.net>
References: <20160120193601.GB24047@imaginationland>
 <56A012D6.6030107@msapiro.net>
 <22176.9477.411589.213445@turnbull.sk.tsukuba.ac.jp>
 <56A03345.4040805@msapiro.net>
 <22176.17930.403438.521009@turnbull.sk.tsukuba.ac.jp>
 <20160121124348.GA30232@imaginationland>
 <22177.517.393657.987628@turnbull.sk.tsukuba.ac.jp>
 <20160121172740.GA20179@imaginationland> <56A12045.9090304@msapiro.net>
Message-ID: <56A15D98.9050101@msapiro.net>

On 01/21/2016 10:15 AM, Mark Sapiro wrote:
> 
> Here's what I think is happening.
> 
> Mailman is munging the From: header from
> 
> From: valerie at mydomain.org
> 
> to
> 
> From: val?rie via List <list at example.com>
> 
> where val?rie comes from the member's name field in the list membership.
> This is done to try to identify the member without actually including an
> email address in the display name as that is said to be a red flag to
> some ISPs.
> 
> Mailman should really RFC 2047 encode the resultant display name or at
> least the non-ascii part of it, but it doesn't. I accept this as a
> Mailman bug and will work on fixing it.
> 
> Anyway, Mailman sends the message with that non-ascii From: header and
> the outgoing MTA attempts to fix it and in the process makes it even worse.


Actually, further testing of this scenario shows that Mailman is likely
responsible for the mis-encoding as well. Once I realized the key was in
providing the poster's real name from the list's membership, I was able
to duplicate the issue including the mis-encoding.

So I don't need to see any further samples, and I have developed a fix.
The basic fix is very simple. In Mailman/Handlers/Cookheaders.py around
line 155 are the lines:

        change_header('From',
                      formataddr(('%s via %s' % (realname, mlist.real_name),
                                  mlist.GetListEmail())),
                       mlist, msg, msgdata)

Immediately before the change_header insert (indented the same 8 spaces)

        realname = str(uheader(mlist, realname))

which will RFC 2047 encode the realname if it contains non-ascii.

This is now reported as
<https://bugs.launchpad.net/mailman/+bug/1536816> and the fix will be
committed soon.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From lists at boywithwings.co.uk  Fri Jan 22 11:59:21 2016
From: lists at boywithwings.co.uk (Leo Francisco)
Date: Fri, 22 Jan 2016 16:59:21 +0000
Subject: [Mailman-Users] Themeing/Skining Mailman
In-Reply-To: <56A12278.4050206@msapiro.net>
References: <56A00759.2060205@boywithwings.co.uk>
 <56A035F5.6020106@msapiro.net> <56A10F15.8030104@boywithwings.co.uk>
 <56A12278.4050206@msapiro.net>
Message-ID: <56A25FE9.2010005@boywithwings.co.uk>

Yep, that's pretty much what we'd like to do, just a simple re-theme of
the sign up page. Thanks Mark for your help.

I'm going to have a mess around it with it.

Leo


On 21/01/16 18:24, Mark Sapiro wrote:
> On 01/21/2016 09:02 AM, Leo Francisco wrote:
>> Thanks for the info, I will look into what version the group are
>> running. So it isn't really simple essentially?
>
> I suspect that an existing list is Mailman 2.1. Mailman 3 is still
> pretty new.
>
> The theming itself is not too difficult depending on what you want to
> do. Theming the entire UI requires server access, but a few of the user
> visible pages, i.e. the list's listinfo page, the 'subscribe results'
> page for a list and the user options page for a list are customizable
> through the web admin UI.
>



From mark at msapiro.net  Fri Jan 22 19:43:30 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Fri, 22 Jan 2016 16:43:30 -0800
Subject: [Mailman-Users] diacritics in from header with from_is_list set
 to munge
In-Reply-To: <56A15D98.9050101@msapiro.net>
References: <56A15D98.9050101@msapiro.net>
Message-ID: <56A2CCB2.5000400@msapiro.net>

Mark Sapiro wrote:

> Actually, further testing of this scenario shows that Mailman is likely
> responsible for the mis-encoding as well.


To summarize and wrap up. It was possible for mailman to create a

From: User_name via list_name <list at example.com>

header with non-ascii in the User_name when creating the new From: for a
message with Munge From or the From: for the outer mwrapper with Wrap
Message. This would occur when the incoming From: had only an address
and no display name so the poster's list membership real name would be
used and that real name contained non-ascii. It could also occur if a
post arrived containing an unencoded display name in From: if the
sending MUA was non-compliant, but this scenario is unlikely.

Thanks to gabriel for reporting the problem so it could ultimately be
found and fixed.

The underlying bug is that Mailman should RFC 2047 encode the User_name
at that point, and now it does (bug at
<https://bugs.launchpad.net/mailman/+bug/1536816>, fix at
<http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1597>[1]).

This was further exacerbated by the fact that Mailman's SMTPDirect.py
module uses the Mailman Message as_string() method to flatten the
message object to plain text for sending, and sometimes, but not always,
depending on characteristics of the message itself this method would
refold or rewrite certain headers and in so doing would 'see' the
non-ascii in the From: header and RFC 2047 encode the entire header
content without regard for that fact that it contained address specs
that are not to be encoded.

Debugging this issue was complicated by the fact that the header
encoding by as_string() didn't occur with every message and my initial
test messages didn't trigger it.

[1] The only part of the fix at
<http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1597>
that is necessary to fix this issue is the addition of

        realname = str(uheader(mlist, realname))

The rest of it is for l10n of the "User_name via list_name" phrase.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan



From skip.montanaro at gmail.com  Sat Jan 23 14:46:33 2016
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Sat, 23 Jan 2016 13:46:33 -0600
Subject: [Mailman-Users] Looking for a server
Message-ID: <CANc-5Uxdrv8vcDRAbhyfcdv++eU01BiqJuaLkdpQOrjnCs5THw@mail.gmail.com>

I've not hosted a mailman list in a long while. I'm trying to help
some folks migrate a list with ten years of archives and no current
subscribers just to make the archives indexable by the various search
engines. I see the list of hosting services on wiki.list.org, but I'd
appreciate actual recommendations (or pointers to prioritized lists),
not an unprioritized list.

Thanks,

Skip Montanaro

From fmouse at fmp.com  Sat Jan 23 20:35:08 2016
From: fmouse at fmp.com (Lindsay Haisley)
Date: Sat, 23 Jan 2016 19:35:08 -0600
Subject: [Mailman-Users] Persistent spam source!
Message-ID: <1453599308.110041.4.camel@pudina.fmp.com>

I got the following from one of my list admins. Any suggestions as to
what might be going on? I've verified that the address in question is
indeed on the discard_these_nonmembers list and that the problem address
isn't already subscribed.

----
        So we keep getting spam postings from  jeffrey at elespectaddor.com
        caught in the "needs approval" filter. He's not a member. I've
        added him to the non-member filter under "List of non-member
        addresses whose postings will be automatically discarded" and
        banned him from membership. But half a dozen times a day we're
        still getting requests to approve a spam posting when the
        options I've set up are to discard anything from him.
        
        How are these slipping through? And is there anything we can do
        about it? It's not a disaster, but it sure is a pain.
----

Our MM version is 2.1.18-1.

-- 
Lindsay Haisley      | "The first casualty when war comes is truth"
FMP Computer Servces |      
512-259-1190         |        -- Hiram W Johnson    
http://www.fmp.com   |


From fmouse at fmp.com  Sat Jan 23 21:19:20 2016
From: fmouse at fmp.com (Lindsay Haisley)
Date: Sat, 23 Jan 2016 20:19:20 -0600
Subject: [Mailman-Users] Persistent spam source!
In-Reply-To: <1453599308.110041.4.camel@pudina.fmp.com>
References: <1453599308.110041.4.camel@pudina.fmp.com>
Message-ID: <1453601960.110041.29.camel@pudina.fmp.com>

What appears to be happening here is that the envelope sender address
for these spams is AdrianaCaldwell at archive.elespectaddor.com which
matches a rule in header_filter_rules to hold mail addresses matching
the regexp ".*Archive". So it _appears_ that directives in
header_filter_rules trump directives from discard_these_nonmembers. Is
this indeed the case? Would specifying the above address verbatim in
discard_these_nonmembers give this directive precedence? 

On Sat, 2016-01-23 at 19:35 -0600, Lindsay Haisley wrote:
> I got the following from one of my list admins. Any suggestions as to
> what might be going on? I've verified that the address in question is
> indeed on the discard_these_nonmembers list and that the problem address
> isn't already subscribed.
> 
> ----
>         So we keep getting spam postings from  jeffrey at elespectaddor.com
>         caught in the "needs approval" filter. He's not a member. I've
>         added him to the non-member filter under "List of non-member
>         addresses whose postings will be automatically discarded" and
>         banned him from membership. But half a dozen times a day we're
>         still getting requests to approve a spam posting when the
>         options I've set up are to discard anything from him.
>         
>         How are these slipping through? And is there anything we can do
>         about it? It's not a disaster, but it sure is a pain.
> ----
> 
> Our MM version is 2.1.18-1.
> 

-- 
Lindsay Haisley      | "The first casualty when war comes is truth"
FMP Computer Servces |      
512-259-1190         |        -- Hiram W Johnson    
http://www.fmp.com   |


From beta at admilon.net  Sun Jan 24 02:00:52 2016
From: beta at admilon.net (Matthias Schmidt)
Date: Sun, 24 Jan 2016 16:00:52 +0900
Subject: [Mailman-Users] MacOS 10.6 ->OS X Server 5.x (=10.11.x)
Message-ID: <667EAF40-8018-4E29-90F4-655BB8A4FA00@admilon.net>

Hello,
I?m currently running a 10.6.8 Server installation with a couple of mailman 2.x mailing lists (?Apple?s mailman? + virtual domains).
Works great :-)

since iOS9 and MacOS 10.11 were published some things are not working anymore via SSL (WebDAV etc) anymore.
So I?m thinking about updating the box.

The current version of OS X Server 5 doesn?t ship with mailman anymore.
Therefore I was thinking to upgrade mailman to v3.
Did anybody upgrade mailman from v2 (apple) to version 3 in connection with an OS update?
If so, how did it go and what?s best practice?
(I did google before, but no results showed up - sorry, if that has been discussed before and I couldn?t find it)

thanks and cheers
Matthias

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/mailman-users/attachments/20160124/d7ec9f17/attachment.sig>

From andrew at hodgsonfamily.org  Sun Jan 24 13:33:42 2016
From: andrew at hodgsonfamily.org (Andrew Hodgson)
Date: Sun, 24 Jan 2016 18:33:42 +0000
Subject: [Mailman-Users] Looking for a server
In-Reply-To: <CANc-5Uxdrv8vcDRAbhyfcdv++eU01BiqJuaLkdpQOrjnCs5THw@mail.gmail.com>
References: <CANc-5Uxdrv8vcDRAbhyfcdv++eU01BiqJuaLkdpQOrjnCs5THw@mail.gmail.com>
Message-ID: <HE1PR04MB12442B73791856E27BB313E7C5C60@HE1PR04MB1244.eurprd04.prod.outlook.com>

Skip Montanaro wrote:

>I've not hosted a mailman list in a long while. I'm trying to help some folks migrate a list with ten years of archives and no current subscribers just to make the archives indexable by the various search engines. I see the list of >hosting services on wiki.list.org, but I'd appreciate actual recommendations (or pointers to prioritized lists), not an unprioritized list.

Have you tried mail-archive.com? Last time I used them I think they can take a Mailman mbox file and turn it into an archive.

Andrew. 

From skip.montanaro at gmail.com  Sun Jan 24 18:37:18 2016
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Sun, 24 Jan 2016 17:37:18 -0600
Subject: [Mailman-Users] Looking for a server
In-Reply-To: <20160124232612.GR21217@hendricks.amyl.org.uk>
References: <CANc-5Uxdrv8vcDRAbhyfcdv++eU01BiqJuaLkdpQOrjnCs5THw@mail.gmail.com>
 <HE1PR04MB12442B73791856E27BB313E7C5C60@HE1PR04MB1244.eurprd04.prod.outlook.com>
 <20160124232612.GR21217@hendricks.amyl.org.uk>
Message-ID: <CANc-5Uyz4Jan2KJ28HT8VnZTJR6=txOEt8+bPBW813_axSPSmw@mail.gmail.com>

> Or google groups?
> https://gist.github.com/adamamyl/02a8f44981c8685f9c88 reminds me of
> https://mail.python.org/pipermail/mailman-users/2014-October/077893.html

Thanks for the suggestion. I've actually already tried that. It was my
first thought. I found a Python Google API which looked straightforward to
use, but I could never figure out the whole setup/authentication process,
and my pleas for assistance on the (I think appropriate) Google help forum,
yielded no help, so I gave up.

I will take a look at your links.

Skip

From adam at amyl.org.uk  Sun Jan 24 18:26:12 2016
From: adam at amyl.org.uk (Adam McGreggor)
Date: Sun, 24 Jan 2016 23:26:12 +0000
Subject: [Mailman-Users] Looking for a server
In-Reply-To: <HE1PR04MB12442B73791856E27BB313E7C5C60@HE1PR04MB1244.eurprd04.prod.outlook.com>
References: <CANc-5Uxdrv8vcDRAbhyfcdv++eU01BiqJuaLkdpQOrjnCs5THw@mail.gmail.com>
 <HE1PR04MB12442B73791856E27BB313E7C5C60@HE1PR04MB1244.eurprd04.prod.outlook.com>
Message-ID: <20160124232612.GR21217@hendricks.amyl.org.uk>

On Sun, Jan 24, 2016 at 06:33:42PM +0000, Andrew Hodgson wrote:
> Skip Montanaro wrote:
> > I've not hosted a mailman list in a long while. I'm trying to help
> > some folks migrate a list with ten years of archives and no current
> > subscribers just to make the archives indexable by the various search
> > engines. I see the list of >hosting services on wiki.list.org, but I'd
> > appreciate actual recommendations (or pointers to prioritized lists),
> > not an unprioritized list.  
> Have you tried mail-archive.com? Last
> time I used them I think they can take a Mailman mbox file and turn it
> into an archive.

Or google groups?
https://gist.github.com/adamamyl/02a8f44981c8685f9c88 reminds me of
https://mail.python.org/pipermail/mailman-users/2014-October/077893.html


-- 
"We are not knocking down doors at four in the morning with people
 booted and suited in riot gear. Most of the removals occur around
 half-five, half-six, seven in the morning."
    -- Tony McNulty denies reports of 'dawn raids' on asylum seekers

From mailman at rissel.it  Mon Jan 25 16:59:47 2016
From: mailman at rissel.it (Sascha Rissel)
Date: Mon, 25 Jan 2016 22:59:47 +0100
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
Message-ID: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>

Hello,

I have some user on my lists, who complain they don't receive the list
mails anymore, since a few days.

Am am running Mailman 2.1.15 on Debian.
The error log is empty, the archive seems to correctly contain all recent
mails.
The people saying, they don't receive the mails are on GMail and German
Web.de hosts and they don't have bounces logged by Mailman.

Can you help me with this?
Are there any means, how I can analyse this issue?

Kind regards,
Sascha.

From mark at msapiro.net  Mon Jan 25 17:35:20 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 25 Jan 2016 14:35:20 -0800
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
Message-ID: <56A6A328.2000308@msapiro.net>

On 01/25/2016 01:59 PM, Sascha Rissel wrote:
> 
> I have some user on my lists, who complain they don't receive the list
> mails anymore, since a few days.
> 
> Am am running Mailman 2.1.15 on Debian.
> The error log is empty, the archive seems to correctly contain all recent
> mails.
> The people saying, they don't receive the mails are on GMail and German
> Web.de hosts and they don't have bounces logged by Mailman.


If there is nothing in Mailman's error log or bounce log, it is almost
certain the mail is being delivered by Mailman to the outgoing MTA and
the outgoing MTA is successfully delivering the mail to the mail
exchange server for the recipient domain.

You can confirm this if you have access to the Mailman (and it's
outgoing MTA) server's mail.log.

If this is the case and the users have checked their gmail or web.de
spam or junk folders and the messages aren't there, the messages are
likely being silently discarded somewhere in the delivery chain after
leaving the outgoing MTA.

Solving this is difficult. Some steps are outlined in the FAQ article at
<http://wiki.list.org/x/4030690>.

I sometimes will copy the specific MTA log messages indicating
acceptance by the receiving MTA, e.g., messages like

Jan 24 19:09:41 sbh16 postfix/smtp[1053]: 1279111E1A8F:
to=<user at gmail.com>, relay=gmail-smtp-in.l.google.com[74.125.28.26]:25,
delay=5.8, delays=4.9/0.66/0.09/0.13, dsn=2.0.0, status=sent (250 2.0.0
OK 1453691381 tt2si3153213pac.167 - gsmtp)

and tell the user to ask gmail what happened to that message. In that
message, (250 2.0.0 OK 1453691381 tt2si3153213pac.167 - gsmtp) is the
acceptance from gmail-smtp-in.l.google.com indicating the message was
accepted at unix time stamp 1453691381 with the accepting server's ID
tt2si3153213pac.167. I suspect gmail actually ignores such requests from
their users or provides only a generic response. People who actually pay
for their email service may have a bit more leverage.

Anyway, apart from the things in the FAQ article, there's not a whole
lot you can do. If you can possibly identify something about the missing
mail that triggers it, e.g., only mail From: a certain user or domain,
or something in a specific thread (copied in everyone's reply), you can
try to avoid that, but if it's all list mail To: particular domains
(gmail), I think it's more likely to be a block on mail from your
server's IP, but in the US at least, gmail ordinarily bounces such mail
with a fairly specific reason.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mailman at rissel.it  Mon Jan 25 17:48:16 2016
From: mailman at rissel.it (Sascha Rissel)
Date: Mon, 25 Jan 2016 23:48:16 +0100
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <56A6A328.2000308@msapiro.net>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
 <56A6A328.2000308@msapiro.net>
Message-ID: <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>

Mark,

thanks for your answer.
I just checked the bounce log.

And I discovered a lot of entries like:

Jan 25 18:33:24 2016 (2741) <BounceRunner at 12019976> processing 9 queued
bounces
Jan 25 18:33:24 2016 (2741) <my-list-name>: <some mail address> already
scored a bounce for date 25-Jan-2016
Jan 25 18:33:24 2016 (2741) <my-list-name>: <some mail address> already
scored a bounce for date 25-Jan-2016
Jan 25 18:33:24 2016 (2741) <my-list-name>: <some mail address> already
scored a bounce for date 25-Jan-2016

where all mail addresses are from German Web.de and GMX. Both providers
belong to the same company.
Thinking about this, I remembered a bounce notification I received last
week, where some of these addresses were set to disabled by Mailman, with
the notification below.

Did GMX/Web.de maybe change their mail processing policies with the start
of the new year?
Below follows an excerpt from the bounce notification.

Kind regards,
Sascha.


Bounce Mail:

<some_user at web.de>: host mx-ha02.web.de[212.227.17.8] refused to talk
    to me: 554-web.de (mxweb107) Nemesis ESMTP Service not available 554-No
    SMTP service 554-Bad DNS PTR resource record. 554 For explanation visit
    http://postmaster.web.de/error-messages?ip=62.75.175.182&c=rdns

Final-Recipient: rfc822; <second_user>@gmx.de
Original-Recipient: rfc822;<second_user>@gmx.de
Action: failed
Status: 4.0.0
Remote-MTA: dns; mx00.emig.gmx.net
Diagnostic-Code: smtp; 554-gmx.net (mxgmx002) Nemesis ESMTP Service not
    available 554-No SMTP service 554-Bad DNS PTR resource record. 554 For
    explanation visit
    http://postmaster.gmx.com/en/error-messages?ip=62.75.175.182&c=rdns

Final-Recipient: rfc822; <some_third_user>@web.de
Original-Recipient: rfc822;andreas_hacker at web.de
Action: failed
Status: 4.0.0
Remote-MTA: dns; mx-ha02.web.de
Diagnostic-Code: smtp; 554-web.de (mxweb107) Nemesis ESMTP Service not
    available 554-No SMTP service 554-Bad DNS PTR resource record. 554 For
    explanation visit
    http://postmaster.web.de/error-messages?ip=62.75.175.182&c=rdns

2016-01-25 23:35 GMT+01:00 Mark Sapiro <mark at msapiro.net>:

> On 01/25/2016 01:59 PM, Sascha Rissel wrote:
> >
> > I have some user on my lists, who complain they don't receive the list
> > mails anymore, since a few days.
> >
> > Am am running Mailman 2.1.15 on Debian.
> > The error log is empty, the archive seems to correctly contain all recent
> > mails.
> > The people saying, they don't receive the mails are on GMail and German
> > Web.de hosts and they don't have bounces logged by Mailman.
>
>
> If there is nothing in Mailman's error log or bounce log, it is almost
> certain the mail is being delivered by Mailman to the outgoing MTA and
> the outgoing MTA is successfully delivering the mail to the mail
> exchange server for the recipient domain.
>
> You can confirm this if you have access to the Mailman (and it's
> outgoing MTA) server's mail.log.
>
> If this is the case and the users have checked their gmail or web.de
> spam or junk folders and the messages aren't there, the messages are
> likely being silently discarded somewhere in the delivery chain after
> leaving the outgoing MTA.
>
> Solving this is difficult. Some steps are outlined in the FAQ article at
> <http://wiki.list.org/x/4030690>.
>
> I sometimes will copy the specific MTA log messages indicating
> acceptance by the receiving MTA, e.g., messages like
>
> Jan 24 19:09:41 sbh16 postfix/smtp[1053]: 1279111E1A8F:
> to=<user at gmail.com>, relay=gmail-smtp-in.l.google.com[74.125.28.26]:25,
> delay=5.8, delays=4.9/0.66/0.09/0.13, dsn=2.0.0, status=sent (250 2.0.0
> OK 1453691381 tt2si3153213pac.167 - gsmtp)
>
> and tell the user to ask gmail what happened to that message. In that
> message, (250 2.0.0 OK 1453691381 tt2si3153213pac.167 - gsmtp) is the
> acceptance from gmail-smtp-in.l.google.com indicating the message was
> accepted at unix time stamp 1453691381 with the accepting server's ID
> tt2si3153213pac.167. I suspect gmail actually ignores such requests from
> their users or provides only a generic response. People who actually pay
> for their email service may have a bit more leverage.
>
> Anyway, apart from the things in the FAQ article, there's not a whole
> lot you can do. If you can possibly identify something about the missing
> mail that triggers it, e.g., only mail From: a certain user or domain,
> or something in a specific thread (copied in everyone's reply), you can
> try to avoid that, but if it's all list mail To: particular domains
> (gmail), I think it's more likely to be a block on mail from your
> server's IP, but in the US at least, gmail ordinarily bounces such mail
> with a fairly specific reason.
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/mailman%40rissel.it
>

From mark at msapiro.net  Mon Jan 25 18:11:48 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 25 Jan 2016 15:11:48 -0800
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
 <56A6A328.2000308@msapiro.net>
 <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
Message-ID: <56A6ABB4.4@msapiro.net>

On 01/25/2016 02:48 PM, Sascha Rissel wrote:
> 
> thanks for your answer.
> I just checked the bounce log.
> 
> And I discovered a lot of entries like:
> 
> Jan 25 18:33:24 2016 (2741) <BounceRunner at 12019976> processing 9 queued
> bounces
> Jan 25 18:33:24 2016 (2741) <my-list-name>: <some mail address> already
> scored a bounce for date 25-Jan-2016
...
> where all mail addresses are from German Web.de and GMX. Both providers
> belong to the same company.
> Thinking about this, I remembered a bounce notification I received last
> week, where some of these addresses were set to disabled by Mailman, with
> the notification below.
> 
> Did GMX/Web.de maybe change their mail processing policies with the start
> of the new year?


Possibly. See below.


> Below follows an excerpt from the bounce notification.
> 
> Bounce Mail:
> 
> <some_user at web.de>: host mx-ha02.web.de[212.227.17.8] refused to talk
>     to me: 554-web.de (mxweb107) Nemesis ESMTP Service not available 554-No
>     SMTP service 554-Bad DNS PTR resource record. 554 For explanation visit
>     http://postmaster.web.de/error-messages?ip=62.75.175.182&c=rdns


OK. The recipient's mail exchange server is refusing your mail. web.de's
explanation of this is at
<http://postmaster.web.de/error-messages?ip=62.75.175.182&c=rdns#nordns>.

In simple terms, it claims your mail server is not properly configured.
I don't know what your Mailman server's domain name is, but it appears
from the error message that its IP is 62.75.175.182 and that IP has an
rDNS PTR record to euve51864.serverprofi24.de.

In any case you need to ensure the IP address your server sends from has
a rDNS (PTR) record pointing to its host name and that host name in turn
needs an A record with the same IP address. Also, it should identify
itself with this same name in the SMTP HELO or EHLO command.

According to the tool at
<http://mxtoolbox.com/SuperTool.aspx?action=smtp%3a62.75.175.182&run=toolpage>,
this is all OK, so I don't know why web.de is complaining. I suggest you
use the form at
<http://postmaster.web.de/en/contact/?ip=62.75.175.182&c=rdns> to ask them.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mark at msapiro.net  Mon Jan 25 18:24:56 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 25 Jan 2016 15:24:56 -0800
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <56A6ABB4.4@msapiro.net>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
 <56A6A328.2000308@msapiro.net>
 <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
 <56A6ABB4.4@msapiro.net>
Message-ID: <56A6AEC8.40008@msapiro.net>

On 01/25/2016 03:11 PM, Mark Sapiro wrote:
> On 01/25/2016 02:48 PM, Sascha Rissel wrote:
>>
>> <some_user at web.de>: host mx-ha02.web.de[212.227.17.8] refused to talk
>>     to me: 554-web.de (mxweb107) Nemesis ESMTP Service not available 554-No
>>     SMTP service 554-Bad DNS PTR resource record. 554 For explanation visit
>>     http://postmaster.web.de/error-messages?ip=62.75.175.182&c=rdns
> 
> 
> OK. The recipient's mail exchange server is refusing your mail. web.de's
> explanation of this is at
> <http://postmaster.web.de/error-messages?ip=62.75.175.182&c=rdns#nordns>.
> 
> In simple terms, it claims your mail server is not properly configured.
> I don't know what your Mailman server's domain name is, but it appears
> from the error message that its IP is 62.75.175.182 and that IP has an
> rDNS PTR record to euve51864.serverprofi24.de.
> 
> In any case you need to ensure the IP address your server sends from has
> a rDNS (PTR) record pointing to its host name and that host name in turn
> needs an A record with the same IP address. Also, it should identify
> itself with this same name in the SMTP HELO or EHLO command.
> 
> According to the tool at
> <http://mxtoolbox.com/SuperTool.aspx?action=smtp%3a62.75.175.182&run=toolpage>,
> this is all OK, so I don't know why web.de is complaining. I suggest you
> use the form at
> <http://postmaster.web.de/en/contact/?ip=62.75.175.182&c=rdns> to ask them.


I looked more carefully at
<http://postmaster.web.de/error-messages?ip=62.75.175.182&c=rdns#nordns>, and
I'm guessing that web.de has decided that euve51864.serverprofi24.de is
a 'generic' name rather than an independent and fully qualified domain name.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mailman at rissel.it  Mon Jan 25 18:27:15 2016
From: mailman at rissel.it (Sascha Rissel)
Date: Tue, 26 Jan 2016 00:27:15 +0100
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <56A6ABB4.4@msapiro.net>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
 <56A6A328.2000308@msapiro.net>
 <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
 <56A6ABB4.4@msapiro.net>
Message-ID: <CAKGvnEAMKc-+AdiyhyPqne8Dc55=pDK1-MVUnUph1ama_0yjPA@mail.gmail.com>

Hi Mark!

> In simple terms, it claims your mail server is not properly configured.
> I don't know what your Mailman server's domain name is, but it appears
> from the error message that its IP is 62.75.175.182 and that IP has an
> rDNS PTR record to euve51864.serverprofi24.de.

this is all pretty correct.

"euve51864.serverprofi24.de" is the natural name of the vServer, I am
running.
I don't have a real domain configured, since I mainly just use it for
mailman.
I configured all Mailman lists to use something like <list name>@rc-worms.de,
where <rc-worms.de> is the www domain of my sports club, whose
administrator I am.
For better readability I named and configured all lists this way, to use <
members at rc-worms.de>, <team1 at rc-worms.de> as exlicity reply-to address and
so on.

This worked pretty well for years.
Did it now turn into a problem, that reply-to address and domain () bear a
different IP/rDNS result?
Is there something you could suggest in this case?

Should I maybe configure a subdomain on my vServer, like <lists.rc-worms.de>,
from which the mails will be send from and can bear own DNS config?

Kind regards,
Sascha.




2016-01-26 0:11 GMT+01:00 Mark Sapiro <mark at msapiro.net>:

> On 01/25/2016 02:48 PM, Sascha Rissel wrote:
> >
> > thanks for your answer.
> > I just checked the bounce log.
> >
> > And I discovered a lot of entries like:
> >
> > Jan 25 18:33:24 2016 (2741) <BounceRunner at 12019976> processing 9
> queued
> > bounces
> > Jan 25 18:33:24 2016 (2741) <my-list-name>: <some mail address> already
> > scored a bounce for date 25-Jan-2016
> ...
> > where all mail addresses are from German Web.de and GMX. Both providers
> > belong to the same company.
> > Thinking about this, I remembered a bounce notification I received last
> > week, where some of these addresses were set to disabled by Mailman, with
> > the notification below.
> >
> > Did GMX/Web.de maybe change their mail processing policies with the start
> > of the new year?
>
>
> Possibly. See below.
>
>
> > Below follows an excerpt from the bounce notification.
> >
> > Bounce Mail:
> >
> > <some_user at web.de>: host mx-ha02.web.de[212.227.17.8] refused to talk
> >     to me: 554-web.de (mxweb107) Nemesis ESMTP Service not available
> 554-No
> >     SMTP service 554-Bad DNS PTR resource record. 554 For explanation
> visit
> >     http://postmaster.web.de/error-messages?ip=62.75.175.182&c=rdns
>
>
> OK. The recipient's mail exchange server is refusing your mail. web.de's
> explanation of this is at
> <http://postmaster.web.de/error-messages?ip=62.75.175.182&c=rdns#nordns>.
>
> In simple terms, it claims your mail server is not properly configured.
> I don't know what your Mailman server's domain name is, but it appears
> from the error message that its IP is 62.75.175.182 and that IP has an
> rDNS PTR record to euve51864.serverprofi24.de.
>
> In any case you need to ensure the IP address your server sends from has
> a rDNS (PTR) record pointing to its host name and that host name in turn
> needs an A record with the same IP address. Also, it should identify
> itself with this same name in the SMTP HELO or EHLO command.
>
> According to the tool at
> <
> http://mxtoolbox.com/SuperTool.aspx?action=smtp%3a62.75.175.182&run=toolpage
> >,
> this is all OK, so I don't know why web.de is complaining. I suggest you
> use the form at
> <http://postmaster.web.de/en/contact/?ip=62.75.175.182&c=rdns> to ask
> them.
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/mailman%40rissel.it
>

From heller at deepsoft.com  Mon Jan 25 17:43:09 2016
From: heller at deepsoft.com (Robert Heller)
Date: Mon, 25 Jan 2016 17:43:09 -0500
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
Message-ID: <201601252243.u0PMh9gd013511@sharky2.deepsoft.com>

At Mon, 25 Jan 2016 22:59:47 +0100 Sascha Rissel <mailman at rissel.it> wrote:

> 
> Hello,
> 
> I have some user on my lists, who complain they don't receive the list
> mails anymore, since a few days.
> 
> Am am running Mailman 2.1.15 on Debian.
> The error log is empty, the archive seems to correctly contain all recent
> mails.
> The people saying, they don't receive the mails are on GMail and German
> Web.de hosts and they don't have bounces logged by Mailman.

Are these messages that these people have posted themselves ("I don't see the 
message *I* posted...")?  If so, this is 'normal' for GMail.  GMail 'hides' 
messages users post on E-Mail lists.

Otherwise you need to check your SMTP logs (/var/log/mailman/smtp and/or
/var/log/maillog).

The only other thought is for the users to check their SPAM folders.  It is 
also possible that your server has somehow been added to a SPAMCOP type 
blacklist.

> 
> Can you help me with this?
> Are there any means, how I can analyse this issue?
> 
> Kind regards,
> Sascha.
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/heller%40deepsoft.com
> 
>                                                     

-- 
Robert Heller             -- 978-544-6933
Deepwoods Software        -- Custom Software Services
http://www.deepsoft.com/  -- Linux Administration Services
heller at deepsoft.com       -- Webhosting Services
                                                                                                                    

From jaybird at bluegrasspals.com  Mon Jan 25 18:10:01 2016
From: jaybird at bluegrasspals.com (Jayson Smith)
Date: Mon, 25 Jan 2016 18:10:01 -0500
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
 <56A6A328.2000308@msapiro.net>
 <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
Message-ID: <56A6AB49.90708@bluegrasspals.com>

Hello,

This bounce message clearly indicates that your IP address doesn't have 
a reverse DNS entry. This is a standard lookup done as a part of spam 
fighting efforts. You need to get your provider to set the reverse DNS 
for your IP address to the domain name of your mail server if possible.

Newer Mailman versions (I know 2.1.20 at least) have an option to alert 
list admins when anyone's bounce score is incremented. I have this 
feature enabled, and it's helped me catch some bounces before they 
resulted in disabling addresses. Years ago, I switched to a new provider 
and thought everything was going well. Then five days later I received a 
bunch of bounce notices telling me that addresses had been disabled. It 
turns out my IP address was apparently on some blacklists, and the way 
Mailman used to work, I was left blissfully unaware there was a problem 
until days later, when possibly many messages to these people could have 
been rejected and lost. Until Mailman offered the choice of receiving 
alerts as soon as someone bounces a message and gets their bounce score 
incremented, the only way I could be sure my Emails weren't bouncing was 
to set Mailman to disable member addresses after just one bounce. This 
worked, but had the unpleasant side effect of disabling member addresses 
who bounced a single message because their ISP thought it was spam for 
some reason, or for some other temporary reason.

Jayson

On 1/25/2016 5:48 PM, Sascha Rissel wrote:
> Mark,
>
> thanks for your answer.
> I just checked the bounce log.
>
> And I discovered a lot of entries like:
>
> Jan 25 18:33:24 2016 (2741) <BounceRunner at 12019976> processing 9 queued
> bounces
> Jan 25 18:33:24 2016 (2741) <my-list-name>: <some mail address> already
> scored a bounce for date 25-Jan-2016
> Jan 25 18:33:24 2016 (2741) <my-list-name>: <some mail address> already
> scored a bounce for date 25-Jan-2016
> Jan 25 18:33:24 2016 (2741) <my-list-name>: <some mail address> already
> scored a bounce for date 25-Jan-2016
>
> where all mail addresses are from German Web.de and GMX. Both providers
> belong to the same company.
> Thinking about this, I remembered a bounce notification I received last
> week, where some of these addresses were set to disabled by Mailman, with
> the notification below.
>
> Did GMX/Web.de maybe change their mail processing policies with the start
> of the new year?
> Below follows an excerpt from the bounce notification.
>
> Kind regards,
> Sascha.
>
>
> Bounce Mail:
>
> <some_user at web.de>: host mx-ha02.web.de[212.227.17.8] refused to talk
>      to me: 554-web.de (mxweb107) Nemesis ESMTP Service not available 554-No
>      SMTP service 554-Bad DNS PTR resource record. 554 For explanation visit
>      http://postmaster.web.de/error-messages?ip=62.75.175.182&c=rdns
>
> Final-Recipient: rfc822; <second_user>@gmx.de
> Original-Recipient: rfc822;<second_user>@gmx.de
> Action: failed
> Status: 4.0.0
> Remote-MTA: dns; mx00.emig.gmx.net
> Diagnostic-Code: smtp; 554-gmx.net (mxgmx002) Nemesis ESMTP Service not
>      available 554-No SMTP service 554-Bad DNS PTR resource record. 554 For
>      explanation visit
>      http://postmaster.gmx.com/en/error-messages?ip=62.75.175.182&c=rdns
>
> Final-Recipient: rfc822; <some_third_user>@web.de
> Original-Recipient: rfc822;andreas_hacker at web.de
> Action: failed
> Status: 4.0.0
> Remote-MTA: dns; mx-ha02.web.de
> Diagnostic-Code: smtp; 554-web.de (mxweb107) Nemesis ESMTP Service not
>      available 554-No SMTP service 554-Bad DNS PTR resource record. 554 For
>      explanation visit
>      http://postmaster.web.de/error-messages?ip=62.75.175.182&c=rdns
>
> 2016-01-25 23:35 GMT+01:00 Mark Sapiro <mark at msapiro.net>:
>
>> On 01/25/2016 01:59 PM, Sascha Rissel wrote:
>>> I have some user on my lists, who complain they don't receive the list
>>> mails anymore, since a few days.
>>>
>>> Am am running Mailman 2.1.15 on Debian.
>>> The error log is empty, the archive seems to correctly contain all recent
>>> mails.
>>> The people saying, they don't receive the mails are on GMail and German
>>> Web.de hosts and they don't have bounces logged by Mailman.
>>
>> If there is nothing in Mailman's error log or bounce log, it is almost
>> certain the mail is being delivered by Mailman to the outgoing MTA and
>> the outgoing MTA is successfully delivering the mail to the mail
>> exchange server for the recipient domain.
>>
>> You can confirm this if you have access to the Mailman (and it's
>> outgoing MTA) server's mail.log.
>>
>> If this is the case and the users have checked their gmail or web.de
>> spam or junk folders and the messages aren't there, the messages are
>> likely being silently discarded somewhere in the delivery chain after
>> leaving the outgoing MTA.
>>
>> Solving this is difficult. Some steps are outlined in the FAQ article at
>> <http://wiki.list.org/x/4030690>.
>>
>> I sometimes will copy the specific MTA log messages indicating
>> acceptance by the receiving MTA, e.g., messages like
>>
>> Jan 24 19:09:41 sbh16 postfix/smtp[1053]: 1279111E1A8F:
>> to=<user at gmail.com>, relay=gmail-smtp-in.l.google.com[74.125.28.26]:25,
>> delay=5.8, delays=4.9/0.66/0.09/0.13, dsn=2.0.0, status=sent (250 2.0.0
>> OK 1453691381 tt2si3153213pac.167 - gsmtp)
>>
>> and tell the user to ask gmail what happened to that message. In that
>> message, (250 2.0.0 OK 1453691381 tt2si3153213pac.167 - gsmtp) is the
>> acceptance from gmail-smtp-in.l.google.com indicating the message was
>> accepted at unix time stamp 1453691381 with the accepting server's ID
>> tt2si3153213pac.167. I suspect gmail actually ignores such requests from
>> their users or provides only a generic response. People who actually pay
>> for their email service may have a bit more leverage.
>>
>> Anyway, apart from the things in the FAQ article, there's not a whole
>> lot you can do. If you can possibly identify something about the missing
>> mail that triggers it, e.g., only mail From: a certain user or domain,
>> or something in a specific thread (copied in everyone's reply), you can
>> try to avoid that, but if it's all list mail To: particular domains
>> (gmail), I think it's more likely to be a block on mail from your
>> server's IP, but in the US at least, gmail ordinarily bounces such mail
>> with a fairly specific reason.
>>
>> --
>> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
>> San Francisco Bay Area, California    better use your sense - B. Dylan
>> ------------------------------------------------------
>> Mailman-Users mailing list Mailman-Users at python.org
>> https://mail.python.org/mailman/listinfo/mailman-users
>> Mailman FAQ: http://wiki.list.org/x/AgA3
>> Security Policy: http://wiki.list.org/x/QIA9
>> Searchable Archives:
>> http://www.mail-archive.com/mailman-users%40python.org/
>> Unsubscribe:
>> https://mail.python.org/mailman/options/mailman-users/mailman%40rissel.it
>>
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/jaybird%40bluegrasspals.com
>
>


From mailman at rissel.it  Mon Jan 25 18:47:14 2016
From: mailman at rissel.it (Sascha Rissel)
Date: Tue, 26 Jan 2016 00:47:14 +0100
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <CAKGvnEAMKc-+AdiyhyPqne8Dc55=pDK1-MVUnUph1ama_0yjPA@mail.gmail.com>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
 <56A6A328.2000308@msapiro.net>
 <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
 <56A6ABB4.4@msapiro.net>
 <CAKGvnEAMKc-+AdiyhyPqne8Dc55=pDK1-MVUnUph1ama_0yjPA@mail.gmail.com>
Message-ID: <CAKGvnECmV0j5D1PxCwcuyCzKHjo9rJFEyJehqP8PFkbnU9BU1Q@mail.gmail.com>

EDIT:
For better clarification, the www domain <rc-worms.de> is hosted on a
different (dedicated) web server. Mailman is running on a vServer of its
own, it just uses <rc-worms.de> as a nice readable domain for mail
addresses, configured on the web server to forward each mail to the real
physical (and not so nicely readable) address <any
<any_mailing_list at euve51864.serverprofi24.de>_
<any_mailing_list at euve51864.serverprofi24.de>mailing
<any_mailing_list at euve51864.serverprofi24.de>
_list at euve51864.serverprofi24.de
<any_mailing_list at euve51864.serverprofi24.de>>.
Am 26.01.2016 12:27 vorm. schrieb "Sascha Rissel" <mailman at rissel.it>:

> Hi Mark!
>
> > In simple terms, it claims your mail server is not properly configured.
> > I don't know what your Mailman server's domain name is, but it appears
> > from the error message that its IP is 62.75.175.182 and that IP has an
> > rDNS PTR record to euve51864.serverprofi24.de.
>
> this is all pretty correct.
>
> "euve51864.serverprofi24.de" is the natural name of the vServer, I am
> running.
> I don't have a real domain configured, since I mainly just use it for
> mailman.
> I configured all Mailman lists to use something like <list name>@
> rc-worms.de, where <rc-worms.de> is the www domain of my sports club,
> whose administrator I am.
> For better readability I named and configured all lists this way, to use <
> members at rc-worms.de>, <team1 at rc-worms.de> as exlicity reply-to address
> and so on.
>
> This worked pretty well for years.
> Did it now turn into a problem, that reply-to address and domain () bear a
> different IP/rDNS result?
> Is there something you could suggest in this case?
>
> Should I maybe configure a subdomain on my vServer, like <
> lists.rc-worms.de>, from which the mails will be send from and can bear
> own DNS config?
>
> Kind regards,
> Sascha.
>
>
>
>
> 2016-01-26 0:11 GMT+01:00 Mark Sapiro <mark at msapiro.net>:
>
>> On 01/25/2016 02:48 PM, Sascha Rissel wrote:
>> >
>> > thanks for your answer.
>> > I just checked the bounce log.
>> >
>> > And I discovered a lot of entries like:
>> >
>> > Jan 25 18:33:24 2016 (2741) <BounceRunner at 12019976> processing 9
>> queued
>> > bounces
>> > Jan 25 18:33:24 2016 (2741) <my-list-name>: <some mail address> already
>> > scored a bounce for date 25-Jan-2016
>> ...
>> > where all mail addresses are from German Web.de and GMX. Both providers
>> > belong to the same company.
>> > Thinking about this, I remembered a bounce notification I received last
>> > week, where some of these addresses were set to disabled by Mailman,
>> with
>> > the notification below.
>> >
>> > Did GMX/Web.de maybe change their mail processing policies with the
>> start
>> > of the new year?
>>
>>
>> Possibly. See below.
>>
>>
>> > Below follows an excerpt from the bounce notification.
>> >
>> > Bounce Mail:
>> >
>> > <some_user at web.de>: host mx-ha02.web.de[212.227.17.8] refused to talk
>> >     to me: 554-web.de (mxweb107) Nemesis ESMTP Service not available
>> 554-No
>> >     SMTP service 554-Bad DNS PTR resource record. 554 For explanation
>> visit
>> >     http://postmaster.web.de/error-messages?ip=62.75.175.182&c=rdns
>>
>>
>> OK. The recipient's mail exchange server is refusing your mail. web.de's
>> explanation of this is at
>> <http://postmaster.web.de/error-messages?ip=62.75.175.182&c=rdns#nordns>.
>>
>> In simple terms, it claims your mail server is not properly configured.
>> I don't know what your Mailman server's domain name is, but it appears
>> from the error message that its IP is 62.75.175.182 and that IP has an
>> rDNS PTR record to euve51864.serverprofi24.de.
>>
>> In any case you need to ensure the IP address your server sends from has
>> a rDNS (PTR) record pointing to its host name and that host name in turn
>> needs an A record with the same IP address. Also, it should identify
>> itself with this same name in the SMTP HELO or EHLO command.
>>
>> According to the tool at
>> <
>> http://mxtoolbox.com/SuperTool.aspx?action=smtp%3a62.75.175.182&run=toolpage
>> >,
>> this is all OK, so I don't know why web.de is complaining. I suggest you
>> use the form at
>> <http://postmaster.web.de/en/contact/?ip=62.75.175.182&c=rdns> to ask
>> them.
>>
>> --
>> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
>> San Francisco Bay Area, California    better use your sense - B. Dylan
>> ------------------------------------------------------
>> Mailman-Users mailing list Mailman-Users at python.org
>> https://mail.python.org/mailman/listinfo/mailman-users
>> Mailman FAQ: http://wiki.list.org/x/AgA3
>> Security Policy: http://wiki.list.org/x/QIA9
>> Searchable Archives:
>> http://www.mail-archive.com/mailman-users%40python.org/
>> Unsubscribe:
>> https://mail.python.org/mailman/options/mailman-users/mailman%40rissel.it
>>
>
>

From mark at msapiro.net  Mon Jan 25 18:48:34 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 25 Jan 2016 15:48:34 -0800
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <CAKGvnEAMKc-+AdiyhyPqne8Dc55=pDK1-MVUnUph1ama_0yjPA@mail.gmail.com>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
 <56A6A328.2000308@msapiro.net>
 <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
 <56A6ABB4.4@msapiro.net>
 <CAKGvnEAMKc-+AdiyhyPqne8Dc55=pDK1-MVUnUph1ama_0yjPA@mail.gmail.com>
Message-ID: <56A6B452.5020806@msapiro.net>

On 01/25/2016 03:27 PM, Sascha Rissel wrote:
> Hi Mark!
> 
>> In simple terms, it claims your mail server is not properly configured.
>> I don't know what your Mailman server's domain name is, but it appears
>> from the error message that its IP is 62.75.175.182 and that IP has an
>> rDNS PTR record to euve51864.serverprofi24.de.
> 
> this is all pretty correct.
> 
> "euve51864.serverprofi24.de" is the natural name of the vServer, I am
> running.
> I don't have a real domain configured, since I mainly just use it for
> mailman.


As I indicated in a follow-up post web.de is apparently not allowing
'generic' domains any more.


> I configured all Mailman lists to use something like <list name>@rc-worms.de,
> where <rc-worms.de> is the www domain of my sports club, whose
> administrator I am.
> For better readability I named and configured all lists this way, to use <
> members at rc-worms.de>, <team1 at rc-worms.de> as exlicity reply-to address and
> so on.
> 
> This worked pretty well for years.
> Did it now turn into a problem, that reply-to address and domain () bear a
> different IP/rDNS result?


That is not the problem as far as I can tell. The problem is that the
specific domain you are using for your mail server,
euve51864.serverprofi24.de, is now seen as a 'generic' name and is not
acceptable to web.de.


> Is there something you could suggest in this case?
> 
> Should I maybe configure a subdomain on my vServer, like <lists.rc-worms.de>,
> from which the mails will be send from and can bear own DNS config?


Yes. lists.rc-worms.de should be good. I would have suggested just
rc-worms.de, but that already has an A record pointing to a different
server.

The issue you will run into is that while you can configure the MTA to
identify itself as lists.rc-worms.de and you can create the A record for
it, you don't control the rDNS PTR record for 62.75.175.182. That's
controlled by intergenia.de. It may be no problem to get them to change
it, but it could be a stumbling block. You'll find out when you ask.

With respect to the underlying issue here, ISPs are getting more and
more fussy about only accepting mail from what they consider to be real
mail servers. Trying to run a mail server with a dynamic IP or an IP
with a 'generic' host name is just not possible if you want reliable
acceptance of your mail.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mailman at rissel.it  Mon Jan 25 19:27:24 2016
From: mailman at rissel.it (Sascha Rissel)
Date: Tue, 26 Jan 2016 01:27:24 +0100
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <56A6B452.5020806@msapiro.net>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
 <56A6A328.2000308@msapiro.net>
 <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
 <56A6ABB4.4@msapiro.net>
 <CAKGvnEAMKc-+AdiyhyPqne8Dc55=pDK1-MVUnUph1ama_0yjPA@mail.gmail.com>
 <56A6B452.5020806@msapiro.net>
Message-ID: <CAKGvnECDFe-_EJ2EpB2=_uvS0pse1OJgXU4KuTdJTt9B-zfjzw@mail.gmail.com>

Dear Mark and all others,

many thanks for your help so far.
I am just starting to setup this subdomain, Mark advised me to.

One thing regarding the Reverse DNS entry:
How can it be necessary, to set up this record? Just imagine a server,
where multiple domains are hosted, but the server only has one IP.
If the ISPs really require rDNS entries, which suit to the mail originating
domain, one would always need a 1:1 mapping of IP and domain, preventing
multiple domains on a one-IP server, isn't it?

Kind regards,
Sascha.

2016-01-26 0:48 GMT+01:00 Mark Sapiro <mark at msapiro.net>:

> On 01/25/2016 03:27 PM, Sascha Rissel wrote:
> > Hi Mark!
> >
> >> In simple terms, it claims your mail server is not properly configured.
> >> I don't know what your Mailman server's domain name is, but it appears
> >> from the error message that its IP is 62.75.175.182 and that IP has an
> >> rDNS PTR record to euve51864.serverprofi24.de.
> >
> > this is all pretty correct.
> >
> > "euve51864.serverprofi24.de" is the natural name of the vServer, I am
> > running.
> > I don't have a real domain configured, since I mainly just use it for
> > mailman.
>
>
> As I indicated in a follow-up post web.de is apparently not allowing
> 'generic' domains any more.
>
>
> > I configured all Mailman lists to use something like <list name>@
> rc-worms.de,
> > where <rc-worms.de> is the www domain of my sports club, whose
> > administrator I am.
> > For better readability I named and configured all lists this way, to use
> <
> > members at rc-worms.de>, <team1 at rc-worms.de> as exlicity reply-to address
> and
> > so on.
> >
> > This worked pretty well for years.
> > Did it now turn into a problem, that reply-to address and domain () bear
> a
> > different IP/rDNS result?
>
>
> That is not the problem as far as I can tell. The problem is that the
> specific domain you are using for your mail server,
> euve51864.serverprofi24.de, is now seen as a 'generic' name and is not
> acceptable to web.de.
>
>
> > Is there something you could suggest in this case?
> >
> > Should I maybe configure a subdomain on my vServer, like <
> lists.rc-worms.de>,
> > from which the mails will be send from and can bear own DNS config?
>
>
> Yes. lists.rc-worms.de should be good. I would have suggested just
> rc-worms.de, but that already has an A record pointing to a different
> server.
>
> The issue you will run into is that while you can configure the MTA to
> identify itself as lists.rc-worms.de and you can create the A record for
> it, you don't control the rDNS PTR record for 62.75.175.182. That's
> controlled by intergenia.de. It may be no problem to get them to change
> it, but it could be a stumbling block. You'll find out when you ask.
>
> With respect to the underlying issue here, ISPs are getting more and
> more fussy about only accepting mail from what they consider to be real
> mail servers. Trying to run a mail server with a dynamic IP or an IP
> with a 'generic' host name is just not possible if you want reliable
> acceptance of your mail.
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/mailman%40rissel.it
>

From mark at msapiro.net  Mon Jan 25 20:00:51 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 25 Jan 2016 17:00:51 -0800
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <CAKGvnECDFe-_EJ2EpB2=_uvS0pse1OJgXU4KuTdJTt9B-zfjzw@mail.gmail.com>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
 <56A6A328.2000308@msapiro.net>
 <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
 <56A6ABB4.4@msapiro.net>
 <CAKGvnEAMKc-+AdiyhyPqne8Dc55=pDK1-MVUnUph1ama_0yjPA@mail.gmail.com>
 <56A6B452.5020806@msapiro.net>
 <CAKGvnECDFe-_EJ2EpB2=_uvS0pse1OJgXU4KuTdJTt9B-zfjzw@mail.gmail.com>
Message-ID: <56A6C543.9030500@msapiro.net>

On 01/25/2016 04:27 PM, Sascha Rissel wrote:
> 
> One thing regarding the Reverse DNS entry:
> How can it be necessary, to set up this record? Just imagine a server,
> where multiple domains are hosted, but the server only has one IP.
> If the ISPs really require rDNS entries, which suit to the mail originating
> domain, one would always need a 1:1 mapping of IP and domain, preventing
> multiple domains on a one-IP server, isn't it?


No. It doesn't matter how many From: or SMTP MAIL FROM domains the
server supports. the crucial things are:

The IP address of the server has an rDNS PTR record pointing to a name.
That name has an A record with the same IP address and that name is the
name with which the MTA identifies itself in SMTP HELO or EHLO.

In addition, web.de and lots of other ISPs require that the name itself
not be a generic looking name, and some go so far as to not like IP
addresses that are in a netblock used by an ISP for 'home' connections.
On this latter point I run an MTA on msapiro.net which at one time was
on a 'home' DSL connection with fixed IP with rDNS to msapiro.net, but
in spite of this proper configuration with a non-generic domain, I would
occasionally get flagged by Microsoft for "bot like behavior", the only
plausible reason for which is the IP was in a range used by the DSL
provider for 'home use'

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From Richard at Damon-Family.org  Mon Jan 25 20:02:23 2016
From: Richard at Damon-Family.org (Richard Damon)
Date: Mon, 25 Jan 2016 20:02:23 -0500
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <CAKGvnECDFe-_EJ2EpB2=_uvS0pse1OJgXU4KuTdJTt9B-zfjzw@mail.gmail.com>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
 <56A6A328.2000308@msapiro.net>
 <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
 <56A6ABB4.4@msapiro.net>
 <CAKGvnEAMKc-+AdiyhyPqne8Dc55=pDK1-MVUnUph1ama_0yjPA@mail.gmail.com>
 <56A6B452.5020806@msapiro.net>
 <CAKGvnECDFe-_EJ2EpB2=_uvS0pse1OJgXU4KuTdJTt9B-zfjzw@mail.gmail.com>
Message-ID: <56A6C59F.2040302@Damon-Family.org>

On 1/25/16 7:27 PM, Sascha Rissel wrote:
> Dear Mark and all others,
>
> many thanks for your help so far.
> I am just starting to setup this subdomain, Mark advised me to.
>
> One thing regarding the Reverse DNS entry:
> How can it be necessary, to set up this record? Just imagine a server,
> where multiple domains are hosted, but the server only has one IP.
> If the ISPs really require rDNS entries, which suit to the mail originating
> domain, one would always need a 1:1 mapping of IP and domain, preventing
> multiple domains on a one-IP server, isn't it?
>
> Kind regards,
> Sascha.
>
The server name does not need to be connected to the domain the mail is 
'from'. My outgoing mail uses an outgoing SMTP server not in anyway 
connected to my domain. If someone REALLY wants their mail to fully 
appear (even to people digging into headers and server logs) as if it is 
coming through a dedicated mail path, then they are going to need to pay 
their MSP for the additional IP addresses to make that happen.

-- 
Richard Damon


From heller at deepsoft.com  Mon Jan 25 21:45:15 2016
From: heller at deepsoft.com (Robert Heller)
Date: Mon, 25 Jan 2016 21:45:15 -0500
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <CAKGvnECDFe-_EJ2EpB2=_uvS0pse1OJgXU4KuTdJTt9B-zfjzw@mail.gmail.com>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com> 
 <56A6A328.2000308@msapiro.net> 
 <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com> 
 <56A6ABB4.4@msapiro.net> 
 <CAKGvnEAMKc-+AdiyhyPqne8Dc55=pDK1-MVUnUph1ama_0yjPA@mail.gmail.com> 
 <56A6B452.5020806@msapiro.net> 
 <CAKGvnECDFe-_EJ2EpB2=_uvS0pse1OJgXU4KuTdJTt9B-zfjzw@mail.gmail.com>
Message-ID: <201601260245.u0Q2jFHM018848@sharky2.deepsoft.com>

At Tue, 26 Jan 2016 01:27:24 +0100 Sascha Rissel <mailman at rissel.it> wrote:

> 
> Dear Mark and all others,
> 
> many thanks for your help so far.
> I am just starting to setup this subdomain, Mark advised me to.
> 
> One thing regarding the Reverse DNS entry:
> How can it be necessary, to set up this record? Just imagine a server,
> where multiple domains are hosted, but the server only has one IP.
> If the ISPs really require rDNS entries, which suit to the mail originating
> domain, one would always need a 1:1 mapping of IP and domain, preventing
> multiple domains on a one-IP server, isn't it?

You just need one 'real' canonical domain name, like lists.rc-worms.de, and
have a rDNS PTR to lists.rc-worms.de. It does not really matter if there is a
foo-list at foobar.com hosted there. What is being objected to is the
name<number>.hosting-provider.tld type of name, as well as things like a HELO
that does not match, and such. It has the look-and-feel of a spammer sending
anonymous E-Mails or E-Mails with forged / spoofed addresses (including
spoofed HELO's). Also the list server's MTA should be using the
lists.rc-worms.de name in the HELO. And you probably should also have
a spf record as well. Everything should 'match': the A record, the rDNS PTR
record, the HELO name used, the SPF record, etc. That makes everything look
legit and everything fits together.

> 
> Kind regards,
> Sascha.
> 
> 2016-01-26 0:48 GMT+01:00 Mark Sapiro <mark at msapiro.net>:
> 
> > On 01/25/2016 03:27 PM, Sascha Rissel wrote:
> > > Hi Mark!
> > >
> > >> In simple terms, it claims your mail server is not properly configured.
> > >> I don't know what your Mailman server's domain name is, but it appears
> > >> from the error message that its IP is 62.75.175.182 and that IP has an
> > >> rDNS PTR record to euve51864.serverprofi24.de.
> > >
> > > this is all pretty correct.
> > >
> > > "euve51864.serverprofi24.de" is the natural name of the vServer, I am
> > > running.
> > > I don't have a real domain configured, since I mainly just use it for
> > > mailman.
> >
> >
> > As I indicated in a follow-up post web.de is apparently not allowing
> > 'generic' domains any more.
> >
> >
> > > I configured all Mailman lists to use something like <list name>@
> > rc-worms.de,
> > > where <rc-worms.de> is the www domain of my sports club, whose
> > > administrator I am.
> > > For better readability I named and configured all lists this way, to use
> > <
> > > members at rc-worms.de>, <team1 at rc-worms.de> as exlicity reply-to address
> > and
> > > so on.
> > >
> > > This worked pretty well for years.
> > > Did it now turn into a problem, that reply-to address and domain () bear
> > a
> > > different IP/rDNS result?
> >
> >
> > That is not the problem as far as I can tell. The problem is that the
> > specific domain you are using for your mail server,
> > euve51864.serverprofi24.de, is now seen as a 'generic' name and is not
> > acceptable to web.de.
> >
> >
> > > Is there something you could suggest in this case?
> > >
> > > Should I maybe configure a subdomain on my vServer, like <
> > lists.rc-worms.de>,
> > > from which the mails will be send from and can bear own DNS config?
> >
> >
> > Yes. lists.rc-worms.de should be good. I would have suggested just
> > rc-worms.de, but that already has an A record pointing to a different
> > server.
> >
> > The issue you will run into is that while you can configure the MTA to
> > identify itself as lists.rc-worms.de and you can create the A record for
> > it, you don't control the rDNS PTR record for 62.75.175.182. That's
> > controlled by intergenia.de. It may be no problem to get them to change
> > it, but it could be a stumbling block. You'll find out when you ask.
> >
> > With respect to the underlying issue here, ISPs are getting more and
> > more fussy about only accepting mail from what they consider to be real
> > mail servers. Trying to run a mail server with a dynamic IP or an IP
> > with a 'generic' host name is just not possible if you want reliable
> > acceptance of your mail.
> >
> > --
> > Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> > San Francisco Bay Area, California    better use your sense - B. Dylan
> > ------------------------------------------------------
> > Mailman-Users mailing list Mailman-Users at python.org
> > https://mail.python.org/mailman/listinfo/mailman-users
> > Mailman FAQ: http://wiki.list.org/x/AgA3
> > Security Policy: http://wiki.list.org/x/QIA9
> > Searchable Archives:
> > http://www.mail-archive.com/mailman-users%40python.org/
> > Unsubscribe:
> > https://mail.python.org/mailman/options/mailman-users/mailman%40rissel.it
> >
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe: https://mail.python.org/mailman/options/mailman-users/heller%40deepsoft.com
> 
>                                                                             

-- 
Robert Heller             -- 978-544-6933
Deepwoods Software        -- Custom Software Services
http://www.deepsoft.com/  -- Linux Administration Services
heller at deepsoft.com       -- Webhosting Services
                                                                                                       

From mailman at icomdesign.com  Mon Jan 25 17:20:29 2016
From: mailman at icomdesign.com (Matt Milunic)
Date: Mon, 25 Jan 2016 17:20:29 -0500
Subject: [Mailman-Users] Web Admin Preferences
Message-ID: <C68EF114-4958-4FCA-8B77-0BA598B94F51@icomdesign.com>

I am running mailman as part of the bundled Mac OS X 10.6 Server. Recently, we had a hard drive failure. All was reinstalled from a cloned backup. Mailman "works? in that I can send messages and they go out and are received properly. The issue is that i cannot log into any of the list web admin?s to delete/add addresses, etc. When I try I get:

-----------------------------------------

Bug in Mailman version 2.1.14

We're sorry, we hit a bug!

Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs. 

????????????????????

If I put in an incorrect password I get the red ?Authorization Failed? message up top.

I have looked-up and run various terminal commands, followed by restarting mail services and web services. Among these are:

chmod 02775 /private/var/mailman

chmod 02775 /usr/share/mailman

chmod 02775 /usr/share/mailman/messages/ar

chown _mailman:_mailman /private/var/mailman

chmod a+rw,g+ws /private/var/mailman

Nothing seems to reset the permissions to allow me to log into the admin. 

I am not as savvy as the rest of you on this list, so simple terms and plain English appreciated :)

From mark at msapiro.net  Tue Jan 26 13:28:08 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 26 Jan 2016 10:28:08 -0800
Subject: [Mailman-Users] Web Admin Preferences
In-Reply-To: <C68EF114-4958-4FCA-8B77-0BA598B94F51@icomdesign.com>
References: <C68EF114-4958-4FCA-8B77-0BA598B94F51@icomdesign.com>
Message-ID: <56A7BAB8.7030303@msapiro.net>

On 01/25/2016 02:20 PM, Matt Milunic wrote:
> I am running mailman as part of the bundled Mac OS X 10.6 Server. Recently, we had a hard drive failure. All was reinstalled from a cloned backup. Mailman "works? in that I can send messages and they go out and are received properly. The issue is that i cannot log into any of the list web admin?s to delete/add addresses, etc. When I try I get:
> 
> -----------------------------------------
> 
> Bug in Mailman version 2.1.14
> 
> We're sorry, we hit a bug!
> 
> Please inform the webmaster for this site of this problem. Printing of traceback and other system information has been explicitly inhibited, but the webmaster can find this information in the Mailman error logs. 
> 
> ????????????????????
> 
> If I put in an incorrect password I get the red ?Authorization Failed? message up top.
> 
> I have looked-up and run various terminal commands, followed by restarting mail services and web services. Among these are:

> chmod 02775 /private/var/mailman
> 
> chmod 02775 /usr/share/mailman
> 
> chmod 02775 /usr/share/mailman/messages/ar
> 
> chown _mailman:_mailman /private/var/mailman
> 
> chmod a+rw,g+ws /private/var/mailman
> 
> Nothing seems to reset the permissions to allow me to log into the admin. 


Have you run Mailman's 'bin/check_perms -f' as root or via sudo?

If you've done this, or you do it now and it doesn't help, please post
the entire traceback from Mailman's error log for one of these exceptions.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mailman at rissel.it  Tue Jan 26 18:56:46 2016
From: mailman at rissel.it (Sascha Rissel)
Date: Wed, 27 Jan 2016 00:56:46 +0100
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <201601260245.u0Q2jFHM018848@sharky2.deepsoft.com>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
 <56A6A328.2000308@msapiro.net>
 <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
 <56A6ABB4.4@msapiro.net>
 <CAKGvnEAMKc-+AdiyhyPqne8Dc55=pDK1-MVUnUph1ama_0yjPA@mail.gmail.com>
 <56A6B452.5020806@msapiro.net>
 <CAKGvnECDFe-_EJ2EpB2=_uvS0pse1OJgXU4KuTdJTt9B-zfjzw@mail.gmail.com>
 <201601260245.u0Q2jFHM018848@sharky2.deepsoft.com>
Message-ID: <CAKGvnEA5YWW7vOsqz3HW1uSpjOL9O-54=x4hWsFk+ZNCOPiD+w@mail.gmail.com>

Hi Guys!

Many thanks for all your help so far!
I managed to configure a valid rDNS record to day and the users from the
blockings ISPs startet to receive mails again!

Now I use the domain lists.rc-worms.de. If I enter that into the tool Mark
mentioned (mxtoolbox.com), everything is stated as fine, except for
"Reverse DNS does not match SMTP Banner".

Could you please try to explain this a bit for me?
I use Postfix as MTA...

Many thanks again,
Sascha.
Am 26.01.2016 3:45 vorm. schrieb "Robert Heller" <heller at deepsoft.com>:

> At Tue, 26 Jan 2016 01:27:24 +0100 Sascha Rissel <mailman at rissel.it>
> wrote:
>
> >
> > Dear Mark and all others,
> >
> > many thanks for your help so far.
> > I am just starting to setup this subdomain, Mark advised me to.
> >
> > One thing regarding the Reverse DNS entry:
> > How can it be necessary, to set up this record? Just imagine a server,
> > where multiple domains are hosted, but the server only has one IP.
> > If the ISPs really require rDNS entries, which suit to the mail
> originating
> > domain, one would always need a 1:1 mapping of IP and domain, preventing
> > multiple domains on a one-IP server, isn't it?
>
> You just need one 'real' canonical domain name, like lists.rc-worms.de,
> and
> have a rDNS PTR to lists.rc-worms.de. It does not really matter if there
> is a
> foo-list at foobar.com hosted there. What is being objected to is the
> name<number>.hosting-provider.tld type of name, as well as things like a
> HELO
> that does not match, and such. It has the look-and-feel of a spammer
> sending
> anonymous E-Mails or E-Mails with forged / spoofed addresses (including
> spoofed HELO's). Also the list server's MTA should be using the
> lists.rc-worms.de name in the HELO. And you probably should also have
> a spf record as well. Everything should 'match': the A record, the rDNS PTR
> record, the HELO name used, the SPF record, etc. That makes everything look
> legit and everything fits together.
>
> >
> > Kind regards,
> > Sascha.
> >
> > 2016-01-26 0:48 GMT+01:00 Mark Sapiro <mark at msapiro.net>:
> >
> > > On 01/25/2016 03:27 PM, Sascha Rissel wrote:
> > > > Hi Mark!
> > > >
> > > >> In simple terms, it claims your mail server is not properly
> configured.
> > > >> I don't know what your Mailman server's domain name is, but it
> appears
> > > >> from the error message that its IP is 62.75.175.182 and that IP has
> an
> > > >> rDNS PTR record to euve51864.serverprofi24.de.
> > > >
> > > > this is all pretty correct.
> > > >
> > > > "euve51864.serverprofi24.de" is the natural name of the vServer, I
> am
> > > > running.
> > > > I don't have a real domain configured, since I mainly just use it for
> > > > mailman.
> > >
> > >
> > > As I indicated in a follow-up post web.de is apparently not allowing
> > > 'generic' domains any more.
> > >
> > >
> > > > I configured all Mailman lists to use something like <list name>@
> > > rc-worms.de,
> > > > where <rc-worms.de> is the www domain of my sports club, whose
> > > > administrator I am.
> > > > For better readability I named and configured all lists this way, to
> use
> > > <
> > > > members at rc-worms.de>, <team1 at rc-worms.de> as exlicity reply-to
> address
> > > and
> > > > so on.
> > > >
> > > > This worked pretty well for years.
> > > > Did it now turn into a problem, that reply-to address and domain ()
> bear
> > > a
> > > > different IP/rDNS result?
> > >
> > >
> > > That is not the problem as far as I can tell. The problem is that the
> > > specific domain you are using for your mail server,
> > > euve51864.serverprofi24.de, is now seen as a 'generic' name and is not
> > > acceptable to web.de.
> > >
> > >
> > > > Is there something you could suggest in this case?
> > > >
> > > > Should I maybe configure a subdomain on my vServer, like <
> > > lists.rc-worms.de>,
> > > > from which the mails will be send from and can bear own DNS config?
> > >
> > >
> > > Yes. lists.rc-worms.de should be good. I would have suggested just
> > > rc-worms.de, but that already has an A record pointing to a different
> > > server.
> > >
> > > The issue you will run into is that while you can configure the MTA to
> > > identify itself as lists.rc-worms.de and you can create the A record
> for
> > > it, you don't control the rDNS PTR record for 62.75.175.182. That's
> > > controlled by intergenia.de. It may be no problem to get them to
> change
> > > it, but it could be a stumbling block. You'll find out when you ask.
> > >
> > > With respect to the underlying issue here, ISPs are getting more and
> > > more fussy about only accepting mail from what they consider to be real
> > > mail servers. Trying to run a mail server with a dynamic IP or an IP
> > > with a 'generic' host name is just not possible if you want reliable
> > > acceptance of your mail.
> > >
> > > --
> > > Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> > > San Francisco Bay Area, California    better use your sense - B. Dylan
> > > ------------------------------------------------------
> > > Mailman-Users mailing list Mailman-Users at python.org
> > > https://mail.python.org/mailman/listinfo/mailman-users
> > > Mailman FAQ: http://wiki.list.org/x/AgA3
> > > Security Policy: http://wiki.list.org/x/QIA9
> > > Searchable Archives:
> > > http://www.mail-archive.com/mailman-users%40python.org/
> > > Unsubscribe:
> > >
> https://mail.python.org/mailman/options/mailman-users/mailman%40rissel.it
> > >
> > ------------------------------------------------------
> > Mailman-Users mailing list Mailman-Users at python.org
> > https://mail.python.org/mailman/listinfo/mailman-users
> > Mailman FAQ: http://wiki.list.org/x/AgA3
> > Security Policy: http://wiki.list.org/x/QIA9
> > Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> > Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/heller%40deepsoft.com
> >
> >
>
> --
> Robert Heller             -- 978-544-6933
> Deepwoods Software        -- Custom Software Services
> http://www.deepsoft.com/  -- Linux Administration Services
> heller at deepsoft.com       -- Webhosting Services
>
>

From mark at msapiro.net  Tue Jan 26 19:11:02 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 26 Jan 2016 16:11:02 -0800
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <CAKGvnEA5YWW7vOsqz3HW1uSpjOL9O-54=x4hWsFk+ZNCOPiD+w@mail.gmail.com>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
 <56A6A328.2000308@msapiro.net>
 <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
 <56A6ABB4.4@msapiro.net>
 <CAKGvnEAMKc-+AdiyhyPqne8Dc55=pDK1-MVUnUph1ama_0yjPA@mail.gmail.com>
 <56A6B452.5020806@msapiro.net>
 <CAKGvnECDFe-_EJ2EpB2=_uvS0pse1OJgXU4KuTdJTt9B-zfjzw@mail.gmail.com>
 <201601260245.u0Q2jFHM018848@sharky2.deepsoft.com>
 <CAKGvnEA5YWW7vOsqz3HW1uSpjOL9O-54=x4hWsFk+ZNCOPiD+w@mail.gmail.com>
Message-ID: <56A80B16.6070303@msapiro.net>

On 01/26/2016 03:56 PM, Sascha Rissel wrote:
> 
> Now I use the domain lists.rc-worms.de. If I enter that into the tool Mark
> mentioned (mxtoolbox.com), everything is stated as fine, except for
> "Reverse DNS does not match SMTP Banner".
> 
> Could you please try to explain this a bit for me?
> I use Postfix as MTA...


Postfix is still identifying itself as euve51864.serverprofi24.de. You need

myhostname = lists.rc-worms.de

in Postfix main.cf

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

From mailman at rissel.it  Wed Jan 27 01:19:57 2016
From: mailman at rissel.it (Sascha Rissel)
Date: Wed, 27 Jan 2016 07:19:57 +0100
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <56A80B16.6070303@msapiro.net>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
 <56A6A328.2000308@msapiro.net>
 <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
 <56A6ABB4.4@msapiro.net>
 <CAKGvnEAMKc-+AdiyhyPqne8Dc55=pDK1-MVUnUph1ama_0yjPA@mail.gmail.com>
 <56A6B452.5020806@msapiro.net>
 <CAKGvnECDFe-_EJ2EpB2=_uvS0pse1OJgXU4KuTdJTt9B-zfjzw@mail.gmail.com>
 <201601260245.u0Q2jFHM018848@sharky2.deepsoft.com>
 <CAKGvnEA5YWW7vOsqz3HW1uSpjOL9O-54=x4hWsFk+ZNCOPiD+w@mail.gmail.com>
 <56A80B16.6070303@msapiro.net>
Message-ID: <CAKGvnEDVd_XurSi1LHriQykuFNEwB4JgbiGpx7XquTBQ==FiaA@mail.gmail.com>

Thanks Mark!

Is this the SMTP HELO/EHLO configuration, you spoke of?

Regards,
Sascha.
Am 27.01.2016 1:11 vorm. schrieb "Mark Sapiro" <mark at msapiro.net>:

> On 01/26/2016 03:56 PM, Sascha Rissel wrote:
> >
> > Now I use the domain lists.rc-worms.de. If I enter that into the tool
> Mark
> > mentioned (mxtoolbox.com), everything is stated as fine, except for
> > "Reverse DNS does not match SMTP Banner".
> >
> > Could you please try to explain this a bit for me?
> > I use Postfix as MTA...
>
>
> Postfix is still identifying itself as euve51864.serverprofi24.de. You
> need
>
> myhostname = lists.rc-worms.de
>
> in Postfix main.cf
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
> ------------------------------------------------------
> Mailman-Users mailing list Mailman-Users at python.org
> https://mail.python.org/mailman/listinfo/mailman-users
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Security Policy: http://wiki.list.org/x/QIA9
> Searchable Archives:
> http://www.mail-archive.com/mailman-users%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-users/mailman%40rissel.it
>

From mark at msapiro.net  Wed Jan 27 01:23:31 2016
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 26 Jan 2016 22:23:31 -0800
Subject: [Mailman-Users] Users complain,
 they don't receive mails from the list
In-Reply-To: <CAKGvnEDVd_XurSi1LHriQykuFNEwB4JgbiGpx7XquTBQ==FiaA@mail.gmail.com>
References: <CAKGvnECFbTPk+j0pUXGXPd-JTyixO=qeDd0oNEAcWGaY8+JJbw@mail.gmail.com>
 <56A6A328.2000308@msapiro.net>
 <CAKGvnEBT8d9k-FomBFyNVkjA2qZhXDwys_0oYEDMXv+0YUAu7A@mail.gmail.com>
 <56A6ABB4.4@msapiro.net>
 <CAKGvnEAMKc-+AdiyhyPqne8Dc55=pDK1-MVUnUph1ama_0yjPA@mail.gmail.com>
 <56A6B452.5020806@msapiro.net>
 <CAKGvnECDFe-_EJ2EpB2=_uvS0pse1OJgXU4KuTdJTt9B-zfjzw@mail.gmail.com>
 <201601260245.u0Q2jFHM018848@sharky2.deepsoft.com>
 <CAKGvnEA5YWW7vOsqz3HW1uSpjOL9O-54=x4hWsFk+ZNCOPiD+w@mail.gmail.com>
 <56A80B16.6070303@msapiro.net>
 <CAKGvnEDVd_XurSi1LHriQykuFNEwB4JgbiGpx7XquTBQ==FiaA@mail.gmail.com>
Message-ID: <56A86263.60605@msapiro.net>

On 01/26/2016 10:19 PM, Sascha Rissel wrote:
> Thanks Mark!
> 
> Is this the SMTP HELO/EHLO configuration, you spoke of?


Yes it is.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan