From barry at python.org  Fri Sep  1 00:12:34 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 31 Aug 2006 18:12:34 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8005] trunk/mailman/messages/de
In-Reply-To: <E1GItul-0001pU-6F@sc8-pr-svn1.sourceforge.net>
References: <E1GItul-0001pU-6F@sc8-pr-svn1.sourceforge.net>
Message-ID: <290B84FB-BABD-41CE-B687-3BBEE098DC4F@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Aug 31, 2006, at 5:17 PM, pheinlein at users.sourceforge.net wrote:

> Revision: 8005
>           http://svn.sourceforge.net/mailman/?rev=8005&view=rev
> Author:   pheinlein
> Date:     2006-08-31 14:17:24 -0700 (Thu, 31 Aug 2006)

Peer, don't forget to commit any necessary changes to the Release_2_1- 
maint branch, if you want to get them into 2.1.9.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRPde13EjvBPtnXfVAQIwDgQApBgULRT9gi/bSuXWSFkBWSwECxkQcmta
guEHOs1spkPlm26MHK+ub0ezInN5OzI0hYnWo1Y+gu0rSK6kRHLXrcGMPjhUoVbK
YR9ClYNoqJuJINj3SCJAxM45H9sQuSZ/20ToDL4fvxHw7G7Hw2dKmJMUY59fhD1e
v4tXzSY3yto=
=eQBN
-----END PGP SIGNATURE-----

From barry at python.org  Sat Sep  2 20:47:32 2006
From: barry at python.org (Barry Warsaw)
Date: Sat, 2 Sep 2006 14:47:32 -0400
Subject: [Mailman-Developers] RELEASED Mailman 2.1.9 release candidate 1
Message-ID: <243CCA3B-B7E9-42E4-9D67-A24E7AC91154@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Mailman development team, I'm happy to announce the  
release of Mailman 2.1.9 release candidate 1.  This is primarily a  
bug fix and security release, although it also contains two new  
languages: Arabic and Vietnamese.

This version is not yet recommended for production environments,  
however testing and feedback is greatly encouraged.  My plan is to  
release 2.1.9 final by 10-Sep-2006.  A more detailed list of changes  
will be included in the final release announcement.

Mailman 2.1.9rc1.tgz is available from SourceForge:

https://sourceforge.net/project/showfiles.php?group_id=103

Translators: If you have any language updates you'd like to see in  
2.1.9 final, please commit them now or send them to me.

Enjoy,
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRPnRyXEjvBPtnXfVAQKuLAP/aUjiZEelQy/oObIadsrhVl9YzP9dcbfE
jK1rJuGQsB7VKHe2X/uQWuaAy95pxhNwo/j/N9qtTaSjlZvjTC74E8WboxJmCemf
A+azXWgaVY/C3L+HDneqFGBVZLXkKg+4IfxmKPALcEs88jHGzOpvlnuZEVzpyaon
bOYFooMqc3c=
=NWI4
-----END PGP SIGNATURE-----

From imacat at mail.imacat.idv.tw  Sun Sep  3 14:48:32 2006
From: imacat at mail.imacat.idv.tw (imacat)
Date: Sun, 03 Sep 2006 20:48:32 +0800
Subject: [Mailman-Developers] Traditional Chinese in Mailman 2.1.9rc1
Message-ID: <20060903204824.6DBF.IMACAT@mail.imacat.idv.tw>

Dear all,

    I saw that you are releasing Mailman 2.1.9rc1.  I hope that this
issue can be seen before 2.1.9 is released.

    The options.html and verify.txt in Traditional Chinese are seriously
outdated.  They seem to be from 2.0.x.  The form buttons are not right. 
I don't know who is in charge of the Traditional Chinese translation and
localization.  Could this be resolved before 2.1.9?  Could anybody tell
me who should I contact to?

--
Best regards,
imacat ^_*' <imacat at mail.imacat.idv.tw>
PGP Key: http://www.imacat.idv.tw/me/pgpkey.txt

<<Woman's Voice>> News: http://www.wov.idv.tw/
Tavern IMACAT's: http://www.imacat.idv.tw/
TLUG List Manager: http://lists.linux.org.tw/cgi-bin/mailman/listinfo/tlug
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060903/ca398569/attachment.pgp 

From barry at python.org  Sun Sep  3 16:48:31 2006
From: barry at python.org (Barry Warsaw)
Date: Sun, 3 Sep 2006 10:48:31 -0400
Subject: [Mailman-Developers] Traditional Chinese in Mailman 2.1.9rc1
In-Reply-To: <20060903204824.6DBF.IMACAT@mail.imacat.idv.tw>
References: <20060903204824.6DBF.IMACAT@mail.imacat.idv.tw>
Message-ID: <0B1CFEA6-5056-4437-88E1-05EBEA15AF2B@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 3, 2006, at 8:48 AM, imacat wrote:

>     I saw that you are releasing Mailman 2.1.9rc1.  I hope that this
> issue can be seen before 2.1.9 is released.
>
>     The options.html and verify.txt in Traditional Chinese are  
> seriously
> outdated.  They seem to be from 2.0.x.  The form buttons are not  
> right.
> I don't know who is in charge of the Traditional Chinese  
> translation and
> localization.  Could this be resolved before 2.1.9?  Could anybody  
> tell
> me who should I contact to?

imacat, you should look at this wiki page for all i18n details:

http://wiki.list.org/display/DEV/Languages

Please contact the language coordinator.  If we can get an update  
this week, I will make sure it gets into Mailman 2.1.9.  If the  
translators need a few more days, I can let the release slip a little  
bit, but not much more than that.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRPrrQHEjvBPtnXfVAQJ0hgP8CoDYj9nHam5CYL+hC3iI0DDQ0Qwop9Pv
X4t+tuOIJHTi64M79yRX6UF2+lUKksdfQdW7Z5nB2wObSZ5uj79Hg2ZXTz4ow50e
5KWisk/sCYPu/UT8tEftZYTnvK0Yyol/AN77el00jmWySPOpJEe6Ida0b48Ovdzt
Wf28/rBbSGk=
=x+uZ
-----END PGP SIGNATURE-----

From imacat at mail.imacat.idv.tw  Sun Sep  3 17:18:06 2006
From: imacat at mail.imacat.idv.tw (imacat)
Date: Sun, 03 Sep 2006 23:18:06 +0800
Subject: [Mailman-Developers] Traditional Chinese in Mailman 2.1.9rc1
In-Reply-To: <0B1CFEA6-5056-4437-88E1-05EBEA15AF2B@python.org>
References: <20060903204824.6DBF.IMACAT@mail.imacat.idv.tw>
	<0B1CFEA6-5056-4437-88E1-05EBEA15AF2B@python.org>
Message-ID: <20060903231755.1980.IMACAT@mail.imacat.idv.tw>

On Sun, 3 Sep 2006 10:48:31 -0400
Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> On Sep 3, 2006, at 8:48 AM, imacat wrote:
> imacat, you should look at this wiki page for all i18n details:
> http://wiki.list.org/display/DEV/Languages
> Please contact the language coordinator.  If we can get an update  
> this week, I will make sure it gets into Mailman 2.1.9.  If the  
> translators need a few more days, I can let the release slip a little  
> bit, but not much more than that.

    Well, then, maybe we can just forget it.  I do not think I can get
it done in a week.  I'll follow up with your information on this issue,
maybe in the future Mailman release.  Thank you for your help.

--
Best regards,
imacat ^_*' <imacat at mail.imacat.idv.tw>
PGP Key: http://www.imacat.idv.tw/me/pgpkey.txt

<<Woman's Voice>> News: http://www.wov.idv.tw/
Tavern IMACAT's: http://www.imacat.idv.tw/
TLUG List Manager: http://lists.linux.org.tw/cgi-bin/mailman/listinfo/tlug
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060903/e576acca/attachment.pgp 

From barry at python.org  Sun Sep  3 22:16:20 2006
From: barry at python.org (Barry Warsaw)
Date: Sun, 3 Sep 2006 16:16:20 -0400
Subject: [Mailman-Developers] Traditional Chinese in Mailman 2.1.9rc1
In-Reply-To: <20060903231755.1980.IMACAT@mail.imacat.idv.tw>
References: <20060903204824.6DBF.IMACAT@mail.imacat.idv.tw>
	<0B1CFEA6-5056-4437-88E1-05EBEA15AF2B@python.org>
	<20060903231755.1980.IMACAT@mail.imacat.idv.tw>
Message-ID: <A2844BE5-3FEB-4341-90B8-1CD10BCA4BBF@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 3, 2006, at 11:18 AM, imacat wrote:

>     Well, then, maybe we can just forget it.  I do not think I can get
> it done in a week.  I'll follow up with your information on this  
> issue,
> maybe in the future Mailman release.  Thank you for your help.

It looks like Ping is going to work on it.  It's great if we can get  
updates into 2.1.9, but don't worry, if you don't finish it in time  
please continue to send updates; we can always release a 2.1.10 at  
some point with language updates.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRPs4GXEjvBPtnXfVAQKN6wP/TvzvCZ7HLnjr7jR8XrqGK75DLmG9BTXh
5yAAl16fTMHaLo8QMWBRldKk/cRqQxSbF3zvlcn180wEjR/kIbL5hSeYbl6saatf
ibjPnCSzcRz4ciSKUkAUD1I3PGVRHPW5PoKR9XZhp1SEeuZC6KJlmiN4Jmh5IDeN
cZr0fqAFg2U=
=orZ+
-----END PGP SIGNATURE-----

From fehwalker at gmail.com  Wed Sep  6 06:58:40 2006
From: fehwalker at gmail.com (Bryan Fullerton)
Date: Wed, 6 Sep 2006 00:58:40 -0400
Subject: [Mailman-Developers] RELEASED Mailman 2.1.9 release candidate 1
In-Reply-To: <243CCA3B-B7E9-42E4-9D67-A24E7AC91154@python.org>
References: <243CCA3B-B7E9-42E4-9D67-A24E7AC91154@python.org>
Message-ID: <35de0c300609052158j66f71691y3df8007fb9af4f00@mail.gmail.com>

On 9/2/06, Barry Warsaw <barry at python.org> wrote:
> This version is not yet recommended for production environments,
> however testing and feedback is greatly encouraged.  My plan is to
> release 2.1.9 final by 10-Sep-2006.  A more detailed list of changes
> will be included in the final release announcement.

Just so you have some feedback...

I upgraded my small production machine to 2.1.9rc1 today, everything
seems to be working as usual.

Thanks,
Bryan

From brad at stop.mail-abuse.org  Wed Sep  6 08:39:02 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Wed, 6 Sep 2006 01:39:02 -0500
Subject: [Mailman-Developers] RELEASED Mailman 2.1.9 release candidate 1
In-Reply-To: <35de0c300609052158j66f71691y3df8007fb9af4f00@mail.gmail.com>
References: <243CCA3B-B7E9-42E4-9D67-A24E7AC91154@python.org>
	<35de0c300609052158j66f71691y3df8007fb9af4f00@mail.gmail.com>
Message-ID: <p0624050ac1241d651e35@[10.0.1.210]>

At 12:58 AM -0400 2006-09-06, Bryan Fullerton wrote:

>  I upgraded my small production machine to 2.1.9rc1 today, everything
>  seems to be working as usual.

We're also running it in production on python.org, and I have yet to 
see anything out of the ordinary.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From barry at python.org  Wed Sep  6 14:24:44 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 6 Sep 2006 08:24:44 -0400
Subject: [Mailman-Developers] RELEASED Mailman 2.1.9 release candidate 1
In-Reply-To: <p0624050ac1241d651e35@[10.0.1.210]>
References: <243CCA3B-B7E9-42E4-9D67-A24E7AC91154@python.org>
	<35de0c300609052158j66f71691y3df8007fb9af4f00@mail.gmail.com>
	<p0624050ac1241d651e35@[10.0.1.210]>
Message-ID: <66870292-D2CC-4319-9016-64644E9726D5@python.org>

On Sep 6, 2006, at 2:39 AM, Brad Knowles wrote:

> At 12:58 AM -0400 2006-09-06, Bryan Fullerton wrote:
>
>>  I upgraded my small production machine to 2.1.9rc1 today, everything
>>  seems to be working as usual.
>
> We're also running it in production on python.org, and I have yet to
> see anything out of the ordinary.

Excellent, thanks for the feedback!
-Barry


From guillomovitch at zarb.org  Sat Sep  9 16:09:27 2006
From: guillomovitch at zarb.org (Guillaume Rousse)
Date: Sat, 09 Sep 2006 16:09:27 +0200
Subject: [Mailman-Developers] Patches in mandriva package
Message-ID: <4502CB17.4060808@zarb.org>

Hello.

I'm the mailman maintainer for mandriva Linux. I'm currently upgrading
2.1.8 to 2.1.9rc1, due to the recent security fixes therein.

I'd like to use this occasion to drop a maximum of patches we still have:
- is 2.1.9 still vulnearble to CVE-2005-3573 ? I didn't found any
reference to it in the release notes, and the patch [1]  still apply
- the default build procedure is not suited to package building: it
check target directory directly (which doesn't exist), and leave
reference to package build root in python bytecode files. The patches
[2] and [3] fixes those issues, maybe they could get integrated.
- we have a patch for the embeded email module that just fix an encoding
name [4]. I didn't found reference to a website or a standalone
distribution of this module elsewhere. Could you please transmit the
patch to its authors ?


[1] CVE-2005-3573 fix:
http://cvs.mandriva.com/cgi-bin/viewvc.cgi/SPECS/mailman/mailman-2.1.6-CVE-2005-3573.patch?view=log
[2] buildroot references in bytecode fix:
http://cvs.mandriva.com/cgi-bin/viewvc.cgi/SPECS/mailman/mailman-2.1.6-CVE-2005-3573.patch?view=log
[3] buildroot check fix:
http://cvs.mandriva.com/cgi-bin/viewvc.cgi/SPECS/mailman/mailman-buildroot-check.patch?view=log


From mailman-developers at mlists.thewrittenword.com  Sun Sep 10 10:25:16 2006
From: mailman-developers at mlists.thewrittenword.com (Albert Chin)
Date: Sun, 10 Sep 2006 03:25:16 -0500
Subject: [Mailman-Developers] Patches for security issues in 2.1.9rc1
Message-ID: <20060910082515.GA32253@mail1.thewrittenword.com>

We'd like to create a patch on top of 2.1.8 for the security issues in
2.1.9rc1.

For the log injection vulnerability, we applied a diff of revisions
7822-7918 for Mailman/Utils.py from the Release_2_1-maint branch.

For CVE-2006-3636, we applied a diff of revisions 7975-8001 from the
Release_2_1-maint branch.

What revisions contain the patch for CVE-2006-2941?
  - Fixed denial of service attack which can be caused by some
    standards-breaking RFC 2231 formatted headers.  CVE-2006-2941.

-- 
albert chin (china at thewrittenword.com)

From Dale at Newfield.org  Mon Sep 11 07:40:49 2006
From: Dale at Newfield.org (Dale Newfield)
Date: Mon, 11 Sep 2006 01:40:49 -0400
Subject: [Mailman-Developers] D'oh!  CVS -> SVN...
In-Reply-To: <44A4173D.3060802@mindlace.net>
References: <ACBA54DA2410A50DFAAA81EA@lcarslon.d.umn.edu>
	<44A4173D.3060802@mindlace.net>
Message-ID: <4504F6E1.50906@Newfield.org>

So, it appears I'm still running version 2.1.8b1, and I'm doing so from 
code checked out of CVS (Tag = NRelease_2_1_8b1, but probably with 
modifications...)

I'd love to be able to do a cvs diff to see what differences I have, 
then I could check out a subversion repository and integrate any 
important changes (if any)...

...but CVS on sourceforge for Mailman is turned off...

...any suggestions?

-Dale Newfield
  Dale at Newfield.org

From barry at python.org  Mon Sep 11 14:21:38 2006
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Sep 2006 08:21:38 -0400
Subject: [Mailman-Developers] D'oh!  CVS -> SVN...
In-Reply-To: <4504F6E1.50906@Newfield.org>
References: <ACBA54DA2410A50DFAAA81EA@lcarslon.d.umn.edu>
	<44A4173D.3060802@mindlace.net> <4504F6E1.50906@Newfield.org>
Message-ID: <4C96C920-E47E-415D-B327-59FAA2E53FA5@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 11, 2006, at 1:40 AM, Dale Newfield wrote:

> So, it appears I'm still running version 2.1.8b1, and I'm doing so  
> from
> code checked out of CVS (Tag = NRelease_2_1_8b1, but probably with
> modifications...)
>
> I'd love to be able to do a cvs diff to see what differences I have,
> then I could check out a subversion repository and integrate any
> important changes (if any)...
>
> ...but CVS on sourceforge for Mailman is turned off...
>
> ...any suggestions?

You should probably do an svn checkout and then use recursive diff  
(ignoring CVS and .svn directories) to find local modifications.   
Since your checkout is from a specific cvs tag, you should probably  
also check out the equivalent svn tag:

https://svn.sourceforge.net/svnroot/mailman/tags/Release_2_1_8b1

although I would suggest running your production system from

https://svn.sourceforge.net/svnroot/mailman/branches/Release_2_1-maint

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRQVU0nEjvBPtnXfVAQLRXAP+L8ARmIvsWd6VHhpPuGF6Et/iZd2IYVAn
nUa2boA1ZUB47MF+loZ5ofbjFmYmTMQCI6zB5TtjJwtgjKAQHcDruLXKkQWs5QJ6
+/XvpUQv1ZeSoc46asq1Hgw0+NbjMkUcjQHW0Gm7nU3sw73OAqr4l1Ky4Z5zTGC0
30YTyQtuVNM=
=Dj7t
-----END PGP SIGNATURE-----

From barry at python.org  Mon Sep 11 16:52:03 2006
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Sep 2006 10:52:03 -0400
Subject: [Mailman-Developers] Patches for security issues in 2.1.9rc1
In-Reply-To: <20060910082515.GA32253@mail1.thewrittenword.com>
References: <20060910082515.GA32253@mail1.thewrittenword.com>
Message-ID: <CEE33BD2-7516-4DFF-817C-EA28BE38A6FC@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 10, 2006, at 4:25 AM, Albert Chin wrote:

> What revisions contain the patch for CVE-2006-2941?
>   - Fixed denial of service attack which can be caused by some
>     standards-breaking RFC 2231 formatted headers.  CVE-2006-2941.

http://svn.sourceforge.net/viewvc/mailman?view=rev&revision=7959

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRQV4FHEjvBPtnXfVAQJSMwP/XfkqiDL2B2IF3g4bF7yA1w3A/zdhNPdN
2bO+XP4HRnLk0/Ka+NpVVyt7si7aAV/vfK3eEyB2cV/rPEdaUtIAmosB8egHT6sN
tTXl8shpZUT4q9mMxQxwUyHcQ/K+pC0HOVNfj7rk/lNFmSzF9BR274Jzx4aWkLbA
/JRq/higSAc=
=alO0
-----END PGP SIGNATURE-----

From mailman-developers at mlists.thewrittenword.com  Mon Sep 11 16:58:54 2006
From: mailman-developers at mlists.thewrittenword.com (Albert Chin)
Date: Mon, 11 Sep 2006 09:58:54 -0500
Subject: [Mailman-Developers] Patches for security issues in 2.1.9rc1
In-Reply-To: <CEE33BD2-7516-4DFF-817C-EA28BE38A6FC@python.org>
References: <20060910082515.GA32253@mail1.thewrittenword.com>
	<CEE33BD2-7516-4DFF-817C-EA28BE38A6FC@python.org>
Message-ID: <20060911145854.GA98955@mail1.thewrittenword.com>

On Mon, Sep 11, 2006 at 10:52:03AM -0400, Barry Warsaw wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Sep 10, 2006, at 4:25 AM, Albert Chin wrote:
> 
> > What revisions contain the patch for CVE-2006-2941?
> >   - Fixed denial of service attack which can be caused by some
> >     standards-breaking RFC 2231 formatted headers.  CVE-2006-2941.
> 
> http://svn.sourceforge.net/viewvc/mailman?view=rev&revision=7959

Thanks!

-- 
albert chin (china at thewrittenword.com)

From barry at python.org  Tue Sep 12 03:51:33 2006
From: barry at python.org (Barry Warsaw)
Date: Mon, 11 Sep 2006 21:51:33 -0400
Subject: [Mailman-Developers] Patches in mandriva package
In-Reply-To: <4502CB17.4060808@zarb.org>
References: <4502CB17.4060808@zarb.org>
Message-ID: <72A98460-0FDE-4930-98F0-C78B858FE9DE@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 9, 2006, at 10:09 AM, Guillaume Rousse wrote:

> I'd like to use this occasion to drop a maximum of patches we still  
> have:
> - is 2.1.9 still vulnearble to CVE-2005-3573 ? I didn't found any
> reference to it in the release notes, and the patch [1]  still apply

This is the first I've seen of this CVE, but it sounds like bugs that  
have been addressed in the email package.

> - the default build procedure is not suited to package building: it
> check target directory directly (which doesn't exist), and leave
> reference to package build root in python bytecode files. The patches
> [2] and [3] fixes those issues, maybe they could get integrated.

I don't think your links are correct.  Link [1] is the same as link [2].

> - we have a patch for the embeded email module that just fix an  
> encoding
> name [4]. I didn't found reference to a website or a standalone
> distribution of this module elsewhere. Could you please transmit the
> patch to its authors ?

There's no [4] link so I don't understand what this one tries to  
fix.  The email package is developed at the email-sig: <http:// 
www.python.org/sigs/email>.  You should probably post the issue over  
there on that sig.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRQYSqnEjvBPtnXfVAQLJpgP/SMowdWbnpicbK9rqeisSIKMffjHP6B1v
6xvFsR3IRmfSin6wIQSMe2yjsVv2Bb7o+tK4u4ts5RE+F0XmBuSu+yeVzq0858y+
dX5iSZHK6b5nrajpR0JFjNTDnir2KPHGr1XlY3vQUhaISeC5wnvWqIiW9KKLat9+
m4F22T7Aqdk=
=mzRh
-----END PGP SIGNATURE-----

From pierre-marc.fournier at polymtl.ca  Tue Sep 12 02:17:00 2006
From: pierre-marc.fournier at polymtl.ca (Pierre-Marc Fournier)
Date: Mon, 11 Sep 2006 20:17:00 -0400
Subject: [Mailman-Developers] create.py: passing form values by URL
Message-ID: <4505FC7C.4080407@polymtl.ca>

Hello,

At our site, we have a web page with a form that users fill to request a
new mailing list. If that mailing list gets approved, the admins use
Mailman's create.py to effectively create it.

A trivial patch allows the approval system (or any script) to pass the
list name, owner, etc to create.py via the url, and have the create form
already filled with those values, leaving only the password for the
administrators to enter. This removes the need for a lot of copy-pasting
into the create.py form and saves time.

Here is the patch. I suggest that it be integrated in Mailman, as it
would enable everyone to use a similar system easily.

pmf




-------------- next part --------------
--- create.py.orig	2006-08-23 12:48:15.213282750 -0400
+++ create.py	2006-08-23 13:39:24.585106500 -0400
@@ -58,7 +58,7 @@
         request_creation(doc)
     else:
         # Put up the list creation request form
-        request_creation(doc)
+        request_creation(doc, cgidata)
     doc.AddItem('<hr>')
     # Always add the footer and print the document
     doc.AddItem(_('Return to the ') +





From brad at stop.mail-abuse.org  Tue Sep 12 03:59:44 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Mon, 11 Sep 2006 20:59:44 -0500
Subject: [Mailman-Developers] create.py: passing form values by URL
In-Reply-To: <4505FC7C.4080407@polymtl.ca>
References: <4505FC7C.4080407@polymtl.ca>
Message-ID: <p06240506c12bc4dc6843@[10.0.1.210]>

At 8:17 PM -0400 2006-09-11, Pierre-Marc Fournier wrote:

>  Here is the patch. I suggest that it be integrated in Mailman, as it
>  would enable everyone to use a similar system easily.

If you want this considered for incorporation into Mailman, I would 
encourage you to upload it to the Mailman Patch page at SourceForge.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From tkikuchi at is.kochi-u.ac.jp  Tue Sep 12 07:16:27 2006
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Tue, 12 Sep 2006 14:16:27 +0900
Subject: [Mailman-Developers] Patches in mandriva package
In-Reply-To: <72A98460-0FDE-4930-98F0-C78B858FE9DE@python.org>
References: <4502CB17.4060808@zarb.org>
	<72A98460-0FDE-4930-98F0-C78B858FE9DE@python.org>
Message-ID: <450642AB.4000601@is.kochi-u.ac.jp>

Hi,

Sorry that I was unable to respond.

Barry Warsaw wrote:

> On Sep 9, 2006, at 10:09 AM, Guillaume Rousse wrote:
> 
>> I'd like to use this occasion to drop a maximum of patches we still  
>> have:
>> - is 2.1.9 still vulnearble to CVE-2005-3573 ? I didn't found any
>> reference to it in the release notes, and the patch [1]  still apply
> 
> This is the first I've seen of this CVE, but it sounds like bugs that  
> have been addressed in the email package.

This is mentioned in the NEWS of version 2.1.7.

- A note on CVE-2005-3573: Although the RFC2231 bug example in the CVE has
   been solved in Mailman 2.1.6, there may be more cases where
   ToDigest.send_digests() can block regular delivery.  We put the
   send_digests() calling part in a try/except clause and leave a message
   in the error log if something happened in send_digests().  Daily call of
   cron/senddigests will provide more detail to the site administrator.

Therefore, 2.1.9 is also not vulnerable.  CVE-2005-3573 is a false 
(delayed) alert.

-- 
Tokio Kikuchi tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From guillomovitch at zarb.org  Tue Sep 12 09:16:48 2006
From: guillomovitch at zarb.org (Guillaume Rousse)
Date: Tue, 12 Sep 2006 09:16:48 +0200
Subject: [Mailman-Developers] Patches in mandriva package
In-Reply-To: <450642AB.4000601@is.kochi-u.ac.jp>
References: <4502CB17.4060808@zarb.org>
	<72A98460-0FDE-4930-98F0-C78B858FE9DE@python.org>
	<450642AB.4000601@is.kochi-u.ac.jp>
Message-ID: <45065EE0.8010209@zarb.org>

Tokio Kikuchi wrote:
> Hi,
> 
> Sorry that I was unable to respond.
> 
> Barry Warsaw wrote:
> 
>> On Sep 9, 2006, at 10:09 AM, Guillaume Rousse wrote:
>>
>>> I'd like to use this occasion to drop a maximum of patches we still 
>>> have:
>>> - is 2.1.9 still vulnearble to CVE-2005-3573 ? I didn't found any
>>> reference to it in the release notes, and the patch [1]  still apply
>>
>> This is the first I've seen of this CVE, but it sounds like bugs that 
>> have been addressed in the email package.
> 
> This is mentioned in the NEWS of version 2.1.7.
> 
> - A note on CVE-2005-3573: Although the RFC2231 bug example in the CVE has
>   been solved in Mailman 2.1.6, there may be more cases where
>   ToDigest.send_digests() can block regular delivery.  We put the
>   send_digests() calling part in a try/except clause and leave a message
>   in the error log if something happened in send_digests().  Daily call of
>   cron/senddigests will provide more detail to the site administrator.
> 
> Therefore, 2.1.9 is also not vulnerable.  CVE-2005-3573 is a false
> (delayed) alert.
Thanks, I'll remove it.


From guillomovitch at zarb.org  Tue Sep 12 09:34:22 2006
From: guillomovitch at zarb.org (Guillaume Rousse)
Date: Tue, 12 Sep 2006 09:34:22 +0200
Subject: [Mailman-Developers] Patches in mandriva package
In-Reply-To: <72A98460-0FDE-4930-98F0-C78B858FE9DE@python.org>
References: <4502CB17.4060808@zarb.org>
	<72A98460-0FDE-4930-98F0-C78B858FE9DE@python.org>
Message-ID: <450662FE.3090302@zarb.org>

Barry Warsaw wrote:
>>> - the default build procedure is not suited to package building: it
>>> check target directory directly (which doesn't exist), and leave
>>> reference to package build root in python bytecode files. The patches
>>> [2] and [3] fixes those issues, maybe they could get integrated.
> 
> I don't think your links are correct.  Link [1] is the same as link [2].
Sorry, here they are:
http://cvs.mandriva.com/cgi-bin/viewvc.cgi/SPECS/mailman/mailman-buildroot-check.patch?view=log
http://cvs.mandriva.com/cgi-bin/viewvc.cgi/SPECS/mailman/mailman-2.1.5-build.patch?view=log

>>> - we have a patch for the embeded email module that just fix an encoding
>>> name [4]. I didn't found reference to a website or a standalone
>>> distribution of this module elsewhere. Could you please transmit the
>>> patch to its authors ?
> 
> There's no [4] link so I don't understand what this one tries to fix. 
> The email package is developed at the email-sig:
> <http://www.python.org/sigs/email>.  You should probably post the issue
> over there on that sig.
The patch is
http://cvs.mandriva.com/cgi-bin/viewvc.cgi/SPECS/mailman/mailman-2.1.8-Charset.patch?view=log

Just submited as bug #1556895 on python tracker.

BTW, why ship a private copy of this module instead of using the system
one ?

From tkikuchi at is.kochi-u.ac.jp  Tue Sep 12 12:38:36 2006
From: tkikuchi at is.kochi-u.ac.jp (=?windows-1252?Q?=3F=3F=3F=3F?=)
Date: Tue, 12 Sep 2006 19:38:36 +0900
Subject: [Mailman-Developers] Patches in mandriva package
In-Reply-To: <450642AB.4000601@is.kochi-u.ac.jp>
References: <4502CB17.4060808@zarb.org>	<72A98460-0FDE-4930-98F0-C78B858FE9DE@python.org>
	<450642AB.4000601@is.kochi-u.ac.jp>
Message-ID: <45068E2C.5060805@is.kochi-u.ac.jp>

Tokio Kikuchi wrote:

>    ToDigest.send_digests() can block regular delivery.  We put the
>    send_digests() calling part in a try/except clause and leave a message
>    in the error log if something happened in send_digests().  Daily call of
>    cron/senddigests will provide more detail to the site administrator.

I noticed this may lead to yet another DoS for digest delivery.  The 
malicious (non-compliant MIME) message may cause other digest deliveries 
to stop as long as the malicious message remains in the digest.mbox 
file.  I created a patch for this situation and uploaded in the patch 
area of SF:
http://sourceforge.net/tracker/index.php?func=detail&aid=1556858&group_id=103&atid=300103

I think I will commit in the Release-2.1-maint branch and include in the 
2.1.9 final release.  I appreciate anyone can review the patch.

At this point of writing, I should note 2.1.9(rc1) has no known 
vulnerablities by which this patch is required.


-- 
Tokio Kikuchi tkikuchi@ is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/


From barry at python.org  Tue Sep 12 14:28:22 2006
From: barry at python.org (Barry Warsaw)
Date: Tue, 12 Sep 2006 08:28:22 -0400
Subject: [Mailman-Developers] Patches in mandriva package
In-Reply-To: <45068E2C.5060805@is.kochi-u.ac.jp>
References: <4502CB17.4060808@zarb.org>	<72A98460-0FDE-4930-98F0-C78B858FE9DE@python.org>
	<450642AB.4000601@is.kochi-u.ac.jp>
	<45068E2C.5060805@is.kochi-u.ac.jp>
Message-ID: <BB20DB5E-D4E6-4F36-B6A8-00AC63FB4DD9@python.org>

On Sep 12, 2006, at 6:38 AM, ???? wrote:

> Tokio Kikuchi wrote:
>
>>    ToDigest.send_digests() can block regular delivery.  We put the
>>    send_digests() calling part in a try/except clause and leave a  
>> message
>>    in the error log if something happened in send_digests().   
>> Daily call of
>>    cron/senddigests will provide more detail to the site  
>> administrator.
>
> I noticed this may lead to yet another DoS for digest delivery.  The
> malicious (non-compliant MIME) message may cause other digest  
> deliveries
> to stop as long as the malicious message remains in the digest.mbox
> file.  I created a patch for this situation and uploaded in the patch
> area of SF:
> http://sourceforge.net/tracker/index.php? 
> func=detail&aid=1556858&group_id=103&atid=300103
>
> I think I will commit in the Release-2.1-maint branch and include  
> in the
> 2.1.9 final release.  I appreciate anyone can review the patch.

Let's wait for 2.1.10, because otherwise we're really going to need  
another release candidate and another week or so of testing.

-Barry


From barry at python.org  Tue Sep 12 19:35:15 2006
From: barry at python.org (Barry Warsaw)
Date: Tue, 12 Sep 2006 13:35:15 -0400
Subject: [Mailman-Developers] Patches in mandriva package
In-Reply-To: <450662FE.3090302@zarb.org>
References: <4502CB17.4060808@zarb.org>
	<72A98460-0FDE-4930-98F0-C78B858FE9DE@python.org>
	<450662FE.3090302@zarb.org>
Message-ID: <BF17806B-B452-4F87-B048-3962533E6D42@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 12, 2006, at 3:34 AM, Guillaume Rousse wrote:

> Sorry, here they are:
> http://cvs.mandriva.com/cgi-bin/viewvc.cgi/SPECS/mailman/mailman- 
> buildroot-check.patch?view=log
> http://cvs.mandriva.com/cgi-bin/viewvc.cgi/SPECS/mailman/ 
> mailman-2.1.5-build.patch?view=log

Okay thanks, we'll look into these for a future release.

> The patch is
> http://cvs.mandriva.com/cgi-bin/viewvc.cgi/SPECS/mailman/ 
> mailman-2.1.8-Charset.patch?view=log
>
> Just submited as bug #1556895 on python tracker.

Thanks, I've assigned this to myself.

> BTW, why ship a private copy of this module instead of using the  
> system
> one ?

Because we can never be sure which version the system Python has and  
we don't want to rely on a version we haven't vetted.  The email  
package is way too important to Mailman.

Thanks,
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRQbv3XEjvBPtnXfVAQLi0wQAmUyuZDeMIq3Z7WdJVucEbhcScF6+32O7
tyIKB3FnLvzlN1BUHMk1VXPRP5OnHT2jCfxyDBsuP6UVIa1zsjk+hwtVzxKZVVUm
pIZDydmEGq3CjkNDVDxCMaaZZ72uyG1BzhZAw7iK2FEhm4cXoFR6xY2Jg1lWhfPY
T8QccEmQZGs=
=Gdmb
-----END PGP SIGNATURE-----

From tkikuchi at is.kochi-u.ac.jp  Wed Sep 13 01:43:55 2006
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Wed, 13 Sep 2006 08:43:55 +0900
Subject: [Mailman-Developers] Patches in mandriva package
In-Reply-To: <BB20DB5E-D4E6-4F36-B6A8-00AC63FB4DD9@python.org>
References: <4502CB17.4060808@zarb.org>	<72A98460-0FDE-4930-98F0-C78B858FE9DE@python.org>	<450642AB.4000601@is.kochi-u.ac.jp>	<45068E2C.5060805@is.kochi-u.ac.jp>
	<BB20DB5E-D4E6-4F36-B6A8-00AC63FB4DD9@python.org>
Message-ID: <4507463B.9080306@is.kochi-u.ac.jp>

>> http://sourceforge.net/tracker/index.php? 
>> func=detail&aid=1556858&group_id=103&atid=300103
>>
>> I think I will commit in the Release-2.1-maint branch and include  
>> in the
>> 2.1.9 final release.  I appreciate anyone can review the patch.
> 
> Let's wait for 2.1.10, because otherwise we're really going to need  
> another release candidate and another week or so of testing.

Oh, yes. I understand.

-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From barry at python.org  Wed Sep 13 16:00:57 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 13 Sep 2006 10:00:57 -0400
Subject: [Mailman-Developers] RELEASED: Mailman 2.1.9
Message-ID: <8538763C-6FBD-49CD-A252-E1A5991F1837@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the GNU Mailman development team, I'm please to announce  
GNU Mailman 2.1.9.  This is primarily a security and bug fix release  
and it is highly recommended that all sites upgrade to this version.   
Mailman 2.1.9 also contains support for two new languages: Arabic and  
Vietnamese.

Mailman is free software for managing email mailing lists and e- 
newsletters.  Mailman is used for all the python.org and  
SourceForge.net mailing lists, as well as at hundreds of other sites.

For more information, including download links, please see:

	http://www.list.org
	http://mailman.sf.net
	http://www.gnu.org/software/mailman

A more detailed change list is included below.

Enjoy,
- -Barry

2.1.9 (12-Sep-2006)

   Security

     - A malicious user could visit a specially crafted URI and  
inject an
       apparent log message into Mailman's error log which might  
induce an
       unsuspecting administrator to visit a phishing site.  This has  
been
       blocked.  Thanks to Moritz Naumann for its discovery.

     - Fixed denial of service attack which can be caused by some
       standards-breaking RFC 2231 formatted headers.  CVE-2006-2941.

     - Several cross-site scripting issues have been fixed.  Thanks  
to Moritz
       Naumann for their discovery.  CVE-2006-3636

     - Fixed an unexploitable format string vulnerability.  Discovery  
and fix
       by Karl Chen.  Analysis of non-exploitability by Martin 'Joey'  
Schulze.
       Also thanks go to Lionel Elie Mamane.  CVE-2006-2191.

   Internationalization

     - New languages: Arabic, Vietnamese.

   Bug fixes and other patches

     - Fixed Decorate.py so that characters in message header/footer  
which
       are not in the character set of the list's language are  
ignored rather
       than causing shunted messages (1507248).

     - Switchboard.py - Closed very tiny holes at the upper ends of  
queue
       slices that could result in unprocessable queue entries.   
Improved FIFO
       processing when two queue entries have the same timestamp.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRQgPGnEjvBPtnXfVAQIVoQP/R2DffgpcPMzUrsef+ZEcYUeuQ1mOcol2
Z2+iQiHkCx6SP2B/NzOzqMQybvQAAe/TzJWzcfqDDoDDdF+vhJH+kkQIuRwHc5jd
+TDF1NOUBegTyxQnoyCHVQddcVNMg9HTTkdwHuvE8MhP1gNuHEnefxf2wbf5+hRq
h5/qlBiANn0=
=VCTA
-----END PGP SIGNATURE-----

From barry at python.org  Wed Sep 13 16:40:15 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 13 Sep 2006 10:40:15 -0400
Subject: [Mailman-Developers] [Mailman-Users] RELEASED: Mailman 2.1.9
In-Reply-To: <20060913141006.GY6362@charite.de>
References: <8538763C-6FBD-49CD-A252-E1A5991F1837@python.org>
	<20060913141006.GY6362@charite.de>
Message-ID: <97F683C1-578A-4E1E-98E0-8FD90A115DB5@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 13, 2006, at 10:10 AM, Ralf Hildebrandt wrote:
> The download link on http://www.gnu.org/software/mailman/download.html
> pointing to http://www.list.org/mailman.tar.gz doesn't work.
>
> The download link on http://www.gnu.org/software/mailman/download.html
> pointing to http://ftp.gnu.org/gnu/mailman/
> lacks 2.1.9

Due to a recent server move and bandwidth cap, we can no longer  
provide tarball downloads from list.org.  Also, because of gnu.org's  
ftp upload procedure, it sometimes lags behind SourceForge.  Your  
best bet immediately is to get it from SF.

I've updated the download.html page to explain these issues.

Thanks,
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRQgYT3EjvBPtnXfVAQLWqgQAth4R8JC313TxhXlNBx1maJiYZWJVA9Qg
6IUmlhrAetCFHzIWArXDKncLE+s85iOJg24eFroiQwGTw3a7tcNMkaG1vCa++E7R
bl4phJ2BB0xnpDi0lKe/f5Wt8ewjNEuUwLDFImFFJguKZEMl+4RUrHo9awL19R/K
HLqaksjmCeU=
=UyOs
-----END PGP SIGNATURE-----

From barry at python.org  Mon Sep 18 05:01:47 2006
From: barry at python.org (Barry Warsaw)
Date: Sun, 17 Sep 2006 23:01:47 -0400
Subject: [Mailman-Developers] Patches in mandriva package
In-Reply-To: <4507463B.9080306@is.kochi-u.ac.jp>
References: <4502CB17.4060808@zarb.org>	<72A98460-0FDE-4930-98F0-C78B858FE9DE@python.org>	<450642AB.4000601@is.kochi-u.ac.jp>	<45068E2C.5060805@is.kochi-u.ac.jp>
	<BB20DB5E-D4E6-4F36-B6A8-00AC63FB4DD9@python.org>
	<4507463B.9080306@is.kochi-u.ac.jp>
Message-ID: <52BC1298-C70F-4E5A-9329-447E098B3AA4@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 12, 2006, at 7:43 PM, Tokio Kikuchi wrote:

>>> http://sourceforge.net/tracker/index.php?  
>>> func=detail&aid=1556858&group_id=103&atid=300103
>>>
>>> I think I will commit in the Release-2.1-maint branch and  
>>> include  in the
>>> 2.1.9 final release.  I appreciate anyone can review the patch.
>> Let's wait for 2.1.10, because otherwise we're really going to  
>> need  another release candidate and another week or so of testing.
>
> Oh, yes. I understand.

BTW, what do you think about changing the way we hold messages for  
digests?  E.g. instead of putting them in an mbox file, where it's  
more difficult to skip bad messages, stick them in a queue-like  
directory and pull them from there.  Any messages that can't be read  
and scrubbed would just be removed.

Might be too big a change for the 2.1 series, but perhaps we should  
look at that for 2.2.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRQ4MG3EjvBPtnXfVAQLybgP/UQse39i+9tcte043CqeQ36kdyFM0Xesq
q1EkG90HonLc/YS8XOXj7jsGN+WZ7W8jP7zAev6cmBJNUcKOPYJAhTCBfhelRLsN
fviC+dc3WkhQjdVXEcQ18I5O3NykvhgtSfMOyBO521c9HIRSLTYecNRni4gHrzyr
/wVUENzom/w=
=0xiv
-----END PGP SIGNATURE-----

From jwblist3 at olympus.net  Mon Sep 18 21:53:52 2006
From: jwblist3 at olympus.net (John W. Baxter)
Date: Mon, 18 Sep 2006 12:53:52 -0700
Subject: [Mailman-Developers] Patches in mandriva package
In-Reply-To: <52BC1298-C70F-4E5A-9329-447E098B3AA4@python.org>
Message-ID: <C1344760.EA69%jwblist3@olympus.net>

On 9/17/06 8:01 PM, "Barry Warsaw" <barry at python.org> wrote:

> BTW, what do you think about changing the way we hold messages for
> digests?  E.g. instead of putting them in an mbox file, where it's
> more difficult to skip bad messages, stick them in a queue-like
> directory and pull them from there.  Any messages that can't be read
> and scrubbed would just be removed.
> 
> Might be too big a change for the 2.1 series, but perhaps we should
> look at that for 2.2.

Sounds like a good change (with 2.2 being the earliest reasonable time to do
it).

  --John (cheering from the sidelines)



From bob at nleaudio.com  Thu Sep 21 06:28:10 2006
From: bob at nleaudio.com (Bob Puff)
Date: Thu, 21 Sep 2006 00:28:10 -0400
Subject: [Mailman-Developers] Patches in mandriva package
Message-ID: <451214DA.30701@nleaudio.com>

(Reposting this, since Python.org's mail servers are having issues)

 > BTW, what do you think about changing the way we hold messages for
 > digests?  E.g. instead of putting them in an mbox file, where it's
 > more difficult to skip bad messages, stick them in a queue-like
 > directory and pull them from there.  Any messages that can't be read
 >  and scrubbed would just be removed.

Hey Barry,

Do you mean as mailman builds the file during the day for the digest?
If so, why would there be a bad message in there?  If mailman's already
processed a message and blasted it out to all the normal delivery
people, why would there be a problem with it?

Also, while we are sort of on the topic, would it be possible to put the
message count back in the subject line?  I recall there was a problem
with the count being off from the actual number of messages... is this
what you were getting at?

One thing I hacked into a 2.0.x version was a "brief" digest - one that
only gave the message subject list.  The changes in ToDigest in 2.1.x
were such that it made it a lot more difficult to make this happen.  It
would be great to get it back.  I can revisit the code and see if I can
come up with a patch, but other things need to stablize a bit.

Bob

From msapiro at value.net  Fri Sep 22 15:40:52 2006
From: msapiro at value.net (Mark Sapiro)
Date: Fri, 22 Sep 2006 06:40:52 -0700
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8036]trunk/mailman/Mailman/loginit.py
In-Reply-To: <E1GQkij-0007S8-5j@sc8-pr-svn1.sourceforge.net>
Message-ID: <PC17602006092206405200938f3d1f51@msapiro>

>Revision: 8036
>          http://svn.sourceforge.net/mailman/?rev=8036&view=rev
>Author:   tkikuchi
>Date:     2006-09-22 06:05:30 -0700 (Fri, 22 Sep 2006)
>
>Log Message:
>-----------
>I needed this for web ui to work.
>
>Modified Paths:
>--------------
>    trunk/mailman/Mailman/loginit.py
>
>Modified: trunk/mailman/Mailman/loginit.py
>===================================================================
>--- trunk/mailman/Mailman/loginit.py	2006-09-21 18:23:48 UTC (rev 8035)
>+++ trunk/mailman/Mailman/loginit.py	2006-09-22 13:05:30 UTC (rev 8036)
>@@ -25,6 +25,7 @@
> import logging
> 
> from Mailman.configuration import config
>+config.load()
> 
> FMT     = '%(asctime)s (%(process)d) %(message)s'
> DATEFMT = '%b %d %H:%M:%S %Y'
>
>


Yes, but this is not the proper fix. I think the place to put
config.load() for the web interface is in scripts/driver.

I sent the following to Barry a few weeks ago, but to date have seen no
reply.

----------- excerpt from message to Barry -------------------
Unfortunately, I neglected to report something else I saw because I
didn't have time to figure it out. I'm still not sure about this, but
here's the problem:

[msapiro at msapiro ...f/test-mailman]$ scripts/post
Traceback (most recent call last):
  File "scripts/post", line 48, in ?
    main()
  File "scripts/post", line 39, in main
    status = sys.modules[module_name].main()
  File "/cygdrive/f/test-mailman/Mailman/bin/post.py", line 44, in main
    loginit.initialize(propagate=True)
  File "/cygdrive/f/test-mailman/Mailman/loginit.py", line 119, in
initialize
    handler = ReopenableFileHandler(os.path.join(config.LOG_DIR,
logger))
AttributeError: 'Configuration' object has no attribute 'LOG_DIR'

In this case, nothing has called config.load(). The same problem exists
in the web interface. That can be fixed with

--- f:/MM-Trunk/mailman/scripts/driver  2006-05-16 20:09:56.437500000
-0700
+++ scripts/driver      2006-08-29 09:51:32.375000000 -0700
@@ -68,6 +68,8 @@
     log = None
     try:
         import paths
+        from Mailman.configuration import config
+        config.load()
         # When running in non-stealth mode, we need to escape entities,
         # otherwise we're vulnerable to cross-site scripting attacks.
         try:

but this then leads to other places where we try to sort the set
returned by Utils.list_names()

I can work on cleaning this up, but I'd like confirmation that
scripts/driver is the right place for config.load() and also what to
do about the various scripts invoked by the mail wrapper.
--------------------end excerpt-------------------------

The problem with calling config.load() from loginit is I think it will
override the loading of a alternate config which may have been
specified as a command line option for those commands that support it
(at least the ones that do logging too).

-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From barry at python.org  Fri Sep 22 17:03:38 2006
From: barry at python.org (Barry Warsaw)
Date: Fri, 22 Sep 2006 11:03:38 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8036]trunk/mailman/Mailman/loginit.py
In-Reply-To: <PC17602006092206405200938f3d1f51@msapiro>
References: <PC17602006092206405200938f3d1f51@msapiro>
Message-ID: <7362ED20-933E-49C1-88E7-32BBD0448EEF@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 22, 2006, at 9:40 AM, Mark Sapiro wrote:

> Yes, but this is not the proper fix. I think the place to put
> config.load() for the web interface is in scripts/driver.
>
> I sent the following to Barry a few weeks ago, but to date have  
> seen no
> reply.

Oh dang, this got buried in all the 2.1.9 traffic.  I'll look at this  
this weekend.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRP7TnEjvBPtnXfVAQLXYQP/Znd2EUWARO5cPQUC72ulIBX1JFP1iQnG
vU/yVF49AYRIf2JS6pWU6MTFRhVxdkoJFZwu0RLr21vNpmZuxqPHhePtUGJQTA9h
rnQiAPEkbNuCqUSNTL+3ssdOKDKxXHMdmTtfro/qJf6qxyojmzZPNMDYtomoLkRV
eEUeoK5OaD4=
=2HB3
-----END PGP SIGNATURE-----

From tkikuchi at is.kochi-u.ac.jp  Sun Sep 24 01:51:26 2006
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Sun, 24 Sep 2006 08:51:26 +0900
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
 [8036]trunk/mailman/Mailman/loginit.py
In-Reply-To: <PC17602006092206405200938f3d1f51@msapiro>
References: <PC17602006092206405200938f3d1f51@msapiro>
Message-ID: <4515C87E.308@is.kochi-u.ac.jp>

Mark,

> 
> The problem with calling config.load() from loginit is I think it will
> override the loading of a alternate config which may have been
> specified as a command line option for those commands that support it
> (at least the ones that do logging too).
> 

Looks like you are right.  I've backed out the last change.

Well, I installed coLinux on my lap top and was testing if I can debug 
mailman while travelling. ;-)

-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From barry at python.org  Sun Sep 24 22:44:51 2006
From: barry at python.org (Barry Warsaw)
Date: Sun, 24 Sep 2006 16:44:51 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8036]trunk/mailman/Mailman/loginit.py
In-Reply-To: <PC17602006092206405200938f3d1f51@msapiro>
References: <PC17602006092206405200938f3d1f51@msapiro>
Message-ID: <3E547054-512C-4A40-BB96-709546F14A5B@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 22, 2006, at 9:40 AM, Mark Sapiro wrote:

> Yes, but this is not the proper fix. I think the place to put
> config.load() for the web interface is in scripts/driver.

r8039.  I'm still looking at the mail script interface.
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRbuSXEjvBPtnXfVAQJTNgQAjbTUBgvn+lwy0fHOk1CJQGs+Gf9pq0WW
4jVUKnBDCZ6Vnbu0dzVyV+q4D3VHB+vaWU37/h8Th5ZCwm0yJo+sCVHW+6WObJZF
ja3Sjr9ZlLOrDES1y+meg9p9m2LflAKmoh0i8gdpKZZsc1CD7m/sxOUJQ6/yk0y7
wIYo+bHrJKU=
=mOIF
-----END PGP SIGNATURE-----

From barry at python.org  Mon Sep 25 09:56:13 2006
From: barry at python.org (Barry Warsaw)
Date: Mon, 25 Sep 2006 03:56:13 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8036]trunk/mailman/Mailman/loginit.py
In-Reply-To: <PC17602006092206405200938f3d1f51@msapiro>
References: <PC17602006092206405200938f3d1f51@msapiro>
Message-ID: <BDC85EC6-C33B-48DA-93AE-05ACFB798BE0@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 22, 2006, at 9:40 AM, Mark Sapiro wrote:

> ----------- excerpt from message to Barry -------------------
> Unfortunately, I neglected to report something else I saw because I
> didn't have time to figure it out. I'm still not sure about this, but
> here's the problem:
>
> [msapiro at msapiro ...f/test-mailman]$ scripts/post
> Traceback (most recent call last):
>   File "scripts/post", line 48, in ?
>     main()
>   File "scripts/post", line 39, in main
>     status = sys.modules[module_name].main()
>   File "/cygdrive/f/test-mailman/Mailman/bin/post.py", line 44, in  
> main
>     loginit.initialize(propagate=True)
>   File "/cygdrive/f/test-mailman/Mailman/loginit.py", line 119, in
> initialize
>     handler = ReopenableFileHandler(os.path.join(config.LOG_DIR,
> logger))
> AttributeError: 'Configuration' object has no attribute 'LOG_DIR'
>
> In this case, nothing has called config.load(). The same problem  
> exists
> in the web interface. That can be fixed with

posting should work now.   r8041.
- -Barry


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRReLpHEjvBPtnXfVAQIZ2wP/YozcIslfQnBHgV3mWlR039ao1TmLOOIy
PKeTejZOU11O+8f1ofcO5ngTKRiJcEOjLkxbBbEdGlsxZOq/8gAi8vOBThofbROC
/ZDp41FCcLN+Nm7ETDyDd6UPGJLLV0zSKXHbnVTttFN0HbqM5a+03JNnDz+NkepM
dT/0hdM0dJQ=
=M90k
-----END PGP SIGNATURE-----

From tkikuchi at is.kochi-u.ac.jp  Wed Sep 27 04:35:06 2006
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Wed, 27 Sep 2006 11:35:06 +0900
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8041] trunk/mailman/Mailman
In-Reply-To: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
Message-ID: <4519E35A.6070906@is.kochi-u.ac.jp>

bwarsaw at users.sourceforge.net wrote:
> Revision: 8041
>           http://svn.sourceforge.net/mailman/?rev=8041&view=rev
> Author:   bwarsaw
> Date:     2006-09-25 00:53:58 -0700 (Mon, 25 Sep 2006)
> 
> Log Message:
> -----------
> Another milestone: you can now post to lists.  Converted the following to use
> the new configuration object: admin, admindb, bounces, confirm, inject, join,
> leave, owner, post, request, unshunt, version.
> 
> Also change MailList.GetScriptURL() to return the list's fully qualified name
> in links.
> 
I've tested a bit.

Postfix.py: got duplicate warning while creating virtual-mailman.
Shouldn't we include hostname in aliases? Otherwise, true virtual 
hosting breaks.

MailList.py: got error when name is None. Looks like full_path is 
hostname at listname order. Is this name + '@' + self.host_name?

I needed this patch to create new lists:

mailman at colinux:~/src/svn.trunk$ svn diff Mailman
Index: Mailman/MTA/Postfix.py
===================================================================
--- Mailman/MTA/Postfix.py      (revision 8041)
+++ Mailman/MTA/Postfix.py      (working copy)
@@ -142,8 +142,9 @@
      print >> fp, '# CREATED:', time.ctime(time.time())
      # Now add all the standard alias entries
      for k, v in makealiases(mlist):
+        fqdnaddr = '%s@%s' % (k, hostname)
          # Format the text file nicely
-        print >> fp, mlist.fqdn_listname, ((fieldsz - len(k)) * ' '), k
+        print >> fp, fqdnaddr, ((fieldsz - len(k)) * ' '), k
      # Finish the text file stanza
      print >> fp, '# STANZA END:', listname
      print >> fp
Index: Mailman/MailList.py
===================================================================
--- Mailman/MailList.py (revision 8041)
+++ Mailman/MailList.py (working copy)
@@ -287,14 +287,17 @@
              os.path.join(config.LOCK_DIR, name or '<site>') + '.lock',
              lifetime=config.LIST_LOCK_LIFETIME)
          # XXX FIXME Sometimes name is fully qualified, sometimes it's not.
-        if '@' in name:
-            self._internal_name, self.host_name = name.split('@', 1)
-            self._full_path = os.path.join(config.LIST_DATA_DIR, name)
-        else:
-            self._internal_name = name
-            self.host_name = config.DEFAULT_EMAIL_HOST
-            self._full_path = os.path.join(config.LIST_DATA_DIR,
-                                           self.host_name + '@' + name)
+       if name:
+            if '@' in name:
+                self._internal_name, self.host_name = name.split('@', 1)
+                self._full_path = os.path.join(config.LIST_DATA_DIR, name)
+            else:
+                self._internal_name = name
+                self.host_name = config.DEFAULT_EMAIL_HOST
+                self._full_path = os.path.join(config.LIST_DATA_DIR,
+                                               self.host_name + '@' + name)
+       else:
+           self._full_path = ''
          # Only one level of mixin inheritance allowed
          for baseclass in self.__class__.__bases__:
              if hasattr(baseclass, 'InitTempVars'):

-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From bob at nleaudio.com  Wed Sep 27 05:01:05 2006
From: bob at nleaudio.com (Bob Puff@NLE)
Date: Tue, 26 Sep 2006 23:01:05 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
 [8041] trunk/mailman/Mailman
In-Reply-To: <4519E35A.6070906@is.kochi-u.ac.jp>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>
Message-ID: <4519E971.8070202@nleaudio.com>



Tokio Kikuchi wrote:


> Postfix.py: got duplicate warning while creating virtual-mailman.
> Shouldn't we include hostname in aliases? Otherwise, true virtual 
> hosting breaks.

the virtual file should indeed contain the hostname.  The aliases file should not.  These are two
separate files, both of which are necessary.

Example:

data/virtual-mailman:
mailman at nle.com              mailman
mailman-admin at nle.com        mailman-admin
...etc...

data/aliases:
mailman:             "|/home/mailman/mail/mailman post mailman"
mailman-admin:       "|/home/mailman/mail/mailman admin mailman"
...etc...

Both should be created, and both should be deleted when the list is removed.  Currently, this is not
the case.

Bob


From Dale at Newfield.org  Wed Sep 27 06:51:07 2006
From: Dale at Newfield.org (Dale Newfield)
Date: Wed, 27 Sep 2006 00:51:07 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
 [8041] trunk/mailman/Mailman
In-Reply-To: <4519E971.8070202@nleaudio.com>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>	<4519E35A.6070906@is.kochi-u.ac.jp>
	<4519E971.8070202@nleaudio.com>
Message-ID: <451A033B.7090505@Newfield.org>

Bob Puff at NLE wrote:
> the virtual file should indeed contain the hostname.  The aliases
> file should not.  These are two separate files, both of which are 
> necessary.

I mostly agree with you, but your solution won't allow true virtual
hosting (having test at domain1.com and test at domain2.com be separate lists
running on the same machine/mailman instance).

Maybe something like this modified example?

data/virtual-mailman:
> mailman at nle.com           mailman_at_nle_com
> mailman-admin at nle.com     mailman-admin_at_nle_com
> ...etc...


data/aliases:
> mailman_at_nle_com:       "|/home/mailman/mail/mailman post mailman at nle.com
> mailman-admin_at_nle_com: "|/home/mailman/mail/mailman admin mailman at nle.com
> ...etc...

This of course begs the questions of how mailman distinguishes between
the lists (what's the appropriate argument to the mailman binary, and 
whether there are any characters besides "." that are disallowed in 
domain names (what is a good encoding to use as a valid local address)?

-Dale

From barry at python.org  Wed Sep 27 17:54:43 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 27 Sep 2006 11:54:43 -0400
Subject: [Mailman-Developers] [Mailman-Users] OS X & Mailman & Python
In-Reply-To: <44D71836-7288-4E65-BE16-BC40BD785DCF@gmail.com>
References: <44D71836-7288-4E65-BE16-BC40BD785DCF@gmail.com>
Message-ID: <CB1452CD-4978-42EE-AEBB-938A55DD123D@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'm including mailman-developers on this message, because I want to  
discuss the issue of which Python versions to support.

On Sep 27, 2006, at 11:04 AM, Stubbs Jeff wrote:

> Got a question. I picked up another Mac, so I'm going to rebuild my
> list server from scratch, using Postfix 2.3.3 and Mailman 2.1.9. The
> default python install supplied by Apple is version 2.3.5. Last week
> or so , I noticed the version 2.4.3 was available from the Python
> website in an OS X installer. This morning, I noticed that the 2.5
> version was available. (whew, the developers must be sucking in the
> coffee).
>
> All things being equal, would Mailman benefit from using the 2.5
> version as opposed to Apple's 2.3.5?
>
> The old server seemed to run fine, but I wouldn't mind upgrading
> python, if it wasn't going to be problematic.

My recommendation would be to run at least Python 2.4.  Mailman 2.1.9  
should run on Python 2.5, but earlier versions of Mailman won't.   
While we still claim to be able to run Mailman 2.1 on versions of  
Python back to 2.1, I believe we broke that claim in Mailman 2.1.9.   
Does anybody care?

I would dearly love to drop support for Python 2.1 and Python 2.2  
even in the Mailman 2.1 series.  As I'm developing on OSX myself  
these days, I can't even build these earlier versions of Python and  
they haven't been supported by the PSF for years.  In fact, Python  
2.3 is no longer supported and I would even love to drop support for  
Python 2.3 in Mailman 2.1.x, although I know we can't.

Then there is the question of what versions we support for Mailman  
2.2, which is currently under development.  Previously we've said  
we'll support Python 2.3 but I think we should revisit that decision.

In summary my preferences would be:

Mailman 2.1.x supported on Python 2.3, 2.4, and 2.5.  Drop support  
for Python 2.1 and 2.2.  We've done this accidentally in Mailman  
2.1.9, so let's make it official.

Mailman 2.2 supported on Python 2.4 and 2.5.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRqeyHEjvBPtnXfVAQLYNwQAmaO7uxgkORkfy0ebOYU2Ez8jgSlcg4uT
8kz26FBgN0sKzp4hPN/kFK2Mqtao0FFdWxDfzUvb0f96o0moh71yqyUa3lfsoW0Y
eTXXGMyACtjFasCYE2IcFuhHCeNHZlxu/yvlfznFQ6RXMi4c83AC/qeutY9o3Jl9
xTdNyISdvmc=
=/bCq
-----END PGP SIGNATURE-----

From barry at python.org  Wed Sep 27 21:04:06 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 27 Sep 2006 15:04:06 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8041] trunk/mailman/Mailman
In-Reply-To: <451A033B.7090505@Newfield.org>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>	<4519E35A.6070906@is.kochi-u.ac.jp>
	<4519E971.8070202@nleaudio.com> <451A033B.7090505@Newfield.org>
Message-ID: <9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 27, 2006, at 12:51 AM, Dale Newfield wrote:

> This of course begs the questions of how mailman distinguishes between
> the lists (what's the appropriate argument to the mailman binary, and
> whether there are any characters besides "." that are disallowed in
> domain names (what is a good encoding to use as a valid local  
> address)?

I've just read the Postfix documentation again to see if there are  
any new features since the last time we implemented integration with  
Postfix virtual domains.  Here's the man page:

http://www.postfix.org/VIRTUAL_README.html

What looks interesting is the section entitled:

Postfix virtual MAILBOX example: separate domains, non-UNIX accounts

This appears to allow us to set up true virtual domains without  
having to encode destination aliases.  The trick though is that we  
would use Maildir delivery for all incoming messages, something I'm  
keen on switching to for Mailman 2.2 anyway.  Maildir is way more  
efficient than invoking a mail program per incoming message, Mailman  
already supports Maildir (although it isn't the default), and AFAIK  
all major MTAs support Maildir.

I'd like to know what you think about updating our Postfix virtual  
delivery hooks to use this technique, and about making Maildir  
delivery the default.  We'd keep the old way around for MTAs that  
don't support Maildir (which would be...?) but we'd deprecate that  
delivery mechanism.

One thing I'm not sure of is the minimum version of Postfix this  
would tie us to.  On OS X 10.4 it looks like I've got Postfix 2.1.5  
and according to postconf, it supports the  necessary variables.

Thoughts?
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRrLK3EjvBPtnXfVAQK4sAQAkkFgWXxxdk1TxLVYxbb3CEO2y59XJtb5
dRcCT8qg0LyxTkFNGEqAQTi9iRIs2/RVuK1KUBVXpSpQbt7zrGfs0s/ukfPVlVbO
/h45S+cQ8d9cF2s77PHSqiUgdGckMy7W6v15qJ9zrrMC+M38yqV5oAzpxNc660y8
UtXNleCdeGM=
=dgKK
-----END PGP SIGNATURE-----

From tkikuchi at is.kochi-u.ac.jp  Thu Sep 28 02:29:08 2006
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Thu, 28 Sep 2006 09:29:08 +0900
Subject: [Mailman-Developers] [Mailman-Users] OS X & Mailman & Python
In-Reply-To: <CB1452CD-4978-42EE-AEBB-938A55DD123D@python.org>
References: <44D71836-7288-4E65-BE16-BC40BD785DCF@gmail.com>
	<CB1452CD-4978-42EE-AEBB-938A55DD123D@python.org>
Message-ID: <451B1754.2000808@is.kochi-u.ac.jp>

> In summary my preferences would be:
> 
> Mailman 2.1.x supported on Python 2.3, 2.4, and 2.5.  Drop support  
> for Python 2.1 and 2.2.  We've done this accidentally in Mailman  
> 2.1.9, so let's make it official.
> 
> Mailman 2.2 supported on Python 2.4 and 2.5.

+1

-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From carson at taltos.org  Thu Sep 28 03:29:49 2006
From: carson at taltos.org (Carson Gaspar)
Date: Wed, 27 Sep 2006 18:29:49 -0700
Subject: [Mailman-Developers] [Mailman-Users] OS X & Mailman & Python
In-Reply-To: <CB1452CD-4978-42EE-AEBB-938A55DD123D@python.org>
References: <44D71836-7288-4E65-BE16-BC40BD785DCF@gmail.com>
	<CB1452CD-4978-42EE-AEBB-938A55DD123D@python.org>
Message-ID: <AA9EDB809B2815C17EDE4CDA@[192.168.1.160]>

--On Wednesday, September 27, 2006 11:54 AM -0400 Barry Warsaw 
<barry at python.org> wrote:

> Then there is the question of what versions we support for Mailman
> 2.2, which is currently under development.  Previously we've said
> we'll support Python 2.3 but I think we should revisit that decision.

If you drop python 2.3, you drop RHEL4. It doesn't effect me personally, as 
I don't run mailman on my RHEL4 servers, but I suspect it would make many 
folks unhappy.

-- 
Carson

From stephen at xemacs.org  Thu Sep 28 03:32:39 2006
From: stephen at xemacs.org (stephen at xemacs.org)
Date: Thu, 28 Sep 2006 10:32:39 +0900
Subject: [Mailman-Developers] [Mailman-Users] OS X & Mailman & Python
In-Reply-To: <CB1452CD-4978-42EE-AEBB-938A55DD123D@python.org>
References: <44D71836-7288-4E65-BE16-BC40BD785DCF@gmail.com>
	<CB1452CD-4978-42EE-AEBB-938A55DD123D@python.org>
Message-ID: <17691.9783.556058.461008@uwakimon.sk.tsukuba.ac.jp>

Barry Warsaw writes:

 > Mailman 2.2 supported on Python 2.4 and 2.5.

+1.

 > Mailman 2.1.x supported on Python 2.3, 2.4, and 2.5.  Drop support  
 > for Python 2.1 and 2.2.  We've done this accidentally in Mailman  
 > 2.1.9, so let's make it official.

Would it be possible to maintain a rough list of Python-2.3-and-later
features that are required for current Mailman as the requirements are
added?  That would at least give folks who think they need an older
Python some idea of what would be involved in adapting their Python
installation to Mailman needs.



From carson at taltos.org  Thu Sep 28 03:33:33 2006
From: carson at taltos.org (Carson Gaspar)
Date: Wed, 27 Sep 2006 18:33:33 -0700
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN:
 mailman:	[8041] trunk/mailman/Mailman
In-Reply-To: <9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
Message-ID: <1EC9A0DAC4500B094D771196@[192.168.1.160]>

--On Wednesday, September 27, 2006 3:04 PM -0400 Barry Warsaw 
<barry at python.org> wrote:

> I'd like to know what you think about updating our Postfix virtual
> delivery hooks to use this technique, and about making Maildir
> delivery the default.  We'd keep the old way around for MTAs that
> don't support Maildir (which would be...?) but we'd deprecate that
> delivery mechanism.
>
> One thing I'm not sure of is the minimum version of Postfix this
> would tie us to.  On OS X 10.4 it looks like I've got Postfix 2.1.5
> and according to postconf, it supports the  necessary variables.

I love the idea. A fork/exec per message always makes me twitch... I have a 
feeling it would also provide better fault-tolerance, especially in a 
replicated filesystem cluster, where you have clear atomic behaviour at 
your disposal.

-- 
Carson

From barry at python.org  Thu Sep 28 04:09:20 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 27 Sep 2006 22:09:20 -0400
Subject: [Mailman-Developers] [Mailman-Users] OS X & Mailman & Python
In-Reply-To: <AA9EDB809B2815C17EDE4CDA@[192.168.1.160]>
References: <44D71836-7288-4E65-BE16-BC40BD785DCF@gmail.com>
	<CB1452CD-4978-42EE-AEBB-938A55DD123D@python.org>
	<AA9EDB809B2815C17EDE4CDA@[192.168.1.160]>
Message-ID: <EEAB3E35-E327-46F4-B95F-8655AC7B49F4@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 27, 2006, at 9:29 PM, Carson Gaspar wrote:

> --On Wednesday, September 27, 2006 11:54 AM -0400 Barry Warsaw
> <barry at python.org> wrote:
>
>> Then there is the question of what versions we support for Mailman
>> 2.2, which is currently under development.  Previously we've said
>> we'll support Python 2.3 but I think we should revisit that decision.
>
> If you drop python 2.3, you drop RHEL4. It doesn't effect me  
> personally, as
> I don't run mailman on my RHEL4 servers, but I suspect it would  
> make many
> folks unhappy.

I'm not up on the development/release plans of RHEL.  When do you  
think they might release a version with Python 2.4 support?  The same  
kind of goes for OS X, which ships with Python 2.3 on Tiger.  I have  
no idea what Leopard will ship with, but I'd have to hope that it  
would be at least 2.4.x if not 2.5.x.

I guess my standard answers would be (In no particular order :)

- - It's easy to build an appropriate Python on whatever *nix platform  
you've got
- - I'm sure there are RPMs available for Python 2.4
- - RHEL5 will probably beat Mailman 2.2 :)

I suspect that Mailman won't be the only Python app out there  
applying pressure to upgrade to a more modern (read "supported")  
version of Python.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRsu0XEjvBPtnXfVAQJwxAP/ard0TFLRDrSalg2W4Fz92gkwFuEdCBPQ
1mSwo7/c9ORPxzPPQ8hgLE2SoN2LeaBFKzFw0HV20Av4g5coQhV2XjwZO+1PbhXt
SXXx7A9iwVgsMrqo8qFvUpUBBc9BQptCF1WrTzkbkTdmvSxMEmcpoW0mbOd+Po5U
aTnvtFK6ccU=
=bi7z
-----END PGP SIGNATURE-----

From carson at taltos.org  Thu Sep 28 04:19:02 2006
From: carson at taltos.org (Carson Gaspar)
Date: Wed, 27 Sep 2006 19:19:02 -0700
Subject: [Mailman-Developers] [Mailman-Users] OS X & Mailman & Python
In-Reply-To: <EEAB3E35-E327-46F4-B95F-8655AC7B49F4@python.org>
References: <44D71836-7288-4E65-BE16-BC40BD785DCF@gmail.com>
	<CB1452CD-4978-42EE-AEBB-938A55DD123D@python.org>
	<AA9EDB809B2815C17EDE4CDA@[192.168.1.160]>
	<EEAB3E35-E327-46F4-B95F-8655AC7B49F4@python.org>
Message-ID: <E229E29F7CC8DC0FE26DFD6D@[192.168.1.2]>

--On Wednesday, September 27, 2006 10:09 PM -0400 Barry Warsaw 
<barry at python.org> wrote:

> I'm not up on the development/release plans of RHEL.  When do you
> think they might release a version with Python 2.4 support?  The same
> kind of goes for OS X, which ships with Python 2.3 on Tiger.  I have
> no idea what Leopard will ship with, but I'd have to hope that it
> would be at least 2.4.x if not 2.5.x.

RHEL5 is in beta. It currently has python-2.4.3 (humorously, the RPM is 
still called python-2.4.3-14.fc6). As I said, _I_ don't care about mailman 
on RHEL4 (I run it on Solaris x86 as I like servers that don't crash), I 
just wanted to raise the issue. Feel free to say "legacy OS, legacy 
mailman". But keeping 2.3 support for future 2.1.x security/bugfix releases 
would be a good idea.

-- 
Carson

From barry at python.org  Thu Sep 28 04:22:38 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 27 Sep 2006 22:22:38 -0400
Subject: [Mailman-Developers] [Mailman-Users] OS X & Mailman & Python
In-Reply-To: <17691.9783.556058.461008@uwakimon.sk.tsukuba.ac.jp>
References: <44D71836-7288-4E65-BE16-BC40BD785DCF@gmail.com>
	<CB1452CD-4978-42EE-AEBB-938A55DD123D@python.org>
	<17691.9783.556058.461008@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <49139071-22FB-4987-A5A1-064C21633B04@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 27, 2006, at 9:32 PM, <stephen at xemacs.org> wrote:

>> Mailman 2.1.x supported on Python 2.3, 2.4, and 2.5.  Drop support
>> for Python 2.1 and 2.2.  We've done this accidentally in Mailman
>> 2.1.9, so let's make it official.
>
> Would it be possible to maintain a rough list of Python-2.3-and-later
> features that are required for current Mailman as the requirements are
> added?  That would at least give folks who think they need an older
> Python some idea of what would be involved in adapting their Python
> installation to Mailman needs.

Mark Sapiro wrote this message describing the unintended breakage in  
Mailman 2.1.9:

http://mail.python.org/pipermail/mailman-users/2006-September/ 
053290.html

So the big difference between 2.1 and 2.2 was the unification of  
classes and types, which also changed the built-in factory functions  
like int() and str() to be types instead of functions.

No one should use Python 2.2 for anything really.  It was a fairly  
radical release and many of the new features didn't stabilize until  
Python 2.3.  The main reason I want to drop Python 2.1 and 2.2 is  
that I simply can't build them on OS X any more, so I can't  
effectively test them.  I'm not sure if Tokio and Mark are in the  
same boat though.  I can't build Python 2.3 either, but at least  
there, I don't have to (thanks Apple!).

As for Mailman 2.2, there are lots and lots of features I want to use  
from Python 2.4.  Built-in sets, generators, PEP 292 $-strings  
(pioneered in Mailman), decorators, and the subprocess module to name  
a few.  Of the new-in-Python 2.5 features I'd use but can live  
without, probably conditional expressions absolute imports, and the  
with statement are the most interesting.  Oh, and the built-in  
sqlite3 package <wink>.

http://www.python.org/doc/2.3/whatsnew/whatsnew23.html
http://www.python.org/doc/2.4/whatsnew/whatsnew24.html
http://www.python.org/doc/2.5/whatsnew/whatsnew25.html

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRsx7nEjvBPtnXfVAQI3SgP/b3jeJti1AVveujcH1gwfwcwtG1LpU23X
ECNQP2wybm6xwIhIl2Hjop58A6CjrauAZvWtF2YspMHeg6l/NnZ7DcCzc1VbKZQT
cAhmsrHOh+MK5tIdLaOkQtl4T8D8i8tmtLrTDO+Wh6rhfG/oVhDa2IbNrdUZ59LQ
yDvB1Nc+1m0=
=yYt8
-----END PGP SIGNATURE-----

From barry at python.org  Thu Sep 28 04:26:45 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 27 Sep 2006 22:26:45 -0400
Subject: [Mailman-Developers] [Mailman-Users] OS X & Mailman & Python
In-Reply-To: <E229E29F7CC8DC0FE26DFD6D@[192.168.1.2]>
References: <44D71836-7288-4E65-BE16-BC40BD785DCF@gmail.com>
	<CB1452CD-4978-42EE-AEBB-938A55DD123D@python.org>
	<AA9EDB809B2815C17EDE4CDA@[192.168.1.160]>
	<EEAB3E35-E327-46F4-B95F-8655AC7B49F4@python.org>
	<E229E29F7CC8DC0FE26DFD6D@[192.168.1.2]>
Message-ID: <1F4ABC97-E0AE-4AB9-8456-ED1E71AC0531@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 27, 2006, at 10:19 PM, Carson Gaspar wrote:

> --On Wednesday, September 27, 2006 10:09 PM -0400 Barry Warsaw
> <barry at python.org> wrote:
>
>> I'm not up on the development/release plans of RHEL.  When do you
>> think they might release a version with Python 2.4 support?  The same
>> kind of goes for OS X, which ships with Python 2.3 on Tiger.  I have
>> no idea what Leopard will ship with, but I'd have to hope that it
>> would be at least 2.4.x if not 2.5.x.
>
> RHEL5 is in beta. It currently has python-2.4.3 (humorously, the  
> RPM is
> still called python-2.4.3-14.fc6). As I said, _I_ don't care about  
> mailman
> on RHEL4 (I run it on Solaris x86 as I like servers that don't crash),

:)  Yeah, unless someone wants to donate an Xserve, I'll probably  
keep my production Gentoo x86 box around for a while longer.  I'm  
still running my own lists and websites on that machine, but I won't  
do development on it.

> I
> just wanted to raise the issue. Feel free to say "legacy OS, legacy
> mailman". But keeping 2.3 support for future 2.1.x security/bugfix  
> releases
> would be a good idea.

Oh yes, sorry I wasn't clear.  We'll definitely continue to support  
Python 2.3 for Mailman 2.1.x.  We'll drop Python 2.3 for Mailman  
2.2.  I'll write this up in the wiki.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRsy5nEjvBPtnXfVAQKgSgP/Vi1HasUpVSRtF+kB8p8L2qX8tlit/hD8
3cuiGonuBejOtmUi0qQF0Y4e1ma4vRqE8sKloPQn9UYHE990fqgqpxlclCa0P1K7
kMGxR3V+aHqhYvd8MjglvFvvhffbf2lkkf92YtV7+gK6S2Utknci8nx2kR9u1wFa
UAW7BGllEM8=
=6M6d
-----END PGP SIGNATURE-----

From brad at stop.mail-abuse.org  Thu Sep 28 04:34:04 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Wed, 27 Sep 2006 21:34:04 -0500
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
 [8041] trunk/mailman/Mailman
In-Reply-To: <9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
Message-ID: <p06240500c140e36ee174@[10.0.1.4]>

At 3:04 PM -0400 9/27/06, Barry Warsaw wrote:

>  This appears to allow us to set up true virtual domains without
>  having to encode destination aliases.  The trick though is that we
>  would use Maildir delivery for all incoming messages, something I'm
>  keen on switching to for Mailman 2.2 anyway.  Maildir is way more
>  efficient than invoking a mail program per incoming message, Mailman
>  already supports Maildir (although it isn't the default), and AFAIK
>  all major MTAs support Maildir.

Sendmail knows nothing of the mailbox delivery method.  That is left 
up to the Local Delivery Agent, which is usually /bin/mail and knows 
about 7th edition mbox-format, and not much else.  You could always 
substitute a different LDA (e.g., procmail), but that would not be a 
standard part of sendmail.

Moreover, I'm not keen on Maildir.  It makes a lot of trade-offs to 
try to get something that is NFS-safe, and I'm not convinced those 
trade-offs are worthwhile, especially not in a non-NFS environment. 
I think there are other ways that you could get the same benefits (in 
a mailbox directory solution), without getting the major drawbacks of 
Maildir per se.


Among other things Maildir creates really hairy long filenames, which 
can easily blow the iname/inode caching built into most filesystems 
-- you could get the same benefit by using a better filename 
naming/hashing scheme with fewer characters.  It also does a lot of 
excessive synchronous meta-data operations (e.g., creating files, 
renaming files, etc...), and that can place a heavy load on the 
underlying filesystem.

In his paper regarding what they built for the Earthlinnk mail 
system, Nick Christenson has clearly proven that you can use the 
atomic creat() system call in a way that eliminates the need for file 
locking on NFS, without all the various baggage that Maildir brings 
to the table.

Mark Crispin also has a lot of good things to say about the 
weaknesses inherent in Maildir.  You should read his comments, too.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From brad at stop.mail-abuse.org  Thu Sep 28 04:36:14 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Wed, 27 Sep 2006 21:36:14 -0500
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
 [8041] trunk/mailman/Mailman
In-Reply-To: <1EC9A0DAC4500B094D771196@[192.168.1.160]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
Message-ID: <p06240501c140e520474b@[10.0.1.4]>

At 6:33 PM -0700 9/27/06, Carson Gaspar wrote:

>  I love the idea. A fork/exec per message always makes me twitch... I have a
>  feeling it would also provide better fault-tolerance, especially in a
>  replicated filesystem cluster, where you have clear atomic behaviour at
>  your disposal.

I agree that fork()/exec() is not an ideal model here, but then 
postfix doesn't use that model internally -- it uses a single parent 
with multiple child processes, and then hands off sockets.  It also 
keeps pretty much the entire working queue in memory, as opposed to 
single-threading through the filesystem.

I don't see how using Maildir is going to solve any of these 
problems.  IMO, if we're going to learn from postfix, I think we 
should learn the right things and take away the right lessons, and 
not just glom onto some alternative technique that has been known to 
have a whole host of other problems.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From brad at stop.mail-abuse.org  Thu Sep 28 04:45:42 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Wed, 27 Sep 2006 21:45:42 -0500
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
 [8041] trunk/mailman/Mailman
In-Reply-To: <p06240500c140e36ee174@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<p06240500c140e36ee174@[10.0.1.4]>
Message-ID: <p06240502c140e657901b@[10.0.1.4]>

At 9:34 PM -0500 9/27/06, Brad Knowles wrote:

>  Moreover, I'm not keen on Maildir.  It makes a lot of trade-offs to
>  try to get something that is NFS-safe, and I'm not convinced those
>  trade-offs are worthwhile, especially not in a non-NFS environment.

One other problem with Maildir -- it throws all message files into 
the same directory, and doesn't use a hashed directory scheme.  If 
we're going to do directory hashing, I would think we'd want to do it 
within our mailbox storage mechanism as well as elsewhere in the 
queueing system.

>  In his paper regarding what they built for the Earthlinnk mail
>  system, Nick Christenson has clearly proven that you can use the
>  atomic creat() system call in a way that eliminates the need for file
>  locking on NFS, without all the various baggage that Maildir brings
>  to the table.

If you want to read Nick's paper, go to 
<http://www.jetcafe.org/npc/doc/mail_arch.html>.  Note that he's also 
the author of the book _sendmail Performance Tuning_, see 
<http://www.jetcafe.org/npc/book/sendmail/>.

>  Mark Crispin also has a lot of good things to say about the
>  weaknesses inherent in Maildir.  You should read his comments, too.

Mark's comments can be read at 
<http://www.washington.edu/imap/documentation/formats.txt.html>.  Do 
a "find" on the page for "maildir", and read from there.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From barry at python.org  Thu Sep 28 05:54:30 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 27 Sep 2006 23:54:30 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8041] trunk/mailman/Mailman
In-Reply-To: <p06240501c140e520474b@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
Message-ID: <91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 27, 2006, at 10:36 PM, Brad Knowles wrote:

> At 6:33 PM -0700 9/27/06, Carson Gaspar wrote:
>
>>  I love the idea. A fork/exec per message always makes me  
>> twitch... I have a
>>  feeling it would also provide better fault-tolerance, especially  
>> in a
>>  replicated filesystem cluster, where you have clear atomic  
>> behaviour at
>>  your disposal.
>
> I agree that fork()/exec() is not an ideal model here, but then
> postfix doesn't use that model internally -- it uses a single parent
> with multiple child processes, and then hands off sockets.  It also
> keeps pretty much the entire working queue in memory, as opposed to
> single-threading through the filesystem.
>
> I don't see how using Maildir is going to solve any of these
> problems.  IMO, if we're going to learn from postfix, I think we
> should learn the right things and take away the right lessons, and
> not just glom onto some alternative technique that has been known to
> have a whole host of other problems.

Remember that we're only talking about how to most efficiently get  
mail from the MTA via local delivery into the Mailman incoming  
queue.  Once in Mailman we'll handle the message our own way, not  
with maildir.

This means that we're limited by what the various MTAs we want to  
integrate with support.  Until now, we've always gone with delivery  
to a program, because that's supported by all MTAs.  This is what  
Carson was referring to by fork/exec -- the MTA must fork/exec  
Mailman's mail wrapper (i.e. post) which basically just sucks the  
message text from stdin and write it to a file.  It seems clear that  
if you could eliminate that extra process (essentially a glorified  
cat), you'd get a win.  I think you're going to get about the same  
amount of filesystem thrashing in either case, so you might as well  
avoid the extra process overhead.

Looking at Postfix, what other options are readily available?  I  
suppose you could try to hook into the transport maps, but if I  
understand them correctly, you're still talking about forking a  
process per message.  Maybe LMTP to a daemon process is another  
option, but there appears to be no documentation on www.postfix.org  
about LMTP.

As for what's best for other MTAs, that's a good question.  I think  
we'll always have to support delivery-to-program since that seems  
like it's the lowest common method.  If there are delivery mechanisms  
that make more sense for specific MTAs, then I'm all for including  
them, but others who are more familiar with those mailer servers will  
have to help (read: donate code).

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRtHdnEjvBPtnXfVAQIVKQP+PBYbnBeG7M7MySaZdGyUy74d/cRwModC
p19xJNMDsSvKKR4dy+uWzyzTF9uM3xPSKQUokvMWQHudyMmtf980E6db4THF1/do
i87z/B3gLS+7SwyXYeRPqnMlEjuHkmFGjPS4t9jz9sGaraixVE9qMqJ3F8S+n7hi
6DuAdUtRqOw=
=s19m
-----END PGP SIGNATURE-----

From barry at python.org  Thu Sep 28 05:56:25 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 27 Sep 2006 23:56:25 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8041] trunk/mailman/Mailman
In-Reply-To: <p06240500c140e36ee174@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<p06240500c140e36ee174@[10.0.1.4]>
Message-ID: <822D7395-BA6D-47C2-894C-132D8FB7E601@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 27, 2006, at 10:34 PM, Brad Knowles wrote:

> At 3:04 PM -0400 9/27/06, Barry Warsaw wrote:
>
>>  This appears to allow us to set up true virtual domains without
>>  having to encode destination aliases.  The trick though is that we
>>  would use Maildir delivery for all incoming messages, something I'm
>>  keen on switching to for Mailman 2.2 anyway.  Maildir is way more
>>  efficient than invoking a mail program per incoming message, Mailman
>>  already supports Maildir (although it isn't the default), and AFAIK
>>  all major MTAs support Maildir.
>
> Sendmail knows nothing of the mailbox delivery method.  That is left
> up to the Local Delivery Agent, which is usually /bin/mail and knows
> about 7th edition mbox-format, and not much else.  You could always
> substitute a different LDA (e.g., procmail), but that would not be a
> standard part of sendmail.

I'm definitely not proposing to get rid of deliver to program, so at  
worst, Sendmail users will continue to use this method.  Is there a  
better way to get the message from Sendmail into Mailman's incoming  
queue?

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRtH6XEjvBPtnXfVAQLnawP9FhQDaHH4TtFlV2oo/FwT1YipNuxl3Kr1
vjswKQtKgQN7QUYuSZwnOSZ3O7PBHjzSNXbuu2GQt2hEYYm0VQkbO4I173EO4HKR
xSkPGDrBU6n3NDC3WjV7BSedBSHlPnJXmnEsTobxeUQTCeeJRwsxO3QdPef22kn5
qmVOmMwfkBw=
=0R2g
-----END PGP SIGNATURE-----

From carson at taltos.org  Thu Sep 28 06:15:44 2006
From: carson at taltos.org (Carson Gaspar)
Date: Wed, 27 Sep 2006 21:15:44 -0700
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
 [8041] trunk/mailman/Mailman
In-Reply-To: <91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
Message-ID: <7749FAF4A2F117A94DCE2D37@[192.168.1.2]>

--On Wednesday, September 27, 2006 11:54 PM -0400 Barry Warsaw 
<barry at python.org> wrote:

> Looking at Postfix, what other options are readily available?  I  suppose
> you could try to hook into the transport maps, but if I  understand them
> correctly, you're still talking about forking a  process per message.
> Maybe LMTP to a daemon process is another  option, but there appears to
> be no documentation on www.postfix.org  about LMTP.

LMTP is fully supported in postfix. You just set mailbox_transport (or trhe 
right hand side of your transport map) to lmtp:... - see postfix's smtp(8). 
I've used it to deliver to cyrus imapd for a long time now. See RFC 2033 
for the LMTP standard.

-- 
Carson

From brad at stop.mail-abuse.org  Thu Sep 28 06:25:50 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Wed, 27 Sep 2006 23:25:50 -0500
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
 [8041] trunk/mailman/Mailman
In-Reply-To: <91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
Message-ID: <p06240501c140fe121fbc@[10.0.1.4]>

At 11:54 PM -0400 9/27/06, Barry Warsaw wrote:

>  Looking at Postfix, what other options are readily available?  I
>  suppose you could try to hook into the transport maps, but if I
>  understand them correctly, you're still talking about forking a
>  process per message.

Use LMTP instead.  This will allow you to completely avoid an 
intermediate (and unnecessary) queue-to-disk stage.

>                        Maybe LMTP to a daemon process is another
>  option, but there appears to be no documentation on
>  www.postfix.org about LMTP.

Wietse didn't invent LMTP, he's just using the technique that was 
invented by others.  Look in the sendmail source code, it includes an 
LDA that implements LMTP.  For that matter, I think postfix also 
includes an LDA that implements LMTP.

LMTP is basically just exactly like SMTP, except that it's via the 
localhost interface only, and simplifies a number of other 
assumptions as well.  So, if you've got code that can do SMTP, then 
you've already got code that can do LMTP.


Frankly, there's not much to LMTP, which I think is a large part of 
why you're not likely to see a great deal written about it within the 
sendmail and postfix codebases.

>  As for what's best for other MTAs, that's a good question.  I
>  think we'll always have to support delivery-to-program since
>  that seems like it's the lowest common method.  If there are
>  delivery mechanisms that make more sense for specific MTAs,
>  then I'm all for including them,

LMTP is probably the best and most native method for both sendmail 
and postfix.  I can't speak for other MTAs.

>                                   but others who are more
>  familiar with those mailer servers will have to help (read:
>  donate code).

IIRC, sendmail has a Berkeley-style copyright, so the "donation" of 
code should not be an issue.  Converting the code from C to Python, 
now that may be a bit more of a problem.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From brad at stop.mail-abuse.org  Thu Sep 28 06:27:07 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Wed, 27 Sep 2006 23:27:07 -0500
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
 [8041] trunk/mailman/Mailman
In-Reply-To: <822D7395-BA6D-47C2-894C-132D8FB7E601@python.org>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<p06240500c140e36ee174@[10.0.1.4]>
	<822D7395-BA6D-47C2-894C-132D8FB7E601@python.org>
Message-ID: <p06240502c140ff5b6cfe@[10.0.1.4]>

At 11:56 PM -0400 9/27/06, Barry Warsaw wrote:

>  I'm definitely not proposing to get rid of deliver to program,
>  so at worst, Sendmail users will continue to use this method.
>  Is there a better way to get the message from Sendmail into
>  Mailman's incoming queue?

Well, sendmail does LMTP to their custom LDA that they include which 
is not officially part of the actual sendmail package itself.  But 
you could easily plug in any other LMTP-capable LDA.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From barry at python.org  Thu Sep 28 06:34:12 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Sep 2006 00:34:12 -0400
Subject: [Mailman-Developers] [Mailman-Users]   OS X & Mailman & Python
In-Reply-To: <451B1754.2000808@is.kochi-u.ac.jp>
References: <44D71836-7288-4E65-BE16-BC40BD785DCF@gmail.com>
	<CB1452CD-4978-42EE-AEBB-938A55DD123D@python.org>
	<451B1754.2000808@is.kochi-u.ac.jp>
Message-ID: <4F3FD161-AA25-40F0-B46D-4C8E978F1D71@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 27, 2006, at 8:29 PM, Tokio Kikuchi wrote:

>> In summary my preferences would be:
>>
>> Mailman 2.1.x supported on Python 2.3, 2.4, and 2.5.  Drop support
>> for Python 2.1 and 2.2.  We've done this accidentally in Mailman
>> 2.1.9, so let's make it official.
>>
>> Mailman 2.2 supported on Python 2.4 and 2.5.
>
> +1

Cool, it's official then. :)

http://wiki.list.org/x/8Q
http://wiki.list.org/x/IQ

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRtQxHEjvBPtnXfVAQLtcAQAlc7v5W0a6uUJgdvn8C0QC9crWKt1yqzJ
U7Mylo33yFiUyXzbI1qkos6AJaAVij2q/elWdRj2+8sUOfBMdHI4NKpZQDcrFe2A
wzzPZTG7HfTyckFMfOb0TYFqkzonlKAbBZTuqrTqagLh79k5FUFE8mPuqITpOiZ4
k4c3H4qU+2k=
=4SI6
-----END PGP SIGNATURE-----

From barry at python.org  Thu Sep 28 06:40:21 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Sep 2006 00:40:21 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8041] trunk/mailman/Mailman
In-Reply-To: <7749FAF4A2F117A94DCE2D37@[192.168.1.2]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<7749FAF4A2F117A94DCE2D37@[192.168.1.2]>
Message-ID: <F9A077B2-15D2-4073-B5B9-D8FCD573104F@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 28, 2006, at 12:15 AM, Carson Gaspar wrote:

> --On Wednesday, September 27, 2006 11:54 PM -0400 Barry Warsaw
> <barry at python.org> wrote:
>
>> Looking at Postfix, what other options are readily available?  I   
>> suppose
>> you could try to hook into the transport maps, but if I   
>> understand them
>> correctly, you're still talking about forking a  process per message.
>> Maybe LMTP to a daemon process is another  option, but there  
>> appears to
>> be no documentation on www.postfix.org  about LMTP.
>
> LMTP is fully supported in postfix. You just set mailbox_transport  
> (or trhe
> right hand side of your transport map) to lmtp:... - see postfix's  
> smtp(8).
> I've used it to deliver to cyrus imapd for a long time now. See RFC  
> 2033
> for the LMTP standard.

Cool, thank for the reference.  I'll read up on that.
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRtSNXEjvBPtnXfVAQLjfAP9GLgoymsgU+O8oCp+yHn2qR1qE4lkqnId
WtSNvUP7cAzL/CtJ6HGdp+Kbyqw/raC9yS797EGBniHu3Y8ACCaInyY0mVS5a8h+
0oIejbeGCvxJCsvPt653H7MSjgHwdv/xeLZBZx4xKxw3c8fCuYG9eUQrli6eXXIq
TkF93BbrtOQ=
=Oeh2
-----END PGP SIGNATURE-----

From barry at python.org  Thu Sep 28 07:07:26 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Sep 2006 01:07:26 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8041] trunk/mailman/Mailman
In-Reply-To: <p06240501c140fe121fbc@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
Message-ID: <DC077DEF-D090-436F-886E-1836A246A179@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 28, 2006, at 12:25 AM, Brad Knowles wrote:

> At 11:54 PM -0400 9/27/06, Barry Warsaw wrote:
>
>>  Looking at Postfix, what other options are readily available?  I
>>  suppose you could try to hook into the transport maps, but if I
>>  understand them correctly, you're still talking about forking a
>>  process per message.
>
> Use LMTP instead.  This will allow you to completely avoid an  
> intermediate (and unnecessary) queue-to-disk stage.

The only problem I see is that I'm not sure Postfix can be configured  
to deliver via LMTP on a per-recipient basis.  It looks like you have  
to specify LTMP via transports and so must deliver all mail for a  
particular domain to that transport.  That's not good because of  
course we want Mailman to only receive messages for the aliases  
corresponding to mailing lists, while other things like local  
recipients or non-list aliases get delivered in the "normal way".

Or is there some way I'm missing that would allow us to segregate  
some domain traffic to Mailman's LMTP server and other traffic to  
Postfix's standard transports?  What about Sendmail?

Other than that, we'd need a reliable standards-compliant LMTP server  
written in Python (and no, smtpd.py- or Twisted-based versions are  
not acceptable ;).

- -Barry


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRtYj3EjvBPtnXfVAQKIowP/anqUQekOOQO8v18sE9PuDtqfeIt8gamF
r+AUEyLFQJ9D2RiCcrJm74mXbGGoVVOxsTaMpIzLdfLL20K+MtX8UtgAcEYT40+C
TzrT34p3I8pVriplpsLIXMb539If3/CR00I3XRhlHhYGbMt14PC/KlpAQ0TCkzqH
uRRDtEWXDlk=
=HwLP
-----END PGP SIGNATURE-----

From carson at taltos.org  Thu Sep 28 07:57:20 2006
From: carson at taltos.org (Carson Gaspar)
Date: Wed, 27 Sep 2006 22:57:20 -0700
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN:
 mailman:	[8041] trunk/mailman/Mailman
In-Reply-To: <DC077DEF-D090-436F-886E-1836A246A179@python.org>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
	<DC077DEF-D090-436F-886E-1836A246A179@python.org>
Message-ID: <FDDC8C5F3D4B3B60E15BC669@[192.168.1.160]>

--On Thursday, September 28, 2006 1:07 AM -0400 Barry Warsaw 
<barry at python.org> wrote:

> Or is there some way I'm missing that would allow us to segregate
> some domain traffic to Mailman's LMTP server and other traffic to
> Postfix's standard transports?  What about Sendmail?

Shouldn't be an issue with postfix. From the default postfix transport map 
template:

# TABLE LOOKUP
#        With lookups from indexed files such as DB or DBM, or from
#        networked  tables  such  as NIS, LDAP or SQL, patterns are
#        tried in the order as listed below:
#
#        user+extension at domain transport:nexthop
#               Mail for user+extension at domain is delivered through
#               transport to nexthop.
#
#        user at domain transport:nexthop
#               Mail for user at domain is delivered through transport
#               to nexthop.
#
#        domain transport:nexthop
#               Mail for domain is delivered through  transport  to
#               nexthop.
#
#        .domain transport:nexthop
#               Mail  for  any  subdomain  of  domain  is delivered
#               through transport to  nexthop.  This  applies  only
#               when the string transport_maps is not listed in the
#               parent_domain_matches_subdomains configuration set-
#               ting.   Otherwise, a domain name matches itself and
#               its subdomains.
#
#        Note 1: the special pattern * represents any address (i.e.
#        it functions as the wild-card pattern).
#
#        Note  2:  the  null  recipient  address  is  looked  up as
#        $empty_address_recipient@$myhostname (default: mailer-dae-
#        mon at hostname).

From brad at stop.mail-abuse.org  Thu Sep 28 09:10:43 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Thu, 28 Sep 2006 02:10:43 -0500
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
 [8041] trunk/mailman/Mailman
In-Reply-To: <FDDC8C5F3D4B3B60E15BC669@[192.168.1.160]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
	<DC077DEF-D090-436F-886E-1836A246A179@python.org>
	<FDDC8C5F3D4B3B60E15BC669@[192.168.1.160]>
Message-ID: <p06240504c14125aa6774@[10.0.1.4]>

At 10:57 PM -0700 9/27/06, Carson Gaspar wrote:

>>  Or is there some way I'm missing that would allow us to segregate
>>  some domain traffic to Mailman's LMTP server and other traffic to
>>  Postfix's standard transports?  What about Sendmail?
>
>  Shouldn't be an issue with postfix. From the default postfix transport map
>  template:

Sendmail should be able to do something comparable, although I have 
not yet looked at the documentation to see just exactly how you'd 
implement that.  Certainly, looking up addresses in a variety of 
database types is not a new idea.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From tkikuchi at is.kochi-u.ac.jp  Thu Sep 28 10:06:25 2006
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Thu, 28 Sep 2006 17:06:25 +0900
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8043] trunk/mailman/Mailman
In-Reply-To: <E1GSqiw-0001yu-T2@sc8-pr-svn1.sourceforge.net>
References: <E1GSqiw-0001yu-T2@sc8-pr-svn1.sourceforge.net>
Message-ID: <451B8281.4070909@is.kochi-u.ac.jp>

Hi Barry and all,

I've checked in this patch today.  Barry, please fix or backout them if 
you don't like them.

It looked like that MaildirRunner.py couldn't deal with the list name 
like 'mailman-users', so I rewrote it with a primitive method instead of 
using the re module.  One problem: I manage a list called 
'system-admin at ...'; this can't go well with this method.  Isn't it time 
to get rid of the -admin address in mailman-2.2? ;-)

tkikuchi at users.sourceforge.net wrote:
> Revision: 8043
>           http://svn.sourceforge.net/mailman/?rev=8043&view=rev
> Author:   tkikuchi
> Date:     2006-09-28 00:54:21 -0700 (Thu, 28 Sep 2006)
> 
> Log Message:
> -----------
> Here are the patches needed in order to create new lists on my test
> installation.  I've tested four cases of combination of 
> POSTFIX_STYLE_VIRTUAL_DOMAINS (Any/None) and USE_MAIL_DIR (Yes/No).
> Also, I've added POSTFIX_VIRTUAL_SEPARATOR = '_at_' for unique mapping of
> local part in the virtual-mailman/aliases files.
> 
> Modified Paths:
> --------------
>     trunk/mailman/Mailman/Defaults.py.in
>     trunk/mailman/Mailman/MTA/Postfix.py
>     trunk/mailman/Mailman/MailList.py
>     trunk/mailman/Mailman/Queue/MaildirRunner.py



-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From Nigel.Metheringham at dev.intechnology.co.uk  Thu Sep 28 10:11:14 2006
From: Nigel.Metheringham at dev.intechnology.co.uk (Nigel Metheringham)
Date: Thu, 28 Sep 2006 09:11:14 +0100
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN:
	mailman:	[8041] trunk/mailman/Mailman
In-Reply-To: <p06240501c140fe121fbc@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
Message-ID: <1159431074.5512.3.camel@angua.localnet>

On Wed, 2006-09-27 at 23:25 -0500, Brad Knowles wrote:
> LMTP is probably the best and most native method for both sendmail 
> and postfix.  I can't speak for other MTAs.

Exim can do LMTP, over a pipe (ie fork/exec program), a socket or
TCP/IP.

	Nigel.
-- 
[ Nigel Metheringham           Nigel.Metheringham at InTechnology.co.uk ]
[ - Comments in this message are my own and not ITO opinion/policy - ]


From barry at python.org  Thu Sep 28 14:12:55 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Sep 2006 08:12:55 -0400
Subject: [Mailman-Developers] LTMP for incoming mail
In-Reply-To: <1159431074.5512.3.camel@angua.localnet>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
	<1159431074.5512.3.camel@angua.localnet>
Message-ID: <DD55D877-197E-4CD3-BB08-AF1A83532880@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I've changed the subject to more accurately reflect this thread.

On Sep 28, 2006, at 4:11 AM, Nigel Metheringham wrote:

> On Wed, 2006-09-27 at 23:25 -0500, Brad Knowles wrote:
>> LMTP is probably the best and most native method for both sendmail
>> and postfix.  I can't speak for other MTAs.
>
> Exim can do LMTP, over a pipe (ie fork/exec program), a socket or
> TCP/IP.

What I find really intriguing about this approach is the ability to  
reject some messages immediately, presumably allowing the MTA to  
bounce them.  It's not clear how much work we could get away with at  
message receipt time in our mythical LMTP server, but let's imagine  
we could at least parse the message and do some preliminary sanity  
checks on it.  Say the message had MIME breakage or couldn't be  
parse.  Or say we could do a quick check of the sender's permissions  
to post, etc.  We could reject the message then before it entered  
Mailman's incoming queue.

What worries me most about this though is that we'd have to (probably  
write and) maintain an LMTP server, and make sure that it was  
efficient enough to keep up with incoming mail pressures, both steady  
state and peak, and be highly robust against errors.  Plus, this is  
another moving part that mailmanctl would have to manage, and of  
course, if our LMTP server breaks, your Mailman is no longer  
accepting incoming mail.  We should not underestimate the work  
involved here.

I did a quick Google search to see if there were any GPL'd LMTP  
servers we could piggyback on, the idea being that if we could find a  
shell of a C program we could embed Python in and talk directly to  
Mailman during the LMTP protocol.  I found this one in the PLL project:

http://pll.sourceforge.net/

but it looks like the code is several years old so I doubt it's being  
maintained, and it doesn't appear to have been released as a 1.0  
tool.  Postfix has an lmtp server, but it seems fairly heavyweight  
(being tied into the smtp server) and it's not clear to me we could  
combine our GPL code with Postfix's license.

Of course there's always Twisted, but that seems like a big bite to  
take for this task.

ISTM that the trade-off then is rolling our own LMTP server vs. doing  
maildir delivery.  Are we confident that we can implement a high  
performance enough server that would give us better throughput than  
maildir would?  In Python?

It might be fun to try, but OTOH it /is/ a distraction from other MM  
2.2 work that needs to get done.  So unless anybody has any leads on  
existing GPL-compatible code we could use, or feels really motivated  
to work on a Python version, I'm inclined to go with maildir for  
MM2.2.  It's not like we couldn't add LMTP at some later point.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRu8TXEjvBPtnXfVAQICiAP8DahfyRAcVrrbYIFAUlC8R8AA7oZiuJxV
NPQ/7Juaf0FrU/nrK+2uWZ6Zf614Tv4lQh3TwRxaOgPHgfsSB8a0UknN+Zy+nXyK
gs9DMHQ2iIan/uIDxo1E4Qtu9sDuh1nctbm8pd3NW8bbyvqmNyli9bknHqE/LtDu
12RcvYEBIdc=
=h3r9
-----END PGP SIGNATURE-----

From barry at python.org  Thu Sep 28 14:16:23 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Sep 2006 08:16:23 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8043] trunk/mailman/Mailman
In-Reply-To: <451B8281.4070909@is.kochi-u.ac.jp>
References: <E1GSqiw-0001yu-T2@sc8-pr-svn1.sourceforge.net>
	<451B8281.4070909@is.kochi-u.ac.jp>
Message-ID: <D0DA58B9-C383-43E1-8274-D5AC7A9BBCA2@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 28, 2006, at 4:06 AM, Tokio Kikuchi wrote:

> I've checked in this patch today.  Barry, please fix or backout  
> them if
> you don't like them.

Thanks Tokio!

> It looked like that MaildirRunner.py couldn't deal with the list name
> like 'mailman-users', so I rewrote it with a primitive method  
> instead of
> using the re module.  One problem: I manage a list called
> 'system-admin at ...'; this can't go well with this method.  Isn't it  
> time
> to get rid of the -admin address in mailman-2.2? ;-)

+1 :)

- -Barry



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRu9GHEjvBPtnXfVAQJicQP+MJxjyB8jRv9kzj7GvxKKi4OYc4dT4AAi
nfvSlwugonY24v4ec4cUv/XwuxchslgYKLCSC7W+AE9SGcZAwVFDIBVtwKpjokkP
UVbKse8eH2rlYpKR0T7oiBu6+vcvBa/hJVJtR5MWso4XzqKmQtqQUvYGT2iXhK4p
7IAQerQmwHo=
=2EoY
-----END PGP SIGNATURE-----

From brad at stop.mail-abuse.org  Thu Sep 28 15:21:05 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Thu, 28 Sep 2006 08:21:05 -0500
Subject: [Mailman-Developers] LTMP for incoming mail
In-Reply-To: <DD55D877-197E-4CD3-BB08-AF1A83532880@python.org>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
	<1159431074.5512.3.camel@angua.localnet>
	<DD55D877-197E-4CD3-BB08-AF1A83532880@python.org>
Message-ID: <p06240500c14177f56ade@[10.0.1.4]>

At 8:12 AM -0400 9/28/06, Barry Warsaw wrote:

>  What I find really intriguing about this approach is the ability to
>  reject some messages immediately, presumably allowing the MTA to
>  bounce them.

Yup.

>                We could reject the message then before it entered
>  Mailman's incoming queue.

Indeed, that's a key advantage.  IIRC, procmail does this with the 
system-wide and user-defined rulesets.

>  I did a quick Google search to see if there were any GPL'd LMTP
>  servers we could piggyback on, the idea being that if we could find a
>  shell of a C program we could embed Python in and talk directly to
>  Mailman during the LMTP protocol.

Does it have to be GPL?  Is a Berkeley-type license not okay? 
Checking the source for sendmail 8.13.8, I find that there is an 
official part of the package which includes the LDA "mail.local", 
which is LMTP-capable, among other things.  It can also do user 
mailbox hashing, based on the username.  You can either hash directly 
to a path like /var/mail/u/s/user or use an MD5 hash of the username 
in a base64 representation (changing "/" to "_"), and you can control 
how many levels of hashing are to be used.

Seems to me like this would be a pretty obvious candidate.

>        Postfix has an lmtp server, but it seems fairly heavyweight
>(being tied into the smtp server) and it's not clear to me we could
>combine our GPL code with Postfix's license.

Please check out the sendmail mail.local stuff and tell me if this is 
a better alternative.  If you need a different license, please let me 
know -- I've known Eric for many years (since way before the company 
existed).  While I won't make any guarantees, I will say that if we 
need a different license, I imagine that I can get a more sympathetic 
ear than you might otherwise be able to find.

>  ISTM that the trade-off then is rolling our own LMTP server vs. doing
>  maildir delivery.  Are we confident that we can implement a high
>  performance enough server that would give us better throughput than
>  maildir would?  In Python?

Dunno about doing it in Python, but I will say that going to Maildir 
as an additional queue-on-disk mechanism on top of everything else 
we're already doing seems to be a big step backward in terms of 
potential performance issues and I don't really see any significant 
positive benefit.


At AOL, we used to use a queue-on-disk method for the Internet mail 
gateways.  Sendmail would take the incoming message, hand it off to a 
custom LDA, the custom LDA would then dump that in a disk queue 
asynchronously, then a synchronous queue runner process would come 
along and pick up the messages and send them over to Stratus. 
Believe me, this system sucked big time -- we had never ending 
problems with disk queues building up to the point where the queue 
runners could never possibly catch up, etc....

And I'm not seeing any real significant operational differences here 
between what you're talking about doing and what AOL abandoned years 
ago.  Okay, so you're talking about using Maildir instead of a 
typical "linear" queue-on-disk and you don't have to do file locking 
to guarantee queue entry creation, but that's still dumping 
everything into a single directory from which we then have to scan 
and pull stuff out and you probably do still have to do some sort of 
file locking in order to make sure that the input and output queue 
mechanisms don't step on each others toes.

>  It might be fun to try, but OTOH it /is/ a distraction from other MM
>  2.2 work that needs to get done.  So unless anybody has any leads on
>  existing GPL-compatible code we could use, or feels really motivated
>  to work on a Python version, I'm inclined to go with maildir for
>  MM2.2.  It's not like we couldn't add LMTP at some later point.

The single queue directory on disk is already one of our biggest 
single bottlenecks.  I don't see how using Maildir as a delivery 
mechanism from the MTA to Mailman is going to improve that.

In fact, it seems to me like we're just adding yet another bottleneck 
of exactly the same sort that we're trying to eliminate elsewhere, 
but with some additional drawbacks that are unique to Maildir and 
which will make our overall system performance even worse than it is 
today.


If we're going to make a big change, it seems to me that LMTP makes 
much more sense than Maildir.  If we can't do LMTP, then I think we'd 
be much better off working on eliminating other bottlenecks in the 
system as opposed to adding yet another totally new source of 
bottlenecks that result from implementing Maildir.


It seems to me that this idea is a case of:

	1.  We have to do Something.
	2.  This is something.
	3.  Therefore, we have to do This.

I think we want to think long and hard about this idea and all it's 
potential drawbacks and new bottleneck sources, before we take that 
first step off the cliff.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From barry at python.org  Thu Sep 28 16:09:22 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Sep 2006 10:09:22 -0400
Subject: [Mailman-Developers] LTMP for incoming mail
In-Reply-To: <p06240500c14177f56ade@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
	<1159431074.5512.3.camel@angua.localnet>
	<DD55D877-197E-4CD3-BB08-AF1A83532880@python.org>
	<p06240500c14177f56ade@[10.0.1.4]>
Message-ID: <6DDEBFCA-410D-4E4B-8BFF-AA4BA45592DC@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 28, 2006, at 9:21 AM, Brad Knowles wrote:

> At 8:12 AM -0400 9/28/06, Barry Warsaw wrote:
>
> Does it have to be GPL?  Is a Berkeley-type license not okay?

GPL would be best, but Berkeley is probably okay.  We'd probably want  
to get confirmation of that from the FSF.  The key thing is that it  
has to be compatible with the GPL (and the Python Software LIcense --  
see below) so that we can combine the whole kit and kaboodle.

> Checking the source for sendmail 8.13.8, I find that there is an  
> official part of the package which includes the LDA "mail.local",  
> which is LMTP-capable, among other things.  It can also do user  
> mailbox hashing, based on the username.  You can either hash  
> directly to a path like /var/mail/u/s/user or use an MD5 hash of  
> the username in a base64 representation (changing "/" to "_"), and  
> you can control how many levels of hashing are to be used.
>
> Seems to me like this would be a pretty obvious candidate.

So a sketch of the architecture would be to embed Python via its C  
API into mail.local, adding callbacks at each point in the LMTP  
protocol.  From there, we'd write the rules in Python so that they'd  
do the message parsing, sanity checking, etc, returning status codes  
which mail.local would use to respond to the LMTP command.  There has  
to be that Python hook, otherwise you can't get to Mailman's data  
structures.

>>        Postfix has an lmtp server, but it seems fairly heavyweight
>> (being tied into the smtp server) and it's not clear to me we could
>> combine our GPL code with Postfix's license.
>
> Please check out the sendmail mail.local stuff and tell me if this  
> is a better alternative.  If you need a different license, please  
> let me know -- I've known Eric for many years (since way before the  
> company existed).  While I won't make any guarantees, I will say  
> that if we need a different license, I imagine that I can get a  
> more sympathetic ear than you might otherwise be able to find.

Cool, that's good to know.

>
>>  ISTM that the trade-off then is rolling our own LMTP server vs.  
>> doing
>>  maildir delivery.  Are we confident that we can implement a high
>>  performance enough server that would give us better throughput than
>>  maildir would?  In Python?
>
> Dunno about doing it in Python, but I will say that going to  
> Maildir as an additional queue-on-disk mechanism on top of  
> everything else we're already doing seems to be a big step backward  
> in terms of potential performance issues and I don't really see any  
> significant positive benefit.

I don't think it's an additional queue-on-disk mechanism, certainly  
in comparison to what we're doing today.  In fact, thinking about it  
more, a maildir approach would be better than what we have today not  
only because it eliminates the extra process fork/exec, but because  
we could segregate queue/in files into per-list subdirectories.  That  
way, you're not dumping all message destined for Mailman into one  
directory.  Not as good as directory hashing, but better than what we  
have today.

I'll grant you that LMTP delivery has the potential to be the most  
efficient mechanism by which messages get from the MTA into Mailman.   
But it's certainly more work and more complicated than maildir;  will  
you grant that maildir is better than what we have today?  Think of  
it as a waystation on the road to the ultimate uber-performing list  
server. :)

Let me just say that ideally, I think LMTP would be a great way to  
go.  It's not my top priority though.  I'm looking for ways to get  
more developers involved in the project, and this seems like a  
perfect thing for someone seeking Mailman fame and fortune <wink>.   
So, anyone care to take the challenge?

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRvXmHEjvBPtnXfVAQLdQAP8DRQdxi/rnHPIB4I+q3xReOpq2yeW7Wsp
y+1AerklGjAhy9+NHOAV2h28rj9YBaS9cp0euJuSWTAoceJfeqBvYM6voL/mfs0I
elTntMROq5diyzytfTxBz9qMDM+QAfuutcp7nuxlPCuYv7CskeAySZll7+v0P8eD
1unF56C5Dns=
=R86i
-----END PGP SIGNATURE-----

From iane at sussex.ac.uk  Thu Sep 28 15:49:35 2006
From: iane at sussex.ac.uk (Ian Eiloart)
Date: Thu, 28 Sep 2006 14:49:35 +0100
Subject: [Mailman-Developers] LTMP for incoming mail
In-Reply-To: <p06240500c14177f56ade@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>	<1159431074.5512.3.camel@angua.localnet>
	<DD55D877-197E-4CD3-BB08-AF1A83532880@python.org>
	<p06240500c14177f56ade@[10.0.1.4]>
Message-ID: <DE29A4A5BBE0AE9351F7DAA3@lewes.staff.uscs.susx.ac.uk>



--On 28 September 2006 08:21:05 -0500 Brad Knowles 
<brad at stop.mail-abuse.org> wrote:

>
>>  What I find really intriguing about this approach is the ability to
>>  reject some messages immediately, presumably allowing the MTA to
>>  bounce them.
>
> Yup.
>
>>                We could reject the message then before it entered
>>  Mailman's incoming queue.
>
> Indeed, that's a key advantage.  IIRC, procmail does this with the
> system-wide and user-defined rulesets.

If that's a reason for using LMTP, then I'd prefer SMTP. Exim can call 
forward to an SMTP server to see if it will accept a message before it's 
too late to reject it at SMTP time. I don't think it can do that with LMTP, 
though I may be wrong.

So, with SMTP if Mailman were to reject a sender to a list, my MTA could 
reject it without causing a bounce.

-- 
Ian Eiloart
IT Services, University of Sussex

From brad at stop.mail-abuse.org  Thu Sep 28 18:06:29 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Thu, 28 Sep 2006 11:06:29 -0500
Subject: [Mailman-Developers] LTMP for incoming mail
In-Reply-To: <6DDEBFCA-410D-4E4B-8BFF-AA4BA45592DC@python.org>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
	<1159431074.5512.3.camel@angua.localnet>
	<DD55D877-197E-4CD3-BB08-AF1A83532880@python.org>
	<p06240500c14177f56ade@[10.0.1.4]>
	<6DDEBFCA-410D-4E4B-8BFF-AA4BA45592DC@python.org>
Message-ID: <p06240501c1419be7d777@[10.0.1.4]>

At 10:09 AM -0400 9/28/06, Barry Warsaw wrote:

>>  Does it have to be GPL?  Is a Berkeley-type license not okay?
>
>  GPL would be best, but Berkeley is probably okay.  We'd probably
>  want to get confirmation of that from the FSF.  The key thing is
>  that it has to be compatible with the GPL (and the Python
>  Software LIcense -- see below) so that we can combine the whole
>  kit and kaboodle.

Is there any license questions or issues that we would need to have 
answered or confirmed by the Sendmail Consortium?  Or should we wait 
on that until we've heard back from the FSF?

>>  Dunno about doing it in Python, but I will say that going
>>  to Maildir as an additional queue-on-disk mechanism on top of
>>  everything else we're already doing seems to be a big step
>>  backward in terms of potential performance issues and I don't
>>  really see any significant positive benefit.
>
>  I don't think it's an additional queue-on-disk mechanism,
>  certainly in comparison to what we're doing today.

Maildir was not designed as an efficient queue-on-disk strategy.  It 
was designed to allow multiple simultaneous parallel deliveries to 
the NFS-mounted mailbox of a given user, and we know that it does a 
number of additional unnecessary things that seriously hurt its 
performance even in that relatively tightly defined context.

It does unnecessary file renames (which cause additional synchronous 
meta-data filesystem operations), it uses filenames that are too long 
and bust iname/inode caching schemes, and it doesn't make use of 
obvious significant performance-enhancing mechanisms like directory 
hashing.

It's pretty easy to design a mechanism that is much more efficient -- 
and scalable -- in handling multiple simultaneous deliveries to a 
user mailbox on NFS.


So why would we want to abuse a bad scheme for user-mailbox-on-NFS as 
an alternative scheme for queue-on-disk?

If we have queue-on-disk problems, why not solve them by implementing 
a more efficient queue-on-disk scheme, instead of abusing a poorly 
designed user-mailbox-on-NFS scheme?

>                                                That way,
>  you're not dumping all message destined for Mailman into
>  one directory.  Not as good as directory hashing, but
>  better than what we have today.

That would be somewhat of an improvement in some respects, but 
Maildir also brings along a lot of additional baggage and I'm not at 
all convinced that it's worth the effort.

>  I'll grant you that LMTP delivery has the potential to be
>  the most efficient mechanism by which messages get from
>  the MTA into Mailman.  But it's certainly more work and
>  more complicated than maildir;  will you grant that maildir
>  is better than what we have today?  Think of it as a
>  waystation on the road to the ultimate uber-performing
>  list server. :)

I'm not at all convinced that Maildir would be an overall improvement 
over what we have today.  I think that adding a directory hashing 
scheme on a fork()/exec() model would probably be a bigger 
improvement than changing our inbox delivery mechanism from a 
fork()/exec() model and using Maildir instead.

At least by sticking with fork()/exec() and adding a directory 
hashing scheme on top of that, we wouldn't need to make any changes 
to the way we interface with MTAs today -- all the changes could be 
kept completely internal to Mailman.  If we were to switch to Maildir 
as an inbox delivery method, not only would we have to change the way 
we interface with MTAs, we would also have to make internal changes 
to Mailman to support the use of Maildir as our queue-on-disk 
mechanism.  That's a bigger overall change with bigger risk and 
relatively lower potential payoff.


If we were to work on implementing a directory hashing scheme instead 
of working on Maildir, we could still add LMTP at a later date.

That would allow us to go back at a later time and enhance our 
features that we provide to Mailing list administrators, while also 
giving us time to look more deeply into the potential performance 
issues and make sure that we're not causing more problems than we're 
solving.

>  Let me just say that ideally, I think LMTP would be a
>  great way to go.  It's not my top priority though.  I'm
>  looking for ways to get more developers involved in the
>  project, and this seems like a perfect thing for someone
>  seeking Mailman fame and fortune <wink>.

I'm not convinced that this is an improvement.

>  So, anyone care to take the challenge?

I'm not a developer, but I do have experience with building 
large-scale mail and mailing list systems, and if you're willing to 
listen to me then I'm willing to give you the benefit of my 
experience.


IMO, Maildir is a Red Herring.  The one and only reason to ever 
consider using Maildir is if you're implementing a large-scale IMAP 
mail server system and you're required to store user mailboxes on NFS.

Even then, you'd be well-served to look for better storage 
mechanisms, because throwing potentially hundreds of thousands of 
messages into a single directory is guaranteed to cause huge 
performance issues, even if every single mailbox operation didn't 
involve scanning the entire directory and doing a stat() on every 
single file, locking the entire directory, creating/renaming/deleting 
the file(s) as appropriate, and then unlocking the directory.


I think we're better off spending our resources working on trying to 
resolve the real bottleneck issues that we already know are present 
in our system as opposed to working on cool stuff that may or may not 
help but would require more overall changes to more parts of the 
system and with relatively lower potential payoff.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From Dale at Newfield.org  Thu Sep 28 18:16:55 2006
From: Dale at Newfield.org (Dale Newfield)
Date: Thu, 28 Sep 2006 12:16:55 -0400
Subject: [Mailman-Developers] LTMP for incoming mail
In-Reply-To: <p06240501c1419be7d777@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>	<451A033B.7090505@Newfield.org>	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>	<1EC9A0DAC4500B094D771196@[192.168.1.160]>	<p06240501c140e520474b@[10.0.1.4]>	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>	<p06240501c140fe121fbc@[10.0.1.4]>	<1159431074.5512.3.camel@angua.localnet>	<DD55D877-197E-4CD3-BB08-AF1A83532880@python.org>	<p06240500c14177f56ade@[10.0.1.4]>	<6DDEBFCA-410D-4E4B-8BFF-AA4BA45592DC@python.org>
	<p06240501c1419be7d777@[10.0.1.4]>
Message-ID: <451BF577.8000104@Newfield.org>

Brad Knowles wrote:
> I think we're better off spending our resources working on trying to 
> resolve the real bottleneck issues that we already know are present 
> in our system as opposed to working on cool stuff that may or may not 
> help but would require more overall changes to more parts of the 
> system and with relatively lower potential payoff.


While I agree with Brad some days more than others, he has a good point 
here.  Presumably every mailman installation is a multiplier for email 
messages (I.E.: many more go out than come in), so if we're worrying 
about the bottlenecks at the inbound side, I'm willing to bet they 
result in much more important bottlenecks on the outbound side...

-Dale

From jwblist3 at olympus.net  Thu Sep 28 19:01:44 2006
From: jwblist3 at olympus.net (John W. Baxter)
Date: Thu, 28 Sep 2006 10:01:44 -0700
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
 [8041] trunk/mailman/Mailman
In-Reply-To: <1159431074.5512.3.camel@angua.localnet>
Message-ID: <C1414E08.ED28%jwblist3@olympus.net>

On 9/28/06 1:11 AM, "Nigel Metheringham"
<Nigel.Metheringham at dev.intechnology.co.uk> wrote:

> On Wed, 2006-09-27 at 23:25 -0500, Brad Knowles wrote:
>> LMTP is probably the best and most native method for both sendmail
>> and postfix.  I can't speak for other MTAs.
> 
> Exim can do LMTP, over a pipe (ie fork/exec program), a socket or
> TCP/IP.

Exim can, indeed.  But for some cases only if built with a special build
flag.  From the (4.6.1) spec:

The lmtp transport runs the LMTP protocol (RFC 2033) over a pipe to a
specified command or by interacting with a Unix domain socket. This
transport is something of a cross between the pipe and smtp transports. Exim
also has support for using LMTP over TCP/IP; this is implemented as an
option for the smtp transport. Because LMTP is expected to be of minority
interest, the default build-time configure in src/EDITME has it commented
out. You need to ensure that

TRANSPORT_LMTP=yes
is present in your Local/Makefile in order to have the lmtp transport
included in the Exim binary.

However, we seem to be interested in LMTP over TCP (to localhost), and I
*think* that is available without the TRANSPORT_LMTP=yes build.

As one data point, the Exim (4.54) shipped with CentOS-4.4 is built without
the TRANSPORT_LMTP flag:
# exim -bV
...
Transports: appendfile/maildir autoreply pipe smtp
...

A quick test with exim -bV -C testConfig
suggests that the protocol = lmtp option in an smtp transport is at least
not a syntax error (and I believe it will work).

  --John



From jwblist3 at olympus.net  Thu Sep 28 19:14:25 2006
From: jwblist3 at olympus.net (John W. Baxter)
Date: Thu, 28 Sep 2006 10:14:25 -0700
Subject: [Mailman-Developers] [Mailman-Users] OS X & Mailman & Python
In-Reply-To: <AA9EDB809B2815C17EDE4CDA@[192.168.1.160]>
Message-ID: <C1415101.ED2B%jwblist3@olympus.net>

On 9/27/06 6:29 PM, "Carson Gaspar" <carson at taltos.org> wrote:

> --On Wednesday, September 27, 2006 11:54 AM -0400 Barry Warsaw
> <barry at python.org> wrote:
> 
>> Then there is the question of what versions we support for Mailman
>> 2.2, which is currently under development.  Previously we've said
>> we'll support Python 2.3 but I think we should revisit that decision.
> 
> If you drop python 2.3, you drop RHEL4. It doesn't effect me personally, as
> I don't run mailman on my RHEL4 servers, but I suspect it would make many
> folks unhappy.

Well, to run Mailman later than 2.1.5 (with backports) (I think it is) in
RHEL 4 (or, therefore, CentOS-4), one is looking at building Mailman from
source rather than using official packages.  If one is capable of doing
that, one is also capable of doing an alternative install of a newer Python,
and telling one's Mailman to use that.

Unofficial packages would have to take the Python version problem into
account.

So while the requirement is a nuisance, it's not a show stopper IMHO.

  --John



From jam at jamux.com  Thu Sep 28 19:29:45 2006
From: jam at jamux.com (John A. Martin)
Date: Thu, 28 Sep 2006 13:29:45 -0400
Subject: [Mailman-Developers] LTMP for incoming mail
In-Reply-To: <p06240501c1419be7d777@[10.0.1.4]> (Brad Knowles's message of
	"Thu, 28 Sep 2006 11:06:29 -0500")
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp> <4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
	<1159431074.5512.3.camel@angua.localnet>
	<DD55D877-197E-4CD3-BB08-AF1A83532880@python.org>
	<p06240500c14177f56ade@[10.0.1.4]>
	<6DDEBFCA-410D-4E4B-8BFF-AA4BA45592DC@python.org>
	<p06240501c1419be7d777@[10.0.1.4]>
Message-ID: <87bqozucfq.fsf@athene.jamux.com>

>>>>> "Brad" == Brad Knowles
>>>>> "Re: [Mailman-Developers] LTMP for incoming mail"
>>>>>  Thu, 28 Sep 2006 11:06:29 -0500

    Brad> Maildir was not designed as an efficient queue-on-disk
    Brad> strategy. 

Is in not possible to do Postfix virtual mailbox domains _without_
maildir style delivery?  (Considering Postfix virtual mailbox domains
is what lead to this, no?)  Doesn't the example given in the Postfix
VIRTUAL_README show delivery to a mailbox on line 11 and delivery to a
maildir on line 12?  Why not consider Postfix virtual mailbox domains
and their workalikes with other MTAs separately and independently from
the choices of delivery format and local delivery agents some of which
may also be chosen independently?

Independently of the above and at least for Postfix, would it be
worthwhile looking at the Postfix policy daemon plug-in as a way to
query Mailman information during the rfc821 conversation and rejecting
a lot of messages before DATA?  Even more interesting, because of
perhaps being portable, might be the milter facility that appeared in
Postfix 2.3.  I have not yet even looked at the later so it may AFIK
be very wide of the mark.

        jam

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 154 bytes
Desc: not available
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060928/668021da/attachment.pgp 

From barry at python.org  Thu Sep 28 20:40:51 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Sep 2006 14:40:51 -0400
Subject: [Mailman-Developers] LTMP for incoming mail
In-Reply-To: <p06240501c1419be7d777@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
	<1159431074.5512.3.camel@angua.localnet>
	<DD55D877-197E-4CD3-BB08-AF1A83532880@python.org>
	<p06240500c14177f56ade@[10.0.1.4]>
	<6DDEBFCA-410D-4E4B-8BFF-AA4BA45592DC@python.org>
	<p06240501c1419be7d777@[10.0.1.4]>
Message-ID: <389F23E7-926C-4958-8B7F-BE3C2D993415@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 28, 2006, at 12:06 PM, Brad Knowles wrote:

> Is there any license questions or issues that we would need to have  
> answered or confirmed by the Sendmail Consortium?  Or should we  
> wait on that until we've heard back from the FSF?

I would ask them if their license is GPL compatible.  IOW, do they  
believe we can combine GPL code with theirs?  Better yet would be  
cases where that's actually been done before.

> Maildir was not designed as an efficient queue-on-disk strategy.   
> It was designed to allow multiple simultaneous parallel deliveries  
> to the NFS-mounted mailbox of a given user, and we know that it  
> does a number of additional unnecessary things that seriously hurt  
> its performance even in that relatively tightly defined context.
>
> It does unnecessary file renames (which cause additional  
> synchronous meta-data filesystem operations), it uses filenames  
> that are too long and bust iname/inode caching schemes, and it  
> doesn't make use of obvious significant performance-enhancing  
> mechanisms like directory hashing.
>
> It's pretty easy to design a mechanism that is much more efficient  
> -- and scalable -- in handling multiple simultaneous deliveries to  
> a user mailbox on NFS.
>
> So why would we want to abuse a bad scheme for user-mailbox-on-NFS  
> as an alternative scheme for queue-on-disk?
>
> If we have queue-on-disk problems, why not solve them by  
> implementing a more efficient queue-on-disk scheme, instead of  
> abusing a poorly designed user-mailbox-on-NFS scheme?

Remember, this discussion all started because Postfix virtual host  
delivery is broken on the trunk.  The virtual_mailbox_maps feature is  
a new one since we last looked at how to integrate Mailman and  
Postfix.  What looks appealing about this is that we can actually pre- 
sort message based on recipient address, and in fact we could pre- 
sort by domain, list name, and list alias.  This really has a big  
advantage in that Mailman's incoming runner can do less message  
inspection to determine where that message is supposed to go.  If a  
file came from /usr/local/mailman/queue/in/mydomain/mylist/post, we  
know immediately that it's destined for list members.  Etc.  We also  
get a layout with fewer messages in more subdirectories for free.

virtual_mailbox_maps don't appear to be useful for delivery-to- 
program, and delivery-to-program with Postfix virtual domains as  
we're doing them now has important disadvantages, most notably that  
we have to create both an alias and a virtual recipient, and we have  
to encode the domain name such that it's a valid alias, without  
introducing additional collisions.  That's icky.

I think John was asking about using virtual_mailbox_maps with  
delivery to mbox, but I think that's worse, because mbox delivery  
forces you to implement locking to avoid contention on the shared  
file.  So if we're going to utilize virtual_mailbox_maps I think  
we're stuck with a maildir layout in queues/in.

>>  I'll grant you that LMTP delivery has the potential to be
>>  the most efficient mechanism by which messages get from
>>  the MTA into Mailman.  But it's certainly more work and
>>  more complicated than maildir;  will you grant that maildir
>>  is better than what we have today?  Think of it as a
>>  waystation on the road to the ultimate uber-performing
>>  list server. :)
>
> I'm not at all convinced that Maildir would be an overall  
> improvement over what we have today.  I think that adding a  
> directory hashing scheme on a fork()/exec() model would probably be  
> a bigger improvement than changing our inbox delivery mechanism  
> from a fork()/exec() model and using Maildir instead.

I don't think there's anyway to really know without implementing it  
and doing some measurements.  Since we won't be losing delivery-to- 
program, that would be possible.

> At least by sticking with fork()/exec() and adding a directory  
> hashing scheme on top of that, we wouldn't need to make any changes  
> to the way we interface with MTAs today -- all the changes could be  
> kept completely internal to Mailman.  If we were to switch to  
> Maildir as an inbox delivery method, not only would we have to  
> change the way we interface with MTAs, we would also have to make  
> internal changes to Mailman to support the use of Maildir as our  
> queue-on-disk mechanism.  That's a bigger overall change with  
> bigger risk and relatively lower potential payoff.

Nope, we simply have to implement a MaildirRunner to pull messages  
out of queue/in using the directory layout format we decide on.  We  
have to do something anyway because the current Postfix integration  
method for virtual domains is broken, and I think the fix is uglier  
and more error prone that switching to a different integration  
method.  I have no problem continuing to maintain delivery-to-program  
for other MTAs, or even Postfix where there's only a single domain.

> If we were to work on implementing a directory hashing scheme  
> instead of working on Maildir, we could still add LMTP at a later  
> date.

A directory hashing scheme is orthogonal to a maildir based queue/in  
scheme.  We should definitely do the former because it buys us  
advantages for the other queues.  We could definitely do LMTP later.   
Or someone running a huge site that would really benefit from LMTP  
could funnel a portion of their profits into paying us to add it <wink>.

>>  Let me just say that ideally, I think LMTP would be a
>>  great way to go.  It's not my top priority though.  I'm
>>  looking for ways to get more developers involved in the
>>  project, and this seems like a perfect thing for someone
>>  seeking Mailman fame and fortune <wink>.
>
> I'm not convinced that this is an improvement.

Was that a comment on the preceding paragraph? :)

>>  So, anyone care to take the challenge?
>
> I'm not a developer, but I do have experience with building large- 
> scale mail and mailing list systems, and if you're willing to  
> listen to me then I'm willing to give you the benefit of my  
> experience.

Absolutely.  But getting LMTP support into Mailman will still require  
a developer to step up and write code.  Maybe Tokio or Mark can be  
convinced, or maybe there's another developer lurking out there who  
would be interested.  I just want to unbreak Postfix virtual domains  
and then fry our bigger fish.

> IMO, Maildir is a Red Herring.  The one and only reason to ever  
> consider using Maildir is if you're implementing a large-scale IMAP  
> mail server system and you're required to store user mailboxes on NFS.

I think the thing you're missing is that we need to get the messages  
from the MTA into Mailman's incoming queue /somehow/, and we're  
basically limited by what the various MTAs have to offer.  This is  
primarily an integration issue, so it's necessarily MTA-specific,  
even if we were to do nothing and stick with delivery-to-program.  We  
can -- and must -- do better with Postfix virtual domains, and as I  
see it,  using virtual_mailbox_maps with maildir delivery is the best  
option available.  I'm still open to other suggestions but as yet, I  
don't see a better way.

BTW, all this discussion of Postfix integration should not make Exim,  
Sendmail, or qmail users feel left out!  If there are betters ways to  
get mail from those MTAs into Mailman's incoming queue, I'm all for  
improving those integration points too.  I just need guidance from  
those more knowledgeable with those MTAs as to what changes we should  
make, if any.  We're not playing favorites, and we're not going to  
make any design choices that would improve Postfix integration at the  
expense of other MTAs.

> I think we're better off spending our resources working on trying  
> to resolve the real bottleneck issues that we already know are  
> present in our system as opposed to working on cool stuff that may  
> or may not help but would require more overall changes to more  
> parts of the system and with relatively lower potential payoff.

Fixing Postfix virtual domain integration is a real problem that  
needs solving, which is how this whole thread started.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRwXOXEjvBPtnXfVAQIYogP/R0+WjnzoYylVdWR9779e9Giht6euldTQ
OjRYXw1IkLGoZOgbXCQF9UvUASw+3NGKVj5nRGKPVBaXOqAZZCYuQHkSTa0ZsIe/
oRBMtbYokHGxV9DFz5g7b6aoSLaHW8u0ieMdk1uvxcrVveVt8jjxD9IifDvhXYBV
V3HYgOrg7Dg=
=pdD3
-----END PGP SIGNATURE-----

From barry at python.org  Thu Sep 28 20:44:25 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Sep 2006 14:44:25 -0400
Subject: [Mailman-Developers] [Mailman-Users]   OS X & Mailman & Python
In-Reply-To: <DE3B460E-E113-4963-AA0F-618A2B4DB7C7@gmail.com>
References: <44D71836-7288-4E65-BE16-BC40BD785DCF@gmail.com>
	<CB1452CD-4978-42EE-AEBB-938A55DD123D@python.org>
	<451B1754.2000808@is.kochi-u.ac.jp>
	<4F3FD161-AA25-40F0-B46D-4C8E978F1D71@python.org>
	<DE3B460E-E113-4963-AA0F-618A2B4DB7C7@gmail.com>
Message-ID: <BEA3D8C5-18E9-493D-879D-06C8B46392D0@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 28, 2006, at 2:13 PM, Stubbs Jeff wrote:

> Barry - thanks for the advice
>
> I just wanted to report that Tiger (10.4.7 : ppc), Python 2.5 (from  
> the OS X installer), and Mailman 2.1.9 works perfectly. Install  
> went without a hitch.
>
> Stumbled a little, setting up virtual domains, but, I've been down  
> that road before, so I knew what to fix.

Excellent, thanks for the feedback Jeff!
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRwYCnEjvBPtnXfVAQIsQAQAqlgmUL/iZuOa+Fxy0e8b/BCkrKOqvujX
fHzLs3nvozDXVu2+FZ6bPOJ89nVdSchIJiUjpVfUvQxmSu3NKy8Dn6szfIgg1zuT
Bk/2gkl8jMPlgh6aknxSkaNKMUJut8P74z/pjbfbBYgdm36AS9K5v1aLBvVhIx5E
sB8vEZ4HP04=
=vN/W
-----END PGP SIGNATURE-----

From jam at jamux.com  Thu Sep 28 22:41:54 2006
From: jam at jamux.com (John A. Martin)
Date: Thu, 28 Sep 2006 16:41:54 -0400
Subject: [Mailman-Developers] LTMP for incoming mail
In-Reply-To: <389F23E7-926C-4958-8B7F-BE3C2D993415@python.org> (Barry Warsaw's
	message of "Thu, 28 Sep 2006 14:40:51 -0400")
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp> <4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
	<1159431074.5512.3.camel@angua.localnet>
	<DD55D877-197E-4CD3-BB08-AF1A83532880@python.org>
	<p06240500c14177f56ade@[10.0.1.4]>
	<6DDEBFCA-410D-4E4B-8BFF-AA4BA45592DC@python.org>
	<p06240501c1419be7d777@[10.0.1.4]>
	<389F23E7-926C-4958-8B7F-BE3C2D993415@python.org>
Message-ID: <87r6xvsoz1.fsf@athene.jamux.com>

A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 154 bytes
Desc: not available
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060928/f2848c36/attachment-0001.pgp 

From brad at stop.mail-abuse.org  Thu Sep 28 22:59:39 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Thu, 28 Sep 2006 15:59:39 -0500
Subject: [Mailman-Developers] LTMP for incoming mail
In-Reply-To: <389F23E7-926C-4958-8B7F-BE3C2D993415@python.org>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
	<1159431074.5512.3.camel@angua.localnet>
	<DD55D877-197E-4CD3-BB08-AF1A83532880@python.org>
	<p06240500c14177f56ade@[10.0.1.4]>
	<6DDEBFCA-410D-4E4B-8BFF-AA4BA45592DC@python.org>
	<p06240501c1419be7d777@[10.0.1.4]>
	<389F23E7-926C-4958-8B7F-BE3C2D993415@python.org>
Message-ID: <p06240501c141df56a57a@[10.0.1.4]>

At 2:40 PM -0400 9/28/06, Barry Warsaw wrote:

>  I would ask them if their license is GPL compatible.  IOW, do they
>  believe we can combine GPL code with theirs?  Better yet would be
>  cases where that's actually been done before.

I'll send a note and ask.

>  Remember, this discussion all started because Postfix virtual host
>  delivery is broken on the trunk.  The virtual_mailbox_maps feature
>  is a new one since we last looked at how to integrate Mailman and
>  Postfix.

But this is a pure postfix issue, and now we're talking about making 
potentially large architectural changes to the system to support this 
one MTA, and without necessarily giving consideration to whether that 
buys us anything for any of the other MTAs.

I understand that integration with postfix is sub-optimal today, but 
I'm not convinced that it makes sense to seriously consider an option 
that may result in throwing out all the other MTAs, in order to fix 
things with postfix.  Worse yet would be trying to maintain two 
different systems, one for postfix and one for everyone else.  Or 
making architectural changes to support the new stuff for postfix, 
which may hurt us for other MTAs.


At the very least, I think it makes sense to look at the overall 
cost/benefit ratio.

Let's assume that we have two systems that are otherwise identical, 
with roughly equivalent traffic.  System A has a single big list, 
while system B has a number of smaller lists, but the overall 
aggregate traffic is equal.

With a Maildir solution, system A will see no benefit to the inbound 
queueing, because you're going to get the same level of contention 
within a single inbound directory for the one big list as you would 
for a single inbound directory for all lists on the system.  System B 
would get a benefit, since each list would not be competing for 
immediate synchronous meta-data update resources with the other 
lists, although there would still be some intra-list competition.

With a hashed directory solution, both systems would see the same 
level of benefit, and intra-list competition would be no worse than 
inter-list competition.  And if the competition were to get too high, 
you simply increase the level of directory hashing.


With a Maildir solution, you give up your ability to implement a 
hashed directory solution, because the MTA would no longer know how 
to write messages to your hashed mailbox-directory-per-list, and to 
get around that you'd have to have some sort of customized local 
delivery agent no matter what.

With a hashed directory solution, if necessary or desirable you could 
still implement a separate directory tree per list within your 
customized local delivery agent, and that directory tree per list 
could look however you want.


Moreover, a Maildir mailbox-per-list solution doesn't do anything for 
outbound queues, whereas a properly implemented hashed directory 
solution should affect outbound at least as much as inbound, at no 
additional implementation cost.

>  virtual_mailbox_maps don't appear to be useful for
>  delivery-to-program, and delivery-to-program with Postfix
>  virtual domains as we're doing them now has important
>  disadvantages, most notably that we have to create both an
>  alias and a virtual recipient, and we have to encode the
>  domain name such that it's a valid alias, without
>  introducing additional collisions.  That's icky.

I know that our current solution is sub-optimal, but I'm not 
convinced that it's the only way to skin this cat.  Moreover, I'm 
also not convinced that Maildir is the only effective way to make use 
of virtual_mailbox_maps.


I am pretty much convinced that using Maildir will effectively 
preclude the ability to make use of directory hashing, precisely 
because you're letting the MTA write directly to a poor standard 
interface instead of handling the internal issues in a manner that is 
opaque to the MTA.

>  I don't think there's anyway to really know without
>  implementing it and doing some measurements.  Since we
>  won't be losing delivery-to-program, that would be possible.

True enough, but there is a cost in terms of lost opportunity, and 
pushing out the delivery schedule by long enough to determine which 
method is going to work better overall.

>  Nope, we simply have to implement a MaildirRunner to pull
>  messages out of queue/in using the directory layout format
>  we decide on.

With Maildir, you don't have any choice in what the directory layout 
will look like.  That's standardized within the Maildir 
implementation, and you can't change that.  Otherwise, you wouldn't 
be using Maildir anymore, you'd be using 
mailbox-directory-solution-that-looks-kinda-semi-sorta-like-maildir-but-modified-by-Mailman-and-incompatible-with-everything-else.

>  A directory hashing scheme is orthogonal to a maildir
>  based queue/in scheme.

I'm not convinced of that.  In fact, I'm convinced that they are 
pretty much mutually exclusive.  That is, unless you're talking about 
using Maildir as a second level of queue-on-disk, before you get to 
the Mailman-internal queue-on-disk mechanism.

Now, if you are talking about two levels of queue-on-disk so that we 
can get both Maildir and queue directory hashing, I think that's 
going to be much, much worse than sticking with the existing postfix 
virtual domain solution.

>  Or someone running a huge site that would really benefit
>  from LMTP could funnel a portion of their profits into
>  paying us to add it <wink>.

I don't think we're doing enough traffic on python.org for them to 
justify paying for it.  I don't think that Apple is doing enough 
traffic with Mailman for them to justify paying for it -- not with 
what we've heard about how the new(er) MacOS X hardware is 
performing, and especially not with the total lack of any support (or 
even acknowledgement) that we get from the corporate types.

I don't think that any of the open-source projects (like FreeBSD) are 
going to be in a position to pay for something like this, or to 
develop & contribute the necessary code, although they might be doing 
enough traffic that they could certainly use these features if they 
were available.

I think that only leaves us with a site like SourceForge, and I think 
you've probably got better contacts there than any of the rest of us.

>  Absolutely.  But getting LMTP support into Mailman will
>  still require a developer to step up and write code.

I'm not that concerned about LMTP.  I think that's a big enough issue 
that we can leave that alone for now.

>  Maybe Tokio or Mark can be convinced, or maybe there's
>  another developer lurking out there who would be
>  interested.  I just want to unbreak Postfix virtual
>  domains and then fry our bigger fish.

I would like to see them unbroken, but I also don't want to see 
anything done that would preclude the use of hashed queue 
directories, or that would add a second level of queue-on-disk and 
yet another source of potential bottlenecks.

>  I think the thing you're missing is that we need to get
>  the messages from the MTA into Mailman's incoming queue
>  /somehow/, and we're basically limited by what the
>  various MTAs have to offer.

I certainly was not understanding your point that you wanted to use 
this as a way to unbreak postfix virtual domains, no.

No, I didn't get that at all.


I'm still not convinced that this is the best way to unbreak postfix 
virtual domains, however.

>  Fixing Postfix virtual domain integration is a real problem that
>  needs solving, which is how this whole thread started.

Agreed, this is a real problem that needs to be resolved.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From brad at stop.mail-abuse.org  Thu Sep 28 22:21:09 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Thu, 28 Sep 2006 15:21:09 -0500
Subject: [Mailman-Developers] LTMP for incoming mail
In-Reply-To: <87bqozucfq.fsf@athene.jamux.com>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp> <4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
	<1159431074.5512.3.camel@angua.localnet>
	<DD55D877-197E-4CD3-BB08-AF1A83532880@python.org>
	<p06240500c14177f56ade@[10.0.1.4]>
	<6DDEBFCA-410D-4E4B-8BFF-AA4BA45592DC@python.org>
	<p06240501c1419be7d777@[10.0.1.4]> <87bqozucfq.fsf@athene.jamux.com>
Message-ID: <p06240500c141de0656ba@[10.0.1.4]>

At 1:29 PM -0400 9/28/06, John A. Martin wrote:

>  Is in not possible to do Postfix virtual mailbox domains _without_
>  maildir style delivery?

Probably, but I'm not sure it really buys us much of anything to have 
separate mailboxes for each list, as opposed to a queue processing 
mechanism that is generally more robust and capable of easily 
handling lots of simultaneous transactions.

>  Independently of the above and at least for Postfix, would it be
>  worthwhile looking at the Postfix policy daemon plug-in as a way to
>  query Mailman information during the rfc821 conversation and rejecting
>  a lot of messages before DATA?

That's going to have the same issues as implementing LMTP, at least 
as far as it concerns performance and ability to handle these kinds 
of operations on a large-scale/real-time basis.

>                                 Even more interesting, because of
>  perhaps being portable, might be the milter facility that appeared in
>  Postfix 2.3.  I have not yet even looked at the later so it may AFIK
>  be very wide of the mark.

Same issues for all.  The particular protocol is not particularly 
relevant to this part of the discussion.  Regardless of how you 
implement the system, it's going to have to do certain types of 
message scanning and parsing, checking against databases and/or 
blacklists, etc....  In all cases, so long as you're holding the 
sender open while you check all these things, you're going to have 
pretty much the same concerns regarding performance and scalability.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From barry at python.org  Fri Sep 29 01:33:30 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Sep 2006 19:33:30 -0400
Subject: [Mailman-Developers] LTMP for incoming mail
In-Reply-To: <p06240501c141df56a57a@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
	<1159431074.5512.3.camel@angua.localnet>
	<DD55D877-197E-4CD3-BB08-AF1A83532880@python.org>
	<p06240500c14177f56ade@[10.0.1.4]>
	<6DDEBFCA-410D-4E4B-8BFF-AA4BA45592DC@python.org>
	<p06240501c1419be7d777@[10.0.1.4]>
	<389F23E7-926C-4958-8B7F-BE3C2D993415@python.org>
	<p06240501c141df56a57a@[10.0.1.4]>
Message-ID: <BC876CA0-5DF9-4CED-9300-66A152524590@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 28, 2006, at 4:59 PM, Brad Knowles wrote:

>>  Remember, this discussion all started because Postfix virtual host
>>  delivery is broken on the trunk.  The virtual_mailbox_maps feature
>>  is a new one since we last looked at how to integrate Mailman and
>>  Postfix.
>
> But this is a pure postfix issue, and now we're talking about  
> making potentially large architectural changes to the system to  
> support this one MTA, and without necessarily giving consideration  
> to whether that buys us anything for any of the other MTAs.

No, we're not.  The /only/ difference to support this will be a new  
incoming queue runner.  Actually, not even that.  We'll just  
modifying the existing MaildirRunner to work better with the file  
system layout that I propose Postfix's virtual_mailbox_maps should be  
configured to use.

MaildirRunner /only/ manages queue/in.  Once the message is pulled  
from the incoming queue, everything else will be MTA agnostic, just  
as it is now.  No other queue will be maildir based (just as they  
aren't now).

It's just one class, and if you don't have Postfix or don't have  
virtual domains, or just don't like the idea at all, then you just  
keep the regular IncomingRunner you've got now and keep delivering to  
the post program.  Nothing changes.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRxb0HEjvBPtnXfVAQJMWgQAidKPtOdIHStZtK2ONcjWTMZKOUJaHsnm
XxNNHdeycnDQY6zzZMnov5QFLT0IDr9a5ASuMnd/XxZy3iHPL8By34Hc7n8+Yly+
b8u7FCN+vbOnuf+s1IoDaQETd05X0AYtdIkdmpfQvassfENmTKGLIp1LqKv8WcGG
4XCniTRWtr8=
=e7G9
-----END PGP SIGNATURE-----

From barry at python.org  Fri Sep 29 01:40:55 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Sep 2006 19:40:55 -0400
Subject: [Mailman-Developers] LTMP for incoming mail
In-Reply-To: <p06240501c141df56a57a@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<1EC9A0DAC4500B094D771196@[192.168.1.160]>
	<p06240501c140e520474b@[10.0.1.4]>
	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>
	<p06240501c140fe121fbc@[10.0.1.4]>
	<1159431074.5512.3.camel@angua.localnet>
	<DD55D877-197E-4CD3-BB08-AF1A83532880@python.org>
	<p06240500c14177f56ade@[10.0.1.4]>
	<6DDEBFCA-410D-4E4B-8BFF-AA4BA45592DC@python.org>
	<p06240501c1419be7d777@[10.0.1.4]>
	<389F23E7-926C-4958-8B7F-BE3C2D993415@python.org>
	<p06240501c141df56a57a@[10.0.1.4]>
Message-ID: <C42F4359-694F-4C7F-B4A6-35005DD4C2E5@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 28, 2006, at 4:59 PM, Brad Knowles wrote:

> I know that our current solution is sub-optimal, but I'm not  
> convinced that it's the only way to skin this cat.  Moreover, I'm  
> also not convinced that Maildir is the only effective way to make  
> use of virtual_mailbox_maps.

It may not be.  And maybe there's some way other than  
virtual_mailbox_maps to better skin the Postfix virtual domain cat.   
I haven't heard it yet though.

> I am pretty much convinced that using Maildir will effectively  
> preclude the ability to make use of directory hashing, precisely  
> because you're letting the MTA write directly to a poor standard  
> interface instead of handling the internal issues in a manner that  
> is opaque to the MTA.

Not at all.  If we implement directory hashing, all the other queues  
will gain from them equally, regardless of what MTA is being used,  
because after the incoming queue, the MTA doesn't figure into it.   
The incoming queue has always been a bit special anyway.

> With Maildir, you don't have any choice in what the directory  
> layout will look like.  That's standardized within the Maildir  
> implementation, and you can't change that.  Otherwise, you wouldn't  
> be using Maildir anymore, you'd be using mailbox-directory-solution- 
> that-looks-kinda-semi-sorta-like-maildir-but-modified-by-Mailman- 
> and-incompatible-with-everything-else.

Except that above the maildirs, you could partition the directories  
by domain/list/recipient.  At least, that's the way I read the docs  
for virtual_mailbox_maps.  I haven't yet tried it though, so maybe it  
doesn't work the way I expect it to.

>>  A directory hashing scheme is orthogonal to a maildir
>>  based queue/in scheme.
>
> I'm not convinced of that.  In fact, I'm convinced that they are  
> pretty much mutually exclusive.  That is, unless you're talking  
> about using Maildir as a second level of queue-on-disk, before you  
> get to the Mailman-internal queue-on-disk mechanism.
>
> Now, if you are talking about two levels of queue-on-disk so that  
> we can get both Maildir and queue directory hashing, I think that's  
> going to be much, much worse than sticking with the existing  
> postfix virtual domain solution.

This maildir proposal /only affects the incoming queue/ which has  
always been special anyway.  None of the other queues will be  
maildir, and they can all be hashed directories.  The two are  
completely independent.

- -Barry


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRxdh3EjvBPtnXfVAQKkcwP/aJYv25I8jt2sh6YizKTB7iz8VKEenLRY
04DsuGv4RwZZqxsMO1a385XyjXGo7229mrdU14PHhoNMQ1MGoYPYAKVWc8B05n8X
aBWUqNdnwobAnOTWMlK1TNY85wb5nWQ4O9KIgi/HmEz+9r/ujPtKUgeCLL3YyDvJ
XEH9zOL7GOU=
=rqoL
-----END PGP SIGNATURE-----

From barry at python.org  Fri Sep 29 04:16:42 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 28 Sep 2006 22:16:42 -0400
Subject: [Mailman-Developers] [Mailman-Users]   OS X & Mailman & Python
In-Reply-To: <C141D6D6.2A0F2%lstone19@stonejongleux.com>
References: <C141D6D6.2A0F2%lstone19@stonejongleux.com>
Message-ID: <89EC65CC-0F75-4852-A712-45FD031242AC@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 28, 2006, at 8:45 PM, Larry Stone wrote:

> This all made me curious. I'm just a user of Mailman on Mac OS X - no
> development of any sort by me - so I'm good with 2.1.9 and Python  
> 2.3.5 on
> 10.4.7 - but this topic made me look at the Python 2.5 package at
> python.org. I finally figured out that the MacOS X python installer  
> installs
> a new version in /usr/local/bin (as well as other places) separate  
> from the
> Tiger provided version in /usr/bin. But how would you get mailman  
> to use the
> user installed version? mailmanctl has #! /usr/bin/python at the  
> top which
> will send it to the Tiger provided version. Of course you could modify
> mailmanctl but that would be subject to being overwritten when a  
> new version
> of mailman is installed. Or is that the way it would need to be done?

I'm assuming you built Mailman from source.  If so, run configure  
with --with-python=/usr/local/bin/python or put that python on your  
$PATH first, and Mailman will use that one.

If you look at the source, you'll see that the #! line is actually  
@PYTHON@ which gets substituted by configure at build time.  I forget  
exactly why, but the standard #! /usr/bin/env python invocation  
caused problems for people, so now we hardcode it via configure.

- -Barry



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRyCCnEjvBPtnXfVAQKYxAP+Ibg1hUcx6nE/7/hH8g6sbJ/m82+ApBl9
WkSNuR3e01gyLm0ogIu3npk0pQeflVrmsvjqSCP6iBw81gaO+oeo7zkrIyFag7Ck
bqMCPIBkSDzblEUrueyTPhBvlpDlP4+vaaQSHe/EGDYSz+r/nzY/PJR6PU+lLgn4
c3SF+iHwsWU=
=FnoT
-----END PGP SIGNATURE-----

From i at mindlace.net  Fri Sep 29 05:19:01 2006
From: i at mindlace.net (emf)
Date: Thu, 28 Sep 2006 23:19:01 -0400
Subject: [Mailman-Developers] Incoming Queue format
In-Reply-To: <p06240500c140e36ee174@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>	<451A033B.7090505@Newfield.org>	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<p06240500c140e36ee174@[10.0.1.4]>
Message-ID: <451C90A5.8060607@mindlace.net>

Brad Knowles wrote:
> Among other things Maildir creates really hairy long filenames, which 
> can easily blow the iname/inode caching built into most filesystems 

I can't find a filesystem that has a filename dependency for inode 
caching, so I suspect I'm completely misunderstanding this. Could you 
expand on that a bit?

> -- you could get the same benefit by using a better filename 
> naming/hashing scheme with fewer characters.  It also does a lot of 
> excessive synchronous meta-data operations (e.g., creating files, 
> renaming files, etc...), and that can place a heavy load on the 
> underlying filesystem.

Maybe; but there are at least two filesystems (XFS, reiserfs) and likely 
more that handle file renaming/creating really cheaply, and have their 
own ninja ways of dealing with really large directories that are the 
product of a rather large amount of coding hours.

Maildir has the advantage of being bog standard and readily 
comprehended. While I'm all in favor of some lmtp delivery mechanism, I 
don't see why we should continue inventing our own queue-on-disk 
approach merely to cater to poorly designed filesystems.

It seems to me like anyone likely to end up with a huge enough incoming 
mailman queue to care about Maildir's inefficiencies would also be able 
to put a sensible filesystem underneath it.

~ethan fremen

From barry at python.org  Fri Sep 29 06:28:26 2006
From: barry at python.org (Barry Warsaw)
Date: Fri, 29 Sep 2006 00:28:26 -0400
Subject: [Mailman-Developers] [Mailman-Users]   OS X & Mailman & Python
In-Reply-To: <C141F720.2A133%lstone19@stonejongleux.com>
References: <C141F720.2A133%lstone19@stonejongleux.com>
Message-ID: <CBC9BA4C-1D51-4F9F-A10E-77538E1921D4@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 28, 2006, at 11:03 PM, Larry Stone wrote:

> On 9/28/06 9:16 PM, Barry Warsaw at barry at python.org wrote:
>
> So, that leads to the question, is there any reason to install  
> python 2.5
> while running 2.1.9 or are we fine with 2.3.5 if we aren't doing  
> anything
> else that would benefit from 2.5?

If it ain't broke... :)

You're probably fine with 2.3.5.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRRyg9nEjvBPtnXfVAQLEhgP/dGuGbr+fzIzaw7/GZbZFtbXW3DfUUbed
luVI4Rhm4fgBjFbqh35d1fu12/qpIPbLWv4TKFVVV522obDJaXnY9o2sxF7H/bkF
rUrXIEIVOe5usd1jq1btZlrgHrsZUOwZ64MBaGekXOhjp4XS/zTsSgex1clGeQSp
ZBvXFiNSKYs=
=ec3Q
-----END PGP SIGNATURE-----

From brad at stop.mail-abuse.org  Fri Sep 29 06:39:27 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Thu, 28 Sep 2006 23:39:27 -0500
Subject: [Mailman-Developers] Incoming Queue format
In-Reply-To: <451C90A5.8060607@mindlace.net>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<p06240500c140e36ee174@[10.0.1.4]> <451C90A5.8060607@mindlace.net>
Message-ID: <p06240502c14251e87f6f@[10.0.1.4]>

At 11:19 PM -0400 9/28/06, emf wrote:

>  I can't find a filesystem that has a filename dependency for inode
>  caching, so I suspect I'm completely misunderstanding this. Could you
>  expand on that a bit?

Some filesystems implement an in-memory hash of recently accessed 
files, but the filenames are typically truncated to fourteen 
characters, and the paths to the files may likewise be truncated.

>  Maybe; but there are at least two filesystems (XFS, reiserfs) and likely
>  more that handle file renaming/creating really cheaply, and have their
>  own ninja ways of dealing with really large directories that are the
>  product of a rather large amount of coding hours.

XFS and ReiserFS do not comprise the entire universe of all 
filesystems in the world in which Mailman will be operated.

There will be plenty of BSD, Solaris, HP-UX, MacOS X, and other OSes 
where Mailman will be used, and even on Linux you're much more likely 
to run into ext2fs or ext3fs than either XFS or ReiserFS on most of 
the several hundred distributions that are available.

>  Maildir has the advantage of being bog standard and readily
>  comprehended. While I'm all in favor of some lmtp delivery mechanism, I
>  don't see why we should continue inventing our own queue-on-disk
>  approach merely to cater to poorly designed filesystems.

While XFS and ReiserFS may have their advantages (and XFS on SGI Irix 
is much better than XFS on Linux), we can't assume that any portion 
of the Mailman community will be using these kinds of filesystems. 
We must be more conservative in our estimates of what filesystem 
features will be available, and code accordingly.

If we were to assume that everyone had XFS, then let's assume they 
all have XFS on Irix, or even Veritas VxFS.

>  It seems to me like anyone likely to end up with a huge enough incoming
>  mailman queue to care about Maildir's inefficiencies would also be able
>  to put a sensible filesystem underneath it.

That may simply not be possible.  Moreover, I have some real 
operational problems with both XFS on Linux and ReiserFS, and I would 
not run a production mail system using them.  Maybe IBM's JFS, if I 
were forced to run a production mail system on Linux at all, but 
certainly not XFS or ReiserFS.

To be honest, I wouldn't run a real production mail system on 
anything less than Veritas VxFS, and I'd be real choosy about my 
underlying hardware, too -- think Hitachi, not EMC.


So your assumptions about what kinds of filesystems may or may not be 
appropriate are not necessarily going to coincide with the decisions 
that other people make, or the kinds of hardware and OS they may be 
forced to live with.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From sveniu at ifi.uio.no  Thu Sep 28 10:55:14 2006
From: sveniu at ifi.uio.no (Sven Ingebrigt Ulland)
Date: Thu, 28 Sep 2006 10:55:14 +0200
Subject: [Mailman-Developers] Debugging mailman (resolving inline signature
	attachments)
Message-ID: <20060928085514.GA29794@buri.ifi.uio.no>

Hello there. How would I go about debugging mailman, by --
for example -- polluting the .py libraries with print or
assert statements? If I do that (which seems like a pretty
thoughtless idea), the output would end up .. in /dev/null?
It would probably be better to write to file in stead?

Right now I'm having a specific problem, but it would be
nice to have some general debugging methods to use in the
future for other problems.

I'm using version 2.1.5-8sarge2 (yes, Debian sarge).

The current problem is that list msg_footer, that is list
signatures, are inline attached as base64- encoded UTF-8
mime parts, but only in some cases:

Original message: iso8859-15 text/plain.
Result: Mailman distributes the message to the list members
as mime multipart, first part is the original iso8859-15
message, and the list footer is appended like this:

 --===============0410419818==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: inline

WWVzLCB0aGlzIGlzIG5vdCB0aGUgcmVhbCBsaXN0IHNpZ25hdHVyZSwgeW91IGhheG9yIG
NyYXhvcg==

 --===============0410419818==--

The same happens for charset 'us-ascii' and 'iso8859-1',
which were some examples I could find now after a quick
look. When the original message is UTF-8 text/plain,
however, the body and signature are merged together as they
should be. The footer contains *only* standard 7-bit ascii
values.. no 8-bit stuff or strangeness.

Now, mailman determines this footer business in
Handlers/Decorate.py on lines 80 to 110 (roughly).  It calls
the following to determine list charset: lcset =
Utils.GetCharSet(mlist.preferred_language) GetCharSet calls
mm_cfg.LC_DESCRIPTIONS, which imports Defaults.py, where --
since my lists have lang set to 'en' -- 'us-ascii' is
returned. I also tried mapping 'en' to 'utf-8', without
success.

This is the snippet that determines if the message should be
multipart or not (linenums included):

86 if not msg.is_multipart() and msgtype == 'text/plain' and \
87   msg.get('content-transfer-encoding', '').lower() <> 'base64' and \
88   (lcset == 'us-ascii' or mcset == lcset):

The incoming messages from users are:
- not multipart
- charset is text/plain
- content-transfer-encoding is mostly '7bit' (not base64)
- lcset *should* be 'us-ascii', but it appears it might not be
- mcset (message charset) is what the user used, normally
  iso8859-1, iso8859-15 or utf-8.

I'd really like to debug that code passage to verify what
variables are screwed up, and which are not.

Any tips?

best regards,
sven

From tkikuchi at is.kochi-u.ac.jp  Fri Sep 29 15:52:56 2006
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Fri, 29 Sep 2006 22:52:56 +0900
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
 [8041] trunk/mailman/Mailman
In-Reply-To: <DC077DEF-D090-436F-886E-1836A246A179@python.org>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>	<451A033B.7090505@Newfield.org>	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>	<1EC9A0DAC4500B094D771196@[192.168.1.160]>	<p06240501c140e520474b@[10.0.1.4]>	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>	<p06240501c140fe121fbc@[10.0.1.4]>
	<DC077DEF-D090-436F-886E-1836A246A179@python.org>
Message-ID: <451D2538.3020201@is.kochi-u.ac.jp>

Barry Warsaw wrote:

> Other than that, we'd need a reliable standards-compliant LMTP server  
> written in Python (and no, smtpd.py- or Twisted-based versions are  
> not acceptable ;).

Why no smtpd.py ?  There is a MailmanProxy Object in the code which was 
written by you, Barry.  Any SMTP server can become a LMTP server IIRC.

http://docs.python.org/lib/node624.html

-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From barry at python.org  Fri Sep 29 16:21:55 2006
From: barry at python.org (Barry Warsaw)
Date: Fri, 29 Sep 2006 10:21:55 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8041] trunk/mailman/Mailman
In-Reply-To: <451D2538.3020201@is.kochi-u.ac.jp>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>	<451A033B.7090505@Newfield.org>	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>	<1EC9A0DAC4500B094D771196@[192.168.1.160]>	<p06240501c140e520474b@[10.0.1.4]>	<91A97AAD-C7BC-48B6-B1BE-986D6B1AD7CE@python.org>	<p06240501c140fe121fbc@[10.0.1.4]>
	<DC077DEF-D090-436F-886E-1836A246A179@python.org>
	<451D2538.3020201@is.kochi-u.ac.jp>
Message-ID: <4EB35FCA-CDE2-4D0F-9BD1-8879D5D2E59F@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 29, 2006, at 9:52 AM, Tokio Kikuchi wrote:

> Barry Warsaw wrote:
>
>> Other than that, we'd need a reliable standards-compliant LMTP server
>> written in Python (and no, smtpd.py- or Twisted-based versions are
>> not acceptable ;).
>
> Why no smtpd.py ?  There is a MailmanProxy Object in the code which  
> was
> written by you, Barry.  Any SMTP server can become a LMTP server IIRC.
>
> http://docs.python.org/lib/node624.html

Only because smtpd.py is asyncore based and I'm not convinced it can  
be made high performance enough.  An smtpd.py based LTMP server could  
provide an interesting proof of concept though.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRR0sCHEjvBPtnXfVAQJPTwP/XONZhqcXwU29qfROOOJ29+ITWUl0uuu4
Eeafwq+15ndxOItEB5xXeEIc7/ctGDGAARuXsYME+ylvTk9bG3fWhN+kb+VdYuP1
GbAEPaNjJ90+3zjhyDGkKRjru4gH01KKEh5YJygYvIEyor8FcEr+fkrPpMK2ZTNX
Mfw+IJBc73o=
=VHNo
-----END PGP SIGNATURE-----

From msapiro at value.net  Fri Sep 29 17:01:56 2006
From: msapiro at value.net (Mark Sapiro)
Date: Fri, 29 Sep 2006 08:01:56 -0700
Subject: [Mailman-Developers] Debugging mailman (resolving inline
	signatureattachments)
In-Reply-To: <20060928085514.GA29794@buri.ifi.uio.no>
Message-ID: <PC1760200609290801560937368f2601@msapiro>

Sven Ingebrigt Ulland wrote:

>Hello there. How would I go about debugging mailman, by --
>for example -- polluting the .py libraries with print or
>assert statements? If I do that (which seems like a pretty
>thoughtless idea), the output would end up .. in /dev/null?
>It would probably be better to write to file in stead?
>
<snip>
>
>I'd really like to debug that code passage to verify what
>variables are screwed up, and which are not.
>
>Any tips?


Output to stderr from the queue runners (the running Mailman processes)
is copied to Mailman's error log so you can print to sys.stderr and
the output goes to the error log.

You can also use Mailman.Logging.Syslog.syslog() calls to write to a
'debug' or other log (which will be created if necessary).

-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From i at mindlace.net  Fri Sep 29 18:10:42 2006
From: i at mindlace.net (emf)
Date: Fri, 29 Sep 2006 12:10:42 -0400
Subject: [Mailman-Developers] Digest queue possibilities
In-Reply-To: <52BC1298-C70F-4E5A-9329-447E098B3AA4@python.org>
References: <4502CB17.4060808@zarb.org>	<72A98460-0FDE-4930-98F0-C78B858FE9DE@python.org>	<450642AB.4000601@is.kochi-u.ac.jp>	<45068E2C.5060805@is.kochi-u.ac.jp>	<BB20DB5E-D4E6-4F36-B6A8-00AC63FB4DD9@python.org>	<4507463B.9080306@is.kochi-u.ac.jp>
	<52BC1298-C70F-4E5A-9329-447E098B3AA4@python.org>
Message-ID: <451D4582.4000403@mindlace.net>

Barry Warsaw wrote:

> BTW, what do you think about changing the way we hold messages for  
> digests?  E.g. instead of putting them in an mbox file, where it's  
> more difficult to skip bad messages, stick them in a queue-like  
> directory and pull them from there.  Any messages that can't be read  
> and scrubbed would just be removed.

I think this would be a great idea, and something like this will be 
necessary for the Association of Computing in the Humanities for whom I 
may be doing some mailman contract work.

Aside from the specifics that they want, this approach would allow for 
much easier implementation of custom digest approaches, like:

* send me a digest of only the Topics I care about
* similarly, send out Topic-specific digests
* let moderators edit digests, minimally selecting what gets included 
but maybe up to editing posts, reordering their occurence in the digest, 
providing summaries, etc.

None of this is impossible with mbox, but would be easier with 
maildir/queue.

~ethan fremen


From i at mindlace.net  Fri Sep 29 18:23:55 2006
From: i at mindlace.net (emf)
Date: Fri, 29 Sep 2006 12:23:55 -0400
Subject: [Mailman-Developers] Incoming Queue format
In-Reply-To: <p06240502c14251e87f6f@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<p06240500c140e36ee174@[10.0.1.4]> <451C90A5.8060607@mindlace.net>
	<p06240502c14251e87f6f@[10.0.1.4]>
Message-ID: <451D489B.7070100@mindlace.net>

Brad Knowles wrote:

> So your assumptions about what kinds of filesystems may or may not be 
> appropriate are not necessarily going to coincide with the decisions 
> that other people make, or the kinds of hardware and OS they may be 
> forced to live with.

I don't disagree with this assertion, nor am I making assumptions about 
what people get to live with.

I observe that there is a very finite amount of Mailman developer-hours 
to be had, and that the problems you're discussing have been addressed 
by people who spent far more time on the problem than we have available 
to us.

Furthermore, many MTAs *do* understand Maildir, and most admins do as 
well; using our own queue-on-disk format means MTAs must access Mailman 
via LTMP, pipe invocation, or the like, and if there are issues with the 
queue the administrator likely must learn our queue-on-disk format.

Being able to deliver to mailman even if mailman isn't currently running 
strikes me as a potential win for some configurations.

Most of the maildir phenomena you have an issue with wouldn't even arise 
in the use case under discussion; a mail would enter maildir/new , 
mailman would suck it out, and that would be that; renaming wouldn't 
occur and the number of elements in the queue is unlikely to become 
large enough to pressure filesystem indexing schemes.

~ethan fremen


From jwblist3 at olympus.net  Fri Sep 29 19:23:48 2006
From: jwblist3 at olympus.net (John W. Baxter)
Date: Fri, 29 Sep 2006 10:23:48 -0700
Subject: [Mailman-Developers] [Mailman-Users]   OS X & Mailman & Python
In-Reply-To: <89EC65CC-0F75-4852-A712-45FD031242AC@python.org>
Message-ID: <C142A4B4.EDA4%jwblist3@olympus.net>

On 9/28/06 7:16 PM, "Barry Warsaw" <barry at python.org> wrote:

> If you look at the source, you'll see that the #! line is actually
> @PYTHON@ which gets substituted by configure at build time.  I forget
> exactly why, but the standard #! /usr/bin/env python invocation
> caused problems for people, so now we hardcode it via configure.

#! /usr/bin/env python
in an old-enough RedHat would have launched Python 1.5.2 when that version
was long dead (deceased, etc).  That's probably not the only example of the
reason for configure to select the Python to be used--essentially any
installation with multiple Pythons is subject to env picking the wrong one.

  --John



From brad at stop.mail-abuse.org  Fri Sep 29 19:45:29 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Fri, 29 Sep 2006 12:45:29 -0500
Subject: [Mailman-Developers] Incoming Queue format
In-Reply-To: <451D489B.7070100@mindlace.net>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<p06240500c140e36ee174@[10.0.1.4]> <451C90A5.8060607@mindlace.net>
	<p06240502c14251e87f6f@[10.0.1.4]> <451D489B.7070100@mindlace.net>
Message-ID: <p06240503c14306b7dfb4@[10.0.1.4]>

At 12:23 PM -0400 9/29/06, emf wrote:

>  Furthermore, many MTAs *do* understand Maildir,

MTAs should be sticking to the job of transmitting e-mail between 
themselves.  If they're spending any time mucking about with local 
mailbox formats, they're making a mistake.  That's a job for a Local 
Delivery Agent, and not an MTA.

Now, granted many mail system packages also include one or more 
sample LDAs in the tarball, either as an official part of the system 
or in some sort of contrib/ directory somewhere, but let's make sure 
that we're using the proper terms for the proper objects.

Therefore, by definition, MTA != LDA.

>                                                  and most admins
>  do as well;

If this discussion has taught me anything, it's that after all these 
years we still have virtually no one in this business that really 
does understand Maildir or any other mailbox format, although many 
claim to do so.

>              using our own queue-on-disk format means MTAs must
>  access Mailman via LTMP, pipe invocation, or the like, and if
>  there are issues with the queue the administrator likely must
>  learn our queue-on-disk format.

Our queue-on-disk format is already much simpler than Maildir, at 
least when it comes to directory structure, and the directory hashing 
schemes that I've been talking about have been around for many years. 
No new thought needs to be put into implementing them.

I even convinced Wietse that he should implement a lot of the same 
concepts, back when I first got involved in postfix in '98, when it 
was still being called VMailer.


Now, if you want to get outside of directory structure, our 
queue-on-disk format includes a lot of things that are 
Mailman-specific, such as creating message pickles, and I don't think 
that anyone is talking about getting rid of those aspects.

>  Most of the maildir phenomena you have an issue with
>  wouldn't even arise in the use case under discussion; a
>  mail would enter maildir/new , mailman would suck it out,
>  and that would be that; renaming wouldn't occur and the
>  number of elements in the queue is unlikely to become
>  large enough to pressure filesystem indexing schemes.

You really need to go back and review exactly how messages are 
created using Maildir.


With Maildir, when a message comes in, a temporary file is opened 
with truncate (with certain measures taken to try to ensure that the 
selected filename will be unique), and if that system call succeeds, 
then the system appends the incoming message and renames the file, 
before it ever closes it.

If that creat() system call fails, then there is already a file by 
that name, and the LDA has to try again.  This is how they "safely" 
create files on NFS, with an operation that is supposed to be atomic, 
and allows them to avoid file locking.

I'd have to check, but I think there are some more synchronous 
meta-data operations in here, too.  Certainly, every time you look to 
see if more messages have come in, you have to scan the entire 
directory, and you have to stat() each and every file, and if you 
want to pick up the message and move that somewhere else, you're 
going to have to do further synchronous meta-data operations that 
involve locking the entire directory structure while they are taking 
place.

Now, your application may not see those locking, scanning, and stat 
operations, you may simply see "move this file to another location", 
but the underlying filesystem code has to do a lot of work to support 
that.  And regardless of whether you're using locking or not, you 
still have race conditions that you have to code for -- or at least 
be aware of, because they are potential areas where you may have 
problems in the future.


If you don't read the comments that Mark Crispin has written about 
the weaknesses in Maildir, and you haven't read the code to see what 
it's actually doing, then I don't see how you can participate in this 
discussion.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From brad at stop.mail-abuse.org  Fri Sep 29 20:24:20 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Fri, 29 Sep 2006 13:24:20 -0500
Subject: [Mailman-Developers] Incoming Queue format
In-Reply-To: <451D489B.7070100@mindlace.net>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<p06240500c140e36ee174@[10.0.1.4]> <451C90A5.8060607@mindlace.net>
	<p06240502c14251e87f6f@[10.0.1.4]> <451D489B.7070100@mindlace.net>
Message-ID: <p06240500c1431365d867@[10.0.1.4]>

At 12:23 PM -0400 9/29/06, emf wrote:

>>  So your assumptions about what kinds of filesystems may
>>  or may not be appropriate are not necessarily going to
>>  coincide with the decisions that other people make, or
>>  the kinds of hardware and OS they may be forced to live
>>  with.
>
>  I don't disagree with this assertion, nor am I making
>  assumptions about what people get to live with.

Actually, that is precisely what you did.  You said that we shouldn't 
bother implementing something that XFS and ReiserFS would fix anyway, 
and that people who would be running Mailman would "obviously" choose 
to run the "best" filesystem for their application.

Your claims to the contrary are disingenuous, at best.

>  Most of the maildir phenomena you have an issue with
>  wouldn't even arise in the use case under discussion;

We're talking about maildir-mailbox-per-list.  So, all known issues 
with Maildir mailboxes would be applicable, because they would *be* 
Maildir mailboxes.  The fact that we would be using Maildir mailboxes 
as a method of handling incoming messages instead of having them 
written to a 7th edition mbox-format mailbox, is purely an 
application-level implementation detail.


That said, I'm done arguing with you and Barry.  You guys go do 
whatever you want.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From lists at yggdrasill.net  Fri Sep 29 21:35:55 2006
From: lists at yggdrasill.net (Jim)
Date: Fri, 29 Sep 2006 13:35:55 -0600 (MDT)
Subject: [Mailman-Developers] Incoming Queue format
In-Reply-To: <p06240503c14306b7dfb4@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp> <4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<p06240500c140e36ee174@[10.0.1.4]> <451C90A5.8060607@mindlace.net>
	<p06240502c14251e87f6f@[10.0.1.4]> <451D489B.7070100@mindlace.net>
	<p06240503c14306b7dfb4@[10.0.1.4]>
Message-ID: <Pine.LNX.4.64.0609291256220.6084@localhost>

On Fri, 29 Sep 2006, Brad Knowles wrote:

> If you don't read the comments that Mark Crispin has written about 
> the weaknesses in Maildir, and you haven't read the code to see what 
> it's actually doing, then I don't see how you can participate in this 
> discussion.

I did read through both of the references you provided, more out of 
curiosity than any particular interest in whether or not Maildir is 
used to address the issues under discussion. To be honest, I don't see 
how either reference is particularly relevant. The Christenson paper is 
a nine year old architecture document that doesn't appear to make any 
mention of Maildir. At best it seems to imply that there are perhaps 
some better ways to solve some of the problems Maildir was designed to 
address. Crispin's comments appear to specifically address Maildir 
inefficiencies associated with management by an IMAP server. These 
inefficiencies are mostly (entirely?) unrelated to the delivery and 
one-time read scenario that is under discussion for Mailman (assuming I 
correctly understand the proposed feature). And to be fair, even the 
nature and severity of these Maildir inefficiencies is open to debate. 
See for example http://www.courier-mta.org/mbox-vs-maildir/#theend.

And since when is reading all source code for all programs, system
calls, etc. associated with a Mailman function a requirement for making
a comment or expressing an opinion on this list? If by chance you feel
that you have such insight into the source code, share it (as you have)
and leave it at that. Don't try to exclude people out of hand and scare
off others who might have useful input. Among other things a list is a
place for learning and gaining a deeper understanding of the issues
associated with the topic.

Jim

From carson at taltos.org  Sat Sep 30 00:37:54 2006
From: carson at taltos.org (Carson Gaspar)
Date: Fri, 29 Sep 2006 15:37:54 -0700
Subject: [Mailman-Developers] Incoming Queue format
In-Reply-To: <p06240502c14251e87f6f@[10.0.1.4]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<p06240500c140e36ee174@[10.0.1.4]> <451C90A5.8060607@mindlace.net>
	<p06240502c14251e87f6f@[10.0.1.4]>
Message-ID: <49E2D4D28748BA3B54803E51@[192.168.1.2]>

--On Thursday, September 28, 2006 11:39 PM -0500 Brad Knowles 
<brad at stop.mail-abuse.org> wrote:

> At 11:19 PM -0400 9/28/06, emf wrote:
>
>>  It seems to me like anyone likely to end up with a huge enough incoming
>>  mailman queue to care about Maildir's inefficiencies would also be able
>>  to put a sensible filesystem underneath it.
>
> That may simply not be possible.  Moreover, I have some real
> operational problems with both XFS on Linux and ReiserFS, and I would
> not run a production mail system using them.  Maybe IBM's JFS, if I
> were forced to run a production mail system on Linux at all, but
> certainly not XFS or ReiserFS.

Brad, if your _incoming_ queue is so big that you have to worry, your 
servers are woefully underspec'd. I understand your dislike for Maildir, 
but for the _incoming_ queue case, it just shouldn't matter. If you can 
provide a detailed use case where it matters for the _incoming_ queue, 
please do so.

-- 
Carson

From brad at stop.mail-abuse.org  Sat Sep 30 03:52:00 2006
From: brad at stop.mail-abuse.org (Brad Knowles)
Date: Fri, 29 Sep 2006 20:52:00 -0500
Subject: [Mailman-Developers] Incoming Queue format
In-Reply-To: <49E2D4D28748BA3B54803E51@[192.168.1.2]>
References: <E1GRlI4-0006O0-FF@sc8-pr-svn1.sourceforge.net>
	<4519E35A.6070906@is.kochi-u.ac.jp>	<4519E971.8070202@nleaudio.com>
	<451A033B.7090505@Newfield.org>
	<9B0599C0-D151-4943-B55E-A8A7F90DE660@python.org>
	<p06240500c140e36ee174@[10.0.1.4]> <451C90A5.8060607@mindlace.net>
	<p06240502c14251e87f6f@[10.0.1.4]>
	<49E2D4D28748BA3B54803E51@[192.168.1.2]>
Message-ID: <p06240506c1437cdfe99c@[10.0.1.11]>

At 3:37 PM -0700 9/29/06, Carson Gaspar wrote:

>  Brad, if your _incoming_ queue is so big that you have to worry, your
>  servers are woefully underspec'd.

That may or may not be true, but that doesn't make the problem 
magically go away.

>                                                           If you can
>provide a detailed use case where it matters for the _incoming_ queue,
>please do so.

Basically, it's any site that has one or more lists that have 
relatively high levels of incoming traffic, and which run into 
synchronous meta-data bottleneck issues.  This could be a lower-end 
machine with a filesystem that does not perform as well as could be, 
and a more moderately sized list.  Or, this could be a site where 
they've already thrown the biggest/best configured machine at the 
problem that they can, and yet they're still seeing problems.

On the high end, from the reports we got after Apple upgraded the 
system for lists.apple.com, I don't think they're too likely to have 
these kinds of problems.  But the FreeBSD folks might already be 
there, and I imagine that the SourceForge people are definitely there.

But I'm sure there are more relatively smaller lists running on 
relatively smaller/less well configured hardware, and which are 
running into the same kinds of problems.


All this aside, it's clear that I'm not going to convince anyone here 
of anything, so I'm just going to unsubscribe from the list and I'll 
ask everyone to make sure that they do not include me on any further 
messages on this subject.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

  Founding Individual Sponsor of LOPSA.  See <http://www.lopsa.org/>.

From sveniu at ifi.uio.no  Sat Sep 30 10:53:36 2006
From: sveniu at ifi.uio.no (Sven Ulland)
Date: Sat, 30 Sep 2006 10:53:36 +0200
Subject: [Mailman-Developers] Debugging mailman (resolving inline
	signatureattachments)
In-Reply-To: <PC1760200609290801560937368f2601@msapiro>
References: <PC1760200609290801560937368f2601@msapiro>
Message-ID: <451E3090.90908@ifi.uio.no>

Mark Sapiro wrote:
> Sven Ingebrigt Ulland wrote:
> 
>> Hello there. How would I go about debugging mailman, by --
>> for example -- polluting the .py libraries with print or
>> assert statements? If I do that (which seems like a pretty
>> thoughtless idea), the output would end up .. in /dev/null?
>> It would probably be better to write to file in stead?
>>
> <snip>
>> I'd really like to debug that code passage to verify what
>> variables are screwed up, and which are not.
>>
>> Any tips?
> 
> 
> Output to stderr from the queue runners (the running Mailman processes)
> is copied to Mailman's error log so you can print to sys.stderr and
> the output goes to the error log.
> 
> You can also use Mailman.Logging.Syslog.syslog() calls to write to a
> 'debug' or other log (which will be created if necessary).

Thanks, that sounds perfect.

s