From mk2s at digitalcommute.com  Fri May  2 04:21:40 2008
From: mk2s at digitalcommute.com (mk2s at digitalcommute.com)
Date: Thu, 01 May 2008 22:21:40 -0400
Subject: [Mailman-Developers] How to get started with the REST server(was
	Re: Hello!! Help	Needed)
In-Reply-To: <f0084680804301450q1aa509a5u80cea1ddd47c8d01@mail.gmail.com>
References: <f0084680804301450q1aa509a5u80cea1ddd47c8d01@mail.gmail.com>
Message-ID: <20080501222140.typb0gofko8kowgs@mail.matrixgroup.net>

This is a quick response to "How do you get started with the  
restserver?" question.   
https://code.launchpad.net/~mk2s/mailman/restserver

All of this is very much work in progress, but server.py is probably  
where you want to start.  There isn't much there, mostly because the  
routes package takes care of the disptach.

I run my server as follows:
$ cd ~/mailman22runtime
$ PYTHONPATH=. python Mailman/rest/server.py

listConroller.py is the only controller currently implemented it  
responds to a get request like the following:
$ lynx http://localhost:8000/lists/makitest
where makitest is the name of my test list that already exists.  The  
response is a HTML representation of the attributes of the list.

it also responds to a request like:
lynx http://localhost:8000/lists/
with a list of all the lists

the controller uses the .html file as templates for returning HTML  
representation responses.  Currently I have the response hardwired to  
HTML, but in theory the json representation should also work.

also see:
http://www.makotemplates.org/
http://routes.groovie.org/manual.html



Quoting aman parnami <amanparnami at gmail.com>:
> Hi Mike,
> I was able to get your code from launchpad after some manipulations
> and lots of patience. Now onwards downloads will be smooth for me.
>
> I just swept through the directory structure of restserver and got a
> little confused. It would be grateful if you could tell me the
> starting point and
> if possible (will be highly appreciated) the order in which files
> should be read to understand them better and sooner.
>
> Thanks,
> --
> AMAN PARNAMI
> CSE DEPTT
> Indian Institute of Technology Bombay
>



----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.



From iane at sussex.ac.uk  Fri May  2 16:59:31 2008
From: iane at sussex.ac.uk (Ian Eiloart)
Date: Fri, 02 May 2008 15:59:31 +0100
Subject: [Mailman-Developers] Intern
Message-ID: <51043A076CCD3E8087F74D81@lewes.staff.uscs.susx.ac.uk>

Hi,

I have an intern coming to work for a couple of months in the summer. 
Neither of us will be python experts, but part of the point is to learn, 
right?

I'd like him to work on some aspects of Mailman 3.

I plan to get LMTP working to my taste, by (a) rejecting messages for 
non-existent lists at RCTP time, and (b) rejecting messages from unwanted 
sender addresses at RCPT time. I Think (a) will be easy to achieve, and (b) 
only slightly more challenging.

So, I'm after ideas for what to do next. I'm vaguely thinking about (c) 
integration (which will probably begin from working up a specification for 
exactly what that means), and possibly (d) user interface design for end 
users and list admins.

Any suggestions for (c) and (d) or alternative areas that need some focus 
would be useful.

Also, can you give me pointers to a good place to learn about doctest, 
please?

Finally, is there any overview documentation for developers new to the 
Mailman architecture?

-- 
Ian Eiloart
IT Services, University of Sussex
x3148

From stephen at xemacs.org  Fri May  2 20:43:02 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 03 May 2008 03:43:02 +0900
Subject: [Mailman-Developers]  Intern
In-Reply-To: <51043A076CCD3E8087F74D81@lewes.staff.uscs.susx.ac.uk>
References: <51043A076CCD3E8087F74D81@lewes.staff.uscs.susx.ac.uk>
Message-ID: <87od7ownrd.fsf@uwakimon.sk.tsukuba.ac.jp>

Ian Eiloart writes:

 > Also, can you give me pointers to a good place to learn about doctest, 
 > please?

In public, please!  Inquiring minds want to know ...

From barry at list.org  Sat May  3 04:19:42 2008
From: barry at list.org (Barry Warsaw)
Date: Fri, 2 May 2008 22:19:42 -0400
Subject: [Mailman-Developers] Important Mailman 2.1.9 to 2.1.10 upgrade
	note.
In-Reply-To: <4814CA2B.7050203@msapiro.net>
References: <PC1760200804221144050468c671b4f9@msapiro>
	<C5C4A17B-EF8A-4EC9-9F6C-5A43F2B48D49@python.org>
	<4814CA2B.7050203@msapiro.net>
Message-ID: <C7D7344D-87E4-4895-8715-AB4B7C9A413D@list.org>

On Apr 27, 2008, at 2:47 PM, Mark Sapiro wrote:
>
> I would appreciate feedback on the 1074 and 1075 revisions from those
> interested.

Mark, these changes look pretty good.  I have some minor comments  
about the formatting but that's it.

-Barry

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: diff.txt
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20080502/881551b9/attachment.txt>
-------------- next part --------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20080502/881551b9/attachment.pgp>

From mark at msapiro.net  Sat May  3 18:15:34 2008
From: mark at msapiro.net (Mark Sapiro)
Date: Sat, 03 May 2008 09:15:34 -0700
Subject: [Mailman-Developers] Important Mailman 2.1.9 to 2.1.10 upgrade
 note.
In-Reply-To: <C7D7344D-87E4-4895-8715-AB4B7C9A413D@list.org>
References: <PC1760200804221144050468c671b4f9@msapiro>
	<C5C4A17B-EF8A-4EC9-9F6C-5A43F2B48D49@python.org>
	<4814CA2B.7050203@msapiro.net>
	<C7D7344D-87E4-4895-8715-AB4B7C9A413D@list.org>
Message-ID: <481C8FA6.9080709@msapiro.net>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Barry Warsaw wrote:
| On Apr 27, 2008, at 2:47 PM, Mark Sapiro wrote:
|>
|> I would appreciate feedback on the 1074 and 1075 revisions from those
|> interested.
|
| Mark, these changes look pretty good.  I have some minor comments about
| the formatting but that's it.
|
| -Barry
|
|
| ------------------------------------------------------------------------
[...]
|> > +                    preserve=True
|
| There should be spaces around the = sign.


Agreed. I must have been thinking ahead (or not thinking) to it's
ultimate use as an argument in

~                self._switchboard.finish(filebase, preserve=preserve)

[...]
|> > +                    preserve=False
|
| Same here.


Yep.


[...]
| +    if (mlist._internal_name <> 'en' and mlist._internal_name not in
| +            os.listdir(mm_cfg.MESSAGES_DIR)):
|
| I would format this code like this:
|
|     if (mlist._internal_name <> 'en' and
|         mlist._internal_name not in os.listdir(mm_cfg.MESSAGES_DIR)):
|         # It's okay to move the template.


Yes, that's clearly better. It always helps to have someone else review
stuff. Thanks.

BTW, the above stuff all comes from changes I made on the 2.1 branch at
revs 1072 and 1074. Did anyone look at the cull_bad_shunt cron and
bin/update change at revs 1074 and 1075 of
<https://code.launchpad.net/~msapiro/mailman/cull_bad_shunt>?

- --
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)

iD8DBQFIHI+mVVuXXpU7hpMRAt3tAJ0TIj6tBaoK2n5IXqxDWC5COT0tQwCggy2h
ITXy+VKlxj/Gd8HxKwuqYnc=
=DVJz
-----END PGP SIGNATURE-----

From mk2s at digitalcommute.com  Tue May  6 06:53:40 2008
From: mk2s at digitalcommute.com (mk2s at digitalcommute.com)
Date: Tue, 06 May 2008 00:53:40 -0400
Subject: [Mailman-Developers] Using REST Interface and WADL
In-Reply-To: <44853CF4-1B83-4527-B62E-2051A0D5FF27@list.org>
References: <E6C6D0F2-ED02-41FD-99DD-216AA4B25959@list.org>
	<BF8C9162-B172-4735-B3E0-D1E07BF2C726@matrixgroup.net>
	<9DF8F405-6E17-4B4B-93F0-731B8B39A472@digitalcommute.com>
	<44853CF4-1B83-4527-B62E-2051A0D5FF27@list.org>
Message-ID: <20080506005340.bbbscgs3kw08gwc8@mail.matrixgroup.net>

Quoting Barry Warsaw <barry at list.org>:

> On Apr 13, 2008, at 5:35 PM, Maki Kato wrote:
[stuff deleted]
>>    - Documentation.  I'm hoping the xHTML representation will have   
>> a way of being able to serve reasonable documentation.  So one   
>> could point their browser to the REST interface and start browsing   
>> to learn how the server is implemented.  We may want to distribute   
>> a static set of HTML pages that has documentation and test(dummy)   
>> data.
>
> Leonard Richardson is very hot on WADL.  Will we be able to generate
> that automatically or does that not have a place in the architecture
> you're thinking about?

I went to read up some more on WADL and ran into this:
http://bitworking.org/news/193/Do-we-need-WADL

I don't have any plans(yet) on auto generating the WADL, or did you  
mean auto generating the documentation from the WADL file?  Are there  
tools that do that?  I didn't find any such.

Currently I'm returning human readable text containing something like  
documentation when you request the xHTML representation.  I've seen  
this done before and figured it was a pretty good idea.  It is of no  
use of course if you want to auto generate a client, as you would from  
a description file.

Anyone else have any opinions or experiences with WADL?

....maki....


----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.



From mk2s at digitalcommute.com  Tue May  6 06:58:27 2008
From: mk2s at digitalcommute.com (mk2s at digitalcommute.com)
Date: Tue, 06 May 2008 00:58:27 -0400
Subject: [Mailman-Developers] What do I need to do to add a package
	dependency in MM2.2?
Message-ID: <20080506005827.tbktkg7yqs880ggc@mail.matrixgroup.net>

I'm doing some work in mm2.2 and wanted to add some python package  
dependencies like routes, lxml, and simplejason.  How do I go about  
declaring these dependencies so that the dependent packages will be  
installed for the end user who wants to run mm2.2?

Can someone point me in the direction of the stuff that I need to read?

....maki....


----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.



From barry at list.org  Tue May  6 20:23:53 2008
From: barry at list.org (Barry Warsaw)
Date: Tue, 6 May 2008 14:23:53 -0400
Subject: [Mailman-Developers] Intern
In-Reply-To: <51043A076CCD3E8087F74D81@lewes.staff.uscs.susx.ac.uk>
References: <51043A076CCD3E8087F74D81@lewes.staff.uscs.susx.ac.uk>
Message-ID: <9BFAD07F-9EBA-4A36-9129-2F16629342D4@list.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On May 2, 2008, at 10:59 AM, Ian Eiloart wrote:

> I have an intern coming to work for a couple of months in the  
> summer. Neither of us will be python experts, but part of the point  
> is to learn, right?

Absolutely!

> I'd like him to work on some aspects of Mailman 3.
>
> I plan to get LMTP working to my taste, by (a) rejecting messages  
> for non-existent lists at RCTP time, and (b) rejecting messages from  
> unwanted sender addresses at RCPT time. I Think (a) will be easy to  
> achieve, and (b) only slightly more challenging.

This sounds great.  With (b) there could be multiple list-specific  
policies, so we should make this pluggable.

> So, I'm after ideas for what to do next. I'm vaguely thinking about  
> (c) integration (which will probably begin from working up a  
> specification for exactly what that means), and possibly (d) user  
> interface design for end users and list admins.
>
> Any suggestions for (c) and (d) or alternative areas that need some  
> focus would be useful.

By user interface design do you mean the web u/i?  None of that really  
works in MM3 right now.  Some of the other work happening in the REST  
interface may allow us to separate the web u/i too.

> Also, can you give me pointers to a good place to learn about  
> doctest, please?
>
> Finally, is there any overview documentation for developers new to  
> the Mailman architecture?

Actually these two subjects go hand-in-hand, although there should be  
better overview documentation.

I'm a big fan of separate file doctests (as opposed to docstring  
doctests which I'm not a fan of at all).  The important thing to  
remember is that doctests are documentation first and tests second.   
Or another way of saying it is that doctests are testable documentation.

This is important because you really want to take care to write the  
narrative of a doctest in a clear and concise way.  I'm experimenting  
with this idea that all tests can be done using doctest, with two  
different audiences in mind.  First is the developer (or user) who  
wants to understand a piece of the system, and the second is maybe a  
maintainer who wants to write comprehensive unit tests with total  
coverage.  I don't know if I can achieve "all tests are doctests", but  
I'm trying.

Although it's a bit centered on docstring doctests (which I don't  
like), the Python doctest module has some good background: http://www.python.org/doc/current/lib/module-doctest.html

 From there, search the Mailman 3 tree for 'doc/' subdirectories.    
All .txt files in those directories are doctests.  There's a little  
bit of harness hooking in mailman/tests/test_documentation.py,  
including the doctest flags that I think are the most useful.  A good  
(well, recent ;) example of a doctest that I like is mailman/doc/ 
chains.txt.

I think it's better to have fewer magical globs in the doctest harness  
and do most of the work, including imports in the doctest.  It's just  
a better way to explain how a particular subsystem works or how you  
would use it.

Use ellipsis liberally but judiciously to improve readability.  Always  
be as explicit as possible in showing return values.  E.g. it's better  
to have something like this:

     >>> chain.name
     'reject'

than this:

     >>> chain.name == 'reject'
     True

because when the latter fails, it's more difficult to figure the  
problem out.  The one common exception is in printing the value None  
because None doesn't show up in the interactive prompt.  E.g. instead of

     >>> chain.foobar = None
     >>> chain.foobar
     >>> chain.baz = 7

do this:

     >>> chain.foobar = None
     >>> print chain.foobar
     None
     >>> chain.baz = 7

I'm beginning to think that its better to print strings than return  
them because ascii strs and unicodes print with the same value, but  
have different reprs, and the u-prefixes are annoying.

Does any of that help?  You should definitely look at the Mailman 3  
code and I'll be happy to answer any more questions you might have.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iEYEARECAAYFAkggojkACgkQ2YZpQepbvXHOSACgnUcbW7CMi9mvLXfWgVAxzimv
BhEAnj439jpogZTMm+h25uzq7pDE5HWq
=evuZ
-----END PGP SIGNATURE-----

From iane at sussex.ac.uk  Wed May  7 11:23:23 2008
From: iane at sussex.ac.uk (Ian Eiloart)
Date: Wed, 07 May 2008 10:23:23 +0100
Subject: [Mailman-Developers] Intern
In-Reply-To: <9BFAD07F-9EBA-4A36-9129-2F16629342D4@list.org>
References: <51043A076CCD3E8087F74D81@lewes.staff.uscs.susx.ac.uk>
	<9BFAD07F-9EBA-4A36-9129-2F16629342D4@list.org>
Message-ID: <BB46A1A7669FAC82DE060624@lewes.staff.uscs.susx.ac.uk>



--On 6 May 2008 14:23:53 -0400 Barry Warsaw <barry at list.org> wrote:

>
>> I plan to get LMTP working to my taste, by (a) rejecting messages
>> for non-existent lists at RCTP time, and (b) rejecting messages from
>> unwanted sender addresses at RCPT time. I Think (a) will be easy to
>> achieve, and (b) only slightly more challenging.
>
> This sounds great.  With (b) there could be multiple list-specific
> policies, so we should make this pluggable.
>

The intention is to get the policy from the list. The simplest way to do it 
would be to treat "reject" as an SMTP time reject, rather than a bounce. At 
my site, at least, I would not offer "bounce" as a choice if smtp rejection 
is available.

Indeed, we currently recommend that list owners don't use "reject".



-- 
Ian Eiloart
IT Services, University of Sussex
x3148

From iane at sussex.ac.uk  Wed May  7 11:37:44 2008
From: iane at sussex.ac.uk (Ian Eiloart)
Date: Wed, 07 May 2008 10:37:44 +0100
Subject: [Mailman-Developers] Intern
In-Reply-To: <9BFAD07F-9EBA-4A36-9129-2F16629342D4@list.org>
References: <51043A076CCD3E8087F74D81@lewes.staff.uscs.susx.ac.uk>
	<9BFAD07F-9EBA-4A36-9129-2F16629342D4@list.org>
Message-ID: <E76526D88CE123C526247A7E@lewes.staff.uscs.susx.ac.uk>



--On 6 May 2008 14:23:53 -0400 Barry Warsaw <barry at list.org> wrote:

>
> Does any of that help?  You should definitely look at the Mailman 3 code
> and I'll be happy to answer any more questions you might have.
>

I'm sure all of that will help when I get into the kitchen, but I'm having 
problems finding the right street. I thought I'd be able to leap right in, 
but the more I look, the more I find that I need a map. Perhaps something 
describing the lifecycle of a typical email, and which bits of top level 
code are responsible for handling it at which stage. Or maybe just a 
recommended order for reading the doctests would do the job.

With that understanding, I can follow the API calls to get at the detail.


-- 
Ian Eiloart
IT Services, University of Sussex
x3148

From ecsd at transpacific.net  Sun May  4 00:31:26 2008
From: ecsd at transpacific.net (Eric Dynamic)
Date: Sat, 03 May 2008 15:31:26 -0700
Subject: [Mailman-Developers] Errors in 2.1.10 and 2.1.10.rc1
Message-ID: <481CE7BE.7050806@transpacific.net>

The error arose in 2.1.10rc1 and so I tried to downgrade the version to 
a stable one, 2.1.10.
I get the same error:

Updating mailing list: academicchoice
Traceback (most recent call last):
  File "./update", line 789, in <module>
    errors = main()
  File "./update", line 679, in main
    errors = errors + dolist(listname)
  File "./update", line 192, in dolist
    mlist = MailList.MailList(listname, lock=0)
  File "/usr/local/mailman/Mailman/MailList.py", line 130, in __init__
    self.Load()
  File "/usr/local/mailman/Mailman/MailList.py", line 632, in Load
    dict, e = self.__load(file)
  File "/usr/local/mailman/Mailman/MailList.py", line 605, in __load
    dict = loadfunc(fp)
ImportError: No module named e

I had an older version (maybe 2.1.5) on machine A which was being 
blocked by AT&T,
so I installed the later version 2.1.10rc1 on machine B and copied the
"lists" and "archives" contents over to machine B. The system continues
to produce references to machine A in the web code, and finally I see I
have to run

withlist -l -r fix_urls

but that fails with the same "No module named e" error. We're running
python 2.5 on FreeBSD 6.2. I desperately need to fix the lists as they are
active and people are hosed until I can. I would make the changes manually
if I knew what "fix_urls" does; I changed the domain name inside the files
lists/*/config.pck but that didn't help. I'm not using any virtual-host 
stuff.

From mark at msapiro.net  Wed May  7 23:32:33 2008
From: mark at msapiro.net (Mark Sapiro)
Date: Wed, 7 May 2008 14:32:33 -0700
Subject: [Mailman-Developers] Errors in 2.1.10 and 2.1.10.rc1
In-Reply-To: <481CE7BE.7050806@transpacific.net>
Message-ID: <PC18302008050714323309889f973bbd@msapiro>

Eric Dynamic wrote to mailman-developers and also similarly to
mailman-users at
<http://mail.python.org/pipermail/mailman-users/2008-May/061542.html>:

>The error arose in 2.1.10rc1 and so I tried to downgrade the version to 
>a stable one, 2.1.10.


2.1.10 is not a downgrade from 2.1.10rc1.


>I get the same error:
>
>Updating mailing list: academicchoice
>Traceback (most recent call last):
>  File "./update", line 789, in <module>
>    errors = main()
>  File "./update", line 679, in main
>    errors = errors + dolist(listname)
>  File "./update", line 192, in dolist
>    mlist = MailList.MailList(listname, lock=0)
>  File "/usr/local/mailman/Mailman/MailList.py", line 130, in __init__
>    self.Load()
>  File "/usr/local/mailman/Mailman/MailList.py", line 632, in Load
>    dict, e = self.__load(file)
>  File "/usr/local/mailman/Mailman/MailList.py", line 605, in __load
>    dict = loadfunc(fp)
>ImportError: No module named e
>
>I had an older version (maybe 2.1.5) on machine A which was being 
>blocked by AT&T,
>so I installed the later version 2.1.10rc1 on machine B and copied the
>"lists" and "archives" contents over to machine B. The system continues
>to produce references to machine A in the web code, and finally I see I
>have to run
>
>withlist -l -r fix_urls
>
>but that fails with the same "No module named e" error. We're running
>python 2.5 on FreeBSD 6.2. I desperately need to fix the lists as they are
>active and people are hosed until I can. I would make the changes manually
>if I knew what "fix_urls" does; I changed the domain name inside the files
>lists/*/config.pck but that didn't help. I'm not using any virtual-host 
>stuff.


As indicated in my reply on mailman-users and subsequent off-list
correspondence, this is cause by some kind of corruption in the list's
config.pck files.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From f.schubert at espresto.com  Thu May  8 14:32:37 2008
From: f.schubert at espresto.com (Frank Schubert)
Date: Thu, 08 May 2008 14:32:37 +0200
Subject: [Mailman-Developers] webui templates or not?
Message-ID: <4822F2E5.1040409@espresto.com>

Hi all,

the plans in
http://wiki.list.org/display/DEV/TemplatingNotes
and
http://wiki.list.org/display/DEV/StyledPages
sound great, but mean a lot of work.

For example there is options.py with functions for rendering output named
options_page and loginpage.

options_page supports the old template mechanism using options.html
However loginpage has hard-coded tables including "BGCOLOR" and such ugly things.

I don't know if someone is still working on the new template mechanism and I couldn't find any changes on options.py either.
(What I think is fundamentally necessary for a complete template mechanism.)
(I looked in http://bazaar.launchpad.net/~mailman-coders/mailman/2.2/files and http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/files)

So my plan is:
For a customer I want to customize more than the standard pages (for example the function loginpage from options.py).
I will use the old template system (like in function options_page in options.py), using new template names like <SCRIPT>_<FUNCTION>.html
As I think this is useful for others it could maybe be merged in mailman.

So what do you think, is it a waste of time because code for the old template mechanism will never get merged?

I'm looking forward to your suggestions.

bye,
Frank

-- 
Frank Schubert
Systemadministration
EsPresto AG
Breite Str. 30-31
10178 Berlin/Germany
Tel: +49.(0)30.90 226.750
Fax: +49.(0)30.90 226.760
f.schubert at espresto.com

HRB 77554 AG - Berlin-Charlottenburg
Vorstand: Alexander Biersack, Maya Biersack, Peter Biersack
Vorsitzender des Aufsichtsrats: Oli Kai Paulus

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20080508/682acc66/attachment.pgp>

From jdavies at ubuntu.com  Sun May 11 17:53:15 2008
From: jdavies at ubuntu.com (Jonathan Patrick Davies)
Date: Sun, 11 May 2008 17:53:15 +0200
Subject: [Mailman-Developers] Implementing multiple admin/mod passwords into
	3.x
Message-ID: <1210521195.27595.0.camel@halmos>

Hey there everyone,

I am currently looking into implementing multiple administrator or
moderator passwords into the 3.x series of Mailman, and would like some
guidance as to what parts of the code I should poke around with.

When I say multiple passwords, I mean that an administrator of a mailing
list can log into the admin interface with his password for the mailing
list - instead of having one password to rule them all. Similar to the
way moderators on forum software work.

For example; I would like to be able to say:

    "jdavies is now a moderator and may log in using his list password"

Also, I understand that there may be situations where the
administrator/moderator may not wish to follow the threads in the list
but would like to perform the admin/mod role nonetheless - in this case
he/she would not have a list password to log in with. Does anyone know
where I could possibly implement a 'Don't send me email from this list'
option?

Right now, I'm looking into 'def AuthContextInfo(...)' in
3.0/mailman/SecurityManager.py - is this a good place to start working
on this?

Also, that snippet of code refers to getMemberPassword(user) - however I
cannot find this function defined in any part of the 3.x code. While in
2.2, I have found it under '2.2/Mailman/OldStyleMemberships.py'. Does
someone know what its status in 3.x is?

Cheers,
Jonathan
-- 
Ubuntu - Linux for human beings | www.ubuntu.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 827 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20080511/6fbd849e/attachment.pgp>

From barry at python.org  Mon May 12 04:34:30 2008
From: barry at python.org (Barry Warsaw)
Date: Sun, 11 May 2008 22:34:30 -0400
Subject: [Mailman-Developers] GNU Mailman Site Redesign
In-Reply-To: <20080421124735.GA6803@amk-desktop.matrixgroup.net>
References: <42a95f60803102010i47ea00ft4af82f46b4cdf1e8@mail.gmail.com>
	<901387E0-D87B-427F-9C6A-1018D8B0F42B@list.org>
	<20080409115957.GA5988@amk-desktop.matrixgroup.net>
	<ABA524F3-9489-4A69-A5F5-AFD385726F05@list.org>
	<20080421124735.GA6803@amk-desktop.matrixgroup.net>
Message-ID: <7A065EA3-340B-458D-93DA-98814287209B@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Apr 21, 2008, at 8:47 AM, A.M. Kuchling wrote:

> On Sun, Apr 20, 2008 at 09:01:38PM -0400, Barry Warsaw wrote:
>> Confluence does support getting a dump of its pages in XML format,
>> which I've done.  I'm leery of publishing the url widely though
>> because I haven't had a chance to look through it, so I don't know if
>> there is any sensitive information in there.  Andrew, I'll send you
>> the url if you want to take a look.
>
> I've looked through it and it's straightforward to pull out the text
> of wiki pages; the complicated task would be converting them from
> Confluence's wiki markup to MoinMoin.  The dump does include a list of
> users, their e-mail addresses, and hashed versions of their passwords,
> so you shouldn't post it publicly.
>
> Do you want me to actually start working on this conversion, or are
> you still deciding whether to use some alternative?  It would be a
> good impetus to make me upgrade python.org to MoinMoin 1.6.  (No rush
> here; it would probably be a few weeks before the upgrade can be
> completed because I can't work on it this week.)

Hi Andrew, thanks for taking a look at this.  It's not a high priority  
for me, but I know the FSF is interested in this.

One question though (for Terri).  I know that she was thinking about  
migrating the documentation to the wiki, and using the PDF export  
feature of Confluence to provide hard copy.  Is this something that we  
can do with Moin?

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iQCVAwUBSCestnEjvBPtnXfVAQIk3gP/fZXMLDt33W5ZD6EBN1aH7Dg8G8ro0Q9d
ixUzm/K7zlTUKFhrb+YM3DbwiNS3UsBDpS6oZgvKZ7Rl/dKwl0QyehCrdEnvK7ue
0aO8Qg/eGqs2y4vPeJzqSKznmXIitRfhwQjzrmDF8Kz9ZLEFLgTEoR2medyzkDlG
pgielvIGp94=
=M+rU
-----END PGP SIGNATURE-----

From amk at amk.ca  Mon May 12 14:09:01 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Mon, 12 May 2008 08:09:01 -0400
Subject: [Mailman-Developers] GNU Mailman Site Redesign
In-Reply-To: <7A065EA3-340B-458D-93DA-98814287209B@python.org>
References: <42a95f60803102010i47ea00ft4af82f46b4cdf1e8@mail.gmail.com>
	<901387E0-D87B-427F-9C6A-1018D8B0F42B@list.org>
	<20080409115957.GA5988@amk-desktop.matrixgroup.net>
	<ABA524F3-9489-4A69-A5F5-AFD385726F05@list.org>
	<20080421124735.GA6803@amk-desktop.matrixgroup.net>
	<7A065EA3-340B-458D-93DA-98814287209B@python.org>
Message-ID: <20080512120901.GA6014@amk-desktop.matrixgroup.net>

On Sun, May 11, 2008 at 10:34:30PM -0400, Barry Warsaw wrote:
> Hi Andrew, thanks for taking a look at this.  It's not a high priority  
> for me, but I know the FSF is interested in this.
 
Noted; I haven't been looking any further at the task, and don't
expect to have time for this at all until after Memorial Day (the
weekend of May 24-25).

> One question though (for Terri).  I know that she was thinking about  
> migrating the documentation to the wiki, and using the PDF export  
> feature of Confluence to provide hard copy.  Is this something that we  
> can do with Moin?

Out-of-the-box Moin doesn't have such a feature, but there's at least
one plug-in for it <http://moinmo.in/ActionMarket/CreatePdfDocument>,
so it would be possible to set up PDF exports.

--amk


From cite at incertum.net  Mon May 12 14:51:35 2008
From: cite at incertum.net (Stefan =?utf-8?Q?F=C3=B6rster?=)
Date: Mon, 12 May 2008 14:51:35 +0200
Subject: [Mailman-Developers] Regarding Handlers/SMTPDirect.py and "chunkify"
Message-ID: <20080512125134.GA23014@mail.incertum.net>

Hello there,

I don't really know how to start this topic, I'm fairly new to Mailman
and I don't speak anything but the most basic Python, but I am unsure
whether the function "def chunkify(recips, chunksize):" which is
defined in "Mailman/Handlers/SMTPDirect.py" might not have an adverse
effect on mail delivery performance.

The problem is: At the moment, I'm looking at a specific setup without
knowing anything about the "historical" development of "chunkify".
There is a comment which mentions an initial suggestion made by Chuq
Von Rospach and further improvements made by Barry Warsaw.
Unfortunately, I was unable to find these original discussion, and
frankly, without knowing why "chunkify" is implemented the way it is,
I don't think I'm qualified to discuss my concern (which involves VERP
delivery using a specific MTAs VERP implementation and a highly I/O
saturated mail server) on this list.

What I found out from the archives is that "chunkfiy" has two
purposes:

1. Make sure that not more than mm_cfg.SMTP_MAX_RCPTS are passed to
the MTA in a single transaction.
2. To improve delivery performance by grouping destinations which will
prevent messages which are addressed to dead/congested destinations
blocking delivery of messages to working destinations.

Is this assumption of mine basically correct? If it is and if I didn't
miss any important parts, I would like to point out a specific
scenario in which the way "chunkify" is implemented and the way in
which it is called based on delivery style (verp, bulk) and the
setting of mm_cfg.SMTP_MAX_RCPTS might actually worsen overall mail
delivery performance.

If I am totally wrong and "chunkfiy" serves a totally different
purpose, please feel free to ignore this posting.


Cheers
Stefan

From mark at msapiro.net  Mon May 12 19:49:41 2008
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 12 May 2008 10:49:41 -0700
Subject: [Mailman-Developers] Regarding Handlers/SMTPDirect.py and
	"chunkify"
In-Reply-To: <20080512125134.GA23014@mail.incertum.net>
Message-ID: <PC1830200805121049410140270d06f5@msapiro>

Stefan F?rster wrote:
>
>I don't really know how to start this topic, I'm fairly new to Mailman
>and I don't speak anything but the most basic Python, but I am unsure
>whether the function "def chunkify(recips, chunksize):" which is
>defined in "Mailman/Handlers/SMTPDirect.py" might not have an adverse
>effect on mail delivery performance.
>
>The problem is: At the moment, I'm looking at a specific setup without
>knowing anything about the "historical" development of "chunkify".
>There is a comment which mentions an initial suggestion made by Chuq
>Von Rospach and further improvements made by Barry Warsaw.
>Unfortunately, I was unable to find these original discussion, and
>frankly, without knowing why "chunkify" is implemented the way it is,
>I don't think I'm qualified to discuss my concern (which involves VERP
>delivery using a specific MTAs VERP implementation and a highly I/O
>saturated mail server) on this list.


See
<http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.012.htp>
for some background.


>What I found out from the archives is that "chunkfiy" has two
>purposes:
>
>1. Make sure that not more than mm_cfg.SMTP_MAX_RCPTS are passed to
>the MTA in a single transaction.
>2. To improve delivery performance by grouping destinations which will
>prevent messages which are addressed to dead/congested destinations
>blocking delivery of messages to working destinations.


1. is correct.

2. is only partially correct. The purpose of grouping is to try to
minimize the number of transactions between the outgoing MTA and the
remote MTA. E.g., if all the aol.com addresses are passed from mailman
to the outgoing MTA in one SMTP transaction, then the MTA is able (if
so configured, etc., ...) to deliver the message to aol.com in one
transaction with the same recipients. It has nothing to do with trying
to prevent blocking, as the chunks are delivered serially anyway.

None of this is relevant if your MTA is VERPing the outgoing message,
because the MTA will send each VERPed message to a single recipient
because each recipient's message has a different envelope from
(separate SMTP MAIL FROM command).


>Is this assumption of mine basically correct? If it is and if I didn't
>miss any important parts, I would like to point out a specific
>scenario in which the way "chunkify" is implemented and the way in
>which it is called based on delivery style (verp, bulk) and the
>setting of mm_cfg.SMTP_MAX_RCPTS might actually worsen overall mail
>delivery performance.
>
>If I am totally wrong and "chunkfiy" serves a totally different
>purpose, please feel free to ignore this posting.


I wouldn't say you are totally wrong, but you are not totally correct.
In any case, if you have 'tuning' suggestions, either in general or
for specific cases, we would be glad to receive them and either
incorporate them in the code or add them to the FAQ as appropriate.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From cite at incertum.net  Mon May 12 21:24:41 2008
From: cite at incertum.net (Stefan =?utf-8?Q?F=C3=B6rster?=)
Date: Mon, 12 May 2008 21:24:41 +0200
Subject: [Mailman-Developers] Regarding Handlers/SMTPDirect.py and
	"chunkify"
In-Reply-To: <PC1830200805121049410140270d06f5@msapiro>
References: <20080512125134.GA23014@mail.incertum.net>
	<PC1830200805121049410140270d06f5@msapiro>
Message-ID: <20080512192440.GC26037@mail.incertum.net>

* Mark Sapiro <mark at msapiro.net> wrote:
> Stefan F?rster wrote:
>> I don't think I'm qualified to discuss my concern (which involves VERP
>> delivery using a specific MTAs VERP implementation and a highly I/O
>> saturated mail server) on this list.
> 
> See
> <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.012.htp>
> for some background.

I apologize, I should have found that entry myself. This sure makes up
for an interesting reading. Was bandwith really that expensive back in
2001? :)

> 2. is only partially correct. The purpose of grouping is to try to
> minimize the number of transactions between the outgoing MTA and the
> remote MTA. E.g., if all the aol.com addresses are passed from mailman
> to the outgoing MTA in one SMTP transaction, then the MTA is able (if
> so configured, etc., ...) to deliver the message to aol.com in one
> transaction with the same recipients. It has nothing to do with trying
> to prevent blocking, as the chunks are delivered serially anyway.

I can fully understand the thoughts behind this. OTOH, as the FAQ
entry you mentioned, if you do parallel deliveries to a single
location, you get a greater throughput: "Cut network bandwidth but
slow delivery to the larger domains." I remember a posting on this
list about cuddling MTAs or trying to outsmart them ;-) [1]

> In any case, if you have 'tuning' suggestions, either in general or
> for specific cases, we would be glad to receive them and either
> incorporate them in the code or add them to the FAQ as appropriate.

I don't know whether this is really a "tuning" suggestion. Let me
please describe my specific setup in hopes that the reason for my
request becomes clearer:

I recently had to take care of a server which was MX for a lagre
company and their outgoing mail relay (common setup). This machine was
doing fine until another server, dedicated to handle the companys many
mailig lists, broke down and it had to take care of mailing list
deliveries, too. The added load of permanent VERPed deliveries was
totally killing the machine in terms of I/O capacity, a situation
which could be resolved as of yesterday by letting the MTA do the
VERPing.

Now, the reason this change was so effective was that the number of
(disk) I/O operations was greatly reduced: Where a posting to a list
with 25000 subscribers would inevitably lead to the creation of 25000
queue files (even more if message content and control information are
kept in different files) before the change, only 25000/500 were
created afterwards (or "should have been", see next paragraph).
Additionally (but this has not really that much of an influence), a
larger number of destinations is available much faster to the MTA,
allowing it's queue manager to make decisions in a "more educated"
way.

Since the server is still quite close to it's I/O limits, I would very
much like to further reduce the number of I/O transactions. Therefore,
I would like to see a configurable parameter, something like
FLAT_RECIPIENT_CHUNKIFY: Right now, if mm_cfg.SMTP_MAX_RCPTS is not
set <= 0, "chunkify" will always (i.e. for any real world recipient
distribution) return more than one chunk of recipients, even if the
total number of recipients is smaller than  mm_cfg.SMTP_MAX_RCPTS.
With the addition of the aforementioned "tunable", we could have
chunkify simple split recipients into batches of mm_cfg.SMTP_MAX_RCPTS
and don't do any groupings. This way, a delivery to 25000 recipients
would really only involve (25000/mm_cfg.SMTP_MAX_RCPTS)*(number of
queue files per message) file creation operations.

Let me please emphasize that I'm fully aware that I will need better
disks or a new server, or get rid off VERP in the long run. I was just
thinking that the addition of a tunable like that _might_ be a
reasonable thing to propose if one absolutely wants to reduce the
number of I/O operations during a "bulkdeliver" without having to set
mm_cfg.SMTP_MAX_RCPTS to <= 0. But I think having one more option (and
I don't think this is hard to code, even I could do that and I have
read my first lines of Python on Friday evening) to adapt Mailman (or
it's delivery strategy) to a local site's needs would not really hurt.
If this option defaults to false, nobody would even notice it.

OTOH, it is one configuration parameter more, and one should not
easily include these things to avoid unnecessary configuration
complexity.

If you feel this is an unreasonable request, please do say so. If not,
I'd be glad to get some feedback from the people on this list
regarding "optimal" delivery strategies to the local MTA in this (or
other) cases.


Cheers
Stefan

[1] This was meant to be a joke. Just to be safe.
-- 
Stefan F?rster     http://www.incertum.net/     Public Key: 0xBBE2A9E9
"Wenn es sich um Geld handelt, geh?rt jeder der gleichen Religion an."
    -- Voltaire

From mark at msapiro.net  Mon May 12 23:20:09 2008
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 12 May 2008 14:20:09 -0700
Subject: [Mailman-Developers] Regarding Handlers/SMTPDirect.py and
	"chunkify"
In-Reply-To: <20080512192440.GC26037@mail.incertum.net>
Message-ID: <PC1830200805121420090906b9eab353@msapiro>

Stefan F?rster wrote:
>
>Now, the reason this change was so effective was that the number of
>(disk) I/O operations was greatly reduced: Where a posting to a list
>with 25000 subscribers would inevitably lead to the creation of 25000
>queue files (even more if message content and control information are
>kept in different files) before the change, only 25000/500 were
>created afterwards (or "should have been", see next paragraph).
>Additionally (but this has not really that much of an influence), a
>larger number of destinations is available much faster to the MTA,
>allowing it's queue manager to make decisions in a "more educated"
>way.
>
>Since the server is still quite close to it's I/O limits, I would very
>much like to further reduce the number of I/O transactions. Therefore,
>I would like to see a configurable parameter, something like
>FLAT_RECIPIENT_CHUNKIFY: Right now, if mm_cfg.SMTP_MAX_RCPTS is not
>set <= 0, "chunkify" will always (i.e. for any real world recipient
>distribution) return more than one chunk of recipients, even if the
>total number of recipients is smaller than  mm_cfg.SMTP_MAX_RCPTS.
>With the addition of the aforementioned "tunable", we could have
>chunkify simple split recipients into batches of mm_cfg.SMTP_MAX_RCPTS
>and don't do any groupings. This way, a delivery to 25000 recipients
>would really only involve (25000/mm_cfg.SMTP_MAX_RCPTS)*(number of
>queue files per message) file creation operations.


I understand what you are saying, but I wonder what the real world
difference would be. As currently written, chunkify returns at most 4
partially filled chunks. Granted, 4 is significantly bigger than one,
but given that the MTA is VERPing the deliveries, it may ultimately
create an outgoing queue entry for each recipient anyway, so the extra
3 on the inbound side doesn't seem that significant (and it might
increase parallelism in the MTA).

Given your 25000 member list, and assuming SMTP_MAX_RCPTS = 500, you
would have at most 54 chunks (and more likely 53 or 52) instead of 50.

In any case, If I were coding this, I would be inclined to not make it
an option, but just to change chunkify so it still grouped, but
continued to fill the last chunk of a group from the next group so
there would be at most one partial chunk.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From cite at incertum.net  Mon May 12 23:59:06 2008
From: cite at incertum.net (=?ISO-8859-1?Q?Stefan_F=F6rster?=)
Date: Mon, 12 May 2008 23:59:06 +0200
Subject: [Mailman-Developers] Regarding Handlers/SMTPDirect.py and
	"chunkify"
In-Reply-To: <PC1830200805121420090906b9eab353@msapiro>
References: <PC1830200805121420090906b9eab353@msapiro>
Message-ID: <DCE0D970-6D0F-4B1F-9994-2244CABA3D9A@incertum.net>


Am 12.05.2008 um 23:20 schrieb Mark Sapiro:
> I understand what you are saying, but I wonder what the real world
> difference would be. As currently written, chunkify returns at most 4
> partially filled chunks. Granted, 4 is significantly bigger than one,
> but given that the MTA is VERPing the deliveries, it may ultimately
> create an outgoing queue entry for each recipient anyway, so the extra
> 3 on the inbound side doesn't seem that significant (and it might
> increase parallelism in the MTA).

First of all, I just noticed that the official code does indeed only  
create at most 4 partially filled buckets. That's the problem when you  
have to jump in for someone else: My SMTPDirect.py contains 26 TLDs.  
Two thoughts:

1. Even with only four buckets, when we have a real world distribution  
amongst recipient addresses, this is four times the I/O needed. The  
ratio get's better with the number of list subscribers growing, but if  
there are less recipients than SMTP_MAX_RCPTS, it's exactly at 1:4.
2. Why even split recipients the way it's done now at all? You have to  
either add new buckets (add new TLDs) or have all recipients outside  
the hard coded TLDs be thrown into the same bucket. I could understand  
it if you first created a list of TLDs involved and sorted by those -  
though I don't know if it's a good idea if you run a really large list  
and examine all recipients...

I didn't understand what you said about VERPing and outgoing queue  
entries - surely any MTA will keep track of recipients on a per  
message basis? As for parallelism, I think the best way to ensure fast  
delivery is to make all target destinations known to the MTA as fast  
as possible.

> Given your 25000 member list, and assuming SMTP_MAX_RCPTS = 500, you
> would have at most 54 chunks (and more likely 53 or 52) instead of 50.
>
> In any case, If I were coding this, I would be inclined to not make it
> an option, but just to change chunkify so it still grouped, but
> continued to fill the last chunk of a group from the next group so
> there would be at most one partial chunk.

At the moment, I changed the code to simply return SMTP_MAX_RCPTS per  
chunk - or all recipients if there are less than that. Hardcoded, not  
configurable. The way it is done now I can't see any real advantages -  
especially living outside the U.S. Either improve the sorting  
algorithm (all TLDs, don't return partial chunks) or make it  
configurable to skip sorting altogether. Or at least that's what I  
feel would be an improvement. Have it default to flat chunking. It  
saves CPU time, I/O operations and gives the MTAs queue manager more  
time to do it's job.


Cheers
Stefan
-- 
Stefan F?rster     http://www.incertum.net/     Public Key: 0xBBE2A9E9
Written on OSX. Who ate my ~/.signature?


From mark at msapiro.net  Tue May 13 00:57:29 2008
From: mark at msapiro.net (Mark Sapiro)
Date: Mon, 12 May 2008 15:57:29 -0700
Subject: [Mailman-Developers] Regarding Handlers/SMTPDirect.py and
	"chunkify"
In-Reply-To: <DCE0D970-6D0F-4B1F-9994-2244CABA3D9A@incertum.net>
Message-ID: <PC18302008051215572906092d7056be@msapiro>

Stefan F?rster wrote:
>
>Am 12.05.2008 um 23:20 schrieb Mark Sapiro:
>> I understand what you are saying, but I wonder what the real world
>> difference would be. As currently written, chunkify returns at most 4
>> partially filled chunks. Granted, 4 is significantly bigger than one,
>> but given that the MTA is VERPing the deliveries, it may ultimately
>> create an outgoing queue entry for each recipient anyway, so the extra
>> 3 on the inbound side doesn't seem that significant (and it might
>> increase parallelism in the MTA).
>
>First of all, I just noticed that the official code does indeed only
>create at most 4 partially filled buckets. That's the problem when you 
>have to jump in for someone else: My SMTPDirect.py contains 26 TLDs. 
>Two thoughts:
>
>1. Even with only four buckets, when we have a real world distribution 
>amongst recipient addresses, this is four times the I/O needed. The
>ratio get's better with the number of list subscribers growing, but if 
>there are less recipients than SMTP_MAX_RCPTS, it's exactly at 1:4.


True.


>2. Why even split recipients the way it's done now at all? You have to 
>either add new buckets (add new TLDs) or have all recipients outside
>the hard coded TLDs be thrown into the same bucket. I could understand 
>it if you first created a list of TLDs involved and sorted by those - 
>though I don't know if it's a good idea if you run a really large list 
>and examine all recipients...


This predates my experience with Mailman. It is based on the statistics
provided by Chuq and outlined in the FAQ. It's true that these
statistics may only be applicable to lists with primarily US members,
and may be outdated in any case, but I can't provide any more
information on why it's done that way. Perhaps it's an idea that's
outlived its usefulness.


>I didn't understand what you said about VERPing and outgoing queue
>entries - surely any MTA will keep track of recipients on a per
>message basis?


I wasn't thinking clearly. I'm sure you're correct.


>As for parallelism, I think the best way to ensure fast 
>delivery is to make all target destinations known to the MTA as fast
>as possible.
>
>> Given your 25000 member list, and assuming SMTP_MAX_RCPTS = 500, you
>> would have at most 54 chunks (and more likely 53 or 52) instead of 50.
>>
>> In any case, If I were coding this, I would be inclined to not make it
>> an option, but just to change chunkify so it still grouped, but
>> continued to fill the last chunk of a group from the next group so
>> there would be at most one partial chunk.
>
>At the moment, I changed the code to simply return SMTP_MAX_RCPTS per
>chunk - or all recipients if there are less than that. Hardcoded, not
>configurable. The way it is done now I can't see any real advantages - 
>especially living outside the U.S. Either improve the sorting 
>algorithm (all TLDs, don't return partial chunks) or make it
>configurable to skip sorting altogether. Or at least that's what I
>feel would be an improvement. Have it default to flat chunking. It
>saves CPU time, I/O operations and gives the MTAs queue manager more 
>time to do it's job.


I think you make a good argument. I'd like to hear from others on this.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From barry at list.org  Tue May 13 06:26:08 2008
From: barry at list.org (Barry Warsaw)
Date: Tue, 13 May 2008 00:26:08 -0400
Subject: [Mailman-Developers] Regarding Handlers/SMTPDirect.py and
	"chunkify"
In-Reply-To: <PC18302008051215572906092d7056be@msapiro>
References: <PC18302008051215572906092d7056be@msapiro>
Message-ID: <043EE3FC-AE8F-4D94-934F-301BDF99D2EF@list.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On May 12, 2008, at 6:57 PM, Mark Sapiro wrote:

> This predates my experience with Mailman. It is based on the  
> statistics
> provided by Chuq and outlined in the FAQ. It's true that these
> statistics may only be applicable to lists with primarily US members,
> and may be outdated in any case, but I can't provide any more
> information on why it's done that way. Perhaps it's an idea that's
> outlived its usefulness.

It may have.  I see from the NEWS file that SMTP_MAX_RCPTS was  
introduced on the last day of 1998, and these chunking tricks were  
essential back then given the state of the art in MTAs.  A decade is a  
long time in this field, so it makes sense to re-evaluate Mailman's  
MTA hand-off machinery.

What makes the most sense for us to do today?  In principle, I would  
like to simplify this code rather than complicate it with yet another  
configuration option.  So what use cases do we need to support?  Can  
we get closer interaction with the upstream MTA to hand it the  
outgoing message and recipients more efficiently?  The more we can  
push these decisions into the MTA, the better off Mailman will be.

As an example of an efficiency I'm likely to add to MM3: I believe I  
can avoid calculating the full recipient list, requiring storing it in  
memory or in a queue file.  I think I can build it so that we only  
iterate through the recipients one at a time, as we're talking to the  
upstream MTA.  That would be a big I/O and memory footprint savings on  
the Mailman side.  Are there other thing we can do to push most of the  
processing, chunking, and efficiency calculations into the MTA, where  
it belongs (IMO)?  To gain this benefit, we may have to ditch  
chunking, or only support what grouping is available in SQL.

We clearly need to support personalization[1] in Mailman, although it  
would be nice if there were extensions we could rely on in the MTA to  
push even that out farther.  I'd like to hear from any Sendmail,  
Postfix, or Exim experts on this list to see if the chunking even  
makes sense any more.

I will note that we still get requests to batch deliveries to the  
outgoing MTA.  People want to be able to throttle Mailman's outgoing  
bandwidth to X number of messages per hour or day.  Ideally, this  
shouldn't be Mailman's task, but it's a valid request and one we  
should keep in mind.

I would be happy if we could determine the two or three strategies  
that address 80% of Mailman's users, package them up as simple-to- 
configure defaults, and then provide a robust plugin architecture that  
people can use to build all kinds of wacky strategies for the other 20%.

- -Barry

[1] Technically, not VERP, but close enough for our purposes.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iEYEARECAAYFAkgpGGAACgkQ2YZpQepbvXHNwACgpBxMYwh136zybvpKWJtEKol4
Ge0An1vLzJWz9mRGOSwd/nR6bM+OkdEo
=DNsQ
-----END PGP SIGNATURE-----

From iane at sussex.ac.uk  Tue May 13 11:46:51 2008
From: iane at sussex.ac.uk (Ian Eiloart)
Date: Tue, 13 May 2008 10:46:51 +0100
Subject: [Mailman-Developers] Regarding Handlers/SMTPDirect.py
 and	"chunkify"
In-Reply-To: <043EE3FC-AE8F-4D94-934F-301BDF99D2EF@list.org>
References: <PC18302008051215572906092d7056be@msapiro>
	<043EE3FC-AE8F-4D94-934F-301BDF99D2EF@list.org>
Message-ID: <3A978FDD3469364C6F22277D@lewes.staff.uscs.susx.ac.uk>



--On 13 May 2008 00:26:08 -0400 Barry Warsaw <barry at list.org> wrote:

> We clearly need to support personalization[1] in Mailman, although it
> would be nice if there were extensions we could rely on in the MTA to
> push even that out farther.  I'd like to hear from any Sendmail, Postfix,
> or Exim experts on this list to see if the chunking even makes sense any
> more.

Yes, it does. At least, it makes sense to offer the option. There are two 
cases - (a) Mailman delivers all its mail to a smarthost, (b) Mailman 
delivers some or all of its mail directly to the MX hosts of the recipients.

Exim, by default, doesn't care about the number of recipients per message. 
It can be configured to using "recipients_max", and in that case will 
either defer or reject "recipients_max_reject" the excess. Both these 
configuration variables are global. Fine tuning of similar limits is 
available in ACLs, though, and can be configured differently for a known 
Mailman host.

If Mailman is configured to route mail through a smarthost, then an Exim 
smarthost should be configured to cope with whatever Mailman throws at it.

Otherwise, as the Exim docs caution, you might fall foul of RFC2821 section 
4.5.3.1 Size limits and minimums:

recipients buffer
      The minimum total number of recipients that must be buffered is
      100 recipients.  Rejection of messages (for excessive recipients)
      with fewer than 100 RCPT commands is a violation of this
      specification.  The general principle that relaying SMTP servers
      MUST NOT, and delivery SMTP servers SHOULD NOT, perform validation
      tests on message headers suggests that rejecting a message based
      on the total number of recipients shown in header fields is to be
      discouraged.  A server which imposes a limit on the number of
      recipients MUST behave in an orderly fashion,  such as to reject
      additional addresses over its limit rather than silently
      discarding addresses previously accepted.  A client that needs to
      deliver a message containing over 100 RCPT commands SHOULD be
      prepared to transmit in 100-recipient "chunks" if the server
      declines to accept more than 100 recipients in a single message.... 
etc...

-- 
Ian Eiloart
IT Services, University of Sussex
x3148

From mark at msapiro.net  Wed May 14 03:14:49 2008
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 13 May 2008 18:14:49 -0700
Subject: [Mailman-Developers] Any known issues with list locking?
Message-ID: <PC18302008051318144903755cac0f68@msapiro>

I had a strange occurrance yesterday on one of my lists

Mailman 2.1.10
Python 2.4.3
CentOS release 5 (Final)

PREFIX          = '/usr/local/mailman'
VAR_PREFIX      = '/var/lib/mailman'

/usr/local/ and /var/ are separate partitions on the same local disk.
Access is strictly local; no NFS or anything like that.

What happened is a user was logged in to her options page. She was
going on vacation, so she first changed her password, but then she was
confused and instead of disabling mail delivery, she unsubscribed. A
notice of the unsubscribe was sent to the list admin and to her, and
the unsubscribe via the member options page was logged, but later, she
was still on the list with the new password. There was no log or
notice of any new subscribes, and the rest of the list members were
the same so no address was changed to her address.

There was a list post being processed at the same time. In fact, it
appears that the post was approved from the list's admindb interface
about 1 second earlier than the unsub from the member options page.

I've looked over the code and I can't see anywhere where list data are
manipulated without being locked, and besides calling the Save()
method on an unlocked list throws an exception and locking a list
refreshes current data from config.pck, so it should not be possible
for two processes to be updating the list concurrently unless there is
some issue in LockFile itself.

There's nothing in Mailman's locks log, and grep of all the system logs
for 'listname.lock' finds nothing either.

Has anyone seen anything like this?

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From barry at list.org  Wed May 14 03:49:56 2008
From: barry at list.org (Barry Warsaw)
Date: Tue, 13 May 2008 21:49:56 -0400
Subject: [Mailman-Developers] Any known issues with list locking?
In-Reply-To: <PC18302008051318144903755cac0f68@msapiro>
References: <PC18302008051318144903755cac0f68@msapiro>
Message-ID: <4975D07E-0567-4B9D-AAA8-6DFC312C4A01@list.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On May 13, 2008, at 9:14 PM, Mark Sapiro wrote:
>
> I've looked over the code and I can't see anywhere where list data are
> manipulated without being locked, and besides calling the Save()
> method on an unlocked list throws an exception and locking a list
> refreshes current data from config.pck, so it should not be possible
> for two processes to be updating the list concurrently unless there is
> some issue in LockFile itself.
>
> There's nothing in Mailman's locks log, and grep of all the system  
> logs
> for 'listname.lock' finds nothing either.
>
> Has anyone seen anything like this?

Mark, I'm not aware of any bugs like this.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iEYEARECAAYFAkgqRUQACgkQ2YZpQepbvXG+0gCghWTz1oDYvqEG7woKQr/QH9K6
NnsAnRmBwRKPl3qlmOAIXHT47FISF24M
=9Gm4
-----END PGP SIGNATURE-----

From cite at incertum.net  Wed May 14 14:16:28 2008
From: cite at incertum.net (Stefan =?utf-8?Q?F=C3=B6rster?=)
Date: Wed, 14 May 2008 14:16:28 +0200
Subject: [Mailman-Developers] Regarding Handlers/SMTPDirect.py and
	"chunkify"
In-Reply-To: <043EE3FC-AE8F-4D94-934F-301BDF99D2EF@list.org>
References: <PC18302008051215572906092d7056be@msapiro>
	<043EE3FC-AE8F-4D94-934F-301BDF99D2EF@list.org>
Message-ID: <20080514121628.GE992@mail.incertum.net>

* Barry Warsaw <barry at list.org> wrote:
> On May 12, 2008, at 6:57 PM, Mark Sapiro wrote:
> 
>> This predates my experience with Mailman. It is based on the
>> statistics
>> provided by Chuq and outlined in the FAQ. It's true that these
>> statistics may only be applicable to lists with primarily US members,
>> and may be outdated in any case, but I can't provide any more
>> information on why it's done that way. Perhaps it's an idea that's
>> outlived its usefulness.
> 
> It may have.  I see from the NEWS file that SMTP_MAX_RCPTS was
> introduced on the last day of 1998, and these chunking tricks were
> essential back then given the state of the art in MTAs.  A decade is a
> long time in this field, so it makes sense to re-evaluate Mailman's
> MTA hand-off machinery.

As Ian Eiloart already pointed out, Mailman MUST comply with RFC 2821.
I'd even say lower the default setting of SMTP_MAX_RCPTS to 100 (mark
this change in the NEWS file or changelog file, and add MTA specific
instructions in the README.<MTA> files).

> We clearly need to support personalization[1] in Mailman, although it
> would be nice if there were extensions we could rely on in the MTA to
> push even that out farther.  I'd like to hear from any Sendmail,
> Postfix, or Exim experts on this list to see if the chunking even
> makes sense any more.

My inquiry on postfix-users on this matter hasn't yielded any
satisfying answers yet, and I'm by no means a Postfix "expert", but I
have several years of experience and I can read the documentation,
available at http://www.postfix.org/SCHEDULER_README.html - especially
the part talking about how the preemptive schedueler picks delivery
jobs. Fro, what's written there, it seems Postfix's queue manager -
qmgr - is able to group recipients on a roughly "per destination" base
even if different mails are assigned to those recipients. That means,
if the third batch contains an entry for "foo at aol.com.invalid" and the
sixth batch has a recipient "bar at aol.com.invalid", by the time the
qmgr begins to deliver to "@aol.com.invalid" it will correctly group
delivery to those two recipients, applying concurrency feedback (see
first part of SCHEDULER_README.html) as needed. Therefore, in an
environment without latency, a zero message delivery time, an
unlimited amount of delivery agents and an unlimited number of memory
for recipient entries (well, last two don't need to be "unlimited",
jst large enough so that every recipient of every incoming message can
be processed instantly), we would lose performance by NOT grouping
those "@aol.com.invalid" recipients.

In reality, the number of delivery agents, the maximum number of
concurrent connections to a destination and the number of recipients
which can be kept in Postfix's active_queue are limited, queue file
generation, setting up a TCP connection and transmitting of a message
do take more than zero time (the last two steps may even fail!). If in
the above example batch six enters the active_queue before all
delivery to "@aol.com.invalid" is done, the only drawback of NOT
grouping recipients would then be the overhead to either keep the
actual message contents in memory twice or to do two "read file"
operations to get those contents to memory for transmission.

OTOH, if by grouping we increase the overall number if "DATA" commands
(and therefore messages) transmitted to Postfix by Mailman, we might
have even higher overhead.

Please don't take that statement for granted. I didn't write "qmgr".
And to be honest, I didn't even start to think about network bandwith
usage. I strongly believe chunking the way it's done doesn't make
sense anymore, if you are not running some old qmail ;)

> I will note that we still get requests to batch deliveries to the
> outgoing MTA.  People want to be able to throttle Mailman's outgoing
> bandwidth to X number of messages per hour or day.  Ideally, this
> shouldn't be Mailman's task, but it's a valid request and one we
> should keep in mind.

> I would be happy if we could determine the two or three strategies
> that address 80% of Mailman's users, package them up as simple-to-
> configure defaults, and then provide a robust plugin architecture that
> people can use to build all kinds of wacky strategies for the other 20%

Ok, from my point of view, I'd say 80% of users are made up of two
groups:

The first group doesn't care at all. They want their MTA to do the
work, they fine tune it (or don't), but basically, as long as their
machines don't crash, they couldn't care less about the order in which
messages are delivered to the MTA. For those people, Mailman needs a
way to get messages out to the downstream MTA quickly, which implies
two constraints:

1. The recipient list must be stored in a way that allows quick
   generation of the necessary SMTP commands.
2. Either the MTA or Mailman must have an option similar to
   SMTP_MAX_RCPTS to avoid unnecessary delays and another option to
   not exhaust the maximum number of concurrent connections the MTA
   will acceppt.

Advantages of this approach:
1. It is never Mailman's fault.
2. All "tuning" can be done in the MTA's configuration, if "tuining"
   is needed.
3. Recipient ordering is only donw once, saving a (tiny) amount of CPU
   time.

Drawbacks of ths approach:
1. Mailman has absolutely zero control about the order or time
   messages are delivered to their final recipients.
2. If, for any reason, delivery to the MTA fails, we (actually, you
   *g*) need to have code in place to handle a queue of outgoing
   messages anyways.

The second group of those 80% want a finer control, and from what you
describe, they all want a "per timeslice" behaviour. Delivering only a
certain amount of messages, delivering messages only during off-hours
(don't know if this is real word, I mean evening and early morning),
delivering only a certain volume of messages, all those are basically
timeslice constraints. To handle those, Mailman needs to:

1. Know what time it is. (obviously!)
2. Considering that all delivery strategies are <N>/time_slice,
   Mailman obviously need to know <N>, which can be:
   - the total number of messages, as in "multiply every outgoing
     message whith the number of list subscribers and sum that up"
   - for volume based delivery, the total number of messages and at
     least an educated guess on their size
   - number of recipients for a single destination
   - ...

So if Mailman would now the concept of "per timeslice" and the
characteristics of outgoing mail, you already had a very flexible
interface in place which allowed for refined control of message flow.
OTOH, aside from some very special cases (UUCP, ETRN-only), Mailman
doesn't know the internal state of the MTA it delivers to. If there
were, for example, a network outage and the MTA would queue the
contents of 10 timeslices until the network is restored, all that
refined flow control was for nothing.

Honestly, I don't know what kind of "wacky" the remaining 20% of
mailman users could want, but I can think of some things which are
normally done in the MTA, for example concurrency control to certain
destinations. I have no clue how that could be achieved, because,
again, Mailman does NOT know about the state of the MTA it delivers
to, and implementing an ESMTP engine and queue manager in Mailman
seems like doing the job twice.


Cheers
Stefan

P.S: While writing this message, I didn't have access to a dictionary.
I apologize in advance.
-- 
Stefan F?rster     http://www.incertum.net/     Public Key: 0xBBE2A9E9
218 DSA/RSA host keys to go...
Worst. Nightmare. Ever.

From amk at amk.ca  Wed May 14 14:40:11 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Wed, 14 May 2008 08:40:11 -0400
Subject: [Mailman-Developers] Copy of Spertus dynamic-list patch
Message-ID: <20080514124011.GA7064@amk-desktop.matrixgroup.net>

For work I'm taking a look at Ellen Spertus's dynamic sublist patches.
I've taken the patch from http://sourceforge.net/projects/dsub/,
removed some extraneous stuff (like the diffs for the configure script)
and added a few notes on what the patch does.  The modified patch is at
http://www.amk.ca/files/mailman/spertus-mailman-patch.txt

--amk

From barry at list.org  Mon May 19 00:51:43 2008
From: barry at list.org (Barry Warsaw)
Date: Sun, 18 May 2008 18:51:43 -0400
Subject: [Mailman-Developers] Important Mailman 2.1.9 to 2.1.10 upgrade
	note.
In-Reply-To: <481C8FA6.9080709@msapiro.net>
References: <PC1760200804221144050468c671b4f9@msapiro>
	<C5C4A17B-EF8A-4EC9-9F6C-5A43F2B48D49@python.org>
	<4814CA2B.7050203@msapiro.net>
	<C7D7344D-87E4-4895-8715-AB4B7C9A413D@list.org>
	<481C8FA6.9080709@msapiro.net>
Message-ID: <A51360EE-AA06-4C5F-903E-84A3987C3F72@list.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On May 3, 2008, at 12:15 PM, Mark Sapiro wrote:
>
> BTW, the above stuff all comes from changes I made on the 2.1 branch  
> at
> revs 1072 and 1074. Did anyone look at the cull_bad_shunt cron and
> bin/update change at revs 1074 and 1075 of
> <https://code.launchpad.net/~msapiro/mailman/cull_bad_shunt>?

Mark,

There looks to be something very strange going on with this branch.   
How did you create it?

- -Barry

% bzr info
Repository tree (format: pack-0.92)
Location:
   shared repository: /Users/barry/projects/mailman
   repository branch: .

Related branches:
   parent branch: http://bazaar.launchpad.net/%7Emsapiro/mailman/cull_bad_shunt/


% bzr version-info
revision-id: mark at msapiro.net-20080427175235-ld4mi9cf2aunq27b
date: 2008-04-27 10:52:35 -0700
build-date: 2008-05-18 18:29:34 -0400
revno: 2
branch-nick: cull_bad_shunt


% bzr log | head -n 20
- ------------------------------------------------------------
revno: 1075
committer: Mark Sapiro <mark at msapiro.net>
branch nick: cull_bad_shunt
timestamp: Sun 2008-04-27 10:52:35 -0700
message:
   Changed bin/update to remove .bak qfiles.
- ------------------------------------------------------------
revno: 1074
committer: Mark Sapiro <mark at msapiro.net>
branch nick: cull_bad_shunt
timestamp: Sun 2008-04-27 10:37:39 -0700
message:
   Added new cron/cull_bad_shunt and supporting Defaults.py.in settings.
- ------------------------------------------------------------
revno: 1073
committer: Mark Sapiro <mark at msapiro.net>
branch nick: 2.1
timestamp: Sat 2008-04-26 17:59:18 -0700
message:


% bzr log -r 1074..1075
bzr: ERROR: Requested revision: u'1074' does not exist in branch:  
BzrBranch6('file:///Users/barry/projects/mailman/cull_bad_shunt/')


% bzr diff -r 1073..1075
bzr: ERROR: Requested revision: u'1073' does not exist in branch:  
BzrBranch6('file:///Users/barry/projects/mailman/cull_bad_shunt/')


% bzr diff -r 1..-1
bzr: ERROR: exceptions.AssertionError: revno/history mismatch

Traceback (most recent call last):
[...]
AssertionError: revno/history mismatch

bzr 1.4 on python 2.5.2 (darwin)
arguments: ['/opt/local/bin/bzr', 'diff', '-r', '1..-1']
encoding: 'US-ASCII', fsenc: 'utf-8', lang: None
plugins:
   bzrtools             /opt/local/lib/python2.5/site-packages/bzrlib/ 
plugins/bzrtools [1.4.0]
   fastimport           /Users/barry/.bazaar/plugins/fastimport  
[unknown]
   hgimport             /Users/barry/.bazaar/plugins/hgimport [unknown]
   launchpad            /opt/local/lib/python2.5/site-packages/bzrlib/ 
plugins/launchpad [unknown]
   loom                 /Users/barry/.bazaar/plugins/loom [1.4.0dev0]
   touch                /Users/barry/.bazaar/plugins/touch.py [unknown]
*** Bazaar has encountered an internal error.
     Please report a bug at https://bugs.launchpad.net/bzr/+filebug
     including this traceback, and a description of what you
     were doing when the error occurred.

(Note that this last exception seems to be fixed in bzr 1.5rc1 on  
Ubuntu.)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iEYEARECAAYFAkgwsv8ACgkQ2YZpQepbvXE2MwCeMjJ7cvbgPkVnT2ZRcac8XQw8
tLkAn3OkvfFZZ8WKTfyTuuxZ9qIKH/fh
=2aRC
-----END PGP SIGNATURE-----

From mark at msapiro.net  Mon May 19 02:31:46 2008
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 18 May 2008 17:31:46 -0700
Subject: [Mailman-Developers] Important Mailman 2.1.9 to 2.1.10 upgrade
 note.
In-Reply-To: <A51360EE-AA06-4C5F-903E-84A3987C3F72@list.org>
References: <PC1760200804221144050468c671b4f9@msapiro>
	<C5C4A17B-EF8A-4EC9-9F6C-5A43F2B48D49@python.org>
	<4814CA2B.7050203@msapiro.net>
	<C7D7344D-87E4-4895-8715-AB4B7C9A413D@list.org>
	<481C8FA6.9080709@msapiro.net>
	<A51360EE-AA06-4C5F-903E-84A3987C3F72@list.org>
Message-ID: <4830CA72.4060109@msapiro.net>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Barry Warsaw wrote:
| On May 3, 2008, at 12:15 PM, Mark Sapiro wrote:
|
|> BTW, the above stuff all comes from changes I made on the 2.1 branch at
|> revs 1072 and 1074. Did anyone look at the cull_bad_shunt cron and
|> bin/update change at revs 1074 and 1075 of
|> <https://code.launchpad.net/~msapiro/mailman/cull_bad_shunt>?
|
| Mark,
|
| There looks to be something very strange going on with this branch.  How
| did you create it?


I have a directory, MM_bzr, on my local workstation that has been 'bzr
init'd and has subdirectories 2.1, 2.2, 3.0 and admin which initially
were populated via 'bzr co' from launchpad. Everything has subsequently
been upgraded with 'bzr upgrade --pack-0.92'. Local bzr is 1.2.0.

I created MM_bzr/cull_bad_shunt locally with bzr branch from MM_bzr/2.1
at rev 1074. I then committed revs 1074 and 1075. I created the
cull_bad_shunt branch on launchpad and then 'bzr push'd my local branch.



| % bzr info
| Repository tree (format: pack-0.92)
| Location:
|   shared repository: /Users/barry/projects/mailman
|   repository branch: .
|
| Related branches:
|   parent branch:
| http://bazaar.launchpad.net/%7Emsapiro/mailman/cull_bad_shunt/


$ bzr info
Standalone tree (format: pack-0.92)
Location:
~  branch root: .

Related branches:
~    push branch:
bzr+ssh://msapiro at bazaar.launchpad.net/%7Emsapiro/mailman/cull_bad_shunt/
~  submit branch: /cygdrive/c/MM_bzr/2.1


| % bzr version-info
| revision-id: mark at msapiro.net-20080427175235-ld4mi9cf2aunq27b
| date: 2008-04-27 10:52:35 -0700
| build-date: 2008-05-18 18:29:34 -0400
| revno: 2
| branch-nick: cull_bad_shunt


$ bzr version-info
revision-id: mark at msapiro.net-20080427175235-ld4mi9cf2aunq27b
date: 2008-04-27 10:52:35 -0700
build-date: 2008-05-18 17:21:11 -0700
revno: 1075
branch-nick: cull_bad_shunt


| % bzr log | head -n 20
| ------------------------------------------------------------
| revno: 1075
| committer: Mark Sapiro <mark at msapiro.net>
| branch nick: cull_bad_shunt
| timestamp: Sun 2008-04-27 10:52:35 -0700
| message:
|   Changed bin/update to remove .bak qfiles.
| ------------------------------------------------------------
| revno: 1074
| committer: Mark Sapiro <mark at msapiro.net>
| branch nick: cull_bad_shunt
| timestamp: Sun 2008-04-27 10:37:39 -0700
| message:
|   Added new cron/cull_bad_shunt and supporting Defaults.py.in settings.
| ------------------------------------------------------------
| revno: 1073
| committer: Mark Sapiro <mark at msapiro.net>
| branch nick: 2.1
| timestamp: Sat 2008-04-26 17:59:18 -0700
| message:


$ bzr log | head -n 20
- ------------------------------------------------------------
revno: 1075
committer: Mark Sapiro <mark at msapiro.net>
branch nick: cull_bad_shunt
timestamp: Sun 2008-04-27 10:52:35 -0700
message:
~  Changed bin/update to remove .bak qfiles.
- ------------------------------------------------------------
revno: 1074
committer: Mark Sapiro <mark at msapiro.net>
branch nick: cull_bad_shunt
timestamp: Sun 2008-04-27 10:37:39 -0700
message:
~  Added new cron/cull_bad_shunt and supporting Defaults.py.in settings.
- ------------------------------------------------------------
revno: 1073
committer: Mark Sapiro <mark at msapiro.net>
branch nick: 2.1
timestamp: Sat 2008-04-26 17:59:18 -0700
message:


| % bzr log -r 1074..1075
| bzr: ERROR: Requested revision: u'1074' does not exist in branch:
| BzrBranch6('file:///Users/barry/projects/mailman/cull_bad_shunt/')


$ bzr log -r 1074..1075
- ------------------------------------------------------------
revno: 1075
committer: Mark Sapiro <mark at msapiro.net>
branch nick: cull_bad_shunt
timestamp: Sun 2008-04-27 10:52:35 -0700
message:
~  Changed bin/update to remove .bak qfiles.
- ------------------------------------------------------------
revno: 1074
committer: Mark Sapiro <mark at msapiro.net>
branch nick: cull_bad_shunt
timestamp: Sun 2008-04-27 10:37:39 -0700
message:
~  Added new cron/cull_bad_shunt and supporting Defaults.py.in settings.


| % bzr diff -r 1073..1075
| bzr: ERROR: Requested revision: u'1073' does not exist in branch:
| BzrBranch6('file:///Users/barry/projects/mailman/cull_bad_shunt/')
|
|
| % bzr diff -r 1..-1
| bzr: ERROR: exceptions.AssertionError: revno/history mismatch


Both of the above bzr diff commands work in my local branch.

Also, browsing the branch and changes at
<https://code.launchpad.net/~msapiro/mailman/cull_bad_shunt> seems to
produce expected results.

- --
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)

iD8DBQFIMMpyVVuXXpU7hpMRAspEAKD76xlQh7OyOHBBby3aWgRd4B5j/wCgoaJG
9ph9yf/lka0Yqx1ylTH1ygk=
=3/rS
-----END PGP SIGNATURE-----

From barry at list.org  Mon May 19 03:21:39 2008
From: barry at list.org (Barry Warsaw)
Date: Sun, 18 May 2008 21:21:39 -0400
Subject: [Mailman-Developers] Important Mailman 2.1.9 to 2.1.10 upgrade
	note.
In-Reply-To: <4830CA72.4060109@msapiro.net>
References: <PC1760200804221144050468c671b4f9@msapiro>
	<C5C4A17B-EF8A-4EC9-9F6C-5A43F2B48D49@python.org>
	<4814CA2B.7050203@msapiro.net>
	<C7D7344D-87E4-4895-8715-AB4B7C9A413D@list.org>
	<481C8FA6.9080709@msapiro.net>
	<A51360EE-AA06-4C5F-903E-84A3987C3F72@list.org>
	<4830CA72.4060109@msapiro.net>
Message-ID: <C98EF029-2BD7-4AA3-8587-F29A4C7F9A8E@list.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On May 18, 2008, at 8:31 PM, Mark Sapiro wrote:

> I have a directory, MM_bzr, on my local workstation that has been 'bzr
> init'd and has subdirectories 2.1, 2.2, 3.0 and admin which initially
> were populated via 'bzr co' from launchpad. Everything has  
> subsequently
> been upgraded with 'bzr upgrade --pack-0.92'. Local bzr is 1.2.0.
>
> I created MM_bzr/cull_bad_shunt locally with bzr branch from MM_bzr/ 
> 2.1
> at rev 1074. I then committed revs 1074 and 1075. I created the
> cull_bad_shunt branch on launchpad and then 'bzr push'd my local  
> branch.

What happens if you

bzr branch http://bazaar.launchpad.net/~msapiro/mailman/cull_bad_shunt

?

When I do this, either in my shared repository or in a fresh  
directory.  I get only 2 revisions.

> Both of the above bzr diff commands work in my local branch.

What about in a pristine branch?

> Also, browsing the branch and changes at
> <https://code.launchpad.net/~msapiro/mailman/cull_bad_shunt> seems to
> produce expected results.

Does it?  It looks weird to me.  For example, why all the renamed  
files in rev 1074?

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iEYEARECAAYFAkgw1iMACgkQ2YZpQepbvXE3lwCdF3CqO9iOFP06kKEdSkiq6yDF
Qn8An1VDhluLghnacPv19twbl8nIErjj
=SZ28
-----END PGP SIGNATURE-----

From mark at msapiro.net  Mon May 19 03:28:14 2008
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 18 May 2008 18:28:14 -0700
Subject: [Mailman-Developers] Important Mailman 2.1.9 to 2.1.10 upgrade
 note.
In-Reply-To: <C98EF029-2BD7-4AA3-8587-F29A4C7F9A8E@list.org>
References: <PC1760200804221144050468c671b4f9@msapiro>
	<C5C4A17B-EF8A-4EC9-9F6C-5A43F2B48D49@python.org>
	<4814CA2B.7050203@msapiro.net>
	<C7D7344D-87E4-4895-8715-AB4B7C9A413D@list.org>
	<481C8FA6.9080709@msapiro.net>
	<A51360EE-AA06-4C5F-903E-84A3987C3F72@list.org>
	<4830CA72.4060109@msapiro.net>
	<C98EF029-2BD7-4AA3-8587-F29A4C7F9A8E@list.org>
Message-ID: <4830D7AE.6080307@msapiro.net>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Barry Warsaw wrote:
|
| What happens if you
|
| bzr branch http://bazaar.launchpad.net/~msapiro/mailman/cull_bad_shunt
|
| ?
|
| When I do this, either in my shared repository or in a fresh directory.
| I get only 2 revisions.


I'm doing this now in a new directory. I'll post the result after it
completes.


|> Both of the above bzr diff commands work in my local branch.
|
| What about in a pristine branch?
|
|> Also, browsing the branch and changes at
|> <https://code.launchpad.net/~msapiro/mailman/cull_bad_shunt> seems to
|> produce expected results.
|
| Does it?  It looks weird to me.  For example, why all the renamed files
| in rev 1074?


Yes, that is strange, but that is in my local rev 1074 too. I didn't
understand it, but it seemed harmless at the time.

- --
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)

iD8DBQFIMNeuVVuXXpU7hpMRAhM6AKCEywK0Fkb31W4/zSA/iFLzMBZI4gCeN+ax
QqVcKMxm5yVwvhTpmtk0Qpg=
=n6b0
-----END PGP SIGNATURE-----

From barry at list.org  Mon May 19 03:31:38 2008
From: barry at list.org (Barry Warsaw)
Date: Sun, 18 May 2008 21:31:38 -0400
Subject: [Mailman-Developers] Using REST Interface and WADL
In-Reply-To: <20080506005340.bbbscgs3kw08gwc8@mail.matrixgroup.net>
References: <E6C6D0F2-ED02-41FD-99DD-216AA4B25959@list.org>
	<BF8C9162-B172-4735-B3E0-D1E07BF2C726@matrixgroup.net>
	<9DF8F405-6E17-4B4B-93F0-731B8B39A472@digitalcommute.com>
	<44853CF4-1B83-4527-B62E-2051A0D5FF27@list.org>
	<20080506005340.bbbscgs3kw08gwc8@mail.matrixgroup.net>
Message-ID: <C255078A-1E7C-4951-8FA5-26FCB195A439@list.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On May 6, 2008, at 12:53 AM, mk2s at digitalcommute.com wrote:
>
> I went to read up some more on WADL and ran into this:
> http://bitworking.org/news/193/Do-we-need-WADL

Thanks, I hadn't seen that before.

> I don't have any plans(yet) on auto generating the WADL, or did you  
> mean auto generating the documentation from the WADL file?  Are  
> there tools that do that?  I didn't find any such.
>
> Currently I'm returning human readable text containing something  
> like documentation when you request the xHTML representation.  I've  
> seen this done before and figured it was a pretty good idea.  It is  
> of no use of course if you want to auto generate a client, as you  
> would from a description file.
>
> Anyone else have any opinions or experiences with WADL?

I'm not aware of any tools to automatically build clients from a WADL  
description.  Personally, I'm rather skeptical it can be done, or at  
least done well.  I've had far too many experiences looking at  
horrible generated code to have much hope of it being a fruitful  
avenue.  I really only asked because I've talked to a few other folks  
about it w.r.t. different applications, so I was interested in getting  
your opinion on it.

Thanks,
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iEYEARECAAYFAkgw2HsACgkQ2YZpQepbvXFo+gCcCmoC4z8/JelOWL5/Oqh0z0OW
oW0AoLMj763toT/s21gQPjNYBDkywvDX
=EPw+
-----END PGP SIGNATURE-----

From barry at list.org  Mon May 19 03:38:10 2008
From: barry at list.org (Barry Warsaw)
Date: Sun, 18 May 2008 21:38:10 -0400
Subject: [Mailman-Developers] What do I need to do to add a package
	dependency in MM2.2?
In-Reply-To: <20080506005827.tbktkg7yqs880ggc@mail.matrixgroup.net>
References: <20080506005827.tbktkg7yqs880ggc@mail.matrixgroup.net>
Message-ID: <28713CE7-17C5-43F2-8981-B7CBE7281E34@list.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On May 6, 2008, at 12:58 AM, mk2s at digitalcommute.com wrote:

> I'm doing some work in mm2.2 and wanted to add some python package  
> dependencies like routes, lxml, and simplejason.  How do I go about  
> declaring these dependencies so that the dependent packages will be  
> installed for the end user who wants to run mm2.2?
>
> Can someone point me in the direction of the stuff that I need to  
> read?

It's more difficult to do in Mailman 2.2, because it is not  
(currently) setuptools based.

The way it's done in 2.1 and 2.2 is to add the packages to the misc/  
directory and edit misc/Makefile.in to install them into the pythonlib  
directory when that gets created.  There's examples there for the  
email package, JapaneseCodes and KoreanCodecs.

We should target Python 2.5 at least for Mailman 2.2.  None of the  
above packages are part of that afaik, but for example, I wonder if it  
still makes sense to include those codecs packages?

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iEYEARECAAYFAkgw2gIACgkQ2YZpQepbvXH9BgCggVX3Vpq/yDetXb+NQ5wE8BqC
A8MAn2DV5ufUrn9um19/IYlbGA+adRZ3
=TiMq
-----END PGP SIGNATURE-----

From barry at list.org  Mon May 19 03:40:00 2008
From: barry at list.org (Barry Warsaw)
Date: Sun, 18 May 2008 21:40:00 -0400
Subject: [Mailman-Developers] Intern
In-Reply-To: <BB46A1A7669FAC82DE060624@lewes.staff.uscs.susx.ac.uk>
References: <51043A076CCD3E8087F74D81@lewes.staff.uscs.susx.ac.uk>
	<9BFAD07F-9EBA-4A36-9129-2F16629342D4@list.org>
	<BB46A1A7669FAC82DE060624@lewes.staff.uscs.susx.ac.uk>
Message-ID: <9CE6ED8A-C674-4CB7-9E39-FCF20C4C5242@list.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On May 7, 2008, at 5:23 AM, Ian Eiloart wrote:

>>> I plan to get LMTP working to my taste, by (a) rejecting messages
>>> for non-existent lists at RCTP time, and (b) rejecting messages from
>>> unwanted sender addresses at RCPT time. I Think (a) will be easy to
>>> achieve, and (b) only slightly more challenging.
>>
>> This sounds great.  With (b) there could be multiple list-specific
>> policies, so we should make this pluggable.
>>
>
> The intention is to get the policy from the list. The simplest way  
> to do it would be to treat "reject" as an SMTP time reject, rather  
> than a bounce. At my site, at least, I would not offer "bounce" as a  
> choice if smtp rejection is available.
>
> Indeed, we currently recommend that list owners don't use "reject".

Interesting.  I'll have to think about this, but it seems to me that  
if you wanted to turn off 'reject' in Mailman 3, you'd have to end up  
disabling that chain (for things that make it past LMTP).  For the  
LMTP hooks, it should be no problem, because I think the way you're  
going to do it is to just do the SMTP-reject if the right rules match  
during the LMTP phase, i.e. that part of the system wouldn't run the  
full chain processor.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iEYEARECAAYFAkgw2nEACgkQ2YZpQepbvXEBuQCdHVpYJDNa4RLgUwSOTrm/C7Bz
ajkAnRpdrFYH3tTbEUfnhBXsQL33bsZA
=xk4G
-----END PGP SIGNATURE-----

From barry at list.org  Mon May 19 03:48:00 2008
From: barry at list.org (Barry Warsaw)
Date: Sun, 18 May 2008 21:48:00 -0400
Subject: [Mailman-Developers] Intern
In-Reply-To: <E76526D88CE123C526247A7E@lewes.staff.uscs.susx.ac.uk>
References: <51043A076CCD3E8087F74D81@lewes.staff.uscs.susx.ac.uk>
	<9BFAD07F-9EBA-4A36-9129-2F16629342D4@list.org>
	<E76526D88CE123C526247A7E@lewes.staff.uscs.susx.ac.uk>
Message-ID: <004BD56C-9A2D-4537-B435-26B3ACA1A9EE@list.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On May 7, 2008, at 5:37 AM, Ian Eiloart wrote:

>> Does any of that help?  You should definitely look at the Mailman 3  
>> code
>> and I'll be happy to answer any more questions you might have.
>>
> I'm sure all of that will help when I get into the kitchen, but I'm  
> having problems finding the right street. I thought I'd be able to  
> leap right in, but the more I look, the more I find that I need a  
> map. Perhaps something describing the lifecycle of a typical email,  
> and which bits of top level code are responsible for handling it at  
> which stage. Or maybe just a recommended order for reading the  
> doctests would do the job.
>
> With that understanding, I can follow the API calls to get at the  
> detail.

I would definitely start with the docs in mailman/queue/docs.  These  
describe the various queues, what their responsibilities are, where  
they get mail from and where they send mail to.  Most messages will  
come in via the lmtp or incoming queues, so think of these as the  
entrances for email into the system.

There are some general docs in mailman/docs which describe the data  
objects you'll be dealing with, such as users, addresses, mailing  
lists, etc.

Next, look at the mailman/pipeline/docs for how the message pipeline  
processor works, and at mailman/rules/docs for how the individual rule  
implementations work.

If that still doesn't help, let me know and I will attempt to add more  
doctests to fill in the gaps.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iEYEARECAAYFAkgw3FEACgkQ2YZpQepbvXGsBQCfaa37srmcZ5iQpo0NEra70TUX
8+YAoKjk1KmCb8qre6R5Ff8mTodLFDn5
=hE8+
-----END PGP SIGNATURE-----

From mark at msapiro.net  Mon May 19 03:48:31 2008
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 18 May 2008 18:48:31 -0700
Subject: [Mailman-Developers] Important Mailman 2.1.9 to 2.1.10 upgrade
 note.
In-Reply-To: <4830D7AE.6080307@msapiro.net>
References: <PC1760200804221144050468c671b4f9@msapiro>
	<C5C4A17B-EF8A-4EC9-9F6C-5A43F2B48D49@python.org>
	<4814CA2B.7050203@msapiro.net>
	<C7D7344D-87E4-4895-8715-AB4B7C9A413D@list.org>
	<481C8FA6.9080709@msapiro.net>
	<A51360EE-AA06-4C5F-903E-84A3987C3F72@list.org>
	<4830CA72.4060109@msapiro.net>
	<C98EF029-2BD7-4AA3-8587-F29A4C7F9A8E@list.org>
	<4830D7AE.6080307@msapiro.net>
Message-ID: <4830DC6F.4070206@msapiro.net>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Mark Sapiro wrote:
| Barry Warsaw wrote:
| |
| | What happens if you
| |
| | bzr branch http://bazaar.launchpad.net/~msapiro/mailman/cull_bad_shunt
| |
| | ?
| |
| | When I do this, either in my shared repository or in a fresh directory.
| | I get only 2 revisions.
|
|
| I'm doing this now in a new directory. I'll post the result after it
| completes.


It says it branched 2 revisions, but 'bzr diff -r 1073..1075' and 'bzr
diff -r 1..-1' produce what seems to be the expected output - at least
the 1073..1075 diff is correct and the 1..-1 diff produces the same
540077 lines as the one in the original MM_bzr/cull_bad_shunt directory.

- --
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)

iD8DBQFIMNxvVVuXXpU7hpMRAnrTAJ0cPJTv0lI3gp8pZ3gvBmQZmbs6agCcCJnX
ltyYyNUFozY2o/sQWjJ+4Ag=
=UvGk
-----END PGP SIGNATURE-----

From f.schubert at espresto.com  Tue May 20 18:21:27 2008
From: f.schubert at espresto.com (Frank Schubert)
Date: Tue, 20 May 2008 18:21:27 +0200
Subject: [Mailman-Developers] template patch for Cgi/confirm.py (was: webui
	templates or not?)
In-Reply-To: <4822F2E5.1040409@espresto.com>
References: <4822F2E5.1040409@espresto.com>
Message-ID: <4832FA87.3080008@espresto.com>

Hi,

I started with Cgi/confirm.py

I removed bad_confirmation and replaced it with a more general method (show_informational_message) that takes an optional argument.
The optional argument is a message that can be rendered for the web-user.

All directly rendered messages - outside the methods mentioned below - are changed to use the new method show_informational_message.

Because show_informational_message can be used when mlist is unavailable, it searches for a template-file in the default template directory with the default language. (e.g. /var/lib/mailman/templates/en)

The attached patch does *not* cover the following methods, because another template-file is needed for them:
* ask_for_cookie
* subscription_prompt
* unsubscription_prompt
* addrchange_prompt
* heldmsg_prompt
* reenable_prompt

I will cover these with the following patches.

It is my first work with python and Mailman and surly not ready for inclusion.
I wanted to get this out as fast as possible to have a base to work on - so please bear with me.

Bye,
Frank

-- 
Frank Schubert
Systemadministration
EsPresto AG
Breite Str. 30-31
10178 Berlin/Germany
Tel: +49.(0)30.90 226.750
Fax: +49.(0)30.90 226.760
f.schubert at espresto.com

HRB 77554 AG - Berlin-Charlottenburg
Vorstand: Alexander Biersack, Maya Biersack, Peter Biersack
Vorsitzender des Aufsichtsrats: Oli Kai Paulus
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: patch-cgi-confirm.py-replace-bad_confirmation.diff
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20080520/3122ca7f/attachment.txt>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20080520/3122ca7f/attachment.pgp>

From terri at zone12.com  Tue May 27 21:32:40 2008
From: terri at zone12.com (Terri Oda)
Date: Tue, 27 May 2008 15:32:40 -0400
Subject: [Mailman-Developers] Mailman FAQ: a call for help
Message-ID: <9F24FE12-8A65-4E1B-AFC6-68001D6C362D@zone12.com>

Ever wanted to help out with Mailman but didn't know how?   Here's  
your chance!

I've been trying to make the wiki into the one-stop-shop for Mailman  
documentation, and as part of that process, I made a script that  
moved the Mailman FAQ entries into the wiki.

http://wiki.list.org/display/DOC/Frequently+Asked+Questions

You'll see that there's around 200 entries in there now, organized  
into sections as they were originally in the FAQ Wizard:

http://www.python.org/cgi-bin/faqw-mm.py

I need some people who've got a few minutes to help out with cleaning  
up the wiki FAQ entries.  You don't need to know the answers -- I  
just need some volunteers who can fix some links and maybe do some  
organization.  I don't need any huge time commitment, just sign up  
for a wiki account and do an entry or two when you've got some spare  
time.
  	
Most of the pages just need someone to convert the FAQ Wizard links  
to internal wiki links (eg: in http://wiki.list.org/x/PIA9), maybe  
fix some other formatting (eg: look at http://wiki.list.org/x/p4A9  
and how the preformatted stuff is inexplicably in three blocks rather  
than one).    It's an easy job, it's just that at 200 pages, I could  
use some help!

If you have any questions or need any help getting started, please  
let me know!

  Terri

From barry at list.org  Tue May 27 22:32:02 2008
From: barry at list.org (Barry Warsaw)
Date: Tue, 27 May 2008 16:32:02 -0400
Subject: [Mailman-Developers] [Mailman-Users] Mailman FAQ: a call for
	help
In-Reply-To: <9F24FE12-8A65-4E1B-AFC6-68001D6C362D@zone12.com>
References: <9F24FE12-8A65-4E1B-AFC6-68001D6C362D@zone12.com>
Message-ID: <BD08AA0A-55FC-4D9E-91AC-7F53B92B173B@list.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On May 27, 2008, at 3:32 PM, Terri Oda wrote:

> Ever wanted to help out with Mailman but didn't know how?   Here's  
> your chance!
>
> I've been trying to make the wiki into the one-stop-shop for Mailman  
> documentation, and as part of that process, I made a script that  
> moved the Mailman FAQ entries into the wiki.
>
> http://wiki.list.org/display/DOC/Frequently+Asked+Questions

Terri, this is superfantastic!  Thanks so much for doing this.  Two  
things:

- - Does it make sense to turn off write permission on the FAQ wizard?   
If we don't, the wiki will slowly get out of date.  I think we can do  
this easily by changing the password and maybe adding some pointers to  
the wiki.

- - We should commit your script to the mailman-administrivia repository.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iEYEARECAAYFAkg8b8IACgkQ2YZpQepbvXFCEACfTIG7qCSv4AUB2aJMvGNBA3B3
WNoAoLnfA4vF+n9xvujeUHUsmt6qqsTV
=2Ilh
-----END PGP SIGNATURE-----