From tkikuchi at is.kochi-u.ac.jp  Sat Mar  3 07:13:24 2007
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Sat, 03 Mar 2007 15:13:24 +0900
Subject: [Mailman-Developers] How can I reload mlist data from SA database
Message-ID: <45E91204.9060909@is.kochi-u.ac.jp>

Hi Barry,

Since we at Japanese universities are in Spring vacation, I started 
testing 2.2 code on my PC.  What is curious about the LMTP/HTTP 
interfaces with SQLAlchemy based database is that changes commited in 
LMTP/HTTP sessions cannot be reflected on the other sessions.  If I 
subscribe an address via HTTP then it cannot be unsubscribed from LMTP 
and vise versa, until I restart the runners.  Perhaps we need a reload 
mechanism but I don't know how to implement it.

Other than that, I think I've checked with i18n plain text message 
deliveries for both regular and digest.  Still to go are mime multipart 
messages and archives.

I've also added a comment on Testing Mailman 2.2 wiki page how to test 
run the 2.2 code.

Cheers,

-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/


From sven at anderson.de  Mon Mar  5 03:24:33 2007
From: sven at anderson.de (Sven Anderson)
Date: Mon, 05 Mar 2007 03:24:33 +0100
Subject: [Mailman-Developers] Feature request: Reply-To Munging etc.
Message-ID: <45EB7F61.8050801@anderson.de>

Hi,

as many of you probably would agree, whether you munge the Reply-To headers 
or not, both ways are not perfect. Just today I had a hard time again, as 
I'm not happy with both options, and finally came up with some ideas how the 
situation could be improved, about which I would like to read your comments:

1) Receiver clean-up (if Reply-To munging is NOT used)
One of the problems not using Reply-To munging is, that by using the 
reply-to-all function of MUAs the list of receivers tends to accumulate. As 
in open lists the users cannot know, if an address is part of the list, they 
cannot just delete them without risking to drop people from the discussion. 
So what about an option to clean-up the receivers list in Mailman, that is 
Mailman removes all To/CC addresses which are members of the list?

2) Add non-member-senders to Reply-To (if Reply-To munging IS used)
One of the annoying things using Reply-To munging in open lists is, that you 
cannot easily include external authors in replies, whether you are using 
reply-to or reply-to-all in your MUA. It would be nice, if the sender could 
be added to the Reply-To addresses in the case, that the sender is not a 
member of the list, so that replies automatically go to the list AND the 
author. (Of course, this should be done only, if the sender didn't set his 
own Reply-To header already.)

3) Add sender to Reply-To (if Reply-To munging IS used)
As an variation to the proposal 2) it also would be nice to have the sender 
in general be added to the Reply-To header. That would make replies behave 
similar as the reply-to-all in case of no Reply-To munging. This makes it 
easier to reply only to the author, and not the whole list, by just deleting 
the list address while replying. Because you can also have address 
accumulation in that case, if users use the reply-to-all function, you could 
combine this option with the "receiver clean-up" option from proposal 1).

As proposal 3) doesn't need to know the member status of addresses, you can 
also easily implement it in the MTA, therefore it is the least important one.

What do you think about that? (With some help I can also try to implement 
this on my own, if there are chances to include it into the official Mailman.)


Cheers,

Sven

From jbglaw at lug-owl.de  Mon Mar  5 09:53:35 2007
From: jbglaw at lug-owl.de (Jan-Benedict Glaw)
Date: Mon, 5 Mar 2007 09:53:35 +0100
Subject: [Mailman-Developers] Feature request: Reply-To Munging etc.
In-Reply-To: <45EB7F61.8050801@anderson.de>
References: <45EB7F61.8050801@anderson.de>
Message-ID: <20070305085335.GR29565@lug-owl.de>

On Mon, 2007-03-05 03:24:33 +0100, Sven Anderson <sven at anderson.de> wrote:
> 1) Receiver clean-up (if Reply-To munging is NOT used)
> One of the problems not using Reply-To munging is, that by using the 
> reply-to-all function of MUAs the list of receivers tends to accumulate. As 
> in open lists the users cannot know, if an address is part of the list, they 
> cannot just delete them without risking to drop people from the discussion. 
> So what about an option to clean-up the receivers list in Mailman, that is 
> Mailman removes all To/CC addresses which are members of the list?

Depending on the system Mailman is installed on, the email going over
the list may take waaay longer time than the direrect mail. So the
active persons have a chance to rapidly answer, instead of waiting for
the list software.

That is, from my point of view, receiver filtering removes
functionality :-)

> 2) Add non-member-senders to Reply-To (if Reply-To munging IS used)
> One of the annoying things using Reply-To munging in open lists is, that you 
> cannot easily include external authors in replies, whether you are using 
> reply-to or reply-to-all in your MUA. It would be nice, if the sender could 
> be added to the Reply-To addresses in the case, that the sender is not a 
> member of the list, so that replies automatically go to the list AND the 
> author. (Of course, this should be done only, if the sender didn't set his 
> own Reply-To header already.)

With persons like me that either do strict to-list-only or
group-replys (depending on the actual audience), this would probably
end up with Cc'ed non-subscribers getting mails twice? So here we
depend on the MUA's intelligence to filter out doubled email
addresses.


But maybe I'm just too used to old habits, though :)

MfG, JBG

-- 
      Jan-Benedict Glaw      jbglaw at lug-owl.de              +49-172-7608481
Signature of:              What we do for ourselves dies with us. What we do for
the second  :         others and the world remains and is immortal. (Albert Pine)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20070305/f4b03a65/attachment.pgp 

From stephen at xemacs.org  Tue Mar  6 08:25:13 2007
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 06 Mar 2007 16:25:13 +0900
Subject: [Mailman-Developers]  Feature request: Reply-To Munging etc.
In-Reply-To: <45EB7F61.8050801@anderson.de>
References: <45EB7F61.8050801@anderson.de>
Message-ID: <87fy8ial5y.fsf@uwakimon.sk.tsukuba.ac.jp>

Sven Anderson writes:

 > 1) Receiver clean-up (if Reply-To munging is NOT used)
 > So what about an option to clean-up the receivers list in Mailman,
 > that is Mailman removes all To/CC addresses which are members of
 > the list?

IIRC, if the user sets their subscription to "no-dupes", that user's
address will be removed from the addressee list, as well as from the
list of addresses that Mailman actually distributes to the post to.

Rather than have this be a list-level option, I would say that you
should set the default for new subscribers to no-dupes.  (This may
also require a patch to Mailman for convenient administration.)

 > 2) Add non-member-senders to Reply-To (if Reply-To munging IS used)
 > One of the annoying things using Reply-To munging in open lists is,
 > that you cannot easily include external authors in replies, whether
 > you are using reply-to or reply-to-all in your MUA. It would be
 > nice, if the sender could be added to the Reply-To addresses in the
 > case, that the sender is not a member of the list, so that replies
 > automatically go to the list AND the author.

This is simply not enough.  You need to add all the non-list
addressees, too.  That is equivalent to (1).

 > 3) Add sender to Reply-To (if Reply-To munging IS used) As an
 > variation to the proposal 2) it also would be nice to have the
 > sender in general be added to the Reply-To header. That would make
 > replies behave similar as the reply-to-all in case of no Reply-To
 > munging.

In other words, you are suggesting that Mailman should munge Reply-To
simply because some users don't know where the reply-to-all function
is.

I realize that non-technical users are human beings and have their
rights, but it's the MUA vendors, not Mailman, who violate those
rights by supplying broken software.

If you have a list where the majority of users are so non-technical
that they can't find reply-to-all, just munge and don't worry about
it.


From sven at anderson.de  Tue Mar  6 14:33:01 2007
From: sven at anderson.de (Sven Anderson)
Date: Tue, 06 Mar 2007 14:33:01 +0100
Subject: [Mailman-Developers] Feature request: Reply-To Munging etc.
In-Reply-To: <87fy8ial5y.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <45EB7F61.8050801@anderson.de>
	<87fy8ial5y.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <45ED6D8D.1000601@anderson.de>

Stephen J. Turnbull, 06.03.2007 08:25:
> Sven Anderson writes:
> 
>  > 1) Receiver clean-up (if Reply-To munging is NOT used)
>  > So what about an option to clean-up the receivers list in Mailman,
>  > that is Mailman removes all To/CC addresses which are members of
>  > the list?
> 
> IIRC, if the user sets their subscription to "no-dupes", that user's
> address will be removed from the addressee list, as well as from the
> list of addresses that Mailman actually distributes to the post to.

I just checked this to be sure, but only the latter is true. The user's
address stays in the To/Cc header.

And anyway, that's not what I want. I want an option so that the addressee
list is cleaned up, no matter if the users wants duplicates or not. That's
because addressee accumulation is the most heard argument against
reply-to-all usage, which again is needed to live without Reply-To
munging. I claim, that such an option would be very useful not just for me
but for many Mailman administrators, especially for those who don't want
to munge the Reply-To header.

>  > 2) Add non-member-senders to Reply-To (if Reply-To munging IS used)
>  > One of the annoying things using Reply-To munging in open lists is,
>  > that you cannot easily include external authors in replies, whether
>  > you are using reply-to or reply-to-all in your MUA. It would be
>  > nice, if the sender could be added to the Reply-To addresses in the
>  > case, that the sender is not a member of the list, so that replies
>  > automatically go to the list AND the author.
> 
> This is simply not enough.  You need to add all the non-list
> addressees, too.  That is equivalent to (1).

No, that is not true, as the "reply-to-all" function includes all the
addressees already, but the From address is _never_ included, if the
Reply-To is set, neither with simple reply nor with reply-to-all. I'm just
addressing this problem with my proposal, that - if Reply-To is set - the
>From address is not included in _any_ case with a standard conform MUA.

Anyway, I refined my ideas in the meantime, and I will try to write them
down in a further email tonight.


Cheers,

Sven

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 4840 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20070306/f0f51d4d/attachment.bin 

From stephen at xemacs.org  Wed Mar  7 18:30:12 2007
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 08 Mar 2007 02:30:12 +0900
Subject: [Mailman-Developers] Feature request: Reply-To Munging etc.
In-Reply-To: <45ED6D8D.1000601@anderson.de>
References: <45EB7F61.8050801@anderson.de>
	<87fy8ial5y.fsf@uwakimon.sk.tsukuba.ac.jp>
	<45ED6D8D.1000601@anderson.de>
Message-ID: <87fy8hdkrf.fsf@uwakimon.sk.tsukuba.ac.jp>

Sven Anderson writes:

 > > IIRC, if the user sets their subscription to "no-dupes", that user's
 > > address will be removed from the addressee list, as well as from the
 > > list of addresses that Mailman actually distributes to the post to.
 > 
 > I just checked this to be sure, but only the latter is true. The user's
 > address stays in the To/Cc header.

Hm.  I know both are true for my lists (Mailman 2.1.5).

 > And anyway, that's not what I want. I want an option so that the addressee
 > list is cleaned up, no matter if the users wants duplicates or not.

Ah.  You don't care what your users want.  I can't argue with that.





From sven at anderson.de  Thu Mar  8 15:39:15 2007
From: sven at anderson.de (Sven Anderson)
Date: Thu, 08 Mar 2007 15:39:15 +0100
Subject: [Mailman-Developers] Feature request: Reply-To Munging etc.
In-Reply-To: <87fy8hdkrf.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <45EB7F61.8050801@anderson.de>	<87fy8ial5y.fsf@uwakimon.sk.tsukuba.ac.jp>	<45ED6D8D.1000601@anderson.de>
	<87fy8hdkrf.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <45F02013.4010606@anderson.de>

Stephen J. Turnbull, 07.03.2007 18:30:
> Sven Anderson writes:
> 
>  > > IIRC, if the user sets their subscription to "no-dupes", that user's
>  > > address will be removed from the addressee list, as well as from the
>  > > list of addresses that Mailman actually distributes to the post to.
>  > 
>  > I just checked this to be sure, but only the latter is true. The user's
>  > address stays in the To/Cc header.
> 
> Hm.  I know both are true for my lists (Mailman 2.1.5).

Mine is 2.1.6.

>  > And anyway, that's not what I want. I want an option so that the addressee
>  > list is cleaned up, no matter if the users wants duplicates or not.
> 
> Ah.  You don't care what your users want.  I can't argue with that.

Of course I care, that _is_ what _my_ users want. You have to distinguish
between the option if the user wants to receive duplicates (what he/she
still can choose) and the proposed list-wide option, if addresses of list
members are removed from the To/Cc headers (not from the distibution list
though!) If the users chooses to receive duplicates, he still gets them in
form of a Bcc.


Cheers,

Sven
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 4840 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20070308/a0e2d64f/attachment.bin 

From barry at python.org  Thu Mar  8 16:03:38 2007
From: barry at python.org (Barry Warsaw)
Date: Thu, 8 Mar 2007 10:03:38 -0500
Subject: [Mailman-Developers] Feature request: Reply-To Munging etc.
In-Reply-To: <87fy8hdkrf.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <45EB7F61.8050801@anderson.de>
	<87fy8ial5y.fsf@uwakimon.sk.tsukuba.ac.jp>
	<45ED6D8D.1000601@anderson.de>
	<87fy8hdkrf.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <6229DA37-4DA2-46B2-80EB-96D9CE799D8A@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 7, 2007, at 12:30 PM, Stephen J. Turnbull wrote:

> Sven Anderson writes:
>
>>> IIRC, if the user sets their subscription to "no-dupes", that user's
>>> address will be removed from the addressee list, as well as from the
>>> list of addresses that Mailman actually distributes to the post to.
>>
>> I just checked this to be sure, but only the latter is true. The  
>> user's
>> address stays in the To/Cc header.
>
> Hm.  I know both are true for my lists (Mailman 2.1.5).

I think you're both right :).  Mailman 2.1 will strip recipients from  
the CC header if explicitly named recipients are members of the list  
and have nodup set.  But Mailman won't strip To headers, nor juggle  
CC and To headers after stripping.

It may not be a bad idea to go all the way with this as Sven  
suggests, at least when not munging.  I do think it will address a  
common complaint about no-munge, and one that's harder to write off  
as "fix your MUA".

I'm not inclined to spend much brain power or LoC trying to "fix"  
reply-to-munging.  I think that's a fundamentally broken use case.   
Good MUAs do the right thing here and if Sven's idea addresses the  
other major complaint then I'd be happy.  If I could drop reply-to  
munging in the next version altogether, I'd be ecstatic.

Thanks, I've added this to the wiki.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRfAly3EjvBPtnXfVAQJrLwP/Sk9LbmFfG/hDJ5RddGYpGMdi6YMfFpyl
1/zGj2EeqCThziP0cC+zOMkY7QgdDaQkt8zgOQ1jLsv1UVBEOJ1uxm5Y3JZ/KzDi
W5ZE1KGIEs/X8vDctSsUPMhYNTa9yUnVEBCjatxJ/S4c2dT311/gYtb9NgmWhoux
jfq8Nqd7QSY=
=RjoH
-----END PGP SIGNATURE-----

From stephen at xemacs.org  Fri Mar  9 12:40:45 2007
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 09 Mar 2007 20:40:45 +0900
Subject: [Mailman-Developers] Feature request: Reply-To Munging etc.
In-Reply-To: <6229DA37-4DA2-46B2-80EB-96D9CE799D8A@python.org>
References: <45EB7F61.8050801@anderson.de>
	<87fy8ial5y.fsf@uwakimon.sk.tsukuba.ac.jp>
	<45ED6D8D.1000601@anderson.de>
	<87fy8hdkrf.fsf@uwakimon.sk.tsukuba.ac.jp>
	<6229DA37-4DA2-46B2-80EB-96D9CE799D8A@python.org>
Message-ID: <87slced4qq.fsf@uwakimon.sk.tsukuba.ac.jp>

Barry Warsaw writes:

 > I think you're both right :).  Mailman 2.1 will strip recipients from  
 > the CC header if explicitly named recipients are members of the list  
 > and have nodup set.  But Mailman won't strip To headers, nor juggle  
 > CC and To headers after stripping.

Ah, right.  I rarely see multiple addresses in the To header from @AOL
though, so this should be sufficient.  I have never seen multiple
addresses in From (except in the examples section of RFC 822 :-), so
that means that at most one will leak per reply.

 > It may not be a bad idea to go all the way with this as Sven  
 > suggests, at least when not munging.  I do think it will address a  
 > common complaint about no-munge, and one that's harder to write off  
 > as "fix your MUA".

I see your point, but why won't my suggestion of defaulting the per
user no-dupes to "on" do fine from that point of view?

Thing is, my experience has been that people who care much about their
personal setting invariably want it *on* (the ones who care and want
it off use procmail anyway; those who can't or won't use procmail
chalk it off as Insh'allah).  That's neither here nor there, except
that they would be very unhappy if the option were administratively
removed (and I would be unhappy too, "ben" is "one of *them*", and
what if "jwz" or "mly" were?  Ouch!)

I think it would be reasonable to remove *all* no-dupes members, and
if the addressees end up empty, then put the list address in To: (or
does Mailman already force the list to appear explicitly ... I guess
not, since that's one of the addressee filters).


From stephen at xemacs.org  Fri Mar  9 14:51:03 2007
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 09 Mar 2007 22:51:03 +0900
Subject: [Mailman-Developers] Feature request: Reply-To Munging etc.
In-Reply-To: <45F154EE.8090905@thecsl.org>
References: <45EB7F61.8050801@anderson.de>
	<87fy8ial5y.fsf@uwakimon.sk.tsukuba.ac.jp>
	<45ED6D8D.1000601@anderson.de>
	<87fy8hdkrf.fsf@uwakimon.sk.tsukuba.ac.jp>
	<6229DA37-4DA2-46B2-80EB-96D9CE799D8A@python.org>
	<87slced4qq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<45F154EE.8090905@thecsl.org>
Message-ID: <87mz2mcypk.fsf@uwakimon.sk.tsukuba.ac.jp>

Dan MacNeil writes:

 > Stephen J. Turnbull quoted out of context:
 > > I see your point, but why won't my suggestion of defaulting the per
 > > user no-dupes to "on" do fine from that point of view?

 > For my selfish purposes, "no-dupes" off is a better default setting.
 > Without a copy of the message to their mailbox, many of my users will
 > keep sending the same message because it "didn't go".

Are you thinking of not-metoo?


From dan at thecsl.org  Fri Mar  9 13:37:02 2007
From: dan at thecsl.org (Dan MacNeil)
Date: Fri, 09 Mar 2007 07:37:02 -0500
Subject: [Mailman-Developers] Feature request: Reply-To Munging etc.
In-Reply-To: <87slced4qq.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <45EB7F61.8050801@anderson.de>	<87fy8ial5y.fsf@uwakimon.sk.tsukuba.ac.jp>	<45ED6D8D.1000601@anderson.de>	<87fy8hdkrf.fsf@uwakimon.sk.tsukuba.ac.jp>	<6229DA37-4DA2-46B2-80EB-96D9CE799D8A@python.org>
	<87slced4qq.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <45F154EE.8090905@thecsl.org>

Stephen J. Turnbull quoted out of context:
> I see your point, but why won't my suggestion of defaulting the per
> user no-dupes to "on" do fine from that point of view?

For my selfish purposes, "no-dupes" off is a better default setting.
Without a copy of the message to their mailbox, many of my users will
keep sending the same message because it "didn't go".


From msapiro at value.net  Fri Mar  9 18:07:47 2007
From: msapiro at value.net (Mark Sapiro)
Date: Fri, 9 Mar 2007 09:07:47 -0800
Subject: [Mailman-Developers] Feature request: Reply-To Munging etc.
In-Reply-To: <87mz2mcypk.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <PC1760200703090907470109b8e1aefc@msapiro>

Stephen J. Turnbull wrote:

>Dan MacNeil writes:
>
> > Stephen J. Turnbull quoted out of context:
> > > I see your point, but why won't my suggestion of defaulting the per
> > > user no-dupes to "on" do fine from that point of view?
>
> > For my selfish purposes, "no-dupes" off is a better default setting.
> > Without a copy of the message to their mailbox, many of my users will
> > keep sending the same message because it "didn't go".
>
>Are you thinking of not-metoo?


I think Dan must be thinking of not-metoo, but note that even ensuring
that not-metoo is off won't do the job for gmail users thanks to
gmail's 'feature' of not showing 'duplicate' messages.

BTW, just for the record, the Defaults.py setting for
new_member_options for a new list (DEFAULT_NEW_MEMBER_OPTIONS) is
no-dupes.

-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From stephen at xemacs.org  Fri Mar  9 18:29:16 2007
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 10 Mar 2007 02:29:16 +0900
Subject: [Mailman-Developers] Feature request: Reply-To Munging etc.
In-Reply-To: <PC1760200703090907470109b8e1aefc@msapiro>
References: <87mz2mcypk.fsf@uwakimon.sk.tsukuba.ac.jp>
	<PC1760200703090907470109b8e1aefc@msapiro>
Message-ID: <87hcsucolv.fsf@uwakimon.sk.tsukuba.ac.jp>

Mark Sapiro writes:

 > I think Dan must be thinking of not-metoo, but note that even ensuring
 > that not-metoo is off won't do the job for gmail users thanks to
 > gmail's 'feature' of not showing 'duplicate' messages.

Yeah, but that's irrelevant to use of no-dupes, because gmail is
keying off a locally saved copy as I understand it.




From barry at python.org  Fri Mar  9 23:00:25 2007
From: barry at python.org (Barry Warsaw)
Date: Fri, 9 Mar 2007 17:00:25 -0500
Subject: [Mailman-Developers] How can I reload mlist data from SA
	database
In-Reply-To: <45E91204.9060909@is.kochi-u.ac.jp>
References: <45E91204.9060909@is.kochi-u.ac.jp>
Message-ID: <52CF95C1-DCE9-42BC-B39B-5783D60FD729@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 3, 2007, at 1:13 AM, Tokio Kikuchi wrote:

> Hi Barry,

Hi Tokio,

Man, I don't know what happened but I just got /plastered/ with email  
this week.  Between that and work, I haven't even touched the code. :/

> Since we at Japanese universities are in Spring vacation, I started
> testing 2.2 code on my PC.  What is curious about the LMTP/HTTP
> interfaces with SQLAlchemy based database is that changes commited in
> LMTP/HTTP sessions cannot be reflected on the other sessions.  If I
> subscribe an address via HTTP then it cannot be unsubscribed from LMTP
> and vise versa, until I restart the runners.  Perhaps we need a reload
> mechanism but I don't know how to implement it.

I think I've noticed a similar problem.  I think it has to do with  
concurrency between processes using SQLAlchemy.  As you said, it  
seems like we need an explicit "reload" but I'm really hoping that  
that's just because we're not doing something right.  I haven't had  
time to figure it out yet, or to contact the SA mailing list to get  
their feedback.  We definitely need to address that issue before we  
get too far down the SA road.

> Other than that, I think I've checked with i18n plain text message
> deliveries for both regular and digest.  Still to go are mime  
> multipart
> messages and archives.
>
> I've also added a comment on Testing Mailman 2.2 wiki page how to test
> run the 2.2 code.

Thanks, and I saw your recent patches.  One thing we should be extra  
careful about is to include unit tests for anything we fix.  I really  
want to improve our test suite so having test that cover the things  
you change are really important.  I'm also trying to be very diligent  
about keeping the test suite working.  Eventually I'd like to  
integrate our stuff with one of the Python community buildbots.

Thanks, and I hope you haven't wasted your entire spring vacation  
waiting for me to respond! :)

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRfHY+nEjvBPtnXfVAQK7GQP9FgHP8VkbuRQYcPSYmC7m8J3OPAsCEUe7
qdeYCdiJk9EIaOxzUJKWT6KrkRsBTs6Rj9LbPtyR9ZwP4xpW838eMakCoRGaFUCr
veJY7J7zVtF26igcwytAJcb5LID/FRJpCIylJxHs9WQwLgJuypRyoqSRAOSgnwkQ
FBFOWpPY1UA=
=+UnM
-----END PGP SIGNATURE-----

From tkikuchi at is.kochi-u.ac.jp  Sat Mar 10 07:07:15 2007
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Sat, 10 Mar 2007 15:07:15 +0900
Subject: [Mailman-Developers] How can I reload mlist data from
	SA	database
In-Reply-To: <52CF95C1-DCE9-42BC-B39B-5783D60FD729@python.org>
References: <45E91204.9060909@is.kochi-u.ac.jp>
	<52CF95C1-DCE9-42BC-B39B-5783D60FD729@python.org>
Message-ID: <45F24B13.4050604@is.kochi-u.ac.jp>

Barry,

>> Other than that, I think I've checked with i18n plain text message
>> deliveries for both regular and digest.  Still to go are mime  
>> multipart
>> messages and archives.
>>
>> I've also added a comment on Testing Mailman 2.2 wiki page how to test
>> run the 2.2 code.
> 
> Thanks, and I saw your recent patches.  One thing we should be extra  
> careful about is to include unit tests for anything we fix.  I really  
> want to improve our test suite so having test that cover the things  
> you change are really important.  I'm also trying to be very diligent  
> about keeping the test suite working.  Eventually I'd like to  
> integrate our stuff with one of the Python community buildbots.

Ok, I will try to contribute test codes as well as the i18n fixes in the 
main codes.  BTW, I realized I might have messed code and some tests 
fail.  Do you want password scheme to be called by ENUM or TAG.  I saw 
the default scheme in Defaults.py is string and changed to be called by 
TAG but it made test_passwords.py to fail.  I think we can change the 
test code like so:
class TestPBKDF2Passwords(TestPasswordsBase):
     scheme = 'pbkdf2'

Thoughts?

> 
> Thanks, and I hope you haven't wasted your entire spring vacation  
> waiting for me to respond! :)

Never mind.  The new semester starts in April. :)

-- 
Tokio Kikuchi tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From sven at anderson.de  Sat Mar 10 17:30:40 2007
From: sven at anderson.de (Sven Anderson)
Date: Sat, 10 Mar 2007 17:30:40 +0100
Subject: [Mailman-Developers] Feature request: Reply-To Munging etc.
In-Reply-To: <6229DA37-4DA2-46B2-80EB-96D9CE799D8A@python.org>
References: <45EB7F61.8050801@anderson.de>
	<87fy8ial5y.fsf@uwakimon.sk.tsukuba.ac.jp>
	<45ED6D8D.1000601@anderson.de>
	<87fy8hdkrf.fsf@uwakimon.sk.tsukuba.ac.jp>
	<6229DA37-4DA2-46B2-80EB-96D9CE799D8A@python.org>
Message-ID: <45F2DD30.8010505@anderson.de>

Barry Warsaw wrote:
> I think you're both right :).  Mailman 2.1 will strip recipients from 
> the CC header if explicitly named recipients are members of the list and 
> have nodup set.  But Mailman won't strip To headers, nor juggle CC and 
> To headers after stripping.

Is see. But wouldn't it be better, that the stripping is controlled by an 
list wide option or (if individual mails are generated anyway, for VERP 
etc.) by the receiving user, and not by the owner of the address? I would 
separate the address stripping completely from the no-dups option.

> It may not be a bad idea to go all the way with this as Sven suggests, 
> at least when not munging.  I do think it will address a common 
> complaint about no-munge, and one that's harder to write off as "fix 
> your MUA".

Just make it an "addressee clean-up" option, then the list admin or the 
receiving user can choose, if he/she wants it or not.

> I'm not inclined to spend much brain power or LoC trying to "fix" 
> reply-to-munging.  I think that's a fundamentally broken use case.  Good 
> MUAs do the right thing here and if Sven's idea addresses the other 
> major complaint then I'd be happy.  If I could drop reply-to munging in 
> the next version altogether, I'd be ecstatic.

Well, as I'm using this "broken use case" on almost every list, and me and 
my users are happy with this (and that's all that counts for me), I clearly 
vote against dropping it. ;-) I most probably would still use it, even if 
the  proposed addressee clean-up would exist. But at the moment the Mailman 
reply-to-munging is not fundamentally but specifically broken. ;-)

I will explain my refined proposal in a separate mail.


Cheers,

Sven

From dan at thecsl.org  Fri Mar  9 15:54:12 2007
From: dan at thecsl.org (Dan MacNeil)
Date: Fri, 09 Mar 2007 09:54:12 -0500
Subject: [Mailman-Developers] Feature request: Reply-To Munging etc.
In-Reply-To: <87mz2mcypk.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <45EB7F61.8050801@anderson.de>	<87fy8ial5y.fsf@uwakimon.sk.tsukuba.ac.jp>	<45ED6D8D.1000601@anderson.de>	<87fy8hdkrf.fsf@uwakimon.sk.tsukuba.ac.jp>	<6229DA37-4DA2-46B2-80EB-96D9CE799D8A@python.org>	<87slced4qq.fsf@uwakimon.sk.tsukuba.ac.jp>	<45F154EE.8090905@thecsl.org>
	<87mz2mcypk.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <45F17514.2000800@thecsl.org>

> Dan MacNeil writes:
>  > For my selfish purposes, "no-dupes" off is a better default setting.
>  > Without a copy of the message to their mailbox, many of my users will
>  > keep sending the same message because it "didn't go".

Stephen J. Turnbull wrote:
> Are you thinking of not-metoo?

Err, yes, I am. Sorry for the distraction.


From sven at anderson.de  Sat Mar 10 19:00:15 2007
From: sven at anderson.de (Sven Anderson)
Date: Sat, 10 Mar 2007 19:00:15 +0100
Subject: [Mailman-Developers] Advanced Reply-To-Munging
Message-ID: <45F2F22F.8060403@anderson.de>

Hi all,

here's finally my next proposal for a less broken but yet very useful 
reply-to-munging:

- Motivation

Non-technical users (generally) love defaults. They don't wanna have to 
choose between three different kind of reply buttons. And no-way they want 
to change their MUA they just got painfully used to (or even their mail 
address, as most of them are using webmail providers), just because of a 
stupid mailing list. So there's a user base who are mostly using webmailers 
and Outlook, and who just want to press "Reply" as they usually do, no 
matter if there is even a "Reply-To-All" button right next to it.

Although this is a big pain for us geeks, I think a great software like 
Mailman should offer options (not default behavior) to serve such a user 
base. That's why I think, that Mailman should offer an option to set the 
default behavior of the standard reply button. The question what the default 
behavior of the reply button should be has to be discussed in each lists 
user base, Mailman should offer the choice. Many list admins are very 
thankful, if they don't have to educate their users.

- Proposed solution

There is no RFC-way to set the default address for replies. The Reply-To 
header _replaces_ the From address in replies, which is something else than 
setting a default. That's why setting the Reply-To completely hides the From 
address (regarding replies) and therefore abusing it as a default breaks the 
reply-to-all function for instance, as it doesn't include the From address 
neither (which is correct, as long as the Reply-To address is an 
_replacement_ for the From address). Since there is no other way but to 
abuse the Reply-To header to control the default reply behavior, 
reply-to-munging has to take care of the From address too.

There are four possible default reply targets that I consider as useful and 
should be offered at least as a list-wide option. (If individual mails are 
sent out, like with VERP, a per-user option is the best of course.)

1. Author
2. List
3. All recipients and author
4. Explicit address

And this is, what I propose that Mailman should do in these cases:

ad 1) Author is the "default default target" ;-). So no munging needed at 
all, no problems.

ad 2) If Reply-To is already set, it is removed. Reply-To is set to the 
lists address. The old Reply-To or - if not existing - the From address is 
checked if it is a list member. If not, it is added as a "fake Cc" to the Cc 
header, in order to make the reply-to-all function work.

ad 3) If Reply-To is already set, it is removed. Reply-To is set to:
  - the lists address,
  - all addresses in To/Cc headers, that are no list members, and
  - the old Reply-To or - if not existing - the From address, if not a list 
member

ad 4) If Reply-To is already set, it is removed. Reply-To is set to the 
explicit address. The old Reply-To or - if not existing - the From address 
is checked if it is a list member. If not, it is added as a "fake Cc" to the 
Cc header, in order to make the reply-to-all function work.

Of course, with option 2-4 you still lose the reply-to-author function, but 
at least for 2 and 4 in exchange for a new function, which a standard MUA 
with just a reply and reply-to-all button doesn't offer.

Example: Me and two colleagues are the management board of a local radio 
station. We use Mailman as a "Deluxe-Alias" for both communication among us 
and as Alias for people who want to contact us. So a lot of mails on this 
list come from non-members. In these cases I either want to answer only to 
the list (my colleagues) or to the author and the list, but _never_ to the 
author alone. So it's obvious that in this case we would like to have the 
reply button going to the list, and reply-to-all going to all (including the 
author, what at the moment doesn't work). We want the default going to the 
list, not only because of our MUAs (Thunderbird, Mutt, Webmailer) only Mutt 
supports reply-to-list, but also because we want the default reply to be the 
least dangerous, which in our case is the list. Funnily enough, the mutt 
user wants the most, that the normal reply is going to the list. ;-)

I hope I could make clear my ideas and point of view.


Cheers,

Sven


From stephen at xemacs.org  Sun Mar 11 19:34:26 2007
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 12 Mar 2007 03:34:26 +0900
Subject: [Mailman-Developers]  Advanced Reply-To-Munging
In-Reply-To: <45F2F22F.8060403@anderson.de>
References: <45F2F22F.8060403@anderson.de>
Message-ID: <877itnd3yl.fsf@uwakimon.sk.tsukuba.ac.jp>

Sven Anderson writes:

 > There is no RFC-way to set the default address for replies.

Of course there is.  For authors, it's Reply-To.  For lists, it's
List-Post.  There is nothing that says the MUA can't offer other
choices; just the that MUA should offer the address list in Reply-To
to the user by default because *that's what the author is explicitly
requesting*.

The fact that the MUAs used by the mob don't offer other choices is a
completely different issue.

 > ad 2) If Reply-To is already set, it is removed.

That's definitely a violation of the RFC.  The author has explicitly
requested delivery of responses to that address, and this proposal
prevents it from working correctly.

Reply-To, like the other headers, accepts multiple addresses.  The
list's address should be added in this case, and users who bitch
should be told that it's required by "the rules" since it was already
present.

 > The old Reply-To or - if not existing - the From address is checked
 > if it is a list member. If not, it is added as a "fake Cc" to the
 > Cc header, in order to make the reply-to-all function work.

This is insufficient.  Somebody who actually sets Reply-To to an
address that's on the list in a post to the list clearly Really Means
It.  It's not my place to explain why; you should ask *them*.

Your assumption is that such users don't exist on the list in
question, at least not in numbers large enough to matter.  Why not let
them have their way?

 > Of course, with option 2-4 you still lose the reply-to-author function, but 
 > at least for 2 and 4 in exchange for a new function, which a standard MUA 
 > with just a reply and reply-to-all button doesn't offer.

It's not a new function, it's a new bug.  Much of the specification
for MUAs is in RFC 2822, including the use of the Reply-To function.
Your proposal prevents conforming MUAs from performing the function
that their users ask of them.

 > author, what at the moment doesn't work). We want the default going to the 
 > list, not only because of our MUAs (Thunderbird, Mutt, Webmailer) only Mutt 
 > supports reply-to-list, but also because we want the default reply to be the 
 > least dangerous, which in our case is the list. Funnily enough, the mutt 
 > user wants the most, that the normal reply is going to the list. ;-)

This is a valid use case, but not for Mailman as currently designed.
What you want is customer relations management software.  If you
really want this, then you can write a custom handler for it.  It
should not be provided with software intended for general use in
discussion lists on the Internet IMO.


From sven at anderson.de  Wed Mar 14 16:51:44 2007
From: sven at anderson.de (Sven Anderson)
Date: Wed, 14 Mar 2007 16:51:44 +0100
Subject: [Mailman-Developers] Advanced Reply-To-Munging
In-Reply-To: <877itnd3yl.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <45F2F22F.8060403@anderson.de>
	<877itnd3yl.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <45F81A10.50609@anderson.de>

Hi Stephen,

thank you for reading and commenting my proposal.

Stephen J. Turnbull, 11.03.2007 19:34:
> Sven Anderson writes:
> 
>  > There is no RFC-way to set the default address for replies.
> 
> Of course there is.  For authors, it's Reply-To.  For lists, it's
> List-Post.

No, it's not. What I was referring here as the "default address" (and I
hoped this was clear) was the address, which is used by the normal reply
button/function of the MUA. There is no way to define a default adressee
for that function, it is always hardwired to the From header, which can be
overridden by a Reply-To header. That's something different, as the
Reply-To overrides the From for _any_ reply function, that is also the
reply-to-all function for instance. There is no RFC way to replace the
>From address only in the "normal" reply, but not in any other reply functions.

> The fact that the MUAs used by the mob don't offer other choices is a
> completely different issue.

You complained, that I don't care what my users want, and at the same time
you call them "mob"? ;-)

>  > ad 2) If Reply-To is already set, it is removed.
> 
> That's definitely a violation of the RFC.  The author has explicitly
> requested delivery of responses to that address, and this proposal
> prevents it from working correctly.

I don't agree. Reply-To tells, which address you should use _if_ you want
to contact that user. Reply-To is not meant to force you to _whom_ you
want to reply to. That's why my proposal replaces the From by an
author-set Reply-To whenever it is used.

> Reply-To, like the other headers, accepts multiple addresses.  The
> list's address should be added in this case, and users who bitch
> should be told that it's required by "the rules" since it was already
> present.

Which rule? A list processor is not an MTA. And don't forget, we are
talking about an _option_. List admins, who want to treat their users like
this, don't have to (and won't) switch on reply-to-munging.

>  > The old Reply-To or - if not existing - the From address is checked
>  > if it is a list member. If not, it is added as a "fake Cc" to the
>  > Cc header, in order to make the reply-to-all function work.
> 
> This is insufficient.  Somebody who actually sets Reply-To to an
> address that's on the list in a post to the list clearly Really Means
> It.  It's not my place to explain why; you should ask *them*.
> 
> Your assumption is that such users don't exist on the list in
> question, at least not in numbers large enough to matter.  Why not let
> them have their way?

A user sending a mail to a list processor cannot expect, that his headers
will be kept in any case. A list processor is munging anyway in many
aspects. The only clean solution here (also in respect to digital
signatures for instance) would be to nest the complete email including
headers into a new email. Then you have to levels of headers, and
everything is kept.

I think the interests of the users to have a mailing list, which behaves
like they think it's the most practical way, outweigh the interest of
single users, who are not happy, that their Reply-To is touched. And I
bet, that the latter is not true for most users, who set the Reply-To, as
my proposal regards their Reply-To in the way it is usually meant.

And again, it's an option. Let the list admins choose.

>  > Of course, with option 2-4 you still lose the reply-to-author function, but 
>  > at least for 2 and 4 in exchange for a new function, which a standard MUA 
>  > with just a reply and reply-to-all button doesn't offer.
> 
> It's not a new function, it's a new bug.  Much of the specification
> for MUAs is in RFC 2822, including the use of the Reply-To function.
> Your proposal prevents conforming MUAs from performing the function
> that their users ask of them.

No. The users didn't ask for "if the receiving users presses 'Reply', the
mail should go to whatever I put into Reply-To". They ask: "If you want to
reach me, don't use the From address, use the Reply-To instead." And this
is done by my proposal.

>  > author, what at the moment doesn't work). We want the default going to the 
>  > list, not only because of our MUAs (Thunderbird, Mutt, Webmailer) only Mutt 
>  > supports reply-to-list, but also because we want the default reply to be the 
>  > least dangerous, which in our case is the list. Funnily enough, the mutt 
>  > user wants the most, that the normal reply is going to the list. ;-)
> 
> This is a valid use case, but not for Mailman as currently designed.
> What you want is customer relations management software.  If you
> really want this, then you can write a custom handler for it.  It
> should not be provided with software intended for general use in
> discussion lists on the Internet IMO.

I don't get the point in referring to another software, while Mailman
offers 95% of what that use case needs and is installed already. All these
fuzzy classifications like CRM shouldn't prevent a software to cross the
border, if it easily can do.


Regards,

Sven

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 4840 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20070314/2b29dc78/attachment.bin 

From stephen at xemacs.org  Thu Mar 15 17:19:14 2007
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 16 Mar 2007 01:19:14 +0900
Subject: [Mailman-Developers] Advanced Reply-To-Munging
In-Reply-To: <45F81A10.50609@anderson.de>
References: <45F2F22F.8060403@anderson.de>
	<877itnd3yl.fsf@uwakimon.sk.tsukuba.ac.jp>
	<45F81A10.50609@anderson.de>
Message-ID: <87mz2eo4xp.fsf@uwakimon.sk.tsukuba.ac.jp>

Sven Anderson writes:

 > Stephen J. Turnbull, 11.03.2007 19:34:
 > > Sven Anderson writes:
 > > 
 > >  > There is no RFC-way to set the default address for replies.
 > > 
 > > Of course there is.  For authors, it's Reply-To.  For lists, it's
 > > List-Post.
 > 
 > No, it's not. What I was referring here as the "default address" (and I
 > hoped this was clear) was the address, which is used by the normal reply
 > button/function of the MUA. There is no way to define a default adressee
 > for that function, it is always hardwired to the From header, which can be
 > overridden by a Reply-To header.

Let me note that that's false for all of the dozen or so different
(Emacs-based, so not usable for the masses, but proof of concept) MUAs
I've used over the last 15 years.

What you're missing is that the best-practice-based RFCs *presume*
your point.  *There is no way to force the recipient's MUA to do
anything.*  Therefore, the RFC headers under discussion are intended
to provide ways for *authors* and other agents (mailing lists) to
*express* their wishes.  The use of Reply-munging, and even more the
other changes you propose, subvert this expression, so that
well-designed conforming MUAs do not (and if you remove the author's
reply-to entirely, *cannot*) do what is expected by their users.

Here's a good (bad?) example:

 > That's why my proposal replaces the From by an author-set Reply-To
 > whenever it is used.

Which is again a non-conforming use of the originator headers.  From
is *not* Reply-To, it identifies the author of the message.  So you've
broken Reply-To, it no longers tells where the author wants replies
sent as it's supposed to.  Now you break From, too!  Like this:

Suppose that because I'm traveling, I set my reply-to to
careof at somewhere.net.  Then my headers will look like:

From: stephen at xemacs.org
Sender: steve at uwakimon.sk.tsukuba.ac.jp
Reply-To: careof at somewhere.net

but what you'll see is

From: careof at somewhere.net
Sender: mailman-users-bounces at python.org
Reply-To: mailman-users at python.org

That's *wrong*, it confuses both my faithful audience, and the
killfiles of those who never want to hear from me again.  Furthermore,
it identifies me as being "at" an address that is evidently temporary.

You simply can't get this right.  I really don't think attempts to do
so should be included in the Mailman distribution at this point in
time.

 > I don't get the point in referring to another software, while Mailman
 > offers 95% of what that use case needs and is installed already. All these
 > fuzzy classifications like CRM shouldn't prevent a software to cross the
 > border, if it easily can do.

I don't want to "prevent" anything.  And couldn't if I wanted to!  You
have the source, you can do what you want.  I'm saying if you want
something that violates the RFCs, use something that has no stake in
being seen to be a conformant implementation of them.

I see the use case.  But what I don't see is a body of experience to
point out best practice.  Your proposed facility will be used in
contexts you don't expect, and it *will* break, in ways that you (or
I!) cannot predict.  I don't think it's a good idea for Mailman to
accept that maintenance burden, until best practice starts to be
understood.

Also, Mailman 3 will involve substantial refactoring as I understand
it.  It will be much more friendly to such experiments.


From sven at anderson.de  Thu Mar 15 18:36:12 2007
From: sven at anderson.de (Sven Anderson)
Date: Thu, 15 Mar 2007 18:36:12 +0100
Subject: [Mailman-Developers] Advanced Reply-To-Munging
In-Reply-To: <87mz2eo4xp.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <45F2F22F.8060403@anderson.de>	<877itnd3yl.fsf@uwakimon.sk.tsukuba.ac.jp>	<45F81A10.50609@anderson.de>
	<87mz2eo4xp.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <45F9840C.5070800@anderson.de>

Hi Stephen,

Stephen J. Turnbull, 15.03.2007 17:19:
>  > That's why my proposal replaces the From by an author-set Reply-To
>  > whenever it is used.
> 
> Which is again a non-conforming use of the originator headers.  From
> is *not* Reply-To, it identifies the author of the message.  So you've
> broken Reply-To, it no longers tells where the author wants replies
> sent as it's supposed to.  Now you break From, too!  Like this:
> 
> Suppose that because I'm traveling, I set my reply-to to
> careof at somewhere.net.  Then my headers will look like:
> 
> From: stephen at xemacs.org
> Sender: steve at uwakimon.sk.tsukuba.ac.jp
> Reply-To: careof at somewhere.net
> 
> but what you'll see is
> 
> From: careof at somewhere.net
> Sender: mailman-users-bounces at python.org
> Reply-To: mailman-users at python.org
> 
> That's *wrong*, it confuses both my faithful audience, and the
> killfiles of those who never want to hear from me again.  Furthermore,
> it identifies me as being "at" an address that is evidently temporary.

Just a short pre-answer to that point, because you got it wrong:

Of course it's wrong. I don't propose that. Sorry, my quoted sentence
above was not clear enough, please stick to the proposal I made.

Assuming you chose the option "reply to list" (option 2) according to my
proposal, the headers would look like:

From: stephen at xemacs.org
Sender: steve at uwakimon.sk.tsukuba.ac.jp
Reply-To: mailman-users at python.org
Cc: careof at somewhere.net (added to existing Cc:, but only if
careof at somewhere.net is not a member of mailman-users at python.org)

I don't see how this additional Cc: can be more dangerous, than what
Mailman is doing right now. The opposite is true, it fixes the
reply-to-all function, because the answer would also go to
careof at somewhere.net, what you explicitly requested by your Reply-To.

All my proposal is about, is bringing the reply-to-munging closer to the
RFC _and_ usability than it is now.

Reply-to-munging _is_ heavily used in existing Mailman installations. Why
not fix it then?


Regards,

Sven

-- 
http://sven.anderson.de    "Believe those who are seeking the truth.
tel:    +49-551-9969285     Doubt those who find it."
mobile: +49-179-4939223                                 (Andr? Gide)

From stephen at xemacs.org  Fri Mar 16 01:17:09 2007
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 16 Mar 2007 09:17:09 +0900
Subject: [Mailman-Developers] Advanced Reply-To-Munging
In-Reply-To: <45F9840C.5070800@anderson.de>
References: <45F2F22F.8060403@anderson.de>
	<877itnd3yl.fsf@uwakimon.sk.tsukuba.ac.jp>
	<45F81A10.50609@anderson.de>
	<87mz2eo4xp.fsf@uwakimon.sk.tsukuba.ac.jp>
	<45F9840C.5070800@anderson.de>
Message-ID: <87ird2nit6.fsf@uwakimon.sk.tsukuba.ac.jp>

Sven Anderson writes:

 > All my proposal is about, is bringing the reply-to-munging closer to the
 > RFC _and_ usability than it is now.

Well, there is a difference of opinion about this, but my opinion is
that Reply-To munging is absolutely broken vis-a-vis the RFC: it's an
originator header, and unless the mailing list can claim to be the
author (eg, announce list or digest), the mailing list should never
touch it.

Whether you can get an effective reply-to-all is a separate question.

 > Reply-to-munging _is_ heavily used in existing Mailman
 > installations. Why not fix it then?

Even the reply-to -> cc proposal isn't a fix, at least in the sense
above.

And whether it makes things more usable, especially for the
maintainers and list admins, is an open question, until it's tried in
practice.  Among other things, the behavior becomes more complicated,
varying across lists.  Consider our disagreement about whether Mailman
removes subscribed addresses from the CC, the ease of confusing the
no-dupes option with not-metoo, and the frequent confusion between
Mailman's no-dupes and Gmail's behavior with respect to reflection of
own posts by mailing lists.

There's such a thing as being too smart for your own good, and I think
it's very easy to cross that line here.


From fmouse-mailman at fmp.com  Mon Mar 19 17:45:02 2007
From: fmouse-mailman at fmp.com (Lindsay Haisley)
Date: Mon, 19 Mar 2007 11:45:02 -0500
Subject: [Mailman-Developers] [Fwd: A work based on your nmzproc]
Message-ID: <1174322702.10115.56.camel@vishnu.fmp.com>

This link may be of interest to the Mailman dev team.

-------- Forwarded Message --------
From: Kiss Gabor (Bitman) <kissg at ssg.ki.iif.hu>
To: fmouse at fmp.com
Subject: A work based on your nmzproc
Date: Mon, 19 Mar 2007 13:17:35 +0100 (CET)

Hi Lindsay,

Maybe you are interested in this document.
http://bakacsin.ki.iif.hu/~kissg/project/mailman+namazu/

Regards

Gabor


From barry at python.org  Wed Mar 21 15:03:49 2007
From: barry at python.org (Barry Warsaw)
Date: Wed, 21 Mar 2007 10:03:49 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8169] trunk/mailman/Mailman/OldStyleMemberships.py
In-Reply-To: <E1HTnWB-0007vp-1B@sc8-pr-svn1.sourceforge.net>
References: <E1HTnWB-0007vp-1B@sc8-pr-svn1.sourceforge.net>
Message-ID: <DA89FEFD-4A12-4B60-BD25-4F32B6E693E9@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 20, 2007, at 7:13 PM, msapiro at users.sourceforge.net wrote:

> Revision: 8169
>           http://svn.sourceforge.net/mailman/?rev=8169&view=rev
> Author:   msapiro
> Date:     2007-03-20 16:13:26 -0700 (Tue, 20 Mar 2007)
>
> Log Message:
> -----------
> - Fixed a bug in OldStyleMemberships.addNewMember that allowed  
> adding an address
>   with upper case in the domain if the local part was all lower case.
> - Changed the semantics of OldStyleMemberships.changeMemberAddress  
> os that
>   in the case of a straightforward address change, i.e. nodelete = 0,
>   delivery status and time are preserved if BYUSER or BYADMIN.
>
> Modified Paths:
> --------------
>     trunk/mailman/Mailman/OldStyleMemberships.py

Thanks for fixing this Mark.  For the change on the trunk, do you  
think we can get test cases in there for these two bugs?  (Probably  
test_membership.py is a good place to add them.)

Cheers,
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRgE7S3EjvBPtnXfVAQLpSgP/XO+95FBO1yabTt5/L+76JLM3xX0PyqvP
Xc2Wob4tJUdFBq9ibP8OXnfFXR3eZxjVEdMTMHMwJ6L96y53sbBgnwyVALh+A3T7
oK/Isqlw8ThITN8UgQB2tTYekaA4O5Ay5lNxW0Pyhd1v8pangK9NO6Q/2ZkdJtQI
fsTT0nJXNJE=
=TnNK
-----END PGP SIGNATURE-----

From msapiro at value.net  Fri Mar 23 22:46:47 2007
From: msapiro at value.net (Mark Sapiro)
Date: Fri, 23 Mar 2007 14:46:47 -0700
Subject: [Mailman-Developers] Problem with format=flowed patch
Message-ID: <PC176020070323144647029653bb77f7@msapiro>

I have been working on a fix for the problem of Mailman's dropping
format=flowed from Content-Type: headers. There is a bug report and a
patch at
<https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1495122&group_id=103>.

The patch to Scrubber.py has just been revised (revised patch is
attached to the above tracker item).

Scrubber would throw an exception when processing a message with no
text/plain part. If you are using a prior version of this patch,
please get the current version to avoid this problem.

-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From tkikuchi at is.kochi-u.ac.jp  Sat Mar 24 22:12:21 2007
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Sun, 25 Mar 2007 06:12:21 +0900
Subject: [Mailman-Developers] Problem with format=flowed patch
In-Reply-To: <PC176020070323144647029653bb77f7@msapiro>
References: <PC176020070323144647029653bb77f7@msapiro>
Message-ID: <46059435.3030808@is.kochi-u.ac.jp>

Hi Mark,

I was working this patch on the trunk.

I think the indent level of this part in Scrubber.py should be

              if charset is None:
                  charset = part.get_content_charset(lcset)
+            format = part.get_param('format')
+            delsp = part.get_param('delsp')

Sorry if I misunderstand.

Mark Sapiro wrote:
> I have been working on a fix for the problem of Mailman's dropping
> format=flowed from Content-Type: headers. There is a bug report and a
> patch at
> <https://sourceforge.net/tracker/?func=detail&atid=100103&aid=1495122&group_id=103>.
> 
> The patch to Scrubber.py has just been revised (revised patch is
> attached to the above tracker item).
> 
> Scrubber would throw an exception when processing a message with no
> text/plain part. If you are using a prior version of this patch,
> please get the current version to avoid this problem.
> 


-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From msapiro at value.net  Sat Mar 24 23:10:18 2007
From: msapiro at value.net (Mark Sapiro)
Date: Sat, 24 Mar 2007 15:10:18 -0700
Subject: [Mailman-Developers] Problem with format=flowed patch
In-Reply-To: <46059435.3030808@is.kochi-u.ac.jp>
Message-ID: <PC1760200703241510180109f2486aff@msapiro>

Tokio Kikuchi wrote:
>
>I think the indent level of this part in Scrubber.py should be
>
>              if charset is None:
>                  charset = part.get_content_charset(lcset)
>+            format = part.get_param('format')
>+            delsp = part.get_param('delsp')
>


Tokio,

Thanks for looking at this.

I don't think your suggestion is correct. What I am trying to do is get
the format= and delsp= parameters from only the first text/plain part
in the message. Often, this will be the only part in which case it
doesn't matter.

Consider however a message with a format=flowed text/plain 'body' and a
subsequent attached text/plain part perhaps without format=flowed. We
want to remember the format parameter from the first 'body' part and
not override it from the second text/plain part.

I recognize that there can always be a pathological case where it might
be more appropriate to get the parameters from a subsequent part, but
I think in general, the first text/plain part is the safest.

-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From tkikuchi at is.kochi-u.ac.jp  Sun Mar 25 00:52:44 2007
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Sun, 25 Mar 2007 08:52:44 +0900
Subject: [Mailman-Developers] Problem with format=flowed patch
In-Reply-To: <PC1760200703241510180109f2486aff@msapiro>
References: <PC1760200703241510180109f2486aff@msapiro>
Message-ID: <4605B9CC.5070602@is.kochi-u.ac.jp>

Mark Sapiro wrote:

> I don't think your suggestion is correct. What I am trying to do is get
> the format= and delsp= parameters from only the first text/plain part
> in the message. Often, this will be the only part in which case it
> doesn't matter.

Sorry that I misunderstood.  It was a little bit too early in the 
morning to start working. ;-) (6AM Japan)

I've almost done with this 'format=flowed' patch and others for 
Scrubber.py and Decorate.py for the trunk.  I'll commit them after 
adding some more test codes.

-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From barry at python.org  Tue Mar 27 03:02:52 2007
From: barry at python.org (Barry Warsaw)
Date: Mon, 26 Mar 2007 21:02:52 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8181] trunk/mailman/Mailman
In-Reply-To: <E1HVIv0-0004xn-LQ@sc8-pr-svn1.sourceforge.net>
References: <E1HVIv0-0004xn-LQ@sc8-pr-svn1.sourceforge.net>
Message-ID: <1523B489-D0A3-4B38-8BB9-80762B1F44A0@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Tokio,

Thanks for the patches, and especially the test cases!  I have a  
couple of questions for you.

I noticed that current trunk is failing three tests:

test_no_multipart_mixed_charset
test_scrub_image
test_scrub_text

Can you check to see if those are failing for you?

Also I was wondering about the latest change to passwords.py:

On Mar 24, 2007, at 10:57 PM, tkikuchi at users.sourceforge.net wrote:

> Modified: trunk/mailman/Mailman/passwords.py
> ===================================================================
> --- trunk/mailman/Mailman/passwords.py	2007-03-24 06:01:35 UTC (rev  
> 8180)
> +++ trunk/mailman/Mailman/passwords.py	2007-03-25 02:57:18 UTC (rev  
> 8181)
> @@ -240,6 +240,9 @@
>      scheme       = scheme_parts[0].lower()
>      scheme_enum  = _SCHEMES_BY_TAG.get(scheme, _DEFAULT_SCHEME)
>      scheme_class = _SCHEMES_BY_ENUM[scheme_enum]
> +    if isinstance(rest_group, unicode):
> +        # decode() fails. (challenge is from database)
> +        rest_group = str(rest_group)
>      return scheme_class.check_response(rest_group, response,  
> *scheme_parts[1:])

Earlier, I was using s.encode('utf-8') to ensure that we had an 8-bit  
string for the hash function.  I'm wondering if you prefer str()  
over .encode('utf-8') here and if so, what the reason is?

Cheers,
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRghtRXEjvBPtnXfVAQJAXQP/enWB9iOvVDJ6B+k3ggN32EjszSH4XvKU
lY9A6gomMIdE+gIivNtbBx6ziv6w2AvYkV2fBC3sc5ooyZGQesjl5gpeZ53AwCyF
HF9bVQuW72L9TxvWIYOO+wnADr97YyYSU/rNHT7GPua5Z+1SnwF5FQknFtpVFZdO
yqzwK5b5C5E=
=moCg
-----END PGP SIGNATURE-----

From tkikuchi at is.kochi-u.ac.jp  Tue Mar 27 03:32:09 2007
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Tue, 27 Mar 2007 10:32:09 +0900
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
 [8181] trunk/mailman/Mailman
In-Reply-To: <1523B489-D0A3-4B38-8BB9-80762B1F44A0@python.org>
References: <E1HVIv0-0004xn-LQ@sc8-pr-svn1.sourceforge.net>
	<1523B489-D0A3-4B38-8BB9-80762B1F44A0@python.org>
Message-ID: <46087419.6090808@is.kochi-u.ac.jp>

Hi Barry,

> I noticed that current trunk is failing three tests:
> 
> test_no_multipart_mixed_charset
> test_scrub_image
> test_scrub_text
> 
> Can you check to see if those are failing for you?

I've confirmed sucess (colinux/debian python2.4) but check them more, on 
different environments.
> 
> Also I was wondering about the latest change to passwords.py:

> Earlier, I was using s.encode('utf-8') to ensure that we had an 8-bit  
> string for the hash function.  I'm wondering if you prefer str()  
> over .encode('utf-8') here and if so, what the reason is?

I was just re-entering the last patch but there may remain older data in 
my installation.  I'll check it in a fresh install and make correction. 
If 'rest_group' is a hashed password, it should be in the ascii range 
but we are going to allow plain password which I fogot.


-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From tkikuchi at is.kochi-u.ac.jp  Tue Mar 27 04:25:13 2007
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Tue, 27 Mar 2007 11:25:13 +0900
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
 [8181] trunk/mailman/Mailman
In-Reply-To: <1523B489-D0A3-4B38-8BB9-80762B1F44A0@python.org>
References: <E1HVIv0-0004xn-LQ@sc8-pr-svn1.sourceforge.net>
	<1523B489-D0A3-4B38-8BB9-80762B1F44A0@python.org>
Message-ID: <46088089.80107@is.kochi-u.ac.jp>

Hi,

> I noticed that current trunk is failing three tests:
> 
> test_no_multipart_mixed_charset
> test_scrub_image
> test_scrub_text

I was using the latest svn of email. :-)

BTW, I can't create a new list in my fresh installation.

# bin/newlist test
Traceback (most recent call last):
   File "bin/newlist", line 48, in <module>
     main()
   File "bin/newlist", line 39, in main
     status = sys.modules[module_name].main()
   File "/usr/local/mailman/Mailman/bin/newlist.py", line 101, in main
     parser, opts, args = parseargs()
   File "/usr/local/mailman/Mailman/bin/newlist.py", line 92, in parseargs
     if opts.password_scheme.lower() not in passwords.SCHEMES:
AttributeError: 'module' object has no attribute 'SCHEMES'


-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From barry at python.org  Tue Mar 27 04:43:03 2007
From: barry at python.org (Barry Warsaw)
Date: Mon, 26 Mar 2007 22:43:03 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8181] trunk/mailman/Mailman
In-Reply-To: <46088089.80107@is.kochi-u.ac.jp>
References: <E1HVIv0-0004xn-LQ@sc8-pr-svn1.sourceforge.net>
	<1523B489-D0A3-4B38-8BB9-80762B1F44A0@python.org>
	<46088089.80107@is.kochi-u.ac.jp>
Message-ID: <E6E421E4-165D-4C75-98C5-666A206E91AA@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 26, 2007, at 10:25 PM, Tokio Kikuchi wrote:

> Hi,
>
>> I noticed that current trunk is failing three tests:
>> test_no_multipart_mixed_charset
>> test_scrub_image
>> test_scrub_text
>
> I was using the latest svn of email. :-)

Hmm, I /think/ I'm using the email 4.0.1 from misc.  One of the  
things I'm trying to do next (other than resolve and commit all the  
changes I made while at Pycon) is to convert things to use setuptools  
so I don't have to keep installing everything just to test them!

> BTW, I can't create a new list in my fresh installation.
>
> # bin/newlist test
> Traceback (most recent call last):
>   File "bin/newlist", line 48, in <module>
>     main()
>   File "bin/newlist", line 39, in main
>     status = sys.modules[module_name].main()
>   File "/usr/local/mailman/Mailman/bin/newlist.py", line 101, in main
>     parser, opts, args = parseargs()
>   File "/usr/local/mailman/Mailman/bin/newlist.py", line 92, in  
> parseargs
>     if opts.password_scheme.lower() not in passwords.SCHEMES:
> AttributeError: 'module' object has no attribute 'SCHEMES'

Confirmed.  I'll fix this.
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRgiEwnEjvBPtnXfVAQLT6AQAhB+XXg7VIxmn4Yl7KmTHy/8fTrQoe+mG
eawINEzzwj/JbJm/6al9paEv8UvDaJk6L1+yCbzGleE5spdR4E0cYEkxPEQ5RW+6
lkTFcVs/+9ulIcQ2Plew710hEg6qd2851ip0EJ0paGcaAK9aRE0V6029ze2N6xmf
lcjWCmseitE=
=C59g
-----END PGP SIGNATURE-----

From tkikuchi at is.kochi-u.ac.jp  Tue Mar 27 10:22:18 2007
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Tue, 27 Mar 2007 17:22:18 +0900
Subject: [Mailman-Developers] How can I reload mlist data from
	SA	database
In-Reply-To: <52CF95C1-DCE9-42BC-B39B-5783D60FD729@python.org>
References: <45E91204.9060909@is.kochi-u.ac.jp>
	<52CF95C1-DCE9-42BC-B39B-5783D60FD729@python.org>
Message-ID: <4608D43A.1070708@is.kochi-u.ac.jp>

Hi again,

Barry Warsaw wrote:

>> testing 2.2 code on my PC.  What is curious about the LMTP/HTTP
>> interfaces with SQLAlchemy based database is that changes commited in
>> LMTP/HTTP sessions cannot be reflected on the other sessions.  If I
>> subscribe an address via HTTP then it cannot be unsubscribed from LMTP
>> and vise versa, until I restart the runners.  Perhaps we need a reload
>> mechanism but I don't know how to implement it.
> 
> I think I've noticed a similar problem.  I think it has to do with  
> concurrency between processes using SQLAlchemy.  As you said, it  
> seems like we need an explicit "reload" but I'm really hoping that  
> that's just because we're not doing something right.  I haven't had  
> time to figure it out yet, or to contact the SA mailing list to get  
> their feedback.  We definitely need to address that issue before we  
> get too far down the SA road.

I've submitted a patch to address this problem on SF patch area because 
I'm not confident whether to check this in the svn.  I hope you have 
time to review and write a better patch.

Cheers,

-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From barry at python.org  Tue Mar 27 15:27:18 2007
From: barry at python.org (Barry Warsaw)
Date: Tue, 27 Mar 2007 09:27:18 -0400
Subject: [Mailman-Developers] How can I reload mlist data from
	SA	database
In-Reply-To: <4608D43A.1070708@is.kochi-u.ac.jp>
References: <45E91204.9060909@is.kochi-u.ac.jp>
	<52CF95C1-DCE9-42BC-B39B-5783D60FD729@python.org>
	<4608D43A.1070708@is.kochi-u.ac.jp>
Message-ID: <79A9F25D-C3D0-49DF-8573-5E322DF2713C@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 27, 2007, at 4:22 AM, Tokio Kikuchi wrote:

> I've submitted a patch to address this problem on SF patch area  
> because I'm not confident whether to check this in the svn.  I hope  
> you have time to review and write a better patch.

Hi Tokio,

Thanks... what's the patch #?
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRgkbvXEjvBPtnXfVAQIRAwQAkRTNctv/r0DNdOD/Mzx2kXLihvpad/Pq
MgkuG7exUPZtUYvsakvA2hNfZIHUAxAOdOfal9KzNFW+mgpCZ48/VEdJZgx2A/cy
Rb0yACBAc2Y49+03OScOwDfb/wp7Y4KhtDvwqUWMeER7w2SFJ833DDPv8grci1Mj
9ijpDLnhL+s=
=csH6
-----END PGP SIGNATURE-----

From tkikuchi at is.kochi-u.ac.jp  Wed Mar 28 01:17:09 2007
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Wed, 28 Mar 2007 08:17:09 +0900
Subject: [Mailman-Developers] How can I reload mlist data
	from	SA	database
In-Reply-To: <79A9F25D-C3D0-49DF-8573-5E322DF2713C@python.org>
References: <45E91204.9060909@is.kochi-u.ac.jp>	<52CF95C1-DCE9-42BC-B39B-5783D60FD729@python.org>	<4608D43A.1070708@is.kochi-u.ac.jp>
	<79A9F25D-C3D0-49DF-8573-5E322DF2713C@python.org>
Message-ID: <4609A5F5.10306@is.kochi-u.ac.jp>

Barry Warsaw wrote:

> On Mar 27, 2007, at 4:22 AM, Tokio Kikuchi wrote:
> 
>> I've submitted a patch to address this problem on SF patch area  
>> because I'm not confident whether to check this in the svn.  I hope  
>> you have time to review and write a better patch.
> 
> Hi Tokio,
> 
> Thanks... what's the patch #?
> - -Barry

http://sourceforge.net/tracker/index.php?func=detail&aid=1688957&group_id=103&atid=300103

Cheers,

-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From barry at python.org  Thu Mar 29 07:43:13 2007
From: barry at python.org (Barry Warsaw)
Date: Thu, 29 Mar 2007 01:43:13 -0400
Subject: [Mailman-Developers] How can I reload mlist data from
	SA	database
In-Reply-To: <4608D43A.1070708@is.kochi-u.ac.jp>
References: <45E91204.9060909@is.kochi-u.ac.jp>
	<52CF95C1-DCE9-42BC-B39B-5783D60FD729@python.org>
	<4608D43A.1070708@is.kochi-u.ac.jp>
Message-ID: <BA255251-DAFB-46CA-8751-C56F2A73CBD0@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 27, 2007, at 4:22 AM, Tokio Kikuchi wrote:

>>> testing 2.2 code on my PC.  What is curious about the LMTP/HTTP
>>> interfaces with SQLAlchemy based database is that changes  
>>> commited in
>>> LMTP/HTTP sessions cannot be reflected on the other sessions.  If I
>>> subscribe an address via HTTP then it cannot be unsubscribed from  
>>> LMTP
>>> and vise versa, until I restart the runners.  Perhaps we need a  
>>> reload
>>> mechanism but I don't know how to implement it.
>> I think I've noticed a similar problem.  I think it has to do  
>> with  concurrency between processes using SQLAlchemy.  As you  
>> said, it  seems like we need an explicit "reload" but I'm really  
>> hoping that  that's just because we're not doing something right.   
>> I haven't had  time to figure it out yet, or to contact the SA  
>> mailing list to get  their feedback.  We definitely need to  
>> address that issue before we  get too far down the SA road.
>
> I've submitted a patch to address this problem on SF patch area  
> because I'm not confident whether to check this in the svn.  I hope  
> you have time to review and write a better patch.

Hi Tokio,

I wonder if we shouldn't just be adding always_refresh=True to the  
mappers?  I can't get to the SQLAlchemy site atm to provide a reference.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRgtR8nEjvBPtnXfVAQJjVAP7BnREssCZMUD4Hp1nCOF7t1EzP76YIbc2
X0Wju9HdiMfRM3TyLnWaKWVB2J4ItJJ1AB73hwY6DtgTf7Yyqp1icaXxhL1ZwKtg
Wl0XWdzC/3rftcc6VA3yABEMRAVwT5w5eu/mdsOBLRw1B5+p81lB4LirWkLaKeKP
ObnSn3ZoY9s=
=N0nP
-----END PGP SIGNATURE-----

From tkikuchi at is.kochi-u.ac.jp  Thu Mar 29 08:35:34 2007
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Thu, 29 Mar 2007 15:35:34 +0900
Subject: [Mailman-Developers] How can I reload mlist data from SA
	database
In-Reply-To: <BA255251-DAFB-46CA-8751-C56F2A73CBD0@python.org>
References: <45E91204.9060909@is.kochi-u.ac.jp>	<52CF95C1-DCE9-42BC-B39B-5783D60FD729@python.org>	<4608D43A.1070708@is.kochi-u.ac.jp>
	<BA255251-DAFB-46CA-8751-C56F2A73CBD0@python.org>
Message-ID: <460B5E36.4010604@is.kochi-u.ac.jp>

Barry Warsaw wrote:

> Hi Tokio,
> 
> I wonder if we shouldn't just be adding always_refresh=True to the  
> mappers?  I can't get to the SQLAlchemy site atm to provide a reference.

I can't access the sqlalchemy site either now.  But, I tested my code 
with two terminals using bin/withlist seeing if I can see a change in 
one in another.  What is odd was that if I used only session.refresh() 
in Load(), the locks are broken.  We must be sure that 
always_refresh=True don't break the lock.

-- 
Tokio Kikucih, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From tkikuchi at is.kochi-u.ac.jp  Thu Mar 29 14:49:58 2007
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Thu, 29 Mar 2007 21:49:58 +0900
Subject: [Mailman-Developers] How can I reload mlist data from SA
	database
In-Reply-To: <460B5E36.4010604@is.kochi-u.ac.jp>
References: <45E91204.9060909@is.kochi-u.ac.jp>	<52CF95C1-DCE9-42BC-B39B-5783D60FD729@python.org>	<4608D43A.1070708@is.kochi-u.ac.jp>	<BA255251-DAFB-46CA-8751-C56F2A73CBD0@python.org>
	<460B5E36.4010604@is.kochi-u.ac.jp>
Message-ID: <460BB5F6.5030407@is.kochi-u.ac.jp>


> We must be sure that always_refresh=True don't break the lock.
> 
No good.  Can't get lock at all.

$ bin/withlist test
Loading list test at colinux.pc (unlocked)
The variable 'm' is the test MailList instance
 >>> m.Lock()
 >>> m.Locked()
False
 >>> m.Save()
Traceback (most recent call last):
   File "<console>", line 1, in <module>
   File "/usr/local/mailman/Mailman/MailList.py", line 576, in Save
     self.__lock.refresh()
   File "/usr/local/mailman/Mailman/LockFile.py", line 187, in refresh
     raise NotLockedError('%s: %s' % (repr(self), self._read()))
NotLockedError: <LockFile 3078782956: 
/usr/local/mailman/locks/test at colinux.pc.lock [unlocked: 18000sec] 
pid=6644>: None
 >>>

-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From tkikuchi at is.kochi-u.ac.jp  Fri Mar 30 13:16:24 2007
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Fri, 30 Mar 2007 20:16:24 +0900
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8184] trunk/mailman/Mailman
In-Reply-To: <E1HX9Mm-0006k5-W8@sc8-pr-svn1.sourceforge.net>
References: <E1HX9Mm-0006k5-W8@sc8-pr-svn1.sourceforge.net>
Message-ID: <460CF188.20102@is.kochi-u.ac.jp>

Hi Barry,

bwarsaw at users.sourceforge.net wrote:
> Revision: 8184
>           http://svn.sourceforge.net/mailman/?rev=8184&view=rev
> Author:   bwarsaw
> Date:     2007-03-29 22:09:36 -0700 (Thu, 29 Mar 2007)
> 
> Log Message:
> -----------
> api_lock(): When locking the MailList object, tell the SQLAlchemy session to
> expire the object.  This way, when the MailList attributes are next accessed,
> the ORM will reload them from the database, getting any new values possibly
> set in other processes.
> 
> This works better than trying to use always_refresh=True on the mapper, or
> trying to do a reload() because both of those approaches blow away locks.  I'm
> not sure why this, but I suspect that it's because the identity map is handing
> us back a different object, rather than invalidating the object's attributes.
> 
> Modified Paths:
> --------------
>     trunk/mailman/Mailman/database/dbcontext.py
>     trunk/mailman/Mailman/testing/test_handlers.py
> 
> Modified: trunk/mailman/Mailman/database/dbcontext.py
> ===================================================================
> --- trunk/mailman/Mailman/database/dbcontext.py	2007-03-27 06:35:12 UTC (rev 8183)
> +++ trunk/mailman/Mailman/database/dbcontext.py	2007-03-30 05:09:36 UTC (rev 8184)
> @@ -129,6 +129,7 @@
>  
>      # Higher level interface
>      def api_lock(self, mlist):
> +        self.session.expire(mlist)
>          # Don't try to re-lock a list
>          if mlist.fqdn_listname in self._mlist_txns:
>              return
> 

Wow!  Fix is so simple.

I've done a test with this using two terminals:

terminal A                         terminal B

$ bin/withlist test                $ bin/withlist test
 >>> m.description
u''
 >>> m.Lock()
 >>> m.description = 'test'
 >>> m.Save()
                                    >>> m.Load()
                                    >>> m.description
                                    u''
 >>> m.Unlock()
                                    >>> m.Lock()
                                    >>> m.description
                                    u'test'

Mere Load() can't refresh the list attributes but needs Lock() to get 
them.  A few problem remains like decorating the message where 
OutgoingRunner is used.  This runner don't update the database so it 
doesn't use Lock() but only Load().  We should fix them to use Lock() 
(and Unlock()) to reload data.


-- 
Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From barry at python.org  Fri Mar 30 14:10:11 2007
From: barry at python.org (Barry Warsaw)
Date: Fri, 30 Mar 2007 08:10:11 -0400
Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman:
	[8184] trunk/mailman/Mailman
In-Reply-To: <460CF188.20102@is.kochi-u.ac.jp>
References: <E1HX9Mm-0006k5-W8@sc8-pr-svn1.sourceforge.net>
	<460CF188.20102@is.kochi-u.ac.jp>
Message-ID: <80EB35CB-DC3E-410B-9A25-E1EF1647D611@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Tokio,

On Mar 30, 2007, at 7:16 AM, Tokio Kikuchi wrote:

> Wow!  Fix is so simple.

I know!  I was shocked too. :)

> Mere Load() can't refresh the list attributes but needs Lock() to get
> them.  A few problem remains like decorating the message where
> OutgoingRunner is used.  This runner don't update the database so it
> doesn't use Lock() but only Load().  We should fix them to use Lock()
> (and Unlock()) to reload data.

Funny, I did probably the exact same test as you did. :)

You're right that the Lock() is necessary to refresh state, so it my  
indeed be better to put this change on Load(), or to fix those few  
other places to use Lock() to refresh.  But I also worry that this  
isn't a full solution ultimately.

The problem is that this will only refresh MailList objects.  I'm  
still trying to find time to port my PyCon changes back into the  
trunk, and part of those changes begins to exchange the PickleType  
columns for real tables.  Refreshing the MailList I suspect won't  
refresh those tables so we'll need to add each related table to the  
refresh set.  I'm going to try to contact the SQLAlchemy folks to see  
if there isn't a more general, or better way, of doing what we're  
trying to do here.  There may not be though because istm that an  
application still needs a way of clearly defining such reload  
boundaries.  Mailman's way is through MailList.Load()/.Lock()/.Unlock 
(), which is tied to that class.

Still, it's important to be aware of this issue, so I'd like to get  
the SA folks take on it.

Thanks for testing this out!
- -Barry

P.S. I'm still getting 3 failures in the test suite.  You mentioned  
it's because we're using different email packages.  I'll try to  
figure that out this weekend.

Cheers!

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRgz+LnEjvBPtnXfVAQLVtgQAkKY2mK7SdiDVn587oM2O6N68MvvxohZv
7wFYZ7uOOt40gKT7YGTySstzuHe3tHC2V1LnxV0/jgQRQ4Qz73jhvQYaYnGO34qn
kgUcBza8OmjxsOM466SjhG5aPRl0GcxENGP1t46+XrocqL4lE95jAdCn96BLz6lx
FsO7lYpJIVw=
=8mRh
-----END PGP SIGNATURE-----