From paitkenhead at draper.com  Sun Dec 19 02:23:54 2010
From: paitkenhead at draper.com (Aitkenhead, Paul C.)
Date: Sat, 18 Dec 2010 20:23:54 -0500
Subject: [Mailman-Developers] Hacking .py scripts to show  config content
Message-ID: <B3B05EF0-EE09-496F-87AD-5CEF6A3A5236@mimectl>

A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3825 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20101218/d6f65c44/attachment.bin>

From mark at msapiro.net  Sun Dec 19 21:33:10 2010
From: mark at msapiro.net (Mark Sapiro)
Date: Sun, 19 Dec 2010 12:33:10 -0800
Subject: [Mailman-Developers] Hacking .py scripts to show config content
In-Reply-To: <B3B05EF0-EE09-496F-87AD-5CEF6A3A5236@mimectl>
References: <B3B05EF0-EE09-496F-87AD-5CEF6A3A5236@mimectl>
Message-ID: <4D0E6C06.5090301@msapiro.net>

On 12/18/2010 5:23 PM, Aitkenhead, Paul C. wrote:
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers at python.org
[...]


It appears that your original post's body was of some non-text/plain
content type and was thus removed by content filtering leaving only the
signature part.

Please re-post your question as plain text.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From superticker2 at iastate.edu  Mon Dec 27 09:16:48 2010
From: superticker2 at iastate.edu (Superticker2 (Mark))
Date: Mon, 27 Dec 2010 02:16:48 -0600
Subject: [Mailman-Developers] Suggestions about Mailman bounce processing
Message-ID: <201012270216480765.31F10804@mailhub.iastate.edu>

I'm not a subscriber to this list, so please include me
superticker2{at}iastate.edu on your cc: line.

I have two suggestions I would like to make concerning Mailman bounce
processing:

1) All bounces from either a Mass Subscribe or Mass Invite operation should
be unconditionally and immediately sent to the list owner.

There can be many typos in these Mass Subscribe/Invite lists, and the list
owner needs to correct these typos _immediately_.  In addition, some of
these mass subscription lists come from paper sign-up lists by foreign
students where the English alphabet is new to them.  Trying to decipher
their email addresses can result in 10% typos and bounces, which need
immediate corrections.

2) All bounces should be immediately directed to the list owner if bounce
processing is switched off.

For small lists, it makes the most sense for the list owner to handle
bounces manually.  Often he'll know the recipients personally and their new
email addresses, which he can fix himself when they bounce.  As it stand
today, there could be 100s of bad addresses on an email list without
anyone's knowledge if bounce processing is off; this is bad design.


From stephen at xemacs.org  Tue Dec 28 05:11:09 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 28 Dec 2010 13:11:09 +0900
Subject: [Mailman-Developers]  Suggestions about Mailman bounce processing
In-Reply-To: <201012270216480765.31F10804@mailhub.iastate.edu>
References: <201012270216480765.31F10804@mailhub.iastate.edu>
Message-ID: <87fwtileoy.fsf@uwakimon.sk.tsukuba.ac.jp>

Superticker2 (Mark) writes:

 > I'm not a subscriber to this list, so please include me
 > superticker2{at}iastate.edu on your cc: line.
 > 
 > I have two suggestions I would like to make concerning Mailman bounce
 > processing:
 > 
 > 1) All bounces from either a Mass Subscribe or Mass Invite operation should
 > be unconditionally and immediately sent to the list owner.

-1

You seem to be confusing your personal (including job-related)
idiosyncracies with universal requirements.  Mailman is used by many
different people with many different needs.  For example, in my use
case, I don't want to see such bounces; I never hand-type addresses,
but rather use cut and paste from mails sent to me personally or one
of my administrative aliases (incorrectly; my target users are
sophisticated users, and are expected to handle subscriptions
themselves).  If that bounces, there is *nothing* I can do to correct
it (as it turns out, in over ten years of this practice AFAIK I have
never made a mistake in the process of copy/paste; doesn't mean it
won't happen tomorrow but I'm not going to worry about it!), and
ordinary bounce processing works fine for cleaning the list.  (It
takes about a month for an address to be removed, but during that
period it is disabled and uses negligible resources.)

It is possible to set up bounce processing so that your requirements
are met (bounce_processing = 1, bounce_score_threshold = 0.0,
bounce_you_are_disabled_warnings = 0, bounce_notify_owner_on_removal = 1
-- granted, the user is removed from the list ;-).

AFAIK those conditions apply to bounces from admin-originated mail as
well as ordinary list traffic.  If not, I agree that it should be made
possible, and probably there should be an option to have different
options applied to bounces caused by admin-originated mail, although
if it's that important that admin-originated mail be checked, you
probably want to know immediately if a subscriber goes AWOL, too.

 > 2) All bounces should be immediately directed to the list owner if
 >    bounce processing is switched off.

-1

Bounce processing is on by default.  If an owner switches it off,
presumably they know what they're doing.

 > For small lists, it makes the most sense for the list owner to handle
 > bounces manually.

Not if the list address is visible from the "InterSewer".  Eventually
either the -bounce address will be used as a Sender by a spammer or it
will end up on the spammer's To list.  Enough of these will get
through to make the list owner's task not merely a chore, but a
disgusting chore.

Not to mention that a large fraction of bounces are actually due to
transient causes (host goes offline on a holiday weekend, for
example).  And in many use cases the most common cause of bounces is
"undeliverable; over-quota".

Doesn't bounce_processing = 1, bounce_unrecognized_goes_to_list_owner = 1
do exactly what you want?  If it doesn't, it should be fixed to do so.
Hm ... see below.

 > As it stand today, there could be 100s of bad addresses on an email
 > list without anyone's knowledge if bounce processing is off; this
 > is bad design.

Looks like an oversight to me.  The people I know just don't run
without automatic bounce processing in this day and age if they want
to do useful work besides manually processing bounces, so nobody has
noticed the bug.  At l.224 of Mailman/Queue/BounceRunner.py, this
change should allow you to configure things so that you get all the
bounces you want (and if the addresses of your lists are visible from
the InterNet, quite a few you don't want):

         # List isn't doing bounce processing?
         if not mlist.bounce_processing:
+            maybe_forward(mlist, msg)
             return

(This is Mailman 2.10, my host is a bit behind the times, so the line
number may be a bit off.)

From stephen at xemacs.org  Tue Dec 28 07:27:35 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 28 Dec 2010 15:27:35 +0900
Subject: [Mailman-Developers] Suggestions about Mailman bounce processing
In-Reply-To: <201012272358300000.3698C2F6@mailhub.iastate.edu>
References: <201012270216480765.31F10804@mailhub.iastate.edu>
	<87fwtileoy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<201012272358300000.3698C2F6@mailhub.iastate.edu>
Message-ID: <877heul8dk.fsf@uwakimon.sk.tsukuba.ac.jp>

Executive summary:

It seems to me that your best bet is to modify Mailman as I suggest,
and at times when you have a lot of paper signups, switch off "bounce
processing" temporarily, and switch on (preferably it wasn't off in
the first place) "bounce unrecognized goes to list owner".  If
modification scares you, surely somebody around is willing to check
your work.

Of course you may not have access to the Mailman installation, but I
would think that the ssite admin would at least consider this simple
change, especially if Mark or Barry wants it for their next
release(s), since it at worst ends up sending some extra mail to list
owners who have volunteered for admin work in some sense, and only
those who switch off bounce processing.

Superticker2 (Mark) writes:

 > 1) We are a University campus group and the University currently
 >    provides a Mailman service to us.

Sure, been there, done that, and the settings you propose make sense
for you.  They'd just really annoy a lot of people, to no purpose
whatsoever, as defaults.

 > During our bi-annual university Club Fair, we can collect 200 email
 > addresses--on paper--from prospective students/staff wanting on our
 > list.

Surely somebody has a cheap old notebook that you could let people
type on at the booth?  It doesn't have to be hooked to the net there.
(That's a suggestion; I understand it may not work for you for various
reasons.)  Or have them mail you from their cellphones.  For people
who wish to use their cellphones as list address, set up a cellphone
with the subscribe address as the prmary address of the owner,
temporarily and exchange addresses.  (Ditto the previous caveat.)

 > I appreciate your suggestion to set bounce_processing=1,
 > bounce_score_threshold=0, but wouldn't this would cause "existing
 > addresses" with scores at "2" to be removed immediately, which we don't
 > want to do.  We want the existing addresses to remain until their bounce
 > score reaches 2.5.

I don't think it causes them to be removed immediately, but it would
cause those with scores at 1.5 to get removed on the next bounce, so
it's probably out for you.

 > If so, maybe you're on to something.  But wouldn't it be prudent to
 > make this the default behavior for all Mass Subscribes/Invites?

Not default, for the reasons given.

 > Isn't it better to correct typos and such right away 100% of the
 > time?

Remember, I don't make typos of that kind, though I'm human enough in
all other contexts.  I suspect that most people using mass subscribe/
mass invite are not transcribing from paper, and if so, they probably
don't make typos very often either.  People do make such typos,
though, and if they do, there's little you can do about it even if you
want to.  It's a bad idea to try to guess a correct address from an
incorrect one, especially if you don't know the spelling rules of the
language in question.

 > 2) I agree with your comments.

Then AFAICS you agree it should not be default. ;-)

 > Thanks for listening and your suggestion for #1.  I'll try that, but I
 > still maintain that for any Mass Operation, you should be informed
 > immediately of the typos

Sure, I'd want that too.  But the computer can't tell if they *are*
the list admin's typos.  You have a special case which generates a lot
of correctable typos, and in my (limited but non-nil) experience, it's
a rare one, and getting more so all the time.  So it's not a great
default.

Note that the final decision is Marks (v2.x) or Barry's (v3)q, of
course.  But I tend to think they'll agree with me.

From superticker2 at iastate.edu  Tue Dec 28 06:58:30 2010
From: superticker2 at iastate.edu (Superticker2 (Mark))
Date: Mon, 27 Dec 2010 23:58:30 -0600
Subject: [Mailman-Developers] Suggestions about Mailman bounce processing
In-Reply-To: <87fwtileoy.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <201012270216480765.31F10804@mailhub.iastate.edu>
	<87fwtileoy.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <201012272358300000.3698C2F6@mailhub.iastate.edu>

Let me respond to each of your comments:

1) We are a University campus group and the University currently provides a
Mailman service to us.  During our bi-annual university Club Fair, we can
collect 200 email addresses--on paper--from prospective students/staff
wanting on our list.  Many are Asian students, and their English
handwriting is poor at best, but we even have trouble reading American
email addresses.  We guess on the bad handwriting and 10% (20 of them)
bounce.  We can correct those write away and get most of them working on
the second try (or do an LDAP lookup on the problem ones).

Using a paper sign-up list for our booth at the Club Fair is _not_ ideal,
but it's all we have available in this theater.  It's nice you don't have
too use a paper sign-up list like we do.

I appreciate your suggestion to set bounce_processing=1,
bounce_score_threshold=0, but wouldn't this would cause "existing
addresses" with scores at "2" to be removed immediately, which we don't
want to do.  We want the existing addresses to remain until their bounce
score reaches 2.5.

Or are you saying the existing scores will be unaffected by temporarily
setting bounce_processing=1,threshold=0 for just the Mass Subscribe
operation?  If so, maybe you're on to something.  But wouldn't it be
prudent to make this the default behavior for all Mass Subscribes/Invites?
Isn't it better to correct typos and such right away 100% of the time?

2) I agree with your comments.  But I currently run a 1500 subscriber
announcement-only Majordomo list for a monthly newsletter with very good
success processing my own bounces manually.  I get about 8 to 10 "hard"
bounces per newsletter, which I can handle.  Yes, the soft bounces (over
quota) are annoying.

But for a discussion list (not announcement-only) with messages everyday,
you would be crazy not to run automatic bounce processing.  I totally agree
there.

The big advantage with manual bounce processing (small, monthly,
announcement-only list) is you can just fix the bouncing problems yourself
if you know everyone on the list.  That would be the _only_ reason to
disable bounce processing, that is, for manual bounce processing.

Thanks for listening and your suggestion for #1.  I'll try that, but I
still maintain that for any Mass Operation, you should be informed
immediately of the typos--no special temporary settings required.  Happy
computing to you, Mark



*********** REPLY SEPARATOR  ***********

On 12/27/2010 at 10:11 PM Stephen J. Turnbull wrote:

>Superticker2 (Mark) writes:
>
> > I'm not a subscriber to this list, so please include me
> > superticker2{at}iastate.edu on your cc: line.
> > 
> > I have two suggestions I would like to make concerning Mailman bounce
> > processing:
> > 
> > 1) All bounces from either a Mass Subscribe or Mass Invite operation
>should
> > be unconditionally and immediately sent to the list owner.
>
>-1
>
>You seem to be confusing your personal (including job-related)
>idiosyncracies with universal requirements.  Mailman is used by many
>different people with many different needs.  For example, in my use
>case, I don't want to see such bounces; I never hand-type addresses,
>but rather use cut and paste from mails sent to me personally or one
>of my administrative aliases (incorrectly; my target users are
>sophisticated users, and are expected to handle subscriptions
>themselves).  If that bounces, there is *nothing* I can do to correct
>it (as it turns out, in over ten years of this practice AFAIK I have
>never made a mistake in the process of copy/paste; doesn't mean it
>won't happen tomorrow but I'm not going to worry about it!), and
>ordinary bounce processing works fine for cleaning the list.  (It
>takes about a month for an address to be removed, but during that
>period it is disabled and uses negligible resources.)
>
>It is possible to set up bounce processing so that your requirements
>are met (bounce_processing = 1, bounce_score_threshold = 0.0,
>bounce_you_are_disabled_warnings = 0, bounce_notify_owner_on_removal = 1
>-- granted, the user is removed from the list ;-).
>
>AFAIK those conditions apply to bounces from admin-originated mail as
>well as ordinary list traffic.  If not, I agree that it should be made
>possible, and probably there should be an option to have different
>options applied to bounces caused by admin-originated mail, although
>if it's that important that admin-originated mail be checked, you
>probably want to know immediately if a subscriber goes AWOL, too.
>
> > 2) All bounces should be immediately directed to the list owner if
> >    bounce processing is switched off.
>
>-1
>
>Bounce processing is on by default.  If an owner switches it off,
>presumably they know what they're doing.
>
> > For small lists, it makes the most sense for the list owner to handle
> > bounces manually.
>
>Not if the list address is visible from the "InterSewer".  Eventually
>either the -bounce address will be used as a Sender by a spammer or it
>will end up on the spammer's To list.  Enough of these will get
>through to make the list owner's task not merely a chore, but a
>disgusting chore.
>
>Not to mention that a large fraction of bounces are actually due to
>transient causes (host goes offline on a holiday weekend, for
>example).  And in many use cases the most common cause of bounces is
>"undeliverable; over-quota".
>
>Doesn't bounce_processing = 1, bounce_unrecognized_goes_to_list_owner = 1
>do exactly what you want?  If it doesn't, it should be fixed to do so.
>Hm ... see below.
>
> > As it stand today, there could be 100s of bad addresses on an email
> > list without anyone's knowledge if bounce processing is off; this
> > is bad design.
>
>Looks like an oversight to me.  The people I know just don't run
>without automatic bounce processing in this day and age if they want
>to do useful work besides manually processing bounces, so nobody has
>noticed the bug.  At l.224 of Mailman/Queue/BounceRunner.py, this
>change should allow you to configure things so that you get all the
>bounces you want (and if the addresses of your lists are visible from
>the InterNet, quite a few you don't want):
>
>         # List isn't doing bounce processing?
>         if not mlist.bounce_processing:
>+            maybe_forward(mlist, msg)
>             return
>
>(This is Mailman 2.10, my host is a bit behind the times, so the line
>number may be a bit off.)




From superticker2 at iastate.edu  Tue Dec 28 08:27:20 2010
From: superticker2 at iastate.edu (Superticker2 (Mark))
Date: Tue, 28 Dec 2010 01:27:20 -0600
Subject: [Mailman-Developers] Suggestions about Mailman bounce processing
In-Reply-To: <877heul8dk.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <201012270216480765.31F10804@mailhub.iastate.edu>
	<87fwtileoy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<201012272358300000.3698C2F6@mailhub.iastate.edu>
	<877heul8dk.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <201012280127200656.36EA19D7@mailhub.iastate.edu>

What might be simplest solution here would be to temporarily (say for 40
minutes) switch bounce processing off just _before_ a Mass Subscribe/Invite
operation so that all bounces are copied to the list owner.  Ideally, we
only care about the bounces for the last Mass Operation, but the
implementation would be simplest to just copy all of them to the list owner
while bounce processing is temporarily disabled.

Seeing the bounces may also help with debugging/optimizing the bounce
processing threshold and such.

I can't imagine why a list owner would want to switch bounce processing off
without seeing what's bouncing?  And one reason we want to switch from
Majordomo to Mailman is to use its automatic bounce processing (except on
Mass Subscribes/Invites).  :)

If there's a service or utility that can verify a list of email addresses
(prior to a mass operation), please suggest it.

Thanks again for listening and your suggestions, Mark


*********** REPLY SEPARATOR  ***********

On 12/28/2010 at 12:27 AM Stephen J. Turnbull wrote:

>Executive summary:
>
>It seems to me that your best bet is to modify Mailman as I suggest,
>and at times when you have a lot of paper signups, switch off "bounce
>processing" temporarily, and switch on (preferably it wasn't off in
>the first place) "bounce unrecognized goes to list owner".  If
>modification scares you, surely somebody around is willing to check
>your work.
>
>Of course you may not have access to the Mailman installation, but I
>would think that the ssite admin would at least consider this simple
>change, especially if Mark or Barry wants it for their next
>release(s), since it at worst ends up sending some extra mail to list
>owners who have volunteered for admin work in some sense, and only
>those who switch off bounce processing.
>
>Superticker2 (Mark) writes:
>
> > 1) We are a University campus group and the University currently
> >    provides a Mailman service to us.
>
>Sure, been there, done that, and the settings you propose make sense
>for you.  They'd just really annoy a lot of people, to no purpose
>whatsoever, as defaults.
>
> > During our bi-annual university Club Fair, we can collect 200 email
> > addresses--on paper--from prospective students/staff wanting on our
> > list.
>
>Surely somebody has a cheap old notebook that you could let people
>type on at the booth?  It doesn't have to be hooked to the net there.
>(That's a suggestion; I understand it may not work for you for various
>reasons.)  Or have them mail you from their cellphones.  For people
>who wish to use their cellphones as list address, set up a cellphone
>with the subscribe address as the prmary address of the owner,
>temporarily and exchange addresses.  (Ditto the previous caveat.)
>
> > I appreciate your suggestion to set bounce_processing=1,
> > bounce_score_threshold=0, but wouldn't this would cause "existing
> > addresses" with scores at "2" to be removed immediately, which we don't
> > want to do.  We want the existing addresses to remain until their
bounce
> > score reaches 2.5.
>
>I don't think it causes them to be removed immediately, but it would
>cause those with scores at 1.5 to get removed on the next bounce, so
>it's probably out for you.
>
> > If so, maybe you're on to something.  But wouldn't it be prudent to
> > make this the default behavior for all Mass Subscribes/Invites?
>
>Not default, for the reasons given.
>
> > Isn't it better to correct typos and such right away 100% of the
> > time?
>
>Remember, I don't make typos of that kind, though I'm human enough in
>all other contexts.  I suspect that most people using mass subscribe/
>mass invite are not transcribing from paper, and if so, they probably
>don't make typos very often either.  People do make such typos,
>though, and if they do, there's little you can do about it even if you
>want to.  It's a bad idea to try to guess a correct address from an
>incorrect one, especially if you don't know the spelling rules of the
>language in question.
>
> > 2) I agree with your comments.
>
>Then AFAICS you agree it should not be default. ;-)
>
> > Thanks for listening and your suggestion for #1.  I'll try that, but I
> > still maintain that for any Mass Operation, you should be informed
> > immediately of the typos
>
>Sure, I'd want that too.  But the computer can't tell if they *are*
>the list admin's typos.  You have a special case which generates a lot
>of correctable typos, and in my (limited but non-nil) experience, it's
>a rare one, and getting more so all the time.  So it's not a great
>default.
>
>Note that the final decision is Marks (v2.x) or Barry's (v3)q, of
>course.  But I tend to think they'll agree with me.




From mark at msapiro.net  Tue Dec 28 21:54:54 2010
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 28 Dec 2010 12:54:54 -0800
Subject: [Mailman-Developers] Suggestions about Mailman bounce processing
In-Reply-To: <877heul8dk.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <PC195201012281254540796435a6612@MSAPIRO>

Stephen J. Turnbull wrote:
>
>Superticker2 (Mark) writes:
>
> > I appreciate your suggestion to set bounce_processing=1,
> > bounce_score_threshold=0, but wouldn't this would cause "existing
> > addresses" with scores at "2" to be removed immediately, which we don't
> > want to do.  We want the existing addresses to remain until their bounce
> > score reaches 2.5.
>
>I don't think it causes them to be removed immediately, but it would
>cause those with scores at 1.5 to get removed on the next bounce, so
>it's probably out for you.


Stephen is correct. Those with non-stale scores >= 1 will not be
removed immediately, but they will be removed the next time
cron/disabled runs (default 09:00 daily) if the threshold is not
raised before that.

It seems pretty safe to do this. Sometime after cron/disabled runs
reduce the threshold to <= 1, make sure that
bounce_notify_owner_on_disable is Yes and do the mass subscribe. Since
no list members are mailed in this process, they won't bounce. After
allowing time for bounces to be returned and processed (bounces are
queued in Mailman and only processed at 15 minute intervals), raise
the threshold to the original value.

Note however that this will only work for mass subscribes. For
invitations, the returned bounces will be for addresses which aren't
list members and these are always ignored.


>Note that the final decision is Marks (v2.x) or Barry's (v3)q, of
>course.  But I tend to think they'll agree with me.


FWIW, I do agree.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From mark at msapiro.net  Tue Dec 28 22:29:35 2010
From: mark at msapiro.net (Mark Sapiro)
Date: Tue, 28 Dec 2010 13:29:35 -0800
Subject: [Mailman-Developers] Suggestions about Mailman bounce processing
In-Reply-To: <201012280127200656.36EA19D7@mailhub.iastate.edu>
Message-ID: <PC195201012281329350328435a642b@MSAPIRO>

Superticker2 (Mark) wrote:
>
>If there's a service or utility that can verify a list of email addresses
>(prior to a mass operation), please suggest it.


There are some. See for example
<http://www.tienhuis.nl/php-email-address-validation-with-verify-probe>
or try this google search
<http://www.google.com/#q=verify+email+address>.

There are problems with trying to do address verification without
actually sending mail. The obvious way to verify an address is to send
an SMTP VRFY command to the MX of the domain of the address, but these
days most SMTP servers either don't implement VRFY or won't accept
VRFY from untrusted sources.

The alternative is to begin an SMTP dialog with the MX and send MAIL
FROM and RCPT TO commands. As long as you don't send DATA, no mail
will actually be sent, but this is not reliable. If the MX rejects the
RCPT, you may know from the status that the address is bad, but some
MXs will accept an invalid RCPT address and not respond with failure
until after DATA, and you can't continue to that point if you don't
want to actually send mail.


For example, consider the following two SMTP sessions

$ telnet a.mx.mail.yahoo.com 25
Trying 67.195.168.31...
Connected to a.mx.mail.yahoo.com (67.195.168.31).
Escape character is '^]'.
220 mta178.mail.ac4.yahoo.com ESMTP YSmtp service ready
HELO msapiro.net
250 mta178.mail.ac4.yahoo.com
MAIL FROM: <mark at msapiro.net>
250 sender <mark at msapiro.net> ok
RCPT TO: <totally_bogus_mark_sapiro at yahoo.com>
250 recipient <totally_bogus_mark_sapiro at yahoo.com> ok
DATA
354 go ahead
.
554 Transaction failed: empty DATA not allowed
Connection closed by foreign host.

and

$ telnet a.mx.mail.yahoo.com 25
Trying 67.195.168.31...
Connected to a.mx.mail.yahoo.com (67.195.168.31).
Escape character is '^]'.
220 mta112.mail.ac4.yahoo.com ESMTP YSmtp service ready
HELO msapiro.net
250 mta112.mail.ac4.yahoo.com
MAIL FROM: <mark at msapiro.net>
250 sender <mark at msapiro.net> ok
RCPT TO: <totally_bogus_mark_sapiro at yahoo.com>
250 recipient <totally_bogus_mark_sapiro at yahoo.com> ok
DATA
354 go ahead
From: me
To: you

body
.
554 delivery error: dd This user doesn't have a yahoo.com account
(totally_bogus_mark_sapiro at yahoo.com) [0] - mta112.mail.ac4.yahoo.com
Connection closed by foreign host.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From dave at aasv.org  Wed Dec 29 00:25:17 2010
From: dave at aasv.org (David Brown)
Date: Tue, 28 Dec 2010 18:25:17 -0500
Subject: [Mailman-Developers] Suggestions about Mailman bounce processing
Message-ID: <007c01cba6e6$7cb6e140$7624a3c0$@aasv.org>

Years and years ago (2000, I think), I implemented a script in PHP that
allows us to test whether our server will be able to deliver to an email
address. It's based on some code by Jon S. Stevens <jon at clearink.com>
(google that + "validateEmail" and you'll find examples). It first does a
DNS lookup for the MX record, then if that's successful it attempts a
standard SMTP HELO/MAIL FROM/RCPT TO/QUIT sequence and checks for errors in
the mail server's response.  

If you have some familiarity with PHP, I'd be happy to send you the version
I wrapped into a function which you could easily loop through. The RFC822
class from PEAR could help process the text file for addresses. Or perhaps
someone here knows more about how to do it in python. Such a utility could
make a slick addition to the admin page in version 3.

There are also commercial tools to do this.
--
David Brown
dave at aasv.org ; webmaster at aasv.org


-----Original Message-----
[...]
If there's a service or utility that can verify a list of email addresses
(prior to a mass operation), please suggest it.
[...]



From mailman-developers at ml.th-h.de  Wed Dec 29 00:50:26 2010
From: mailman-developers at ml.th-h.de (Thomas Hochstein)
Date: Wed, 29 Dec 2010 00:50:26 +0100
Subject: [Mailman-Developers] Suggestions about Mailman bounce processing
References: <201012270216480765.31F10804@mailhub.iastate.edu>
	<87fwtileoy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<201012272358300000.3698C2F6@mailhub.iastate.edu>
	<877heul8dk.fsf@uwakimon.sk.tsukuba.ac.jp>
	<201012280127200656.36EA19D7@mailhub.iastate.edu>
Message-ID: <alm.1012290050.4149@thorondor.akallabeth.de>

"Superticker2 (Mark)" schrieb:

> If there's a service or utility that can verify a list of email addresses
> (prior to a mass operation), please suggest it.

<http://th-h.de/download/checkmail-0.4.tar.gz> is a Perl script to
check the validity of a mail address (by trying a SMTP dialog up to
RCPT TO).

>From the README:
| DESCRIPTION
|
|     checkmail checks the vailidity / deliverability of a mail address. You
|     may submit just one address as the last argument or a file containing
|     one address on each line using the -f option.

Regards,
-thh

From superticker2 at iastate.edu  Wed Dec 29 06:19:35 2010
From: superticker2 at iastate.edu (Superticker2 (Mark))
Date: Tue, 28 Dec 2010 23:19:35 -0600
Subject: [Mailman-Developers] Suggestions about Mailman bounce processing
In-Reply-To: <PC195201012281254540796435a6612@MSAPIRO>
References: <PC195201012281254540796435a6612@MSAPIRO>
Message-ID: <201012282319350562.3B9B8011@mailhub.iastate.edu>

This is VERY interesting.  I really appreciate this suggestion and may try
it after confirming that disables occur at 9:00AM local time.  Thanks for
your insights!

Another thought I had would be to setup a list-owner web page for each
emailing list that would run a grep script on the Mailman "Bounces Log"
that would (1) show recent (last hour) bounce activity for (2) that
specific list following a Mass Subscribe/Invite operation.  I don't have
access to the Mailman server itself, but if such a web page could give me a
window into the Bounces Log for my specific list, that would be all I need.
 I just need to identify which addresses are bouncing.


*********** REPLY SEPARATOR  ***********

On 12/28/2010 at 2:54 PM Mark Sapiro wrote:

>Stephen J. Turnbull wrote:
>>
>>Superticker2 (Mark) writes:
>>
>> > I appreciate your suggestion to set bounce_processing=1,
>> > bounce_score_threshold=0, but wouldn't this would cause "existing
>> > addresses" with scores at "2" to be removed immediately, which we
don't
>> > want to do.  We want the existing addresses to remain until their
>bounce
>> > score reaches 2.5.
>>
>>I don't think it causes them to be removed immediately, but it would
>>cause those with scores at 1.5 to get removed on the next bounce, so
>>it's probably out for you.
>
>
>Stephen is correct. Those with non-stale scores >= 1 will not be
>removed immediately, but they will be removed the next time
>cron/disabled runs (default 09:00 daily) if the threshold is not
>raised before that.
>
>It seems pretty safe to do this. Sometime after cron/disabled runs
>reduce the threshold to <= 1, make sure that
>bounce_notify_owner_on_disable is Yes and do the mass subscribe. Since
>no list members are mailed in this process, they won't bounce. After
>allowing time for bounces to be returned and processed (bounces are
>queued in Mailman and only processed at 15 minute intervals), raise
>the threshold to the original value.
>
>Note however that this will only work for mass subscribes. For
>invitations, the returned bounces will be for addresses which aren't
>list members and these are always ignored.
>
>
>>Note that the final decision is Marks (v2.x) or Barry's (v3)q, of
>>course.  But I tend to think they'll agree with me.
>
>
>FWIW, I do agree.
>
>-- 
>Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
>San Francisco Bay Area, California    better use your sense - B. Dylan




From superticker2 at iastate.edu  Wed Dec 29 06:58:25 2010
From: superticker2 at iastate.edu (Superticker2 (Mark))
Date: Tue, 28 Dec 2010 23:58:25 -0600
Subject: [Mailman-Developers] Suggestions about Mailman bounce processing
In-Reply-To: <PC195201012281329350328435a642b@MSAPIRO>
References: <PC195201012281329350328435a642b@MSAPIRO>
Message-ID: <201012282358250515.3BBF0D72@mailhub.iastate.edu>

Trying to verify the email address prior to the Mass Subscribe/Invite is
the easiest solution.  I'm going to look into that first. Thanks for all
the tips below.

But as pointed out below, because of spammers trying to randomly guess
email addresses, I think today's mail servers don't always respond with
reliable answers.  This makes address verification problematic, although it
can't hurt to try finding some of bad addresses that way.

Sorry, I haven't learned PHP as yet.  My most fluent programming languages
are C, C++, G (LabView), and MatLab.

Happy computing, Mark


*********** REPLY SEPARATOR  ***********

On 12/28/2010 at 3:29 PM Mark Sapiro wrote:

>Superticker2 (Mark) wrote:
>>
>>If there's a service or utility that can verify a list of email addresses
>>(prior to a mass operation), please suggest it.
>
>
>There are some. See for example
><http://www.tienhuis.nl/php-email-address-validation-with-verify-probe>
>or try this google search
><http://www.google.com/#q=verify+email+address>.
>
>There are problems with trying to do address verification without
>actually sending mail. The obvious way to verify an address is to send
>an SMTP VRFY command to the MX of the domain of the address, but these
>days most SMTP servers either don't implement VRFY or won't accept
>VRFY from untrusted sources.
>
>The alternative is to begin an SMTP dialog with the MX and send MAIL
>FROM and RCPT TO commands. As long as you don't send DATA, no mail
>will actually be sent, but this is not reliable. If the MX rejects the
>RCPT, you may know from the status that the address is bad, but some
>MXs will accept an invalid RCPT address and not respond with failure
>until after DATA, and you can't continue to that point if you don't
>want to actually send mail.
>
>
>For example, consider the following two SMTP sessions
>
>$ telnet a.mx.mail.yahoo.com 25
>Trying 67.195.168.31...
>Connected to a.mx.mail.yahoo.com (67.195.168.31).
>Escape character is '^]'.
>220 mta178.mail.ac4.yahoo.com ESMTP YSmtp service ready
>HELO msapiro.net
>250 mta178.mail.ac4.yahoo.com
>MAIL FROM: <mark at msapiro.net>
>250 sender <mark at msapiro.net> ok
>RCPT TO: <totally_bogus_mark_sapiro at yahoo.com>
>250 recipient <totally_bogus_mark_sapiro at yahoo.com> ok
>DATA
>354 go ahead
>.
>554 Transaction failed: empty DATA not allowed
>Connection closed by foreign host.
>
>and
>
>$ telnet a.mx.mail.yahoo.com 25
>Trying 67.195.168.31...
>Connected to a.mx.mail.yahoo.com (67.195.168.31).
>Escape character is '^]'.
>220 mta112.mail.ac4.yahoo.com ESMTP YSmtp service ready
>HELO msapiro.net
>250 mta112.mail.ac4.yahoo.com
>MAIL FROM: <mark at msapiro.net>
>250 sender <mark at msapiro.net> ok
>RCPT TO: <totally_bogus_mark_sapiro at yahoo.com>
>250 recipient <totally_bogus_mark_sapiro at yahoo.com> ok
>DATA
>354 go ahead
>From: me
>To: you
>
>body
>.
>554 delivery error: dd This user doesn't have a yahoo.com account
>(totally_bogus_mark_sapiro at yahoo.com) [0] - mta112.mail.ac4.yahoo.com
>Connection closed by foreign host.
>
>-- 
>Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
>San Francisco Bay Area, California    better use your sense - B. Dylan