[Mailman-Developers] mailpasswds

Thu, 2 Jul 1998 17:07:36 -0400 (EDT)

--UfXfOrm0J+
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Gergely Madarasz writes:
> I have problems with mailpasswds... first it did not work because of this
> line:
>             url = list.GetOptionsURL(user)
> I changed it to:
>             url = list.GetAbsoluteOptionsURL(user)
> Now it starts (the text is a bit misplaced in the mails it sends), but
> forks too many times (I have two lists, 500 and 120 subscribers):
> [...]
> There should be a better way to handle these mails... either to have a
> better mailing algorithm, or to handle this error, wait for a minute, then
> continue.

You don't say which version you're running, but we ran into the same
problem here at python.org (yesterday, when the passwords went out),
and i've attacked the problem on two fronts in the current code.  I'll
describe what i did for the first one, if you want to try to reproduce
it in your own code, and include a new version of cron/mailpasswds for
1.0b4 for the second.

One fix is to change the "os.forks()" in the scripts/deliver script to
use the following 'forker()' routine, which recognizes errno.EAGAIN,
and retries to fork a specified number of times.  (You have to have
settings in the module for TRIES and REFRACT - i use 5 and 15,
respectively.)

TRIES = 5
REFRACT = 15

def forker(tries=TRIES, refract=REFRACT):
    """Fork, retrying on EGAIN errors with refract secs pause between tries.

    Returns value of os.fork(), or raises the exception for:
     (1) non-EAGAIN exception, or
     (2) EGAIN exception encountered more than tries times."""
    got = 0
    for i in range(tries):
        # Loop until we successfully fork or the number tries is exceeded.
        try:
            got = os.fork()
            break
        except os.error, val:
            import errno, sys, time
            if val[0] == errno.EAGAIN:
                # Resource temporarily unavailable.
                time.sleep(refract)
            else:
                # No go - reraise original exception, same stack frame and all.
                raise val, None, sys.exc_info()[2]
    return got

Another solution is to have the mailpasswds script take some time to
do an os.wait() every several users - i'm attaching a version of
cron/mailpasswds for 1.0b4 that does this.

--UfXfOrm0J+
Content-Type: text/plain
Content-Description: 1.0b4 cron/mailpasswds which periodically waits for forked processes
Content-Disposition: inline;
	filename="mailpasswds"
Content-Transfer-Encoding: 7bit

#! /usr/bin/env python
#
# Copyright (C) 1998 by the Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
# 
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
# 
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software 
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

"""Send password reminders for all lists to all users.

We accumulate users and their passwords, and use the last list to send a
single message to each user with their complete collection of passwords,
rather than sending a single message for each password."""

# This puppy should probably do lots of logging.

import sys, os, string, time, errno
import paths
import maillist, mm_cfg, mm_message, mm_utils

# Give time for the delivery process-forks to clear every so often, to
# avoid saturation of the process table.  Set zero or negative for no
# pauses.
PAUSE_FREQUENCY = 20

USERPASSWORDSTEXT = """
This is a reminder, sent out once a month, about your %s
mailing list memberships.  It includes your subscription info and
how to use it to change it or unsubscribe from a list.

Passwords for %s:

%s
%s
You can visit the URLs to change your membership status or configuration,
including unsubscribing, setting digest-style delivery or disabling
delivery altogether (e.g., for a vacation), and so on.

In addition to the URL interfaces, you can also use email to make such
changes.  For more info, send a message to the '-request' address of the
list (for example, %s-request@%s) containing just
the word 'help' in the message body, and an email message will be sent to
you with instructions.

If you have questions, problems, comments, etc, send them to
mailman-owner@%s.  Thanks!
"""

def MailAllPasswords(list, users):
    """Send each user their complete list of passwords.

    The list can be any random one - it is only used for the message
    delivery mechanism."""
    subj = '%s maillist memberships reminder\n' % list.host_name
    count = PAUSE_FREQUENCY
    for user, data in users.items():
	table = []
	for l, p, u in data:
	    if len(l) > 9:
		table.append("%s\n           %-10s\n%s\n" % (l, p, u))
	    else:
		table.append("%-10s %-10s\n%s\n" % (l, p, u))
	header = ("%-10s %-10s\n%-10s %-10s"
		  % ("List", "Password // URL", "----", "--------"))
	text = USERPASSWORDSTEXT % (list.host_name,
                                    user,
				    header,
				    string.join(table, "\n"),
				    l, list.host_name,
				    list.host_name)
   	list.SendTextToUser(subject = subj,
   			    recipient = user,
   			    text = text,
 			    sender = mm_cfg.MAILMAN_OWNER,
                            add_headers = ["X-No-Archive: yes"],
                            raw=1)
        count = count - 1
        if count == 0:
            # The pause that refreshes.
            waitall()
            count = PAUSE_FREQUENCY

def main():
    """Consolidate all the list/url/password info for each user, so we send 
    the user a single message with the info for all their lists on this
    site."""
    list = None
    users = {}				# user: (listname, password, url)
    for name in mm_utils.list_names():
	list = maillist.MailList(name)
	list_name = list.real_name
	reminders_to_admins = list.reminders_to_admins
	for user, password in list.passwords.items():
	    url = list.GetAbsoluteOptionsURL(user)
	    if reminders_to_admins:
		recipient = "%s-admin@%s" % tuple(string.split(user, '@'))
	    else:
		recipient = user
	    if users.has_key(recipient):
		users[recipient].append(list_name, password, url)
	    else:
		users[recipient] = [(list_name, password, url)]
	# Unlocking each list after identifying passwords, but before having
	# the consolidated list, means that there is a window for discrepancy
	# between the reported and actual password.  Big deal - if the user
	# changed the password in the meanwhile, they'll realize it, and it's
	# not worth the extra deadlock risk.
	list.Unlock()

    if list:
	MailAllPasswords(list, users)

def waitall():
    """Return only when there are no forked subprocesses running."""
    try:
        while 1:
            os.wait()
    except os.error, val:
        if val[0] == errno.ECHILD:
            # errno.ECHILD: "No child processes"
            return
        else:
            raise val, None, sys.exc_info()[2]

if __name__ == "__main__":
    main()

--UfXfOrm0J+
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I've done a little testing of both these approaches, but since the
problem exists at the resource limit, and entails sending out messages 
to lots of people, i'm limited in the testing i could do.  Caveat
emptor!  (And "everyone scrutinize" - the more eyes the better.)

Ken Manheimer		  klm@python.org	    703 620-8990 x268
	    (orporation for National Research |nitiatives

	# If you appreciate Python, consider joining the PSA! #
		  # <http://www.python.org/psa/>. #

--UfXfOrm0J+--