[Mailman-Users] List Sync Problem

[Michael Caplan]

Jon and Greg,

Thanks for the quick assistance.  Jon, I made the changes you suggested.
That is some brilliant coding.  I've got a lot to learn.  I recommend that
this be entered into the FAQ, as I am sure others will find this very
useful.

Best,

Michael Caplan
Institute for Social Ecology
http://www.social-ecology.org/


-----Original Message-----
From: mailman-users-admin at python.org
[mailto:mailman-users-admin at python.org]On Behalf Of Jon Carnes
Sent: December 4, 2001 12:05 AM
To: ISE; mailman-users at python.org
Subject: Re: [Mailman-Users] List Sync Problem


Ack!  I can think of three possiblities that might be causing this...  but
all are long shots.

 1) The volume you are using is full and there is no room to right the
files.  Check your space available on the server (df).

 2) The following line (added to the script) may not work as you desire:
    "cat /tmp/ise-all_list | grep -vi archive > /tmp/ise-all_list"
You are outputing from the same file that you are inputing to... If it
can't do this in one read, then you may end up with a blank file.

 3) Some other process wipes out your temp file before the sync can start
or finish up.  You end up syncing with a blank or non-existant file.  Do
you run a cron job that cleans out the /tmp directory on a regular basis?

===
If the volumes aren't full then let's make some changes to the script and
see if this helps.
First lets move the created file out of the /tmp volume.  Second, remove
the redirects to /dev/null so that you will recieve emails containing the
output of those commands.  Third, lets version the sync file so that you
can look at it and see if it is ever created incorrectly.

Below are the changes...

You are going to get a lot of output if you use the below script, but it
should help trouble shoot what is going on.

Plus, I have modified the generation of the .htpasswd file so that works
much better and changes a blank password to a generic password (here
defined as the word "password").

I hope this helps,

Jon Carnes

On Monday 03 December 2001 20:43, ISE wrote:
> Hello,
>
> I have compiled a script based upon the great feedback from Jon Carnes
> and the Mailman FAQ that generates a "main" which consists of all the
> subscribers from a set of lists.  This script then  syncs this list with
> a .htpasswd file.  My script works fine via a hourly cron job minus one
> glitch.  Ever few days or so (this does not happen at any consistent
> time) when the script is filtering through the list set, it come back
> with zero list subscribers.  Following this, the script then updates the
> main list. Because It comes back with zero subscribers, the main list's
> members are all removed.  An hour later when the job is run again, the
> script locates all the expected list members from the list set and then
> re-subscribes everyone. The main problem with this is that all the
> re-subscribed members are assigned new passwords, ultimately making my
> attempt to sync this list to a .htpasswd file unfruitful as the list
> members can never be certain of what their password is.
>
> I have included the script below.  I don't think the problem is with the
> script, but perhaps has something to do with Mailman.  Any ideas?
>
> Thank you,
>
> Michael Caplan
> Institute for Social Ecology
> http://www.social-ecology.org/
>
>
#!/usr/local/bin/bash
# Sync the ise-all mailling list
# The list includes everyone on an ISE project mailling list.
# Users on the ise-all list are then synced with the htpass file
# for the ISE "internal" web page
#
# create an empty file

echo " " > /home/mailman/ise_list

LISTS="`/usr/local/mailman/bin/list_lists | grep -i ise | \
  grep -vi food | \
  grep -vi all | \
  grep -vi new-test | \
  grep -vi organise | \
  awk '{ print $1 }'`"

for i in $LISTS
do
   /usr/local/mailman/bin/list_members $i >> /home/mailman/ise_list
done

cat /home/mailman/ise_list |sort -u |grep -vi archive \
      > /home/mailman/ise-all_list

/usr/local/mailman/bin/sync_members -f /home/mailman/ise-all_list ise-all


mv /home/mailman/ise_list /home/mailman/ise_list"`date '+-%d-%k'`"
mv /home/mailman/ise-all_list /home/mailman/ise-all_list"`date '+-%d-%k'`"

#Sync list with htpasswd file

touch /var/tmp/.htpasswd.ise-all
CONFIG="/home/mailman/lists/ise-all/config.db"
for i in `/home/mailman/bin/list_members ise-all `
do
  PASS=`/home/mailman/bin/dumpdb $CONFIG |grep -i $i |head -2 | \
          tail -1 |cut -f4 "-d'"`
  if [ "xxx$PASS" = "xxx" ]; then PASS="password" ; fi
  htpasswd -b /var/tmp/.htpasswd.ise-all $i $PASS
done
#
mv /var/tmp/.htpasswd.ise-all /etc/httpd/.htpasswd.ise-all