[Mailman-Users] Phantom moderation pending requests & heldmsg files

[Chris Waltham]

On Apr 7, 2008, at 6:03 PM, Mark Sapiro wrote:
> Chris Waltham wrote:
>
>> A couple of weeks ago our Mailman 2.0 server crashed (a physical
>> problem, it wasn't Mailman's fault!) and we had to upgrade to 2.1  
>> in a
>> real hurry. As a result, I had to copy the entire ~mailman/data
>> directory... which may not have been such a good idea:
>>
>> [mailman at list ~/data]$ ls -al | wc -l
>> 40054
>>
>> There are a _lot_ of files prefixed with "heldmsg-" in the data
>> directory, as you can see. There are entries all the way back to  
>> 2005,
>> in fact! I realize that I'm going to have to deal with all those  
>> files
>> at some stage, but I currently have another problem.
>
>
> Depending on what else you did, these files may now be orphans, but  
> see
> <http://www.python.org/cgi-bin/faqw-mm.py? 
> req=show&file=faq04.074.htp>.

Thanks Mark, I'll check that out.

>> A user
>> administers a list called "ams-announce", and they are receiving
>> emails with the subject "The AMS-announce at list.bowdoin.edu mailing
>> list has 31 request(s) waiting for your consideration". However, when
>> they or I go to the admin interface, there are no pending requests.
>> The user has been getting the same message for more than a week now,
>> always with the same amount (31) of pending messages but there are
>> never actually any messages there for the admin to discard.
>>
>> Oddly, there are no files prefixed with "heldmsg-ams-announce" in  
>> the /
>> home/mailman/data directory, however I did grep for the string "ams-
>> announce" in _all_ heldmsg files and there are many matches --
>> virtually all of it looks like spam. I suppose, then, that I have two
>> questions:
>>
>> * how can I print how many moderation requests a list has pending,  
>> and
>> is there a way to flush the pending requests from the command-line?
>> (The FAQ entries deal with doing it via the web interface, before
>> 2.1.5's feature of "discard all messages")
>> * can I print a list of what lists have what files pending in the
>> format of heldmsg-$listname-$msgID, so that I can cross-reference the
>> files I have on the filesystem (many of which are really old) with
>> what the lists think need to be delievered?
>
>
> First of all, the bogus '31 ams-announce moderator request(s)' email  
> is
> almost certainly not coming from your current Mailman 2.1.9
> installation. I have never seen a report of a case like this where the
> email was not coming from some old/backup/test/whatever installation
> whose cron/checkdbs was still being run.

Guess what? This report also fails your criteria... it was the old  
server. :-( I thought I removed the NFS mount for mailman, but it was  
in the automounter so the cron job could still successfully run!  
That'll teach me for not just removing the user. And because the old  
server had the same name as the new server and I wasn't a list admin,  
I couldn't compare headers. Color me embarrassed...

> Second, the '31' has nothing to do with data/heldmsg-* files. These
> files have the messages themselves, but not the pending request
> information. The data for the 'moderator request(s)' email come from
> the lists/listname/request.pck file. If you want to see what's there,
> you can dump it with bin/dumpdb, and you can clear it by simply
> removing the file, although if you dump it first, I think you'll see
> it's devoid of requests just like the admindb interface says.

Yep, I bet it is:

[mailman at list ~/lists/ams-announce]$ dumpdb request.pck
[----- start pickle file -----]
<----- start object 1 ----->
{'version': (0, 1)}
[----- end pickle file -----]

Whereas if I run the "dumpdb" command on a list I know has outstanding  
requests, I see the request itself. Guess I know how this works, now!

> As far as getting a list of what messages are pending for each list in
> the form heldmsg-$listname-$msgID is concerned, there's nothing that
> will give you that in a nice list, but if you go to a specific list's
> request.pck file and dump it with bin/dumpdb, you'll see one entry  
> like
>
> 'version': (0, 1)
>
> and for each held message, entries like
>
> id: (1, (time, sender, subject, reason, filename, msgdata))
>
> and entries with 2 or 3 as the first item of the value which are held
> subscribe and unsubscribe requests.
>
> In the held message entries, the key, id, is the number you call
> $msgID; time is the time in seconds that the message was held; sender,
> subject and reason are self explanitory; filename is the name of the
> held message file and msgdata is a dictionary of the message's
> metadata at the time it was held.
>
> Note that if you have any request.db files, these are residue from  
> your
> prior installation. I don't know if you did an actual upgrade of your
> 2.0 installation to 2.1.9, but if you did, the data from the
> request,db files should have been automatically migrated to the
> request.pck files by bin/update as part of the 2.1.9 install. If that
> isn't what happened, then I'm not sure what you're faced with in terms
> of recovering any held messages.

With the exception of 3-4 lists (out of 800+), I let the "make update"  
command run so I presume that actually upgraded the lists. I think I  
might just delete the holdmsg files en masse, I can't see why  
(organizationally) I should need to keep them. I think I did remove  
pending_requests.db (or whatever it's called) when I did the upgrade,  
but I think it was empty anyway.


Chris

>
>
> -- 
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
>