[Mailman-Users] Phantom moderation pending requests & heldmsg files

[Mark Sapiro]

Chris Waltham wrote:

>Hi Mark,
>
>Just to re-visit this...
>
>On Apr 8, 2008, at 1:24 PM, Mark Sapiro wrote:
>
>> Chris Waltham wrote:
>>>
>>> With the exception of 3-4 lists (out of 800+), I let the "make  
>>> update"
>>> command run so I presume that actually upgraded the lists. I think I
>>> might just delete the holdmsg files en masse, I can't see why
>>> (organizationally) I should need to keep them.
>>
>>
>> When you delete these files, it is better to use bin/discard as
>> discussed in
>> <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.074.htp>
>> rather than just rm. This is because bin/discard actually removes the
>> held message from the requests.pck file (see below). However, if the
>> heldmsg-* file is an orphan, it won't be removed by bin/discard, so
>> you need to run bin/discard first and then rm any files left.
>>
>> The problem with just removing the heldmsg-* file is if there are a
>> large number of held messages for a single list, the admindb CGI can
>> time out trying to create the held messages page, and just removing
>> the heldmsg-* files without also removing the request.pck entries
>> won't cure this time out.
>
>I looked at my heldmsg files (all 40,000 of them :-)) and there are a  
>number patterns. Most heldmsg files are from a handful of lists (let's  
>call them baseball and football). If I do a dumpdb of the hockey  
>list's pending.pck file, this is the output:
>
>[mailman at list ~/bin]$ ./dumpdb ../lists/hockey/pending.pck
>[----- start pickle file -----]
><----- start object 1 ----->
>{   '6183e43cf99b4a6850537d2a4837b26886bade2a': ('H', 5145),
>     'd22c46a0b5c704ec8be5c63a791e53adc23dc7fc': ('H', 5146),
>     'df7eae0e0a275c43b9c4bfb8d5f037df578e6cc2': ('H', 5147),


Note that the entries in lists/hockey/pending.pck are only the
confirmation tokens and they expire after (default) 3 days, so there
may normally be lots of held messages that are not in pending.pck.

The tokens in pending.pck are for the submitter of the held message to
be able to cancel the post or for the admin/moderator to be able to
cancel or approve the post by email. They have nothing to do with
what's in the admindb interface which is controlled by what's in the
request.pck file.


>Now, the first line ends with the value "5145". Here is what I can  
>find on the filesystem:
>
>[mailman at list ~/data]$ ls -al heldmsg-hockey* | grep 5145
>-rw-rw-r--  1 mailman mailman   3395 May  4 12:54 heldmsg- 
>hockey-5145.pck
>
>I rinsed & repeated that process with the other two entries in the  
>pending.pck file, i.e. 5146 and 5147. For both of those two, I can  
>find heldmsg-hockey-NNNN.pck (where NNNN equals the ID above).  
>However, I still have an awful lot of heldmsg-hockey files that  
>are .txt files, not .pck files:


The fact that they are .txt files is not a problem per se. As long as
there are entries in request.pck for them, they should appear in the
admindb interface and can be approved if you wish, or rejected or
discarded.

If they are not in request.pck and do not appear in the admindb
interface, all you can do is remove the files or 'repost' the messages
using bin/inject


>[mailman at list ~/data]$ ls -al heldmsg-hockey*.txt | wc -l
>3610
>
>[mailman at list ~/data]$ ls -alt heldmsg-hockey*.txt | head
>-rw-rw-r--  1 mailman mailman   2210 Mar 13 08:42 heldmsg- 
>hockey-4920.txt
>-rw-rw-r--  1 mailman mailman   3250 Mar 13 08:14 heldmsg- 
>hockey-4919.txt
>-rw-rw-r--  1 mailman mailman   3646 Mar 13 01:18 heldmsg- 
>hockey-4918.txt
>-rw-rw-r--  1 mailman mailman   2505 Mar 12 03:50 heldmsg- 
>hockey-4917.txt
>-rw-rw-r--  1 mailman mailman   2377 Mar 12 03:50 heldmsg- 
>hockey-4916.txt
>-rw-rw-r--  1 mailman mailman   2815 Mar 11 14:38 heldmsg- 
>hockey-4915.txt
>-rw-rw-r--  1 mailman mailman   2354 Mar 10 12:02 heldmsg- 
>hockey-4914.txt
>-rw-rw-r--  1 mailman mailman   2591 Mar 10 11:28 heldmsg- 
>hockey-4913.txt
>-rw-rw-r--  1 mailman mailman   2227 Mar 10 11:19 heldmsg- 
>hockey-4912.txt
>-rw-rw-r--  1 mailman mailman   2551 Mar 10 11:19 heldmsg- 
>hockey-4911.txt
>
>I think March 10 is around the time that I went from Mailman 2.0.12 to  
>2.1.9. Now my question is this: does Mailman 2.1.9 reference .txt  
>files for heldmessages, or are we only using .pck files now?


Theis is controlled by HOLD_MESSAGES_AS_PICKLES which defaults to Yes
but can be set to No in mm_cfg.py.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan