[Mailman-Users] Indexing mail right after delivery

[Mark Sapiro]

On 3/2/2010 3:41 AM, Cedric Jeanneret wrote:
> On Fri, 26 Feb 2010 10:15:13 -0800
> Mark Sapiro <mark at msapiro.net> wrote:
>>
>> At this point, you have a list object (locked) and a message object. You
>> might think you could just do
>>
>> mlist.ArchiveMail(msg)
>>
>> to archive the mail to the listname.mbox file and the pipermail archive,
>> but that wouldn't quite work because that method would re-invoke the
>> external archiver. Also, you don't need to worry about the listname.mbox
>> file because the ArchiveMail() method already did that before invoking
>> the external archiver, so what you would need is
>>
>> from Mailman.Archiver import HyperArch
>> from cStringIO import StringIO
>> f = StringIO(str(msg))
>> h = HyperArch.HyperArchive(mlist)
>> h.processUnixMailbox(f)
>> h.close()
>> f.close()
>>
>> Which is what the ArchiveMail() method would do. Now you still have the
>> mlist and msg objects, and you need to save and unlock the list at some
>> point
>>
>> mlist.Save()
>> mlist.Unlock()
>>
>> and the message is now in the pipermail archive and can be indexed.
>>
> 
> Hello again,
> 
> I'm having some troubles with my code. According to what Mark said, I've done this :
> 
> #!/usr/bin/env python
> import sys
> sys.path.insert(0,'/usr/lib/mailman')
> 
> import syslog
> 
> syslog.syslog('begin script')
> 
> import email
> from Mailman import MailList
> from Mailman import Message
> ## archive part
> from Mailman.Archiver import HyperArch
> from cStringIO import StringIO
> 
> maillist = sys.argv[2]
> hostname = sys.argv[1]
> 
> msg = email.message_from_file(sys.stdin, Message.Message)
> syslog.syslog(maillist)
> 
> mlist = MailList.MailList(maillist, lock=True)
> 
> syslog.syslog('processing archiver')
> ## let archive it
> f = StringIO(str(msg))
> h = HyperArch.HyperArchive(mlist)
> h.processUnixMailbox(f)
> h.close()
> f.close()
> mlist.Save()
> mlist.Unlock()
> 
> mlist.ArchiveMail(msg)


Here is one problem. Remove the above line. As I tried to say above you
can't do this. The lines above from "f = StringIO(str(msg))" through
"f.close()" archive the message. When you call mlist.ArchiveMail(msg),
it reinvokes your external archiver in an endless loop.

You need to remove the mlist.ArchiveMail(msg).

The locking problem is something else. The external archiver is called
with the list locked, thus when we try to instantiate the list 'locked',
we have a deadlock. Thus, you never saw the loop because of the deadlock.

The good news is we don't have to pass a locked list instance to
HyperArch.HyperArchive() as it uses a special archiver lock.

So, replace

mlist = MailList.MailList(maillist, lock=True)

with

mlist = MailList.MailList(maillist, lock=False)

and remove the "mlist.Unlock()" as your instance isn't locked, and
ArchRunner will unlock its list instance when you exit.


> syslog.syslog('processing indexer')
> ### coming soon
> 
> syslog.syslog('exiting - all ok')
> sys.exit(0)
> 
> "syslog" is for debug purpose only.
> 
> And if I send an email on my ML, I have this kind of error:
> 
> Mar 02 12:38:33 2010 (28380) toto.lock lifetime has expired, breaking


-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan