[Mailman-Developers] Pipermail, large archives and performance

[Barry A. Warsaw]

>>>>> "BT" == Bob Tanner <tanner@real-time.com> writes:

    BT> I am wondering what that road map is for mailman.

I've been swamped with other work, specifically Python 2.1a1 and
"paying gig" assignments. :) OTOH, I have a big chunk of rough Mailman
2.1 code that I'm working on, specifically finishing up the i18n stuff
and a complete reworking of the qrunner subsystem along the lines of
stuff discussed last year.  It needs more testing before I can start
checking things in.

I'd like to have something released by the Python conference in
March, though that will realistically just be a beta of 2.1.  I'm on
the hook for giving a talk about I18N at the conference, so expect a
big push to get all that working. :)

I had hoped to rewrite the persistent storage subsystem using zodb,
but that's not going to happen until I can finish up some zodb
enhancements (part of that aforementioned "paying gig").

My goal is still to add at least the features described on the Mailman
2.1 wiki page:

http://www.zope.org/Members/bwarsaw/MailmanDesignNotes/MailmanTwoDotOne

Feel free to add you're own wish list to this page as a comment (which
you should be able to do without logging into zope.org -- let me know
if there are any problems with that).

    BT> And pipermail is really causing problems. I have an mbox file
    BT> that is 1.92Gb in size and it takes almost 14 hours to process
    BT> the qfiles.

This will be improved in 2.1 because there will be a separate qrunner
for the archiver.  It won't be in the critical path for getting mail
through the system.  It won't help Pipermail itself -- for that I'm
still looking for someone to "own" that subsystem.  I want Mailman to
include a bundled Python archiver to make it easy to
"download-and-go", but I don't have time to concentrate on improving
Pipermail itself.  Jeremy's done a great job for 2.0 but it really
could use a rewrite.

    BT> I am willing to put my coding skill to work to fix these
    BT> issues if there is some road map to follow.

If you want to start looking at improving Pipermail and generalizing
the interface b/w the archiver and Mailman, that would be way cool.

-Barry