[Mailman-Developers] a patch to scale Cgi/admin.py

[Fil]

> What it sounds like you really want in order to minimize database I/O is
> to implement an in-memory caching system on top of the various methods
> of the MemberAdaptor. So you'd have per MySQLMemberAdaptor object a
> dictionary keyed the same as the database table with dictionaries for
> the various fields per subscriber. If there is a KeyError when trying to
> access the dictionary, hit the database. If the database returns no
> rows, then you raise NotAMemberError or return whatever may be
> appropriate.

I'd love to see it happen, but you also have to be careful (speaking of
scaling, not for my own uses which are served at < 300 k lists) of not
reaching a memory limit. If your lists have 10 million subscribers (say),
you don't want to load the whole list in memory just to retreive one
address. This is the DB's job (ie MySQL itself, or OldStyleMemberAdaptor.py
when using the usual db), not Mailman's job per se.

In parallel I have another idea that could be somehow faster for the
members page: instead of splitting the list into "buckets", and restricting
the display to a chunk of a bucket, just get rid of buckets, and chunk
wherever in the list. And, in order to get the functionality of "buckets"
back, just add the initial letters in the list of links to the different
"chunks".

The links to a specific chunk would be styled as
        /members/?start=jane at doe.com
and the display test would be
        if ( addr >= start ) { prepare the display }

Not sure if my English makes sense, I'll just post the code when I'm done
pythonizing the idea. I don't even know how to compare two strings in
python, so it might take a little while :-D

Note that this would lose no functionality: it may even be a bit more useful
as UI for medium-sized lists of ~100 subscribers --  currently if your list
holds 2 addresses starting by "a", two by "b" and so on, you have to check
26 pages of subscribers, whereas you really need just two (the first 40, and
the last 12).

-- Fil