[Mailman-Developers] Alternative MemberAdaptors

[M.-A. Lemburg]

Thomas Wouters wrote:
> On Thu, Feb 20, 2003 at 10:55:52AM -0500, barry at python.org wrote:
> 
> [ MemberAdaptors ]

FYI, I'm working on a MemberAdaptor for mxODBC and DCOracle2 (which
won't be open sourced though).

>>I've found a few things during this experience that point to things we
>>ought to improve.  I don't have a lot of time right now, but I wanted
>>to put this out there to start the discussion.  I'll quickly mention a
>>few things.
 >>
>>- I wanted to hook into the BDB transaction (txn) machinery, and I
>>  found a convenient hook.  I overloaded MailList.Lock() to include a
>>  txn begin, MailList.Save() to do a txn commit, and MailList.Unlock()
>>  to do a txn abort.  This seems to work well as long as aborting
>>  after committing is harmless (it is in BDB).  I'd like to get
>>  feedback from the SQL folks (or other MemberAdaptor developers) on
>>  whether we need more explicit transaction support or whether the
>>  basically necessary hooks are already there.

This worked for me as well. I did have to use a central database
controller, though, in order to maintain transaction state
independently of the MemberAdaptor.

>>- We really need to optimize the MemberAdaptor API and the
>>  implementations that use them.  Especially methods that return
>>  lists, e.g. getMembers() and friends.  Right now, everything has to
>>  return a list, but I could do much better by returning iterators,
>>  because I can load my iterator up with a BDB cursor.  This has the
>>  advantage of not requiring the entire member database to be loaded
>>  into memory just to iterate over it.  Unfortunately, too much of the
>>  rest of the code assumes these methods return lists, and while I
>>  started to go down the iterator path, I backed out of it because of
>>  the complexity.

I found that I had to recode the admin.py CGI stuff. Most of
the code seems to be built with the idea of having very fast
access to all aspects of the member data.

With databases and other external storages this is not
the case. It is often better to get all the member data for
a chunk of members at once, then calling out to the storage
for each and every bit of information.

>>  There are other optimizations that would require a bit more
>>  thought.  E.g. the admin's Membership List page seems to require
>>  that the entire member database be iterated over to chunkify and
>>  bucketize.  Fixing this probably requires both changes to the u/i
>>  and changes to the interface.  It also makes life more difficult for
>>  OldStyleMemberships, although BDBMemberAdaptor can probably be
>>  fairly easily elaborated.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Software directly from the Source  (#1, Mar 05 2003)
 >>> Python/Zope Products & Consulting ...         http://www.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
Python UK 2003, Oxford:                                     27 days left
EuroPython 2003, Charleroi, Belgium:                       111 days left