[Mailman-Developers] Memory pinned in ram, with huge lists

[Mark Sapiro]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jesus Cea wrote:
> 
> 3. I found a memory reference cycle between a Mailing list and its
> OldStyleMemberships component, linked via "self._memberadaptor". This
> cycle keeps the mailing list alive and, so, the cache never evicted the
> data.
> 
> I changed the OldStyleMemberships constructor to:
> 
> """
> class OldStyleMemberships(MemberAdaptor.MemberAdaptor):
>     def __init__(self, mlist):
>         import weakref
>         self.__mlist = weakref.proxy(mlist)
> """
> 
> to keep only a weak reference to the mailing list, breaking the cycle.


Thanks very much for your efforts in debugging this.


> 2. Now, since cache in evicted frequently, mailing list data must be
> reloaded every time. This is a performance hit, but my mailing list are
> huge but with little traffic (maybe a couple of mails per week), so this
> is a non issue for me.


The use of the cache has been changed for 2.2. See the full thread at
<http://mail.python.org/pipermail/mailman-developers/2008-August/020329.html>
for more information. In 2.2, the cache will be less effective anyway,
and the impact doesn't seem too severe.

I am going to implement your change to OldStyleMemberships for 2.2. I'm
almost inclined to drop the cache all together as I think with the 2.2
logic, hits may be rare. In theory, the logic can avoid a second read of
the pickle if the runner first instantiates the list unlocked and
subsequently locks it, but I suspect this normally happens in the same
clock second so the second read wouldn't be avoided anyway.

- --
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)

iD8DBQFJQswbVVuXXpU7hpMRApSoAKDlxigg49X9N+JiQN2QFwjQvySDzACgrUcZ
JG6h+E9bm29rY/GbriGbSpw=
=rZiy
-----END PGP SIGNATURE-----