[Python-Dev] We got leaks!

[Tim Peters]

If someone is up for a good weekend puzzle, try this on for size.  I'd like
to multitask this, if possible.

I'm attacking the datetime module in the CVS sandbox, aiming to get it into
good shape for 2.3a1.  There's a pure Python implementation and a pure C
implementation (incomplete but already quite hefty).  You build the latter
like so, for release mode:

    setup.py build_ext -i

or for debug mode:

    setup.py build_ext -i --debug

test_both.py is a test suite.  To test the Python implementation,

    test_both.py

and to test the C implementation,

    test_both.py c

The tests should all pass.

Running this in a loop, I didn't notice any memory growth after a minute of
CPU time.  To be sure, I replaced test_main like so:

def test_main():
    import gc
    r = unittest.TextTestRunner(stream=sys.stdout, verbosity=2)
    s = test_suite()
    while True:
        r.run(s)
        gc.collect()
        print gc.garbage  # this is always an empty list
        print '*' * 10, 'total refs:', sys.gettotalrefcount()

and ran a debug build, watching the printout of total outstanding refs.
Alas, the C implementation shows a very slow leak this way:

********** total refs: 29633
********** total refs: 29643
********** total refs: 29653
********** total refs: 29663
********** total refs: 29673

So the C implementation is leaking 10 references per test cycle.  Fine, so I
screwed up the C code.  However, the Python implementation leaks close to 4x
faster, 37 references per test cycle.  Jeremy suggests I look for the
missing decref in datetime.py <wink>:

********** total refs: 33921
********** total refs: 33958
********** total refs: 33995
********** total refs: 34032
********** total refs: 34069

Likely suspects are "newer stuff":  new-style classes and class methods.
Possibly unittest too, but hard to imagine how.

Even if you can't make time to dig into this, it would help a little to know
what the C and Python leak rates are on a different platform.  Heck, it
would help to know whether the C code even compiles and the tests pass on a
different platform (I'm running Win2K + current CVS Python + MSVC 6 here).