[Python-Dev] bsddb tests disabled by default

[skip at pobox.com]

    Brett> Well, it has always been that way for me, so I always assumed
    Brett> test_bsddb3 was just a *really* long test.

Slow for me, but not nearly as bad as for you:

    % time ./python.exe ../Lib/bsddb/test/test_all.py 

    -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    Sleepycat Software: Berkeley DB 4.4.20: (January 10, 2006)
    bsddb.db.version():   (4, 4, 20)
    bsddb.db.__version__: 4.7.3pre5
    bsddb.db.cvsid:       $Id: _bsddb.c 66182 2008-09-03 17:50:32Z jesus.cea $
    py module:            /Users/skip/src/python/trunk/Lib/bsddb/__init__.pyc
    extension module:     /Users/skip/src/python/trunk/regular/build/lib.macosx-10.3-i386-2.6/_bsddb.so
    python version:       2.6rc2+ (trunk:66519M, Sep 20 2008, 08:36:03) 
    [GCC 4.0.1 (Apple Inc. build 5465)]
    My pid:               82520
    -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    ............................................................. ....
    ----------------------------------------------------------------------
    Ran 294 tests in 156.791s

    OK

    real    2m37.188s
    user    0m9.907s
    sys     0m6.196s

One thing I noticed was that you and I are both using BerkDB 4.4.20 while
Jesus is running 4.7.25.  I can't get to 4.7.x with MacPorts, but I can get
to 4.6.21.  I installed that, rebuild bsddb with it and reran the tests:

    % time ./python.exe ../Lib/bsddb/test/test_all.py 

    -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    Berkeley DB 4.6.21: (September 27, 2007)
    bsddb.db.version():   (4, 6, 21)
    bsddb.db.__version__: 4.7.3pre5
    bsddb.db.cvsid:       $Id: _bsddb.c 66182 2008-09-03 17:50:32Z jesus.cea $
    py module:            /Users/skip/src/python/trunk/Lib/bsddb/__init__.pyc
    extension module:     /Users/skip/src/python/trunk/regular/build/lib.macosx-10.3-i386-2.6/_bsddb.so
    python version:       2.6rc2+ (trunk:66519M, Sep 20 2008, 08:36:03) 
    [GCC 4.0.1 (Apple Inc. build 5465)]
    My pid:               21203
    -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    ............................................................................................................................................................................................................................................................................................................
    ----------------------------------------------------------------------
    Ran 300 tests in 557.679s

    OK

    real    9m18.093s
    user    0m10.499s
    sys     0m16.709s

Hmmm...  Those extra six tests are expensive!  I noticed there was a fair
chunk of time where the CPU meter showed the CPU essentially idle and the
dots were not moving.  I trust it was waiting for the rather modest disk on
my laptop.

Next stop, in-memory disk (all commands as root):

    hdid -nomount ram://1024
    newfs /dev/rdisk1
    mkdir /tmp/mem
    mount /dev/disk1 /tmp/mem
    chmod 1777 /tmp/mem

and rerun the tests with TMPDIR pointing at /tmp/mem.  Whoops, it looks like
test_support creates temp files in the current directory, ignoring TMPDIR or
tempfile.gettempdir().  (Why is that???)  So, cd to /tmp/mem first.  Whoops!
Again, the bsddb tests force the test database into /tmp.  Fix test_all.py
to use TMPDIR if set.  Damn!  1gb isn't enough.  I tried boosting it to 2gb.
Still no go.  Jesus, how big is your ramdisk?

Here's a couple line patch for bsddb/test/test_all.py that uses TMPDIR if
it's set.

Index: Lib/bsddb/test/test_all.py
===================================================================
--- Lib/bsddb/test/test_all.py  (revision 66520)
+++ Lib/bsddb/test/test_all.py  (working copy)
@@ -443,6 +443,9 @@
 def set_test_path_prefix(path) :
     get_new_path.prefix=path
 
+if "TMPDIR" in os.environ:
+    set_test_path_prefix(os.path.join(os.environ["TMPDIR"], "z-Berkeley_DB"))
+
 def remove_test_path_directory() :
     test_support.rmtree(get_new_path.prefix)

Skip