[spambayes-dev] DBRunRecoveryError

[Richie Hindle]

[Tim]
> For "stuff like this", it often helps to write a little program driving a
> database with random operations on random data, keeping a mirror dict
> alongside so you can easily check the expected state against the database's
> state.

Here's a script that will either crash Python 2.3c1 ("illegal operation")
or give a DBRunRecoveryError - no need for your mirror dict here!  This is
on Win98, untested on other OSs.  I suspect this is a red herring from
Spambayes' point of view because I don't believe the message DB is ever
shared between threads, but here it is anyway:

----------------------------------------------------------------------

import os, random, bsddb, thread

DB_FILENAME = "hammer_bsddb.db"

def hammer(db):
    for i in range(1000000):
        rnd = random.random()
        if rnd < 0.33:
            # Add/update a value.
            value = "This is a test"[:int(random.random() * 10)]
            db[str(int(random.random() * 100000))] = value
        elif rnd < 0.66:
            # Read a value.
            try:
                x = db[str(int(random.random() * 100000))]
            except KeyError:
                pass
        else:
            # Delete a value
            try:
                del db[str(int(random.random() * 100000))]
            except KeyError:
                pass

def main():
    try: os.remove(DB_FILENAME)
    except OSError: pass
    db = bsddb.hashopen(DB_FILENAME, "c")
    thread.start_new_thread(hammer, (db,))
    thread.start_new_thread(hammer, (db,))
    thread.start_new_thread(hammer, (db,))
    thread.start_new_thread(hammer, (db,))
    hammer(db)

if __name__ == '__main__':
    main()

----------------------------------------------------------------------

The docs for bsddb say nothing about threadsafety, but ISTR people on this
list claiming that it is threadsafe.  Like I say, I don't think threading
is Spambayes' problem, but I'm ready to be corrected...

-- 
Richie Hindle
richie at entrian.com