[spambayes-bugs] [ spambayes-Bugs-759917 ] Pickle vs. DB: inconsistent behaviour when database missing

[SourceForge.net]

Bugs item #759917, was opened at 2003-06-24 11:26
Message generated for change (Comment added) made by montanaro
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=759917&group_id=61702

Category: hammie
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Ward (gward)
Assigned to: Skip Montanaro (montanaro)
Summary: Pickle vs. DB: inconsistent behaviour when database missing

Initial Comment:
hammie.py behaves inconsistently when the database file
is missing, depending on whether it was run with -D
(pickle) or -d (DBM).  In particular, -D scores the
message at 0.5, and -d crashes with a traceback. 
Example (neither spambayes.db or spambayes.pkl exist):

$ hammie.py -f -D -p spambayes.pkl < $msg | grep
"^X-Spambayes"
X-Spambayes-Classification: unsure; 0.50

$ hammie.py -f -d -p spambayes.db < $msg | grep
"^X-Spambayes"
Traceback (most recent call last):
  File "/usr/local/bin/hammie.py", line 6, in ?
    spambayes.hammiebulk.main()
  File "/home/greg/lib/python/spambayes/hammiebulk.py",
line 180, in main
    h = hammie.open(pck, usedb, mode)
  File "/home/greg/lib/python/spambayes/hammie.py",
line 260, in open
    b = storage.DBDictClassifier(filename, mode)
  File "/home/greg/lib/python/spambayes/storage.py",
line 147, in __init__
    self.load()
  File "/home/greg/lib/python/spambayes/storage.py",
line 155, in load
    self.dbm = dbmstorage.open(self.db_name, self.mode)
  File "/home/greg/lib/python/spambayes/dbmstorage.py",
line 59, in open
    return f(*args)
  File "/home/greg/lib/python/spambayes/dbmstorage.py",
line 41, in open_best
    return f(*args)
  File "/home/greg/lib/python/spambayes/dbmstorage.py",
line 12, in open_db3hash
    return bsddb3.hashopen(*args)
  File
"/usr/local/lib/python2.2/site-packages/bsddb3/__init__.py",
line 160, in hashopen
    d.open(file, _db.DB_HASH, flags, mode)
bsddb3._db.DBNoSuchFileError: (2, 'No such file or
directory')

Arguably, both behaviours are wrong: hammie.py should
terminate with a clear, simple, "No such file or
directory" error message.

----------------------------------------------------------------------

>Comment By: Skip Montanaro (montanaro)
Date: 2004-08-03 06:43

Message:
Logged In: YES 
user_id=44345

Yes, sb_filter.py opens the database in read mode because you
will presumably be modifying the database (training) with some
other tool.  This avoids two problems:

1. corruption w/ multiple writers and no locking

2. having to work out some sort of locking scheme and figuring
out what to do when one of the locking programs (say,
sb_server.py) wants to keep the file locked for writing for long
periods of time.

The only time sb_filter.py opens the database with a different
mode is when you explicitly ask it to create a new database with
the -n flag.  In that case it doesn't score any messages.



----------------------------------------------------------------------

Comment By: Tony Meyer (anadelonbrin)
Date: 2004-08-02 23:26

Message:
Logged In: YES 
user_id=552329

This is very old, and should probably be resolved one way or
the other.  Right now, just about all the spambayes scripts
do the pickle behaviour above, i.e. if there is no database
it opens a new one without error.  This seems somewhat nicer
than having to explicitly create one if it doesn't exist,
although I don't use sb_filter.

The reason that sb_filter is different in this respect is
that it specifies 'r' as the mode, rather than the default
'c'.  I'm sure there is a reason for this, so I don't want
to change it - I figure Skip will probably know why it is,
so assigning to him.

Skip: if you think that the 'r' mode is right, then I'll
create a patch to catch the DBNoSuchFileError and print out
a nicer "Use an existing database, fool", error. :)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=759917&group_id=61702