[spambayes-dev] A new and altogether different bsddb breakage

[Kenny Pitt]

Tim Peters wrote:
> [Kenny Pitt]
>> So are you recommending that we avoid using the whole _bsddb.pyd
>> binary package?  I originally thought you were referring only to the
>> Python-coded wrappers.
> 
> We need _bsddb.pyd to use a modern BDB at all from within Python on
> Windows. I'm not familiar with what all is in that DLL, but there
> certainly aren't any Python-coded wrappers in it.  It's about 5000
> lines worth of compiled C code, and wraps Sleepycat's C API so that
> it can be called from Python. Most of it consists of short wrapper
> functions calling Sleepycat functions with similar names, just
> converting raw C bits to and from Python objects at the boundaries ...

OK, then you were talking about what I originally thought you were
talking about.  I was getting worried that the C wrappers themselves
were too buggy to be of use <0.5 wink>.

> Hmm.  Maybe you're not aware of this:
> 
>     http://pybsddb.sourceforge.net/bsddb3.html
> 
> That's effictively the *real* documentation for Python's modern
> Berkeley module.

Yeah, that's the documentation I started from, which basically just
summarizes what each function is for and shows the Python syntax for
calling them.  It then refers you to the Berkeley docs for any level of
detail about option flags and such, so I usually just go straight to the
Sleepycat C++ API docs.  The Python wrappers are basically a direct
mapping of the C++ class structure, and I've previously worked a little
with BDB in C++.

> I'm suggesting avoiding the legacy tricks, avoiding the slow & buggy
> stuff trying to make current BDB "act just like a magical Python
> dict", and writing as directly as possible to Sleepycat's C API
> (which, I hasten to add, is much easier from Python than from C). 

OK, I can go with that, and it should be relatively straightforward.  My
concern still stands re Win98, though.  Maybe I didn't express it
clearly.

Whenever you use direct BDB through the pybsddb/bsddb3/bsddb module in a
multi-thread/multi-user scenario, you always have to start with a call
to initialize the DB environment before you can do anything else.  You
expressed some concern over the breakage on Win98 of the tests in
test_dbshelve.py.  Unfortunately, the line that always fails is that
very first and most basic initialization call, the same one that we
would need to call for any use in SpamBayes.  It is a direct call into
the C-code wrappers, and happens *before* any "legacy tricks".

Since the test suite opens and closes a number of databases and
environments before it gets to the point that fails, there could be some
adverse effects there.  Maybe the best thing is to throw some test code
into SpamBayes and see if it will even start up on Win98.  I don't have
access to a Win98 test system, but if I can code up enough support that
we can try this out, would you be willing to give it a test?  It will
probably be after the holidays before I can get to it, but we'll see.

-- 
Kenny Pitt