bad bsddb key/values on large data

[Barry Schiffman]

--- Skip Montanaro <skip at pobox.com> wrote:

> From: Skip Montanaro <skip at pobox.com>
> Date: Mon, 30 Aug 2004 21:45:28 -0500
> To: Barry Schiffman <schiffo2000 at yahoo.com>
> CC: python-list at python.org
> Subject: Re: bad bsddb key/values on large data
> 
> 
>     Barry> The largest one, about 200 megs, is
> written, and can be read, and
>     Barry> the strings are correctly formatted --
> but the values attached to
>     Barry> the keys are wrong ...
> 
>     ...
> 
>     Barry> Can anyone tell me if there is such a
> Python problem or a bsddb
>     Barry> problem?
> 
> You didn't say what versions of Berkeley DB or
> Python you are using.  If you
> are using Berkeley DB 1.85, problems are to be
> expected.  Upgrade to a later
> version.
> 
> Skip
> 

My apologies, but I wrote this too soon. 

The chief suspect is an interaction between one of the
hash keys and my code. The entire table is shifted
ahead on the very first key, so the value for key_1 is
saved under key_2, and then so and so forth. The shift
stays constant at one step (so key_n contains the
record for key_n-1). This only happens with one of the
three files. In fact, the code is supposed to reject
many of the records in the input, and does so without
messing up the key/value pairs -- except for this one.

However, I won't be able to check this out until
tonight. 

Anyway, I think the versions are OK (though I'm not
sure if the Sleepcat versions are the same as the
Berkeley versions). I tried this twice under these
setups

Python 2.2.2, Sleepycat 3.3.11 on a PC with RH 7.3
Python 2.3.2, Sleepycat 3.2.9 on a PPC with YDL 2.3

Thanks much.

Barry


	
		
__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - 100MB free storage!
http://promotions.yahoo.com/new_mail