It never fails (bsddb retirement in 2.3)

[Peter Hansen]

Nick Vargish wrote:
> 
> A number of these old programs rely on a large text file to determine
> where to go for more information. They do a linear line-based search
> in this 750Kb file. They do this every time someone asks the server
> for a document.
> 
> A couple of weeks ago I figure I can make up the loss of efficiency
> from invoking Python (compared to a native binary) by replacing this
> linear search with a bsddb lookup. The code will generate a fresh .db
> file if the text file changes, since the text file is edited by hand
> about twice a day.

You don't describe exactly how much slower the programs are
compared to the old programs.  Are you optimizing something before
it's time to do that (in other words, before it's working), or are
you doing a "spike" to see whether Python even *can* be fast enough
for you?

If this is just to see how fast Python can be, maybe you can simply 
read the text file into memory and hold it there for searches.  Or
if it must be read by dozens of different programs, maybe they can
all just retrieve a pickled version that is pre-sorted or something.

In other words, there might be some much simpler options than what
you are now trying, and if you're just still experimenting, or if
you don't really need the massive speedup you are currently striving
for but could settle for something a little better than a linear
search, going for the simple approach might be best.

-Peter