Caching Strategies/Database question

[Brian Kelley]

>
> Since you're not deleting anything you might change the physical record
> format to:
> (string, nextInteger)
> and simply append these records, ie. using a heap like structure.
> Eventually replace the string by a smaller key and keep a separate
> table of (string, smaller key).
>

Well, MySQL essentially does this in the file tables no?   The solution was
two tables

Table 1
key varchar(255)
key_id integer

Table 2
key_id integer
list_entry integer

The key_id in table 2 is an index into the table and retrievals to date have
been relatively fast.

This is also very fast for storage as the tables can be loaded from flat files
and retrieval is fast because of the way MySQL works... ISAM (indexed
sequential access method) and all that.  The problem with MySQL is that I
can't deliver it nicely to a customer while bsddb3 and metakit are embedded
and the customer really doesn't have to know anything about them.  While these
solutions work there seems to be a big speed hit after a certain amount of
records are inserted and I can't seem to get around this yet.