very large dictionaries

[Mathias Waack]

robin wrote:
> I need to do a search through about 50 million records, each of
> which are less than 100 bytes wide. A database is actually too slow
> for this, so I thought of optimising the data and putting it all in
> memory.

You'll hardly find something faster than a real database. 

> There is a single key field, so a dictionary is an obvious choice
> for a structure, since Python optimises these nicely.
> 
> But is there a better choice? Is it worth building some sort of
> tree?

It depends on the structure of the data. What kind of data do you
have, what are you looking for and on what machine (at least how
much memory is available) will the whole thing run?

Mathias