memory efficient set/dictionary

[koara]

> > I would recommend you to use a database since it meets your
> > requirements (off-memory, fast, persistent). The bsdddb module
> > (berkeley db) even gives you a dictionary like interface.
> >http://www.python.org/doc/lib/module-bsddb.html
>
> Standard SQL databases can work for this, but generally your
> recommendation of using bsddb works very well for int -> int mappings.
> In particular, I would suggest using a btree, if only because I have had
> troubles in the past with colliding keys in the bsddb.hash (and recno is
> just a flat file, and will attempt to create a file i*(record size) to
> write to record number i .
>
> As an alternative, there are many search-engine known methods for
> mapping int -> [int, int, ...], which can be implemented as int -> int,
> where the second int is a pointer to an address on disk.  Looking into a
> few of the open source search implementations may be worthwhile.

Thanks guys! I will look into bsddb, hopefully this doesn't keep all
keys in memory, i couldn't find answer to that during my (very brief)
look into the documentation.

And how about the extra memory used for set/dict'ing of integers? Is
there a simple answer?