Storing pairs of (int, int) in a database : which db to choose ?

Paul Rubin http
Tue Dec 23 10:09:09 EST 2003


John Hunter <jdhunter at ace.bsd.uchicago.edu> writes:
>     Stormbringer> in my opinion (especially considering the range of
>     Stormbringer> those integers - one is in the range 1..100000 and
>     Stormbringer> the other in the range 1..500000).
> 
> What about using a binary file of unsigned ints which you load into a
> python dictionary and do everything in memory?  There would be no
> extra overhead in the file and it would be very fast, if you are able
> to hold the 100,000 ints in memory.

No it's much worse than that.  The 100,000 ints are index numbers
for individual words.  The 500,000 ints are articles and there can
be thousands of words in each article.  So you may need to store
billions of ints, not just 100,000 of them.




More information about the Python-list mailing list