Writing huge Sets() to disk

Robert Brewer fumanchu at amor.org
Mon Jan 10 11:32:28 EST 2005


Martin MOKREJŠ wrote:
>   I have sets.Set() objects having up to 20E20 items,
> each is composed of up to 20 characters. Keeping
> them in memory on !GB machine put's me quickly into swap.
> I don't want to use dictionary approach, as I don't see a sense
> to store None as a value. The items in a set are unique.
> 
>   How can I write them efficiently to disk?

got shelve*?


Robert Brewer
MIS
Amor Ministries
fumanchu at amor.org

* the "shelve" module in the standard library, that is.



More information about the Python-list mailing list