Large Amount of Data

Larry Bates larry.bates at websafe.com
Fri May 25 18:37:42 EDT 2007


Jack wrote:
> Thanks for the replies!
> 
> Database will be too slow for what I want to do.
> 
> "Marc 'BlackJack' Rintsch" <bj_666 at gmx.net> wrote in message 
> news:pan.2007.05.25.18.03.09.688008 at gmx.net...
>> In <H8udnTwG_s1QvMrbnZ2dnUVZ_o2vnZ2d at comcast.com>, Jack wrote:
>>
>>> I need to process large amount of data. The data structure fits well
>>> in a dictionary but the amount is large - close to or more than the size
>>> of physical memory. I wonder what will happen if I try to load the data
>>> into a dictionary. Will Python use swap memory or will it fail?
>> What about putting the data into a database?  If the keys are strings the
>> `shelve` module might be a solution.
>>
>> Ciao,
>> Marc 'BlackJack' Rintsch 
> 
> 
Purchase more memory.  It is REALLY cheap these days.

-Larry



More information about the Python-list mailing list