Large Amount of Data

[Vyacheslav Maslov]

Larry Bates wrote:
> Jack wrote:
>> Thanks for the replies!
>>
>> Database will be too slow for what I want to do.
>>
>> "Marc 'BlackJack' Rintsch" <bj_666 at gmx.net> wrote in message 
>> news:pan.2007.05.25.18.03.09.688008 at gmx.net...
>>> In <H8udnTwG_s1QvMrbnZ2dnUVZ_o2vnZ2d at comcast.com>, Jack wrote:
>>>
>>>> I need to process large amount of data. The data structure fits well
>>>> in a dictionary but the amount is large - close to or more than the size
>>>> of physical memory. I wonder what will happen if I try to load the data
>>>> into a dictionary. Will Python use swap memory or will it fail?
>>> What about putting the data into a database?  If the keys are strings the
>>> `shelve` module might be a solution.
>>>
>>> Ciao,
>>> Marc 'BlackJack' Rintsch 
>>
> Purchase more memory.  It is REALLY cheap these days.
Not a solution at all. What about if amount of data exceed architecture 
memory limits? i.e. 4Gb at 32bit.

Better solution is to use database for data storage/processing

-- 
Vyacheslav Maslov