Large Amount of Data

[John Nagle]

Jack wrote:
> I need to process large amount of data. The data structure fits well
> in a dictionary but the amount is large - close to or more than the size
> of physical memory. I wonder what will happen if I try to load the data
> into a dictionary. Will Python use swap memory or will it fail?
> 
> Thanks. 

     What are you trying to do?  At one extreme, you're implementing something
like a search engine that needs gigabytes of bitmaps to do joins fast as
hundreds of thousands of users hit the server, and need to talk seriously
about 64-bit address space machines.  At the other, you have no idea how
to either use a database or do sequential processing.  Tell us more.

					John Nagle