64 bit Python

[Mathias Waack]

Mike C. Fletcher wrote:

> Mathias Waack wrote:
> ...
> 
>>My python script allocates a bunch of strings each of 1024
>>characters and writes it in a cStringIO. And it fails after writing
>>512K of strings. Don't know how python restricts the heap size -
>>but I'm fairly sure its not a restriction of the OS.
>>  
>>
> Does cStringIO require contiguous blocks of memory?  I'm wondering
> if maybe there's something going on where you just can't allocate
> more than
> a GB of contiguous space?  

That was the problem. I've used cStringIO to reduce the overhead by
the python memory management. But now I know - that was a bad idea
in this case. 

> What happens if you just store the 
> strings in
> a list (or use StringIO instead of cStringIO)?  

It consumed 10 GB really fast, another GB really slow, some hundred
MB much slower and then it died;)

Mathias