64 bit Python
Mathias Waack
M.Waack at gmx.de
Tue Feb 15 11:58:00 EST 2005
Mike C. Fletcher wrote:
> Mathias Waack wrote:
> ...
>
>>My python script allocates a bunch of strings each of 1024
>>characters and writes it in a cStringIO. And it fails after writing
>>512K of strings. Don't know how python restricts the heap size -
>>but I'm fairly sure its not a restriction of the OS.
>>
>>
> Does cStringIO require contiguous blocks of memory? I'm wondering
> if maybe there's something going on where you just can't allocate
> more than
> a GB of contiguous space?
That was the problem. I've used cStringIO to reduce the overhead by
the python memory management. But now I know - that was a bad idea
in this case.
> What happens if you just store the
> strings in
> a list (or use StringIO instead of cStringIO)?
It consumed 10 GB really fast, another GB really slow, some hundred
MB much slower and then it died;)
Mathias
More information about the Python-list
mailing list