Is there no compression support for large sized strings in Python?

Claudio Grondi claudio.grondi at freenet.de
Fri Dec 2 05:39:12 EST 2005


"Gerald Klix" <Gerald.Klix at klix.ch> schrieb im Newsbeitrag
news:mailman.1452.1133449632.18701.python-list at python.org...
> Did you consider the mmap library?
> Perhaps it is possible to avoid to hold these big stings in memory.
> BTW: AFAIK it is not possible in 32bit windows for an ordinary programm
> to allocate more than 2 GB. That restriction comes from the jurrasic
> MIPS-Processors, that reserved the upper 2 GB for the OS.
>
> HTH,
> Gerald

>>> objMmap = mmap.mmap(fHdl,os.fstat(fHdl)[6])

Traceback (most recent call last):
  File "<pyshell#21>", line 1, in -toplevel-
    objMmap = mmap.mmap(fHdl,os.fstat(fHdl)[6])
OverflowError: memory mapped size is too large (limited by C int)
>>> os.fstat(fHdl)[6]
4498001104L

Max. allowed value is here 256*256*256*128-1
i.e. 2147483647

'jurrasic' lets greet us also in Python.

The only existing 'workaround' seem to be,
to go for a 64 bit machine with a 64 bit Python version.

No other known way?
Can the Python code not be adjusted, so that C long long is used instead of
C int?

 Claudio





More information about the Python-list mailing list