Is there no compression support for large sized strings in Python?

[jepler at unpythonic.net]

On this system (Linux 2.6.x, AMD64, 2 GB RAM, python2.4) I am able to
construct a 1 GB string by repetition, as well as compress a 512MB
string with gzip in one gulp.

    $ cat claudio.py 
    s = '1234567890'*(1048576*50)

    import zlib
    c = zlib.compress(s)
    print len(c)
    open("/tmp/claudio.gz", "wb").write(c)

    $ python claudio.py 
    1017769

    $ python -c 'print len("m" * (1048576*1024))'
    1073741824

I was also able to create a 1GB string on a different system (Linux 2.4.x,
32-bit Dual Intel Xeon, 8GB RAM, python 2.2).

    $ python -c 'print len("m" * 1024*1024*1024)'
    1073741824

I agree with another poster that you may be hitting Windows limitations rather
than Python ones, but I am certainly not familiar with the details of Windows
memory allocation.

Jeff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20051201/c15a150f/attachment.sig>