Is there no compression support for large sized strings in Python?

[Alex Martelli]

Claudio Grondi <claudio.grondi at freenet.de> wrote:
   ...
> In this context I am very curious how many of such
> 2 GByte strings is it possible to create within a
> single Python process?

VM (Virtual Memory) may make the issue difficult to answer precisely.

With a Python build for 64-bit addressing (and running, of course, on a
64-bit machine), you could go on for a long time.  If your virtual
memory space is large enough (say a nice entire terabyte RAID diskset),
and you don't use resource limiting to throttle the process, you could
be trashing (with about 1000 GB of VM backed by only 14 GB of physical
RAM, I predict *LOTS AND LOTS* of disk activity!) for a very, very long
time before you finally get an out-of-memory error.

Change the parameters and the answer will change, of course -- Python
has relatively little to do with it, as you can build it for either
64-bit or 32-bit addressing, on suitable CPUs; the OS's VM
implementation (and of course the CPU) essentially dominate this
"problem space".


Alex