[Image-SIG] Workarounds for MemoryErrors

[Fredrik Lundh]

On Wed, Jun 24, 2009 at 10:06 AM, Gregor Kopka<gregor at kopka.net> wrote:

> could it be that it bails the moment you step with the image size over the
> maximum value of a signed word (32768) ?

The size is stored as C ints, so that shouldn't happen (unless you
compile on a machine that thinks int is 16 bits).  The following works
just fine on my 64-bit machine:

>>> a = Image.new("L", (100000000, 1))
>>> b = Image.new("L", (1, 100000000))

so if you get a MemoryError, the reason is simply that the OS doesn't
want to allocate that much memory for your process, either because it
doesn't have it, or because a process-level or other resource
limitation.

But PIL simply isn't made for processing humongous images in a single
block; I suggest doing some preprocessing to split them up in tiles or
strips, or, if you're reading from an uncompressed format,
manipulating the "size" and "tile" attributes directly after open; see
e.g.

    http://mail.python.org/pipermail/image-sig/2005-September/003525.html

(if the input is TIFF, the tile attribute may contain a bunch of
tiles; in that case, you can simply process one tile at a time by
keeping just one in the list, and adjusting the size/extent to match
that tile).

For output, you're pretty much on your own, even if you can probably
reuse code from e.g. the PPM module, or any other codec that writes
uncompressed files.

</F>