[Image-SIG] reaching memory limits with crop method

[Adam J Smith]

We are encountering memory errors while processing very large images (over
3Gb TIFFs, 17888x45408 px). We are trying to make "tiles" of the image by
calling PIL's crop method and saving the resulting images. Eventually, we
see this error:

Traceback (most recent call last):
          File "/insitu1/lavers2/ZoomifyImage/ZoomifyFileProcessor.py",
line 135, in ?
            processor.ZoomifyProcess(sys.argv[1:])
          File "/insitu1/lavers2/ZoomifyImage/ZoomifyFileProcessor.py",
line 127, in ZoomifyProcess
            self.processImage()
          File "/home/lavers2/ZoomifyImage/ZoomifyBase.py", line 199,
in processImage
            imageRow = image.crop([0, ul_y, self.originalWidth, lr_y])
          File
"/home/lavers2/python/lib/python2.4/site-packages/PIL/Image.py",
line 673, in crop
            self.load()
          File
"/home/lavers2/python/lib/python2.4/site-packages/PIL/ImageFile.py", line
155, in load
            self.load_prepare()
          File
"/home/lavers2/python/lib/python2.4/site-packages/PIL/ImageFile.py", line
221, in load_prepare
            self.im = Image.core.new(self.mode, self.size)
        MemoryError

This is on a linux box with 2Gb of RAM, 3Gb of swap space, using PIL 1.1.4.

As I understand it, the load() method loads the entire image into memory,
and although there appears to be some ways to stream image data through
PIL for processing, it looks like you can't use this type of approach with
the crop() method. Am I wrong about this, or is there another way to
accomplish the same type of thing (saving a region of an image as an
image) without using crop()?

Thanks in advance for any guidance you can give.

____________________________
adam smith
ajs17 at cornell.edu
255-8893
215 ccc