[Image-SIG] Workarounds for MemoryErrors

[B. Bogart]

Thanks Kevin,

I have tried the code, the Image.new() function just bails due to
MemoryError.

I've now enabled some extra swap, giving me about 20GB of free swap+mem.

Still python bails on Image.new(). Looking at memory usage it seems
python does not even try to allocate, just bails before any memory is used.

Does PIL have some intelligence to bail when some pre-test shows there
is not enough available memory? If so is there a way to override it?

I just tried allocating a 32000x32000 RGBA image, which leaves this:

             total       used       free     shared    buffers     cached
Mem:       4065824    4037288      28536          0       3376       6960
-/+ buffers/cache:    4026952      38872
Swap:     15623164     112048   15511116

So very little swap is actually being used, and I'm guessing not by PIL.

creating an image much larger than 32000x32000 bails due to memoryerror.

Seems the memory usage check does not include swap.

Any ideas?

Thanks,
B.

Kevin Cazabon wrote:
> Quite a number of years back I processed a 1.5GB image on a machine with
> 256MB of RAM (the rest virtual memory - 48x96" poster at 304
> pixels/inch).  It took days, but it worked.  The software actually was
> pretty inefficient too, so I was probably using well over 2GB of memory
> space at a time.
> 
> I think your image is about 19GB (at 24bits/pixel)... not that different
> of a ratio than what I did.  Have you tried it?  Besides ensuring you
> have lots of VM available, it might just work as-is.  Memory is cheap
> these days too... probably cheaper than the hours of programming a
> work-around.
> 
> Kevin
> 
> On 22 Jun 2009, at 12:52, B. Bogart wrote:
> 
>> Hello all,
>>
>> I want to create a very large RGBA image (96000x72000 pixels).
>>
>> I have 4GB of RAM.
>>
>> Is there an easy way of getting around this error by having PIL only
>> allocate one section of the image at a time?
>>
>> If PIL does not have any internal trick to work with large images then
>> I'll have to make 4+ smaller images one at a time, but then I'm not sure
>> how I could combine them without needing to allocate a memory chunk for
>> the whole image.
>>
>> Otherwise I suppose I'll have to try with some other language, perhaps
>> C/SDL, though a quick calculation seems to show that such a large RGBA
>> image is just unworkable. Is there some way of using disk space rather
>> than memory? Does not matter if it is slow, just that it is possible!
>>
>> Any advice?
>>
>> Thanks,
>> B.
>> _______________________________________________
>> Image-SIG maillist  -  Image-SIG at python.org
>> http://mail.python.org/mailman/listinfo/image-sig
> 
>