[Image-SIG] Workarounds for MemoryErrors

[Gregor Kopka]

Hi,

could it be that it bails the moment you step with the image size over 
the maximum value of a signed word (32768) ?

Regards,

Gregor

B. Bogart schrieb:
> Thanks Kevin,
>
> I have tried the code, the Image.new() function just bails due to
> MemoryError.
>
> I've now enabled some extra swap, giving me about 20GB of free swap+mem.
>
> Still python bails on Image.new(). Looking at memory usage it seems
> python does not even try to allocate, just bails before any memory is used.
>
> Does PIL have some intelligence to bail when some pre-test shows there
> is not enough available memory? If so is there a way to override it?
>
> I just tried allocating a 32000x32000 RGBA image, which leaves this:
>
>              total       used       free     shared    buffers     cached
> Mem:       4065824    4037288      28536          0       3376       6960
> -/+ buffers/cache:    4026952      38872
> Swap:     15623164     112048   15511116
>
> So very little swap is actually being used, and I'm guessing not by PIL.
>
> creating an image much larger than 32000x32000 bails due to memoryerror.
>
> Seems the memory usage check does not include swap.
>
> Any ideas?
>
> Thanks,
> B.
>
> Kevin Cazabon wrote:
>   
>> Quite a number of years back I processed a 1.5GB image on a machine with
>> 256MB of RAM (the rest virtual memory - 48x96" poster at 304
>> pixels/inch).  It took days, but it worked.  The software actually was
>> pretty inefficient too, so I was probably using well over 2GB of memory
>> space at a time.
>>
>> I think your image is about 19GB (at 24bits/pixel)... not that different
>> of a ratio than what I did.  Have you tried it?  Besides ensuring you
>> have lots of VM available, it might just work as-is.  Memory is cheap
>> these days too... probably cheaper than the hours of programming a
>> work-around.
>>
>> Kevin
>>
>> On 22 Jun 2009, at 12:52, B. Bogart wrote:
>>
>>     
>>> Hello all,
>>>
>>> I want to create a very large RGBA image (96000x72000 pixels).
>>>
>>> I have 4GB of RAM.
>>>
>>> Is there an easy way of getting around this error by having PIL only
>>> allocate one section of the image at a time?
>>>
>>> If PIL does not have any internal trick to work with large images then
>>> I'll have to make 4+ smaller images one at a time, but then I'm not sure
>>> how I could combine them without needing to allocate a memory chunk for
>>> the whole image.
>>>
>>> Otherwise I suppose I'll have to try with some other language, perhaps
>>> C/SDL, though a quick calculation seems to show that such a large RGBA
>>> image is just unworkable. Is there some way of using disk space rather
>>> than memory? Does not matter if it is slow, just that it is possible!
>>>
>>> Any advice?
>>>
>>> Thanks,
>>> B.
>>> _______________________________________________
>>> Image-SIG maillist  -  Image-SIG at python.org
>>> http://mail.python.org/mailman/listinfo/image-sig
>>>       
>>     
> _______________________________________________
> Image-SIG maillist  -  Image-SIG at python.org
> http://mail.python.org/mailman/listinfo/image-sig
>
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/image-sig/attachments/20090624/2ee9bda4/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gregor.vcf
Type: text/x-vcard
Size: 314 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/image-sig/attachments/20090624/2ee9bda4/attachment.vcf>