[Image-SIG] MemoryError

[Guy K. Kloss]

On Wed, 09 Jul 2008 9:36:14 am Shaun Edwards wrote:
> Hello, I'm still a bit new to all of this, and am extremely happy with PIL.
>  However, I'm ending up with a slight memory issue.  I'm converting large
> tif files (ranging from 400 - 1500 mb in size) to much smaller jpg files,
> in large batches.  I open the tif file, resize it, and convert it to jpg.
>  I then use the garbagecollector module in python to clear the memory.
>  However, after running 30-40 of these files through the script, it fails
> on me, with a MemoryError.  The script is attached.  I'm curious if you
> might have any information, tips, or advice that would assist me in getting
> it to work.

It might be good to use the subprocess module for individual conversion jobs 
for two reasons:

 * Each sub-process that ends will automatically free all memory again
 * You can launch several sub-processes to run in parallel on multi-core
   machines. Therefore you may gain some speed as well.

This does of course not resolve the GC problems in their cause, but it should 
help, be a quick way out of it and additionally give the benefit of speed 
increases on today's rather more ubiquitous multi-core systems.

Guy


[1] http://docs.python.org/lib/module-subprocess.html

-- 
Guy K. Kloss
Institute of Information and Mathematical Sciences
Te Kura Putaiao o Mohiohio me Pangarau
Room 2.63, Quad Block A Building
Massey University, Auckland, Albany
Private Bag 102 904, North Shore Mail Centre
voice: +64 9 414-0800 ext. 9585   fax: +64 9 441-8181
eMail: G.Kloss at massey.ac.nz  http://www.massey.ac.nz/~gkloss/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/image-sig/attachments/20080709/98dfcac1/attachment.pgp>