[Image-SIG] PIL Consuming All System Resources

[Andy McCurdy]

Thanks for the explanation. As your and others have suggested, I think
limiting the size of the source file seems like a reasonable idea.

-andy

On Sun, May 30, 2010 at 5:59 AM, Fredrik Lundh <fredrik at pythonware.com>wrote:

> The file has mode=RGB size=20400x28079, so you'd need about 2291246400
> bytes to load it all into memory at once, and twice that to do e.g.
> color conversion (which creates a second image memory), so it's a bit
> on the big side, at least for a 32-bit environment.
>
> Note that PIL doesn't actually read the pixels until you do something
> that needs them, so you can do a sanity check before you proceed; e.g.
>
>    im = Image.open(...)
>    bytes = im.size[0] * im.size[1]
>    if im.mode not in ("1", "L", "P"):
>        bytes = bytes * 4  # 32-bit storage
>    if bytes > some threshold:
>        raise IOError("sorry, to big for us")
>    # do operation that actually needs the pixels
>
> </F>
>
> On Sun, May 30, 2010 at 2:55 AM, Andy McCurdy <sedrik at gmail.com> wrote:
> > Hi,
> > We've been using PIL for the last 2 years to resize our users' uploaded
> > images and have been extremely pleased. Thanks for all your effort.
> > I noticed earlier today that we were experiencing issues of servers
> running
> > out of memory. It looks like the problem might be within PIL. I've put
> > together a few lines of code below that produce the same behavior. This
> is
> > the first time we've seen anything like this, and it seems to be a
> problem
> > with a specific image file (URL included in the code below) being
> converted
> > to RGB or resized. This is the only image file I've encountered that
> > produces this behavior. And no, I'm not quite sure why a user felt the
> need
> > to upload a 17M JPG... :)
> > I've tried both PIL 1.1.6 and 1.1.7, and the same behavior occurs on both
> > Ubuntu 8.04 and OS X 10.5.
> > A bug fix would be great, but given the infrequency of seeing this
> problem,
> > I'd settle for a way to detect whether the image I'm working with will
> cause
> > PIL issues so that I can avoid it.
> > Thanks!
> > -andy
> > ##### Example Code #####
> > import StringIO
> > import urllib2
> > from PIL import Image
> > image_data =
> > urllib2.urlopen('
> http://media.giantbomb.com/uploads/8/84310/1291762-img016.jpg').read()
> > io = StringIO.StringIO(image_data)
> > img = Image.open(io)
> > # either of the following two lines causes Python to consume > 2G of
> memory
> > and not return.
> > converted_img = img.convert('RGB')
> > # or
> > resized_img = img.resize((200, 150), Image.ANTIALIAS)
> > _______________________________________________
> > Image-SIG maillist  -  Image-SIG at python.org
> > http://mail.python.org/mailman/listinfo/image-sig
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/image-sig/attachments/20100530/9ec5bdbf/attachment.html>