Fast Imaging for Webserver

[Prateek]

Thanks for all the responses so far..

The 'imaging' app that I mentioned is actually serving as a library for
a bunch of other apps.
A typical scenario is icon processing. The source image *could* be
large (in this case it was 128x128 PNGA)

Think of it as a Mac Finder/Win Explorer style view (grid + imagelist)
where you're showing a bunch of data in icon form (16x16). For small
amounts of data, everything is ok... but as soon as I get to 600-700
items, the page takes forever to load. The obvious solution is
pagination but I'm doing some aggregate analysis on the data (like a
frequency distribution) where if I were to paginate the data, i'd lose
the client-side sorting ability (I'd have to sort server side which is
a whole other can of worms).

I'm gonna take Paul's advice and report as soon as I can. One other
alternative is to save this information on the filesystem (as temp
files) and route all future requests via the static file processing
mechanism (which can be handled by Apache)... do you think that is a
good idea?

Prateek

On Jan 25, 3:11 pm, "Fredrik Lundh" <fred... at pythonware.com> wrote:
> "Prateek" wrote:
> > Hi. I'm creating a web-application using CherryPy 2.2.1. My application
> > needs to process images (JPG/PNG files) to
>
> > 1) create thumbnails (resize them)
> > 2) overlay them on a custom background (a simple frame)
> > 3) Overlay 'badges' (small 16x16 images) on top of the final thumbnail
>
> > I am using PIL 1.1.5 which I have custom compiled on my development
> > machine (MacBook Pro 2.33Ghz Core Duo)..
> > I am using im.thumbnail for step 1 and im.paste for steps 2 and 3.
>
> > The problem is that this thing is just way too slow.
> > For ab -n 1000 -C session_id=2f55ae2dfefa896f67a80f73045aadfa4b4269f1
> >http://localhost:8080/imaging/icon/def/128/255(where def is the name
> > of the image - default in this case - 128 is the size in pixels and 255
> > is the background color), I am getting:
>
> > Document Path:          /imaging/icon/def/128/255
> > Document Length:        14417 bytes
>
> > Concurrency Level:      1
> > Time taken for tests:   18.664 seconds
> > Complete requests:      1000
> > Failed requests:        0
> > Broken pipe errors:     0
> > Total transferred:      14680000 bytes
> > HTML transferred:       14417000 bytes
> > Requests per second:    53.58 [#/sec] (mean)
> > Time per request:       18.66 [ms] (mean)
> > Time per request:       18.66 [ms] (mean, across all concurrent
> > requests)
> > Transfer rate:          786.54 [Kbytes/sec] received
>
> > FYI: This request returns a PNG image (image/png) and not html
>
> > My understanding is that the problem is either with the CherryPy setup
> > (which is likely because even in other cases, i don't get much more
> > than 65 requests per second) or PIL itself (even though I'm caching the
> > background images and source images)without knowing more about what kind of source images you're using, the
> amount of data they represent etc, it's not entirely obvious to me that 50+
> images per second on a single server is that bad, really.  unless the source
> images are rather tiny, that's plenty of pixels per second to crunch.
>
> are you really receiving more than 4500000 source images per day per
> server?
> 
> </F>