[Image-SIG] floating point color with PIL again

[Georg Mischler]

Hi again,

I hate to follow up to my own message, but it looks like I
have no choice but to annoy this forum just until I get
*some* response.
(Yes, I know this is free software and I shouldn't complain,
but after investing lots of thought into the matter and
potentially even larger amounts of work, calling out
repeatedly without an echo is still frustrating.)


I wrote:
> The result are the floating point RGB radiance pixels.
> How should I store those? None of the image types I could
> see is big enough to handle this. Should I create a new
> "RGBf" (or whatever) image type?
> 
> In Storage.c this would look like:
> 
> 	...
> 	} else if (strcmp(mode, "RGBf") == 0) {
> 		/* 3x32-bit floating point images */
> 		im->bands = 3;
> 		im->pixelsize = 12; /* ?? */
> 		im->linesize = xsize * 3
> 		im->type = IMAGING_TYPE_FLOAT32;
> 	} ...
>
[...] 
> Did I miss anything that would make this break?
> Is there a way to get the same effect without messing with
> the PIL sources?

Reading more carefully through the sources, it dawns on me
that this would break almost everything.

Most functions in PIL silently assume that an image will
have a pixel size of either one or four bytes. This allows
for many optimisations, depending on the values of the
im->image8 and im->image32 members. The typical approach
is to loop first over im->ysize and then over im->xsize
with either UINT8 or UINT32 steps. This often already
breaks the experimental modes currently implemented, and
would leave 2/3 of a floating point color image untouched.

As a consequence, getting a PIL based solution for my
problem requires a more general approach for looping over
the image data. I can see several possible approaches:

a) Define eg. IMAGING_TYPE_FLOAT32x3 and modify the looping
   macros (where macros are used) to accept the xsize to
   loop over as an argument, set to im->xsize*im->bands for
   IMAGING_TYPE_FLOAT32x3.  This still breaks for functions
   that don't access or write the data in each line in
   forward direction (eg. FlipLeftRight()) and requires
   some additional work for those functions where the loop
   can't easily be substituted by a macro. The current
   experimental modes would also benefit from this approach.

b) Have all loops actually loop over im->ysize, im->xsize
   and finally im->bands in im->pixelsize/im->bands
   steps. This is be the most general approach, but 
   defeats all optimisations currently in place. Probably
   not a good idea.

c) Wrap the floating point color data in a "super image"
   data structure, actually containing three normal F type
   images. This object could use the existing procedures
   to have it's subimages treated seperately where possible,
   collecting the results and composing them into a new
   super image. Some of the interesting operations of course
   depend on the full color values and would have to be
   implemented completely from scratch. In any case, this
   rather looks like an ugly hack to me, and would only make
   some sense if there was no way to get my modifications
   back into the official PIL package (to prevent having
   to adapt my code to every new release).

d) Add another if/else clause to every function operating
   on images, and implement extra routines to handle just
   my special case. As a general solution, this looks ugly.
   In some special cases though, method a) probably
   requires to do this for real tricky procedures, and
   ImagingTransformFilter() is already implemented in a
   very flexible variation of this approach. 

e) Some genius will no doubt find a better way.


I am willing to implement one of the solutions above,
preferring a) or whatever the experts convince me to.
Yes, I realize that this means a mayor overhaul of
almost all of PIL. But this still looks far more 
attractive than having to start from scratch or not being
able to implement my image manipulation stuff at all.

I don't expect to cover everything in the first run
anyway. I will implement the changes to those functions
where I need them myself first, and possibly modify others
only to prevent potential disaster. There are several
areas where the floating point color images will not
support the standard ways to do things by principle.

First, translation to other formats will not be possible
through the normal convert() mechanisms, since there is no
direct mapping between radiance values and image brightness.
There will be an extra reduce(?) function, that takes
into account the desired exposure of the image, the gamma
of the target medium and the spectral lighting characteristics
under which the image was originally created. The internal
sequence of operations will be:
  rgb radiance(w/sr/m2) x spectral weight
     -> rgb luminance(cd/m2 or nits) x gamma x exposure
        -> image brightness (0-255 rgb or greyscale).

This also prevents direct display by any of the current
GUI options (Tkinter in my case). It will be necessary
to convert (reduce) the image to one of the shorter formats
before applying to a PhotoImage or similar. As an alternative,
this conversion might be encapsulated in the image object
itself, so it would look like a RGB, L or I image to the
functions that can only handle those, and do the conversion
on the fly. I don't see an easy way to implement this in
current PIL, but it's a theoretical concept for trading
memory consumption against cpu cycles.

I assume that the generalisations of method a) would also
prepare PIL for raster data that uses more than three bands.
If anyone is interested in this, please let me know.  Maybe
we could share some of the work, or I could at least try
not to sabotage your needs with my changes. Any other comments
or suggestions are highly welcome.


Ok, that's all I can currently offer, so let's see if
it finally stirs up some attention... ;)


Regards

-schorsch


PS:
Before the question comes up: Yes, I have considered buying
a support contract, but although my project has a very
commercial look (and I announced it this way), It's still
far from generating any revenues. Since I currently have
more time than money (I'm already about a year behind schedule,
so who cares), I need to stick to the do-it-yourself approach
for now.  We'll see what happens when (or if) I ever start
to sell any of my software.

-- 
Georg Mischler  --  simulation developper  --  schorsch at schorsch.com
+schorsch.com+  --  lighting design tools  --  http://www.schorsch.com/