WSGI/wsgiref: modifying output on windows ?

[tjollans@g-NOSPAM-mail.com]

Hi,

I am currently trying to port my web software AFoC <http://afoc.k.vu>
to Windows and have hit a strange problem: it seams that binary files
(PNG images in this case) get distorted by wsgiref. (I have tried both
the CGIHandler with both Xitami and Apache2 and WSGIServer). My
application is returning a list of one item (the image as a string).
The image is intact at that point. (tested by writing it to disk and
comparing); The client gets a version with a few added \r characters
and something missing at the end, but of the same overall length. Here
are specific examples:

>>> f1 = file('afoc_bak', 'rb')
>>> f2 = file('box_top.png', 'rb')
>>> s1 = f1.read()
>>> s2 = f2.read()
>>> s1 == s2
False
>>> s1
"\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR
\x00\x00\x00\x96\x00\x00\x00\x14\x08\x06\x00
\x00\x00oy\xc41\x00\x00\x00\x04sBIT\x08\x08\x08\x08|\x08d
\x88\x00\x00\x00\x19tEX
tSoftware\x00www.inkscape.org\x9b\xee<\x1a\x00\x00\x01\x95IDATh\x81\xed
\xd9?K#A\
x18\xc7\xf1\xef\xecnv\xb3FQA\x8b\xf3@\xc1?\xe0)*\xda\xd9(v\xd7+r
\xb6\x82\xef\xc3
\xce\x17\xe1\xab\xb0\x15k\x0b\x1b\x1b\xe1@\x8eK\xa1\x85\x044F
\x8d1\x9a8\x16\x93\
x80\xc4F]u\x89\xfc>\xcd\xb0\x0fS<\xc5\x8fyfw\x8d\xb5\x16\x91\x8f
\xe6\xa5\xdd\x80
|O\n\x96|\n\x05K>\x85\x01\x06\xd2nB\xda\xc6\xe3\xb3\xb5\x06<\x00Uk\xed}
\xeb\xc6\
x00\xf8\xf1\x85\x8dI{\xab7\xd6f\xb0\xee\x81\xaa1\xa6\x02\x94\xad
\xb5\x95\xe6\xc6
\x00\x9dX\xf2zU\xc0\xc7\x05\xac\xd6x\xbe\x03n\x81\xd8\x18s\x03\\Yk\xeb:
\xb1\xe4-
n\x80\x08\x17\xac\x07\\\xb0n\x812\x90\x05B\xc07\xc6\x94\x82\xf9\xa5\xad
\xb5\xd4\
xda\x94\xb6\xe0y\xbe\x0fP*\xfe?>\xc9\xef\x15J\xc5\x7f\x97\xb81x
\x87\x0bUL#T\xb8{
\xbb5\xab\xeb\x07\xfaB*oR)\x17\xce\x8f\x0e\xb7w\xf3\xc7;\x7f\x81k
\xe0\n(\x01\x97
@\x11\xb8\xf0'g76S\xecQ\xdaP&\xccu\xfc\x1c\\\x98\n\xa3n\xef\xect\xbf
\xd0([\xdc\x
a5\xbe\x0e\xd4\xf4\x1dK\xdem\xf4\xd7\xca\xc2\xc8\xf8\xf24\xd0\x05t
\x02\x1d\xb8\x
b1\x18+X\xf2n\xc6xfbf}\x11\x17\xa8,\xeeb\x1f\x01\xa1\x82%
\x89\xc4\xb9\xfe\x9e\xb
1\xc9?\x11\xee2_\xc7\x8dC\xab`IbC\xc3\xbf\xe7p\xa3\xb0\xf9f\xa8\x7f
\x85\x92\\\x9
4\xed\xedk\xad)X\x92X&\xcc\xf5\xb6\xd6\x14,I,\x08\xe2\x9e\xd6\x9a\x82%
\x89y~&\xf
b\xa2\x96F#\xf2\xfd=\x01\x19c^\xe3KN&\x89\x00\x00\x00\x00IEND\xaeB`
\x82"
>>> s2
"\x89PNG\r\r\n\x1a\r\n\x00\x00\x00\rIHDR
\x00\x00\x00\x96\x00\x00\x00\x14\x08\x06
\x00\x00\x00oy\xc41\x00\x00\x00\x04sBIT\x08\x08\x08\x08|\x08d
\x88\x00\x00\x00\x1
9tEXtSoftware\x00www.inkscape.org\x9b\xee<\x1a\x00\x00\x01\x95IDATh
\x81\xed\xd9?
K#A\x18\xc7\xf1\xef\xecnv\xb3FQA\x8b\xf3@\xc1?\xe0)*\xda\xd9(v\xd7+r
\xb6\x82\xef
\xc3\xce\x17\xe1\xab\xb0\x15k\x0b\x1b\x1b\xe1@\x8eK\xa1\x85\x044F
\x8d1\x9a8\x16\
x93\x80\xc4F]u\x89\xfc>\xcd\xb0\x0fS<\xc5\x8fyfw\x8d\xb5\x16\x91\x8f
\xe6\xa5\xdd
\x80|O\r\n\x96|\r\n\x05K>\x85\x01\x06\xd2nB\xda\xc6\xe3\xb3\xb5\x06<
\x00Uk\xed}\
xeb\xc6\x00\xf8\xf1\x85\x8dI{\xab7\xd6f\xb0\xee
\x81\xaa1\xa6\x02\x94\xad\xb5\x95
\xe6\xc6\x00\x9dX\xf2zU\xc0\xc7\x05\xac\xd6x\xbe\x03n\x81\xd8\x18s\x03\
\Yk\xeb:\
xb1\xe4-n\x80\x08\x17\xac\x07\\\xb0n\x812\x90\x05B
\xc07\xc6\x94\x82\xf9\xa5\xad\
xb5\xd4\xda\x94\xb6\xe0y\xbe\x0fP*\xfe?>\xc9\xef\x15J\xc5\x7f\x97\xb81x
\x87\x0bU
L#T\xb8{\xbb5\xab\xeb\x07\xfaB*oR)\x17\xce\x8f\x0e\xb7w\xf3\xc7;\x7f
\x81k\xe0\r\
n(\x01\x97@\x11\xb8\xf0'g76S\xecQ\xdaP&\xccu\xfc\x1c\\\x98\r\n\xa3n\xef
\xect\xbf
\xd0([\xdc\xa5\xbe\x0e\xd4\xf4\x1dK\xdem\xf4\xd7\xca
\xc2\xc8\xf8\xf24\xd0\x05t\x
02\x1d\xb8\xb1\x18+X\xf2n\xc6xfbf}\x11\x17\xa8,\xeeb\x1f\x01\xa1\x82%
\x89\xc4\xb
9\xfe\x9e\xb1\xc9?\x11\xee2_\xc7\x8dC\xab`IbC\xc3\xbf\xe7p\xa3\xb0\xf9f
\xa8\x7f\
x85\x92\\\x94\xed\xedk\xad)X\x92X&\xcc\xf5\xb6\xd6\x14,I,\x08\xe2\x9e
\xd6\x9a\x8
2%\x89y~&\xfb\xa2\x96F#\xf2\xfd=\x01\x19c^\xe3KN&
\x89\x00\x00\x00\x00IE"
>>>

a thought that springs to mind is that this may be related to the
barmy DOS relic of binary/ASCII file destinction; however, this should
not be a problem if I understand PEP 333 correctly:

"""The server or gateway should treat the yielded strings as binary
byte sequences: in particular, it should ensure that line endings are
not altered."""

Might this be a bug in wsgiref ? I will hopefully be able to do some
more testing, e.g. with the FLUP FastCGI back-end, in the next few
days.

Kind regards,

Thomas Jollans