[Python-Dev] Unicode entities in XML cause problems :-(

[Matthias Urlichs]

Playing around with xml.dom.minidom, I noticed that this beast is
perfectly able to read HTML which it can't print:

>>> import xml.dom.minidom as md
>>> d=md.parseString("<foo>b&#2000;</foo>"))
>>> d.writexml(sys.stdout)
...
UnicodeError: ASCII encoding error: ordinal not in range(128)

Ouch.

Scanning the sources, which revealed various ways to replace
'&' with '&amp;' but no generic codec for [ht|x]ml-escaped character
entities.

Thus, my proposal (which I'm going to implement since I need it...) is to
write such a codec. For simplicity, I propose to accept &uuml; and &euro;
and friends, but to emit them as &#1234; (or whatever).

After this codec is written, all occurrences of string.replace('&','&amp;')
(and vice versa) within the standard library can be replaced with the
appropriate encode/decode methods. 

Thoughts? Or am I totally blind, such a codec already exists, and I
have missed it?

--
Matthias Urlichs     |     noris network AG     |     http://smurf.noris.de/