Sequential XML parsing with xml.sax

[peter at hardy.dropbear.id.au]

Hi hi.

I'm trying to do sequential decompression of a bzipped XML file and
feed it to a SAX parser with the following code.

remotefh = urllib.urlopen('file:///home/peter/catalog.rdf.bz2')
decompressor = bz2.BZ2Decompressor()
handler = CatalogueDocumentHandler(sys.stdout)
chunksize = 2048
data = remotefh.read(chunksize)
while data != '':
    out = decompressor.decompress(data)
    if out != '':
        xml.sax.parseString(out, handler)
    data = remotefh.read(chunksize)

This fails with the first chunk of decompressed data passed to
xml.sax.parseString. I'm suspecting because it's an incomplete fragment
of XML. I've tried with a number of different chunk sizes, putting the
break in different places, but it always fails on the first call. For
reference, the traceback looks like:

    xml.sax.parseString(out, handler)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/__init__.py",
line 47, in parseString
    parser.parse(inpsrc)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py",
line 109, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/xmlreader.py",
line 125, in parse
    self.close()
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py",
line 226, in close
    self.feed("", isFinal = 1)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py",
line 220, in feed
    self._err_handler.fatalError(exc)
  File "/usr/lib/python2.4/site-packages/_xmlplus/sax/handler.py", line
38, in fatalError
    raise exception
xml.sax._exceptions.SAXParseException: <unknown>:15132:63: no element
found

(line 15132 is the last, incomplete line feed to parseString. FWIW,
it's:
  <pgterms:friendlytitle rdf:parseType="Literal">Searchlights o
)

The API reference isn't clear on whether parseString can only handle
discrete bits of valid XML, or if it's designed to be called in this
way. So I'm not sure if I'm misusing the function, or if I've done
something else wrong.

Any pointers?
Thanks,
-- 
Pete