parsing multiple root element XML into text

[Stefan Behnel]

Burak Arslan, 09.05.2014 18:52:
> On 05/09/14 16:55, Stefan Behnel wrote:
>> ElementTree has gained a nice API in
>> Py3.4 that supports this in a much saner way than SAX, using iterators.
>> Basically, you just dump in some data that you received and get back an
>> iterator over the elements (and their subtrees) that it generated from it.
>> Intercept on the right top elements and you get your next subtree as soon
>> as it's ready.
> 
> Here's a small script:

A bit hard to read, though.


>     events = etree.iterparse(istr, events=("start", "end"))
>     stack = deque()
>     for event, element in events:
>     if event == "start":
>     stack.append(element)
>     elif event == "end":
>     stack.pop()
>      
>     if len(stack) == 0:
>     break
>      
>     print(istr.tell(), "%5s, %4s, %s" % (event, element.tag, element.text))
> 
> where istr is an input-stream. (Fully working example:
> https://gist.github.com/plq/025005a71e8135c46800)
> 
> I was expecting to have istr.tell() return the position where the first
> root element ends, which would make it possible to continue parsing with
> another call to etree.iterparse(). But istr.tell() returns the position
> of EOF after the first call to next() on the iterator it returns.

Correct, because it finished parsing. It controls the reading process and
reads ahead, that's how iterparse() works.


> Without the stack check, the loop eventually throws an exception and the
> offset value in that exception is None.
> 
> So I'm lost here, how it'd possible to parse OP's document with lxml?

See my earlier post. Instead of XMLParser, just use the XMLPullParser for
incremental (non-blocking) parsing and processing.

To make this clear, though: to use an XML parser, you need well formed XML,
and that means exactly one root element.

Stefan