[Expat-discuss] TCP live stream buffer and expat xml parsing

[Nick MacDonald]

Mikhail:

eXpat can handle the supplied data in chunks smaller than the whole
file/message, so I assume you're running into the following problem:

According to the XML spec, a properly formed XML document can have
only ONE root element.... it appears you are attempting to pass more
than one to eXpat...  You would need to detect the end of one
document, and reset the parsing for the next... or you could probably
use a bit of a hack...  Just pass in your own buffer at the
beginning... with your own root tag...
and you won't need to supply the ending root tag until such time as
you want to shut down parsing with eXpat...

Right now, your root tags look like <BGP_MESSAGE> ...  so instead of this
<BGP_MESSAGE>
</BGP_MESSAGE>
<BGP_MESSAGE>
</BGP_MESSAGE>

which looks like two root <BGP_MESSAGE> tags in a row... feed in
something else like

<BGPMessageParser>
<!-- this is the stuff from above -->
<BGP_MESSAGE>
</BGP_MESSAGE>
<BGP_MESSAGE>
</BGP_MESSAGE>
</BGPMessageParser>

where I "magically" prefixed it with a "BGPMessageParser> tag of my
own invention...

As far as I know, that should work for you...  You'd still need to
reset everything on any errors in the supplied data... but you should
have been already thinking about that problem before as nothing
changes in error handling in this new approach...

Hope that helped... Good luck...

Nick



On Sun, Jul 18, 2010 at 12:14 PM, Mikhail Strizhov
<strizhov at cs.colostate.edu> wrote:
> I have live tcp xml stream and each xml message has same format:
>
> <BGP_MESSAGE length="00001914" version="0.2"
> xmlns="urn:ietf:params:xml:ns:xfb-0.2" type_value="3" type="MESSAGE">
> ...other_xml_items_here...
> </BGP_MESSAGE>
>
> <BGP_MESSAGE length="00002918" version="0.2"
> xmlns="urn:ietf:params:xml:ns:xfb-0.2" type_value="3" type="MESSAGE">
> ...other_xml_items_here...
> </BGP_MESSAGE>
>
> <BGP_MESSAGE length="00002184" version="0.2"
> xmlns="urn:ietf:params:xml:ns:xfb-0.2" type_value="3" type="MESSAGE">
> ...other_xml_items_here...
> </BGP_MESSAGE>
>
> When I'm calling TCP recv function to get data from socket I need to specify
> size of buffer, lets say 4096 bytes.
> Usually one <BGP_MESSAGE>..</BGP_MESSAGE> message is around 2500-3000 bytes.
> In this case I'm getting 1st full message and half of next.
> Afterwards I'm forwarding this buffer to XML_Parse function - 1st message
> parsed successfully, but 2nd is half parsed and then error messages.
>
> Is anybody know how to handle live tcp stream with libexpat?
>
> My code is large to attach, its available here -
> http://www.netsec.colostate.edu/~strizhov/bgpmon/bgpmonclient.c