Writing big XML files where beginning depends on end.

[uche.ogbuji at gmail.com]

"""
 Then we need something that allows parts of the XML file
to be written to file and purged from RAM to avoid the
memory problem.

Suggestions for solutions are appreciated.
"""

Multiple XML files is not an option, but what about general entities or
XInclude?  That way you don't need to change your parsing code.

Using 4Suite's MarkupWriter [1] you could write the outer shell and
inner subtrees to separate streams, only filling in values for the
outer stream when the inner stream is complete, and your computations
are ready.  You can then use the writer.xmlFragment method to stitch
the inner subtrees to the outer shell.  MarkupWriter operates in
streaming mode, so you would not be holding much XML in memory at all.

http://www.xml.com/pub/a/2005/04/20/py-xml.html

--
Uche Ogbuji                               Fourthought, Inc.
http://uche.ogbuji.net                    http://fourthought.com
http://copia.ogbuji.net                   http://4Suite.org
Articles: http://uche.ogbuji.net/tech/publications/