10GB XML Blows out Memory, Suggestions?

Diez B. Roggisch deets at nospam.web.de
Tue Jun 6 09:33:25 EDT 2006


K.S.Sreeram schrieb:
> axwack at gmail.com wrote:
>> I wrote a program that takes an XML file into memory using Minidom. I
>> found out that the XML document is 10gb.
> 
> With a 10gb file, you're best bet might be to juse use Expat and C!!

No what exactly makes C grok a 10Gb file where python will fail to do so?

What the OP needs is a different approach to XML-documents that won't 
parse the whole file into one giant tree - but I'm pretty sure that 
(c)ElementTree will do the job as well as expat. And I don't recall the 
OP musing about performances woes, btw.

Diez



More information about the Python-list mailing list