Problem with cPickle and cElementTree

[Stefan Behnel]

Barry wrote:
> I recently tried switching from ElementTree to cElementTree.  My
> application parses a collection of large XML files and creates indexes
> based on certain attributes.  This entire collection is saved as an
> instance of my Database class.  Using ElementTree and cPickle has
> allowed me to save these instances and use them later.
> 
> Using cElementTree significantly reduces parse time (~50%) and memory
> ~(15%) but cPickle refuses to pickle the database object.  I receive:
> 
> TypeError: expecting string or Unicode object, NoneType found
> 
> The offending line of code simple shows my invocation of cPickle,
> which is not helpful.
> 
> Doing exactly the same thing with ElementTree works fine.
> 
> It appears that the objects returned by cElementTree do not pickle
> correctly.  Is this a know issue?  I was unable to find any reports of
> this problem.

Pickling is (almost always) built-in for pure Python classes, but it needs to
be implemented explicitly for C classes. cElementTree simply doesn't support this.

If you need a fast and memory friendly XML engine *and* want to pickle
elements, take a look at lxml.objectify. Note that it's only partially
compatible with ElementTree, so you will have to change your code (lxml.etree
is mostly compatible, but it doesn't support pickling). It's a very
easy-to-use XML library, though.

Stefan