Can XML-RPC performance be improved?

[skip at pobox.com]

    Diez> Sion Arrowsmith wrote:
    >> I've got an established client-server application here where there
    >> is now a need to shovel huge amounts of data (structured as lists of
    >> lists) between the two, and the performance bottleneck has become
    >> the amount of time spent parsing XML ...

    Diez> CORBA. Or TCP/IP directly. I know that isn't really the answer
    Diez> you're looking for - but there is an overhead parsing XML, and
    Diez> that can't go away.

Ah, but if both ends of the pipe happen to be running Python, you can
cheat. ;-)  Run your list through marshal.dumps() before passing it to
xmlrpclib, then undo the marshalling on the other end.

    >>> biglist = ["abc"] * 50
    >>> biglist = [biglist] * 50
    >>> import xmlrpclib
    >>> import marshal
    >>> x = xmlrpclib.dumps((biglist,))
    >>> len(x)
    92331
    >>> x.count("<")
    10310
    >>> y = xmlrpclib.dumps((marshal.dumps(biglist),))
    >>> len(y)
    20324
    >>> y.count("<")
    8

If you can swing it, I'd be willing to bet you a beer your XML parsing time
will all but disappear.

If you're not using xmlrpclib with some sort of C helper (like sgmlop), try
that as well.  Big difference parsing XML in C and Python.

Skip