[Python-Dev] Status of json (simplejson) in cpython

[Michael Foord]

On 17/04/2011 07:28, "Martin v. Löwis" wrote:
>> Well, there was a 5x speedup demonstrated comparing simplejson to the
>> standard library json module.
> Can you kindly point to that demonstration?
>
Hmm... according to a later email in this thread it is 350ms vs 250ms 
for an 11kb sample. That's a nice speedup but not a 5x one. Bob Ippolito 
did claim that simplejson was faster than json for real world workloads 
and I see no reason not to believe him. :-)

>> That sound like *very* worth pursuing (and
>> crazy not to pursue). I've had json serialisation be the bottleneck in
>> web applications generating several megabytes of json for some requests.
> Hmm. I'd claim that the web application that needs to generate several
> megabytes of json for something should be redesigned.

It was displaying (including sorting) large amounts of information in 
tables through a web ui. The customer wanted all the information 
available in the ables, so all the data needed to be sent. We did 
filtering on the server side where possible to minimize the data sent, 
but it was ~10mb for many of the queries. We also cached the data on the 
client and only updated as needed.

We could have "redesigned" the customer requirements I suppose...

>   I also wonder
> whether the bottleneck was the *generation*,
The bottleneck was generation. I benchmarked and optimised. (We were 
using simplejson but I trimmed down the data sent to the absolute 
minimum needed by the client app rather than merely serialising all the 
source data from the django model objects - I didn't optimise within 
simplejson itself...)

> the transmission, or
> the processing of the data on the receiving end.
>
Processing was done in IronPython in Silverlight using the .NET 
de-serialization APIs which were dramatically faster than the Python 
handling on the other side.

All the best,

Michael

> Regards,
> Martin


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html