[pypy-issue] [issue868] Pypy json.dumps >100% slower than CPython 2.7, simplejson

[Xavier Morel]

New submission from Xavier Morel <bugs.pypy.org at masklinn.net>:

Test setup: 2010 15" Macbook Pro (2.4GHz i5, 8GB RAM, Intel X25M 160GB), OSX 10.6.8
Python 2.6.7 (Macports)
Python 2.7.2 (Macports)
Pypy 1.6.0 (Aug 22 2011, 12:40:28, Macports)
simplejson 2.2.1 (pypi/pip, includes C speedups)

* Pypy is very fast at dumping an empty dict (~2.2µs, Python 2.7 takes 5µs)
* Pypy is much slower as soon as there are data in the dict, getting performances closer to CPython 2.6 than CPython 2.7:
  - 5 keys, 2 integer values, 3 string values
    + Pypy runtime is 2x CPython 2.7
    + Pypy runtime is 0.62x CPython 2.6
  - 5 keys, 1 integer value, 1 string value, 3 simple dicts (from above)
    + Pypy runtime is 4.7x CPython 2.7
    + Pypy runtime is 0.7x CPython 2.6
* As can be seen above, Pypy's json.dumps performances worsen (relatively to CPython implementations) as the dict to dump gets bigger and more 
complex. The worst case is a ridiculous test file I found on the 'net (671K JSON file) where pypy completely breaks down, with a runtime of 
260x CPython 2.7 (interestingly, while CPython 2.6's json could not load the file if I dumped a pickle from CPython 2.7 and loaded it in 2.6 
CPython 2.6 could json.dumps it and had a similar breakdown: Pypy's runtime is only 1.55x CPython 2.6 on that ridiculously huge object)

python2.6 bench.py
Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <type 'exceptions.RuntimeError'> ignored
Test case: EMPTY
	 loops per run 1267170
	Best performances: 8.09 usec/loop
Test case: SIMPLE
	 loops per run 387650
	Best performances: 26.95 usec/loop
Test case: NESTED
	 loops per run 110270
	Best performances: 94.09 usec/loop
python2.7 bench.py
Test case: EMPTY
	 loops per run 1248310
	Best performances: 4.97 usec/loop
Test case: SIMPLE
	 loops per run 1219280
	Best performances: 8.25 usec/loop
Test case: NESTED
	 loops per run 724410
	Best performances: 13.99 usec/loop
Test case: HUGE
	 loops per run 3400
	Best performances: 3085.52 usec/loop
python2.7 bench.py --simplejson
Test case: EMPTY
	 loops per run 1726060
	Best performances: 5.80 usec/loop
Test case: SIMPLE
	 loops per run 1030550
	Best performances: 9.65 usec/loop
Test case: NESTED
	 loops per run 574570
	Best performances: 16.22 usec/loop
Test case: HUGE
	 loops per run 3620
	Best performances: 2821.45 usec/loop
pypy-c bench.py
Test case: EMPTY
	 loops per run 517820
	Best performances: 2.19 usec/loop
Test case: SIMPLE
	 loops per run 48360
	Best performances: 16.94 usec/loop
Test case: NESTED
	 loops per run 15900
	Best performances: 65.57 usec/loop
Test case: HUGE
	 loops per run 10
	Best performances: 801478.39 usec/loop

Bench file linked below, HUGE case testfile is http://json-test-suite.googlecode.com/files/sample.zip

----------
files: bench.py
messages: 3121
nosy: masklinn, pypy-issue
priority: bug
status: unread
title: Pypy json.dumps >100% slower than CPython 2.7, simplejson

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue868>
________________________________________