[pypy-dev] Pie-thon benchmark

[Alex Martelli]

On Wednesday 31 December 2003 09:39 pm, Seo Sanghyeon wrote:
> Happy new year, pypy-dev!

And a happy new year to you!

I think it's quite important at this stage to have pypy run as much benchmark
and testing CPython code as feasible (I spent the Amsterdam sprint mostly
working on that, mostly pair programming with Pat Maupin, and Laura and
Jacob also worked a lot on adding and fixing built-ins -- as a result we now
have versions of pystone and of the CPython unit tests for built-in types and
functions running under pypy, though quite a bit of the unit tests has had to
be commented out to get there -- plenty of things we can't fix, or can't fix 
yet / right now, or, like complex numbers, don't really want to spend lots 
and lots of energy at this stage).  So, I heartily approve of your work on
pie-thon!

> For PyPy, I used `time python py.py -S b?.py'. To make relative
> import work I prepended `PYTHONPATH=.' for b4.py and b6.py.
>
> b0.py: error   __new__() takes exactly 4 arguments (1 given)

Hmmm -- guess we gotta work on making some of __new__'s args
optional!

> b1.py: error   maximum recursion depth exceeded

(No hypothesis here).

> b2.py: real    1m25.725s (250x slow)

Wow -- more commonly the speed ratio is observed to be in the
thousands (e.g. the fastest laptop at the sprint got 3 pystones/sec
in pypy, vs I believe 50,000 pystones/sec in CPython 2.3.3).

> b3.py: error   __new__ multimethod got a non-wrapped argument

Ouch -- this is sure to be a serious coding error somewhere in pypy.

> b4.py: error   (this one imports and uses b0.py)
> b5.py: error   global name complex is not defined

Right, we haven't worked on complex... there's some code for complex
but no work has been spent making it work as a builtin type, AFAIK.

> b6.py: error   unbound method must be called with
>                instance as first argument
>
> More comments on b6.py: it took 22 minutes(!) to crash. It printed
> 42, 1000041, 999999, 49999950000, and finally crashed at
> `d = dict.fromkeys(xrange(1000000))'.

Ah, yes, this is a known issue -- we can't yet expose classmethods
from built-in types.  We do have classmethods, for user-coded types,
we just haven't yet gotten around to installing fromkeys as a classmethod
into builtin type dict -- so it looks like a normal method instead.


Alex