[Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

[Maciej Fijalkowski]

On Fri, Feb 12, 2010 at 7:04 PM, Collin Winter <collinwinter at google.com> wrote:
> Hey Maciej,
>
> On Thu, Feb 11, 2010 at 6:39 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> Snippet from:
>>
>> http://codereview.appspot.com/186247/diff2/5014:8003/7002
>>
>> *PyPy*: PyPy [#pypy]_ has good performance on numerical code, but is
>> slower than Unladen Swallow on non-numerical workloads. PyPy only
>> supports 32-bit x86 code generation. It has poor support for CPython
>> extension modules, making migration for large applications
>> prohibitively expensive.
>>
>> That part at the very least has some sort of personal opinion
>> "prohibitively",
>
> Of course; difficulty is always in the eye of the person doing the
> work. Simply put, PyPy is not a drop-in replacement for CPython: there
> is no embedding API, much less the same one exported by CPython;
> important libraries, such as MySQLdb and pycrypto, do not build
> against PyPy; PyPy is 32-bit x86 only.

I like this wording far more. It's at the very least far more precise.
Those examples are fair enough (except the fact that PyPy is not 32bit
x86 only, the JIT is).

> All of these problems can be overcome with enough time/effort/money,
> but I think you'd agree that, if all I'm trying to do is speed up my
> application, adding a new x86-64 backend or implementing support for
> CPython extension modules is certainly north of "prohibitively
> expensive". I stand by that wording. I'm willing to enumerate all of
> PyPy's deficiencies in this regard in the PEP, rather than the current
> vaguer wording, if you'd prefer.
>
>> while the other part is not completely true "slower
>> than US on non-numerical workloads". Fancy providing a proof for that?
>> I'm well aware that there are benchmarks on which PyPy is slower than
>> CPython or US, however, I would like a bit more weighted opinion in
>> the PEP.
>
> Based on the benchmarks you're running at
> http://codespeak.net:8099/plotsummary.html, PyPy is slower than
> CPython on many non-numerical workloads, which Unladen Swallow is
> faster than CPython at. Looking at the benchmarks there at which PyPy
> is faster than CPython, they are primarily numerical; this was the
> basis for the wording in the PEP.
>
> My own recent benchmarking of PyPy and Unladen Swallow (both trunk;
> PyPy wouldn't run some benchmarks):
>
> | Benchmark    | PyPy  | Unladen | Change          |
> +==============+=======+=========+=================+
> | ai           | 0.61  | 0.51    |  1.1921x faster |
> | django       | 0.68  | 0.8     |  1.1898x slower |
> | float        | 0.03  | 0.07    |  2.7108x slower |
> | html5lib     | 20.04 | 16.42   |  1.2201x faster |
> | pickle       | 17.7  | 1.09    | 16.2465x faster |
> | rietveld     | 1.09  | 0.59    |  1.8597x faster |
> | slowpickle   | 0.43  | 0.56    |  1.2956x slower |
> | slowspitfire | 2.5   | 0.63    |  3.9853x faster |
> | slowunpickle | 0.26  | 0.27    |  1.0585x slower |
> | unpickle     | 28.45 | 0.78    | 36.6427x faster |
>
> I'm happy to change the wording to "slower than US on some workloads".
>
> Thanks,
> Collin Winter
>

"slower than US on some workloads" is true, while not really telling
much to a potential reader. For any X and Y implementing the same
language "X is faster than Y on some workloads" is usually true.

To be precise you would need to include the above table in the PEP,
which is probably a bit too much, given that PEP is not about PyPy at
all. I'm fine with any wording that is at least correct.

Cheers,
fijal