[pypy-dev] Speed of numpy vs numpypy

[PIERRE AUGIER]

Hi,

HPy becomes somehow usable :-) I investigated what could give HPy with something similar to Numpy (piconumpy, a C extension providing an array class). Some very simple preliminary results:

https://github.com/hpyproject/piconumpy/tree/microbench_loop_sum/bench/microbench_loop_sum

Just an explicit loop to sum 10000 random floats with PyPy (and Julia for comparison):

{'cache_tag': 'pypy37',
 'version': sys.pypy_version_info(major=7, minor=3, micro=6, releaselevel='final', serial=0)}
list                          : 1.73e-05 s (  1.6 * Julia)
purepy                        : 1.97e-05 s (  1.9 * Julia)
numpy                         : 4.12e-03 s (388.6 * Julia)
_piconumpy_hpy                : 2.14e-04 s ( 20.2 * Julia)
_piconumpy_cpython_capi       : 1.19e-03 s (112.1 * Julia)

I have no idea how much HPy+PyPy could be improved in terms of perf for such code.

Pierre

----- Mail original -----
> De: "Matti Picus" <matti.picus at gmail.com>
> À: "pypy-dev" <pypy-dev at python.org>
> Envoyé: Samedi 9 Octobre 2021 23:56:06
> Objet: Re: [pypy-dev] Speed of numpy vs numpypy

> On 9/10/21 12:31 am, Werner Heisenberg wrote:
>> Hi everyone
>>
>> I read the pypy FAQ and some blog posts, which leaves me puzzled and
>> this is why I'm asking here:
>>
>> I'm doing data science and use numpy as only library in the code
>> I'm talking about.
>> A few years ago I used pypy to great success to speed up an algorithm
>> of mine without much development effort.
>> I had to rewrite the code a bit so numpypy was digesting it but it
>> worked great.
>>
>> Now I am trying to do the same thing and numpypy seems to have been
>> dropped in favor of complete support of numpy. In theory this is great
>> - however now my algorithm is at least 4x slower with pypy. Is numpypy
>> is no longer around or is there a way to install it and try the code
>> with it?
>>
>> Is the numpy support by support of the c extension which the JIT
>> cannot optimise? Does this make pypy slower than Cpython? There have
>> been some blog posts about progress to make it faster from 2017 but
>> then nothing.
>>
>> So is there anything I can do?
>>
>> I do not want to write all the numpy stuff I need in plain python as
>> this would take a lot of time and then I would be so close to C, that
>> I could implement everything in C straight away.
>>
>> Best regards
>> Nikolai
>>
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
> 
> 
> It turns out the approach in numpypy was not sustainable with the
> current level of resources available to PyPy. We have a hard enough time
> trying to catch up with CPython, and are only now releasing a PyPy3.8,
> when CPython has recently released 3.10. Keeping up with NumPy, with
> their twice-early release schedule, in addition to the other maintenance
> tasks for PyPy, was too much, given our very limited budget for ongoing
> maintenance. We ended up disabling the numpypy extension for python3+
> since the code no longer translated. As Dan Stromberg mentioned,
> currently we view HPy as the way forward: we predict it will allow the
> calls to C code to be closer to the speed of CPython. Note this is a
> more modest goal than numpypy, since numpypy could have provided
> Numba-like performance.
> 
> I guess the answer to "what can you do" could be anything from
> 
> - help up move HPy forward
> 
> - run an effort to revive numpypy (this will have to be a long-term
> commitment)
> 
> - get involved in ongoing maintenance of PyPy which would free up other
> resources.
> 
> Matti
> 
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev