[pypy-dev] Question about extension support

[Kevin Modzelewski]

Hi all, thanks for the responses, but I guess I should have been more
explicit -- I'm curious about *why* PyPy is slow on existing extension
modules and why people are being steered away from them.  I completely
support the push to move away from CPython extension modules, but I'm not
sure it's reasonable to expect that programmers will rewrite all the
extension modules they use.

Put another way, I understand how having a JIT-understandable cffi module
will be faster on PyPy than an extension module, but what I don't quite
understand is why CPython extension modules have to be slower on PyPy than
they are on CPython.  I'm not saying that extension modules should be sped
up by PyPy, but I'm curious why they have a reputation for being slower.


On Wed, Mar 26, 2014 at 1:31 AM, Yury V. Zaytsev <yury at shurup.com> wrote:

> On Tue, 2014-03-25 at 16:19 -0700, Kevin Modzelewski wrote:
> >
> > I'm curious, since I've heard a number of people mention that
> > extension modules are the primary reason that PyPy is slower than
> > cPython for their code; definitely an improvement over "PyPy doesn't
> > run my code at all", but it's made me curious about whether or not /
> > why it has to be that way.
>
> In my opinion, it all depends on how do you use CPyExt and what your
> extension modules are for. There are two scenarios here (or combinations
> thereof) that I think cover most of the use cases:
>
> 1) You use C extensions to make it faster.
> 2) You use C extensions to steer external processes.
>
> Ideally with PyPy you should be able to drop (1) altogether and write
> nice Python code that JIT will be able to optimize sometimes even better
> than hand-written C code, so here the answer would be "don't use
> extensions".
>
> Now, if as a part of (2) you are doing some lengthy processing entirely
> outside PyPy, this might still just as fast as with CPython with CPyExt,
> but if the calls to your foreign functions are short and/or you are
> transferring a lot of data C <-> PyPy, then there you go...
>
> Personally, I've been using CPyExt and I'm very happy about it, because
> the function calls take a long time, and whatever happens outside
> doesn't have much to do with objects in PyPy land.
>
> However, if my requirements were different, I would have rather
> re-written everything using cffi, from what I understood it can deliver
> comparable performance to cPython, and also it works both for PyPy and
> cPython, not just PyPy...
>
> --
> Sincerely yours,
> Yury V. Zaytsev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20140326/498a4a99/attachment.html>