From dimaqq at gmail.com Tue Feb 1 21:11:31 2022 From: dimaqq at gmail.com (Dima Tisnek) Date: Wed, 2 Feb 2022 11:11:31 +0900 Subject: [pypy-dev] Making PyPy relevant Message-ID: Dear PyPy folk, I had been a quiet supporter of your project for some years, but lately completely dropped off. I would like to state my reasons in hope that PyPy will not be completely forgotten and thus point a path forward, perhaps hypothetical, but one that I would very much like to see: #1 PyPy must track Python language versions (and CPython stdlib versions) You've released 7.3.8 with 3.8 support and I already use [Python language version] 3.9 in production and 3.10 in CI. (3.9 in prod only because some dependencies are missing a formal update, it will be 3.10 in no time) The components that run [Python language version] 3.8 in prod are a mix of obsolete, unmaintained, and those whose developers are too busy with other things, there's no chance those components would switch to PyPy. You've released 3.9 beta support and I'm running [CPython] 3.11.0a4. I can't use your great work. Ideally PyPy would track these in lock-step (released at the same time); an acceptable compromise may be a 3-month delay. In short: for me (and probably mots developers) PyPy remains an academic exercise. Thank you, Dima Tisnek P.S. My wish list: #2 Move to GitHub already. Your current repo setup makes it impossible for the majority to of developers to contribute. #3 Focus on one major feature, that is visible to the developers, and not old -- it could be, for example, typing or async/await, but probably not multithreading. The impact of your amazing work is proportional to the number of users. The average user is interested more in language usability and frist-class language features; performance comes second. From steve at pearwood.info Wed Feb 2 10:11:35 2022 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 3 Feb 2022 02:11:35 +1100 Subject: [pypy-dev] Making PyPy relevant In-Reply-To: References: Message-ID: <20220202151135.GF16660@ando.pearwood.info> Hi Dima, On Wed, Feb 02, 2022 at 11:11:31AM +0900, Dima Tisnek wrote: > #1 PyPy must track Python language versions (and CPython stdlib versions) > > You've released 7.3.8 with 3.8 support and I already use [Python > language version] 3.9 in production and 3.10 in CI. [...] > Ideally PyPy would track these in lock-step (released at the same > time); an acceptable compromise may be a 3-month delay. Most people do not track the latest Python. They use their vendor's Python, which may be a number of releases back, or even whatever legacy version their application is written for. For example, there are still people using Python 2.7 https://access.redhat.com/solutions/4455511 and the default version of Python that ships with RHEL 8 is changing from 3.6 to 3.8. Not everyone moves fast to the latest version of the language. For some people stability is more valuable than features. I'm sure that the PyPy devs would support 3.11 if they had the manpower. Did you think that they just hadn't noticed that CPython was up to 3.10 and 3.11 is in development? > #2 Move to GitHub already. Your current repo setup makes it impossible > for the majority to of developers to contribute. "Impossible"? Like you literally cannot get your head around a different repo than GitHub? How will you cope if your job requires you to learn new skills? Or even a new language? > performance comes second. That's a strange thing to say to a Python interpreter whose reason for existence is to improve performance. -- Steve From oliver.margetts at gmail.com Wed Feb 2 11:11:30 2022 From: oliver.margetts at gmail.com (Oliver Margetts) Date: Wed, 2 Feb 2022 16:11:30 +0000 Subject: [pypy-dev] Making PyPy relevant In-Reply-To: <20220202151135.GF16660@ando.pearwood.info> References: <20220202151135.GF16660@ando.pearwood.info> Message-ID: As a long term user, I admit I do like the shiny new things - (type hints and f-strings ... bliss). But I actually think pypy's cadence is very promising. CPython releases are now yearly, but on the pypy side the 3.8 rc came out and 3.9 is in beta only 9 months after 3.7 was released. So kudos on that front! If that pace is sustained and CPython is caught up with it might actually mean you actually have to slow down ;-) On Wed, 2 Feb 2022 at 15:12, Steven D'Aprano wrote: > Hi Dima, > > On Wed, Feb 02, 2022 at 11:11:31AM +0900, Dima Tisnek wrote: > > > #1 PyPy must track Python language versions (and CPython stdlib versions) > > > > You've released 7.3.8 with 3.8 support and I already use [Python > > language version] 3.9 in production and 3.10 in CI. > [...] > > Ideally PyPy would track these in lock-step (released at the same > > time); an acceptable compromise may be a 3-month delay. > > Most people do not track the latest Python. They use their vendor's > Python, which may be a number of releases back, or even whatever legacy > version their application is written for. > > For example, there are still people using Python 2.7 > > https://access.redhat.com/solutions/4455511 > > and the default version of Python that ships with RHEL 8 is changing > from 3.6 to 3.8. > > Not everyone moves fast to the latest version of the language. For some > people stability is more valuable than features. > > I'm sure that the PyPy devs would support 3.11 if they had the manpower. > Did you think that they just hadn't noticed that CPython was up to 3.10 > and 3.11 is in development? > > > > > > #2 Move to GitHub already. Your current repo setup makes it impossible > > for the majority to of developers to contribute. > > "Impossible"? Like you literally cannot get your head around a different > repo than GitHub? > > How will you cope if your job requires you to learn new skills? Or even > a new language? > > > > performance comes second. > > That's a strange thing to say to a Python interpreter whose reason for > existence is to improve performance. > > > > -- > Steve > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From phyoakl at hexcode.tech Wed Feb 2 16:25:44 2022 From: phyoakl at hexcode.tech (Phyo Arkar Lwin) Date: Thu, 3 Feb 2022 03:55:44 +0630 Subject: [pypy-dev] Making PyPy relevant In-Reply-To: References: Message-ID: PyPy version is moving up version a lot faster than before. And Pypy 3.8 is mainstream in most development houses and enterprise. It seems P do not People rarely using latest python version But i agree on point that PyPy need to grow its userbase. Many of the people i talked with still think pypy CExt is still slow or incompatible with many C base Exts including Data Science stack. PyPy team needs to demystify those and bring more people in , Mirroring to Github is also a good idea , it can bring more pople and Stars (You know , most CTO choose tech stack base on github stars , not the actual usefulness) On Wed, Feb 2, 2022 at 10:42 PM Oliver Margetts wrote: > As a long term user, I admit I do like the shiny new things - (type hints > and f-strings ... bliss). But I actually think pypy's cadence is very > promising. CPython releases are now yearly, but on the pypy side the 3.8 rc > came out and 3.9 is in beta only 9 months after 3.7 was released. So kudos > on that front! > > If that pace is sustained and CPython is caught up with it might actually > mean you actually have to slow down ;-) > > On Wed, 2 Feb 2022 at 15:12, Steven D'Aprano wrote: > >> Hi Dima, >> >> On Wed, Feb 02, 2022 at 11:11:31AM +0900, Dima Tisnek wrote: >> >> > #1 PyPy must track Python language versions (and CPython stdlib >> versions) >> > >> > You've released 7.3.8 with 3.8 support and I already use [Python >> > language version] 3.9 in production and 3.10 in CI. >> [...] >> > Ideally PyPy would track these in lock-step (released at the same >> > time); an acceptable compromise may be a 3-month delay. >> >> Most people do not track the latest Python. They use their vendor's >> Python, which may be a number of releases back, or even whatever legacy >> version their application is written for. >> >> For example, there are still people using Python 2.7 >> >> https://access.redhat.com/solutions/4455511 >> >> and the default version of Python that ships with RHEL 8 is changing >> from 3.6 to 3.8. >> >> Not everyone moves fast to the latest version of the language. For some >> people stability is more valuable than features. >> >> I'm sure that the PyPy devs would support 3.11 if they had the manpower. >> Did you think that they just hadn't noticed that CPython was up to 3.10 >> and 3.11 is in development? >> >> >> >> >> > #2 Move to GitHub already. Your current repo setup makes it impossible >> > for the majority to of developers to contribute. >> >> "Impossible"? Like you literally cannot get your head around a different >> repo than GitHub? >> >> How will you cope if your job requires you to learn new skills? Or even >> a new language? >> >> >> > performance comes second. >> >> That's a strange thing to say to a Python interpreter whose reason for >> existence is to improve performance. >> >> >> >> -- >> Steve >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev >> > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev On Wed, Feb 2, 2022 at 8:42 AM Dima Tisnek wrote: > Dear PyPy folk, > > I had been a quiet supporter of your project for some years, but > lately completely dropped off. I would like to state my reasons in > hope that PyPy will not be completely forgotten and thus point a path > forward, perhaps hypothetical, but one that I would very much like to > see: > > #1 PyPy must track Python language versions (and CPython stdlib versions) > > You've released 7.3.8 with 3.8 support and I already use [Python > language version] 3.9 in production and 3.10 in CI. > (3.9 in prod only because some dependencies are missing a formal > update, it will be 3.10 in no time) > The components that run [Python language version] 3.8 in prod are a > mix of obsolete, unmaintained, and those whose developers are too busy > with other things, there's no chance those components would switch to > PyPy. > > You've released 3.9 beta support and I'm running [CPython] 3.11.0a4. I > can't use your great work. > > Ideally PyPy would track these in lock-step (released at the same > time); an acceptable compromise may be a 3-month delay. > > > In short: for me (and probably mots developers) PyPy remains an > academic exercise. > > > Thank you, > Dima Tisnek > > > P.S. My wish list: > > #2 Move to GitHub already. Your current repo setup makes it impossible > for the majority to of developers to contribute. > > #3 Focus on one major feature, that is visible to the developers, and > not old -- it could be, for example, typing or async/await, but > probably not multithreading. The impact of your amazing work is > proportional to the number of users. The average user is interested > more in language usability and frist-class language features; > performance comes second. > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mhroncok at redhat.com Fri Feb 4 11:10:38 2022 From: mhroncok at redhat.com (=?UTF-8?Q?Miro_Hron=c4=8dok?=) Date: Fri, 4 Feb 2022 17:10:38 +0100 Subject: [pypy-dev] Making PyPy relevant In-Reply-To: <20220202151135.GF16660@ando.pearwood.info> References: <20220202151135.GF16660@ando.pearwood.info> Message-ID: On 02. 02. 22 16:11, Steven D'Aprano wrote: > and the default version of Python that ships with RHEL 8 is changing > from 3.6 to 3.8. It isn't. The default version of Python that ships with RHEL 8 will always be 3.6. You can also install Python 3.8 and 3.9, but the version of Python that the system actually uses will always be 3.6. -- Miro Hron?ok -- Phone: +420777974800 IRC: mhroncok From dmitryk at lightrun.com Thu Feb 10 01:01:39 2022 From: dmitryk at lightrun.com (Dmitry Kovner) Date: Thu, 10 Feb 2022 16:01:39 +1000 Subject: [pypy-dev] PyCodeObject incompatibility Message-ID: Hello! I'm trying to build a low-level C API extension of cPython to be used in PyPy. The extension extensively uses some fields of PyCodeObject ( https://github.com/python/cpython/blob/f87e616af038ee8963185e11b96841c81e8ef15a/Include/code.h#L23): co_firstlineno, co_stacksize, co_consts and so on. However, it seems these fields are not defined in the similar structure of the PyPy implementation: https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.8/pypy/module/cpyext/include/code.h#L7. Is it possible to get values of these fields using PyPy C API somehow? -------------- next part -------------- An HTML attachment was scrubbed... URL: From armin.rigo at gmail.com Thu Feb 10 08:06:49 2022 From: armin.rigo at gmail.com (Armin Rigo) Date: Thu, 10 Feb 2022 14:06:49 +0100 Subject: [pypy-dev] PyCodeObject incompatibility In-Reply-To: References: Message-ID: Hi, On Thu, 10 Feb 2022 at 14:03, Dmitry Kovner wrote: > Hello! I'm trying to build a low-level C API extension of cPython to be used in PyPy. The extension extensively uses some fields of PyCodeObject (https://github.com/python/cpython/blob/f87e616af038ee8963185e11b96841c81e8ef15a/Include/code.h#L23): co_firstlineno, co_stacksize, co_consts and so on. However, it seems these fields are not defined in the similar structure of the PyPy implementation: https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.8/pypy/module/cpyext/include/code.h#L7. Is it possible to get values of these fields using PyPy C API somehow? Yes, it's a limitation of PyPy not to expose these fields. If you want to write portable code that always works, the simplest is to call `PyObject_GetAttrString(code, "__consts__")` etc. Remember that you need to call `Py_DECREF()` on the result at some point. A bient?t, Armin. From cfbolz at gmx.de Thu Feb 10 08:20:36 2022 From: cfbolz at gmx.de (Carl Friedrich Bolz-Tereick) Date: Thu, 10 Feb 2022 14:20:36 +0100 Subject: [pypy-dev] PyCodeObject incompatibility In-Reply-To: References: Message-ID: Hi Dmitry, those fields are quite implementation-dependant, it's not clear to me that their use can be just carried over from CPython in all cases. Could you maybe tell us which exception module uses them or at least what you are doing with them? But yes, as Armin write, accessing the the .co_* attributes with PyObject_GetAttrString is an approach! Cheers, CF On 10.02.22 07:01, Dmitry Kovner wrote: > Hello! I'm trying to build a low-level C API extension of cPython to be > used in PyPy. The extension extensively uses some fields of PyCodeObject > (https://github.com/python/cpython/blob/f87e616af038ee8963185e11b96841c81e8ef15a/Include/code.h#L23 > ):?co_firstlineno,?co_stacksize,?co_consts > and so on. However, it seems these fields are not defined in the similar > structure of the PyPy? implementation: > https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.8/pypy/module/cpyext/include/code.h#L7 > . > Is it possible to get values of these fields using PyPy C API somehow? > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From armin.rigo at gmail.com Thu Feb 10 08:24:57 2022 From: armin.rigo at gmail.com (Armin Rigo) Date: Thu, 10 Feb 2022 14:24:57 +0100 Subject: [pypy-dev] PyCodeObject incompatibility In-Reply-To: References: Message-ID: Hi, On Thu, 10 Feb 2022 at 14:20, Carl Friedrich Bolz-Tereick wrote: > are doing with them? But yes, as Armin write, accessing the the .co_* > attributes with PyObject_GetAttrString is an approach! Oops, sorry. Python 3 renamed various attributes to use the double-underscore convention, like on function objects, but skipped code objects for some reason. So I meant `PyObject_GetAttrString(code, "co_consts")`. Armin From matti.picus at gmail.com Fri Feb 11 04:36:18 2022 From: matti.picus at gmail.com (Matti Picus) Date: Fri, 11 Feb 2022 11:36:18 +0200 Subject: [pypy-dev] PyPy v7.3.8 rc2 now available In-Reply-To: References: Message-ID: <21554b13-e3dd-5b82-afd5-59bd09e18dec@gmail.com> PyPy v7.2.8 rc2 is now available for testing. Thanks to all who tried out rc1. There are a few bugfixes. The sys.version_info now also correctly reports This is a release of 4 versions of python: 2.7, 3.7, 3.8 (no longer beta) and 3.9 (beta quality). Release note: https://doc.pypy.org/en/latest/release-v7.3.8.html Downloads: https://downloads.python.org/pypy Checksums: https://www.pypy.org/checksums.html Please try these out and report any problems. Note that we are having issues with the ARM8-64bit buildbot, so we are not releasing binary packages for that platform. Hopefully we will have an alternative for the final release. Matti From dmitryk at lightrun.com Mon Feb 14 22:15:33 2022 From: dmitryk at lightrun.com (Dmitry Kovner) Date: Tue, 15 Feb 2022 13:15:33 +1000 Subject: [pypy-dev] PyCodeObject incompatibility In-Reply-To: References: Message-ID: Hi, again! First of all, thanks for the fast reply! The answer helped me a lot. However, during the porting of the extension to the awesome PyPy, I've got more questions: 1. The extension uses opcodes from cPython's opcode.h header file. ( https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/opcode.h) There is no such file in the implementation of C API in PyPy. What could you recommend as the best alternative? 2. The extension uses some fields of PyThreadState structure. For example, its frame field ( https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/cpython/pystate.h#L59). Is it right to use PyObject_GetAttrString() to access that field in PyPy C API? 3. There is no function PyFrame_FastToLocals ( https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Objects/frameobject.c#L931) in the C API of PyPy. Are there any alternatives? 4. Functions PyEval_SetTrace(), PyEval_SetProfile(), PyFrame_GetLineNumber() are defined only in stubs.py in PyPy. ( https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.7/pypy/module/cpyext/stubs.py#L918) Is there a way to get that information in PyPy except patching of its source code? 5. Are there any analogues of all PyTrace_* constants ( https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/cpython/pystate.h#L26) in the PyPy C API? 6. There is no function PyFrame_Check() ( https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/frameobject.h#L53) in the PyPy C API. Are there any alternatives? I'm really sorry for the long question! I hope the list of the problems described above is full enough to port the extension and I will have no more questions. :) Best regards, Dmitrii ??, 10 ????. 2022 ?. ? 23:07, Armin Rigo : > Hi, > > On Thu, 10 Feb 2022 at 14:03, Dmitry Kovner wrote: > > Hello! I'm trying to build a low-level C API extension of cPython to be > used in PyPy. The extension extensively uses some fields of PyCodeObject ( > https://github.com/python/cpython/blob/f87e616af038ee8963185e11b96841c81e8ef15a/Include/code.h#L23): > co_firstlineno, co_stacksize, co_consts and so on. However, it seems these > fields are not defined in the similar structure of the PyPy > implementation: > https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.8/pypy/module/cpyext/include/code.h#L7. > Is it possible to get values of these fields using PyPy C API somehow? > > Yes, it's a limitation of PyPy not to expose these fields. If you > want to write portable code that always works, the simplest is to call > `PyObject_GetAttrString(code, "__consts__")` etc. Remember that you > need to call `Py_DECREF()` on the result at some point. > > > A bient?t, > > Armin. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matti.picus at gmail.com Tue Feb 15 01:23:01 2022 From: matti.picus at gmail.com (Matti Picus) Date: Tue, 15 Feb 2022 08:23:01 +0200 Subject: [pypy-dev] PyCodeObject incompatibility In-Reply-To: References: Message-ID: <2ff9b5d8-681e-65d4-0ca8-ec99bcbfc7fe@gmail.com> On 15/2/22 05:15, Dmitry Kovner wrote: > Hi, again! First of all, thanks for the fast reply! The answer helped > me a lot. However, during the porting of the extension to the awesome > PyPy, I've got more questions: > 1. The extension uses opcodes from cPython's opcode.h header file. > (https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/opcode.h) > There is no such file in the implementation of C API in PyPy. What > could you recommend as the best alternative? > 2. The extension uses some fields of PyThreadState structure. For > example, its frame field > (https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/cpython/pystate.h#L59). > Is it right to use PyObject_GetAttrString() to access that field in > PyPy C API? > 3. There is no function PyFrame_FastToLocals > (https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Objects/frameobject.c#L931) > in the C API of PyPy. Are there any alternatives? > 4. Functions PyEval_SetTrace(), PyEval_SetProfile(), > PyFrame_GetLineNumber() are defined only in stubs.py in PyPy. > (https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.7/pypy/module/cpyext/stubs.py#L918) > Is there a way to get that information in PyPy except patching of its > source code? > 5. Are there any analogues of all PyTrace_* constants > (https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/cpython/pystate.h#L26) > in the PyPy C API? > 6. There is no function PyFrame_Check() > (https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/frameobject.h#L53) > in the PyPy C API. Are there any alternatives? > > I'm really sorry for the long question! I hope the list of the > problems described above is full enough to port the extension and I > will have no more questions. :) > > Best regards, Dmitrii > Even if PyPy could expose the interfaces you desire (and it would be a lot of work to do that), you would lose out on speed since the C-API on PyPy is much much slower than pure python. If you really want to port your extension, you might want to rethink what it is doing and refactor it to do so in a PyPy-friendly way: pure python + CFFI where needed to interface to libraries that expose C interfaces. The C-API is a layer we bolted on top of PyPy to support popular libraries like Cython and NumPy, it is not meant for projects that probe CPython internals. Matti From johnfouf at gmail.com Tue Feb 15 06:35:48 2022 From: johnfouf at gmail.com (Ioannis Foufoulas) Date: Tue, 15 Feb 2022 13:35:48 +0200 Subject: [pypy-dev] Differences between pypy2 and 3 and cffi Message-ID: <3C88391E-E2E8-4B87-A0CC-1C4809BA9FA1@gmail.com> Hi, I have an embedded function: @ffi.def_extern() def toint(input,insize,result): for i in range(insize): x = ffi.string(input[i]) result[i] = float(x) if x else 0 return 1 It seems that this runs around 30-40% percent slower in PyPy3 than in 2. Input is a char** c array and result is a float c array. The difference is not in ffi.string If having `result[i] = 1 if x else 0` it runs same in both versions. Any ideas? From cfbolz at gmx.de Tue Feb 15 13:14:45 2022 From: cfbolz at gmx.de (Carl Friedrich Bolz-Tereick) Date: Tue, 15 Feb 2022 19:14:45 +0100 Subject: [pypy-dev] PyCodeObject incompatibility In-Reply-To: References: Message-ID: <28564e81-5af9-b797-d789-f3e14d5d12e8@gmx.de> Hi Dmitry, I can only reiterate, we would need some information about what that library is actually for to give more useful information at this point. As Matti said, most of these APIs cannot be usefully implemented in PyPy. most jitted functions do not *have* a filled frame object at all in PyPy. so if you want to access the frames, you need to leave the jit-produced machine code, which is very slow. Cheers, CF On 15.02.22 04:15, Dmitry Kovner wrote: > Hi, again! First of all, thanks for the fast reply! The answer helped me > a lot. However, during the porting of the extension to the awesome PyPy, > I've got more questions: > 1. The extension uses opcodes from cPython's opcode.h header file. > (https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/opcode.h > ) > There is no such file in the implementation of C API in PyPy. What could > you recommend as the best alternative? > 2. The extension uses some fields of PyThreadState structure. For > example, its frame field > (https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/cpython/pystate.h#L59 > ). > Is it right to use PyObject_GetAttrString() to access that field in PyPy > C API? > 3. There is no function PyFrame_FastToLocals > (https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Objects/frameobject.c#L931 > ) > in the C API of PyPy. Are there any alternatives? > 4. Functions PyEval_SetTrace(), PyEval_SetProfile(), > PyFrame_GetLineNumber() are defined only in stubs.py in PyPy. > (https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.7/pypy/module/cpyext/stubs.py#L918 > ) > Is there a way to get that information in PyPy except patching of its > source code? > 5. Are there any analogues of all PyTrace_* constants > (https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/cpython/pystate.h#L26 > ) > in the PyPy C API? > 6. There is no function PyFrame_Check() > (https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/frameobject.h#L53 > ) > in the PyPy C API. Are there any alternatives? > > I'm really sorry for the long question! I hope the list of the problems > described above is full enough to port the extension and I will have no > more questions. :) > > Best regards, Dmitrii > > ??, 10 ????. 2022 ?. ? 23:07, Armin Rigo >: > > Hi, > > On Thu, 10 Feb 2022 at 14:03, Dmitry Kovner > wrote: > > Hello! I'm trying to build a low-level C API extension of cPython > to be used in PyPy. The extension extensively uses some fields of > PyCodeObject > (https://github.com/python/cpython/blob/f87e616af038ee8963185e11b96841c81e8ef15a/Include/code.h#L23 > ): > co_firstlineno, co_stacksize, co_consts and so on. However, it seems > these fields are not defined in the similar structure of the PyPy > implementation: > https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.8/pypy/module/cpyext/include/code.h#L7 > . > Is it possible to get values of these fields using PyPy C API somehow? > > Yes, it's a limitation of PyPy not to expose these fields.? If you > want to write portable code that always works, the simplest is to call > `PyObject_GetAttrString(code, "__consts__")` etc.? Remember that you > need to call `Py_DECREF()` on the result at some point. > > > A bient?t, > > Armin. > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From drsalists at gmail.com Tue Feb 15 23:24:29 2022 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 15 Feb 2022 20:24:29 -0800 Subject: [pypy-dev] Differences between pypy2 and 3 and cffi In-Reply-To: <3C88391E-E2E8-4B87-A0CC-1C4809BA9FA1@gmail.com> References: <3C88391E-E2E8-4B87-A0CC-1C4809BA9FA1@gmail.com> Message-ID: I doubt this is an expected result for Pypy3. But what if you use: result[i] = float(x) if x else 0.0 ...to make result be of homogeneous type? On Tue, Feb 15, 2022 at 3:36 AM Ioannis Foufoulas wrote: > Hi, > I have an embedded function: > > @ffi.def_extern() > def toint(input,insize,result): > for i in range(insize): > x = ffi.string(input[i]) > result[i] = float(x) if x else 0 > return 1 > > It seems that this runs around 30-40% percent slower in PyPy3 than in 2. > > Input is a char** c array and result is a float c array. The difference is > not in ffi.string > If having `result[i] = 1 if x else 0` it runs same in both versions. Any > ideas? > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matti.picus at gmail.com Wed Feb 16 02:03:40 2022 From: matti.picus at gmail.com (Matti Picus) Date: Wed, 16 Feb 2022 09:03:40 +0200 Subject: [pypy-dev] Differences between pypy2 and 3 and cffi In-Reply-To: References: <3C88391E-E2E8-4B87-A0CC-1C4809BA9FA1@gmail.com> Message-ID: <85ae9649-4742-7f6f-dd50-5c06c44346de@gmail.com> On Tue, Feb 15, 2022 at 3:36 AM Ioannis Foufoulas wrote: Hi, I have an embedded function: @ffi.def_extern() def toint(input,insize,result): ? ? for i in range(insize): ? ? ? ? x = ffi.string(input[i]) ? ? ? ? result[i] =? float(x) if x else 0 ? ? ? return 1 It seems that this runs around 30-40% percent slower in PyPy3 than in 2. Input is a char** c array and result is a float c array. The difference is not in ffi.string If having `result[i] = 1 if x else 0` it runs same in both versions. Any ideas? _______________________________________________ pypy-dev mailing list pypy-dev at python.org https://mail.python.org/mailman/listinfo/pypy-dev > I converted this to an issue https://foss.heptapod.net/pypy/pypy/-/issues/3682 with some timings and analysis of what might be happening. Matti From johnfouf at gmail.com Thu Feb 17 14:55:52 2022 From: johnfouf at gmail.com (Yannis Foufoulas) Date: Thu, 17 Feb 2022 21:55:52 +0200 Subject: [pypy-dev] Differences between pypy2 and 3 and cffi In-Reply-To: References: <3C88391E-E2E8-4B87-A0CC-1C4809BA9FA1@gmail.com> Message-ID: Nothing changes, the original implementation was int(float(x)) if x else 0 However, I so that the overhead was the same if not using int. ????? ??? BlueMail ??? Android ? ???? 16 ??? 2022, 06:24 ,??? ??? 06:24 ,Dan Stromberg ??????: >I doubt this is an expected result for Pypy3. > >But what if you use: >result[i] = float(x) if x else 0.0 > >...to make result be of homogeneous type? > >On Tue, Feb 15, 2022 at 3:36 AM Ioannis Foufoulas >wrote: > >> Hi, >> I have an embedded function: >> >> @ffi.def_extern() >> def toint(input,insize,result): >> for i in range(insize): >> x = ffi.string(input[i]) >> result[i] = float(x) if x else 0 >> return 1 >> >> It seems that this runs around 30-40% percent slower in PyPy3 than in >2. >> >> Input is a char** c array and result is a float c array. The >difference is >> not in ffi.string >> If having `result[i] = 1 if x else 0` it runs same in both versions. >Any >> ideas? >> >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From cfbolz at gmx.de Thu Feb 17 15:14:50 2022 From: cfbolz at gmx.de (Carl Friedrich Bolz-Tereick) Date: Thu, 17 Feb 2022 21:14:50 +0100 Subject: [pypy-dev] Differences between pypy2 and 3 and cffi In-Reply-To: References: <3C88391E-E2E8-4B87-A0CC-1C4809BA9FA1@gmail.com> Message-ID: We found the problem on the issue that Matti made. It's the float call itself that has gotten slower. It's been fixed and we improved the PyPy2 performance in the process. Tomorrow's nightly should have the change. Cheers, CF On 17.02.22 20:55, Yannis Foufoulas wrote: > Nothing changes, > the original implementation was > int(float(x)) if x else 0 > > However, I so that the overhead was the same if not using int. > > ???? ??? BlueMail ??? Android > ???? 16 ??? 2022 ,??? ??? 06:24 ,Dan Stromberg > ??????: > > > I doubt this is an expected result for Pypy3. > > But what if you use: > result[i] =? float(x) if x else 0.0 > > ...to make result be of homogeneous type? > > On Tue, Feb 15, 2022 at 3:36 AM Ioannis Foufoulas < > johnfouf at gmail.com > wrote: > > Hi, > I have an embedded function: > > @ffi.def_extern() > def toint(input,insize,result): > ? ? for i in range(insize): > ? ? ? ? x = ffi.string(input[i]) > ? ? ? ? result[i] =? float(x) if x else 0 > ? ? ? return 1 > > It seems that this runs around 30-40% percent slower in PyPy3 > than in 2. > > Input is a char** c array and result is a float c array. The > difference is not in ffi.string > If having `result[i] = 1 if x else 0` it runs same in both > versions. Any ideas? > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From matti.picus at gmail.com Sun Feb 20 00:36:50 2022 From: matti.picus at gmail.com (Matti Picus) Date: Sun, 20 Feb 2022 07:36:50 +0200 Subject: [pypy-dev] PyPy 7.3.8 release Message-ID: I have released PyPy v7.3.8. Thanks to all who contributed and tested and for moving PyPy forward. Matti From cfbolz at gmx.de Sun Feb 20 05:43:17 2022 From: cfbolz at gmx.de (Carl Friedrich Bolz-Tereick) Date: Sun, 20 Feb 2022 11:43:17 +0100 Subject: [pypy-dev] PyPy 7.3.8 release In-Reply-To: References: Message-ID: <98157580-eb3b-c764-58da-9fe2c6a43bb4@gmx.de> On 20.02.22 06:36, Matti Picus wrote: > I have released PyPy v7.3.8. Thanks to all who contributed and tested > and for moving PyPy forward. Hi all, Matti and I have discussed that this is the last 3.7 release. We'll stop running 3.7 nightly and can merge directly from default to the py3.8 branch. Just wanted to let everyone know. Cheers, Carl Friedrich From alex.gaynor at gmail.com Sun Feb 20 09:40:50 2022 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Sun, 20 Feb 2022 09:40:50 -0500 Subject: [pypy-dev] PyPy 7.3.8 release In-Reply-To: <98157580-eb3b-c764-58da-9fe2c6a43bb4@gmx.de> References: <98157580-eb3b-c764-58da-9fe2c6a43bb4@gmx.de> Message-ID: FYI there seems to be an issue with the formatting of the blog post: Alex On Sun, Feb 20, 2022 at 5:44 AM Carl Friedrich Bolz-Tereick wrote: > > On 20.02.22 06:36, Matti Picus wrote: > > I have released PyPy v7.3.8. Thanks to all who contributed and tested > > and for moving PyPy forward. > > Hi all, > > Matti and I have discussed that this is the last 3.7 release. We'll stop > running 3.7 nightly and can merge directly from default to the py3.8 > branch. Just wanted to let everyone know. > > Cheers, > > Carl Friedrich > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev -- All that is necessary for evil to succeed is for good people to do nothing. -------------- next part -------------- A non-text attachment was scrubbed... Name: Screen Shot 2022-02-20 at 09.40.19.png Type: image/png Size: 704610 bytes Desc: not available URL: From matti.picus at gmail.com Sun Feb 20 10:44:26 2022 From: matti.picus at gmail.com (Matti Picus) Date: Sun, 20 Feb 2022 17:44:26 +0200 Subject: [pypy-dev] PyPy 7.3.8 release In-Reply-To: References: <98157580-eb3b-c764-58da-9fe2c6a43bb4@gmx.de> Message-ID: <0015de49-8454-ac84-65a8-787810a39c9f@gmail.com> On 20/2/22 16:40, Alex Gaynor wrote: > FYI there seems to be an issue with the formatting of the blog post: > > Alex > Thanks Alex, fixed now. Matti From matti.picus at gmail.com Thu Feb 24 07:35:12 2022 From: matti.picus at gmail.com (matti picus) Date: Thu, 24 Feb 2022 14:35:12 +0200 Subject: [pypy-dev] PyPy3.7-v7.3.8 broke binary compatibility Message-ID: Once again, I broke ABI compatibility with the new release. I re-added a field to PyDateTime_CAPI that should have been part of the original PyPy3.7 release, but was missed, and so should be left out of PyPy3.7 v7.3.x. This causes segfaults when using PyPy3.7-v7.3.8 on wheels built for PyPy3.7-v7.3.7 and older. Please do not build packages with PyPy3.7-v7.3.8 for general release because they will not work on older PyPy3.7. What is doubly annoying is this is exactly why I needed to release v7.3.7 over v7.3.6. PRs welcome to suggest a test for this. A "hg diff" is a start, but it picks up refactoring and valid additions as well, so it needs to be a bit smarter. Maybe something with CFFI? Matti From mgorny at gentoo.org Thu Feb 24 09:00:31 2022 From: mgorny at gentoo.org (=?UTF-8?Q?Micha=C5=82_G=C3=B3rny?=) Date: Thu, 24 Feb 2022 15:00:31 +0100 Subject: [pypy-dev] PyPy3.7-v7.3.8 broke binary compatibility In-Reply-To: References: Message-ID: <598c654c4b30fbabbcabdcc9c80774d5de96165a.camel@gentoo.org> On Thu, 2022-02-24 at 14:35 +0200, matti picus wrote: > Once again, I broke ABI compatibility with the new release. I re-added > a field to PyDateTime_CAPI that should have been part of the original > PyPy3.7 release, but was missed, and so should be left out of PyPy3.7 > v7.3.x. This causes segfaults when using PyPy3.7-v7.3.8 on wheels > built for PyPy3.7-v7.3.7 and older. Please do not build packages with > PyPy3.7-v7.3.8 for general release because they will not work on older > PyPy3.7. > > What is doubly annoying is this is exactly why I needed to release > v7.3.7 over v7.3.6. PRs welcome to suggest a test for this. A "hg > diff" is a start, but it picks up refactoring and valid additions as > well, so it needs to be a bit smarter. Maybe something with CFFI? Perhaps abi-compliance-checker [1] could be of some help. It is quite good in detecting ABI breakage in "normal" C/C++ libraries. [1] https://lvc.github.io/abi-compliance-checker/ -- Best regards, Micha? G?rny From elijahokello90 at gmail.com Sat Feb 26 08:26:25 2022 From: elijahokello90 at gmail.com (Elijah Okello) Date: Sat, 26 Feb 2022 16:26:25 +0300 Subject: [pypy-dev] BUILDING PYPY FROM SOURCE ASSERTION ERROR Message-ID: Hello Pypy Team, I am trying to build pypy from Source and I keep on getting an Assertion error whenever I try to build it with a different gc other than the default. The error I get is this [translation:ERROR] AssertionError Processing block: block at 62[index_362...] is a in (rpython.memory.support:262)AddressDeque.popleft containing the following operations: v3921 = getattr(self_10008, ('oldest_chunk')) v3922 = getattr(v3921, ('items')) result_37 = getitem(v3922, index_362) v3923 = add(index_362, (1)) v3924 = setattr(self_10008, ('index_in_oldest'), v3923) --end? How can I solve this issue? Please help me. Is there something I am not doing right?? Regards Okello Ivan Elijah Software Engineer -------------- next part -------------- An HTML attachment was scrubbed... URL: From cfbolz at gmx.de Sat Feb 26 15:36:01 2022 From: cfbolz at gmx.de (Carl Friedrich Bolz-Tereick) Date: Sat, 26 Feb 2022 21:36:01 +0100 Subject: [pypy-dev] BUILDING PYPY FROM SOURCE ASSERTION ERROR In-Reply-To: References: Message-ID: <5dfeffa3-eefe-aad2-39c0-9f741a3da7a1@gmx.de> Hey! Welcome to the project! This is not enough information for us to answer your question. How exactly are you building PyPy? What's the precise command line? What do you mean by "a different GC"? One of PyPy's GCs? Or one you wrote yourself? Can you point us to your fork with your code changes? Cheers, Carl Friedrich On 26.02.22 14:26, Elijah Okello wrote: > Hello Pypy Team, > > I am trying to build pypy from Source and I keep on getting an Assertion > error whenever I try to build it with a different gc other than the > default. > > The error I get is this > [translation:ERROR] AssertionError > Processing block: > ?block at 62[index_362...] is a 'rpython.flowspace.flowcontext.SpamBlock'> > ?in (rpython.memory.support:262)AddressDeque.popleft > ?containing the following operations: > ?? ? ? v3921 = getattr(self_10008, ('oldest_chunk')) > ?? ? ? v3922 = getattr(v3921, ('items')) > ?? ? ? result_37 = getitem(v3922, index_362) > ?? ? ? v3923 = add(index_362, (1)) > ?? ? ? v3924 = setattr(self_10008, ('index_in_oldest'), v3923) > ?--end? > How can I solve this issue? Please help me. Is there something I am not > doing right?? > > Regards > Okello Ivan Elijah > Software Engineer > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev