From matti.picus at gmail.com Wed Oct 4 08:25:42 2017 From: matti.picus at gmail.com (Matti Picus) Date: Wed, 4 Oct 2017 23:25:42 +1100 Subject: [pypy-dev] PyPy 5.9 release now ready Message-ID: <93dc295d-742e-a6b1-5450-a21d37acf8ae@gmail.com> An HTML attachment was scrubbed... URL: From dcallahan at fb.com Wed Oct 4 08:52:33 2017 From: dcallahan at fb.com (David Callahan) Date: Wed, 4 Oct 2017 12:52:33 +0000 Subject: [pypy-dev] distutils configuration Message-ID: I need to adjust the default configuration used by ?distutils? for compilation. In my default build of branch release-pypy3.5-5.x, the toplevel ?distutils.sysconfig? dispatches as follows: if '__pypy__' in sys.builtin_module_names: from distutils.sysconfig_pypy import * from distutils.sysconfig_pypy import _config_vars # needed by setuptools and sysconfig_pypy.py unconditionally sets some configuration variables such as ?CC? and ?LDSHARED?. CPython reads from a configuration file which can be change with an environment variable and it is this technique we use. This does not yet seem supported in PyPy?s alternate implementation ?distutils.sysconfig_cpython.py? Two questions: Generally, why are there are two implementations of sysconfig and what motivates the default choice? What changes are recommended to allow a configuration file here? Thanks david -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Wed Oct 4 10:45:43 2017 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 4 Oct 2017 16:45:43 +0200 Subject: [pypy-dev] distutils configuration In-Reply-To: References: Message-ID: Hi David As far as I remember, there are two implementations of sysconfig, because CPython one parser Makefile or something like that which we don't even really have, or more precisely didn't really have at the time. Overwriting CC and LDFLAGS can be definitely consider a bug, I would accept the patch that changes it to accept it only conditionally. PS. If you're using PyPy to compile C extensions using distutils (as opposed to say cffi), while it's generally supported, the resulting extensions are quite slow. Best regards, Maciej Fijalkowski On Wed, Oct 4, 2017 at 2:52 PM, David Callahan wrote: > I need to adjust the default configuration used by ?distutils? for > compilation. > > > > In my default build of branch release-pypy3.5-5.x, the toplevel > ?distutils.sysconfig? dispatches as follows: > > > > if '__pypy__' in sys.builtin_module_names: > > from distutils.sysconfig_pypy import * > > from distutils.sysconfig_pypy import _config_vars # needed by setuptools > > > > and sysconfig_pypy.py unconditionally sets some configuration variables such > as ?CC? and ?LDSHARED?. > > > > CPython reads from a configuration file which can be change with an > environment variable and it is this technique we use. This does not yet > seem supported in PyPy?s alternate implementation > ?distutils.sysconfig_cpython.py? > > > > Two questions: > > Generally, why are there are two implementations of sysconfig and what > motivates the default choice? > > What changes are recommended to allow a configuration file here? > > > > Thanks > > david > > > > > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > From matti.picus at gmail.com Thu Oct 5 09:25:02 2017 From: matti.picus at gmail.com (Matti Picus) Date: Fri, 6 Oct 2017 00:25:02 +1100 Subject: [pypy-dev] PyPy 5.9 released Message-ID: An HTML attachment was scrubbed... URL: From dcallahan at fb.com Thu Oct 5 20:30:58 2017 From: dcallahan at fb.com (David Callahan) Date: Fri, 6 Oct 2017 00:30:58 +0000 Subject: [pypy-dev] dist.py:261: UserWarning: Unknown distribution option: 'cffi_modules' Message-ID: <217D45BC-8AEE-4B54-8F43-6ADF127387D8@fb.com> Has anyone experience building the package cryptography-1.9 using PyPy and setup.py? In the build phase I encounter this diagnostic ?/python/pypy.5.8/?/lib-python/3/distutils/dist.py:261: UserWarning: Unknown distribution option: 'cffi_modules' but the stage seems to complete without error. Is this expected or suspicious? Thanks david -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gaynor at gmail.com Fri Oct 6 08:31:32 2017 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Fri, 6 Oct 2017 08:31:32 -0400 Subject: [pypy-dev] dist.py:261: UserWarning: Unknown distribution option: 'cffi_modules' In-Reply-To: <217D45BC-8AEE-4B54-8F43-6ADF127387D8@fb.com> References: <217D45BC-8AEE-4B54-8F43-6ADF127387D8@fb.com> Message-ID: Hi David, We test cryptography against PyPy in our CI, and I install it semi-regularly, so I'd expect it to work :-) Are you able to reproduce this reliably? If yes, can you include full instructions? If not: what versions of pip and setuptools do you have? Alex On Thu, Oct 5, 2017 at 8:30 PM, David Callahan wrote: > Has anyone experience building the package cryptography-1.9 using PyPy and > setup.py? > > > > In the build phase I encounter this diagnostic > > > > ?/python/pypy.5.8/?/lib-python/3/distutils/dist.py:261: UserWarning: > Unknown distribution option: 'cffi_modules' > > > > but the stage seems to complete without error. Is this expected or > suspicious? > > > > Thanks david > > > > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: D1B3 ADC0 E023 8CA6 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcallahan at fb.com Fri Oct 6 10:24:11 2017 From: dcallahan at fb.com (David Callahan) Date: Fri, 6 Oct 2017 14:24:11 +0000 Subject: [pypy-dev] dist.py:261: UserWarning: Unknown distribution option: 'cffi_modules' In-Reply-To: References: <217D45BC-8AEE-4B54-8F43-6ADF127387D8@fb.com> Message-ID: <2A11FCEC-E561-4F6A-A46B-49FFE2A68D82@fb.com> Thanks Alex, We are using setuptools 34.2.0, we don?t use pip. In our environment, each package it built from source and installed into a separate directory and a long PYTHONPATH constructed. This works fine for CPython but apparently has some snag with PyPy. From: Alex Gaynor Date: Friday, October 6, 2017 at 5:32 AM To: David Callahan Cc: PyPy Dev Subject: Re: [pypy-dev] dist.py:261: UserWarning: Unknown distribution option: 'cffi_modules' Hi David, We test cryptography against PyPy in our CI, and I install it semi-regularly, so I'd expect it to work :-) Are you able to reproduce this reliably? If yes, can you include full instructions? If not: what versions of pip and setuptools do you have? Alex On Thu, Oct 5, 2017 at 8:30 PM, David Callahan > wrote: Has anyone experience building the package cryptography-1.9 using PyPy and setup.py? In the build phase I encounter this diagnostic ?/python/pypy.5.8/?/lib-python/3/distutils/dist.py:261: UserWarning: Unknown distribution option: 'cffi_modules' but the stage seems to complete without error. Is this expected or suspicious? Thanks david _______________________________________________ pypy-dev mailing list pypy-dev at python.org https://mail.python.org/mailman/listinfo/pypy-dev -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: D1B3 ADC0 E023 8CA6 -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Fri Oct 6 10:58:37 2017 From: fijall at gmail.com (Maciej Fijalkowski) Date: Fri, 6 Oct 2017 16:58:37 +0200 Subject: [pypy-dev] dist.py:261: UserWarning: Unknown distribution option: 'cffi_modules' In-Reply-To: <2A11FCEC-E561-4F6A-A46B-49FFE2A68D82@fb.com> References: <217D45BC-8AEE-4B54-8F43-6ADF127387D8@fb.com> <2A11FCEC-E561-4F6A-A46B-49FFE2A68D82@fb.com> Message-ID: Hi David Maybe pypy 3 does not come with correct distutils/setuptools gimmick to convince everyone cffi is installed? The way you described should generally work with PyPy. I'll have a look tomorrow at the sprint. Cheers, fijal On Fri, Oct 6, 2017 at 4:24 PM, David Callahan wrote: > Thanks Alex, > > > > We are using setuptools 34.2.0, we don?t use pip. In our environment, each > package it built from source and installed into a separate directory and a > long PYTHONPATH constructed. This works fine for CPython but apparently has > some snag with PyPy. > > > > From: Alex Gaynor > Date: Friday, October 6, 2017 at 5:32 AM > To: David Callahan > Cc: PyPy Dev > Subject: Re: [pypy-dev] dist.py:261: UserWarning: Unknown distribution > option: 'cffi_modules' > > > > Hi David, > > > > We test cryptography against PyPy in our CI, and I install it > semi-regularly, so I'd expect it to work :-) > > > > Are you able to reproduce this reliably? If yes, can you include full > instructions? > > > > If not: what versions of pip and setuptools do you have? > > > > Alex > > > > On Thu, Oct 5, 2017 at 8:30 PM, David Callahan wrote: > > Has anyone experience building the package cryptography-1.9 using PyPy and > setup.py? > > > > In the build phase I encounter this diagnostic > > > > ?/python/pypy.5.8/?/lib-python/3/distutils/dist.py:261: UserWarning: Unknown > distribution option: 'cffi_modules' > > > > but the stage seems to complete without error. Is this expected or > suspicious? > > > > Thanks david > > > > > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > > > > > > -- > > "I disapprove of what you say, but I will defend to the death your right to > say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > > GPG Key fingerprint: D1B3 ADC0 E023 8CA6 > > > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > From ronan.lamy at gmail.com Sat Oct 7 13:28:19 2017 From: ronan.lamy at gmail.com (Ronan Lamy) Date: Sat, 7 Oct 2017 19:28:19 +0200 Subject: [pypy-dev] dist.py:261: UserWarning: Unknown distribution option: 'cffi_modules' In-Reply-To: <2A11FCEC-E561-4F6A-A46B-49FFE2A68D82@fb.com> References: <217D45BC-8AEE-4B54-8F43-6ADF127387D8@fb.com> <2A11FCEC-E561-4F6A-A46B-49FFE2A68D82@fb.com> Message-ID: I couldn't find out how to reproduce your issue, cryptography 1.9 installs fine for me. I'm guessing that there is something specific to your setup that is causing the issue. Ronan 2017-10-06 16:24 GMT+02:00 David Callahan : > Thanks Alex, > > > > We are using setuptools 34.2.0, we don?t use pip. In our environment, each > package it built from source and installed into a separate directory and a > long PYTHONPATH constructed. This works fine for CPython but apparently > has some snag with PyPy. > > > > *From: *Alex Gaynor > *Date: *Friday, October 6, 2017 at 5:32 AM > *To: *David Callahan > *Cc: *PyPy Dev > *Subject: *Re: [pypy-dev] dist.py:261: UserWarning: Unknown distribution > option: 'cffi_modules' > > > > Hi David, > > > > We test cryptography against PyPy in our CI, and I install it > semi-regularly, so I'd expect it to work :-) > > > > Are you able to reproduce this reliably? If yes, can you include full > instructions? > > > > If not: what versions of pip and setuptools do you have? > > > > Alex > > > > On Thu, Oct 5, 2017 at 8:30 PM, David Callahan wrote: > > Has anyone experience building the package cryptography-1.9 using PyPy and > setup.py? > > > > In the build phase I encounter this diagnostic > > > > ?/python/pypy.5.8/?/lib-python/3/distutils/dist.py:261: UserWarning: > Unknown distribution option: 'cffi_modules' > > > > but the stage seems to complete without error. Is this expected or > suspicious? > > > > Thanks david > > > > > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > > > > > > > -- > > "I disapprove of what you say, but I will defend to the death your right > to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > > GPG Key fingerprint: D1B3 ADC0 E023 8CA6 > > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From srhaque at theiet.org Wed Oct 11 05:29:38 2017 From: srhaque at theiet.org (Shaheed Haque) Date: Wed, 11 Oct 2017 10:29:38 +0100 Subject: [pypy-dev] Automated binding generation (and maintenance) In-Reply-To: References:

Message-ID: Hi Wim, After reviewing your comments, I propose to check out rootcling. I initially had some trouble using pip3 to install the newer code, but that seems to have been resolved as of yesterday's 0.2.3 build. I did notice one message during the install which seems to be benign, so I mention it here merely in passing: Running command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-spz01kkp/cppyy-backend/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpe2h6yls0pip-wheel- --python-tag cp36 running bdist_wheel running build running build_ext error: [Errno 2] No such file or directory: 'cling-config': 'cling-config' error Failed building wheel for cppyy-backend Running setup.py clean for cppyy-backend I'll no doubt be back with questions :-). Thanks for all the good work, Shaheed On 23 September 2017 at 06:24, Shaheed Haque wrote: > Wim, > > Thanks for the detailed and thoughtful reply. I will digest and > respond when I am properly back in circulation. > > On 15 September 2017 at 07:43, wrote: >> Shaheed, >> >>> Ah, I had not realised rootcling existed. I've seen that I can invoke >>> it using Python version-specific paths...is this the correct way to >>> invoke it: >>> >>> ROOTCLING=/usr/local/lib/python3.6/dist-packages/cppyy_backend >>> LD_LIBRARY_PATH=$ROOTCLING/lib $ROOTCLING/bin/rootcling -h >> >> >> Yes, and here's a description of the LinkDef.h format: >> >> >> https://root.cern.ch/root/html/guides/users-guide/AddingaClass.html#the-linkdef.h-file >> >>> or is there a recommended wrapper? >> >> >> No, but I'm going to add one for pip, same as I did for genreflex. I've >> been fleshing out the backend generation, taken over from Anto: >> >> https://bitbucket.org/wlav/cppyy-backend >> >> where all that can live. I'm told that I'll need rootcling anyway for >> use of modules (see below). >> >>> I actually get some warnings and then the error: >> >> >> Add this set of exclusions to the selection.xml: >> >> >> >> >> >> >> >> Of course, the larger problem of pulling in these standard libs over and >> over again is that it is a waste of cpu and memory, so I do want to see >> the file_name attribute fixed. As it stands, I'd simply exclude: >> >> >> >> >> especially since they are already available by default. Note that those two >> rules cover the ones needed for new_allocator and Alloc_hider. >> >> However, there is a more efficient approach that is right around the corner >> (and has been right about the corner for a long time, so don't hold me to >> that). Next release now seems likely though. >> >> The long term goal has always been to use modules: >> >> http://clang.llvm.org/docs/Modules.html >> >> but the original drivers (Apple, Google, and the C++ standards committee) >> have been going back and forth on it. Now, things are finally falling into >> place. Here's Google: >> >> https://www.youtube.com/watch?v=dHFNpBfemDI >> >> And here's ROOT: >> >> >> https://indico.cern.ch/event/643728/contributions/2612822/attachments/1494074/2323893/ROOTs_C_modules_status_report.pdf >> >> The big deal is that C++ developers have an incentive to deploy modules, so >> being able to patch into that should be a huge time saver (and where they >> don't, rootcling will soon be able to create modules from headers). Note >> that modules don't come for free: it will require some ambiguity resolution, >> but that is typically a Good Thing (code-quality wise). >> >> Modules allow deserialization of only the piece of the AST that is actually >> being requested, saving memory. This as opposed to header files (whether or >> not precompiled) which pull in everything before them. See the status report >> above for the improvements in memory usage. >> >> And with modules, of course, selection becomes unnecessary (markup for >> automatic streamers may still be useful, but that is not relevant for >> bindings generation). >> >>> I did wonder if I was missing some "-isystem" includes, and tried >>> adding them but the --debug output from genreflex seemed to suggest >>> they were being ignored. >> >> >> Some flags are ignored as no-one was using them (so far). Some others >> are definitely obsolete by now. >> >>> What is interesting, and might possibly throw light on the selection >>> filter issue, is that the file name for the classes in >>> kjsinterpreter.h itself is always the empty string ''. Classes that >>> come from included files return non-empty strings such as >>> 'kjsobject.h' for 'KJSObject'. >> >> >> That's after the fact (i.e. what is stored); I don't see the rule being >> respected/used at all. >> >>> BTW, the reason for doing this is that lots of KDE code has multiple >>> classes and even namespaces in a single header file. Now, for >>> discoverability of the loaded objects, I find the incremental "pop >>> into cppyy,gbl on demand" somewhat limiting and I wanted to play about >>> with that. I could also workaround the filter issue if I precomputed >>> the needed names in a precursor pass. >> >> >> The issue here is the memory cost of loading things that won't get used >> in the end. This is why a functional dir() (which needs nothing but >> strings, after all), in conjunction with lazy loading/creation when a >> real access happens work well. LLVM is fully lookup based, btw. There >> is a custom layer on top of Cling to make enumeration possible. >> >>> Finally, and most importantly given the fidelity with which cppyy >>> renders the C++ code, I'm think about how Pythonisation customisation >>> might be handled: e.g. a Python wrapper layer to allow a >>> pointer-plus-size to render as a Python list/tuple, or generate a dict >>> mapping fora QSet, and so on. (I'm dimly aware of the >>> boost-recognition logic you have alluded to, this is specifically more >>> about Qt-specific patterns and ad-hoc scenarios). >> >> >> In 2015, a GSoC student fleshed this out. I never put it into PyPy b/c of >> a lack of test coverage, but did put in in PyROOT. Here's an example of >> the "pointer-plus-size" pythonization (from ROOT.py): >> >> # python side pythonizations (should live in their own file, if we get >> many) >> def set_size(self, buf): >> buf.SetSize(self.GetN()) >> return buf >> >> # TODO: add pythonization API to pypy-c >> if not PYPY_CPPYY_COMPATIBILITY_FIXME: >> cppyy.add_pythonization( >> cppyy.compose_method("^TGraph(2D)?$|^TGraph.*Errors$", >> "GetE?[XYZ]$", set_size)) >> >> The functions selected by the regexps return naked pointers, but the object >> can be queried for the size (all have a consistent GetN() function). So the >> method composer patches up the return value, making it a sized array, >> instead of an "open-ended" one. >> >> I'm sitting on some patches as I wanted to tweak his APIs a bit. There >> was some ordering that I felt didn't compose well, but that is minor. >> >> Similarly, there's code to apply ownership rules, mapping exceptions, >> the new C++11 smartptrs, controlling auto-casting, handling the GIL, making >> properties, and adding overloads. All driven by regexp matching of patterns. >> See here: >> >> >> https://bitbucket.org/wlav/cppyy/src/4d14ba325e494f13cc11f3f11cbb87b44048b256/python/cppyy/_pythonization.py?at=master >> >> (plus further support inside the bindings layer itself). >> >> Of course, one can hook up completely custom functions, and he made it so >> that that is per C++ namespace, so nicely self-contained. >> >> Again, this is currently only partly available, as I need to write a lot >> more tests for PyPy (which are bound to unearth some problems along the >> way). And then there is documentation to be written ... >> >>> P.S. Please note that after today, I'll likely not have much Internet >>> access for a couple of weeks, so any responses may be limited. >> >> >> I'll make sure I have at least all my local changes pushed by then. :) >> >> >> Best regards, >> Wim >> -- >> WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From srhaque at theiet.org Wed Oct 11 06:51:37 2017 From: srhaque at theiet.org (Shaheed Haque) Date: Wed, 11 Oct 2017 11:51:37 +0100 Subject: [pypy-dev] A small bug in cppyy's setup.py Message-ID: Hi, In cppyy 0.3.1, there is a bug in setup.py for systems where pypy is not detected. I believe an incomplete patch is: ========== $ diff -U5 setup.py.original setup.py --- setup.py.original 2017-10-11 11:36:54.632924969 +0100 +++ setup.py 2017-10-11 11:38:08.732492901 +0100 @@ -18,10 +18,11 @@ else: requirements = ['cppyy-backend'] add_pkg = ['cppyy'] except ImportError: requirements = ['CPyCppyy'] + add_pkg = [] setup( name='cppyy', version='0.3.1', ========== Without this assignment to define add_pkg, the reference to "add_pkg" fails in the setup() call. I describe the patch as incomplete because I'm not sure what the array should contain. I do see that the relevant direction contains both a "cppyy" and a "cppyy_compat" directory. Any clarification welcomed. Thanks, Shaheed P.S. If a bug report, or a PR is preferred, please advise. From srhaque at theiet.org Wed Oct 11 07:03:56 2017 From: srhaque at theiet.org (Shaheed Haque) Date: Wed, 11 Oct 2017 12:03:56 +0100 Subject: [pypy-dev] Automated binding generation (and maintenance) In-Reply-To: References:

Message-ID: Oh wait, I think I see that cling-config is installed by the cppyy package. (Seems a tad confusing, ho-hum). On 11 October 2017 at 10:29, Shaheed Haque wrote: > Hi Wim, > > After reviewing your comments, I propose to check out rootcling. I > initially had some trouble using pip3 to install the newer code, but > that seems to have been resolved as of yesterday's 0.2.3 build. I did > notice one message during the install which seems to be benign, so I > mention it here merely in passing: > > Running command /usr/bin/python3 -u -c "import setuptools, > tokenize;__file__='/tmp/pip-build-spz01kkp/cppyy-backend/setup.py';f=getattr(tokenize, > 'open', open)(__file__);code=f.read().replace('\r\n', > '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d > /tmp/tmpe2h6yls0pip-wheel- --python-tag cp36 > running bdist_wheel > running build > running build_ext > error: [Errno 2] No such file or directory: 'cling-config': 'cling-config' > error > Failed building wheel for cppyy-backend > Running setup.py clean for cppyy-backend > > I'll no doubt be back with questions :-). > > Thanks for all the good work, Shaheed > > > > On 23 September 2017 at 06:24, Shaheed Haque wrote: >> Wim, >> >> Thanks for the detailed and thoughtful reply. I will digest and >> respond when I am properly back in circulation. >> >> On 15 September 2017 at 07:43, wrote: >>> Shaheed, >>> >>>> Ah, I had not realised rootcling existed. I've seen that I can invoke >>>> it using Python version-specific paths...is this the correct way to >>>> invoke it: >>>> >>>> ROOTCLING=/usr/local/lib/python3.6/dist-packages/cppyy_backend >>>> LD_LIBRARY_PATH=$ROOTCLING/lib $ROOTCLING/bin/rootcling -h >>> >>> >>> Yes, and here's a description of the LinkDef.h format: >>> >>> >>> https://root.cern.ch/root/html/guides/users-guide/AddingaClass.html#the-linkdef.h-file >>> >>>> or is there a recommended wrapper? >>> >>> >>> No, but I'm going to add one for pip, same as I did for genreflex. I've >>> been fleshing out the backend generation, taken over from Anto: >>> >>> https://bitbucket.org/wlav/cppyy-backend >>> >>> where all that can live. I'm told that I'll need rootcling anyway for >>> use of modules (see below). >>> >>>> I actually get some warnings and then the error: >>> >>> >>> Add this set of exclusions to the selection.xml: >>> >>> >>> >>> >>> >>> >>> >>> Of course, the larger problem of pulling in these standard libs over and >>> over again is that it is a waste of cpu and memory, so I do want to see >>> the file_name attribute fixed. As it stands, I'd simply exclude: >>> >>> >>> >>> >>> especially since they are already available by default. Note that those two >>> rules cover the ones needed for new_allocator and Alloc_hider. >>> >>> However, there is a more efficient approach that is right around the corner >>> (and has been right about the corner for a long time, so don't hold me to >>> that). Next release now seems likely though. >>> >>> The long term goal has always been to use modules: >>> >>> http://clang.llvm.org/docs/Modules.html >>> >>> but the original drivers (Apple, Google, and the C++ standards committee) >>> have been going back and forth on it. Now, things are finally falling into >>> place. Here's Google: >>> >>> https://www.youtube.com/watch?v=dHFNpBfemDI >>> >>> And here's ROOT: >>> >>> >>> https://indico.cern.ch/event/643728/contributions/2612822/attachments/1494074/2323893/ROOTs_C_modules_status_report.pdf >>> >>> The big deal is that C++ developers have an incentive to deploy modules, so >>> being able to patch into that should be a huge time saver (and where they >>> don't, rootcling will soon be able to create modules from headers). Note >>> that modules don't come for free: it will require some ambiguity resolution, >>> but that is typically a Good Thing (code-quality wise). >>> >>> Modules allow deserialization of only the piece of the AST that is actually >>> being requested, saving memory. This as opposed to header files (whether or >>> not precompiled) which pull in everything before them. See the status report >>> above for the improvements in memory usage. >>> >>> And with modules, of course, selection becomes unnecessary (markup for >>> automatic streamers may still be useful, but that is not relevant for >>> bindings generation). >>> >>>> I did wonder if I was missing some "-isystem" includes, and tried >>>> adding them but the --debug output from genreflex seemed to suggest >>>> they were being ignored. >>> >>> >>> Some flags are ignored as no-one was using them (so far). Some others >>> are definitely obsolete by now. >>> >>>> What is interesting, and might possibly throw light on the selection >>>> filter issue, is that the file name for the classes in >>>> kjsinterpreter.h itself is always the empty string ''. Classes that >>>> come from included files return non-empty strings such as >>>> 'kjsobject.h' for 'KJSObject'. >>> >>> >>> That's after the fact (i.e. what is stored); I don't see the rule being >>> respected/used at all. >>> >>>> BTW, the reason for doing this is that lots of KDE code has multiple >>>> classes and even namespaces in a single header file. Now, for >>>> discoverability of the loaded objects, I find the incremental "pop >>>> into cppyy,gbl on demand" somewhat limiting and I wanted to play about >>>> with that. I could also workaround the filter issue if I precomputed >>>> the needed names in a precursor pass. >>> >>> >>> The issue here is the memory cost of loading things that won't get used >>> in the end. This is why a functional dir() (which needs nothing but >>> strings, after all), in conjunction with lazy loading/creation when a >>> real access happens work well. LLVM is fully lookup based, btw. There >>> is a custom layer on top of Cling to make enumeration possible. >>> >>>> Finally, and most importantly given the fidelity with which cppyy >>>> renders the C++ code, I'm think about how Pythonisation customisation >>>> might be handled: e.g. a Python wrapper layer to allow a >>>> pointer-plus-size to render as a Python list/tuple, or generate a dict >>>> mapping fora QSet, and so on. (I'm dimly aware of the >>>> boost-recognition logic you have alluded to, this is specifically more >>>> about Qt-specific patterns and ad-hoc scenarios). >>> >>> >>> In 2015, a GSoC student fleshed this out. I never put it into PyPy b/c of >>> a lack of test coverage, but did put in in PyROOT. Here's an example of >>> the "pointer-plus-size" pythonization (from ROOT.py): >>> >>> # python side pythonizations (should live in their own file, if we get >>> many) >>> def set_size(self, buf): >>> buf.SetSize(self.GetN()) >>> return buf >>> >>> # TODO: add pythonization API to pypy-c >>> if not PYPY_CPPYY_COMPATIBILITY_FIXME: >>> cppyy.add_pythonization( >>> cppyy.compose_method("^TGraph(2D)?$|^TGraph.*Errors$", >>> "GetE?[XYZ]$", set_size)) >>> >>> The functions selected by the regexps return naked pointers, but the object >>> can be queried for the size (all have a consistent GetN() function). So the >>> method composer patches up the return value, making it a sized array, >>> instead of an "open-ended" one. >>> >>> I'm sitting on some patches as I wanted to tweak his APIs a bit. There >>> was some ordering that I felt didn't compose well, but that is minor. >>> >>> Similarly, there's code to apply ownership rules, mapping exceptions, >>> the new C++11 smartptrs, controlling auto-casting, handling the GIL, making >>> properties, and adding overloads. All driven by regexp matching of patterns. >>> See here: >>> >>> >>> https://bitbucket.org/wlav/cppyy/src/4d14ba325e494f13cc11f3f11cbb87b44048b256/python/cppyy/_pythonization.py?at=master >>> >>> (plus further support inside the bindings layer itself). >>> >>> Of course, one can hook up completely custom functions, and he made it so >>> that that is per C++ namespace, so nicely self-contained. >>> >>> Again, this is currently only partly available, as I need to write a lot >>> more tests for PyPy (which are bound to unearth some problems along the >>> way). And then there is documentation to be written ... >>> >>>> P.S. Please note that after today, I'll likely not have much Internet >>>> access for a couple of weeks, so any responses may be limited. >>> >>> >>> I'll make sure I have at least all my local changes pushed by then. :) >>> >>> >>> Best regards, >>> Wim >>> -- >>> WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From wlavrijsen at lbl.gov Wed Oct 11 12:57:29 2017 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Wed, 11 Oct 2017 09:57:29 -0700 (PDT) Subject: [pypy-dev] Automated binding generation (and maintenance) In-Reply-To: References:

Message-ID: Shaheed, > Oh wait, I think I see that cling-config is installed by the cppyy > package. (Seems a tad confusing, ho-hum). no, it's in cppyy-cling, which was freshly pulled in when starting from cppyy, as all has been updated to take that new split into account. (I'm not sure how to force such updates otherwise.) As for the reasons for splitting and the overall package structure, rather than posting it here, I added it to the docs: http://cppyy.readthedocs.io/en/latest/installation.html#package-structure Basically, I want to avoid having to republish/reinstall all of Cling/LLVM whenever I make a small change in the wrapper, as the former changes only very infrequently (and takes a long time to build, as opposed to the wrapper which is just a single C++ file). I hope this is the last change I need to make to the package structure. :) Once 1.0 is out, I'll look into whether something like conda is better than pip (given the amount of C++ code). For now I think pip will do. Best regards, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From wlavrijsen at lbl.gov Wed Oct 11 13:17:08 2017 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Wed, 11 Oct 2017 10:17:08 -0700 (PDT) Subject: [pypy-dev] A small bug in cppyy's setup.py In-Reply-To: References: Message-ID: Shaheed, yes, or rather: + add_pkg = ['cppyy'] Anyway, is in, thanks! I was off on a tangent now that PyPy 5.9 is released. For that matter, the cppyy-backend package has been updated, too. PyPy 5.7-5.9 will have the 0.2.x branch, later is now 0.3.x. Change is minor, but needed for templated constructors (as is the case for the boost example on the intro page of the documentation). I'm currently trying to fix up all I want for PyPy, to get it in early and be on-time for the next release. :) I was hoping to get more into 5.9, but sat on it for too long. It's all minor stuff: naming etc. to get consistent results between PyPy and CPython in things like help() and exceptions, but these small differences can get very annoying very quickly. After that, first priority will be to setup nightly builds, to catch things like the above. I hardly ever run p3 myself, for example. > P.S. If a bug report, or a PR is preferred, please advise. Well, this is a CPython/cppyy specific problem, so it's highly unlikely to be of interest on pypy-dev. Therefore, I'd recommend using: https://bitbucket.org/wlav/cppyy/issues Or e-mail me directly. Thanks! Best regards, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From tkadm30 at yandex.com Thu Oct 12 16:44:59 2017 From: tkadm30 at yandex.com (Etienne Robillard) Date: Thu, 12 Oct 2017 16:44:59 -0400 Subject: [pypy-dev] AttributeError: 'built in-code' object has no attribute 'co_code' Message-ID: <14765118-48a8-95d6-7f38-d5e2dd78d21f@yandex.com> Hi, I'm trying to port django-hotsauce to PyPy. I'm getting an exception when I use Cython compiled under CPython: DEBUG:Traceback (most recent call last): File "lib/notmm/controllers/wsgi.pyx", line 206, in notmm.controllers.wsgi.BaseController.get_response (lib/notmm/controllers/wsgi.c:4174) File "/home/erob/src/django-hotsauce-pypy/lib/notmm/utils/django_compat.py", line 230, in resolve sub_match = pattern.resolve(new_path) File "/home/erob/src/django-hotsauce-pypy/lib/notmm/utils/django_compat.py", line 155, in resolve return (self.callback, args, kwargs) File "/home/erob/src/django-hotsauce-pypy/lib/notmm/utils/django_compat.py", line 166, in _get_callback raise ViewDoesNotExist, "Tried %s in module %s. Error was: %s" % (func_name, mod_name, str(e)) ViewDoesNotExist: Tried index in module mainapp.research.views. Error was: 'builtin-code' object has no attribute 'co_code' Traceback (most recent call last): File "/usr/lib/pypy/lib-python/2.7/wsgiref/handlers.py", line 85, in run self.result = application(self.environ, self.start_response) File "/usr/lib/pypy/lib-python/2.7/wsgiref/validate.py", line 176, in lint_app iterator = application(environ, start_response_wrapper) File "/home/erob/src/django-hotsauce-oauthclient/lib/wsgi_oauth2/middleware.py", line 292, in __call__ return self.application(environ, start_response) File "lib/notmm/controllers/wsgi.pyx", line 136, in notmm.controllers.wsgi.BaseController.__call__ (lib/notmm/controllers/wsgi.c:2842) File "lib/notmm/controllers/wsgi.pyx", line 151, in notmm.controllers.wsgi.BaseController.application (lib/notmm/controllers/wsgi.c:3177) AttributeError: 'builtin-code' object has no attribute 'co_code' 127.0.0.1 - - [12/Oct/2017 16:37:03] "GET / HTTP/1.1" 500 59 In wsgi.pyx: with sessionmanager(environ): request.environ.update(environ) #assert request.environ['PATH_INFO'] == request.path_url response = self.get_response(request=request) return response(environ, start_response) Any ideas why using a @contextmanager here yield this exception with PyPy 5.6.0 and Cython 0.25.2 ? Thank you in advance, Etienne From yury at shurup.com Fri Oct 13 01:16:15 2017 From: yury at shurup.com (Yury V. Zaytsev) Date: Fri, 13 Oct 2017 07:16:15 +0200 (CEST) Subject: [pypy-dev] AttributeError: 'built in-code' object has no attribute 'co_code' In-Reply-To: <14765118-48a8-95d6-7f38-d5e2dd78d21f@yandex.com> References: <14765118-48a8-95d6-7f38-d5e2dd78d21f@yandex.com> Message-ID: Hi, On Thu, 12 Oct 2017, Etienne Robillard wrote: > Any ideas why using a @contextmanager here yield this exception with > PyPy 5.6.0 and Cython 0.25.2 ? You are using very outdated versions of both PyPy (latest = 5.9.0) and Cython (latest = 0.27.1), could you please try again with the latest versions? There have been a lot of compatibility work going on both sides, and it is not unlikely that the issue has been fixed in the meantime. -- Sincerely yours, Yury V. Zaytsev From tkadm30 at yandex.com Fri Oct 13 07:22:40 2017 From: tkadm30 at yandex.com (Etienne Robillard) Date: Fri, 13 Oct 2017 07:22:40 -0400 Subject: [pypy-dev] AttributeError: 'built in-code' object has no attribute 'co_code' In-Reply-To: References: <14765118-48a8-95d6-7f38-d5e2dd78d21f@yandex.com> Message-ID: <7d322ccb-7c9a-97b9-a634-2b678bb06824@yandex.com> Hi Yury, I upgraded pypy to 5.9.0 and cython to 0.27.1. Compilation of libschevo fails under PyPy: erob at marina:~/src/django-hotsauce-pypy/extras/libschevo$ sudo pypy setup.py develop --prefix=/usr/local/pypy running develop running egg_info writing lib/libschevo.egg-info/PKG-INFO writing dependency_links to lib/libschevo.egg-info/dependency_links.txt writing entry points to lib/libschevo.egg-info/entry_points.txt writing requirements to lib/libschevo.egg-info/requires.txt writing top-level names to lib/libschevo.egg-info/top_level.txt reading manifest file 'lib/libschevo.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching 'README' anywhere in distribution warning: no files found matching '*' under directory 'schevo/templates' writing manifest file 'lib/libschevo.egg-info/SOURCES.txt' running build_ext building 'schevo.store._s_persistent' extension cc -pthread -DNDEBUG -O2 -fPIC -I/usr/local/pypy/pypy2-v5.9.0-linux32/include -c lib/schevo/store/_s_persistent.c -o build/temp.linux-i686-2.7/lib/schevo/store/_s_persistent.o lib/schevo/store/_s_persistent.c:8:27: fatal error: weakrefobject.h: No such file or directory #include "weakrefobject.h" ^ compilation terminated. error: command 'cc' failed with exit status 1 Will need to investigate further why weakrefobject.h is missing from PyPy or find a workaround. Best Regards, Etienne On 13/10/17 01:16 AM, Yury V. Zaytsev wrote: > > You are using very outdated versions of both PyPy (latest = 5.9.0) and > Cython (latest = 0.27.1), could you please try again with the latest > versions? There have been a lot of compatibility work going on both > sides, and it is not unlikely that the issue has been fixed in the > meantime. > From tkadm30 at yandex.com Fri Oct 13 08:13:57 2017 From: tkadm30 at yandex.com (Etienne Robillard) Date: Fri, 13 Oct 2017 08:13:57 -0400 Subject: [pypy-dev] AttributeError: 'built in-code' object has no attribute 'co_code' In-Reply-To: <7d322ccb-7c9a-97b9-a634-2b678bb06824@yandex.com> References: <14765118-48a8-95d6-7f38-d5e2dd78d21f@yandex.com> <7d322ccb-7c9a-97b9-a634-2b678bb06824@yandex.com> Message-ID: <25044364-0650-86f7-daca-2ab69c8c0958@yandex.com> Hi, Can someone please explain what this TypeError message means? Best Regards, Etienne Traceback (most recent call last): ? File "/usr/local/pypy/pypy2-v5.9.0-linux32/lib-python/2.7/wsgiref/handlers.py", line 85, in run ??? self.result = application(self.environ, self.start_response) ? File "/usr/local/pypy/pypy2-v5.9.0-linux32/lib-python/2.7/wsgiref/validate.py", line 176, in lint_app ??? iterator = application(environ, start_response_wrapper) ? File "/home/erob/src/django-hotsauce-oauthclient/lib/wsgi_oauth2/middleware.py", line 292, in __call__ ??? return self.application(environ, start_response) ? File "lib/notmm/controllers/wsgi.pyx", line 136, in notmm.controllers.wsgi.BaseController.__call__ TypeError: __weakref__ slot disallowed: we already got one Le 2017-10-13 ? 07:22, Etienne Robillard a ?crit?: > Hi Yury, > > I upgraded pypy to 5.9.0 and cython to 0.27.1. > > Compilation of libschevo fails under PyPy: > > erob at marina:~/src/django-hotsauce-pypy/extras/libschevo$ sudo pypy > setup.py develop --prefix=/usr/local/pypy > running develop > running egg_info > writing lib/libschevo.egg-info/PKG-INFO > writing dependency_links to lib/libschevo.egg-info/dependency_links.txt > writing entry points to lib/libschevo.egg-info/entry_points.txt > writing requirements to lib/libschevo.egg-info/requires.txt > writing top-level names to lib/libschevo.egg-info/top_level.txt > reading manifest file 'lib/libschevo.egg-info/SOURCES.txt' > reading manifest template 'MANIFEST.in' > warning: no files found matching 'README' anywhere in distribution > warning: no files found matching '*' under directory 'schevo/templates' > writing manifest file 'lib/libschevo.egg-info/SOURCES.txt' > running build_ext > building 'schevo.store._s_persistent' extension > cc -pthread -DNDEBUG -O2 -fPIC > -I/usr/local/pypy/pypy2-v5.9.0-linux32/include -c > lib/schevo/store/_s_persistent.c -o > build/temp.linux-i686-2.7/lib/schevo/store/_s_persistent.o > lib/schevo/store/_s_persistent.c:8:27: fatal error: weakrefobject.h: > No such file or directory > ?#include "weakrefobject.h" > ?????????????????????????? ^ > compilation terminated. > error: command 'cc' failed with exit status 1 > > Will need to investigate further why weakrefobject.h is missing from > PyPy or find a workaround. > > Best Regards, > > Etienne > > > > On 13/10/17 01:16 AM, Yury V. Zaytsev wrote: >> >> You are using very outdated versions of both PyPy (latest = 5.9.0) >> and Cython (latest = 0.27.1), could you please try again with the >> latest versions? There have been a lot of compatibility work going on >> both sides, and it is not unlikely that the issue has been fixed in >> the meantime. >> > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev -- Etienne Robillard tkadm30 at yandex.com http://www.isotopesoftware.ca/ From tkadm30 at yandex.com Fri Oct 13 11:18:01 2017 From: tkadm30 at yandex.com (Etienne Robillard) Date: Fri, 13 Oct 2017 11:18:01 -0400 Subject: [pypy-dev] AttributeError: 'built in-code' object has no attribute 'co_code' In-Reply-To: <25044364-0650-86f7-daca-2ab69c8c0958@yandex.com> References: <14765118-48a8-95d6-7f38-d5e2dd78d21f@yandex.com> <7d322ccb-7c9a-97b9-a634-2b678bb06824@yandex.com> <25044364-0650-86f7-daca-2ab69c8c0958@yandex.com> Message-ID: I opened a ticket for this issue: https://bitbucket.org/pypy/pypy/issues/2681/typeerror-__weakref__-slot-disallowed-we Etienne Le 2017-10-13 ? 08:13, Etienne Robillard a ?crit?: > Traceback (most recent call last): > ? File > "/usr/local/pypy/pypy2-v5.9.0-linux32/lib-python/2.7/wsgiref/handlers.py", > line 85, in run > ??? self.result = application(self.environ, self.start_response) > ? File > "/usr/local/pypy/pypy2-v5.9.0-linux32/lib-python/2.7/wsgiref/validate.py", > line 176, in lint_app > ??? iterator = application(environ, start_response_wrapper) > ? File > "/home/erob/src/django-hotsauce-oauthclient/lib/wsgi_oauth2/middleware.py", > line 292, in __call__ > ??? return self.application(environ, start_response) > ? File "lib/notmm/controllers/wsgi.pyx", line 136, in > notmm.controllers.wsgi.BaseController.__call__ > TypeError: __weakref__ slot disallowed: we already got one -- Etienne Robillard tkadm30 at yandex.com http://www.isotopesoftware.ca/ From tkadm30 at yandex.com Fri Oct 13 12:34:23 2017 From: tkadm30 at yandex.com (Etienne Robillard) Date: Fri, 13 Oct 2017 12:34:23 -0400 Subject: [pypy-dev] AttributeError: 'built in-code' object has no attribute 'co_code' In-Reply-To: References: <14765118-48a8-95d6-7f38-d5e2dd78d21f@yandex.com> <7d322ccb-7c9a-97b9-a634-2b678bb06824@yandex.com> <25044364-0650-86f7-daca-2ab69c8c0958@yandex.com> Message-ID: Hi Jason, How would you tackle this issue? I? may not have defined any __slots__ in my code. Thank? you in advance, Etienne Le 2017-10-13 ? 11:18, Etienne Robillard a ?crit?: > I opened a ticket for this issue: > https://bitbucket.org/pypy/pypy/issues/2681/typeerror-__weakref__-slot-disallowed-we > > Etienne > > Le 2017-10-13 ? 08:13, Etienne Robillard a ?crit?: >> Traceback (most recent call last): >> ? File >> "/usr/local/pypy/pypy2-v5.9.0-linux32/lib-python/2.7/wsgiref/handlers.py", >> line 85, in run >> ??? self.result = application(self.environ, self.start_response) >> ? File >> "/usr/local/pypy/pypy2-v5.9.0-linux32/lib-python/2.7/wsgiref/validate.py", >> line 176, in lint_app >> ??? iterator = application(environ, start_response_wrapper) >> ? File >> "/home/erob/src/django-hotsauce-oauthclient/lib/wsgi_oauth2/middleware.py", >> line 292, in __call__ >> ??? return self.application(environ, start_response) >> ? File "lib/notmm/controllers/wsgi.pyx", line 136, in >> notmm.controllers.wsgi.BaseController.__call__ >> TypeError: __weakref__ slot disallowed: we already got one > -- Etienne Robillard tkadm30 at yandex.com http://www.isotopesoftware.ca/ From jason.madden at nextthought.com Fri Oct 13 12:39:30 2017 From: jason.madden at nextthought.com (Jason Madden) Date: Fri, 13 Oct 2017 11:39:30 -0500 Subject: [pypy-dev] AttributeError: 'built in-code' object has no attribute 'co_code' In-Reply-To: References: <14765118-48a8-95d6-7f38-d5e2dd78d21f@yandex.com> <7d322ccb-7c9a-97b9-a634-2b678bb06824@yandex.com> <25044364-0650-86f7-daca-2ab69c8c0958@yandex.com>

Message-ID: Hi, I don't know anything near like enough about your code or environment to be able to make any productive suggestions. I left a comment on the issue with some ideas on where to look based on something I've seen in the distant past but that's about all I can do. (Oh, one other thing I would try if at all possible: Take Cython out of the mix and see what happens when you just run the code as pure Python.) Jason > On Oct 13, 2017, at 11:34, Etienne Robillard wrote: > > Hi Jason, > > How would you tackle this issue? > > I may not have defined any __slots__ in my code. > > Thank you in advance, > Etienne > > Le 2017-10-13 ? 11:18, Etienne Robillard a ?crit : >> I opened a ticket for this issue: https://bitbucket.org/pypy/pypy/issues/2681/typeerror-__weakref__-slot-disallowed-we >> >> Etienne >> >> Le 2017-10-13 ? 08:13, Etienne Robillard a ?crit : >>> Traceback (most recent call last): >>> File "/usr/local/pypy/pypy2-v5.9.0-linux32/lib-python/2.7/wsgiref/handlers.py", line 85, in run >>> self.result = application(self.environ, self.start_response) >>> File "/usr/local/pypy/pypy2-v5.9.0-linux32/lib-python/2.7/wsgiref/validate.py", line 176, in lint_app >>> iterator = application(environ, start_response_wrapper) >>> File "/home/erob/src/django-hotsauce-oauthclient/lib/wsgi_oauth2/middleware.py", line 292, in __call__ >>> return self.application(environ, start_response) >>> File "lib/notmm/controllers/wsgi.pyx", line 136, in notmm.controllers.wsgi.BaseController.__call__ >>> TypeError: __weakref__ slot disallowed: we already got one >> > > -- > Etienne Robillard > tkadm30 at yandex.com > http://www.isotopesoftware.ca/ > From tkadm30 at yandex.com Fri Oct 13 14:04:36 2017 From: tkadm30 at yandex.com (Etienne Robillard) Date: Fri, 13 Oct 2017 14:04:36 -0400 Subject: [pypy-dev] AttributeError: 'built in-code' object has no attribute 'co_code' In-Reply-To: References: <14765118-48a8-95d6-7f38-d5e2dd78d21f@yandex.com> <7d322ccb-7c9a-97b9-a634-2b678bb06824@yandex.com> <25044364-0650-86f7-daca-2ab69c8c0958@yandex.com>

Message-ID: <4f1835c1-ee3d-855d-73d2-c9b26b4c5ec0@yandex.com> Hi Jason, Thanks for writing. I fixed all the issues and everything seem to work so far. I'm going to add PyPy to the list of supported platforms for Django-hotsauce. :) Cheers, Etienne Le 2017-10-13 ? 12:39, Jason Madden a ?crit?: > Hi, > > I don't know anything near like enough about your code or environment to be able to make any productive suggestions. I left a comment on the issue with some ideas on where to look based on something I've seen in the distant past but that's about all I can do. (Oh, one other thing I would try if at all possible: Take Cython out of the mix and see what happens when you just run the code as pure Python.) > > Jason > >> On Oct 13, 2017, at 11:34, Etienne Robillard wrote: >> >> Hi Jason, >> >> How would you tackle this issue? >> >> I may not have defined any __slots__ in my code. >> >> Thank you in advance, >> Etienne >> >> Le 2017-10-13 ? 11:18, Etienne Robillard a ?crit : >>> I opened a ticket for this issue: https://bitbucket.org/pypy/pypy/issues/2681/typeerror-__weakref__-slot-disallowed-we >>> >>> Etienne >>> >>> Le 2017-10-13 ? 08:13, Etienne Robillard a ?crit : >>>> Traceback (most recent call last): >>>> File "/usr/local/pypy/pypy2-v5.9.0-linux32/lib-python/2.7/wsgiref/handlers.py", line 85, in run >>>> self.result = application(self.environ, self.start_response) >>>> File "/usr/local/pypy/pypy2-v5.9.0-linux32/lib-python/2.7/wsgiref/validate.py", line 176, in lint_app >>>> iterator = application(environ, start_response_wrapper) >>>> File "/home/erob/src/django-hotsauce-oauthclient/lib/wsgi_oauth2/middleware.py", line 292, in __call__ >>>> return self.application(environ, start_response) >>>> File "lib/notmm/controllers/wsgi.pyx", line 136, in notmm.controllers.wsgi.BaseController.__call__ >>>> TypeError: __weakref__ slot disallowed: we already got one >> -- >> Etienne Robillard >> tkadm30 at yandex.com >> http://www.isotopesoftware.ca/ >> -- Etienne Robillard tkadm30 at yandex.com http://www.isotopesoftware.ca/ From yury at shurup.com Fri Oct 13 15:13:41 2017 From: yury at shurup.com (Yury V. Zaytsev) Date: Fri, 13 Oct 2017 21:13:41 +0200 (CEST) Subject: [pypy-dev] AttributeError: 'built in-code' object has no attribute 'co_code' In-Reply-To: <4f1835c1-ee3d-855d-73d2-c9b26b4c5ec0@yandex.com> References: <14765118-48a8-95d6-7f38-d5e2dd78d21f@yandex.com> <7d322ccb-7c9a-97b9-a634-2b678bb06824@yandex.com> <25044364-0650-86f7-daca-2ab69c8c0958@yandex.com>

<4f1835c1-ee3d-855d-73d2-c9b26b4c5ec0@yandex.com> Message-ID: On Fri, 13 Oct 2017, Etienne Robillard wrote: > Thanks for writing. I fixed all the issues and everything seem to work > so far. I'm going to add PyPy to the list of supported platforms for > Django-hotsauce. :) So, in the end, these issues were with your code, rather than PyPy / Cython? I think it would have been helpful if you gave a rundown of the problems you observed and how did you fix them... -- Sincerely yours, Yury V. Zaytsev From tkadm30 at yandex.com Fri Oct 13 15:49:48 2017 From: tkadm30 at yandex.com (Etienne Robillard) Date: Fri, 13 Oct 2017 15:49:48 -0400 Subject: [pypy-dev] AttributeError: 'built in-code' object has no attribute 'co_code' In-Reply-To: References: <14765118-48a8-95d6-7f38-d5e2dd78d21f@yandex.com> <7d322ccb-7c9a-97b9-a634-2b678bb06824@yandex.com> <25044364-0650-86f7-daca-2ab69c8c0958@yandex.com>

<4f1835c1-ee3d-855d-73d2-c9b26b4c5ec0@yandex.com> Message-ID: <6e7eb66d-3e75-839c-aee1-cd5e6870d00c@yandex.com> Hi Yuri I created a new branch for tracking my changes to django-hotsauce on PyPy: % hg clone -b 0.7-pypy https://bitbucket.org/tkadm30/django-hotsauce django-hotsauce Best Regards, Etienne Le 2017-10-13 ? 15:13, Yury V. Zaytsev a ?crit?: > On Fri, 13 Oct 2017, Etienne Robillard wrote: > >> Thanks for writing. I fixed all the issues and everything seem to >> work so far. I'm going to add PyPy to the list of supported platforms >> for Django-hotsauce. :) > > So, in the end, these issues were with your code, rather than PyPy / > Cython? I think it would have been helpful if you gave a rundown of > the problems you observed and how did you fix them... > -- Etienne Robillard tkadm30 at yandex.com http://www.isotopesoftware.ca/ From tkadm30 at yandex.com Mon Oct 16 05:56:39 2017 From: tkadm30 at yandex.com (Etienne Robillard) Date: Mon, 16 Oct 2017 05:56:39 -0400 Subject: [pypy-dev] New django-hotsauce development group Message-ID: <100e51b0-54fe-5cb0-234c-08e7bf10d34b@yandex.com> Hi, I'm pleased to announce the availability of a new public discussion group for Django-hotsauce: https://groups.google.com/forum/#!forum/django-hotsauce Talented CPython and PyPy developers are invited to join and meet the django-hotsauce community. Best regards, Etienne -- Etienne Robillard tkadm30 at yandex.com http://www.isotopesoftware.ca/ From dcallahan at fb.com Mon Oct 16 15:18:07 2017 From: dcallahan at fb.com (David Callahan) Date: Mon, 16 Oct 2017 19:18:07 +0000 Subject: [pypy-dev] fiber support Message-ID: folly:fibers (https://github.com/facebook/folly/tree/master/folly/fibers ) is a C++ package for lightweight, cooperatively scheduled threads. We have an application which extends this to CPython by adding the following save/restore code around task function invocation: auto tstate = PyThreadState_Get(); CHECK_NOTNULL(tstate); auto savedFrame = tstate->frame; auto savedExcType = tstate->exc_type; auto savedExcValue = tstate->exc_value; auto savedExcTraceback = tstate->exc_traceback; func(); tstate->frame = savedFrame; tstate->exc_type = savedExcType; tstate->exc_value = savedExcValue; tstate->exc_traceback = savedExcTraceback; (here func is a boost::python::object) This does not work in PyPy 5.9 immediately because the thread state object does not expose these fields nor are there accessor methods. Is there a way to get similar effect in PyPy? Thanks david -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Tue Oct 17 15:09:07 2017 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 17 Oct 2017 19:09:07 +0000 Subject: [pypy-dev] fiber support In-Reply-To: References: Message-ID: Hi David. You're probably completely aware that such calls (using cpyext) would completely kill performance, right? I didn't look in detail, but in order to get performance, you would need to write such calls using cffi. Best regards, Maciej Fijalkowski On Mon, Oct 16, 2017 at 7:18 PM, David Callahan wrote: > folly:fibers (https://github.com/facebook/folly/tree/master/folly/fibers ) > is a C++ package for lightweight, cooperatively scheduled threads. We have > an application which extends this to CPython by adding the following > save/restore code around task function invocation: > > > > auto tstate = PyThreadState_Get(); > > CHECK_NOTNULL(tstate); > > auto savedFrame = tstate->frame; > > auto savedExcType = tstate->exc_type; > > auto savedExcValue = tstate->exc_value; > > auto savedExcTraceback = tstate->exc_traceback; > > func(); > > tstate->frame = savedFrame; > > tstate->exc_type = savedExcType; > > tstate->exc_value = savedExcValue; > > tstate->exc_traceback = savedExcTraceback; > > > > (here func is a boost::python::object) > > > > This does not work in PyPy 5.9 immediately because the thread state object > does not expose these fields nor are there accessor methods. > > > > Is there a way to get similar effect in PyPy? > > > > Thanks > > david > > > > > > > > > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > From dcallahan at fb.com Tue Oct 17 15:57:01 2017 From: dcallahan at fb.com (David Callahan) Date: Tue, 17 Oct 2017 19:57:01 +0000 Subject: [pypy-dev] fiber support In-Reply-To: References:

Message-ID: <8A933299-D760-4869-B052-6FF4E2271B2C@fb.com> Hi Maciej, thanks for the reply I have been warned before about the performance of cpyext but we have hundreds of source dependencies and removing such reliance does not help us in the short term. Thus, performance is not a concern yet since we are not even at a point to run the code and know what is expensive and what is irrelevant. At this point, it is not clear to me how to mimic this code using the cpyext later at all. Or if it is even necessary if PyPy has a different architecture for exception handling. --david On 10/17/17, 12:10 PM, "Maciej Fijalkowski" wrote: Hi David. You're probably completely aware that such calls (using cpyext) would completely kill performance, right? I didn't look in detail, but in order to get performance, you would need to write such calls using cffi. Best regards, Maciej Fijalkowski On Mon, Oct 16, 2017 at 7:18 PM, David Callahan wrote: > folly:fibers (https://github.com/facebook/folly/tree/master/folly/fibers ) > is a C++ package for lightweight, cooperatively scheduled threads. We have > an application which extends this to CPython by adding the following > save/restore code around task function invocation: > > > > auto tstate = PyThreadState_Get(); > > CHECK_NOTNULL(tstate); > > auto savedFrame = tstate->frame; > > auto savedExcType = tstate->exc_type; > > auto savedExcValue = tstate->exc_value; > > auto savedExcTraceback = tstate->exc_traceback; > > func(); > > tstate->frame = savedFrame; > > tstate->exc_type = savedExcType; > > tstate->exc_value = savedExcValue; > > tstate->exc_traceback = savedExcTraceback; > > > > (here func is a boost::python::object) > > > > This does not work in PyPy 5.9 immediately because the thread state object > does not expose these fields nor are there accessor methods. > > > > Is there a way to get similar effect in PyPy? > > > > Thanks > > david > > > > > > > > > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman_listinfo_pypy-2Ddev&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=lFyiPUrFdOHdaobP7i4hoA&m=J_q51G31j5IKPKCE_2ZALuqBrWgdBs58pczmLTt_ml8&s=ZPehk3U44BJaOoWPII5iCNEv91mTCk_VoMR4Xq9Oh9k&e= > From fijall at gmail.com Tue Oct 17 16:21:04 2017 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 17 Oct 2017 20:21:04 +0000 Subject: [pypy-dev] fiber support In-Reply-To: <8A933299-D760-4869-B052-6FF4E2271B2C@fb.com> References:

<8A933299-D760-4869-B052-6FF4E2271B2C@fb.com> Message-ID: Heh, sure Did you look whether such functionality can be replicated with PyThreadState_New and PyThreadState_Swap? Cheers, fijal On Tue, Oct 17, 2017 at 7:57 PM, David Callahan wrote: > Hi Maciej, thanks for the reply > > I have been warned before about the performance of cpyext but we have hundreds of source dependencies and removing such reliance does not help us in the short term. Thus, performance is not a concern yet since we are not even at a point to run the code and know what is expensive and what is irrelevant. > > At this point, it is not clear to me how to mimic this code using the cpyext later at all. Or if it is even necessary if PyPy has a different architecture for exception handling. > > --david > > On 10/17/17, 12:10 PM, "Maciej Fijalkowski" wrote: > > Hi David. > > You're probably completely aware that such calls (using cpyext) would > completely kill performance, right? I didn't look in detail, but in > order to get performance, you would need to write such calls using > cffi. > > Best regards, > Maciej Fijalkowski > > On Mon, Oct 16, 2017 at 7:18 PM, David Callahan wrote: > > folly:fibers (https://github.com/facebook/folly/tree/master/folly/fibers ) > > is a C++ package for lightweight, cooperatively scheduled threads. We have > > an application which extends this to CPython by adding the following > > save/restore code around task function invocation: > > > > > > > > auto tstate = PyThreadState_Get(); > > > > CHECK_NOTNULL(tstate); > > > > auto savedFrame = tstate->frame; > > > > auto savedExcType = tstate->exc_type; > > > > auto savedExcValue = tstate->exc_value; > > > > auto savedExcTraceback = tstate->exc_traceback; > > > > func(); > > > > tstate->frame = savedFrame; > > > > tstate->exc_type = savedExcType; > > > > tstate->exc_value = savedExcValue; > > > > tstate->exc_traceback = savedExcTraceback; > > > > > > > > (here func is a boost::python::object) > > > > > > > > This does not work in PyPy 5.9 immediately because the thread state object > > does not expose these fields nor are there accessor methods. > > > > > > > > Is there a way to get similar effect in PyPy? > > > > > > > > Thanks > > > > david > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > pypy-dev mailing list > > pypy-dev at python.org > > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman_listinfo_pypy-2Ddev&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=lFyiPUrFdOHdaobP7i4hoA&m=J_q51G31j5IKPKCE_2ZALuqBrWgdBs58pczmLTt_ml8&s=ZPehk3U44BJaOoWPII5iCNEv91mTCk_VoMR4Xq9Oh9k&e= > > > > From armin.rigo at gmail.com Wed Oct 18 06:59:17 2017 From: armin.rigo at gmail.com (Armin Rigo) Date: Wed, 18 Oct 2017 12:59:17 +0200 Subject: [pypy-dev] fiber support In-Reply-To: References: Message-ID: Hi David, On 16 October 2017 at 21:18, David Callahan wrote: > folly:fibers (https://github.com/facebook/folly/tree/master/folly/fibers ) > is a C++ package for lightweight, cooperatively scheduled threads. We have > an application which extends this to CPython by adding the following > save/restore code around task function invocation: This will never work inside PyPy, even using cpyext, because cpyext's notion of exceptions is different from the internal one inside PyPy. Instead, you should look at the ``_continuation`` module or more likely the ``greenlet`` module, which is built-in in PyPy (and available as a CPython extension module, too, if you like). It should give you a replacement, though of course you need to adapt the callers. If you really want to use folly:fibers or a specific package manipulating the C stack, then I'm afraid it's quite hard. The ``greenlet`` module in PyPy is based on ``stacklet``, which is an internal part of PyPy doing the same thing using the necessary bookkeeping for PyPy, which is different than the one for CPython you included in your e-mail. If you really want to look there, you'd need to replicate that logic around folly:fibres (search for all "stacklet" file names in several places in the PyPy source code). A bient?t, Armin. From matti.picus at gmail.com Sat Oct 28 16:54:36 2017 From: matti.picus at gmail.com (Matti Picus) Date: Sat, 28 Oct 2017 23:54:36 +0300 Subject: [pypy-dev] Enhancing the package.py script to download and build dependencies Message-ID: An HTML attachment was scrubbed... URL: From matti.picus at gmail.com Sat Oct 28 16:58:12 2017 From: matti.picus at gmail.com (Matti Picus) Date: Sat, 28 Oct 2017 23:58:12 +0300 Subject: [pypy-dev] Enhancing the package.py script to download and build dependencies Message-ID: <4d498da3-4ee0-c9ab-614e-a5d2a61defe7@gmail.com> danchr has submitted a pull request that enables our cffi build step (executed twice - once during tranlation and once during packaging) to download external dependencies. This is needed for MacOSX where the versions of OpenSSL, lzma, and _gdbm are not current or missing for pypy3. Since it is a departure from current practice, I would like to solicit a wider base of opinions before committing. The pull request can be seen here https://bitbucket.org/pypy/pypy/pull-requests/571/support-for-embedding-pypy3-dependencies and the builds produced are here http://buildbot.pypy.org/nightly/py3.5-mac-embedding Any thoughts? Matti From armin.rigo at gmail.com Sun Oct 29 18:11:52 2017 From: armin.rigo at gmail.com (Armin Rigo) Date: Mon, 30 Oct 2017 00:11:52 +0200 Subject: [pypy-dev] Leysin Winter sprint Message-ID: Hi all, I'm trying to organise the next winter sprint a bit in advance. It might tentatively be occurring in March 2018. If people have preferences for the exact week, please do tell :-) A bient?t, Armin. From anto.cuni at gmail.com Sun Oct 29 18:47:36 2017 From: anto.cuni at gmail.com (Antonio Cuni) Date: Sun, 29 Oct 2017 23:47:36 +0100 Subject: [pypy-dev] Leysin Winter sprint In-Reply-To: References: Message-ID: Hi Armin, I would probably be unable to come during the first two weeks of march, so for me the preference is basically "anything >= 15/03" On Sun, Oct 29, 2017 at 11:11 PM, Armin Rigo wrote: > Hi all, > > I'm trying to organise the next winter sprint a bit in advance. It > might tentatively be occurring in March 2018. If people have > preferences for the exact week, please do tell :-) > > > A bient?t, > > Armin. > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: