From matti.picus at gmail.com Thu Aug 16 19:48:29 2018 From: matti.picus at gmail.com (Matti Picus) Date: Fri, 17 Aug 2018 02:48:29 +0300 Subject: [pypy-dev] PyOpenCL wants to move from cffi to pybind11 Message-ID: <49dd0d67-88c6-78a8-a6b8-4df40f5c20ff@gmail.com> PyOpenCL allows one to write kernels for OpenCL in python and uses numpy arrays as the data to pass around. They now use cffi bindings, but are thinking of moving to PyBind11. https://lists.tiker.net/pipermail/pyopencl/2018-August/002395.html They claim 1500 less code lines and a 2x speedup (on cpython). It seems this is a step backwards for us, PyBind11 uses the python c-api to wrap C++ code and I do not see how we can make that JIT freindly. On the other hand, I do not know how easy it is to move form PyBind11 to cppyy. Any thoughts? I used to follow their codebase closely, now I just lurk on the mailing list, and do not use the package myself. Matti From wlavrijsen at lbl.gov Thu Aug 16 23:56:27 2018 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Thu, 16 Aug 2018 20:56:27 -0700 (PDT) Subject: [pypy-dev] PyOpenCL wants to move from cffi to pybind11 In-Reply-To: <49dd0d67-88c6-78a8-a6b8-4df40f5c20ff@gmail.com> References: <49dd0d67-88c6-78a8-a6b8-4df40f5c20ff@gmail.com> Message-ID: Matti, > They claim 1500 less code lines and a 2x speedup (on cpython). I certainly believe the former, but am surprised by the latter. From some simple benchmarking, I've found PyBind11 to be very slow (as in 3x and up). Perhaps the problem is in the C wrappers more so than in cffi? > On the other hand, I do not know how easy it is to move form PyBind11 to > cppyy. Any thoughts? The question is what they'd think of the LLVM dependency, since the e-mail explicitly states they're happy to be without a boost one. A move itself should be easy enough, but they'd have to be convinced by performance, so it might be good to get a benchmark out of them to measure and see what they consider more important: speed or few dependencies. Best regards, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From matti.picus at gmail.com Mon Aug 27 12:38:35 2018 From: matti.picus at gmail.com (Matti Picus) Date: Mon, 27 Aug 2018 18:38:35 +0200 Subject: [pypy-dev] bencher4 fialing own py3 builds Message-ID: Own nightly builds on p3.5 are failing numerous cpyext tests. The problem seems to be related to the _cffi_backend module somehow hitting an assert at import, but I cannot reproduce the problem locally. I have tried many things on the unicode-utf8-py3 branch to solve the problem, but to no avail. See the last 3 or 4 commits on that branch. Any help would be greatly appreciated. Perhaps we should retire bencher4 and find another build machine, one that we can control a bit more? Matti From armin.rigo at gmail.com Mon Aug 27 15:44:00 2018 From: armin.rigo at gmail.com (Armin Rigo) Date: Mon, 27 Aug 2018 21:44:00 +0200 Subject: [pypy-dev] bencher4 fialing own py3 builds In-Reply-To: References: Message-ID: Hi Matti, On 27 August 2018 at 18:38, Matti Picus wrote: > Own nightly builds on p3.5 are failing numerous cpyext tests. The problem > seems to be related to the _cffi_backend module somehow hitting an assert at > import, but I cannot reproduce the problem locally. It reproduces if we run both test_buffer and test_translate together (py.test test_buffer.py test_translate.py). > Perhaps we should retire bencher4 and find another build machine, one > that we can control a bit more? It wouldn't be a bad idea. Note that we should check again if it would be OK to reinstall bencher4 in a way that we fully control (including giving accounts to people at our discretion), which would save us the trouble of looking elsewhere. But maybe it still isn't possible to do that... A bient?t, Armin. From anto.cuni at gmail.com Wed Aug 29 06:08:40 2018 From: anto.cuni at gmail.com (Antonio Cuni) Date: Wed, 29 Aug 2018 12:08:40 +0200 Subject: [pypy-dev] bencher4 fialing own py3 builds In-Reply-To: References: Message-ID: On Mon, Aug 27, 2018 at 6:38 PM Matti Picus wrote: > . Perhaps we should retire bencher4 and find > another build machine, one that we can control a bit more another thing we could try to have more repeatable builds is to use docker containers; for example, some time ago I used this approach to debug an issue which appeared only on 32bit builds: https://github.com/antocuni/dockerpypy This is orthogonal to using a machine on which we have more control of course. Probably the best is to have full control on the machine AND use docker containers :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From matti.picus at gmail.com Fri Aug 31 02:15:30 2018 From: matti.picus at gmail.com (Matti Picus) Date: Fri, 31 Aug 2018 08:15:30 +0200 Subject: [pypy-dev] translation problems on unicode-utf8-py3 Message-ID: I have two failures with translation that I need some help with on the unicode-utf8-py3 branch. Occasionally while translating the branch locally, I get an error [translation:ERROR] UnsupportedFieldExc: unexpected array(struct(struct)) [translation] start debugger... > /home/matti/pypy_stuff/pypy_35/rpython/jit/codewriter/heaptracker.py(111)all_interiorfielddescrs() -> raise UnsupportedFieldExc("unexpected array(struct(struct))") I can get to the offending struct The struct is from rpython/rlib/rutf8.py: UTF8_INDEX_STORAGE = lltype.GcArray(lltype.Struct('utf8_loc_elem', ??????? ('baseindex', lltype.Signed), ??????? ('ofs', lltype.FixedSizeArray(lltype.Char, 16)), ??? )) and is returned from the function create_utf8_index_storage which is called in that file as well as in objspace/std/unicodeobject.py I put more of the error messsage and the code bits in this gist https://gist.github.com/mattip/057554dd7eba64f8cbf2bc1a70c1c6eb The second strange failure is of a own z_translation test http://buildbot.pypy.org/summary/longrepr?testname=test__codecs_translates&builder=own-linux-x86-64&build=6944&mod=pypy.module._codecs.test.test_ztranslation Any help would be welcome. Matti From matti.picus at gmail.com Fri Aug 31 03:40:55 2018 From: matti.picus at gmail.com (Matti Picus) Date: Fri, 31 Aug 2018 09:40:55 +0200 Subject: [pypy-dev] last benchmark run was June 2 Message-ID: We lost the machine that was running our benchmark suite, the last run sees to have been June2 http://speed.pypy.org/changes/. Choices: - Ask the PSF to run on the machine that is used to run benchmarks for speed.python.org. The machine, speed-python.osuosl.org,? is very powerful and not heavily used, a description of it is https://speed.python.org/about/ under "The Machine". The users with psf-authorized access to the machine, from https://github.com/python/psf-chef, are fijal, zware, mattip, haypo (results of "grep python-speed -r ." in that repo) - Use one of bencher4, baroquesoftware.com, or any other pypy-specific donated machine. - Stop running benchmarks Any thoughts? Do we need nightly benchmarks or should we run them less often? Should we also be running py3.5 benchmarks? Should we upload to speed.python.org, speed.pypy.org or both? Matti From armin.rigo at gmail.com Fri Aug 31 04:03:39 2018 From: armin.rigo at gmail.com (Armin Rigo) Date: Fri, 31 Aug 2018 10:03:39 +0200 Subject: [pypy-dev] translation problems on unicode-utf8-py3 In-Reply-To: References: Message-ID: Hi, On 31 August 2018 at 08:15, Matti Picus wrote: > I have two failures with translation that I need some help with on the > unicode-utf8-py3 branch. > Occasionally while translating the branch locally, I get an error I've tried both on unicode-utf8-py3 and on unicode-utf8 (which seems to have the same logic around these functions). I didn't get the error. The error message is not enough to understand what is wrong, I'll need a pdb prompt and look around. Maybe if you manage to reproduce in a "screen" on bencher4, I could connect. Armin