From armin.rigo at gmail.com Tue May 2 13:07:22 2017 From: armin.rigo at gmail.com (Armin Rigo) Date: Tue, 2 May 2017 19:07:22 +0200 Subject: [pypy-dev] 5.7.1+dfsg-1~ppa1~ubuntu16.04 is incomplete Message-ID: Hi Stefano, We got report that 5.7.1+dfsg-1~ppa1~ubuntu16.04 is incomplete: https://bitbucket.org/pypy/pypy/issues/2550/cpyext_objecth-no-such-file-or-directory Thanks ! Armin From g.rodola at gmail.com Tue May 2 16:23:34 2017 From: g.rodola at gmail.com (Giampaolo Rodola') Date: Tue, 2 May 2017 22:23:34 +0200 Subject: [pypy-dev] Can't compile C extensions on pypy3 aka alternatives for PyModule_GetState Message-ID: Hello, I'm the author of psutil and I received 2 bug reports complaining about lack of PyModule_GetState: https://github.com/giampaolo/psutil/issues/853 https://github.com/giampaolo/psutil/issues/1045 There is also a ticket for pypy: https://bitbucket.org/pypy/pypy/issues/2450/cant-import-psutil-on-pypy3-missing Since it seems this is not going to be fixed any time soon I'm curios: does this mean C extension modules cannot be compiled with pypy3? Are there alternatives to PyModule_GetState() which could be used? Thanks in advance. -- Giampaolo - http://grodola.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From yury at shurup.com Sat May 6 16:45:49 2017 From: yury at shurup.com (Yury V. Zaytsev) Date: Sat, 6 May 2017 22:45:49 +0200 (CEST) Subject: [pypy-dev] win32 buildbot In-Reply-To: References: Message-ID: Hi Matti, I've just updated the win32 slave, hopefully nightly builds will work. It seems that the instructions you linked are not correct anymore though as the paths \tcltk\* no longer exist. Another note: I'm still building pypy with pypy 2.5.1 (which is probably bad, because new versions are probably a lot faster, and are not getting regular testing on the buildbot) since later versions used to abort with mysterious OOM errors. Have you been able to build pypy on win32 with latest pypy and/or reproduce the errors I'm getting? -- Sincerely yours, Yury V. Zaytsev On Fri, 21 Apr 2017, Matti Picus wrote: > I had to update the zip, the sha256 is now different. Use the one on the link > > http://doc.pypy.org/en/latest/windows.html#abridged-method-for-ojit-builds-using-visual-studio-2008 > > > On 21/04/17 16:06, Matti Picus wrote: >> >> Hi. The buildbot seems stuck, could you restart it? >> >> I have updated the external build dependencies, available at >> https://bitbucket.org/pypy/pypy/downloads/local_5.8.zip >> >> Could you also update the buildbot with these? The checksum is >> >> cc375a677aabcb46a50b96f23bdebd0b526cbb02202b85a7148845e2849ffd6d >> >> which I calculated with >> hashlib.sha256(open(r'local_5.8.zip').read()).digesthex() >> >> > > > From matti.picus at gmail.com Sat May 6 17:17:45 2017 From: matti.picus at gmail.com (Matti Picus) Date: Sun, 7 May 2017 00:17:45 +0300 Subject: [pypy-dev] win32 buildbot In-Reply-To: References: Message-ID: On 06/05/17 23:45, Yury V. Zaytsev wrote: > Hi Matti, > > I've just updated the win32 slave, hopefully nightly builds will work. > It seems that the instructions you linked are not correct anymore > though as the paths \tcltk\* no longer exist. > Thanks, that should get us one step closer to a successful win32 packaging run on the py3.5 branch. I hope those files are no longer needed, we will see tonight > Another note: I'm still building pypy with pypy 2.5.1 (which is > probably bad, because new versions are probably a lot faster, and are > not getting regular testing on the buildbot) since later versions used > to abort with mysterious OOM errors. Have you been able to build pypy > on win32 with latest pypy and/or reproduce the errors I'm getting? > Yes, it is worth updating to 5.7.1. How much memory does the buildslave have? The buildslave uses the Microsoft Python Tools for Python 2.7 compiler bundle. Perhaps it needs more memory than the Visual Studio 9.0 compiler I am using, and so requires more than 8GB? Matti From yury at shurup.com Sun May 7 04:05:05 2017 From: yury at shurup.com (Yury V. Zaytsev) Date: Sun, 7 May 2017 10:05:05 +0200 (CEST) Subject: [pypy-dev] win32 buildbot In-Reply-To: References: Message-ID: On Sun, 7 May 2017, Matti Picus wrote: > Yes, it is worth updating to 5.7.1. How much memory does the buildslave > have? The buildslave uses the Microsoft Python Tools for Python 2.7 > compiler bundle. Perhaps it needs more memory than the Visual Studio 9.0 > compiler I am using, and so requires more than 8GB? It currently has 16G RAM, and I'm sorry, but I don't think I can add more at the moment. In as far as compiler is concerned, the Python Tools compiler should be exactly the same as MSVC9 in terms of resource consumption (the difference is in features / headers / ..., i.e. it's closer to the SDK compilers rather than to MSVC itself): https://mail.python.org/pipermail/python-dev/2014-September/136499.html Let's see if the current build runs through, and then I can try it ;-) -- Sincerely yours, Yury V. Zaytsev From yury at shurup.com Sun May 7 04:57:05 2017 From: yury at shurup.com (Yury V. Zaytsev) Date: Sun, 7 May 2017 10:57:05 +0200 (CEST) Subject: [pypy-dev] win32 buildbot In-Reply-To: References: Message-ID: On Sun, 7 May 2017, Yury V. Zaytsev wrote: > Let's see if the current build runs through, and then I can try it ;-) Well, good news is that the compilation has worked, but bad news is that as I mentioned, the directory structure (and contents) has changed, and it now fails to find tcl/tk runtime where it expects it to be, because it picks the libraries from my old pypy installation instead of local_5.8: http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/3092/steps/shell_1/logs/stdio I guess the problem is that you're shipping tcl85t.dll & tk85t.dll in local_5.8, whereas it is looking for tcl85.dll & tk85.dll as they were shipped in local_2.4, and so not finding them picks the ones from pypy, consequently failing to find the runtime, so either pypy packaging script, or local package should be adjusted. I will see now if I can update pypy when the build finishes completely... -- Sincerely yours, Yury V. Zaytsev From yury at shurup.com Mon May 8 01:32:24 2017 From: yury at shurup.com (Yury V. Zaytsev) Date: Mon, 8 May 2017 07:32:24 +0200 (CEST) Subject: [pypy-dev] win32 buildbot In-Reply-To: References: Message-ID: On Sun, 7 May 2017, Yury V. Zaytsev wrote: > I will see now if I can update pypy when the build finishes > completely... So, the good news is that apparently with PyPy 5.7.1 the translation & compilation works again, so I'll stick with it, yay! The not so good news is that the TCL/TK packaging problem didn't magically solve itself, so you'd have to look into it... -- Sincerely yours, Yury V. Zaytsev From matti.picus at gmail.com Mon May 8 16:12:54 2017 From: matti.picus at gmail.com (Matti Picus) Date: Mon, 8 May 2017 23:12:54 +0300 Subject: [pypy-dev] win32 buildbot In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: From yury at shurup.com Mon May 8 16:29:48 2017 From: yury at shurup.com (Yury V. Zaytsev) Date: Mon, 8 May 2017 22:29:48 +0200 (CEST) Subject: [pypy-dev] win32 buildbot In-Reply-To: References: Message-ID: On Mon, 8 May 2017, Matti Picus wrote: > I rebuilt the local_5.8.zip, and fixed the instructions. Please try the > newer package. Hi Matti, I've just updated the build slave, let's see if this works tomorrow morning... keeping the fingers crossed ;-) -- Sincerely yours, Yury V. Zaytsev From yury at shurup.com Tue May 9 14:32:48 2017 From: yury at shurup.com (Yury V. Zaytsev) Date: Tue, 9 May 2017 20:32:48 +0200 (CEST) Subject: [pypy-dev] win32 buildbot In-Reply-To: References: Message-ID: On Mon, 8 May 2017, Yury V. Zaytsev wrote: > On Mon, 8 May 2017, Matti Picus wrote: > >> I rebuilt the local_5.8.zip, and fixed the instructions. Please try >> the newer package. > > I've just updated the build slave, let's see if this works tomorrow > morning... keeping the fingers crossed ;-) So, PyPy builds again, high five! :-) Only PyPy3 bails out while compiling the TCL/TK extension, but the problem is a different one now. -- Sincerely yours, Yury V. Zaytsev From matti.picus at gmail.com Wed May 10 08:20:34 2017 From: matti.picus at gmail.com (Matti Picus) Date: Wed, 10 May 2017 15:20:34 +0300 Subject: [pypy-dev] win32 buildbot In-Reply-To: References: Message-ID: <4404e61f-69f4-b7c2-c156-a776ab5f79da@gmail.com> An HTML attachment was scrubbed... URL: From yury at shurup.com Wed May 10 15:53:05 2017 From: yury at shurup.com (Yury V. Zaytsev) Date: Wed, 10 May 2017 21:53:05 +0200 (CEST) Subject: [pypy-dev] win32 buildbot In-Reply-To: <4404e61f-69f4-b7c2-c156-a776ab5f79da@gmail.com> References: <4404e61f-69f4-b7c2-c156-a776ab5f79da@gmail.com> Message-ID: On Wed, 10 May 2017, Matti Picus wrote: > It seems own tests are taking a very long time on this machine. I have > explored using jom.exe (https://github.com/qt-labs/jom) instead of nmake > to allow parallel compilation, which reduces my run of rpython's > translator\c\test\test_typed.py from ~700 secs to ~600 secs, where on > the same machine it is under 200 secs with linux. Note that the buildbot > requires over 2800 secs for the same test (see > http://buildbot.pypy.org/builders/own-win-x86-32/builds/1395/steps/shell_6/logs/stdio) It is simply so that the machine kindly provided by the folks at Dartmouth College where I migrated the Windows VM to is way slower than my old box, and has only 2 physical cores in total... Also take a note, that the same machine also does build & test runs for git-annex, which take about an hour each, so that also doesn't speedup PyPy builds. Anyways, in the mean time, it looks like I've found colo in Germany for my old machine, but I still need to: * Get it shipped to the new datacenter * Partly replace old hardware * Install it & set it up anew No ETAs, but something is moving in some direction at least. Getting hardware is for now an unsolved problem though. I need 2 x 2/3 TB SATA enterprise HDDs and 2 x enterprise SSDs + would be good to replace unreliable DDR3 RAM sticks... Stuff that I have is 5 years old, and HDDs tend to fail when they are old :-/ -- Sincerely yours, Yury V. Zaytsev From fijall at gmail.com Fri May 12 06:02:33 2017 From: fijall at gmail.com (Maciej Fijalkowski) Date: Fri, 12 May 2017 12:02:33 +0200 Subject: [pypy-dev] win32 buildbot In-Reply-To: References: <4404e61f-69f4-b7c2-c156-a776ab5f79da@gmail.com> Message-ID: Hi Yuri. While I do not have powers to make a decision, I think it's very reasonable if pypy pays for such parts. If you can provide cost estimates, I'll try to go through the appropriate channels to have this approved, if you're interested. On Wed, May 10, 2017 at 9:53 PM, Yury V. Zaytsev wrote: > On Wed, 10 May 2017, Matti Picus wrote: > >> It seems own tests are taking a very long time on this machine. I have >> explored using jom.exe (https://github.com/qt-labs/jom) instead of nmake to >> allow parallel compilation, which reduces my run of rpython's >> translator\c\test\test_typed.py from ~700 secs to ~600 secs, where on the >> same machine it is under 200 secs with linux. Note that the buildbot >> requires over 2800 secs for the same test (see >> http://buildbot.pypy.org/builders/own-win-x86-32/builds/1395/steps/shell_6/logs/stdio) > > > It is simply so that the machine kindly provided by the folks at Dartmouth > College where I migrated the Windows VM to is way slower than my old box, > and has only 2 physical cores in total... > > Also take a note, that the same machine also does build & test runs for > git-annex, which take about an hour each, so that also doesn't speedup PyPy > builds. > > Anyways, in the mean time, it looks like I've found colo in Germany for my > old machine, but I still need to: > > * Get it shipped to the new datacenter > * Partly replace old hardware > * Install it & set it up anew > > No ETAs, but something is moving in some direction at least. Getting > hardware is for now an unsolved problem though. I need 2 x 2/3 TB SATA > enterprise HDDs and 2 x enterprise SSDs + would be good to replace > unreliable DDR3 RAM sticks... Stuff that I have is 5 years old, and HDDs > tend to fail when they are old :-/ > > > -- > Sincerely yours, > Yury V. Zaytsev > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From armin.rigo at gmail.com Sun May 14 11:25:59 2017 From: armin.rigo at gmail.com (Armin Rigo) Date: Sun, 14 May 2017 17:25:59 +0200 Subject: [pypy-dev] Can't compile C extensions on pypy3 aka alternatives for PyModule_GetState In-Reply-To: References: Message-ID: Hi, On 2 May 2017 at 22:23, Giampaolo Rodola' wrote: > I'm the author of psutil and I received 2 bug reports complaining about lack > of PyModule_GetState: Thanks for passing the complains :-) Implemented PyModule_GetState() in 11e536df3c51. Can you check if that's enough for psutil? Wait for tonight's nightly build at http://buildbot.pypy.org/nightly/py3.5/ (at least 2017-05-15). A bient?t, Armin. From bremner at unb.ca Wed May 17 19:58:45 2017 From: bremner at unb.ca (David Bremner) Date: Wed, 17 May 2017 20:58:45 -0300 Subject: [pypy-dev] PhD Positions in programming language runtimes (Python) Message-ID: <87a86bxcga.fsf@tethera.net> 1 Position I: Python on OMR =========================== The candidate will explore the benefits and challenges of using OMR [1] components (e.g. JIT, GC) in Python [2,3] runtimes. This may involve extending OMR in novel ways to better support Python, or vice-versa. If questions like how to balance fast startup time for scripting applications against good performance for compute-intensive workloads sound interesting, this might an opportunity for you. 2 Position II: Python and PGAS ============================== The candidate will implement support for parallel programming with Python, using the PGAS [4] (parallel global address space) model. The PGAS model can significantly simplify the implementation of parallel algorithms on distributed and shared memory computers. Challenges include enabling parallelism while dealing with Python's Global Interpreter Lock (GIL), and implementing distributed garbage collection efficiently. 3 Qualifications ================ Normally applicants to the Computer Science PhD program at UNB require a Master's degree in computer science or a closely related area. This project will require substantial hands-on and experimental work, and being comfortable with Python and C++. Familiarity with Linux and Open Source Software development would be a plus. For position I, exposure to programming language concepts like GC, JIT, and bytecode would be helpful. For position II, experience with parallel programming frameworks and/or languages such as MPI, OpenMP, X10 is an asset. 4 Environment ============= This project is supported by the IBM Centre for Advanced Studies (Atlantic) at the University of New Brunswick [5]. We have a lively group of students, postdocs, and research assistants working on related problems. Students work closely with both UNB researchers and technical experts from IBM, gaining valuable perspective and experience. 5 Support ========= These positions are "fully funded"; the exact funding package is subject to finalization but will be consistent with other fully funded students in Computer Science at UNB. You can find some information about expenses on the UNB web site [6]. 6 Contact ========= Send your resum? and unofficial transcripts by email to Stephen MacKay Please specify which position(s) you are interested in. [1]: [http://www.eclipse.org/omr/] [2]: [http://www.python.org] [3]: [http://www.pypy.org] [4]: [http://www.pgas.org/] [5]: [http://www.unb.ca/research/casatlantic/] [6]: [http://www.unb.ca/fredericton/studentservices/international/new_international_student/moneymatters.html] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 658 bytes Desc: not available URL: From njs at pobox.com Wed May 24 23:16:03 2017 From: njs at pobox.com (Nathaniel Smith) Date: Wed, 24 May 2017 20:16:03 -0700 Subject: [pypy-dev] [Python-ideas] Exposing CPython's subinterpreter C-API in the stdlib. In-Reply-To: References: Message-ID: CC'ing PyPy-dev... On Wed, May 24, 2017 at 6:01 PM, Eric Snow wrote: > Although I haven't been able to achieve the pace that I originally > wanted, I have been able to work on my multi-core Python idea > little-by-little. Most notably, some of the blockers have been > resolved at the recent PyCon sprints and I'm ready to move onto the > next step: exposing multiple interpreters via a stdlib module. > > Initially I just want to expose basic support via 3 successive > changes. Below I've listed the corresponding (chained) PRs, along > with what they add. Note that the 2 proposed modules take some cues > from the threading module, but don't try to be any sort of > replacement. Threading and subinterpreters are two different features > that are used together rather than as alternatives to one another. > > At the very least I'd like to move forward with the _interpreters > module sooner rather than later. Doing so will facilitate more > extensive testing of subinterpreters, in preparation for further use > of them in the multi-core Python project. We can iterate from there, > but I'd at least like to get the basic functionality landed early. > Any objections to (or feedback about) the low-level _interpreters > module as described? Likewise for the high-level interpreters module? > > Discussion on any expanded functionality for the modules or on the > broader topic of the multi-core project are both welcome, but please > start other threads for those topics. > > -eric > > > basic low-level API: https://github.com/python/cpython/pull/1748 > > _interpreters.create() -> id > _interpreters.destroy(id) > _interpreters.run_string(id, code) > _interpreters.run_string_unrestricted(id, code, ns=None) -> ns > > extra low-level API: https://github.com/python/cpython/pull/1802 > > _interpreters.enumerate() -> [id, ...] > _interpreters.get_current() -> id > _interpreters.get_main() -> id > _interpreters.is_running(id) -> bool > > basic high-level API: https://github.com/python/cpython/pull/1803 > > interpreters.enumerate() -> [Interpreter, ...] > interpreters.get_current() -> Interpreter > interpreters.get_main() -> Interpreter > interpreters.create() -> Interpreter > interpreters.Interpreter(id) > interpreters.Interpreter.is_running() > interpreters.Interpreter.destroy() > interpreters.Interpreter.run(code) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -- Nathaniel J. Smith -- https://vorpus.org From tuom.larsen at gmail.com Fri May 26 13:04:22 2017 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Fri, 26 May 2017 19:04:22 +0200 Subject: [pypy-dev] round Message-ID: Dear list! I noticed that `round` seem to run very slow. Is this a known issue or should I file an bug report? Or am I missing something? from timeit import timeit print timeit('int(sqrt(123.456)+0.5)', 'from math import sqrt') print timeit('int(round(sqrt(123.456)))', 'from math import sqrt') prints: 0.00339508056641 1.18832707405 Thanks! From armin.rigo at gmail.com Fri May 26 16:34:13 2017 From: armin.rigo at gmail.com (Armin Rigo) Date: Fri, 26 May 2017 22:34:13 +0200 Subject: [pypy-dev] round In-Reply-To: References: Message-ID: Hi Tuom, On 26 May 2017 at 19:04, Tuom Larsen wrote: > I noticed that `round` seem to run very slow. Is this a known issue or > should I file an bug report? Or am I missing something? You're perfectly right. Yes, please open a bug report. It looks like an easy fix: "def round_double()" in rpython/rlib/rfloat.py is a copy of CPython's version, but without the fast path for the case of "round(x, 0)", which is the default. A bient?t, Armin. From tuom.larsen at gmail.com Fri May 26 16:53:04 2017 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Fri, 26 May 2017 22:53:04 +0200 Subject: [pypy-dev] round In-Reply-To: References: Message-ID: Created at https://bitbucket.org/pypy/pypy/issues/2563/slow-round Thank you for the explanation! On Fri, May 26, 2017 at 10:34 PM, Armin Rigo wrote: > Hi Tuom, > > On 26 May 2017 at 19:04, Tuom Larsen wrote: >> I noticed that `round` seem to run very slow. Is this a known issue or >> should I file an bug report? Or am I missing something? > > You're perfectly right. Yes, please open a bug report. It looks like > an easy fix: "def round_double()" in rpython/rlib/rfloat.py is a copy > of CPython's version, but without the fast path for the case of > "round(x, 0)", which is the default. > > > A bient?t, > > Armin. From phyo.arkarlwin at gmail.com Tue May 30 03:47:41 2017 From: phyo.arkarlwin at gmail.com (Phyo Arkar) Date: Tue, 30 May 2017 07:47:41 +0000 Subject: [pypy-dev] Suggestion : Build Using PyPy page to showcase. Message-ID: We had launched our chat-room for medical consultation , in our country. It is reaching 5000 users in first month and growing fast with 400 concurrent user max , daily. We need a page to showcase projects running 100% on pypy runtime! Regards Phyo. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuom.larsen at gmail.com Wed May 31 11:11:22 2017 From: tuom.larsen at gmail.com (Tuom Larsen) Date: Wed, 31 May 2017 17:11:22 +0200 Subject: [pypy-dev] Integer division Message-ID: Dear list! I noticed that integer division (`//`) is slower than floating-point division and integer cast. For example: from __future__ import division from time import time t = time() for i in range(1, 10000): for j in range(1, 10000): # k = i//j # 2.12 seconds k = int(i/j) # 0.98 seconds print time() - t I now integer division should be slower, but I thought that the `int()` would make floating division even slower. Please, can someone explain what is going on? Is this expected behaviour? Thank you!