From pypy at pocketnix.org Sun May 1 00:07:35 2011 From: pypy at pocketnix.org (pypy at pocketnix.org) Date: Sat, 30 Apr 2011 22:07:35 +0000 Subject: [pypy-dev] PyPy and memory In-Reply-To: References: Message-ID: <20110430220735.GA24306@pocketnix.org> On Sat, Apr 30, 2011 at 05:38:19PM +0200, Massimo Sala wrote: > What about PyPy vs plain CPython ? > > Please see : > http://pushingtheweb.com/2010/06/python-and-tcmalloc/ > > Is it possible for the maintainers to provide > - PyPy > - PyPy with tcmalloc > > so end-users can > - give a try > - do some tests in their different configs > - provide a feedback here on the mailing list > ? if i remember correctly the memory fragmentation was noted by the dropbox guys in http://pycon.blip.tv/file/4878722/ if i also remember correctly one of the last slides is a shout out to pypy indicating it doesn't have this problem if the video is incorrect let me know and i will dig through my archives for the correct video may have a quick peek at this anyway From rogerb at rogerbinns.com Sun May 1 11:14:35 2011 From: rogerb at rogerbinns.com (Roger Binns) Date: Sun, 01 May 2011 02:14:35 -0700 Subject: [pypy-dev] cpyext: Detecting pypy and other issues Message-ID: <4DBD247B.7040102@rogerbinns.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm trying to port my APSW C extension to pypy and am encountering various issues. APSW is somewhat similar to pysqlite in that it wraps SQLite, but APSW wraps all of SQLite and provides SQLite's semantics rather than whatever DBAPI says. APSW supports Python 2.3 through 3.2. What is the correct way to detect pypy compilation? My code includes various enhancements and optimisations but they depend on CPython to some degree. These would need to be disabled at compile time. It looks like PYPY_VERSION being #defined will work but I'd rather use the "official" way whatever that is. One example optimisation I do is object reuse. When the refcount is 1 and it is dealloc then they go to a recycle list. Doing this gave several percentage point performance improvements. This can be disabled with a #define. An example of enhanced functionality is an alternative to PyErr_WriteUnraiseable. The CPython implementation is virtually useless (eg no traceback). My version shows the traceback but again depends on CPython internals to get it. http://code.google.com/p/apsw/source/browse/src/util.c#89 I need the api PyRun_StringFlags as it is used to execute a Python string and attach the results to the module. It is called like this: PyRun_StringFlags(char*, Py_file_input, apswdict, apswdict, NULL); I do have a benchmarking tool that mainly exposes the performance improvement of APSW over pysqlite. Most of the work of the wrapper is taking Python types (especially strings) and providing them to the SQLite APIs, and getting data back from SQLite and converting back to Python types. There is a lot of GIL acquisition and release. In order to improve performance there are also various string related caches. My test suite has 99,6% coverage of the C code so I should be able to find all the places where pypy differs. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk29JHMACgkQmOOfHg372QSTSQCgu3mi8osgcXgpcQ0SpMTJvnVu 7BQAoIxIZsaeq29QCqrJdFRarnvpjay0 =BZfM -----END PGP SIGNATURE----- From arigo at tunes.org Sun May 1 11:36:34 2011 From: arigo at tunes.org (Armin Rigo) Date: Sun, 1 May 2011 11:36:34 +0200 Subject: [pypy-dev] virtualenv 1.6.1 now released Message-ID: Hi, If you are using virtualenv, note that PyPy 1.5 requires virtualenv 1.6.1. It has been released a few hours *after* PyPy 1.5. For those who did not find it, it's there now :-) Armin. From rogerb at rogerbinns.com Sun May 1 12:08:12 2011 From: rogerb at rogerbinns.com (Roger Binns) Date: Sun, 01 May 2011 03:08:12 -0700 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: <4DBD247B.7040102@rogerbinns.com> References: <4DBD247B.7040102@rogerbinns.com> Message-ID: <4DBD310C.9090806@rogerbinns.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The website is totally silent on how to debug C extension issues. I'm encountering an issue where calling PyObject_GetAttrString results in pypy getting SIGSEGV. The first parameter is the module (return of Py_InitModule3) stored in a static variable. If I add an extra Py_INCREF to init then that goes away. It is behaving as though modules are created with a ref count of zero. Do I have to add an incref for everything that has a static pointer to it? My next problem is a list allocated with PyList_New(0). I later call PyList_GET_SIZE on it and get a SIGSEGV. I put an assertion before that of PyList_Check but that crashes too. Unfortunately I can't really do much with the pypy/gdb output: E RPython traceback: File "implement_1.c", line 52621, in PyList_Check File "implement_5.c", line 14533, in CpyTypedescr_realize Program received signal SIGSEGV, Segmentation fault. 0x000000000041118e in ?? () (gdb) bt #0 0x000000000041118e in ?? () #1 0x00000000004ab890 in ?? () #2 0x0000000000f47bfb in ?? () #3 0x000000000043f31f in PyList_Check () #4 0x00007ffff5410795 in Connection_close_internal (self=0x7ffff7f91030, force=1) at src/connection.c:166 #5 0x00007ffff5410b6d in Connection_close (self=0x7ffff7f91030, args=0x7ffff7f8ee60) at src/connection.c:274 #6 0x0000000000f47bfb in ?? () #7 0x0000000000729236 in ?? () ... #154 0x00007ffff5cfbeff in __libc_start_main (main=0x4118b0, argc=2, ubp_av=0x7fffffffe108, init=, fini=, rtld_fini=, stack_end=0x7fffffffe0f8) at libc-start.c:226 ... Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk29MQcACgkQmOOfHg372QRBUgCfVaXt6c8S7CKkWlmSyzOUknIV FVMAniyQCLw7DWaElMF0AYiabbXqXsyV =1Ecf -----END PGP SIGNATURE----- From holger at merlinux.eu Sun May 1 12:20:14 2011 From: holger at merlinux.eu (holger krekel) Date: Sun, 1 May 2011 10:20:14 +0000 Subject: [pypy-dev] virtualenv 1.6.1 now released In-Reply-To: References: Message-ID: <20110501102014.GA4071@merlinux.eu> Hi Armin, all (also CC Jannis Leidel), On Sun, May 01, 2011 at 11:36 +0200, Armin Rigo wrote: > Hi, > > If you are using virtualenv, note that PyPy 1.5 requires virtualenv > 1.6.1. It has been released a few hours *after* PyPy 1.5. For those > who did not find it, it's there now :-) I still encounter a problem with virtualenv/pypy, see transcript on tannit below. it's basically a recursive use of virtualenv which however works nicely with CPython. best, holger (0)hpk at tannit:~$ virtualenv p0 --no-site-packages New python executable in p0/bin/python Installing distribute.................................................................................................................................................................................done. (0)hpk at tannit:~$ p0/bin/pip install virtualenv Downloading/unpacking virtualenv Using download cache from /home/hpk/.tox/downloads/http%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fv%2Fvirtualenv%2Fvirtualenv-1.6.1.tar.gz Running setup.py egg_info for package virtualenv warning: no previously-included files matching '*.*' found under directory 'docs/_templates' Installing collected packages: virtualenv Running setup.py install for virtualenv warning: no previously-included files matching '*.*' found under directory 'docs/_templates' Installing virtualenv script to /home/hpk/p0/bin Successfully installed virtualenv Cleaning up... (0)hpk at tannit:~$ p0/bin/virtualenv p1 # cpython works fine Using real prefix '/usr' New python executable in p1/bin/python Overwriting p1/lib/python2.6/distutils/__init__.py with new content Installing setuptools............done. Installing pip...............done. (0)hpk at tannit:~$ p0/bin/virtualenv -p pypy p1 # this is pypy-1.5 and it fails Running virtualenv with interpreter /usr/local/bin/pypy New pypy executable in p1/bin/pypy Installing setuptools........ Complete output from command /home/hpk/p1/bin/pypy -c "#!python \"\"\"Bootstra...sys.argv[1:]) " --always-copy -U setuptools: Traceback (most recent call last): File "app_main.py", line 53, in run_toplevel File "app_main.py", line 526, in run_it File "", line 67, in ImportError: No module named md5 ---------------------------------------- ...Installing setuptools...done. Traceback (most recent call last): File "app_main.py", line 53, in run_toplevel File "/home/hpk/p0/lib/python2.6/site-packages/virtualenv.py", line 1965, in main() File "/home/hpk/p0/lib/python2.6/site-packages/virtualenv.py", line 795, in main never_download=options.never_download) File "/home/hpk/p0/lib/python2.6/site-packages/virtualenv.py", line 895, in create_environment search_dirs=search_dirs, never_download=never_download) File "/home/hpk/p0/lib/python2.6/site-packages/virtualenv.py", line 588, in install_setuptools search_dirs=search_dirs, never_download=never_download) File "/home/hpk/p0/lib/python2.6/site-packages/virtualenv.py", line 562, in _install_req cwd=cwd) File "/home/hpk/p0/lib/python2.6/site-packages/virtualenv.py", line 863, in call_subprocess % (cmd_desc, proc.returncode)) OSError: Command /home/hpk/p1/bin/pypy -c "#!python \"\"\"Bootstra...sys.argv[1:]) " --always-copy -U setuptools failed with error code 1 (0)hpk at tannit:~$ From arigo at tunes.org Sun May 1 16:35:35 2011 From: arigo at tunes.org (Armin Rigo) Date: Sun, 1 May 2011 16:35:35 +0200 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: <4DBD310C.9090806@rogerbinns.com> References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> Message-ID: Hi Roger, A first warning: even if you manage to compile your C extension with PyPy, the interfacing is much slower than with CPython. Keep that in mind, given that according to what you write you care enough about performance to try to squeeze a few percents. On Sun, May 1, 2011 at 12:08 PM, Roger Binns wrote: > It looks like PYPY_VERSION being #defined will work > but I'd rather use the "official" way whatever that is. That's the "official" way. > I need the api PyRun_StringFlags as it is used to execute a Python string > and attach the results to the module. If it's not already provided, then you are welcome to contribute it to PyPy in pypy/module/cpyext/. It is known that we don't have 100% coverage even of Python.h (not to mention other headers like compile.h, which will probably never be supported). > Do I have to add an incref for everything that has a static pointer to it? It depends on details of exactly what you do, but it is likely. The exact rule is, like CPython, that any object is freed as soon as its reference counter goes down to zero; but unlike CPython, this does *not* include places like a reference stored in the module's dictionary. In other words, if it seems to work on CPython but not on PyPy, then it is likely that there is actually a (potentially very rare) bug in the CPython version too. For example, the object you have in the static variable could be meant to be kept alive by being also stored in the module's dictionary. If so, what if a user actually delete it from your module's dictionary? Then you get a crash on CPython too. > My next problem is a list allocated with PyList_New(0). ?I later call PyList_GET_SIZE on it > and get a SIGSEGV. I put an assertion before that of PyList_Check but that crashes too. >?Unfortunately I can't really do much with the pypy/gdb output: You get easier-to-read tracebacks if you run it without the JIT: "pypy --jit threshold=-1". Of course in order to get the details of, say, the local variables, then you need a custom pypy built with debugging symbols. A bient?t, Armin. From arigo at tunes.org Sun May 1 17:22:06 2011 From: arigo at tunes.org (Armin Rigo) Date: Sun, 1 May 2011 17:22:06 +0200 Subject: [pypy-dev] virtualenv 1.6.1 now released In-Reply-To: <20110501102014.GA4071@merlinux.eu> References: <20110501102014.GA4071@merlinux.eu> Message-ID: Hi Holger, On Sun, May 1, 2011 at 12:20 PM, holger krekel wrote: > I still encounter a problem with virtualenv/pypy, see transcript on > tannit below. I don't know virtualenv, but I tried to repeat the commands you typed (with virtualenv 1.6.1 and pypy 1.5 as released, on tannit 64-bit), and it works for me. I can only guess that you have some older virtualenv or some older pypy around. Armin From arigo at tunes.org Sun May 1 17:35:39 2011 From: arigo at tunes.org (Armin Rigo) Date: Sun, 1 May 2011 17:35:39 +0200 Subject: [pypy-dev] virtualenv 1.6.1 now released In-Reply-To: References: <20110501102014.GA4071@merlinux.eu> Message-ID: Hi all, On Sun, May 1, 2011 at 5:22 PM, Armin Rigo wrote: > I can only guess that you have some older virtualenv or some older pypy around. For reference, I just installed now the pypy 1.5 release as the default pypy on tannit and tannit32. A bient?t, Armin. From holger at merlinux.eu Sun May 1 20:07:54 2011 From: holger at merlinux.eu (holger krekel) Date: Sun, 1 May 2011 18:07:54 +0000 Subject: [pypy-dev] virtualenv 1.6.1 now released In-Reply-To: References: <20110501102014.GA4071@merlinux.eu> Message-ID: <20110501180754.GC4071@merlinux.eu> On Sun, May 01, 2011 at 17:22 +0200, Armin Rigo wrote: > Hi Holger, > > On Sun, May 1, 2011 at 12:20 PM, holger krekel wrote: > > I still encounter a problem with virtualenv/pypy, see transcript on > > tannit below. > > I don't know virtualenv, but I tried to repeat the commands you typed > (with virtualenv 1.6.1 and pypy 1.5 as released, on tannit 64-bit), > and it works for me. I can only guess that you have some older > virtualenv or some older pypy around. i use pypy 1.5 from opt. However, /usr/bin/virtualenv on tannit is at version 1.4.5 and maybe that is the problem. sidenote: the released pypy-1.5 shows this: (0)hpk at tannit:~$ /opt/pypy-1.5-linux64/bin/pypy Python 2.7.1 (b590cf6de419, Apr 30 2011, 02:00:34) [PyPy 1.5.0-alpha0 with GCC 4.4.3] on linux2 so the version string is still at "alpha0" FWIW. holger From arigo at tunes.org Sun May 1 20:17:13 2011 From: arigo at tunes.org (Armin Rigo) Date: Sun, 1 May 2011 20:17:13 +0200 Subject: [pypy-dev] virtualenv 1.6.1 now released In-Reply-To: <20110501180754.GC4071@merlinux.eu> References: <20110501102014.GA4071@merlinux.eu> <20110501180754.GC4071@merlinux.eu> Message-ID: Hi Holger, On Sun, May 1, 2011 at 8:07 PM, holger krekel wrote: > i use pypy 1.5 from opt. ?However, /usr/bin/virtualenv on tannit is at > version 1.4.5 and maybe that is the problem. I assumed that you got virtualenv 1.6.1, because you replied to my message that was saying precisely "virtualenv 1.6.1 is needed, use that version". :-) I will update it on tannit too. Armin From holger at merlinux.eu Sun May 1 20:39:14 2011 From: holger at merlinux.eu (holger krekel) Date: Sun, 1 May 2011 18:39:14 +0000 Subject: [pypy-dev] virtualenv 1.6.1 now released In-Reply-To: References: <20110501102014.GA4071@merlinux.eu> <20110501180754.GC4071@merlinux.eu> Message-ID: <20110501183914.GE4071@merlinux.eu> On Sun, May 01, 2011 at 20:17 +0200, Armin Rigo wrote: > Hi Holger, > > On Sun, May 1, 2011 at 8:07 PM, holger krekel wrote: > > i use pypy 1.5 from opt. ?However, /usr/bin/virtualenv on tannit is at > > version 1.4.5 and maybe that is the problem. > > I assumed that you got virtualenv 1.6.1, because you replied to my > message that was saying precisely "virtualenv 1.6.1 is needed, use > that version". :-) there were two virtualenv invocations. The second one was with 1.6.1 and pypy but the first one was with 1.4.5 and cpython. Maybe that kind of nesting is just not supported across venv and python versions. holger > I will update it on tannit too. > > Armin > From rogerb at rogerbinns.com Mon May 2 06:55:50 2011 From: rogerb at rogerbinns.com (Roger Binns) Date: Sun, 01 May 2011 21:55:50 -0700 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> Message-ID: <4DBE3956.90409@rogerbinns.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/01/2011 07:35 AM, Armin Rigo wrote: > A first warning: even if you manage to compile your C extension with > PyPy, the interfacing is much slower than with CPython. Keep that in > mind, given that according to what you write you care enough about > performance to try to squeeze a few percents. First you make it work right then you make it work fast :-) Even if it is 10% slower than CPython the rest of pypy will be more than 10% faster so things will still be better overall. Worst case some combination of my code, my tests and pypy will be improved by trying to do the port. >> I need the api PyRun_StringFlags as it is used to execute a Python string >> and attach the results to the module. > > If it's not already provided, then you are welcome to contribute it to > PyPy in pypy/module/cpyext/. It is known that we don't have 100% > coverage even of Python.h (not to mention other headers like > compile.h, which will probably never be supported). It would be helpful if the cpyext site said what to do in these situations. For example add missing functions to a wiki page or create a ticket or send to mailing list etc. ie what actions should developers like me take when discovering things like this that ensures the smoothest running of the project. (Obviously just fixing it is good but we aren't all in a position to do that at the moment.) PyErr_Display is another missing function which is widely used, but never documented in the Python docs for some reason. The pypy site also doesn't say where to communicate for cpyext - I assumed this mailing list was the right place. >> Do I have to add an incref for everything that has a static pointer to it? > > It depends on details of exactly what you do, but it is likely. The > exact rule is, like CPython, that any object is freed as soon as its > reference counter goes down to zero; but unlike CPython, this does > *not* include places like a reference stored in the module's > dictionary. Note my problem is not with contents of the module, but with the module itself. static PyObject *themodule=Py_InitModule3(...); Later in one of the methods called back from Python I call PyObject_AttrString(themodule, "foo") which is crashing. If I incref themodule in init then this problem doesn't occur. The behaviour is as though Py_InitModule3 is setting the starting refcount to zero, or with the module's refcount becoming zero inside pypy which then begs the question of how my callback could have been called. > In other words, if it seems to work on CPython but not on > PyPy, then it is likely that there is actually a (potentially very > rare) bug in the CPython version too. For my tests I build a full debug version of CPython will all memory optimisations disabled (quite a feat since it squirrels away stuff all over the place) and then run everything under valgrind. Combined with 100% coverage in almost all source files I'm reasonably confident that things are coded well, although there may be some luck :-) > For example, the object you > have in the static variable could be meant to be kept alive by being > also stored in the module's dictionary. I did just double check my code and every item that I do PyModule_AddObject with is also incref first. That was some diligent coding however many years ago when I wrote it :-) > You get easier-to-read tracebacks if you run it without the JIT: "pypy > --jit threshold=-1". Of course in order to get the details of, say, > the local variables, then you need a custom pypy built with debugging > symbols. Unfortunately disabling the JIT wasn't much help. My goal is to produce the best bug report possible and use the least amount of the pypy team's time. (It is a lot easier to fix and understand things starting from a good bug report.) What would be most helpful is if the cpyext page gave some instructions to follow in particular making an appropriate pypy and using valgrind. (Usually making valgrind work well requires extra options above and beyond regular debugging builds.) Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk2+OVEACgkQmOOfHg372QTkpQCfc+DL/GFDZCpyMsNEsx5tRhvY fIUAn0z1EQ+NxZLQ8PwaouXXSRy4JCTo =vu+1 -----END PGP SIGNATURE----- From anto.cuni at gmail.com Mon May 2 08:30:18 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Mon, 02 May 2011 08:30:18 +0200 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: <4DBE3956.90409@rogerbinns.com> References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> Message-ID: <4DBE4F7A.3030109@gmail.com> Hello Roger, I'm not "the" cpyext expert here, so I'll let the others to answer your specific questions. However: On 02/05/11 06:55, Roger Binns wrote: > Unfortunately disabling the JIT wasn't much help. My goal is to produce the > best bug report possible and use the least amount of the pypy team's time. > (It is a lot easier to fix and understand things starting from a good bug > report.) What would be most helpful is if the cpyext page gave some > instructions to follow in particular making an appropriate pypy and using > valgrind. (Usually making valgrind work well requires extra options above > and beyond regular debugging builds.) you are right, we lack such a document. Do you feel like writing it? :-) (seriously: you are by far in a better position than us for knowing what could or could not help a newcomer. Even if you don't know the answer of all the questions, having a document with XXX that we can fix would be valuable). ciao, Anto From arigo at tunes.org Mon May 2 12:12:07 2011 From: arigo at tunes.org (Armin Rigo) Date: Mon, 2 May 2011 12:12:07 +0200 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: <4DBE3956.90409@rogerbinns.com> References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> Message-ID: Hi Roger, On Mon, May 2, 2011 at 6:55 AM, Roger Binns wrote: > The pypy site also doesn't say where to communicate for cpyext - I assumed > this mailing list was the right place. Yes. > ?static PyObject *themodule=Py_InitModule3(...); > > Later in one of the methods called back from Python I call > PyObject_AttrString(themodule, "foo") which is crashing. ?If I incref > themodule in init then this problem doesn't occur. Indeed, it's an edge case. What you must realize is that your module object is of course kept alive in PyPy, even if its refcount drops to zero. More precisely, in PyPy, every object accessed by the C extensions is actually accessed via a "wrapper", which is declared as PyObject. A wrapper holds just the refcount and a reference to the real object, which is not a PyObject at all (it's an object that can be moved by our GC, to start with). When the refcount drops to zero we assume that the C extension doesn't need the object any more, and we free the PyObject wrapper. This may or may not cause the real object within PyPy to be freed. It may even be the case that later we want again to pass a reference to the same object to the C extension; in this case, we make a new wrapper. So the refcount of -- in your case -- the module object is never zero; it's just that as soon as the main function of your C extension module finishes, the refcount is decremented and reaches zero, so the PyObject wrapper is freed. It is arguably a bug in your module, even though in that case it is hard to write an example that triggers the bug, because it is impossible in CPython to free a loaded extension module. The point is that if it is was possible, then doing so in CPython would cause the module to see its refcount drop to zero, and be freed, which makes the pointer in the static global variable invalid. That's what I meant by saying that such issues are, somehow, showing bugs in the C extension module, although admittedly they are often "bugs" that cannot possibly cause an issue in practice. A bient?t, Armin. From dimaqq at gmail.com Mon May 2 20:14:45 2011 From: dimaqq at gmail.com (Dima Tisnek) Date: Mon, 2 May 2011 11:14:45 -0700 Subject: [pypy-dev] cpyext reference counting and other gc's In-Reply-To: References: Message-ID: Sure, I started with a basic concept of having as few dependencies in pypy rpython code as possible, that is pypy sees object defined/created in c extension as native. Granted, I didn't check all the details yet, e.g. cpython protocols probably need to be wrapped in cpyext anyway. Dunno if possible in general case, I still need to investigate the hack I'm thinking of here. I looked through code for quite a few cpython c extensions, most make only minimal use of cpython api. Moreover it occurs to me it is important to support sheer number of simple cpython extensions (rationale is that python is commonly used to glue large-ish software modules together). The few extensions that do something non-trivial would probably need to be patched to take full advantage of pypy anyway. The consequence is Py* "symbols" (C source uses those symbolically, we are free to inject macros or define functions) can become quite complex functions. If c extension writer assumes that incref is a very simple operation while repeatedly calling a user-defined function is complex and overoptimizes their extension with this view, the results could be funny :P Please correct me if I misunderstood cpyext, I'd be willing to write up cpyext on pypy wiki. Current cpyext creates a shadow object for every pointer that is passed to or from C and keeps this object for every pointer that is used in C internally, lifetime seen through object's reference count. My concept is to shadow only the reference count. When an object crosses pypy-C boundary, current cpyext has to perform an allocation or lookup anyway, it is rather similar with my proposal, perhaps my proposal is even cheaper here. As far as incref/decref is concerned, only first lookup is potentially slow, subsequent lookups are in cpu cache anyway, moreover if there's significant traffic to the reference count dict, whole dict is probably cached. Another possible advantage is unloading C extension modules as we can attribute privately stored python objects to the module that holds them; although as this is not done in cpython, most modules are probably broken. Cheers, Dima Tisnek p.s. I'd really like to know what are reasons some c extension do not work with cpyext right now. can anyone weigh in on dict/custom data structure lookup, e.g. splay tree vs pointer-chasing linked shadow objects? is there a better data structure than a has table (void* -> ssize_t) On 26 April 2011 09:23, Armin Rigo wrote: > Hi Dima, > > On Mon, Apr 25, 2011 at 9:53 PM, Dima Tisnek wrote: >> https://docs.google.com/document/d/1k7t-WIsfKW4tIL9i8-7Y6_9lo18wcsibyDONOF2i_l8/edit?hl=en > > Can you explain a bit more what are the advantages of the solution you > propose, compared to what is already implemented in cpyext? ?Your > description is far too high-level for us to know exactly what you > mean. > > It seems that you want to replace the currently implemented solution > with a very different one. ?I can explain it in a bit more details, > but I would first like to hear what goal you are trying to achieve. > Here is a quick reply based on guessing. ?The issue with your version > is that Py_INCREF() and Py_DECREF() needs to do a slow dictionary > lookup, while ours doesn't. ?Conversely, I believe that your version > doesn't need a dictionary lookup in other cases where ours needs to. > However it seems to me that if you add so much overhead to Py_INCREF() > and Py_DECREF(), you loose all other speed advantages. > > > A bient?t, > > Armin. > From amauryfa at gmail.com Mon May 2 20:44:12 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Mon, 2 May 2011 20:44:12 +0200 Subject: [pypy-dev] cpyext reference counting and other gc's In-Reply-To: References: Message-ID: Hi, 2011/5/2 Dima Tisnek : > I'd really like to know what are reasons some c extension do not work > with cpyext right now. I can see three kinds of reasons: - Reference count mistakes, that normally don't show up in CPython (but many c extensions do crash when you delete them from sys.modules, then reimport them; cpyext just detect the failure the first time :-) - unsupported functions, either because we never encountered them, or because they are really hard to support in pypy - extensions that play too much with CPython inter nals: numpy, Cython are in this case > can anyone weigh in on dict/custom data structure lookup, e.g. splay > tree vs pointer-chasing linked shadow objects? > is there a better data structure than a has table (void* -> ssize_t) Probably! The hash table is used because it's readily available in RPython. But if you care to provide a RPython implementation of the associative container, I'd be happy to test it. -- Amaury Forgeot d'Arc From dimaqq at gmail.com Mon May 2 20:53:11 2011 From: dimaqq at gmail.com (Dima Tisnek) Date: Mon, 2 May 2011 11:53:11 -0700 Subject: [pypy-dev] cpyext reference counting and other gc's In-Reply-To: References: Message-ID: Hi Amaury, thanks for a quick reply, btw, which api functions are hard to support in pypy or why some are? d. On 2 May 2011 11:44, Amaury Forgeot d'Arc wrote: > Hi, > > 2011/5/2 Dima Tisnek : >> I'd really like to know what are reasons some c extension do not work >> with cpyext right now. > > I can see three kinds of reasons: > > - Reference count mistakes, that normally don't show up in CPython > (but many c extensions do crash when you delete them from sys.modules, > then reimport them; cpyext just detect the failure the first time :-) > > - unsupported functions, either because we never encountered them, > or because they are really hard to support in pypy > > - extensions that play too much with CPython inter > nals: numpy, Cython > are in this case > >> can anyone weigh in on dict/custom data structure lookup, e.g. splay >> tree vs pointer-chasing linked shadow objects? >> is there a better data structure than a has table (void* -> ssize_t) > > Probably! The hash table is used because it's readily available in > RPython. But if you care to provide a RPython implementation of > the associative container, I'd be happy to test it. > > -- > Amaury Forgeot d'Arc > From amauryfa at gmail.com Mon May 2 21:09:31 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Mon, 2 May 2011 21:09:31 +0200 Subject: [pypy-dev] cpyext reference counting and other gc's In-Reply-To: References: Message-ID: 2011/5/2 Dima Tisnek : > Hi Amaury, thanks for a quick reply, btw, which api functions are hard > to support in pypy or why some are? Fortunately, there are not so many of them: - PyFile_AsFile(), PyFile_FromFile, because files opened by pypy don't use a FILE* (like python3) - PyThreadState creation and suppression - PyInterpreter creation and suppression - Py_Initialize and Py_Finalize (to embed a python interpreter inside an application) - Some trace and Traceback management functions that are not even documented (but used by Cython :-)) -- Amaury Forgeot d'Arc From rogerb at rogerbinns.com Mon May 2 22:36:48 2011 From: rogerb at rogerbinns.com (Roger Binns) Date: Mon, 02 May 2011 13:36:48 -0700 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: <4DBE4F7A.3030109@gmail.com> References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> <4DBE4F7A.3030109@gmail.com> Message-ID: <4DBF15E0.6080005@rogerbinns.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/01/2011 11:30 PM, Antonio Cuni wrote: > you are right, we lack such a document. Do you feel like writing it? :-) I'll make an initial stab at it this week. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk2/FdwACgkQmOOfHg372QSeZQCeJ7P/5VsMxno6j2eunziJtzAC jHIAoIpPgm4jPHOERSKq4OVJLt/lv+MA =uSH6 -----END PGP SIGNATURE----- From rogerb at rogerbinns.com Mon May 2 23:04:48 2011 From: rogerb at rogerbinns.com (Roger Binns) Date: Mon, 02 May 2011 14:04:48 -0700 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> Message-ID: <4DBF1C70.6040907@rogerbinns.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/02/2011 03:12 AM, Armin Rigo wrote: > It is arguably a bug in your module, The rule seems to be that if the C code saves the value of *any* PyObject* in a static variable then an incref must be done first. I have actually been doing this for everything (including PyTypeObject) except for the module object itself. > The point is that if it is was possible, then doing so in > CPython would cause the module to see its refcount drop to zero, and > be freed, which makes the pointer in the static global variable > invalid. This wouldn't have affected me. There is no way you could have called a function provided by the module if the module itself had been freed. My other issue is happening where the object struct has a PyObject * that is allocated with PyList_New(0) in the _new method. Later on PyList_ methods crash. The debugger is showing a refcount of 140737353674481 so something has gone horribly wrong ... Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk2/HGwACgkQmOOfHg372QT9qgCfWvr9rjt0jsV0r/2wnWk5Tq3Q b+UAniNU0scFslq1TqQeowa/XZYICxh3 =HyOO -----END PGP SIGNATURE----- From rogerb at rogerbinns.com Mon May 2 23:21:46 2011 From: rogerb at rogerbinns.com (Roger Binns) Date: Mon, 02 May 2011 14:21:46 -0700 Subject: [pypy-dev] cpyext reference counting and other gc's In-Reply-To: References: Message-ID: <4DBF206A.70807@rogerbinns.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > - Some trace and Traceback management functions that are not even > documented (but used by Cython :-)) I use them too. One of the problems with the standard documented C api is that tracebacks do not include any methods implemented in C. For example if a Python method calls a C method which calls a Python method which errors then the traceback won't include the C code. This is very confusing if you don't know why the C code called a Python method. I show this in the doc for my project: http://apidoc.apsw.googlecode.com/hg/exceptions.html#augmented-stack-traces As the doc shows I end up adding in synthetic stack frames so you can clearly see the C code. I also augment the synthetic frames with local variables that can be introspected to find out what is going on. The very bottom of the page shows what a difference that makes. You can see the C code to do it here: http://code.google.com/p/apsw/source/browse/src/traceback.c Note use of functions like PyThreadState_Get() and PyTraceback_here. As long as the signature of AddTraceBackHere can remain the same then I don't care what the body inside is for pypy. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk2/IGYACgkQmOOfHg372QRzAgCglUdQlOoF3fQBJAAQ7Lf8vRs2 4OMAn1weWlNDNtHHcccB1vjxSvn7wk59 =jTAS -----END PGP SIGNATURE----- From amauryfa at gmail.com Tue May 3 00:08:42 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 3 May 2011 00:08:42 +0200 Subject: [pypy-dev] cpyext reference counting and other gc's In-Reply-To: <4DBF206A.70807@rogerbinns.com> References: <4DBF206A.70807@rogerbinns.com> Message-ID: Hi, 2011/5/2 Roger Binns : > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > >> - Some trace and Traceback management functions that are not even >> documented (but used by Cython :-)) > > I use them too. ?One of the problems with the standard documented C api is > that tracebacks do not include any methods implemented in C. ?For example if > a Python method calls a C method which calls a Python method which errors > then the traceback won't include the C code. ?This is very confusing if you > don't know why the C code called a Python method. > > I show this in the doc for my project: > > ?http://apidoc.apsw.googlecode.com/hg/exceptions.html#augmented-stack-traces > > As the doc shows I end up adding in synthetic stack frames so you can > clearly see the C code. ?I also augment the synthetic frames with local > variables that can be introspected to find out what is going on. ?The very > bottom of the page shows what a difference that makes. > > You can see the C code to do it here: > > ?http://code.google.com/p/apsw/source/browse/src/traceback.c > > Note use of functions like PyThreadState_Get() and PyTraceback_here. ?As > long as the signature of AddTraceBackHere can remain the same then I don't > care what the body inside is for pypy. Yes, we've implemented PyTraceback_here so that it works exactly for this usage. Can you check whether pypy does the right thing for you as well? -- Amaury Forgeot d'Arc From amauryfa at gmail.com Tue May 3 00:08:42 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 3 May 2011 00:08:42 +0200 Subject: [pypy-dev] cpyext reference counting and other gc's In-Reply-To: <4DBF206A.70807@rogerbinns.com> References: <4DBF206A.70807@rogerbinns.com> Message-ID: Hi, 2011/5/2 Roger Binns : > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > >> - Some trace and Traceback management functions that are not even >> documented (but used by Cython :-)) > > I use them too. ?One of the problems with the standard documented C api is > that tracebacks do not include any methods implemented in C. ?For example if > a Python method calls a C method which calls a Python method which errors > then the traceback won't include the C code. ?This is very confusing if you > don't know why the C code called a Python method. > > I show this in the doc for my project: > > ?http://apidoc.apsw.googlecode.com/hg/exceptions.html#augmented-stack-traces > > As the doc shows I end up adding in synthetic stack frames so you can > clearly see the C code. ?I also augment the synthetic frames with local > variables that can be introspected to find out what is going on. ?The very > bottom of the page shows what a difference that makes. > > You can see the C code to do it here: > > ?http://code.google.com/p/apsw/source/browse/src/traceback.c > > Note use of functions like PyThreadState_Get() and PyTraceback_here. ?As > long as the signature of AddTraceBackHere can remain the same then I don't > care what the body inside is for pypy. Yes, we've implemented PyTraceback_here so that it works exactly for this usage. Can you check whether pypy does the right thing for you as well? -- Amaury Forgeot d'Arc From amauryfa at gmail.com Tue May 3 00:12:44 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 3 May 2011 00:12:44 +0200 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: <4DBF1C70.6040907@rogerbinns.com> References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> <4DBF1C70.6040907@rogerbinns.com> Message-ID: 2011/5/2 Roger Binns : > There is no way you could have called a > function provided by the module if the module itself had been freed. Are you sure? from mymodule import func del sys.modules['mymodule'] func() -- Amaury Forgeot d'Arc From dmalcolm at redhat.com Mon May 2 23:49:09 2011 From: dmalcolm at redhat.com (David Malcolm) Date: Mon, 02 May 2011 17:49:09 -0400 Subject: [pypy-dev] PyPy 1.5 released In-Reply-To: References: Message-ID: <1304372949.2063.5.camel@radiator.bos.redhat.com> On Sat, 2011-04-30 at 17:04 +0200, Armin Rigo wrote: > ====================== > PyPy 1.5: Catching Up > ====================== > > We're pleased to announce the 1.5 release of PyPy. This release updates > PyPy with the features of CPython 2.7.1, including the standard library. Thus > all the features of `CPython 2.6`_ and `CPython 2.7`_ are now supported. It > also contains additional performance improvements. You can download it here: > > http://pypy.org/download.html FWIW, I've now updated Fedora's RPM packages of PyPy to pypy-1.5. See https://bugzilla.redhat.com/show_bug.cgi?id=701121 for links to the builds for various releases/architectures. Enjoy! Dave From exarkun at twistedmatrix.com Tue May 3 00:51:26 2011 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Mon, 02 May 2011 22:51:26 -0000 Subject: [pypy-dev] cpyext reference counting and other gc's In-Reply-To: References: Message-ID: <20110502225126.1992.2104890452.divmod.xquotient.1011@localhost.localdomain> On 07:09 pm, amauryfa at gmail.com wrote: >2011/5/2 Dima Tisnek : >>Hi Amaury, thanks for a quick reply, btw, which api functions are hard >>to support in pypy or why some are? > >Fortunately, there are not so many of them: >- PyFile_AsFile(), PyFile_FromFile, because files opened by pypy don't >use a FILE* (like python3) Do fdopen(3) and fileno(3) not help here? I can understand how there might be synchronization issues if it were implemented this way... I've never used these CPython APIs myself. You didn't have the memoryview APIs on your list. Does that mean they're easy? Then can you implement them? ;) Jean-Paul From rogerb at rogerbinns.com Tue May 3 01:41:07 2011 From: rogerb at rogerbinns.com (Roger Binns) Date: Mon, 02 May 2011 16:41:07 -0700 Subject: [pypy-dev] cpyext reference counting and other gc's In-Reply-To: References: <4DBF206A.70807@rogerbinns.com> Message-ID: <4DBF4113.1020900@rogerbinns.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/02/2011 03:08 PM, Amaury Forgeot d'Arc wrote: > Yes, we've implemented PyTraceback_here so that it works exactly for this usage. > Can you check whether pypy does the right thing for you as well? I will once I can get anything actually working. Currently having problems with pypy crashing. (The code does compile though.) Note that I am perfectly happy to change my code to something that works with pypy - I already had to make it work the different ways that Python 2 and 3 do. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk2/QQ4ACgkQmOOfHg372QRujACeLLylS1qh8jBCc3lmqNmGHvSn EHEAoI+3P8B8iuCIsmkhfl/0Bn1qdggc =8j1u -----END PGP SIGNATURE----- From rogerb at rogerbinns.com Tue May 3 07:04:38 2011 From: rogerb at rogerbinns.com (Roger Binns) Date: Mon, 02 May 2011 22:04:38 -0700 Subject: [pypy-dev] cpyext reference counting and other gc's In-Reply-To: <20110502225126.1992.2104890452.divmod.xquotient.1011@localhost.localdomain> References: <20110502225126.1992.2104890452.divmod.xquotient.1011@localhost.localdomain> Message-ID: <4DBF8CE6.9050906@rogerbinns.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/02/2011 03:51 PM, exarkun at twistedmatrix.com wrote: > On 07:09 pm, amauryfa at gmail.com wrote: >> Fortunately, there are not so many of them: >> - PyFile_AsFile(), PyFile_FromFile, because files opened by pypy don't >> use a FILE* (like python3) > > Do fdopen(3) and fileno(3) not help here? That gets you the OS handle but the FILE* has a whole bunch of other crud inside like a buffer, current offsets and eol handling. If it is just a one time transfer from Python to C or vice versa then it could work, but if both use the file/FILE concurrently then chances are you'll end up with file corruption. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk2/jOEACgkQmOOfHg372QTLWACdHs8UC5UBeby+kONqFV9Q8Jar sKAAn1EB8cXK8wZYWjOEnInTAYcSSXIQ =9/ZX -----END PGP SIGNATURE----- From fijall at gmail.com Tue May 3 08:50:39 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 3 May 2011 08:50:39 +0200 Subject: [pypy-dev] PyPy 1.5 released In-Reply-To: <1304372949.2063.5.camel@radiator.bos.redhat.com> References: <1304372949.2063.5.camel@radiator.bos.redhat.com> Message-ID: On Mon, May 2, 2011 at 11:49 PM, David Malcolm wrote: > On Sat, 2011-04-30 at 17:04 +0200, Armin Rigo wrote: >> ====================== >> PyPy 1.5: Catching Up >> ====================== >> >> We're pleased to announce the 1.5 release of PyPy. This release updates >> PyPy with the features of CPython 2.7.1, including the standard library. Thus >> all the features of `CPython 2.6`_ and `CPython 2.7`_ are now supported. It >> also contains additional performance improvements. You can download it here: >> >> ? ? http://pypy.org/download.html > > FWIW, I've now updated Fedora's RPM packages of PyPy to pypy-1.5. > > See https://bugzilla.redhat.com/show_bug.cgi?id=701121 for links to the > builds for various releases/architectures. Cool! Thanks. Arch linux was first btw ;-) > > Enjoy! > Dave > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From ondrej at certik.cz Tue May 3 09:22:44 2011 From: ondrej at certik.cz (Ondrej Certik) Date: Tue, 3 May 2011 00:22:44 -0700 Subject: [pypy-dev] speed of sympy tests Message-ID: Hi, I have downloaded the pypy 1.5 binary (with jit) and run tests (only sympy/core, so that it's fast) on Ubuntu Natty, 64bit: ondrej at eagle:~/repos/sympy(master)$ bin/test sympy/core/ ============================= test process starts ============================== executable: /usr/bin/python (2.7.1-final-0) ground types: python sympy/core/tests/test_arit.py[48] ...f.......................................... .. [OK] sympy/core/tests/test_assumptions.py[28] ....f....................... [OK] sympy/core/tests/test_basic.py[10] .......... [OK] sympy/core/tests/test_cache.py[1] . [OK] sympy/core/tests/test_complex.py[13] ............. [OK] sympy/core/tests/test_containers.py[5] ..... [OK] sympy/core/tests/test_count_ops.py[2] .. [OK] sympy/core/tests/test_diff.py[6] ...... [OK] sympy/core/tests/test_equal.py[5] ..... [OK] sympy/core/tests/test_eval.py[8] .....f.. [OK] sympy/core/tests/test_eval_power.py[12] ............ [OK] sympy/core/tests/test_evalf.py[23] ....................... [OK] sympy/core/tests/test_expand.py[6] ...... [OK] sympy/core/tests/test_expr.py[60] .............................................. .............. [OK] sympy/core/tests/test_exprtools.py[4] .... [OK] sympy/core/tests/test_facts.py[11] ........... [OK] sympy/core/tests/test_functions.py[23] .....f................. [OK] sympy/core/tests/test_logic.py[11] ........... [OK] sympy/core/tests/test_match.py[26] ...f...................... [OK] sympy/core/tests/test_numbers.py[47] ........................................... .... [OK] sympy/core/tests/test_operations.py[4] .... [OK] sympy/core/tests/test_priority.py[5] ..... [OK] sympy/core/tests/test_relational.py[7] ....... [OK] sympy/core/tests/test_sets.py[18] .................. [OK] sympy/core/tests/test_subs.py[30] .............................. [OK] sympy/core/tests/test_symbol.py[9] ......... [OK] sympy/core/tests/test_sympify.py[23] ....................... [OK] sympy/core/tests/test_truediv.py[3] ... [OK] sympy/core/tests/test_var.py[5] ..... [OK] ======= tests finished: 448 passed, 5 expected to fail, in 2.90 seconds ======== ondrej at eagle:~/repos/sympy(master)$ ~/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy bin/test sympy/core/ ============================= test process starts ============================== executable: /home/ondrej/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy (2.7.1-final-42) ground types: python sympy/core/tests/test_arit.py[48] ...f.......................................... .. [OK] sympy/core/tests/test_assumptions.py[28] ....f....................... [OK] sympy/core/tests/test_basic.py[10] .......... [OK] sympy/core/tests/test_cache.py[1] . [OK] sympy/core/tests/test_complex.py[13] ............. [OK] sympy/core/tests/test_containers.py[5] ..... [OK] sympy/core/tests/test_count_ops.py[2] .. [OK] sympy/core/tests/test_diff.py[6] ...... [OK] sympy/core/tests/test_equal.py[5] ..... [OK] sympy/core/tests/test_eval.py[8] .....f.. [OK] sympy/core/tests/test_eval_power.py[12] ............ [OK] sympy/core/tests/test_evalf.py[23] ....................... [OK] sympy/core/tests/test_expand.py[6] ...... [OK] sympy/core/tests/test_expr.py[60] ..........F................................... .............. [FAIL] sympy/core/tests/test_exprtools.py[4] .... [OK] sympy/core/tests/test_facts.py[11] ........... [OK] sympy/core/tests/test_functions.py[23] .....f................. [OK] sympy/core/tests/test_logic.py[11] ........... [OK] sympy/core/tests/test_match.py[26] ...f...................... [OK] sympy/core/tests/test_numbers.py[47] ........................................... .... [OK] sympy/core/tests/test_operations.py[4] .... [OK] sympy/core/tests/test_priority.py[5] ..... [OK] sympy/core/tests/test_relational.py[7] ....... [OK] sympy/core/tests/test_sets.py[18] .................. [OK] sympy/core/tests/test_subs.py[30] .............................. [OK] sympy/core/tests/test_symbol.py[9] ......... [OK] sympy/core/tests/test_sympify.py[23] ....................... [OK] sympy/core/tests/test_truediv.py[3] ... [OK] sympy/core/tests/test_var.py[5] ..... [OK] ________________________________________________________________________________ _____________ sympy/core/tests/test_expr.py:test_as_leading_term3 ______________ File "/home/ondrej/repos/sympy/sympy/core/tests/test_expr.py", line 212, in test_as_leading_term3 assert (2*x+pi*x+x**2).as_leading_term(x) == (2+pi)*x AssertionError == tests finished: 447 passed, 1 failed, 5 expected to fail, in 13.87 seconds == DO *NOT* COMMIT! The test failure is probably some minor difference between CPython and PyPy.. However, as you can see, the test execution is 4x slower in PyPy, compared to CPython. Is this the supposed slow down, or is there something specific to sympy? I have also done this test in bin/isympy, CPython: >>> from timeit import default_timer as clock >>> t = clock(); a = expand((1+x+y+z)**20); clock()-t 1.4584941864 PyPy: >>> from timeit import default_timer as clock >>> t = clock(); a = expand((1+x+y+z)**20); clock()-t 4.48898005486 so PyPy is roughly 3x slower here. So 3x or 4x slower, I think that's already very usable. Especially the sandbox pypy is an awesome idea, for serving web apps. The only problem is that I use Cython a lot, which I think is incompatible with PyPy... Ondrej From fijall at gmail.com Tue May 3 09:31:39 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 3 May 2011 09:31:39 +0200 Subject: [pypy-dev] speed of sympy tests In-Reply-To: References: Message-ID: On Tue, May 3, 2011 at 9:22 AM, Ondrej Certik wrote: > Hi, > > I have downloaded the pypy 1.5 binary (with jit) and run tests (only > sympy/core, so that it's fast) on Ubuntu Natty, 64bit: > > > ondrej at eagle:~/repos/sympy(master)$ bin/test sympy/core/ > ============================= test process starts ============================== > executable: ? /usr/bin/python ?(2.7.1-final-0) > ground types: python > > sympy/core/tests/test_arit.py[48] ...f.......................................... > .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_assumptions.py[28] ....f....................... ? ? ? [OK] > sympy/core/tests/test_basic.py[10] .......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_cache.py[1] . ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_complex.py[13] ............. ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_containers.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_count_ops.py[2] .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_diff.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_equal.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_eval.py[8] .....f.. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_eval_power.py[12] ............ ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_evalf.py[23] ....................... ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_expand.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_expr.py[60] .............................................. > .............. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_exprtools.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_facts.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_functions.py[23] .....f................. ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_logic.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_match.py[26] ...f...................... ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_numbers.py[47] ........................................... > .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_operations.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_priority.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_relational.py[7] ....... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_sets.py[18] .................. ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_subs.py[30] .............................. ? ? ? ? ? ?[OK] > sympy/core/tests/test_symbol.py[9] ......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_sympify.py[23] ....................... ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_truediv.py[3] ... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_var.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > > ======= tests finished: 448 passed, 5 expected to fail, in 2.90 seconds ======== > ondrej at eagle:~/repos/sympy(master)$ > ~/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy bin/test > sympy/core/ > ============================= test process starts ============================== > executable: ? /home/ondrej/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy > ?(2.7.1-final-42) > ground types: python > > sympy/core/tests/test_arit.py[48] ...f.......................................... > .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_assumptions.py[28] ....f....................... ? ? ? [OK] > sympy/core/tests/test_basic.py[10] .......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_cache.py[1] . ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_complex.py[13] ............. ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_containers.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_count_ops.py[2] .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_diff.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_equal.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_eval.py[8] .....f.. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_eval_power.py[12] ............ ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_evalf.py[23] ....................... ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_expand.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_expr.py[60] ..........F................................... > .............. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[FAIL] > sympy/core/tests/test_exprtools.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_facts.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_functions.py[23] .....f................. ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_logic.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_match.py[26] ...f...................... ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_numbers.py[47] ........................................... > .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_operations.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_priority.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_relational.py[7] ....... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_sets.py[18] .................. ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_subs.py[30] .............................. ? ? ? ? ? ?[OK] > sympy/core/tests/test_symbol.py[9] ......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_sympify.py[23] ....................... ? ? ? ? ? ? ? ?[OK] > sympy/core/tests/test_truediv.py[3] ... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > sympy/core/tests/test_var.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] > > ________________________________________________________________________________ > _____________ sympy/core/tests/test_expr.py:test_as_leading_term3 ______________ > ?File "/home/ondrej/repos/sympy/sympy/core/tests/test_expr.py", line > 212, in test_as_leading_term3 > ? ?assert (2*x+pi*x+x**2).as_leading_term(x) == (2+pi)*x > AssertionError > > == tests finished: 447 passed, 1 failed, 5 expected to fail, in 13.87 seconds == > DO *NOT* COMMIT! > > > > > The test failure is probably some minor difference between CPython and > PyPy.. However, as you can see, the test execution is 4x slower in > PyPy, compared to CPython. > Is this the supposed slow down, or is there something specific to sympy? > > I have also done this test in bin/isympy, CPython: > >>>> from timeit import default_timer as clock >>>> t = clock(); a = expand((1+x+y+z)**20); clock()-t > 1.4584941864 > > PyPy: > >>>> from timeit import default_timer as clock >>>> t = clock(); a = expand((1+x+y+z)**20); clock()-t > 4.48898005486 > > so PyPy is roughly 3x slower here. > > So 3x or 4x slower, I think that's already very usable. Especially the > sandbox pypy is an awesome idea, for serving web apps. The only > problem is that I use Cython a lot, which I think is incompatible with > PyPy... > > Ondrej > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > Hey. I think the test run is somehow within acceptable numbers (in theory tests should run each code once, hence making JIT not really fit well), however I think the other slowdown is not really acceptable. We were looking for a while to include some sympy benchmarks to our benchmark suite, maybe the good time is now :) I'll look at it From ondrej at certik.cz Tue May 3 09:44:05 2011 From: ondrej at certik.cz (Ondrej Certik) Date: Tue, 3 May 2011 00:44:05 -0700 Subject: [pypy-dev] speed of sympy tests In-Reply-To: References: Message-ID: On Tue, May 3, 2011 at 12:31 AM, Maciej Fijalkowski wrote: > On Tue, May 3, 2011 at 9:22 AM, Ondrej Certik wrote: >> Hi, >> >> I have downloaded the pypy 1.5 binary (with jit) and run tests (only >> sympy/core, so that it's fast) on Ubuntu Natty, 64bit: >> >> >> ondrej at eagle:~/repos/sympy(master)$ bin/test sympy/core/ >> ============================= test process starts ============================== >> executable: ? /usr/bin/python ?(2.7.1-final-0) >> ground types: python >> >> sympy/core/tests/test_arit.py[48] ...f.......................................... >> .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_assumptions.py[28] ....f....................... ? ? ? [OK] >> sympy/core/tests/test_basic.py[10] .......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_cache.py[1] . ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_complex.py[13] ............. ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_containers.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_count_ops.py[2] .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_diff.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_equal.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_eval.py[8] .....f.. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_eval_power.py[12] ............ ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_evalf.py[23] ....................... ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_expand.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_expr.py[60] .............................................. >> .............. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_exprtools.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_facts.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_functions.py[23] .....f................. ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_logic.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_match.py[26] ...f...................... ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_numbers.py[47] ........................................... >> .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_operations.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_priority.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_relational.py[7] ....... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_sets.py[18] .................. ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_subs.py[30] .............................. ? ? ? ? ? ?[OK] >> sympy/core/tests/test_symbol.py[9] ......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_sympify.py[23] ....................... ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_truediv.py[3] ... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_var.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> >> ======= tests finished: 448 passed, 5 expected to fail, in 2.90 seconds ======== >> ondrej at eagle:~/repos/sympy(master)$ >> ~/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy bin/test >> sympy/core/ >> ============================= test process starts ============================== >> executable: ? /home/ondrej/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy >> ?(2.7.1-final-42) >> ground types: python >> >> sympy/core/tests/test_arit.py[48] ...f.......................................... >> .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_assumptions.py[28] ....f....................... ? ? ? [OK] >> sympy/core/tests/test_basic.py[10] .......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_cache.py[1] . ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_complex.py[13] ............. ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_containers.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_count_ops.py[2] .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_diff.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_equal.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_eval.py[8] .....f.. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_eval_power.py[12] ............ ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_evalf.py[23] ....................... ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_expand.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_expr.py[60] ..........F................................... >> .............. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[FAIL] >> sympy/core/tests/test_exprtools.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_facts.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_functions.py[23] .....f................. ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_logic.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_match.py[26] ...f...................... ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_numbers.py[47] ........................................... >> .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_operations.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_priority.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_relational.py[7] ....... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_sets.py[18] .................. ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_subs.py[30] .............................. ? ? ? ? ? ?[OK] >> sympy/core/tests/test_symbol.py[9] ......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_sympify.py[23] ....................... ? ? ? ? ? ? ? ?[OK] >> sympy/core/tests/test_truediv.py[3] ... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> sympy/core/tests/test_var.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >> >> ________________________________________________________________________________ >> _____________ sympy/core/tests/test_expr.py:test_as_leading_term3 ______________ >> ?File "/home/ondrej/repos/sympy/sympy/core/tests/test_expr.py", line >> 212, in test_as_leading_term3 >> ? ?assert (2*x+pi*x+x**2).as_leading_term(x) == (2+pi)*x >> AssertionError >> >> == tests finished: 447 passed, 1 failed, 5 expected to fail, in 13.87 seconds == >> DO *NOT* COMMIT! >> >> >> >> >> The test failure is probably some minor difference between CPython and >> PyPy.. However, as you can see, the test execution is 4x slower in >> PyPy, compared to CPython. >> Is this the supposed slow down, or is there something specific to sympy? >> >> I have also done this test in bin/isympy, CPython: >> >>>>> from timeit import default_timer as clock >>>>> t = clock(); a = expand((1+x+y+z)**20); clock()-t >> 1.4584941864 >> >> PyPy: >> >>>>> from timeit import default_timer as clock >>>>> t = clock(); a = expand((1+x+y+z)**20); clock()-t >> 4.48898005486 >> >> so PyPy is roughly 3x slower here. >> >> So 3x or 4x slower, I think that's already very usable. Especially the >> sandbox pypy is an awesome idea, for serving web apps. The only >> problem is that I use Cython a lot, which I think is incompatible with >> PyPy... >> >> Ondrej >> _______________________________________________ >> pypy-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/pypy-dev >> > > Hey. > > I think the test run is somehow within acceptable numbers (in theory > tests should run each code once, hence making JIT not really fit > well), however I think the other slowdown is not really acceptable. We Well, it is also only run once, so JIT probably doesn't help. > were looking for a while to include some sympy benchmarks to our > benchmark suite, maybe the good time is now :) SymPy uses caching, so maybe that's the problem. It is possible to turn it down though. Ping me if you have any questions, or run into any problems with sympy. Ondrej From holger at merlinux.eu Tue May 3 11:55:00 2011 From: holger at merlinux.eu (holger krekel) Date: Tue, 3 May 2011 09:55:00 +0000 Subject: [pypy-dev] docs on doc.pypy.org Message-ID: <20110503095500.GW4071@merlinux.eu> Hi all, i created a CNAME for our use of pypy.readthedocs.org and so you can now reach pypy docs under the frontend URL of http://doc.pypy.org It might be better to promote this link also from pypy.org so that URLs have a higher chance to remain valid even if we decide to move away from readthedocs some day. Moreover, we could see to have (another) sphinx setup for sphinx setup for our current pypy.org so we can host it there as well, maybe under www.pypy.org -- we need to have a subdomain/CNAME as we cannot redirect the root domain to readthedocs. best, holger From anto.cuni at gmail.com Tue May 3 12:04:39 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 03 May 2011 12:04:39 +0200 Subject: [pypy-dev] docs on doc.pypy.org In-Reply-To: <20110503095500.GW4071@merlinux.eu> References: <20110503095500.GW4071@merlinux.eu> Message-ID: <4DBFD337.4080107@gmail.com> On 03/05/11 11:55, holger krekel wrote: > Hi all, > > i created a CNAME for our use of pypy.readthedocs.org and so you can now > reach pypy docs under the frontend URL of > > http://doc.pypy.org > > It might be better to promote this link also from pypy.org so > that URLs have a higher chance to remain valid even if we > decide to move away from readthedocs some day. yes, good idea. Moreover, I think we should change the link from "Dev Documentation" to just "Documentation", because it contains docs which are generally useful also for users (e.g., the getting started, the papers, etc.) Alternatively, we could have a heavier refactoring of the documentation and have clearly separated sections for "user docs" and "dev docs". > Moreover, we could see to have (another) sphinx setup for sphinx > setup for our current pypy.org so we can host it there as well, maybe > under www.pypy.org -- we need to have a subdomain/CNAME as we cannot > redirect the root domain to readthedocs. I'm not a sphinx expert, but I don't think that the default layout (i.e., with a sidebar on the left) is the best for the home page of pypy.org. ciao, Anto From holger at merlinux.eu Tue May 3 12:12:17 2011 From: holger at merlinux.eu (holger krekel) Date: Tue, 3 May 2011 10:12:17 +0000 Subject: [pypy-dev] docs on doc.pypy.org In-Reply-To: <4DBFD337.4080107@gmail.com> References: <20110503095500.GW4071@merlinux.eu> <4DBFD337.4080107@gmail.com> Message-ID: <20110503101217.GY4071@merlinux.eu> On Tue, May 03, 2011 at 12:04 +0200, Antonio Cuni wrote: > On 03/05/11 11:55, holger krekel wrote: > > Hi all, > > > > i created a CNAME for our use of pypy.readthedocs.org and so you can now > > reach pypy docs under the frontend URL of > > > > http://doc.pypy.org > > > > It might be better to promote this link also from pypy.org so > > that URLs have a higher chance to remain valid even if we > > decide to move away from readthedocs some day. > > yes, good idea. > Moreover, I think we should change the link from "Dev Documentation" to just > "Documentation", because it contains docs which are generally useful also for > users (e.g., the getting started, the papers, etc.) > > Alternatively, we could have a heavier refactoring of the documentation and > have clearly separated sections for "user docs" and "dev docs". > > > Moreover, we could see to have (another) sphinx setup for sphinx > > setup for our current pypy.org so we can host it there as well, maybe > > under www.pypy.org -- we need to have a subdomain/CNAME as we cannot > > redirect the root domain to readthedocs. > > I'm not a sphinx expert, but I don't think that the default layout (i.e., with > a sidebar on the left) is the best for the home page of pypy.org. I think sphinx can be tuned to support pypy.org layout but i don't consider myself a sphinx expert either. Maybe someone feels like helping and trying with https://bitbucket.org/pypy/pypy.org/overview to put it into a sphinx format? holger From holger at merlinux.eu Tue May 3 12:40:12 2011 From: holger at merlinux.eu (holger krekel) Date: Tue, 3 May 2011 10:40:12 +0000 Subject: [pypy-dev] docs on doc.pypy.org In-Reply-To: <20110503101217.GY4071@merlinux.eu> References: <20110503095500.GW4071@merlinux.eu> <4DBFD337.4080107@gmail.com> <20110503101217.GY4071@merlinux.eu> Message-ID: <20110503104012.GA4071@merlinux.eu> On Tue, May 03, 2011 at 10:12 +0000, holger krekel wrote: > On Tue, May 03, 2011 at 12:04 +0200, Antonio Cuni wrote: > > On 03/05/11 11:55, holger krekel wrote: > > > Hi all, > > > > > > i created a CNAME for our use of pypy.readthedocs.org and so you can now > > > reach pypy docs under the frontend URL of > > > > > > http://doc.pypy.org > > > > > > It might be better to promote this link also from pypy.org so > > > that URLs have a higher chance to remain valid even if we > > > decide to move away from readthedocs some day. > > > > yes, good idea. > > Moreover, I think we should change the link from "Dev Documentation" to just > > "Documentation", because it contains docs which are generally useful also for > > users (e.g., the getting started, the papers, etc.) > > > > Alternatively, we could have a heavier refactoring of the documentation and > > have clearly separated sections for "user docs" and "dev docs". > > > > > Moreover, we could see to have (another) sphinx setup for sphinx > > > setup for our current pypy.org so we can host it there as well, maybe > > > under www.pypy.org -- we need to have a subdomain/CNAME as we cannot > > > redirect the root domain to readthedocs. > > > > I'm not a sphinx expert, but I don't think that the default layout (i.e., with > > a sidebar on the left) is the best for the home page of pypy.org. > > I think sphinx can be tuned to support pypy.org layout but i don't > consider myself a sphinx expert either. Maybe someone feels like helping > and trying with https://bitbucket.org/pypy/pypy.org/overview > to put it into a sphinx format? seems like there is a limitation: http://read-the-docs.readthedocs.org/en/latest/install.html#what-s-available which i just hit while playing with rtfd and pytest docs :/ Maybe we can get ourselves whitelisted sometime. holger From fijall at gmail.com Tue May 3 13:20:09 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 3 May 2011 13:20:09 +0200 Subject: [pypy-dev] PyPy Assembler SQRT Patch In-Reply-To: References: Message-ID: Hi. Sorry it took so long to review, long weekend and whatnot. The test for x86 backend actually fails for -1.0 on 64bit. Can you reproduce/deny? On Tue, Apr 26, 2011 at 11:51 PM, Joe wrote: > Attached is a patch to allow pypy to use SQRTSD rather than calling > out to libc. ?This resulted in a 2x speedup, that scaled as the > benchmark took longer. > When i added another 0 to the end of the benchmark it was still a 2x speedup. > > benchmark results: > http://paste.pocoo.org/show/378122/ > > I'll be happy to answer any questions about the patch. > Joe > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From cool-rr at cool-rr.com Tue May 3 13:25:40 2011 From: cool-rr at cool-rr.com (cool-RR) Date: Tue, 3 May 2011 13:25:40 +0200 Subject: [pypy-dev] Allow subclassing `staticmethod` and `classmethod` Message-ID: In my project I made an `abstract_static_method` class: https://github.com/cool-RR/GarlicSim/blob/master/garlicsim/garlicsim/general_misc/abc_tools.py#L7 A similar one has been made in Python 3.2: http://hg.python.org/cpython/file/default/Lib/abc.py#l48 In Pypy this currently cannot be done, because if you subclass `staticmethod` into `abstractstaticmethod`, any instances will have a class of `staticmethod` instead of `abstractstaticmethod`: >>>> class A(staticmethod): .... pass .... >>>> def f(): .... pass .... >>>> a = A(f) >>>> a >>>> type(a) What can be done? I want to have `abstract_static_method` in my code. Thanks, Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cfbolz at gmx.de Tue May 3 13:36:43 2011 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 03 May 2011 13:36:43 +0200 Subject: [pypy-dev] Allow subclassing `staticmethod` and `classmethod` In-Reply-To: References: Message-ID: <4DBFE8CB.40300@gmx.de> Hi Ram, On 05/03/2011 01:25 PM, cool-RR wrote: > In my project I made an `abstract_static_method` class: > > https://github.com/cool-RR/GarlicSim/blob/master/garlicsim/garlicsim/general_misc/abc_tools.py#L7 > > A > similar one has been made in Python 3.2: > > http://hg.python.org/cpython/file/default/Lib/abc.py#l48 > > > In Pypy this > currently cannot be done, because if you subclass `staticmethod` into > `abstractstaticmethod`, any instances will have a class > of `staticmethod` instead of `abstractstaticmethod`: > > >>>> class A(staticmethod): > .... pass > .... > >>>> def f(): > .... pass > .... > >>>> a = A(f) > >>>> a > > >>>> type(a) > > > > What can be done? I want to have `abstract_static_method` in my code. This is a bug in PyPy, I fixed it in the repo, tomorrow you can use a fresh nightly build from here: http://buildbot.pypy.org/nightly/trunk/ Carl Friedrich From cool-rr at cool-rr.com Tue May 3 13:38:11 2011 From: cool-rr at cool-rr.com (cool-RR) Date: Tue, 3 May 2011 13:38:11 +0200 Subject: [pypy-dev] Allow subclassing `staticmethod` and `classmethod` In-Reply-To: <4DBFE8CB.40300@gmx.de> References: <4DBFE8CB.40300@gmx.de> Message-ID: Cool. Is there gonna be a 1.5.1 release with stuff like this? On Tue, May 3, 2011 at 1:36 PM, Carl Friedrich Bolz wrote: > Hi Ram, > > On 05/03/2011 01:25 PM, cool-RR wrote: > > In my project I made an `abstract_static_method` class: > > > > > https://github.com/cool-RR/GarlicSim/blob/master/garlicsim/garlicsim/general_misc/abc_tools.py#L7 > > > > < > https://github.com/cool-RR/GarlicSim/blob/master/garlicsim/garlicsim/general_misc/abc_tools.py#L7 > >A > > similar one has been made in Python 3.2: > > > > http://hg.python.org/cpython/file/default/Lib/abc.py#l48 > > > > > > In Pypy this > > currently cannot be done, because if you subclass `staticmethod` into > > `abstractstaticmethod`, any instances will have a class > > of `staticmethod` instead of `abstractstaticmethod`: > > > > >>>> class A(staticmethod): > > .... pass > > .... > > >>>> def f(): > > .... pass > > .... > > >>>> a = A(f) > > >>>> a > > > > >>>> type(a) > > > > > > > > What can be done? I want to have `abstract_static_method` in my code. > > > This is a bug in PyPy, I fixed it in the repo, tomorrow you can use a > fresh nightly build from here: > > http://buildbot.pypy.org/nightly/trunk/ > > Carl Friedrich > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -- Sincerely, Ram Rachum -------------- next part -------------- An HTML attachment was scrubbed... URL: From rndblnch at gmail.com Tue May 3 15:11:15 2011 From: rndblnch at gmail.com (renaud blanch) Date: Tue, 3 May 2011 15:11:15 +0200 Subject: [pypy-dev] BytecodeCorruption error with pypy-1.5 Message-ID: Hello, First, thank you for the pypy-1.5 release! I quickly tested pypy-1.5 (osx64 binary from your download page) with some code I use as benchmark (some solutions to Google code jam problems), and I got a reproductible "Fatal RPython error: BytecodeCorruption". The minimal code sample that reproduce this behaviour I could came up with is: def what(_="whatever"): return (_ if _ else _) if False else _ what() This code works as expected with any cpython and with pypy-1.4.1 on the same machine (Intel Core 2 Duo iMac/Mac OS X 10.6.7). Should I open a bug on the pypy tracker? renaud From fijall at gmail.com Tue May 3 15:18:30 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 3 May 2011 15:18:30 +0200 Subject: [pypy-dev] speed of sympy tests In-Reply-To: References: Message-ID: On Tue, May 3, 2011 at 9:44 AM, Ondrej Certik wrote: > On Tue, May 3, 2011 at 12:31 AM, Maciej Fijalkowski wrote: >> On Tue, May 3, 2011 at 9:22 AM, Ondrej Certik wrote: >>> Hi, >>> >>> I have downloaded the pypy 1.5 binary (with jit) and run tests (only >>> sympy/core, so that it's fast) on Ubuntu Natty, 64bit: >>> >>> >>> ondrej at eagle:~/repos/sympy(master)$ bin/test sympy/core/ >>> ============================= test process starts ============================== >>> executable: ? /usr/bin/python ?(2.7.1-final-0) >>> ground types: python >>> >>> sympy/core/tests/test_arit.py[48] ...f.......................................... >>> .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_assumptions.py[28] ....f....................... ? ? ? [OK] >>> sympy/core/tests/test_basic.py[10] .......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_cache.py[1] . ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_complex.py[13] ............. ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_containers.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_count_ops.py[2] .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_diff.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_equal.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_eval.py[8] .....f.. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_eval_power.py[12] ............ ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_evalf.py[23] ....................... ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_expand.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_expr.py[60] .............................................. >>> .............. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_exprtools.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_facts.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_functions.py[23] .....f................. ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_logic.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_match.py[26] ...f...................... ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_numbers.py[47] ........................................... >>> .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_operations.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_priority.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_relational.py[7] ....... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_sets.py[18] .................. ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_subs.py[30] .............................. ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_symbol.py[9] ......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_sympify.py[23] ....................... ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_truediv.py[3] ... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_var.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> >>> ======= tests finished: 448 passed, 5 expected to fail, in 2.90 seconds ======== >>> ondrej at eagle:~/repos/sympy(master)$ >>> ~/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy bin/test >>> sympy/core/ >>> ============================= test process starts ============================== >>> executable: ? /home/ondrej/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy >>> ?(2.7.1-final-42) >>> ground types: python >>> >>> sympy/core/tests/test_arit.py[48] ...f.......................................... >>> .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_assumptions.py[28] ....f....................... ? ? ? [OK] >>> sympy/core/tests/test_basic.py[10] .......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_cache.py[1] . ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_complex.py[13] ............. ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_containers.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_count_ops.py[2] .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_diff.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_equal.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_eval.py[8] .....f.. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_eval_power.py[12] ............ ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_evalf.py[23] ....................... ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_expand.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_expr.py[60] ..........F................................... >>> .............. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[FAIL] >>> sympy/core/tests/test_exprtools.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_facts.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_functions.py[23] .....f................. ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_logic.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_match.py[26] ...f...................... ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_numbers.py[47] ........................................... >>> .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_operations.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_priority.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_relational.py[7] ....... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_sets.py[18] .................. ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_subs.py[30] .............................. ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_symbol.py[9] ......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_sympify.py[23] ....................... ? ? ? ? ? ? ? ?[OK] >>> sympy/core/tests/test_truediv.py[3] ... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> sympy/core/tests/test_var.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>> >>> ________________________________________________________________________________ >>> _____________ sympy/core/tests/test_expr.py:test_as_leading_term3 ______________ >>> ?File "/home/ondrej/repos/sympy/sympy/core/tests/test_expr.py", line >>> 212, in test_as_leading_term3 >>> ? ?assert (2*x+pi*x+x**2).as_leading_term(x) == (2+pi)*x >>> AssertionError >>> >>> == tests finished: 447 passed, 1 failed, 5 expected to fail, in 13.87 seconds == >>> DO *NOT* COMMIT! >>> >>> >>> >>> >>> The test failure is probably some minor difference between CPython and >>> PyPy.. However, as you can see, the test execution is 4x slower in >>> PyPy, compared to CPython. >>> Is this the supposed slow down, or is there something specific to sympy? >>> >>> I have also done this test in bin/isympy, CPython: >>> >>>>>> from timeit import default_timer as clock >>>>>> t = clock(); a = expand((1+x+y+z)**20); clock()-t >>> 1.4584941864 >>> >>> PyPy: >>> >>>>>> from timeit import default_timer as clock >>>>>> t = clock(); a = expand((1+x+y+z)**20); clock()-t >>> 4.48898005486 >>> >>> so PyPy is roughly 3x slower here. >>> >>> So 3x or 4x slower, I think that's already very usable. Especially the >>> sandbox pypy is an awesome idea, for serving web apps. The only >>> problem is that I use Cython a lot, which I think is incompatible with >>> PyPy... >>> >>> Ondrej >>> _______________________________________________ >>> pypy-dev at codespeak.net >>> http://codespeak.net/mailman/listinfo/pypy-dev >>> >> >> Hey. >> >> I think the test run is somehow within acceptable numbers (in theory >> tests should run each code once, hence making JIT not really fit >> well), however I think the other slowdown is not really acceptable. We > > Well, it is also only run once, so JIT probably doesn't help. > >> were looking for a while to include some sympy benchmarks to our >> benchmark suite, maybe the good time is now :) > > > SymPy uses caching, so maybe that's the problem. It is possible to > turn it down though. Ping me if you have any questions, or run into > any problems with sympy. > > Ondrej > That might sound strange but for me, both pypy and cpython runs within 4s. Which python you use? From fijall at gmail.com Tue May 3 15:20:49 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 3 May 2011 15:20:49 +0200 Subject: [pypy-dev] speed of sympy tests In-Reply-To: References: Message-ID: On Tue, May 3, 2011 at 3:18 PM, Maciej Fijalkowski wrote: > On Tue, May 3, 2011 at 9:44 AM, Ondrej Certik wrote: >> On Tue, May 3, 2011 at 12:31 AM, Maciej Fijalkowski wrote: >>> On Tue, May 3, 2011 at 9:22 AM, Ondrej Certik wrote: >>>> Hi, >>>> >>>> I have downloaded the pypy 1.5 binary (with jit) and run tests (only >>>> sympy/core, so that it's fast) on Ubuntu Natty, 64bit: >>>> >>>> >>>> ondrej at eagle:~/repos/sympy(master)$ bin/test sympy/core/ >>>> ============================= test process starts ============================== >>>> executable: ? /usr/bin/python ?(2.7.1-final-0) >>>> ground types: python >>>> >>>> sympy/core/tests/test_arit.py[48] ...f.......................................... >>>> .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_assumptions.py[28] ....f....................... ? ? ? [OK] >>>> sympy/core/tests/test_basic.py[10] .......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_cache.py[1] . ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_complex.py[13] ............. ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_containers.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_count_ops.py[2] .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_diff.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_equal.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_eval.py[8] .....f.. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_eval_power.py[12] ............ ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_evalf.py[23] ....................... ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_expand.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_expr.py[60] .............................................. >>>> .............. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_exprtools.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_facts.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_functions.py[23] .....f................. ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_logic.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_match.py[26] ...f...................... ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_numbers.py[47] ........................................... >>>> .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_operations.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_priority.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_relational.py[7] ....... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_sets.py[18] .................. ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_subs.py[30] .............................. ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_symbol.py[9] ......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_sympify.py[23] ....................... ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_truediv.py[3] ... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_var.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> >>>> ======= tests finished: 448 passed, 5 expected to fail, in 2.90 seconds ======== >>>> ondrej at eagle:~/repos/sympy(master)$ >>>> ~/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy bin/test >>>> sympy/core/ >>>> ============================= test process starts ============================== >>>> executable: ? /home/ondrej/Downloads/pypy-c-jit-43780-b590cf6de419-linux64/bin/pypy >>>> ?(2.7.1-final-42) >>>> ground types: python >>>> >>>> sympy/core/tests/test_arit.py[48] ...f.......................................... >>>> .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_assumptions.py[28] ....f....................... ? ? ? [OK] >>>> sympy/core/tests/test_basic.py[10] .......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_cache.py[1] . ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_complex.py[13] ............. ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_containers.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_count_ops.py[2] .. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_diff.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_equal.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_eval.py[8] .....f.. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_eval_power.py[12] ............ ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_evalf.py[23] ....................... ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_expand.py[6] ...... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_expr.py[60] ..........F................................... >>>> .............. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[FAIL] >>>> sympy/core/tests/test_exprtools.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_facts.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_functions.py[23] .....f................. ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_logic.py[11] ........... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_match.py[26] ...f...................... ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_numbers.py[47] ........................................... >>>> .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_operations.py[4] .... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_priority.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_relational.py[7] ....... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_sets.py[18] .................. ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_subs.py[30] .............................. ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_symbol.py[9] ......... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_sympify.py[23] ....................... ? ? ? ? ? ? ? ?[OK] >>>> sympy/core/tests/test_truediv.py[3] ... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> sympy/core/tests/test_var.py[5] ..... ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [OK] >>>> >>>> ________________________________________________________________________________ >>>> _____________ sympy/core/tests/test_expr.py:test_as_leading_term3 ______________ >>>> ?File "/home/ondrej/repos/sympy/sympy/core/tests/test_expr.py", line >>>> 212, in test_as_leading_term3 >>>> ? ?assert (2*x+pi*x+x**2).as_leading_term(x) == (2+pi)*x >>>> AssertionError >>>> >>>> == tests finished: 447 passed, 1 failed, 5 expected to fail, in 13.87 seconds == >>>> DO *NOT* COMMIT! >>>> >>>> >>>> >>>> >>>> The test failure is probably some minor difference between CPython and >>>> PyPy.. However, as you can see, the test execution is 4x slower in >>>> PyPy, compared to CPython. >>>> Is this the supposed slow down, or is there something specific to sympy? >>>> >>>> I have also done this test in bin/isympy, CPython: >>>> >>>>>>> from timeit import default_timer as clock >>>>>>> t = clock(); a = expand((1+x+y+z)**20); clock()-t >>>> 1.4584941864 >>>> >>>> PyPy: >>>> >>>>>>> from timeit import default_timer as clock >>>>>>> t = clock(); a = expand((1+x+y+z)**20); clock()-t >>>> 4.48898005486 >>>> >>>> so PyPy is roughly 3x slower here. >>>> >>>> So 3x or 4x slower, I think that's already very usable. Especially the >>>> sandbox pypy is an awesome idea, for serving web apps. The only >>>> problem is that I use Cython a lot, which I think is incompatible with >>>> PyPy... >>>> >>>> Ondrej >>>> _______________________________________________ >>>> pypy-dev at codespeak.net >>>> http://codespeak.net/mailman/listinfo/pypy-dev >>>> >>> >>> Hey. >>> >>> I think the test run is somehow within acceptable numbers (in theory >>> tests should run each code once, hence making JIT not really fit >>> well), however I think the other slowdown is not really acceptable. We >> >> Well, it is also only run once, so JIT probably doesn't help. >> >>> were looking for a while to include some sympy benchmarks to our >>> benchmark suite, maybe the good time is now :) >> >> >> SymPy uses caching, so maybe that's the problem. It is possible to >> turn it down though. Ping me if you have any questions, or run into >> any problems with sympy. >> >> Ondrej >> > > That might sound strange but for me, both pypy and cpython runs within > 4s. Which python you use? > CPython 2.7 runs 2x faster than 2.6. Can we get some more real-time communication set up, so we can discuss a bit From arigo at tunes.org Tue May 3 15:30:46 2011 From: arigo at tunes.org (Armin Rigo) Date: Tue, 3 May 2011 15:30:46 +0200 Subject: [pypy-dev] Allow subclassing `staticmethod` and `classmethod` In-Reply-To: References: <4DBFE8CB.40300@gmx.de> Message-ID: Hi Cool-RR, On Tue, May 3, 2011 at 1:38 PM, cool-RR wrote: > Cool. Is there gonna be a 1.5.1 release with stuff like this? Yes, very likely. Note that the 1.5 release is just one of the nightly binaries :-) Armin From alex.gaynor at gmail.com Tue May 3 15:32:39 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Tue, 3 May 2011 09:32:39 -0400 Subject: [pypy-dev] Allow subclassing `staticmethod` and `classmethod` In-Reply-To: References: <4DBFE8CB.40300@gmx.de> Message-ID: On Tue, May 3, 2011 at 9:30 AM, Armin Rigo wrote: > Hi Cool-RR, > > On Tue, May 3, 2011 at 1:38 PM, cool-RR wrote: > > Cool. Is there gonna be a 1.5.1 release with stuff like this? > > Yes, very likely. Note that the 1.5 release is just one of the > nightly binaries :-) > > Armin > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > FYI classmethod works correctly in 1.5 (because I fixed it a while ago because Django uses it). Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Tue May 3 16:30:23 2011 From: arigo at tunes.org (Armin Rigo) Date: Tue, 3 May 2011 16:30:23 +0200 Subject: [pypy-dev] PyPy Assembler SQRT Patch In-Reply-To: References: Message-ID: Hi, On Tue, May 3, 2011 at 1:20 PM, Maciej Fijalkowski wrote: > Sorry it took so long to review, long weekend and whatnot. The test > for x86 backend actually fails for -1.0 on 64bit. Can you > reproduce/deny? I fixed it, it was really a bug about FINISH(ConstFloat(...)). But the patch, at least as applied now, does not have the intended effect; SQRTSD is never generated because ll_math_sqrt() is called with a residual call. Joe, can you fix it? I can try to if you don't have the time. A bient?t, Armin. From amauryfa at gmail.com Tue May 3 16:36:08 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 3 May 2011 16:36:08 +0200 Subject: [pypy-dev] BytecodeCorruption error with pypy-1.5 In-Reply-To: References: Message-ID: 2011/5/3 renaud blanch : > I quickly tested pypy-1.5 (osx64 binary from your download page) with > some code I use as benchmark (some solutions to Google code jam > problems), and I got a reproductible "Fatal RPython error: > BytecodeCorruption". > > The minimal code sample that reproduce this behaviour I could came up with is: > > def what(_="whatever"): > ? ?return (_ if _ else _) if False else _ > what() > > > This code works as expected with any cpython and with pypy-1.4.1 on > the same machine (Intel Core 2 Duo iMac/Mac OS X 10.6.7). > > Should I open a bug on the pypy tracker? Yes please ! -- Amaury Forgeot d'Arc From alex.gaynor at gmail.com Tue May 3 18:23:16 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Tue, 3 May 2011 12:23:16 -0400 Subject: [pypy-dev] PyPy Assembler SQRT Patch In-Reply-To: References: Message-ID: On Tue, May 3, 2011 at 10:30 AM, Armin Rigo wrote: > Hi, > > On Tue, May 3, 2011 at 1:20 PM, Maciej Fijalkowski > wrote: > > Sorry it took so long to review, long weekend and whatnot. The test > > for x86 backend actually fails for -1.0 on 64bit. Can you > > reproduce/deny? > > I fixed it, it was really a bug about FINISH(ConstFloat(...)). But > the patch, at least as applied now, does not have the intended effect; > SQRTSD is never generated because ll_math_sqrt() is called with a > residual call. Joe, can you fix it? I can try to if you don't have > the time. > > > A bient?t, > > Armin. > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > Another thing is it would be useful to test that the SQRTSD is generated, I don't know a good way to do that though. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Tue May 3 18:28:58 2011 From: arigo at tunes.org (Armin Rigo) Date: Tue, 3 May 2011 18:28:58 +0200 Subject: [pypy-dev] PyPy Assembler SQRT Patch In-Reply-To: References: Message-ID: Hi Alex, On Tue, May 3, 2011 at 6:23 PM, Alex Gaynor wrote: > Another thing is it would be useful to test that the SQRTSD is generated, I > don't know a good way to do that though. In the test, you can hack genop_math_sqrt() in-place to set a flag, and test that the flag is set... And another place it should also be tested is in test_pypy_c_new.py (pypy.module.pypyjit.test_pypy_c), i.e. you should write a test that checks that in a full translated pypy-c, the code corresponding to the Python source "math.sqrt(x)" indeed ends up being a CALL(sqrt_wrapper) and not a CALL(ll_math_sqrt). A bient?t, Armin. From lac at openend.se Tue May 3 18:50:02 2011 From: lac at openend.se (Laura Creighton) Date: Tue, 03 May 2011 18:50:02 +0200 Subject: [pypy-dev] docs on doc.pypy.org In-Reply-To: Message from holger krekel of "Tue, 03 May 2011 10:12:17 -0000." <20110503101217.GY4071@merlinux.eu> References: <20110503095500.GW4071@merlinux.eu> <4DBFD337.4080107@gmail.com> <20110503101217.GY4071@merlinux.eu> Message-ID: <201105031650.p43Go2C8018390@theraft.openend.se> In a message of Tue, 03 May 2011 10:12:17 -0000, holger krekel writes: >> I'm not a sphinx expert, but I don't think that the default layout (i.e >., with >> a sidebar on the left) is the best for the home page of pypy.org. > >I think sphinx can be tuned to support pypy.org layout but i don't >consider myself a sphinx expert either. Maybe someone feels like helping > >and trying with https://bitbucket.org/pypy/pypy.org/overview >to put it into a sphinx format? > >holger Georg Brandl (and you don't get more expert than this) showed up in the channel to volunteer to give us a custom look. Now we only have to decide what it is that we want. Laura From santagada at gmail.com Tue May 3 19:08:49 2011 From: santagada at gmail.com (Leonardo Santagada) Date: Tue, 3 May 2011 14:08:49 -0300 Subject: [pypy-dev] docs on doc.pypy.org In-Reply-To: <201105031650.p43Go2C8018390@theraft.openend.se> References: <20110503095500.GW4071@merlinux.eu> <4DBFD337.4080107@gmail.com> <20110503101217.GY4071@merlinux.eu> <201105031650.p43Go2C8018390@theraft.openend.se> Message-ID: On Tue, May 3, 2011 at 1:50 PM, Laura Creighton wrote: > In a message of Tue, 03 May 2011 10:12:17 -0000, holger krekel writes: >>> I'm not a sphinx expert, but I don't think that the default layout (i.e >>., with >>> a sidebar on the left) is the best for the home page of pypy.org. >> >>I think sphinx can be tuned to support pypy.org layout but i don't >>consider myself a sphinx expert either. ?Maybe someone feels like helping >> >>and trying with https://bitbucket.org/pypy/pypy.org/overview >>to put it into a sphinx format? >> >>holger > > Georg Brandl (and you don't get more expert than this) showed up in > the channel to volunteer to give us a custom look. ?Now we only have to > decide what it is that we want. I found this site themes to be really cheap and good looking: http://themeforest.net/category/site-templates/technology/software http://themeforest.net/category/site-templates/technology Either this or hiring someone I think is the best idea, pypy deserves a great looking website :) -- Leonardo Santagada From holger at merlinux.eu Tue May 3 19:13:05 2011 From: holger at merlinux.eu (holger krekel) Date: Tue, 3 May 2011 17:13:05 +0000 Subject: [pypy-dev] Allow subclassing `staticmethod` and `classmethod` In-Reply-To: References: <4DBFE8CB.40300@gmx.de> Message-ID: <20110503171305.GD4071@merlinux.eu> On Tue, May 03, 2011 at 15:30 +0200, Armin Rigo wrote: > Hi Cool-RR, > > On Tue, May 3, 2011 at 1:38 PM, cool-RR wrote: > > Cool. Is there gonna be a 1.5.1 release with stuff like this? > > Yes, very likely. Note that the 1.5 release is just one of the > nightly binaries :-) modulo a 1.5.1 version string, maybe? :) holger > Armin > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From rogerb at rogerbinns.com Thu May 5 05:54:00 2011 From: rogerb at rogerbinns.com (Roger Binns) Date: Wed, 04 May 2011 20:54:00 -0700 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: <4DBF15E0.6080005@rogerbinns.com> References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> <4DBE4F7A.3030109@gmail.com> <4DBF15E0.6080005@rogerbinns.com> Message-ID: <4DC21F58.8000101@rogerbinns.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/02/2011 01:36 PM, Roger Binns wrote: > On 05/01/2011 11:30 PM, Antonio Cuni wrote: >> you are right, we lack such a document. Do you feel like writing it? :-) > > I'll make an initial stab at it this week. Here you go: https://docs.google.com/document/pub?id=1Ve9B6Gz5laV63zYFOlb_Lhe8ZBIw2V1756lquott6JI Unfortunately building pypy myself didn't really achieve anything for my own code. PyPy behaved really bizarrely for example claiming functions don't exist in my test code (standard unittest). I get different behaviour if I run under gdb or not. Additionally objects aren't particularly useful. With CPython there are some nice macros supplied with a .gdbinit, especially pyo so you can tell what is going on. Pypy is behaving like a randomized black box! If anyone wants to try for themselves then the code is at http://code.google.com/p/apsw/source/checkout Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk3CH1MACgkQmOOfHg372QQsxACdElcBC7z/um35YWZ9d0fTM212 DkMAniD7ibCOYQcfHIeB8U8aSpMu9EQ6 =RRZh -----END PGP SIGNATURE----- From qbproger at gmail.com Thu May 5 22:39:05 2011 From: qbproger at gmail.com (Joe) Date: Thu, 5 May 2011 16:39:05 -0400 Subject: [pypy-dev] docs on doc.pypy.org In-Reply-To: References: <20110503095500.GW4071@merlinux.eu> <4DBFD337.4080107@gmail.com> <20110503101217.GY4071@merlinux.eu> <201105031650.p43Go2C8018390@theraft.openend.se> Message-ID: If you do a blog post asking for help someone would probably help out. Especially since they'd have pypy.org in their portfolio. It may not turn up anything, but worth a shot. Joe On Tue, May 3, 2011 at 1:08 PM, Leonardo Santagada wrote: > On Tue, May 3, 2011 at 1:50 PM, Laura Creighton wrote: >> In a message of Tue, 03 May 2011 10:12:17 -0000, holger krekel writes: >>>> I'm not a sphinx expert, but I don't think that the default layout (i.e >>>., with >>>> a sidebar on the left) is the best for the home page of pypy.org. >>> >>>I think sphinx can be tuned to support pypy.org layout but i don't >>>consider myself a sphinx expert either. ?Maybe someone feels like helping >>> >>>and trying with https://bitbucket.org/pypy/pypy.org/overview >>>to put it into a sphinx format? >>> >>>holger >> >> Georg Brandl (and you don't get more expert than this) showed up in >> the channel to volunteer to give us a custom look. ?Now we only have to >> decide what it is that we want. > > I found this site themes to be really cheap and good looking: > > http://themeforest.net/category/site-templates/technology/software > http://themeforest.net/category/site-templates/technology > > Either this or hiring someone I think is the best idea, pypy deserves > a great looking website :) > > -- > Leonardo Santagada > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From dacjames at gmail.com Thu May 5 23:00:08 2011 From: dacjames at gmail.com (Daniel Collins) Date: Thu, 5 May 2011 14:00:08 -0700 Subject: [pypy-dev] docs on doc.pypy.org In-Reply-To: References: <20110503095500.GW4071@merlinux.eu> <4DBFD337.4080107@gmail.com> <20110503101217.GY4071@merlinux.eu> <201105031650.p43Go2C8018390@theraft.openend.se> Message-ID: Hello! After following the project for a long time, I just joined the pypy mailing list looking to help out. I would love to help on the pypy.orgwebsite; admittedly, I am fairly new to web design but I think I could help. I have never contributed to an open source project before so I need a little guidance about how things work here. Thanks, let me know! --Daniel On Thu, May 5, 2011 at 1:39 PM, Joe wrote: > If you do a blog post asking for help someone would probably help out. > Especially since they'd have pypy.org in their portfolio. It may not > turn up anything, but worth a shot. > > Joe > > On Tue, May 3, 2011 at 1:08 PM, Leonardo Santagada > wrote: > > On Tue, May 3, 2011 at 1:50 PM, Laura Creighton wrote: > >> In a message of Tue, 03 May 2011 10:12:17 -0000, holger krekel writes: > >>>> I'm not a sphinx expert, but I don't think that the default layout > (i.e > >>>., with > >>>> a sidebar on the left) is the best for the home page of pypy.org. > >>> > >>>I think sphinx can be tuned to support pypy.org layout but i don't > >>>consider myself a sphinx expert either. Maybe someone feels like > helping > >>> > >>>and trying with https://bitbucket.org/pypy/pypy.org/overview > >>>to put it into a sphinx format? > >>> > >>>holger > >> > >> Georg Brandl (and you don't get more expert than this) showed up in > >> the channel to volunteer to give us a custom look. Now we only have to > >> decide what it is that we want. > > > > I found this site themes to be really cheap and good looking: > > > > http://themeforest.net/category/site-templates/technology/software > > http://themeforest.net/category/site-templates/technology > > > > Either this or hiring someone I think is the best idea, pypy deserves > > a great looking website :) > > > > -- > > Leonardo Santagada > > _______________________________________________ > > pypy-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/pypy-dev > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Thu May 5 23:13:31 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 5 May 2011 23:13:31 +0200 Subject: [pypy-dev] docs on doc.pypy.org In-Reply-To: References: <20110503095500.GW4071@merlinux.eu> <4DBFD337.4080107@gmail.com> <20110503101217.GY4071@merlinux.eu> <201105031650.p43Go2C8018390@theraft.openend.se> Message-ID: On Thu, May 5, 2011 at 11:00 PM, Daniel Collins wrote: > Hello! > After following the project for a long time, I just joined the pypy mailing > list looking to help out. ?I would love to help on the pypy.org > website;?admittedly, I am fairly new to web design but I think I could help. > > I have never contributed to an open source project before so I need a little > guidance about how things work here. ?Thanks, let me know! > --Daniel It's very simple. You come up with an idea what do you want to work on (this is done, pypy website) so you need to check out an appropriate repository. http://bitbucket.org/pypy/pypy.org then you either create a branch and issue a pull request or create a patch or whatever way is really convinient. Feel free to join #pypy on irc.freenode.net if you have any more questions > > On Thu, May 5, 2011 at 1:39 PM, Joe wrote: >> >> If you do a blog post asking for help someone would probably help out. >> ?Especially since they'd have pypy.org in their portfolio. ?It may not >> turn up anything, but worth a shot. >> >> Joe >> >> On Tue, May 3, 2011 at 1:08 PM, Leonardo Santagada >> wrote: >> > On Tue, May 3, 2011 at 1:50 PM, Laura Creighton wrote: >> >> In a message of Tue, 03 May 2011 10:12:17 -0000, holger krekel writes: >> >>>> I'm not a sphinx expert, but I don't think that the default layout >> >>>> (i.e >> >>>., with >> >>>> a sidebar on the left) is the best for the home page of pypy.org. >> >>> >> >>>I think sphinx can be tuned to support pypy.org layout but i don't >> >>>consider myself a sphinx expert either. ?Maybe someone feels like >> >>> helping >> >>> >> >>>and trying with https://bitbucket.org/pypy/pypy.org/overview >> >>>to put it into a sphinx format? >> >>> >> >>>holger >> >> >> >> Georg Brandl (and you don't get more expert than this) showed up in >> >> the channel to volunteer to give us a custom look. ?Now we only have to >> >> decide what it is that we want. >> > >> > I found this site themes to be really cheap and good looking: >> > >> > http://themeforest.net/category/site-templates/technology/software >> > http://themeforest.net/category/site-templates/technology >> > >> > Either this or hiring someone I think is the best idea, pypy deserves >> > a great looking website :) >> > >> > -- >> > Leonardo Santagada >> > _______________________________________________ >> > pypy-dev at codespeak.net >> > http://codespeak.net/mailman/listinfo/pypy-dev >> _______________________________________________ >> pypy-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/pypy-dev > > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From santagada at gmail.com Fri May 6 02:40:47 2011 From: santagada at gmail.com (Leonardo Santagada) Date: Thu, 5 May 2011 21:40:47 -0300 Subject: [pypy-dev] docs on doc.pypy.org In-Reply-To: References: <20110503095500.GW4071@merlinux.eu> <4DBFD337.4080107@gmail.com> <20110503101217.GY4071@merlinux.eu> <201105031650.p43Go2C8018390@theraft.openend.se> Message-ID: On Thu, May 5, 2011 at 6:13 PM, Maciej Fijalkowski wrote: >> I have never contributed to an open source project before so I need a little >> guidance about how things work here. ?Thanks, let me know! >> --Daniel > > It's very simple. You come up with an idea what do you want to work on > (this is done, pypy website) so you need to check out an appropriate > repository. > > http://bitbucket.org/pypy/pypy.org > > then you either create a branch and issue a pull request or create a > patch or whatever way is really convinient. > > Feel free to join #pypy on irc.freenode.net if you have any more questions Sorry to disagree but wouldn't just doing a sketch in photoshop and posting the design here be enough for a start? Then do the html+css+javascript needed and only then think about doing a template, which if you don't know how to do there are plenty of people that knows that and can do it for you in 2 hours tops if you have everything already on html. I think the problem is the design of the website, not coding the template in jinja2. -- Leonardo Santagada From exarkun at twistedmatrix.com Fri May 6 13:57:17 2011 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Fri, 06 May 2011 11:57:17 -0000 Subject: [pypy-dev] docs on doc.pypy.org In-Reply-To: References: <20110503095500.GW4071@merlinux.eu> <4DBFD337.4080107@gmail.com> <20110503101217.GY4071@merlinux.eu> <201105031650.p43Go2C8018390@theraft.openend.se> Message-ID: <20110506115717.1992.541890468.divmod.xquotient.1245@localhost.localdomain> On 12:40 am, santagada at gmail.com wrote: >On Thu, May 5, 2011 at 6:13 PM, Maciej Fijalkowski >wrote: >>>I have never contributed to an open source project before so I need a >>>little >>>guidance about how things work here. ?Thanks, let me know! >>>--Daniel >> >>It's very simple. You come up with an idea what do you want to work on >>(this is done, pypy website) so you need to check out an appropriate >>repository. >> >>http://bitbucket.org/pypy/pypy.org >> >>then you either create a branch and issue a pull request or create a >>patch or whatever way is really convinient. >> >>Feel free to join #pypy on irc.freenode.net if you have any more >>questions > >Sorry to disagree but wouldn't just doing a sketch in photoshop and Sorry, this isn't really very on topic, but photoshop is a stupid, worse-than-useless tool for designing websites. Sketch it in html+css+javascript. If you can't do that, then you haven't really designed anything. Jean-Paul >posting the design here be enough for a start? Then do the >html+css+javascript needed and only then think about doing a template, >which if you don't know how to do there are plenty of people that >knows that and can do it for you in 2 hours tops if you have >everything already on html. > >I think the problem is the design of the website, not coding the >template in jinja2. > >-- >Leonardo Santagada >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev From qbproger at gmail.com Fri May 6 19:29:08 2011 From: qbproger at gmail.com (Joe) Date: Fri, 6 May 2011 13:29:08 -0400 Subject: [pypy-dev] PyPy Assembler SQRT Patch In-Reply-To: References: Message-ID: Attached an updated patch. It actually generates the assembly now to use SQRTSD. It provides about the same performance as the previous patch. Because of that, I feel as though the performance gain can be attributed to the change in error checking. It might be worth splitting out the other math functions and slimming down the error checking. Right now in the new_unary_math_function it generates a math function that does 3 C library calls 2 errno calls and the math function itself. The patch takes this down to the minimal number of checks necessary for math.sqrt, making only an isfinite() call in addition to the sqrt() call best case. Previously it'd call "_error_reset(); c_func(); rposix.get_errno()" and then at least 1 call to isnan() and a call to either isnan() or isinf(). Does this seem like something worth looking into for the other math functions? It wouldn't be as hard as getting the whole assembly instruction working, and seems like it provides the majority of the performance gain. Joe On Tue, May 3, 2011 at 12:28 PM, Armin Rigo wrote: > Hi Alex, > > On Tue, May 3, 2011 at 6:23 PM, Alex Gaynor wrote: >> Another thing is it would be useful to test that the SQRTSD is generated, I >> don't know a good way to do that though. > > In the test, you can hack genop_math_sqrt() in-place to set a flag, > and test that the flag is set... > > And another place it should also be tested is in test_pypy_c_new.py > (pypy.module.pypyjit.test_pypy_c), i.e. you should write a test that > checks that in a full translated pypy-c, the code corresponding to the > Python source "math.sqrt(x)" indeed ends up being a CALL(sqrt_wrapper) > and not a CALL(ll_math_sqrt). > > > A bient?t, > > Armin. > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -------------- next part -------------- A non-text attachment was scrubbed... Name: sqrtsd.patch Type: application/octet-stream Size: 17795 bytes Desc: not available URL: From kristjan at ccpgames.com Fri May 6 20:30:55 2011 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Sat, 7 May 2011 02:30:55 +0800 Subject: [pypy-dev] cheerio Message-ID: <2E034B571A5CE44E949B9FCC3B6D24EE6D298E70@exchcn.ccp.ad.local> Hi there. I finally got round to subscribe after the G?teborg sprint. Hi there, to all of you sprinters, thanks for the fun! Meanwhile, the Stackless project urgently feels the need to resolve the RCS system. Any suggestions on how to proceed with moving the stackless repository from the python svn one to Hg? Also, there is the idea we mentioned to have a 2.7 branch that we can freely modify for our mutual purposes, e.g. by fixing some of the unittests and other stuff. Any thoughts on that? -Kristj?n -------------- next part -------------- An HTML attachment was scrubbed... URL: From dimaqq at gmail.com Fri May 6 21:53:45 2011 From: dimaqq at gmail.com (Dima Tisnek) Date: Fri, 6 May 2011 12:53:45 -0700 Subject: [pypy-dev] cheerio In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE6D298E70@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE6D298E70@exchcn.ccp.ad.local> Message-ID: go git, much better 2011/5/6 Kristj?n Valur J?nsson : > Hi there. > > I finally got round to subscribe after the G?teborg sprint.? Hi there, to > all of you sprinters, thanks for the fun! > > > > Meanwhile, the Stackless project urgently feels the need to resolve the RCS > system. > > Any suggestions on how to proceed with moving the stackless repository from > the python svn one to Hg? > > Also, there is the idea we mentioned to have a 2.7 branch that we can freely > modify for our mutual purposes, e.g. by fixing some of the unittests and > other stuff.? Any thoughts on that? > > > > -Kristj?n > > > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From Ronny.Pfannschmidt at gmx.de Fri May 6 22:20:42 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Fri, 06 May 2011 22:20:42 +0200 Subject: [pypy-dev] cheerio In-Reply-To: <2E034B571A5CE44E949B9FCC3B6D24EE6D298E70@exchcn.ccp.ad.local> References: <2E034B571A5CE44E949B9FCC3B6D24EE6D298E70@exchcn.ccp.ad.local> Message-ID: <1304713242.4655.6.camel@Klappe2> On Sat, 2011-05-07 at 02:30 +0800, Kristj?n Valur J?nsson wrote: > Hi there. > > I finally got round to subscribe after the G?teborg sprint. Hi there, > to all of you sprinters, thanks for the fun! > > > > Meanwhile, the Stackless project urgently feels the need to resolve > the RCS system. > > Any suggestions on how to proceed with moving the stackless repository > from the python svn one to Hg? that depends mainly on how stackless managed syncing with cpython and if/how you want to record that in the new history also the importance of merges being recorded is a important question its basically choosing the tradeoff between correctness and ease of migration (for pypy we skipped merges and for old files we have to resort to using annotate on svn) > > Also, there is the idea we mentioned to have a 2.7 branch that we can > freely modify for our mutual purposes, e.g. by fixing some of the > unittests and other stuff. Any thoughts on that? there is some talk on splitting the stdlib to a separate repo (which may be more interesting for everyone) -- Ronny > > > > -Kristj?n > > > > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From qbproger at gmail.com Sat May 7 06:15:50 2011 From: qbproger at gmail.com (Joe) Date: Sat, 7 May 2011 00:15:50 -0400 Subject: [pypy-dev] PyPy Assembler SQRT Patch In-Reply-To: References: Message-ID: My mistake. There was a bug so it appeared as though SQRTSD wasn't being used while running the tests. It turns out the first patch I posted probably worked. After benchmarking, SQRTSD is definitely the cause of the speedup. Sorry for the confusion. Joe On Fri, May 6, 2011 at 1:29 PM, Joe wrote: > Attached an updated patch. ?It actually generates the assembly now to > use SQRTSD. ?It provides about the same performance as the previous > patch. ?Because of that, I feel as though the performance gain can be > attributed to the change in error checking. > > It might be worth splitting out the other math functions and slimming > down the error checking. ?Right now in the new_unary_math_function it > generates a math function that does 3 C library calls 2 errno calls > and the math function itself. ?The patch takes this down to the > minimal number of checks necessary for math.sqrt, making only an > isfinite() call in addition to the sqrt() call best case. ?Previously > it'd call "_error_reset(); c_func(); rposix.get_errno()" and then at > least 1 call to isnan() and a call to either isnan() or isinf(). > > Does this seem like something worth looking into for the other math > functions? ?It wouldn't be as hard as getting the whole assembly > instruction working, and seems like it provides the majority of the > performance gain. > > Joe > > On Tue, May 3, 2011 at 12:28 PM, Armin Rigo wrote: >> Hi Alex, >> >> On Tue, May 3, 2011 at 6:23 PM, Alex Gaynor wrote: >>> Another thing is it would be useful to test that the SQRTSD is generated, I >>> don't know a good way to do that though. >> >> In the test, you can hack genop_math_sqrt() in-place to set a flag, >> and test that the flag is set... >> >> And another place it should also be tested is in test_pypy_c_new.py >> (pypy.module.pypyjit.test_pypy_c), i.e. you should write a test that >> checks that in a full translated pypy-c, the code corresponding to the >> Python source "math.sqrt(x)" indeed ends up being a CALL(sqrt_wrapper) >> and not a CALL(ll_math_sqrt). >> >> >> A bient?t, >> >> Armin. >> _______________________________________________ >> pypy-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/pypy-dev >> > From arigo at tunes.org Sat May 7 11:41:31 2011 From: arigo at tunes.org (Armin Rigo) Date: Sat, 7 May 2011 11:41:31 +0200 Subject: [pypy-dev] PyPy Assembler SQRT Patch In-Reply-To: References: Message-ID: Hi Joe, On Sat, May 7, 2011 at 6:15 AM, Joe wrote: > My mistake. ?There was a bug so it appeared as though SQRTSD wasn't > being used while running the tests. ?It turns out the first patch I > posted probably worked. ?After benchmarking, SQRTSD is definitely the > cause of the speedup. ?Sorry for the confusion. Sorry, the confusion probably came from my side. I missed that running the tests with interp_operations() would not set supports_floats=True :-( That's fixed now. Thank you again for your patience. Armin From estama at gmail.com Sun May 8 03:13:57 2011 From: estama at gmail.com (Elefterios Stamatogiannakis) Date: Sun, 08 May 2011 04:13:57 +0300 Subject: [pypy-dev] Pypy bugs Message-ID: <4DC5EE55.4080004@gmail.com> Hi all, and many many thanks for pypy. Your work on pypy is exceptional. Due to Roger Binns doing the first steps of porting APSW on top of pypy, i've tried my project (madIS: http://code.google.com/p/madis ) with pypy and head APSW, and found the following: In python 2.7.1: In [1]: print dict.__setattr__ In pypy 1.5: In [10]: print dict.__setattr__ In essence in python 2.7.1 dict.__setattr__ is the same as object.__setattr__ whereas in pypy dict.__setattr__ works only with dictionaries, so it throws the following error when called with an object: TypeError: unbound method __setattr__() must be called with dict instance as first argument Above problem was solved by changing all dict.__setattr__ invocations into object.__setattr__ invocations, and now the same code works in both CPython and pypy. The second problem that i found with pypy is this: In [12]: import xml.etree.c xml.etree.cElementTree Above is to show that cElementTree exists in pypy's xml.etree. If then i try to import cElementTree: In [11]: import xml.etree.cElementTree --------------------------------------------------------------------------- ImportError Traceback (most recent call last) /home/estama/programs/pypy-15/bin/ in () /home/estama/programs/pypy-15/lib-python/2.7/xml/etree/cElementTree.pyc in () 1 # Wrapper module for _elementtree 2 ----> 3 from _elementtree import * ImportError: No module named _elementtree Thanks again for pypy, and to Roger Binns for your outstanding work on APSW. lefteris. From drsalists at gmail.com Sun May 8 05:06:00 2011 From: drsalists at gmail.com (Dan Stromberg) Date: Sat, 7 May 2011 20:06:00 -0700 Subject: [pypy-dev] Pypy bugs In-Reply-To: <4DC5EE55.4080004@gmail.com> References: <4DC5EE55.4080004@gmail.com> Message-ID: Hi Elefterios. I don't really work on PyPy, but I've written code that runs on PyPy and CPython 2&3 and Jython, unmodified. I tried IronPython too, but it was too different from the others. On Sat, May 7, 2011 at 6:13 PM, Elefterios Stamatogiannakis < estama at gmail.com> wrote: > Hi all, and many many thanks for pypy. Your work on pypy is exceptional. > > Due to Roger Binns doing the first steps of porting APSW on top of pypy, > i've tried my project (madIS: http://code.google.com/p/madis ) with pypy > and head APSW, and found the following: > > In python 2.7.1: > In [1]: print dict.__setattr__ > > > In pypy 1.5: > In [10]: print dict.__setattr__ > > > In essence in python 2.7.1 dict.__setattr__ is the same as > object.__setattr__ whereas in pypy dict.__setattr__ works only with > dictionaries, so it throws the following error when called with an object: > > TypeError: unbound method __setattr__() must be called with dict > instance as first argument > > Above problem was solved by changing all dict.__setattr__ invocations > into object.__setattr__ invocations, and now the same code works in both > CPython and pypy. > Actually, this feels a bit like it was a portability issue in your code to me. Why should dict's __setattr__ apply to all objects? Doesn't it seem more natural that it would apply to dicts alone, and if it applies to more, that that's an implementation detail? > The second problem that i found with pypy is this: > > In [12]: import xml.etree.c > xml.etree.cElementTree > > Above is to show that cElementTree exists in pypy's xml.etree. If then i > try to import cElementTree: > > In [11]: import xml.etree.cElementTree > --------------------------------------------------------------------------- > ImportError Traceback (most recent call last) > > /home/estama/programs/pypy-15/bin/ in () > > /home/estama/programs/pypy-15/lib-python/2.7/xml/etree/cElementTree.pyc > in () > 1 # Wrapper module for _elementtree > > 2 > ----> 3 from _elementtree import * > > ImportError: No module named _elementtree > > Thanks again for pypy, and to Roger Binns for your outstanding work on > APSW. > PyPy in general doesn't include a bunch of C extension modules; it's written in Python and RPython, so this seems somewhat natural. Your code probably should do something like: try: import xml.etree.cElementTree as element_tree except ImportError: import xml.etree.ElementTree as element_tree ...and then use element_tree.whatever in your code. It's 3 more lines, but more portable. HTH. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rubycoder at gmail.com Sun May 8 09:39:54 2011 From: rubycoder at gmail.com (Gustavo Kuerten) Date: Sun, 8 May 2011 15:39:54 +0800 Subject: [pypy-dev] rlb.parsing left recursion support ? Message-ID: Hi, I noticed that there is a test in test_pypackrat.py which seems to suggest that the parser supports left recursion. a: a ":" | "b"; However running with the ebnf above failed with the error: nonterminal a is in its own follow set(['a']) AssertionError assert not self.has_left_recursion() So, does it or does it not support left recursion ? -- Regards, S -------------- next part -------------- An HTML attachment was scrubbed... URL: From estama at gmail.com Sun May 8 12:18:56 2011 From: estama at gmail.com (Elefterios Stamatogiannakis) Date: Sun, 08 May 2011 13:18:56 +0300 Subject: [pypy-dev] Pypy bugs In-Reply-To: References: <4DC5EE55.4080004@gmail.com> Message-ID: <4DC66E10.7060306@gmail.com> On 08/05/11 06:06, Dan Stromberg wrote: > > Hi Elefterios. > > I don't really work on PyPy, but I've written code that runs on PyPy and > CPython 2&3 and Jython, unmodified. I tried IronPython too, but it was > too different from the others. I don't really know how pypy wants to present itself. From reading the site and blog, i got the impression that pypy wishes to become an alternative to (and maybe replace it?) CPython, and not a "parallel universe" Python like Jython and IronPython. If this is indeed true, then pypy has to mirror some of the quirks and historic bugs of CPython, that people have come to depend on. Nevertheless, i don't mind at all, if a porting effort is required to retarget something to also run on pypy. And from what i see, you have a lot more experience in doing this than me :-) . > > On Sat, May 7, 2011 at 6:13 PM, Elefterios Stamatogiannakis > > wrote: > > Hi all, and many many thanks for pypy. Your work on pypy is exceptional. > > Due to Roger Binns doing the first steps of porting APSW on top of pypy, > i've tried my project (madIS: http://code.google.com/p/madis ) with pypy > and head APSW, and found the following: > > In python 2.7.1: > In [1]: print dict.__setattr__ > > > In pypy 1.5: > In [10]: print dict.__setattr__ > > > In essence in python 2.7.1 dict.__setattr__ is the same as > object.__setattr__ whereas in pypy dict.__setattr__ works only with > dictionaries, so it throws the following error when called with an > object: > > TypeError: unbound method __setattr__() must be called with dict > instance as first argument > > Above problem was solved by changing all dict.__setattr__ invocations > into object.__setattr__ invocations, and now the same code works in both > CPython and pypy. > > > Actually, this feels a bit like it was a portability issue in your code > to me. Why should dict's __setattr__ apply to all objects? Doesn't it > seem more natural that it would apply to dicts alone, and if it applies > to more, that that's an implementation detail? You are right, and at first i too thought that CPython's behavior wasn't right. Nevertheless, maybe there is a reason that CPython behaves in this non consistent way, and other programs may hit this too. As it is right now, i prefer pypy's implementation on this, as it makes more sense, and it is easily corrected. > > The second problem that i found with pypy is this: > > In [12]: import xml.etree.c > xml.etree.cElementTree > > Above is to show that cElementTree exists in pypy's xml.etree. If then i > try to import cElementTree: > > In [11]: import xml.etree.cElementTree > --------------------------------------------------------------------------- > ImportError Traceback (most recent > call last) > > /home/estama/programs/pypy-15/bin/ in () > > /home/estama/programs/pypy-15/lib-python/2.7/xml/etree/cElementTree.pyc > in () > 1 # Wrapper module for _elementtree > > 2 > ----> 3 from _elementtree import * > > ImportError: No module named _elementtree > > Thanks again for pypy, and to Roger Binns for your outstanding work > on APSW. > > > PyPy in general doesn't include a bunch of C extension modules; it's > written in Python and RPython, so this seems somewhat natural. Your > code probably should do something like: > > try: > import xml.etree.cElementTree as element_tree > except ImportError: > import xml.etree.ElementTree as element_tree > > ...and then use element_tree.whatever in your code. It's 3 more lines, > but more portable. > > HTH. You are exactly right, and i've more or less used code like the one you presented. My only problem is that i believe that pypy shouldn't have answered at my first example (In [12]:...) above that cElementTree exists in xml.etree .Or maybe it should short circuit cElementTree to point at ElementTree. To declare that something exists, and then when somebody tries to use it, to throw an exception, isn't that nice. l. From arigo at tunes.org Sun May 8 13:17:23 2011 From: arigo at tunes.org (Armin Rigo) Date: Sun, 8 May 2011 13:17:23 +0200 Subject: [pypy-dev] Pypy bugs In-Reply-To: <4DC66E10.7060306@gmail.com> References: <4DC5EE55.4080004@gmail.com> <4DC66E10.7060306@gmail.com> Message-ID: Hi Elefterios, On Sun, May 8, 2011 at 12:18 PM, Elefterios Stamatogiannakis wrote: > You are exactly right, and i've more or less used code like the one you > presented. My only problem is that i believe that pypy shouldn't have > answered at my first example (In [12]:...) above that cElementTree > exists in xml.etree .Or maybe it should short circuit cElementTree to > point at ElementTree. To declare that something exists, and then when > somebody tries to use it, to throw an exception, isn't that nice. Nothing we can do: the module cElementTree is actually written in Python also in CPython --- it is cElementTree.py --- but it import internally "_elementtree". So there is no way PyPy can know that cElementTree cannot be imported before actually trying to. The workaround to write "try: import cXxx; except ImportError: import xxx" is the general solution. It works in other cases too (cProfile, cPickle, cStringIO, etc). And also, it works in case the version of CPython you are running on misses some C extension modules. For example you would get the exact same error as you got on PyPy if you are running on a CPython where the C module "_elementree" was not compiled. A bient?t, Armin. From arigo at tunes.org Sun May 8 13:22:07 2011 From: arigo at tunes.org (Armin Rigo) Date: Sun, 8 May 2011 13:22:07 +0200 Subject: [pypy-dev] rlb.parsing left recursion support ? In-Reply-To: References: Message-ID: Hi Gustavo, On Sun, May 8, 2011 at 9:39 AM, Gustavo Kuerten wrote: > I noticed that there is a test in test_pypackrat.py which seems to suggest > that the parser supports left recursion. > > a: a ":" | "b"; I cannot find this test anywhere. Can you tell more precisely where it is from? I looked in pypy/rlib/parsing/test/test_pypackrat.py. > So, does it or does it not support left recursion ? It does not, as far as I know. Armin From pypy at pocketnix.org Sun May 8 13:22:58 2011 From: pypy at pocketnix.org (Da_Blitz) Date: Sun, 8 May 2011 11:22:58 +0000 Subject: [pypy-dev] Pypy bugs In-Reply-To: <4DC66E10.7060306@gmail.com> References: <4DC5EE55.4080004@gmail.com> <4DC66E10.7060306@gmail.com> Message-ID: <20110508112258.GA14145@pocketnix.org> On Sun, May 08, 2011 at 01:18:56PM +0300, Elefterios Stamatogiannakis wrote: > You are right, and at first i too thought that CPython's behavior wasn't > right. Nevertheless, maybe there is a reason that CPython behaves in > this non consistent way, and other programs may hit this too. > > As it is right now, i prefer pypy's implementation on this, as it makes > more sense, and it is easily corrected. I had a quick look at your code and i don't fully grasp why you didn't just use setattr which in my understanding is the correct way to go about this, touching anything starting with "__" or "_" should be avoided as you should not rely on those attributes, for example when passed in as an argument to a function/method. i understand that you are invoking it on a known object and not a passed argument but i would still apply the same rules and try to avoid relying on direct access to those attributes and instead call them in the proper manner i would have a hard time calling this a pypy bug, i would say the code is too closly coupled to the implemetnation of dict. > You are exactly right, and i've more or less used code like the one you > presented. My only problem is that i believe that pypy shouldn't have > answered at my first example (In [12]:...) above that cElementTree > exists in xml.etree .Or maybe it should short circuit cElementTree to > point at ElementTree. To declare that something exists, and then when > somebody tries to use it, to throw an exception, isn't that nice. looks like the lib is in the lib-python/2.7 and not lib-python/modified-2.7 directory and so it appears to be a side effect of importing the python std-lib as a side note cElementTree is gone in python 3.3 (didn't check the other versions) and most python libs with optional c versions now autodetect if the lib is installed and use that (eg Pickle and cPickle). using the try/except clause is the recommended idiom to avoid this situation when there are multiple versions of the same lib available i also would not call this a pypy bug as you rightly pointed out people may see pypy as a cpython replacment and perhaps noting the above 2 things in the cpython_diffrences.rst file in the doc folder would be a good idea hope i am right, feel free to flame me if i am not From rubycoder at gmail.com Sun May 8 14:53:40 2011 From: rubycoder at gmail.com (Gustavo Kuerten) Date: Sun, 8 May 2011 20:53:40 +0800 Subject: [pypy-dev] rlb.parsing left recursion support ? In-Reply-To: References: Message-ID: Armin, My example is not equivalent to the actual test. But here is the snippet from test_pypackrat.py ?? ?def test_leftrecursion(self): ?? ? ? ?class parser(PackratParser): ?? ? ? ? ? ?""" ?? ? ? ? ? ?b: b 'a' | 'b'; ?? ? ? ? ? ?""" ?? ? ? ?print parser._code ?? ? ? ?p = parser("b") ?? ? ? ?res = p.b() ?? ? ? ?assert res == "b" ?? ? ? ?p = parser("bac") ?? ? ? ?res = p.b() ?? ? ? ?assert p._pos == 2 ?? ? ? ?assert res == "a" ?? ? ? ?p = parser("baaaaaaaaaaaaaac") ?? ? ? ?res = p.b() ?? ? ? ?assert p._pos == 15 ?? ? ? ?assert res == "a" This seems to indicate that it does support?left recursion. 2011/5/8 Armin Rigo > > Hi Gustavo, > > On Sun, May 8, 2011 at 9:39 AM, Gustavo Kuerten wrote: > > I noticed that there is a test in test_pypackrat.py which seems to suggest > > that the parser supports left recursion. > > > > a: a ":" | "b"; > > I cannot find this test anywhere. ?Can you tell more precisely where > it is from? ?I looked in pypy/rlib/parsing/test/test_pypackrat.py. > > > So, does it or does it not support left recursion ? > > It does not, as far as I know. > > > Armin -- Regards, G From arigo at tunes.org Sun May 8 15:07:22 2011 From: arigo at tunes.org (Armin Rigo) Date: Sun, 8 May 2011 15:07:22 +0200 Subject: [pypy-dev] rlb.parsing left recursion support ? In-Reply-To: References: Message-ID: Hi Gustavo, On Sun, May 8, 2011 at 2:53 PM, Gustavo Kuerten wrote: > This seems to indicate that it does support?left recursion. Ah, sorry. I have no real clue, so I can just answer from looking at the source code, which I'm sure you did already for longer than me. There seems to be two classes called PackratParser. You are getting the error from the one in parsing.py, but test_pypackrat uses the one in makepackrat.py. Even if you want to use the class in parsing.py, it takes an argument "check_for_left_recursion" which you can try setting to False. Well, it's probably obvious to you; if so, sorry not to be more useful. Armin. From cfbolz at gmx.de Mon May 9 10:20:18 2011 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Mon, 09 May 2011 10:20:18 +0200 Subject: [pypy-dev] rlb.parsing left recursion support ? In-Reply-To: References: Message-ID: <4DC7A3C2.5090500@gmx.de> On 05/08/2011 02:53 PM, Gustavo Kuerten wrote: > Armin, > > My example is not equivalent to the actual test. > But here is the snippet from test_pypackrat.py > > def test_leftrecursion(self): > class parser(PackratParser): > """ > b: b 'a' | 'b'; > """ > print parser._code > p = parser("b") > res = p.b() > assert res == "b" > p = parser("bac") > res = p.b() > assert p._pos == 2 > assert res == "a" > p = parser("baaaaaaaaaaaaaac") > res = p.b() > assert p._pos == 15 > assert res == "a" > > This seems to indicate that it does support left recursion. There are actually two parser generators in rlib/parsing. This is a test for the second, experimental, one. The "official" one does not support left recursion. The second one is not stable enough to be used, and I guess I should remove it from rlib/ at some point given that I also don't plan to continue working on it. Carl Friedrich From arigo at tunes.org Mon May 9 15:46:57 2011 From: arigo at tunes.org (Armin Rigo) Date: Mon, 9 May 2011 15:46:57 +0200 Subject: [pypy-dev] This is a test message Message-ID: This is a test message, sent to both addresses. From arigo at tunes.org Mon May 9 15:46:57 2011 From: arigo at tunes.org (Armin Rigo) Date: Mon, 9 May 2011 15:46:57 +0200 Subject: [pypy-dev] This is a test message Message-ID: This is a test message, sent to both addresses. From paniq at paniq.org Mon May 9 16:39:44 2011 From: paniq at paniq.org (Leonard Ritter) Date: Mon, 9 May 2011 16:39:44 +0200 Subject: [pypy-dev] This is a test message In-Reply-To: References: Message-ID: this is a test answer, sent from both recipients. ;) On Mon, May 9, 2011 at 3:46 PM, Armin Rigo wrote: > This is a test message, sent to both addresses. > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > From paniq at paniq.org Mon May 9 16:39:44 2011 From: paniq at paniq.org (Leonard Ritter) Date: Mon, 9 May 2011 16:39:44 +0200 Subject: [pypy-dev] This is a test message In-Reply-To: References: Message-ID: this is a test answer, sent from both recipients. ;) On Mon, May 9, 2011 at 3:46 PM, Armin Rigo wrote: > This is a test message, sent to both addresses. > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > From phyo.arkarlwin at gmail.com Mon May 9 21:18:34 2011 From: phyo.arkarlwin at gmail.com (Phyo Arkar) Date: Tue, 10 May 2011 01:48:34 +0630 Subject: [pypy-dev] moving to python.org ... In-Reply-To: <20110509130153.GH15987@merlinux.eu> References: <20110509130153.GH15987@merlinux.eu> Message-ID: Wow so the future of python is really happening! pypy will become python's future! Good Job! Keep it up . still many things to go. On Mon, May 9, 2011 at 7:31 PM, holger krekel wrote: > Hi all, > > We are today / currently migrating pypy mailing lists to python.org, > made possible by the great help from Ralf Hildebrandt. > > Here are the new "From" addresses: > > pypy-dev at codespeak.net -> pypy-dev at python.org > pypy-issue at codespeak.net -> pypy-issue at python.org > pypy-svn at codespeak.net -> pypy-commit at python.org > pypy-z at codespeak.net -> pypy-z at python.org > > Note the move from "pypy-svn" to the "pypy-commit" name. > > All membership information is retained but you will now get mail from > python.org so you may need to adjust your filters. > > If i do things correctly you can still send mail to the old mail > addresses for at least another week in addition to the new ones. > > best, > holger > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ondrej at certik.cz Mon May 9 23:48:23 2011 From: ondrej at certik.cz (Ondrej Certik) Date: Mon, 9 May 2011 14:48:23 -0700 Subject: [pypy-dev] moving to python.org ... In-Reply-To: <20110509130153.GH15987@merlinux.eu> References: <20110509130153.GH15987@merlinux.eu> Message-ID: On Mon, May 9, 2011 at 6:01 AM, holger krekel wrote: > Hi all, > > We are today / currently migrating pypy mailing lists to python.org, > made possible by the great help from Ralf Hildebrandt. > > Here are the new "From" addresses: > > ? ?pypy-dev at codespeak.net ? -> pypy-dev at python.org > ? ?pypy-issue at codespeak.net -> pypy-issue at python.org > ? ?pypy-svn at codespeak.net ? -> pypy-commit at python.org > ? ?pypy-z at codespeak.net ? ? -> pypy-z at python.org > > Note the move from "pypy-svn" to the "pypy-commit" name. > > All membership information is retained but you will now get mail from > python.org so you may need to adjust your filters. > > If i do things correctly you can still send mail to the old mail > addresses for at least another week in addition to the new ones. Wow, congratulations! It's nice to see you sticked with PyPy for so many years and really made it all the way to python.org. Ondrej From cool-rr at cool-rr.com Tue May 10 00:36:09 2011 From: cool-rr at cool-rr.com (cool-RR) Date: Mon, 9 May 2011 18:36:09 -0400 Subject: [pypy-dev] Can't import pyc file Message-ID: Is it true that Pypy can't import a `.pyc` file? Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From amauryfa at gmail.com Tue May 10 00:57:31 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 10 May 2011 00:57:31 +0200 Subject: [pypy-dev] Can't import pyc file In-Reply-To: References: Message-ID: 2011/5/10 cool-RR : > Is it true that Pypy can't import a `.pyc` file? Yes, Pypy refuses to import a .pyc file if there is no corresponding .py file in the same directory. -- Amaury Forgeot d'Arc From stefan_ml at behnel.de Tue May 10 07:54:00 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 10 May 2011 07:54:00 +0200 Subject: [pypy-dev] moving to python.org ... In-Reply-To: References: <20110509130153.GH15987@merlinux.eu> Message-ID: <4DC8D2F8.9090203@behnel.de> Ondrej Certik, 09.05.2011 23:48: > On Mon, May 9, 2011 at 6:01 AM, holger krekel wrote: >> We are today / currently migrating pypy mailing lists to python.org, >> made possible by the great help from Ralf Hildebrandt. >> >> Here are the new "From" addresses: >> >> pypy-dev at codespeak.net -> pypy-dev at python.org >> pypy-issue at codespeak.net -> pypy-issue at python.org >> pypy-svn at codespeak.net -> pypy-commit at python.org >> pypy-z at codespeak.net -> pypy-z at python.org >> >> Note the move from "pypy-svn" to the "pypy-commit" name. >> >> All membership information is retained but you will now get mail from >> python.org so you may need to adjust your filters. >> >> If i do things correctly you can still send mail to the old mail >> addresses for at least another week in addition to the new ones. > > Wow, congratulations! It's nice to see you sticked with PyPy for so > many years and really made it all the way to python.org. Oh, that's just e-mail addresses. Cython-dev has had a python.org address for a couple of months now. It's not like Python moved to pypy.org or something. Stefan From ondrej at certik.cz Tue May 10 08:11:30 2011 From: ondrej at certik.cz (Ondrej Certik) Date: Mon, 9 May 2011 23:11:30 -0700 Subject: [pypy-dev] moving to python.org ... In-Reply-To: <4DC8D2F8.9090203@behnel.de> References: <20110509130153.GH15987@merlinux.eu> <4DC8D2F8.9090203@behnel.de> Message-ID: On Mon, May 9, 2011 at 10:54 PM, Stefan Behnel wrote: > Ondrej Certik, 09.05.2011 23:48: >> >> On Mon, May 9, 2011 at 6:01 AM, holger krekel wrote: >>> >>> We are today / currently migrating pypy mailing lists to python.org, >>> made possible by the great help from Ralf Hildebrandt. >>> >>> Here are the new "From" addresses: >>> >>> ? ?pypy-dev at codespeak.net ? -> ?pypy-dev at python.org >>> ? ?pypy-issue at codespeak.net -> ?pypy-issue at python.org >>> ? ?pypy-svn at codespeak.net ? -> ?pypy-commit at python.org >>> ? ?pypy-z at codespeak.net ? ? -> ?pypy-z at python.org >>> >>> Note the move from "pypy-svn" to the "pypy-commit" name. >>> >>> All membership information is retained but you will now get mail from >>> python.org so you may need to adjust your filters. >>> >>> If i do things correctly you can still send mail to the old mail >>> addresses for at least another week in addition to the new ones. >> >> Wow, congratulations! It's nice to see you sticked with PyPy for so >> many years and really made it all the way to python.org. > > Oh, that's just e-mail addresses. Cython-dev has had a python.org address > for a couple of months now. It's not like Python moved to pypy.org or > something. I understand. What I mean is that it's a great recognition. For Cython of course as well. Ondrej From anto.cuni at gmail.com Tue May 10 09:53:16 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 10 May 2011 09:53:16 +0200 Subject: [pypy-dev] Can't import pyc file In-Reply-To: References: Message-ID: <4DC8EEEC.9000307@gmail.com> On 10/05/11 00:36, cool-RR wrote: > Is it true that Pypy can't import a `.pyc` file? yes, by default pyc files which don't have the corresponding py file are ignored. You can change the behavior by translating pypy by yourself and using the --objspace-lonepycfiles option: $ ./translate.py -Ojit targetpypystandalone --objspace-lonepycfiles ciao, Anto From cool-rr at cool-rr.com Tue May 10 10:27:14 2011 From: cool-rr at cool-rr.com (cool-RR) Date: Tue, 10 May 2011 04:27:14 -0400 Subject: [pypy-dev] Can't import pyc file In-Reply-To: <4DC8EEEC.9000307@gmail.com> References: <4DC8EEEC.9000307@gmail.com> Message-ID: On Tue, May 10, 2011 at 3:53 AM, Antonio Cuni wrote: > On 10/05/11 00:36, cool-RR wrote: > > Is it true that Pypy can't import a `.pyc` file? > > yes, by default pyc files which don't have the corresponding py file are > ignored. > > You can change the behavior by translating pypy by yourself and using the > --objspace-lonepycfiles option: > > $ ./translate.py -Ojit targetpypystandalone --objspace-lonepycfiles > > ciao, > Anto > Is it possible to make an existing Pypy installation accept lone pyc files with some flag or something? Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From anto.cuni at gmail.com Tue May 10 10:35:59 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 10 May 2011 10:35:59 +0200 Subject: [pypy-dev] Can't import pyc file In-Reply-To: References: <4DC8EEEC.9000307@gmail.com> Message-ID: <4DC8F8EF.9020202@gmail.com> On 10/05/11 10:27, cool-RR wrote: > Is it possible to make an existing Pypy installation accept lone pyc files > with some flag or something? no, it's only a translation-time option, sorry. From cool-rr at cool-rr.com Tue May 10 10:57:34 2011 From: cool-rr at cool-rr.com (cool-RR) Date: Tue, 10 May 2011 04:57:34 -0400 Subject: [pypy-dev] Where are the constants? Message-ID: Hello, On the IRC channel arigato showed me how to set the size of Pypy's nursery for testing. But I forgot and I couldn't find it on Google. Is there a page containing all the Pypy constants? If not, what's the constant for Pypy's nursery size? Thanks, Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From amauryfa at gmail.com Tue May 10 11:07:32 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 10 May 2011 11:07:32 +0200 Subject: [pypy-dev] Where are the constants? In-Reply-To: References: Message-ID: Hi, Le mardi 10 mai 2011, cool-RR a ?crit?: > Hello, > On the IRC channel arigato showed me how to set the size of Pypy's nursery for testing. But I forgot and I couldn't find it on Google. Is there a page containing all the Pypy constants? If not, what's the constant for Pypy's nursery size? > See the comments on top of https://bitbucket.org/pypy/pypy/src/8dfcde3a5e0d/pypy/rpython/memory/gc/minimark.py -- Amaury -- Amaury Forgeot d'Arc From vincent.legoll at gmail.com Tue May 10 11:17:22 2011 From: vincent.legoll at gmail.com (Vincent Legoll) Date: Tue, 10 May 2011 11:17:22 +0200 Subject: [pypy-dev] Can't import pyc file In-Reply-To: <4DC8F8EF.9020202@gmail.com> References: <4DC8EEEC.9000307@gmail.com> <4DC8F8EF.9020202@gmail.com> Message-ID: On Tue, May 10, 2011 at 10:35 AM, Antonio Cuni wrote: > On 10/05/11 10:27, cool-RR wrote: > >> Is it possible to make an existing Pypy installation accept lone pyc files >> with some flag or something? > > no, it's only a translation-time option, sorry. Is there a particular reason this is done this way (performance maybe) ? -- Vincent Legoll From arigo at tunes.org Tue May 10 11:20:01 2011 From: arigo at tunes.org (Armin Rigo) Date: Tue, 10 May 2011 11:20:01 +0200 Subject: [pypy-dev] Can't import pyc file In-Reply-To: <4DC8F8EF.9020202@gmail.com> References: <4DC8EEEC.9000307@gmail.com> <4DC8F8EF.9020202@gmail.com> Message-ID: Hi, On Tue, May 10, 2011 at 10:35 AM, Antonio Cuni wrote: >> Is it possible to make an existing Pypy installation accept lone pyc files >> with some flag or something? > > no, it's only a translation-time option, sorry. Well, if the question is "is it possible to change pypy so that..." then the answer is of course yes. But before you want that, note that you will run into all kinds of troubles doing that, including the fact that the details of our .pyc files are slightly different than CPython's. A bient?t, Armin. From amauryfa at gmail.com Tue May 10 11:24:34 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 10 May 2011 11:24:34 +0200 Subject: [pypy-dev] Can't import pyc file In-Reply-To: References: <4DC8EEEC.9000307@gmail.com> <4DC8F8EF.9020202@gmail.com> Message-ID: Le mardi 10 mai 2011, Vincent Legoll a ?crit?: > On Tue, May 10, 2011 at 10:35 AM, Antonio Cuni wrote: >> On 10/05/11 10:27, cool-RR wrote: >> >>> Is it possible to make an existing Pypy installation accept lone pyc files >>> with some flag or something? >> >> no, it's only a translation-time option, sorry. > > Is there a particular reason this is done this way (performance maybe) ? http://readthedocs.org/docs/pypy/latest/config/objspace.lonepycfiles.html -- Amaury -- Amaury Forgeot d'Arc From anto.cuni at gmail.com Tue May 10 11:33:09 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 10 May 2011 11:33:09 +0200 Subject: [pypy-dev] Can't import pyc file In-Reply-To: References: <4DC8EEEC.9000307@gmail.com> <4DC8F8EF.9020202@gmail.com> Message-ID: <4DC90655.9080304@gmail.com> On 10/05/11 11:20, Armin Rigo wrote: > Well, if the question is "is it possible to change pypy so that..." > then the answer is of course yes. But before you want that, note that > you will run into all kinds of troubles doing that, including the fact > that the details of our .pyc files are slightly different than > CPython's. also, if the reason behind that is to distribute an application without distributing the sources, note that in principle it is possible to modify pypy to directly include the compiled modules inside the executable, making it even (slightly) harder to decompile. ciao, Anto From holger at merlinux.eu Tue May 10 12:43:41 2011 From: holger at merlinux.eu (holger krekel) Date: Tue, 10 May 2011 10:43:41 +0000 Subject: [pypy-dev] pypy mailman passwords Message-ID: <20110510104341.GZ15987@merlinux.eu> Hi all, the mailman passwords on python.org are changed compared to codespeak.net. In case you need it you need to trigger sending it: http://mail.python.org/mailman/options/pypy-dev Also, people who had unsubscribed or configured daily digests need to re-do it as we did not transfer those settings, either. best, holger From wilk at flibuste.net Tue May 10 13:31:06 2011 From: wilk at flibuste.net (William) Date: Tue, 10 May 2011 11:31:06 +0000 (UTC) Subject: [pypy-dev] pypy mailman passwords References: <20110510104341.GZ15987@merlinux.eu> Message-ID: Hi, Is there a change to do when we use gmane ? thx -- William Dod? - http://flibuste.net Informaticien Ind?pendant From holger at merlinux.eu Tue May 10 13:37:13 2011 From: holger at merlinux.eu (holger krekel) Date: Tue, 10 May 2011 11:37:13 +0000 Subject: [pypy-dev] pypy mailman passwords In-Reply-To: References: <20110510104341.GZ15987@merlinux.eu> Message-ID: <20110510113713.GD15987@merlinux.eu> On Tue, May 10, 2011 at 11:31 +0000, William wrote: > Hi, > > Is there a change to do when we use gmane ? i am not sure. Did you post this via gmane? :) If there are problems i guess we could re-register the new location with gmane. holger From wilk at flibuste.net Tue May 10 13:44:23 2011 From: wilk at flibuste.net (William) Date: Tue, 10 May 2011 11:44:23 +0000 (UTC) Subject: [pypy-dev] pypy mailman passwords References: <20110510104341.GZ15987@merlinux.eu> <20110510113713.GD15987@merlinux.eu> Message-ID: On 10-05-2011, holger krekel wrote: > On Tue, May 10, 2011 at 11:31 +0000, William wrote: >> Hi, >> >> Is there a change to do when we use gmane ? > > i am not sure. Did you post this via gmane? :) yes and i receive all the messages also > > If there are problems i guess we could re-register > the new location with gmane. > > holger -- William Dod? - http://flibuste.net Informaticien Ind?pendant From cool-rr at cool-rr.com Tue May 10 18:50:21 2011 From: cool-rr at cool-rr.com (cool-RR) Date: Tue, 10 May 2011 12:50:21 -0400 Subject: [pypy-dev] Functions `.__module__` attribute Message-ID: Hey, It seems that in CPython a function gets a `.__module__` attribute according to the current value of `__name__` when the function is being defined. In Pypy this seems not to be working, since I changed `__name__` in globals (as part of a test) but it still didn't change the function's `.__module__`. Is this a deficiency in Pypy? -- Sincerely, Ram Rachum -------------- next part -------------- An HTML attachment was scrubbed... URL: From cool-rr at cool-rr.com Tue May 10 18:59:55 2011 From: cool-rr at cool-rr.com (cool-RR) Date: Tue, 10 May 2011 12:59:55 -0400 Subject: [pypy-dev] Where are the constants? In-Reply-To: References: Message-ID: Thanks! Ram. On Tue, May 10, 2011 at 5:07 AM, Amaury Forgeot d'Arc wrote: > Hi, > > Le mardi 10 mai 2011, cool-RR a ?crit : > > Hello, > > On the IRC channel arigato showed me how to set the size of Pypy's > nursery for testing. But I forgot and I couldn't find it on Google. Is there > a page containing all the Pypy constants? If not, what's the constant for > Pypy's nursery size? > > > > See the comments on top of > > https://bitbucket.org/pypy/pypy/src/8dfcde3a5e0d/pypy/rpython/memory/gc/minimark.py > > -- > Amaury -------------- next part -------------- An HTML attachment was scrubbed... URL: From anto.cuni at gmail.com Tue May 10 20:18:23 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 10 May 2011 20:18:23 +0200 Subject: [pypy-dev] Functions `.__module__` attribute In-Reply-To: References: Message-ID: <4DC9816F.7010707@gmail.com> On 10/05/11 18:50, cool-RR wrote: > Hey, > > It seems that in CPython a function gets a `.__module__` attribute according > to the current value of `__name__` when the function is being defined. In Pypy > this seems not to be working, since I changed `__name__` in globals (as part > of a test) but it still didn't change the function's `.__module__`. Is this a > deficiency in Pypy? could you please write the smallest possible test that shows the behavior? The source of the problem might be that pypy's Function object caches its w_module attribute: so, if you get func.__module__ and *then* change __name__, the modification is not seen. Also, it seems that CPython sets the module name at function-definition time, while pypy does it only the first time that __module__ is actually got. So, for example, the following code: def one(): pass def two(): pass print one.__module__, __name__ = 'foo' print two.__module__ prints '__main__ __main__' on cpython, and '__main__ foo' on pypy. Not sure whether it's a bug or an implementation detail. ciao, Anto From cool-rr at cool-rr.com Tue May 10 20:39:23 2011 From: cool-rr at cool-rr.com (cool-RR) Date: Tue, 10 May 2011 14:39:23 -0400 Subject: [pypy-dev] Functions `.__module__` attribute In-Reply-To: <4DC9816F.7010707@gmail.com> References: <4DC9816F.7010707@gmail.com> Message-ID: On Tue, May 10, 2011 at 2:18 PM, Antonio Cuni wrote: > On 10/05/11 18:50, cool-RR wrote: > > Hey, > > > > It seems that in CPython a function gets a `.__module__` attribute > according > > to the current value of `__name__` when the function is being defined. In > Pypy > > this seems not to be working, since I changed `__name__` in globals (as > part > > of a test) but it still didn't change the function's `.__module__`. Is > this a > > deficiency in Pypy? > > could you please write the smallest possible test that shows the behavior? > > The source of the problem might be that pypy's Function object caches its > w_module attribute: so, if you get func.__module__ and *then* change > __name__, > the modification is not seen. > > Also, it seems that CPython sets the module name at function-definition > time, > while pypy does it only the first time that __module__ is actually got. So, > for example, the following code: > > def one(): pass > def two(): pass > print one.__module__, > __name__ = 'foo' > print two.__module__ > > prints '__main__ __main__' on cpython, and '__main__ foo' on pypy. > > Not sure whether it's a bug or an implementation detail. > > ciao, > Anto > Ah, then my problem is easily solved by accessing `my_function.__module__` before I change `__name__` back to its original value. I tested and it works. I can still make a test case if you have interest in it. Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cool-rr at cool-rr.com Tue May 10 20:47:18 2011 From: cool-rr at cool-rr.com (cool-RR) Date: Tue, 10 May 2011 14:47:18 -0400 Subject: [pypy-dev] Functions `.__module__` attribute In-Reply-To: <4DC986E1.1090604@gmail.com> References: <4DC9816F.7010707@gmail.com> <4DC986E1.1090604@gmail.com> Message-ID: On Tue, May 10, 2011 at 2:41 PM, Antonio Cuni wrote: > On 10/05/11 20:39, cool-RR wrote: > > Ah, then my problem is easily solved by accessing > `my_function.__module__` > > before I change `__name__` back to its original value. I tested and it > works. > > by the way, what do you need it for? > For this test: https://github.com/cool-RR/GarlicSim/blob/master/garlicsim/test_garlicsim/test_general_misc/test_address_tools/test_describe.py#L270 > > I think that, whatever you are doing, the best way is to explicitly assign > func.__module__ to the value you want. > Maybe, if this issue will ever become relevant again I'll consider it. Thanks, Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From anto.cuni at gmail.com Tue May 10 20:41:37 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 10 May 2011 20:41:37 +0200 Subject: [pypy-dev] Functions `.__module__` attribute In-Reply-To: References: <4DC9816F.7010707@gmail.com> Message-ID: <4DC986E1.1090604@gmail.com> On 10/05/11 20:39, cool-RR wrote: > Ah, then my problem is easily solved by accessing `my_function.__module__` > before I change `__name__` back to its original value. I tested and it works. by the way, what do you need it for? I think that, whatever you are doing, the best way is to explicitly assign func.__module__ to the value you want. ciao, Anto From arigo at tunes.org Tue May 10 22:55:34 2011 From: arigo at tunes.org (Armin Rigo) Date: Tue, 10 May 2011 22:55:34 +0200 Subject: [pypy-dev] Functions `.__module__` attribute In-Reply-To: References: <4DC9816F.7010707@gmail.com> <4DC986E1.1090604@gmail.com> Message-ID: Hi, That's a complete implementation detail, but on the other hand, there is no reason I can think of for PyPy to do something different than CPython, so I guess it should just be fixed. A bient?t, Armin. From zejn at kiberpipa.org Tue May 10 23:02:10 2011 From: zejn at kiberpipa.org (Gasper Zejn) Date: Tue, 10 May 2011 23:02:10 +0200 Subject: [pypy-dev] PyPy and GeoDjango Message-ID: <201105102302.11078.zejn@kiberpipa.org> Seeing that there's a postgresql adapter written for PyPy [1], I decided to do a test-drive on a pet Django project. Had I not used GeoDjango, it would have probably worked right away. I didn't need to change much: Django's PostGIS database adapter uses a psycopg extension, psycopg2.extensions.ISQLQuote, which doesn't really provide anything useful. However, I'm guessing due to implementation difference, original psycopg calls `__conforms__` with extra parameter, while rpython implementation does not - that's why there's an `*args`. The other problem was that CPython ctypes and PyPy ctypes implementations differ: where CPython uses `PTR._obj.value`, PyPy uses `PTR.contents.value`. The attached patch made my project work both on CPython and PyPy. Regards, Gasper Zejn [1] https://bitbucket.org/alex_gaynor/pypy-postgresql/overview -------------- next part -------------- A non-text attachment was scrubbed... Name: pypy_geodjango_ctypes.diff Type: text/x-patch Size: 2698 bytes Desc: not available URL: From alex.gaynor at gmail.com Tue May 10 23:18:43 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Tue, 10 May 2011 17:18:43 -0400 Subject: [pypy-dev] PyPy and GeoDjango In-Reply-To: <201105102302.11078.zejn@kiberpipa.org> References: <201105102302.11078.zejn@kiberpipa.org> Message-ID: On Tue, May 10, 2011 at 5:02 PM, Gasper Zejn wrote: > > Seeing that there's a postgresql adapter written for PyPy [1], I decided to > do > a test-drive on a pet Django project. Had I not used GeoDjango, it would > have > probably worked right away. > > > I didn't need to change much: Django's PostGIS database adapter uses a > psycopg > extension, psycopg2.extensions.ISQLQuote, which doesn't really provide > anything useful. However, I'm guessing due to implementation difference, > original psycopg calls `__conforms__` with extra parameter, while rpython > implementation does not - that's why there's an `*args`. > > The other problem was that CPython ctypes and PyPy ctypes implementations > differ: where CPython uses `PTR._obj.value`, PyPy uses > `PTR.contents.value`. > > > The attached patch made my project work both on CPython and PyPy. > > > Regards, > Gasper Zejn > > > [1] https://bitbucket.org/alex_gaynor/pypy-postgresql/overview > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > > The __conform__ thing is a bug in my psycogp2, I'll fix that so it'd be innapropriate to apply that to django. The other changes look like Django relying on internal details of ctypes, the correct fix isn't to add a special case for another ctypes impl. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal at bendowski.pl Wed May 11 23:02:54 2011 From: michal at bendowski.pl (=?UTF-8?Q?Micha=C5=82_Bendowski?=) Date: Wed, 11 May 2011 23:02:54 +0200 Subject: [pypy-dev] JVM backend - status and future work possibilities? Message-ID: Welcome everyone :) I was wondering what is the current status of the JVM backend. I'm asking because I am considering working on it for my master's thesis - I was inspired by a call to action from the "Features" page of pypy.org. Unfortunately it's unclear to me how much work needs to be done here. I understand that RPython translation is already implemented (although might need some care, for the build process to succeed more easily[1]), but the JIT backend is missing. There is also no integration with JVM libraries - am I right? On the other hand there already is a "reference" CLI implementation and a whole thesis about it. I'm not familiar with pypy internals at the moment, nor am I a compiler expert (yet!). I will be very grateful for any advice, as to whether taking care of the JVM backend is a suitable amount of work, and whether it can "scale", i.e. be divided into smaller parts that I can implement along the way and stop if I'm out of time (I imagine I can always find more work in PyPy). Thank you, Micha? Bendowski [1] I was able to build it on Linux from the release-1.5 tag revision. It fails to build on tip or Mac OS X. Then it can execute some code, but crashes on pystone.main(10). To be honest, the CLI backend seems to behave the same with regards to building and pystone. From berdario at gmail.com Wed May 11 23:56:53 2011 From: berdario at gmail.com (Dario Bertini) Date: Wed, 11 May 2011 23:56:53 +0200 Subject: [pypy-dev] JVM backend - status and future work possibilities? In-Reply-To: References: Message-ID: 2011/5/11 Micha? Bendowski : > Welcome everyone :) > > I was wondering what is the current status of the JVM backend. I'm > asking because I am considering working on it for my master's thesis - > I was inspired by a call to action from the "Features" page of > pypy.org. Unfortunately it's unclear to me how much work needs to be > done here. I understand that RPython translation is already > implemented (although might need some care, for the build process to > succeed more easily[1]), but the JIT backend is missing. There is also > no integration with JVM libraries - am I right? On the other hand > there already is a "reference" CLI implementation and a whole thesis > about it. Hi, I'm in a similar situation here :) I'm planning as well to work on the jvm backend for my master thesis: I contacted just today one of my professors who seems to be willing to mentor me, but I haven't started any work yet the integration with JVM libraries basically is a prerequisite for starting the work on the JIT backend, I'm not sure and I don't actually know your university program, but here at my university for the master thesis we have something like 15 ects... so something like the integration with the JVM alone may very well be on the upper side of what effort we could be expect to put in the thesis You're right about the CLI implementation, but after a huge refactoring inside pypy it wasn't functional anymore... Antonio Cuni (who is the author of the CLI backend we're talking about) fixed it, but I think that the ability to access CLI assemblies and class still has some issues... btw, I'm very new to this project as well, and from what I gathered until now, Antonio seems to be the guy to ask to for questions about the OO type system :) As a starting point: he suggested me to start looking into JPype and Jtool http://jpype.sourceforge.net/ http://blog.meresco.org/2010/03/11/integrating-java-in-python-with-jtool/ this kind of tools are needed for calling JVM code from CPython, thus letting us to try/debug the JVM integration when running pypy interpreted on top of CPython ( pypy/bin/py.py ) > > I'm not familiar with pypy internals at the moment, nor am I a > compiler expert (yet!). I will be very grateful for any advice, as to > whether taking care of the JVM backend is a suitable amount of work, > and whether it can "scale", i.e. be divided into smaller parts that I > can implement along the way and stop if I'm out of time (I imagine I > can always find more work in PyPy). > I would also like to know if this kind of work would be feasible to tackle in parallel for 2 students like us :) (btw, my presence on the irc channel has been quite lacking nowadays... I'll try to be more present in the next days) byebye :Dario From rogerb at rogerbinns.com Thu May 12 04:21:35 2011 From: rogerb at rogerbinns.com (Roger Binns) Date: Wed, 11 May 2011 19:21:35 -0700 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: <4DC21F58.8000101@rogerbinns.com> References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> <4DBE4F7A.3030109@gmail.com> <4DBF15E0.6080005@rogerbinns.com> <4DC21F58.8000101@rogerbinns.com> Message-ID: <4DCB442F.1080800@rogerbinns.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/04/2011 08:54 PM, Roger Binns wrote: > On 05/02/2011 01:36 PM, Roger Binns wrote: >> On 05/01/2011 11:30 PM, Antonio Cuni wrote: >>> you are right, we lack such a document. Do you feel like writing it? :-) > >> I'll make an initial stab at it this week. > > Here you go: > > https://docs.google.com/document/pub?id=1Ve9B6Gz5laV63zYFOlb_Lhe8ZBIw2V1756lquott6JI I haven't heard anything since writing this or how to compile a valgrind friendly pypy ... Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk3LRCsACgkQmOOfHg372QSl1gCfRpHUrAX54fM7tZcwtirdxJv1 pUYAn0XQ6nOmbR5aUcesVUa1Z720FgJG =XO82 -----END PGP SIGNATURE----- From headius at headius.com Thu May 12 06:39:32 2011 From: headius at headius.com (Charles Oliver Nutter) Date: Wed, 11 May 2011 23:39:32 -0500 Subject: [pypy-dev] JVM backend - status and future work possibilities? In-Reply-To: References: Message-ID: <5E773F9F-1CDC-4A78-8AFB-9F380BAAA410@headius.com> FWIW I would be happy to help out with advice and direction, especially if the JVM backend could be rejiggered to take advantage of the new invokedynamic bytecode in Java 7. I have been using it for JRuby and have been very pleased with it. Anyway, I really hope the JVM backend can move forward and I am standing by to help out as I am able. - Charlie (mobile) On May 11, 2011, at 16:56, Dario Bertini wrote: > 2011/5/11 Micha? Bendowski : >> Welcome everyone :) >> >> I was wondering what is the current status of the JVM backend. I'm >> asking because I am considering working on it for my master's thesis - >> I was inspired by a call to action from the "Features" page of >> pypy.org. Unfortunately it's unclear to me how much work needs to be >> done here. I understand that RPython translation is already >> implemented (although might need some care, for the build process to >> succeed more easily[1]), but the JIT backend is missing. There is also >> no integration with JVM libraries - am I right? On the other hand >> there already is a "reference" CLI implementation and a whole thesis >> about it. > > Hi, I'm in a similar situation here :) > > I'm planning as well to work on the jvm backend for my master thesis: > I contacted just today one of my professors who seems to be willing to > mentor me, but I haven't started any work yet > > the integration with JVM libraries basically is a prerequisite for > starting the work on the JIT backend, I'm not sure and I don't > actually know your university program, but here at my university for > the master thesis we have something like 15 ects... so something like > the integration with the JVM alone may very well be on the upper side > of what effort we could be expect to put in the thesis > > You're right about the CLI implementation, but after a huge > refactoring inside pypy it wasn't functional anymore... Antonio Cuni > (who is the author of the CLI backend we're talking about) fixed it, > but I think that the ability to access CLI assemblies and class still > has some issues... > > btw, I'm very new to this project as well, and from what I gathered > until now, Antonio seems to be the guy to ask to for questions about > the OO type system :) > > As a starting point: he suggested me to start looking into JPype and Jtool > > http://jpype.sourceforge.net/ > http://blog.meresco.org/2010/03/11/integrating-java-in-python-with-jtool/ > > this kind of tools are needed for calling JVM code from CPython, thus > letting us to try/debug the JVM integration when running pypy > interpreted on top of CPython ( pypy/bin/py.py ) > >> >> I'm not familiar with pypy internals at the moment, nor am I a >> compiler expert (yet!). I will be very grateful for any advice, as to >> whether taking care of the JVM backend is a suitable amount of work, >> and whether it can "scale", i.e. be divided into smaller parts that I >> can implement along the way and stop if I'm out of time (I imagine I >> can always find more work in PyPy). >> > > I would also like to know if this kind of work would be feasible to > tackle in parallel for 2 students like us :) > > (btw, my presence on the irc channel has been quite lacking > nowadays... I'll try to be more present in the next days) > > byebye > > :Dario > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev From fijall at gmail.com Thu May 12 10:34:34 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 12 May 2011 10:34:34 +0200 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: <4DCB442F.1080800@rogerbinns.com> References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> <4DBE4F7A.3030109@gmail.com> <4DBF15E0.6080005@rogerbinns.com> <4DC21F58.8000101@rogerbinns.com> <4DCB442F.1080800@rogerbinns.com> Message-ID: On Thu, May 12, 2011 at 4:21 AM, Roger Binns wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 05/04/2011 08:54 PM, Roger Binns wrote: >> On 05/02/2011 01:36 PM, Roger Binns wrote: >>> On 05/01/2011 11:30 PM, Antonio Cuni wrote: >>>> you are right, we lack such a document. Do you feel like writing it? :-) >> >>> I'll make an initial stab at it this week. >> >> Here you go: >> >> https://docs.google.com/document/pub?id=1Ve9B6Gz5laV63zYFOlb_Lhe8ZBIw2V1756lquott6JI > > I haven't heard anything since writing this or how to compile a valgrind > friendly pypy ... 32bit pypy works out of the box on valgrind. No idea why 64bit does not > > Roger > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > > iEYEARECAAYFAk3LRCsACgkQmOOfHg372QSl1gCfRpHUrAX54fM7tZcwtirdxJv1 > pUYAn0XQ6nOmbR5aUcesVUa1Z720FgJG > =XO82 > -----END PGP SIGNATURE----- > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > From audrius at neutrino.lt Thu May 12 11:59:15 2011 From: audrius at neutrino.lt (Audrius =?utf-8?Q?Ka=C5=BEukauskas?=) Date: Thu, 12 May 2011 12:59:15 +0300 Subject: [pypy-dev] Compiling PyPy 1.5 fails on Slackware 13.37 Message-ID: <20110512095915.GA3960@kiras> Hi, I've reported this on #pypy yesterday, but I thought this might be a better place for this kind of information. I've tried to build PyPy 1.5 from source on Slackware 13.37 (32-bit LXC container, while the host and kernel are 64-bit; both systems Slackware) and got an exception[0] in the last stage of the build process (compilation). [0] http://paste.pocoo.org/show/387121/ Here are some bits of information about the system used: gcc 4.5.2, glibc 2.13. If you need anything else, I'll be happy to report it. I also still have all the .c files created by translation process, if any of those are needed, just tell me where to send them. -- Audrius Ka?ukauskas -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From fijall at gmail.com Thu May 12 12:12:15 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 12 May 2011 12:12:15 +0200 Subject: [pypy-dev] Compiling PyPy 1.5 fails on Slackware 13.37 In-Reply-To: <20110512095915.GA3960@kiras> References: <20110512095915.GA3960@kiras> Message-ID: 2011/5/12 Audrius Ka?ukauskas : > Hi, > > I've reported this on #pypy yesterday, but I thought this might be a > better place for this kind of information. > > I've tried to build PyPy 1.5 from source on Slackware 13.37 (32-bit LXC > container, while the host and kernel are 64-bit; both systems Slackware) > and got an exception[0] in the last stage of the build process > (compilation). > > [0] http://paste.pocoo.org/show/387121/ > > Here are some bits of information about the system used: gcc 4.5.2, > glibc 2.13. ?If you need anything else, I'll be happy to report it. ?I > also still have all the .c files created by translation process, if any > of those are needed, just tell me where to send them. testing_1.s or testing_1.c would be cool to have > > -- > Audrius Ka?ukauskas > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > > From audrius at neutrino.lt Thu May 12 12:20:59 2011 From: audrius at neutrino.lt (Audrius =?utf-8?Q?Ka=C5=BEukauskas?=) Date: Thu, 12 May 2011 13:20:59 +0300 Subject: [pypy-dev] Compiling PyPy 1.5 fails on Slackware 13.37 In-Reply-To: References: <20110512095915.GA3960@kiras> Message-ID: <20110512102059.GA5341@kiras> On Thu, 2011-05-12 at 12:12:15 +0200, Maciej Fijalkowski wrote: > > Here are some bits of information about the system used: gcc 4.5.2, > > glibc 2.13. ?If you need anything else, I'll be happy to report it. ?I > > also still have all the .c files created by translation process, if any > > of those are needed, just tell me where to send them. > > testing_1.s or testing_1.c would be cool to have See attached both testing_1.c and testing_1.s. -- Audrius Ka?ukauskas -------------- next part -------------- #include "common_header.h" #include "structdef.h" #include "forwarddecl.h" /***********************************************************/ /*** Implementations ***/ #define HAVE_RTYPER typedef struct pypy_rpy_string0 RPyString; typedef struct pypy_list0 RPyListOfString; #define _RPyListOfString_SetItem pypy_g__RPyListOfString_SetItem__listPtr_Signed_rpy_str #define _RPyListOfString_GetItem pypy_g__RPyListOfString_GetItem__listPtr_Signed #define RPyString_New pypy_g_RPyString_New__Signed #define _RPyListOfString_New pypy_g__RPyListOfString_New__Signed #define _RPyListOfString_Length pypy_g__RPyListOfString_Length__listPtr typedef struct pypy_object_vtable0 *RPYTHON_EXCEPTION_VTABLE; typedef struct pypy_object0 *RPYTHON_EXCEPTION; #define RPYTHON_EXCEPTION_MATCH pypy_g_ll_issubclass #define RPYTHON_TYPE_OF_EXC_INST pypy_g_ll_type__objectPtr #define RPYTHON_RAISE_OSERROR pypy_g_ll_raise_OSError__Signed #define RPyExceptionOccurred1 pypy_g_RPyExceptionOccurred #define RPyFetchExceptionType pypy_g_RPyFetchExceptionType #define RPyFetchExceptionValue pypy_g_RPyFetchExceptionValue #define RPyClearException pypy_g_RPyClearException #define RPyRaiseException pypy_g_RPyRaiseException #define RPyExc_TypeError (&pypy_g_exceptions_TypeError.te_super.se_super.e_super) #define RPyExc_OverflowError (&pypy_g_exceptions_OverflowError.oe_super.ae_super.se_super.e_super) #define RPyExc_UnicodeEncodeError (&pypy_g_exceptions_UnicodeEncodeError.uee_super.ue_super.ve_super.se_super.e_super) #define RPyExc_IndexError (&pypy_g_exceptions_IndexError.ie_super.le_super.se_super.e_super) #define RPyExc_RuntimeError (&pypy_g_exceptions_RuntimeError.re_super.se_super.e_super) #define RPyExc_MemoryError (&pypy_g_exceptions_MemoryError.me_super.se_super.e_super) #define RPyExc_UnicodeDecodeError (&pypy_g_exceptions_UnicodeDecodeError.ude_super.ue_super.ve_super.se_super.e_super) #define RPyExc_KeyError (&pypy_g_exceptions_KeyError.ke_super.le_super.se_super.e_super) #define RPyExc_NotImplementedError (&pypy_g_exceptions_NotImplementedError.nie_super.re_super.se_super.e_super) #define RPyExc_IOError (&pypy_g_exceptions_IOError.ioe_super.ee_super.se_super.e_super) #define RPyExc_ZeroDivisionError (&pypy_g_exceptions_ZeroDivisionError.zde_super.ae_super.se_super.e_super) #define RPyExc_ValueError (&pypy_g_exceptions_ValueError.ve_super.se_super.e_super) #define RPyExc_AssertionError (&pypy_g_exceptions_AssertionError.ae_super.se_super.e_super) #define RPyExc_StopIteration (&pypy_g_exceptions_StopIteration.si_super.e_super) #define RPyExc_OSError (&pypy_g_exceptions_EnvironmentError.ee_super.se_super.e_super) #define RPyExc_pypy__rlib__rstackovf_StackOverflow (&pypy_g_pypy_rlib_rstackovf_StackOverflow.so_super.re_super.se_super.e_super) #include "src/g_include.h" /* structimpl.c */ /* nonfuncnodes.c */ /* nonfuncnodes_1.c */ /* nonfuncnodes_2.c */ /* nonfuncnodes_3.c */ /* nonfuncnodes_4.c */ /* nonfuncnodes_5.c */ /* nonfuncnodes_6.c */ /* nonfuncnodes_7.c */ /* nonfuncnodes_8.c */ /* nonfuncnodes_9.c */ /* nonfuncnodes_10.c */ /* nonfuncnodes_11.c */ /* nonfuncnodes_12.c */ /* nonfuncnodes_13.c */ /* nonfuncnodes_14.c */ /* nonfuncnodes_15.c */ /* nonfuncnodes_16.c */ /* nonfuncnodes_17.c */ /* nonfuncnodes_18.c */ /* nonfuncnodes_19.c */ /* nonfuncnodes_20.c */ /* nonfuncnodes_21.c */ /* nonfuncnodes_22.c */ /* nonfuncnodes_23.c */ /* nonfuncnodes_24.c */ /* nonfuncnodes_25.c */ /* nonfuncnodes_26.c */ /* nonfuncnodes_27.c */ /* nonfuncnodes_28.c */ /* nonfuncnodes_29.c */ /* nonfuncnodes_30.c */ /* nonfuncnodes_31.c */ /* nonfuncnodes_32.c */ /* nonfuncnodes_33.c */ /* nonfuncnodes_34.c */ /* nonfuncnodes_35.c */ /* nonfuncnodes_36.c */ /* nonfuncnodes_37.c */ /* nonfuncnodes_38.c */ /* nonfuncnodes_39.c */ /* nonfuncnodes_40.c */ /* nonfuncnodes_41.c */ /* nonfuncnodes_42.c */ /* nonfuncnodes_43.c */ /* nonfuncnodes_44.c */ /* nonfuncnodes_45.c */ /* nonfuncnodes_46.c */ /* nonfuncnodes_47.c */ /* implement.c */ /* implement_1.c */ /* implement_2.c */ /* implement_3.c */ /* implement_4.c */ /* implement_5.c */ /* implement_6.c */ /* implement_7.c */ /* implement_8.c */ /* implement_9.c */ /* implement_10.c */ /* implement_11.c */ /* implement_12.c */ /* implement_13.c */ /* implement_14.c */ /* implement_15.c */ /* implement_16.c */ /* implement_17.c */ /* implement_18.c */ /* implement_19.c */ /* implement_20.c */ char *RPython_StartupCode(void) { char *error = NULL; pypy_g_frameworkgc_setup(); pypy_g_pypy_objspace_std_floatobject_W_FloatObject.wfo_inst_floatval = (Py_HUGE_VAL); pypy_g_tuple2_1433.t_item0 = (-Py_HUGE_VAL); pypy_g_tuple2_1434.t_item0 = (-Py_HUGE_VAL); pypy_g_tuple2_1435.t_item0 = (-Py_HUGE_VAL); pypy_g_tuple2_1436.t_item0 = (-Py_HUGE_VAL); pypy_g_tuple2_1437.t_item0 = (-Py_HUGE_VAL); pypy_g_tuple2_1437.t_item1 = (Py_HUGE_VAL/Py_HUGE_VAL); pypy_g_tuple2_1438.t_item0 = (-Py_HUGE_VAL); pypy_g_tuple2_1440.t_item0 = (-Py_HUGE_VAL); pypy_g_tuple2_1441.t_item0 = (Py_HUGE_VAL/Py_HUGE_VAL); pypy_g_tuple2_1441.t_item1 = (Py_HUGE_VAL/Py_HUGE_VAL); pypy_g_tuple2_1444.t_item0 = (Py_HUGE_VAL); pypy_g_tuple2_1447.t_item0 = (Py_HUGE_VAL); pypy_g_tuple2_1448.t_item0 = (Py_HUGE_VAL); pypy_g_tuple2_1449.t_item0 = (Py_HUGE_VAL); pypy_g_tuple2_1450.t_item0 = (Py_HUGE_VAL); pypy_g_tuple2_1451.t_item0 = (Py_HUGE_VAL); pypy_g_tuple2_1452.t_item0 = (Py_HUGE_VAL); pypy_g_tuple2_1452.t_item1 = (Py_HUGE_VAL/Py_HUGE_VAL); pypy_g_tuple2_1453.t_item0 = (Py_HUGE_VAL/Py_HUGE_VAL); pypy_g_tuple2_1454.t_item0 = (Py_HUGE_VAL/Py_HUGE_VAL); pypy_g_tuple2_1457.t_item1 = (Py_HUGE_VAL/Py_HUGE_VAL); pypy_g_tuple2_1458.t_item0 = (Py_HUGE_VAL); pypy_g_tuple2_1459.t_item0 = (Py_HUGE_VAL); pypy_g_tuple2_1460.t_item0 = (Py_HUGE_VAL); pypy_g_tuple2_1461.t_item0 = (Py_HUGE_VAL); pypy_g_tuple2_1468.t_item1 = (Py_HUGE_VAL/Py_HUGE_VAL); pypy_g_tuple2_1469.t_item0 = (Py_HUGE_VAL); pypy_g_tuple2_1469.t_item1 = (-Py_HUGE_VAL); pypy_g_tuple2_1470.t_item1 = (-Py_HUGE_VAL); pypy_g_tuple2_1471.t_item1 = (Py_HUGE_VAL); pypy_g_tuple2_1472.t_item0 = (Py_HUGE_VAL); pypy_g_tuple2_1472.t_item1 = (Py_HUGE_VAL); pypy_g_tuple2_1473.t_item0 = (Py_HUGE_VAL/Py_HUGE_VAL); pypy_g_tuple2_1473.t_item1 = (Py_HUGE_VAL); pypy_g_tuple2_1474.t_item1 = (Py_HUGE_VAL); pypy_g_tuple2_1475.t_item1 = (Py_HUGE_VAL); pypy_g_tuple2_1476.t_item1 = (-Py_HUGE_VAL); pypy_g_tuple2_1477.t_item1 = (-Py_HUGE_VAL); pypy_g_tuple2_1478.t_item1 = (Py_HUGE_VAL); pypy_g_tuple2_1479.t_item1 = (-Py_HUGE_VAL); pypy_g_tuple2_1482.t_item1 = (Py_HUGE_VAL/Py_HUGE_VAL); pypy_g_tuple2_1483.t_item1 = (Py_HUGE_VAL); pypy_g_tuple2_1484.t_item1 = (-Py_HUGE_VAL); pypy_g_tuple2_1485.t_item0 = (Py_HUGE_VAL/Py_HUGE_VAL); pypy_g_tuple2_1485.t_item1 = (-Py_HUGE_VAL); pypy_g_tuple2_1486.t_item0 = (-Py_HUGE_VAL); pypy_g_tuple2_1487.t_item0 = (-Py_HUGE_VAL); return error; } -------------- next part -------------- .file "testing_1.c" .text .p2align 4,,15 .type bootstrap_pthread, @function bootstrap_pthread: subl $12, %esp call *16(%esp) xorl %eax, %eax addl $12, %esp ret .size bootstrap_pthread, .-bootstrap_pthread .section .rodata .LC0: .string "" .string "" .text .p2align 4,,15 .type signal_setflag_handler, @function signal_setflag_handler: subl $12, %esp movl 16(%esp), %eax cmpl $64, %eax ja .L3 movb $1, pypysig_flags(%eax) movl $1, pypysig_occurred movl $-1, pypysig_counter .L3: movl wakeup_fd, %eax cmpl $-1, %eax je .L2 pushl %edx pushl $1 pushl $.LC0 pushl %eax call write addl $16, %esp .L2: addl $12, %esp ret .size signal_setflag_handler, .-signal_setflag_handler .p2align 4,,15 .globl RPyThreadGetIdent .type RPyThreadGetIdent, @function RPyThreadGetIdent: subl $28, %esp call pthread_self movl %eax, (%esp) movl (%esp), %eax addl $28, %esp ret .size RPyThreadGetIdent, .-RPyThreadGetIdent .p2align 4,,15 .globl RPyThreadStart .type RPyThreadStart, @function RPyThreadStart: pushl %esi pushl %ebx subl $64, %esp leal 20(%esp), %ebx pushl %ebx call pthread_attr_init movl _pypythread_stacksize, %eax addl $16, %esp testl %eax, %eax jne .L11 .L8: subl $8, %esp pushl $0 pushl %ebx call pthread_attr_setscope movl 80(%esp), %esi pushl %esi pushl $bootstrap_pthread pushl %ebx leal 72(%esp), %eax pushl %eax call pthread_create movl %eax, %esi addl $20, %esp pushl %ebx call pthread_attr_destroy addl $16, %esp testl %esi, %esi je .L12 movl $-1, %eax addl $52, %esp popl %ebx popl %esi ret .p2align 4,,15 .L12: subl $12, %esp movl 56(%esp), %ecx pushl %ecx call pthread_detach movl 60(%esp), %eax addl $16, %esp addl $52, %esp popl %ebx popl %esi ret .p2align 4,,15 .L11: subl $8, %esp pushl %eax pushl %ebx call pthread_attr_setstacksize addl $16, %esp jmp .L8 .size RPyThreadStart, .-RPyThreadStart .p2align 4,,15 .globl RPyThreadGetStackSize .type RPyThreadGetStackSize, @function RPyThreadGetStackSize: movl _pypythread_stacksize, %eax ret .size RPyThreadGetStackSize, .-RPyThreadGetStackSize .p2align 4,,15 .globl RPyThreadSetStackSize .type RPyThreadSetStackSize, @function RPyThreadSetStackSize: pushl %ebp pushl %edi pushl %esi pushl %ebx subl $60, %esp movl 80(%esp), %ebx testl %ebx, %ebx jne .L15 movl $0, _pypythread_stacksize xorl %esi, %esi .L16: movl %esi, %eax addl $60, %esp popl %ebx popl %esi popl %edi popl %ebp ret .p2align 4,,15 .L15: cmpl $32767, %ebx ja .L20 movl $-1, %esi jmp .L16 .L20: subl $12, %esp leal 24(%esp), %edi pushl %edi call pthread_attr_init addl $16, %esp orl $-1, %esi testl %eax, %eax jne .L16 pushl %ebp pushl %ebp pushl %ebx pushl %edi call pthread_attr_setstacksize movl %eax, %ebp movl %edi, (%esp) call pthread_attr_destroy addl $16, %esp testl %ebp, %ebp jne .L16 movl %ebx, _pypythread_stacksize xorl %esi, %esi jmp .L16 .size RPyThreadSetStackSize, .-RPyThreadSetStackSize .p2align 4,,15 .globl RPyThreadAfterFork .type RPyThreadAfterFork, @function RPyThreadAfterFork: ret .size RPyThreadAfterFork, .-RPyThreadAfterFork .section .rodata.str1.1,"aMS", at progbits,1 .LC1: .string "sem_init" .text .p2align 4,,15 .globl RPyThreadLockInit .type RPyThreadLockInit, @function RPyThreadLockInit: pushl %ebx subl $12, %esp movl 20(%esp), %ebx movl $0, 16(%ebx) pushl $1 pushl $0 pushl %ebx call sem_init addl $16, %esp testl %eax, %eax jne .L25 movl $1, 16(%ebx) movl $1, %eax addl $8, %esp popl %ebx ret .p2align 4,,15 .L25: subl $12, %esp pushl $.LC1 call perror addl $16, %esp xorl %eax, %eax addl $8, %esp popl %ebx ret .size RPyThreadLockInit, .-RPyThreadLockInit .section .rodata.str1.1 .LC2: .string "sem_destroy" .text .p2align 4,,15 .globl RPyOpaqueDealloc_ThreadLock .type RPyOpaqueDealloc_ThreadLock, @function RPyOpaqueDealloc_ThreadLock: subl $12, %esp movl 16(%esp), %eax movl 16(%eax), %edx testl %edx, %edx jne .L28 .L26: addl $12, %esp ret .p2align 4,,15 .L28: subl $12, %esp pushl %eax call sem_destroy addl $16, %esp testl %eax, %eax je .L26 movl $.LC2, 16(%esp) addl $12, %esp jmp perror .size RPyOpaqueDealloc_ThreadLock, .-RPyOpaqueDealloc_ThreadLock .section .rodata.str1.1 .LC3: .string "sem_wait" .LC4: .string "sem_trywait" .text .p2align 4,,15 .globl RPyThreadAcquireLock .type RPyThreadAcquireLock, @function RPyThreadAcquireLock: pushl %ebx subl $24, %esp movl 32(%esp), %ebx movl 36(%esp), %ecx testl %ecx, %ecx jne .L38 jmp .L37 .p2align 4,,15 .L31: cmpl $4, %eax jne .L42 .L37: subl $12, %esp pushl %ebx call sem_trywait addl $16, %esp cmpl $-1, %eax jne .L31 call __errno_location movl (%eax), %eax cmpl $4, %eax je .L37 .p2align 4,,15 .L42: cmpl $11, %eax je .L35 testl %eax, %eax jne .L43 .L35: testl %eax, %eax sete %al andl $255, %eax addl $24, %esp popl %ebx ret .p2align 4,,15 .L45: call __errno_location movl (%eax), %eax cmpl $4, %eax jne .L44 .p2align 4,,15 .L38: subl $12, %esp pushl %ebx call sem_wait addl $16, %esp cmpl $-1, %eax je .L45 cmpl $4, %eax je .L38 .L44: testl %eax, %eax je .L35 subl $12, %esp pushl $.LC3 movl %eax, 28(%esp) call perror addl $16, %esp movl 12(%esp), %eax testl %eax, %eax sete %al andl $255, %eax addl $24, %esp popl %ebx ret .p2align 4,,15 .L43: subl $12, %esp pushl $.LC4 movl %eax, 28(%esp) call perror addl $16, %esp movl 12(%esp), %eax testl %eax, %eax sete %al andl $255, %eax addl $24, %esp popl %ebx ret .size RPyThreadAcquireLock, .-RPyThreadAcquireLock .section .rodata.str1.1 .LC5: .string "sem_post" .text .p2align 4,,15 .globl RPyThreadReleaseLock .type RPyThreadReleaseLock, @function RPyThreadReleaseLock: subl $24, %esp movl 28(%esp), %eax pushl %eax call sem_post addl $16, %esp testl %eax, %eax jne .L48 addl $12, %esp ret .p2align 4,,15 .L48: movl $.LC5, 16(%esp) addl $12, %esp jmp perror .size RPyThreadReleaseLock, .-RPyThreadReleaseLock .section .rodata.str1.4,"aMS", at progbits,1 .align 4 .LC6: .string "out of thread-local storage keys" .text .p2align 4,,15 .globl RPyThreadTLS_Create .type RPyThreadTLS_Create, @function RPyThreadTLS_Create: subl $20, %esp pushl $0 movl 28(%esp), %edx pushl %edx call pthread_key_create movl %eax, %edx xorl %eax, %eax testl %edx, %edx movl $.LC6, %edx cmovne %edx, %eax addl $28, %esp ret .size RPyThreadTLS_Create, .-RPyThreadTLS_Create .p2align 4,,15 .globl RPyThreadYield .type RPyThreadYield, @function RPyThreadYield: ret .size RPyThreadYield, .-RPyThreadYield .p2align 4,,15 .globl pypysig_getaddr_occurred .type pypysig_getaddr_occurred, @function pypysig_getaddr_occurred: movl $pypysig_counter, %eax ret .size pypysig_getaddr_occurred, .-pypysig_getaddr_occurred .p2align 4,,15 .globl pypysig_ignore .type pypysig_ignore, @function pypysig_ignore: pushl %ebx subl $164, %esp movl $1, 16(%esp) leal 16(%esp), %ebx leal 20(%esp), %eax pushl %eax call sigemptyset movl $0, 152(%esp) addl $12, %esp pushl $0 pushl %ebx movl 172(%esp), %ecx pushl %ecx call sigaction addl $168, %esp popl %ebx ret .size pypysig_ignore, .-pypysig_ignore .p2align 4,,15 .globl pypysig_default .type pypysig_default, @function pypysig_default: pushl %ebx subl $164, %esp movl $0, 16(%esp) leal 16(%esp), %ebx leal 20(%esp), %eax pushl %eax call sigemptyset movl $0, 152(%esp) addl $12, %esp pushl $0 pushl %ebx movl 172(%esp), %ebx pushl %ebx call sigaction addl $168, %esp popl %ebx ret .size pypysig_default, .-pypysig_default .p2align 4,,15 .globl pypysig_setflag .type pypysig_setflag, @function pypysig_setflag: pushl %ebx subl $164, %esp movl $signal_setflag_handler, 16(%esp) leal 16(%esp), %ebx leal 20(%esp), %eax pushl %eax call sigemptyset movl $0, 152(%esp) addl $12, %esp pushl $0 pushl %ebx movl 172(%esp), %eax pushl %eax call sigaction addl $168, %esp popl %ebx ret .size pypysig_setflag, .-pypysig_setflag .p2align 4,,15 .globl pypysig_reinstall .type pypysig_reinstall, @function pypysig_reinstall: ret .size pypysig_reinstall, .-pypysig_reinstall .p2align 4,,15 .globl pypysig_poll .type pypysig_poll, @function pypysig_poll: movl pypysig_occurred, %eax testl %eax, %eax jne .L64 movl $-1, %eax ret .p2align 4,,15 .L64: movl $0, pypysig_occurred xorl %eax, %eax jmp .L61 .p2align 4,,15 .L60: incl %eax cmpl $65, %eax je .L65 .L61: movb pypysig_flags(%eax), %dl testb %dl, %dl je .L60 movb $0, pypysig_flags(%eax) movl $1, pypysig_occurred ret .p2align 4,,15 .L65: movl $-1, %eax ret .size pypysig_poll, .-pypysig_poll .p2align 4,,15 .globl pypysig_set_wakeup_fd .type pypysig_set_wakeup_fd, @function pypysig_set_wakeup_fd: movl wakeup_fd, %eax movl 4(%esp), %edx movl %edx, wakeup_fd ret .size pypysig_set_wakeup_fd, .-pypysig_set_wakeup_fd .p2align 4,,15 .globl _LL_stack_growing_direction .type _LL_stack_growing_direction, @function _LL_stack_growing_direction: subl $28, %esp movl 32(%esp), %edx testl %edx, %edx je .L70 leal 15(%esp), %eax subl %edx, %eax .L69: addl $28, %esp ret .p2align 4,,15 .L70: subl $12, %esp leal 27(%esp), %eax pushl %eax call _LL_stack_growing_direction addl $16, %esp jmp .L69 .size _LL_stack_growing_direction, .-_LL_stack_growing_direction .p2align 4,,15 .globl LL_stack_too_big_slowpath .type LL_stack_too_big_slowpath, @function LL_stack_too_big_slowpath: pushl %ebx subl $8, %esp movl 16(%esp), %ebx movl stack_direction, %edx testl %edx, %edx je .L83 .L72: movl %gs:start_tls_key at ntpoff, %eax testl %eax, %eax je .L74 .L85: movl %ebx, %ecx subl %eax, %ecx cmpl $786431, %ecx jbe .L84 testl %edx, %edx jle .L77 addl $786431, %ecx cmpl $786430, %ecx ja .L82 .L78: movl %ebx, %gs:start_tls_key at ntpoff movl %ebx, _LLstacktoobig_stack_start xorl %eax, %eax addl $8, %esp popl %ebx ret .p2align 4,,15 .L84: movl %eax, _LLstacktoobig_stack_start xorl %eax, %eax addl $8, %esp popl %ebx ret .p2align 4,,15 .L83: subl $12, %esp pushl $0 call _LL_stack_growing_direction addl $16, %esp testl %eax, %eax jle .L73 movl $1, stack_direction movl $1, %edx movl %gs:start_tls_key at ntpoff, %eax testl %eax, %eax jne .L85 .L74: testl %edx, %edx jg .L78 jmp .L79 .p2align 4,,15 .L82: movb $1, %al addl $8, %esp popl %ebx ret .p2align 4,,15 .L77: subl $786432, %ecx cmpl $786431, %ecx ja .L82 .L79: subl $786431, %ebx jmp .L78 .p2align 4,,15 .L73: movl $-1, stack_direction movl $-1, %edx jmp .L72 .size LL_stack_too_big_slowpath, .-LL_stack_too_big_slowpath .p2align 4,,15 .globl _RPyRaiseSimpleException .type _RPyRaiseSimpleException, @function _RPyRaiseSimpleException: pushl %ebx subl $20, %esp movl 28(%esp), %ebx pushl %ebx call pypy_g_ll_type__objectPtr popl %edx popl %ecx pushl %ebx pushl %eax call pypy_g_RPyRaiseException addl $24, %esp popl %ebx ret .size _RPyRaiseSimpleException, .-_RPyRaiseSimpleException .p2align 4,,15 .globl op_llong_mul_ovf .type op_llong_mul_ovf, @function op_llong_mul_ovf: pushl %ebp pushl %edi pushl %esi pushl %ebx subl $44, %esp movl 64(%esp), %ecx movl 68(%esp), %ebx movl 72(%esp), %edi movl 76(%esp), %ebp movl %ebp, %esi imull %ecx, %esi movl %ebx, %eax imull %edi, %eax addl %eax, %esi movl %edi, %eax mull %ecx movl %eax, (%esp) addl %esi, %edx movl %edx, 4(%esp) movd %ecx, %xmm1 movd %ebx, %xmm2 punpckldq %xmm2, %xmm1 movq %xmm1, 24(%esp) fildq 24(%esp) fstpl 8(%esp) movsd 8(%esp), %xmm0 movd %edi, %xmm1 movd %ebp, %xmm2 punpckldq %xmm2, %xmm1 movq %xmm1, 24(%esp) fildq 24(%esp) fstpl 16(%esp) mulsd 16(%esp), %xmm0 fildq (%esp) fstpl 8(%esp) movsd 8(%esp), %xmm1 ucomisd %xmm0, %xmm1 jnp .L95 .L93: subsd %xmm0, %xmm1 xorpd %xmm2, %xmm2 ucomisd %xmm2, %xmm1 jb .L96 ucomisd %xmm2, %xmm0 jb .L97 .L91: mulsd .LC9, %xmm1 ucomisd %xmm1, %xmm0 jb .L98 .L88: movl (%esp), %eax movl 4(%esp), %edx addl $44, %esp popl %ebx popl %esi popl %edi popl %ebp ret .p2align 4,,15 .L97: movsd .LC8, %xmm2 xorpd %xmm2, %xmm0 jmp .L91 .p2align 4,,15 .L96: movsd .LC8, %xmm3 xorpd %xmm3, %xmm1 ucomisd %xmm2, %xmm0 jae .L91 jmp .L97 .p2align 4,,15 .L95: jne .L93 movl (%esp), %eax movl 4(%esp), %edx addl $44, %esp popl %ebx popl %esi popl %edi popl %ebp ret .p2align 4,,15 .L98: subl $12, %esp pushl $pypy_g_exceptions_OverflowError call pypy_g_ll_type__objectPtr popl %ebx popl %esi pushl $pypy_g_exceptions_OverflowError pushl %eax call pypy_g_RPyRaiseException addl $16, %esp movl $-1, (%esp) movl $-1, 4(%esp) jmp .L88 .size op_llong_mul_ovf, .-op_llong_mul_ovf .p2align 4,,15 .globl instrument_setup .type instrument_setup, @function instrument_setup: ret .size instrument_setup, .-instrument_setup .section .rodata.str1.1 .LC11: .string "(out of memory!)" .text .p2align 4,,15 .globl RPyString_AsCharP .type RPyString_AsCharP, @function RPyString_AsCharP: pushl %ebp pushl %edi pushl %esi pushl %ebx subl $24, %esp movl 44(%esp), %esi movl 8(%esi), %ebx leal 8(%ebx), %eax pushl %eax call malloc addl $16, %esp testl %eax, %eax je .L105 movl _RPyString_dump, %edx movl %edx, (%eax) movl %eax, _RPyString_dump leal 4(%eax), %ebp movl %ebp, %edi addl $12, %esi cmpl $4, %ebx jae .L106 xorl %ecx, %ecx testb $2, %bl jne .L107 .L103: testb $1, %bl jne .L108 .L104: movb $0, 4(%eax,%ebx) movl %ebp, %eax addl $12, %esp popl %ebx popl %esi popl %edi popl %ebp ret .p2align 4,,15 .L108: movb (%esi,%ecx), %dl movb %dl, (%edi,%ecx) jmp .L104 .p2align 4,,15 .L107: movw (%esi), %cx movw %cx, (%edi) movl $2, %ecx testb $1, %bl je .L104 jmp .L108 .p2align 4,,15 .L106: movl %ebx, %ecx shrl $2, %ecx rep movsl xorl %ecx, %ecx testb $2, %bl je .L103 jmp .L107 .p2align 4,,15 .L105: movl $.LC11, %eax addl $12, %esp popl %ebx popl %esi popl %edi popl %ebp ret .size RPyString_AsCharP, .-RPyString_AsCharP .p2align 4,,15 .globl RPyString_FreeCache .type RPyString_FreeCache, @function RPyString_FreeCache: subl $12, %esp movl _RPyString_dump, %eax testl %eax, %eax je .L109 .p2align 4,,15 .L112: movl (%eax), %edx movl %edx, _RPyString_dump subl $12, %esp pushl %eax call free movl _RPyString_dump, %eax addl $16, %esp testl %eax, %eax jne .L112 .L109: addl $12, %esp ret .size RPyString_FreeCache, .-RPyString_FreeCache .p2align 4,,15 .globl RPyString_FromString .type RPyString_FromString, @function RPyString_FromString: pushl %edi pushl %esi pushl %ebx movl 16(%esp), %esi subl $12, %esp pushl %esi call strlen movl %eax, %ebx movl %eax, (%esp) call pypy_g_RPyString_New__Signed leal 12(%eax), %edi cmpl $4, %ebx jb .L115 movl %ebx, %ecx shrl $2, %ecx rep movsl .L115: xorl %edx, %edx testb $2, %bl je .L116 movw (%esi), %dx movw %dx, (%edi) movl $2, %edx .L116: andl $1, %ebx je .L117 movb (%esi,%edx), %cl movb %cl, (%edi,%edx) .L117: addl $16, %esp popl %ebx popl %esi popl %edi ret .size RPyString_FromString, .-RPyString_FromString .section .rodata.str1.1 .LC12: .string "RPython traceback:\n" .LC13: .string " ...\n" .LC14: .string " File \"%s\", line %d, in %s\n" .section .rodata.str1.4 .align 4 .LC15: .string " Note: this traceback is incomplete or corrupted!\n" .text .p2align 4,,15 .globl pypy_debug_traceback_print .type pypy_debug_traceback_print, @function pypy_debug_traceback_print: pushl %ebp pushl %edi pushl %esi pushl %ebx subl $12, %esp call pypy_g_RPyFetchExceptionType movl %eax, %esi movl stderr, %edi pushl %edi pushl $19 pushl $1 pushl $.LC12 call fwrite movl pypydtcount, %ebp movl %ebp, %ebx addl $16, %esp xorl %ecx, %ecx .L129: decl %ebx andl $127, %ebx cmpl %ebp, %ebx je .L131 .L120: movl pypy_debug_tracebacks(,%ebx,8), %eax movl pypy_debug_tracebacks+4(,%ebx,8), %edi leal -1(%eax), %edx cmpl $-3, %edx setbe %dl andl $255, %edx testl %ecx, %ecx je .L122 testl %edx, %edx je .L130 cmpl %esi, %edi je .L124 .L130: movl $1, %ecx decl %ebx andl $127, %ebx cmpl %ebp, %ebx jne .L120 .L131: movl stderr, %ebx pushl %ebx pushl $6 pushl $1 pushl $.LC13 call fwrite addl $16, %esp .L118: addl $12, %esp popl %ebx popl %esi popl %edi popl %ebp ret .p2align 4,,15 .L122: testl %edx, %edx jne .L124 testl %esi, %esi je .L126 cmpl %edi, %esi jne .L132 .L126: testl %eax, %eax je .L118 movl %edi, %esi jmp .L130 .p2align 4,,15 .L124: subl $12, %esp movl 4(%eax), %ecx pushl %ecx movl 8(%eax), %edx pushl %edx movl (%eax), %eax pushl %eax pushl $.LC14 movl stderr, %ebp pushl %ebp call fprintf addl $32, %esp xorl %ecx, %ecx movl pypydtcount, %ebp jmp .L129 .p2align 4,,15 .L132: movl stderr, %edi pushl %edi pushl $51 pushl $1 pushl $.LC15 call fwrite addl $16, %esp addl $12, %esp popl %ebx popl %esi popl %edi popl %ebp ret .size pypy_debug_traceback_print, .-pypy_debug_traceback_print .section .rodata.str1.1 .LC16: .string "Fatal RPython error: %s\n" .text .p2align 4,,15 .globl pypy_debug_catch_fatal_exception .type pypy_debug_catch_fatal_exception, @function pypy_debug_catch_fatal_exception: subl $12, %esp call pypy_debug_traceback_print call pypy_g_RPyFetchExceptionType pushl %edx movl 12(%eax), %eax addl $4, %eax pushl %eax pushl $.LC16 movl stderr, %eax pushl %eax call fprintf call abort .size pypy_debug_catch_fatal_exception, .-pypy_debug_catch_fatal_exception .section .rodata.str1.1 .LC17: .string "0" .text .p2align 4,,15 .globl LL_strtod_parts_to_float .type LL_strtod_parts_to_float, @function LL_strtod_parts_to_float: pushl %ebp pushl %edi pushl %esi pushl %ebx subl $76, %esp movl 108(%esp), %eax movl $.LC17, %esi cmpb $0, (%eax) cmovne %eax, %esi call localeconv movl (%eax), %ebp subl $12, %esp pushl %ebp call strlen popl %edx movl %eax, %edi movl 108(%esp), %eax pushl %eax call strlen popl %ebx movl %eax, %ebx movl 112(%esp), %ecx pushl %ecx call strlen popl %edx movl %eax, 52(%esp) movl 116(%esp), %eax pushl %eax call strlen movl %eax, 60(%esp) movl %esi, (%esp) call strlen leal 2(%edi,%ebx), %edi addl 56(%esp), %edi addl 60(%esp), %edi addl %eax, %edi movl %edi, (%esp) call malloc movl %eax, %ebx popl %edx popl %ecx movl 104(%esp), %eax pushl %eax pushl %ebx call strcpy popl %edx popl %ecx movl 108(%esp), %eax pushl %eax pushl %ebx call strcat popl %edx popl %ecx pushl %ebp pushl %ebx call strcat popl %ebp popl %eax movl 112(%esp), %ecx pushl %ecx pushl %ebx call strcat movl %ebx, (%esp) call strlen popl %ebp popl %edx movw $101, (%ebx,%eax) pushl %esi pushl %ebx call strcat leal -1(%ebx,%edi), %esi popl %ecx popl %edi leal 56(%esp), %eax pushl %eax pushl %ebx call strtod fstpl 32(%esp) call __errno_location movl %eax, %edi movl $0, (%eax) movl 64(%esp), %edx addl $16, %esp cmpl %edx, %esi fldl 16(%esp) jae .L136 movl %esi, 48(%esp) movl %esi, %edx .L136: cmpl %ebx, %edx je .L145 cmpb $0, (%edx) jne .L146 cmpl %esi, %edx je .L138 fstp %st(0) jmp .L137 .p2align 4,,15 .L145: fstp %st(0) jmp .L137 .p2align 4,,15 .L146: fstp %st(0) .L137: subl $12, %esp pushl %ebx call free movl $42, (%edi) addl $16, %esp fld1 fchs addl $76, %esp popl %ebx popl %esi popl %edi popl %ebp ret .p2align 4,,15 .L138: fstl 8(%esp) movsd 8(%esp), %xmm0 ucomisd .LC7, %xmm0 jnp .L144 .L140: subl $12, %esp pushl %ebx fstpl 32(%esp) call free addl $16, %esp fldl 16(%esp) addl $76, %esp popl %ebx popl %esi popl %edi popl %ebp ret .p2align 4,,15 .L144: jne .L140 fstp %st(0) subl $8, %esp pushl $0 pushl %ebx call strtod movl $0, (%edi) addl $16, %esp jmp .L140 .size LL_strtod_parts_to_float, .-LL_strtod_parts_to_float .section .rodata.str1.1 .LC19: .string "%.*e" .LC20: .string "%.*f" .LC21: .string "%.*g" .text .p2align 4,,15 .globl LL_strtod_formatd .type LL_strtod_formatd, @function LL_strtod_formatd: pushl %edi pushl %esi pushl %ebx movl 16(%esp), %edx movl 20(%esp), %ecx movb 24(%esp), %al cmpb $101, %al je .L156 cmpb $102, %al je .L157 cmpb $103, %al jne .L150 movl $.LC21, %eax .L148: subl $8, %esp pushl %ecx pushl %edx movl 44(%esp), %esi pushl %esi pushl %eax movl buflen, %ebx pushl %ebx pushl $buffer call snprintf addl $32, %esp testl %eax, %eax jle .L150 cmpl buflen, %eax jge .L150 call localeconv movl (%eax), %esi subl $12, %esp pushl %esi call strlen addl $16, %esp movl %eax, %edi cmpb $46, (%esi) je .L161 .L152: movb buffer, %al cmpb $43, %al je .L159 movl $buffer, %ebx cmpb $45, %al movl $buffer+1, %eax cmove %eax, %ebx .L153: call __ctype_b_loc movl (%eax), %edx jmp .L154 .p2align 4,,15 .L155: incl %ebx .L154: xorl %eax, %eax movb (%ebx), %al testb $8, 1(%edx,%eax,2) jne .L155 pushl %ecx pushl %edi pushl %esi pushl %ebx call strncmp addl $16, %esp testl %eax, %eax jne .L149 movb $46, (%ebx) cmpl $1, %edi jle .L149 incl %ebx leal -1(%ebx,%edi), %edi subl $12, %esp pushl %edi call strlen addl $12, %esp movl %eax, %esi pushl %eax pushl %edi pushl %ebx call memmove movb $0, (%ebx,%esi) addl $16, %esp jmp .L149 .p2align 4,,15 .L150: movl $1059995455, buffer movb $0, buffer+4 .L149: movl $buffer, %eax popl %ebx popl %esi popl %edi ret .p2align 4,,15 .L157: movl $.LC20, %eax jmp .L148 .p2align 4,,15 .L161: cmpb $0, 1(%esi) jne .L152 jmp .L149 .p2align 4,,15 .L159: movl $buffer+1, %ebx jmp .L153 .p2align 4,,15 .L156: movl $.LC19, %eax jmp .L148 .size LL_strtod_formatd, .-LL_strtod_formatd .section .rodata.str1.4 .align 4 .LC22: .string "/tmp/SBo/pypy-1.5-src/pypy/translator/c/src/obmalloc.c" .section .rodata.str1.1 .LC23: .string "bp != ((void *)0)" .section .rodata.str1.4 .align 4 .LC24: .string "narenas == 0 && maxarenas == 0" .section .rodata.str1.1 .LC25: .string "narenas < maxarenas" .text .p2align 4,,15 .globl PyObject_Malloc .type PyObject_Malloc, @function PyObject_Malloc: pushl %ebp pushl %edi pushl %esi pushl %ebx subl $28, %esp movl 48(%esp), %esi leal -1(%esi), %ebp cmpl $255, %ebp ja .L163 shrl $3, %ebp leal (%ebp,%ebp), %ebx movl usedpools(,%ebx,4), %edx movl 8(%edx), %ecx cmpl %edx, %ecx je .L164 incl (%edx) movl 4(%edx), %eax testl %eax, %eax je .L188 movl (%eax), %ebx movl %ebx, 4(%edx) testl %ebx, %ebx je .L189 .L166: addl $28, %esp popl %ebx popl %esi popl %edi popl %ebp ret .p2align 4,,15 .L163: testl %esi, %esi movl $1, %eax cmove %eax, %esi .L185: movl %esi, 48(%esp) addl $28, %esp popl %ebx popl %esi popl %edi popl %ebp jmp malloc .p2align 4,,15 .L164: movl freepools, %edx testl %edx, %edx je .L168 movl 8(%edx), %eax movl %eax, freepools movl 20(%edx), %eax .L169: movl %ecx, 8(%edx) movl %ecx, 12(%edx) movl %edx, 8(%ecx) movl %edx, 12(%ecx) movl $1, (%edx) cmpl %eax, %ebp je .L190 .L170: movl %ebp, 20(%edx) leal 8(,%ebp,8), %ecx leal 32(%edx), %eax leal 32(%ecx,%ecx), %ebx movl %ebx, 24(%edx) movl $4096, %ebx subl %ecx, %ebx movl %ebx, 28(%edx) leal (%eax,%ecx), %ecx movl %ecx, 4(%edx) movl $0, (%ecx) jmp .L166 .p2align 4,,15 .L189: movl 24(%edx), %ebx cmpl 28(%edx), %ebx ja .L167 leal (%edx,%ebx), %ecx movl %ecx, 4(%edx) leal 8(%ebx,%ebp,8), %ebx movl %ebx, 24(%edx) movl $0, (%ecx) jmp .L166 .p2align 4,,15 .L167: movl 12(%edx), %edx movl %edx, 12(%ecx) movl %ecx, 8(%edx) jmp .L166 .p2align 4,,15 .L190: movl 4(%edx), %eax movl (%eax), %ecx movl %ecx, 4(%edx) jmp .L166 .p2align 4,,15 .L168: movl nfreepools, %eax testl %eax, %eax je .L171 .L172: decl %eax movl %eax, nfreepools movl arenabase, %edx leal 4096(%edx), %eax movl %eax, arenabase movl narenas, %eax decl %eax movl %eax, 16(%edx) movl $65535, 20(%edx) movl %ecx, 8(%edx) movl %ecx, 12(%edx) movl %edx, 8(%ecx) movl %edx, 12(%ecx) movl $1, (%edx) jmp .L170 .L171: subl $12, %esp pushl $262144 call malloc movl %eax, 20(%esp) addl $16, %esp testl %eax, %eax je .L185 movl %eax, arenabase movl $64, nfreepools movl 4(%esp), %eax andl $4095, %eax je .L174 movl $63, nfreepools movl 4(%esp), %edx subl %eax, %edx movl %edx, %eax addl $4096, %eax movl %eax, arenabase .L174: movl arenas, %eax testl %eax, %eax je .L191 movl narenas, %eax movl maxarenas, %ecx movl %ecx, 8(%esp) cmpl %ecx, %eax je .L192 .L179: movl narenas, %eax cmpl %eax, 8(%esp) jbe .L193 movl arenas, %eax movl narenas, %edx movl 4(%esp), %ecx movl %ecx, (%eax,%edx,4) movl narenas, %eax incl %eax movl %eax, narenas movl nfreepools, %eax movl usedpools(,%ebx,4), %ecx jmp .L172 .L192: addl %ecx, %ecx movl %ecx, 8(%esp) cmpl %ecx, %eax jae .L178 subl $12, %esp leal 0(,%ecx,4), %eax pushl %eax call malloc movl %eax, 28(%esp) addl $16, %esp testl %eax, %eax je .L178 movl narenas, %ecx movl arenas, %edx sall $2, %ecx cmpl $4, %ecx jb .L182 shrl $2, %ecx movl 12(%esp), %edi movl %edx, %esi rep movsl .L182: movl 12(%esp), %eax movl %eax, arenas movl 8(%esp), %edx movl %edx, maxarenas jmp .L179 .L191: movl narenas, %eax testl %eax, %eax jne .L176 cmpl $0, maxarenas jne .L176 subl $12, %esp pushl $64 call malloc movl %eax, arenas movl arenas, %eax addl $16, %esp testl %eax, %eax je .L178 movl $16, maxarenas movl $16, 8(%esp) jmp .L179 .L178: subl $12, %esp pushl 16(%esp) call free movl $0, nfreepools addl $16, %esp jmp .L185 .L188: pushl $__PRETTY_FUNCTION__.254917 pushl $605 pushl $.LC22 pushl $.LC23 call __assert_fail .L176: pushl $__PRETTY_FUNCTION__.254906 pushl $452 pushl $.LC22 pushl $.LC24 call __assert_fail .L193: pushl $__PRETTY_FUNCTION__.254906 pushl $510 pushl $.LC22 pushl $.LC25 call __assert_fail .size PyObject_Malloc, .-PyObject_Malloc .section .rodata.str1.1 .LC26: .string "pool->ref.count > 0" .text .p2align 4,,15 .globl PyObject_Free .type PyObject_Free, @function PyObject_Free: pushl %esi pushl %ebx subl $4, %esp movl 16(%esp), %eax testl %eax, %eax je .L194 movl %eax, %edx andl $-4096, %edx movl 16(%edx), %ecx movl narenas, %ebx cmpl %ebx, %ecx jae .L196 movl arenas, %ebx movl %eax, %esi subl (%ebx,%ecx,4), %esi cmpl $262143, %esi ja .L196 movl (%edx), %ecx testl %ecx, %ecx je .L200 movl 4(%edx), %ecx movl %ecx, (%eax) movl %eax, 4(%edx) testl %ecx, %ecx je .L198 movl (%edx), %eax decl %eax movl %eax, (%edx) testl %eax, %eax je .L201 .L194: addl $4, %esp popl %ebx popl %esi ret .p2align 4,,15 .L196: movl %eax, 16(%esp) addl $4, %esp popl %ebx popl %esi jmp free .p2align 4,,15 .L201: movl 8(%edx), %ecx movl 12(%edx), %eax movl %eax, 12(%ecx) movl %ecx, 8(%eax) movl freepools, %eax movl %eax, 8(%edx) movl %edx, freepools addl $4, %esp popl %ebx popl %esi ret .p2align 4,,15 .L198: movl (%edx), %eax decl %eax movl %eax, (%edx) testl %eax, %eax je .L202 movl 20(%edx), %eax movl usedpools(,%eax,8), %eax movl 12(%eax), %ecx movl %eax, 8(%edx) movl %ecx, 12(%edx) movl %edx, 12(%eax) movl %edx, 8(%ecx) jmp .L194 .L200: pushl $__PRETTY_FUNCTION__.254929 pushl $746 pushl $.LC22 pushl $.LC26 call __assert_fail .L202: pushl $__PRETTY_FUNCTION__.254929 pushl $785 pushl $.LC22 pushl $.LC26 call __assert_fail .size PyObject_Free, .-PyObject_Free .p2align 4,,15 .globl PyObject_Realloc .type PyObject_Realloc, @function PyObject_Realloc: pushl %edi pushl %esi pushl %ebx subl $16, %esp movl 32(%esp), %ebx movl 36(%esp), %eax testl %ebx, %ebx je .L215 movl %ebx, %ecx andl $-4096, %ecx movl 16(%ecx), %edx movl narenas, %esi cmpl %esi, %edx jae .L205 movl arenas, %esi movl %ebx, %edi subl (%esi,%edx,4), %edi cmpl $262143, %edi ja .L205 movl 20(%ecx), %edx leal 8(,%edx,8), %esi cmpl %esi, %eax ja .L206 leal 0(,%eax,4), %ecx leal (%esi,%esi,2), %edx cmpl %edx, %ecx ja .L207 movl %eax, %esi .L206: subl $12, %esp pushl %eax call PyObject_Malloc addl $16, %esp testl %eax, %eax je .L214 movl %esi, %edx movl %eax, %edi movl %ebx, %esi cmpl $4, %edx jae .L216 .L208: xorl %ecx, %ecx testb $2, %dl je .L211 movw (%esi), %cx movw %cx, (%edi) movl $2, %ecx .L211: andl $1, %edx je .L212 movb (%esi,%ecx), %dl movb %dl, (%edi,%ecx) .L212: subl $12, %esp pushl %ebx movl %eax, 28(%esp) call PyObject_Free addl $16, %esp movl 12(%esp), %eax movl %eax, %ebx movl %ebx, %eax addl $16, %esp popl %ebx popl %esi popl %edi ret .p2align 4,,15 .L205: testl %eax, %eax jne .L217 subl $8, %esp pushl $1 pushl %ebx call realloc addl $16, %esp testl %eax, %eax cmovne %eax, %ebx .L207: movl %ebx, %eax addl $16, %esp popl %ebx popl %esi popl %edi ret .p2align 4,,15 .L217: movl %eax, 36(%esp) movl %ebx, 32(%esp) addl $16, %esp popl %ebx popl %esi popl %edi jmp realloc .p2align 4,,15 .L214: xorl %ebx, %ebx jmp .L207 .p2align 4,,15 .L216: testb $1, %al jne .L218 .L209: testl $2, %edi jne .L219 .L210: movl %edx, %ecx shrl $2, %ecx rep movsl jmp .L208 .p2align 4,,15 .L215: movl %eax, 32(%esp) addl $16, %esp popl %ebx popl %esi popl %edi jmp PyObject_Malloc .L219: movw (%esi), %cx movw %cx, (%edi) addl $2, %edi addl $2, %esi subl $2, %edx jmp .L210 .L218: movb (%ebx), %cl movb %cl, (%eax) leal 1(%eax), %edi leal 1(%ebx), %esi decl %edx jmp .L209 .size PyObject_Realloc, .-PyObject_Realloc .p2align 4,,15 .globl RPython_StartupCode .type RPython_StartupCode, @function RPython_StartupCode: pushl %edi pushl %esi pushl %ebx call pypy_g_frameworkgc_setup xorl %eax, %eax movl $2146435072, %edx movl %eax, pypy_g_pypy_objspace_std_floatobject_W_FloatObject+8 movl %edx, pypy_g_pypy_objspace_std_floatobject_W_FloatObject+12 xorl %ecx, %ecx movl $-1048576, %ebx movl %ecx, pypy_g_tuple2_1433+4 movl %ebx, pypy_g_tuple2_1433+8 movl %ecx, pypy_g_tuple2_1434+4 movl %ebx, pypy_g_tuple2_1434+8 movl %ecx, pypy_g_tuple2_1435+4 movl %ebx, pypy_g_tuple2_1435+8 movl %ecx, pypy_g_tuple2_1436+4 movl %ebx, pypy_g_tuple2_1436+8 movl %ecx, pypy_g_tuple2_1437+4 movl %ebx, pypy_g_tuple2_1437+8 xorl %esi, %esi movl $2146959360, %edi movl %esi, pypy_g_tuple2_1437+12 movl %edi, pypy_g_tuple2_1437+16 movl %ecx, pypy_g_tuple2_1438+4 movl %ebx, pypy_g_tuple2_1438+8 movl %ecx, pypy_g_tuple2_1440+4 movl %ebx, pypy_g_tuple2_1440+8 movl %esi, pypy_g_tuple2_1441+4 movl %edi, pypy_g_tuple2_1441+8 movl %esi, pypy_g_tuple2_1441+12 movl %edi, pypy_g_tuple2_1441+16 movl %eax, pypy_g_tuple2_1444+4 movl %edx, pypy_g_tuple2_1444+8 movl %eax, pypy_g_tuple2_1447+4 movl %edx, pypy_g_tuple2_1447+8 movl %eax, pypy_g_tuple2_1448+4 movl %edx, pypy_g_tuple2_1448+8 movl %eax, pypy_g_tuple2_1449+4 movl %edx, pypy_g_tuple2_1449+8 movl %eax, pypy_g_tuple2_1450+4 movl %edx, pypy_g_tuple2_1450+8 movl %eax, pypy_g_tuple2_1451+4 movl %edx, pypy_g_tuple2_1451+8 movl %eax, pypy_g_tuple2_1452+4 movl %edx, pypy_g_tuple2_1452+8 movl %esi, pypy_g_tuple2_1452+12 movl %edi, pypy_g_tuple2_1452+16 movl %esi, pypy_g_tuple2_1453+4 movl %edi, pypy_g_tuple2_1453+8 movl %esi, pypy_g_tuple2_1454+4 movl %edi, pypy_g_tuple2_1454+8 movl %esi, pypy_g_tuple2_1457+12 movl %edi, pypy_g_tuple2_1457+16 movl %eax, pypy_g_tuple2_1458+4 movl %edx, pypy_g_tuple2_1458+8 movl %eax, pypy_g_tuple2_1459+4 movl %edx, pypy_g_tuple2_1459+8 movl %eax, pypy_g_tuple2_1460+4 movl %edx, pypy_g_tuple2_1460+8 movl %eax, pypy_g_tuple2_1461+4 movl %edx, pypy_g_tuple2_1461+8 movl %esi, pypy_g_tuple2_1468+12 movl %edi, pypy_g_tuple2_1468+16 movl %eax, pypy_g_tuple2_1469+4 movl %edx, pypy_g_tuple2_1469+8 movl %ecx, pypy_g_tuple2_1469+12 movl %ebx, pypy_g_tuple2_1469+16 movl %ecx, pypy_g_tuple2_1470+12 movl %ebx, pypy_g_tuple2_1470+16 movl %eax, pypy_g_tuple2_1471+12 movl %edx, pypy_g_tuple2_1471+16 movl %eax, pypy_g_tuple2_1472+4 movl %edx, pypy_g_tuple2_1472+8 movl %eax, pypy_g_tuple2_1472+12 movl %edx, pypy_g_tuple2_1472+16 movl %esi, pypy_g_tuple2_1473+4 movl %edi, pypy_g_tuple2_1473+8 movl %eax, pypy_g_tuple2_1473+12 movl %edx, pypy_g_tuple2_1473+16 movl %eax, pypy_g_tuple2_1474+12 movl %edx, pypy_g_tuple2_1474+16 movl %eax, pypy_g_tuple2_1475+12 movl %edx, pypy_g_tuple2_1475+16 movl %ecx, pypy_g_tuple2_1476+12 movl %ebx, pypy_g_tuple2_1476+16 movl %ecx, pypy_g_tuple2_1477+12 movl %ebx, pypy_g_tuple2_1477+16 movl %eax, pypy_g_tuple2_1478+12 movl %edx, pypy_g_tuple2_1478+16 movl %ecx, pypy_g_tuple2_1479+12 movl %ebx, pypy_g_tuple2_1479+16 movl %esi, pypy_g_tuple2_1482+12 movl %edi, pypy_g_tuple2_1482+16 movl %eax, pypy_g_tuple2_1483+12 movl %edx, pypy_g_tuple2_1483+16 movl %ecx, pypy_g_tuple2_1484+12 movl %ebx, pypy_g_tuple2_1484+16 movl %esi, pypy_g_tuple2_1485+4 movl %edi, pypy_g_tuple2_1485+8 movl %ecx, pypy_g_tuple2_1485+12 movl %ebx, pypy_g_tuple2_1485+16 movl %ecx, pypy_g_tuple2_1486+4 movl %ebx, pypy_g_tuple2_1486+8 movl %ecx, pypy_g_tuple2_1487+4 movl %ebx, pypy_g_tuple2_1487+8 xorl %eax, %eax popl %ebx popl %esi popl %edi ret .size RPython_StartupCode, .-RPython_StartupCode .section .rodata.str1.1 .LC30: .string "out of memory" .section .rodata.str1.4 .align 4 .LC31: .string "Fatal error during initialization: %s\n" .text .p2align 4,,15 .globl main .type main, @function main: leal 4(%esp), %ecx andl $-16, %esp pushl -4(%ecx) pushl %ebp movl %esp, %ebp pushl %edi pushl %esi pushl %ebx pushl %ecx subl $40, %esp movl (%ecx), %eax movl %eax, -32(%ebp) movl 4(%ecx), %ecx movl %ecx, -40(%ebp) call RPython_StartupCode testl %eax, %eax jne .L222 subl $12, %esp movl -32(%ebp), %edi pushl %edi call pypy_g__RPyListOfString_New__Signed movl %eax, -36(%ebp) call pypy_g_RPyExceptionOccurred addl $16, %esp testb %al, %al jne .L230 movl -32(%ebp), %esi testl %esi, %esi jle .L223 movl $0, -28(%ebp) movl -28(%ebp), %edx .p2align 4,,15 .L227: movl -40(%ebp), %eax movl (%eax,%edx,4), %esi subl $12, %esp pushl %esi call strlen movl %eax, %ebx movl %eax, (%esp) call pypy_g_RPyString_New__Signed movl %eax, %edx leal 12(%eax), %edi cmpl $4, %ebx jb .L224 movl %ebx, %ecx shrl $2, %ecx rep movsl .L224: xorl %eax, %eax testb $2, %bl je .L225 movw (%esi), %ax movw %ax, (%edi) movl $2, %eax .L225: andl $1, %ebx je .L226 movb (%esi,%eax), %cl movb %cl, (%edi,%eax) .L226: movl %edx, -44(%ebp) call pypy_g_RPyExceptionOccurred addl $16, %esp testb %al, %al movl -44(%ebp), %edx jne .L230 pushl %ebx pushl %edx movl -28(%ebp), %ecx pushl %ecx movl -36(%ebp), %edx pushl %edx call pypy_g__RPyListOfString_SetItem__listPtr_Signed_rpy_str incl -28(%ebp) addl $16, %esp movl -28(%ebp), %edx cmpl %edx, -32(%ebp) jg .L227 .L223: subl $12, %esp movl -36(%ebp), %eax pushl %eax call pypy_g_entry_point movl %eax, %ebx call pypy_g_RPyExceptionOccurred addl $16, %esp testb %al, %al jne .L232 movl %ebx, %eax leal -16(%ebp), %esp popl %ecx popl %ebx popl %esi popl %edi popl %ebp leal -4(%ecx), %esp ret .p2align 4,,15 .L232: call pypy_debug_traceback_print call pypy_g_RPyFetchExceptionType pushl %edi movl 12(%eax), %eax addl $4, %eax pushl %eax pushl $.LC16 movl stderr, %esi pushl %esi call fprintf call abort .L230: movl $.LC30, %eax .L222: pushl %ebx pushl %eax pushl $.LC31 pushl stderr call fprintf call abort .size main, .-main .globl pypysig_counter .bss .align 16 .type pypysig_counter, @object .size pypysig_counter, 4 pypysig_counter: .zero 4 .globl _LLstacktoobig_stack_start .align 16 .type _LLstacktoobig_stack_start, @object .size _LLstacktoobig_stack_start, 4 _LLstacktoobig_stack_start: .zero 4 .globl stack_direction .align 16 .type stack_direction, @object .size stack_direction, 4 stack_direction: .zero 4 .globl _RPyString_dump .align 16 .type _RPyString_dump, @object .size _RPyString_dump, 4 _RPyString_dump: .zero 4 .globl pypydtcount .align 16 .type pypydtcount, @object .size pypydtcount, 4 pypydtcount: .zero 4 .globl buflen .data .align 16 .type buflen, @object .size buflen, 4 buflen: .long 120 .globl start_tls_key .section .tbss,"awT", at nobits .align 16 .type start_tls_key, @object .size start_tls_key, 4 start_tls_key: .zero 4 .comm pypy_debug_tracebacks,1024,32 .comm buffer,120,32 .local narenas .comm narenas,4,16 .local arenas .comm arenas,4,16 .local freepools .comm freepools,4,16 .data .align 32 .type usedpools, @object .size usedpools, 256 usedpools: .long usedpools-8 .long usedpools-8 .long usedpools .long usedpools .long usedpools+8 .long usedpools+8 .long usedpools+16 .long usedpools+16 .long usedpools+24 .long usedpools+24 .long usedpools+32 .long usedpools+32 .long usedpools+40 .long usedpools+40 .long usedpools+48 .long usedpools+48 .long usedpools+56 .long usedpools+56 .long usedpools+64 .long usedpools+64 .long usedpools+72 .long usedpools+72 .long usedpools+80 .long usedpools+80 .long usedpools+88 .long usedpools+88 .long usedpools+96 .long usedpools+96 .long usedpools+104 .long usedpools+104 .long usedpools+112 .long usedpools+112 .long usedpools+120 .long usedpools+120 .long usedpools+128 .long usedpools+128 .long usedpools+136 .long usedpools+136 .long usedpools+144 .long usedpools+144 .long usedpools+152 .long usedpools+152 .long usedpools+160 .long usedpools+160 .long usedpools+168 .long usedpools+168 .long usedpools+176 .long usedpools+176 .long usedpools+184 .long usedpools+184 .long usedpools+192 .long usedpools+192 .long usedpools+200 .long usedpools+200 .long usedpools+208 .long usedpools+208 .long usedpools+216 .long usedpools+216 .long usedpools+224 .long usedpools+224 .long usedpools+232 .long usedpools+232 .long usedpools+240 .long usedpools+240 .section .rodata .type __PRETTY_FUNCTION__.254929, @object .size __PRETTY_FUNCTION__.254929, 14 __PRETTY_FUNCTION__.254929: .string "PyObject_Free" .local nfreepools .comm nfreepools,4,16 .local arenabase .comm arenabase,4,16 .type __PRETTY_FUNCTION__.254917, @object .size __PRETTY_FUNCTION__.254917, 16 __PRETTY_FUNCTION__.254917: .string "PyObject_Malloc" .local maxarenas .comm maxarenas,4,16 .type __PRETTY_FUNCTION__.254906, @object .size __PRETTY_FUNCTION__.254906, 10 __PRETTY_FUNCTION__.254906: .string "new_arena" .data .align 16 .type wakeup_fd, @object .size wakeup_fd, 4 wakeup_fd: .long -1 .local pypysig_occurred .comm pypysig_occurred,4,16 .local pypysig_flags .comm pypysig_flags,65,32 .local _pypythread_stacksize .comm _pypythread_stacksize,4,16 .section .rodata.cst8,"aM", at progbits,8 .align 8 .LC7: .long 0 .long 0 .section .rodata.cst16,"aM", at progbits,16 .align 16 .LC8: .long 0 .long -2147483648 .long 0 .long 0 .section .rodata.cst8 .align 8 .LC9: .long 0 .long 1077936128 .ident "GCC: (GNU) 4.5.2" .section .note.GNU-stack,"", at progbits -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From rogerb at rogerbinns.com Thu May 12 17:34:48 2011 From: rogerb at rogerbinns.com (Roger Binns) Date: Thu, 12 May 2011 08:34:48 -0700 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> <4DBE4F7A.3030109@gmail.com> <4DBF15E0.6080005@rogerbinns.com> <4DC21F58.8000101@rogerbinns.com> <4DCB442F.1080800@rogerbinns.com> Message-ID: <4DCBFE18.3050804@rogerbinns.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/12/2011 01:34 AM, Maciej Fijalkowski wrote: >>> 32bit pypy works out of the box on valgrind. No idea why 64bit does not Are you sure it plays nice with the standard memory allocation functions (malloc/free)? That is unlikely due to use of a GC. The doc I wrote shows what you have to do to get CPython to play nice. Additionally I need pypy's behaviour to be deterministic. I was getting different behaviour from just running a test versus running under gdb with a breakpoint in one of my C functions and just hitting continue once it hit! The most promising thing looked to me to be the refcounting GC but that doesn't compile - the error message implied it had something to do with not supporting weakrefs. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk3L/hQACgkQmOOfHg372QQVRwCdEFJKlDHJzW7V5bvGD57tNyfy ISMAoL2j4+zO3fM5wxcYTSgoBE63JFU1 =u/ST -----END PGP SIGNATURE----- From glbrt.chen at gmail.com Thu May 12 17:48:00 2011 From: glbrt.chen at gmail.com (Bo Chen) Date: Thu, 12 May 2011 23:48:00 +0800 Subject: [pypy-dev] JVM backend - status and future work possibilities? In-Reply-To: <5E773F9F-1CDC-4A78-8AFB-9F380BAAA410@headius.com> References: <5E773F9F-1CDC-4A78-8AFB-9F380BAAA410@headius.com> Message-ID: Hi, all, I'm a newbie in python and python implementation. I just read this thread. It's no better to me to devote some effort on the JVM backend in pypy. That could be a little hard for me but cool. And I need to write my master's doc the next year too! It's a good opportunity for me to study on the details of a language implementation. Shall we just do something or help each other? Thanks & BR Bo Chen On Thu, May 12, 2011 at 12:39 PM, Charles Oliver Nutter wrote: > FWIW I would be happy to help out with advice and direction, especially if > the JVM backend could be rejiggered to take advantage of the new > invokedynamic bytecode in Java 7. I have been using it for JRuby and have > been very pleased with it. > > Anyway, I really hope the JVM backend can move forward and I am standing by > to help out as I am able. > > - Charlie (mobile) > > On May 11, 2011, at 16:56, Dario Bertini wrote: > > > 2011/5/11 Micha? Bendowski : > >> Welcome everyone :) > >> > >> I was wondering what is the current status of the JVM backend. I'm > >> asking because I am considering working on it for my master's thesis - > >> I was inspired by a call to action from the "Features" page of > >> pypy.org. Unfortunately it's unclear to me how much work needs to be > >> done here. I understand that RPython translation is already > >> implemented (although might need some care, for the build process to > >> succeed more easily[1]), but the JIT backend is missing. There is also > >> no integration with JVM libraries - am I right? On the other hand > >> there already is a "reference" CLI implementation and a whole thesis > >> about it. > > > > Hi, I'm in a similar situation here :) > > > > I'm planning as well to work on the jvm backend for my master thesis: > > I contacted just today one of my professors who seems to be willing to > > mentor me, but I haven't started any work yet > > > > the integration with JVM libraries basically is a prerequisite for > > starting the work on the JIT backend, I'm not sure and I don't > > actually know your university program, but here at my university for > > the master thesis we have something like 15 ects... so something like > > the integration with the JVM alone may very well be on the upper side > > of what effort we could be expect to put in the thesis > > > > You're right about the CLI implementation, but after a huge > > refactoring inside pypy it wasn't functional anymore... Antonio Cuni > > (who is the author of the CLI backend we're talking about) fixed it, > > but I think that the ability to access CLI assemblies and class still > > has some issues... > > > > btw, I'm very new to this project as well, and from what I gathered > > until now, Antonio seems to be the guy to ask to for questions about > > the OO type system :) > > > > As a starting point: he suggested me to start looking into JPype and > Jtool > > > > http://jpype.sourceforge.net/ > > > http://blog.meresco.org/2010/03/11/integrating-java-in-python-with-jtool/ > > > > this kind of tools are needed for calling JVM code from CPython, thus > > letting us to try/debug the JVM integration when running pypy > > interpreted on top of CPython ( pypy/bin/py.py ) > > > >> > >> I'm not familiar with pypy internals at the moment, nor am I a > >> compiler expert (yet!). I will be very grateful for any advice, as to > >> whether taking care of the JVM backend is a suitable amount of work, > >> and whether it can "scale", i.e. be divided into smaller parts that I > >> can implement along the way and stop if I'm out of time (I imagine I > >> can always find more work in PyPy). > >> > > > > I would also like to know if this kind of work would be feasible to > > tackle in parallel for 2 students like us :) > > > > (btw, my presence on the irc channel has been quite lacking > > nowadays... I'll try to be more present in the next days) > > > > byebye > > > > :Dario > > _______________________________________________ > > pypy-dev mailing list > > pypy-dev at python.org > > http://mail.python.org/mailman/listinfo/pypy-dev > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Thu May 12 17:59:28 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 12 May 2011 17:59:28 +0200 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: <4DCBFE18.3050804@rogerbinns.com> References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> <4DBE4F7A.3030109@gmail.com> <4DBF15E0.6080005@rogerbinns.com> <4DC21F58.8000101@rogerbinns.com> <4DCB442F.1080800@rogerbinns.com> <4DCBFE18.3050804@rogerbinns.com> Message-ID: On Thu, May 12, 2011 at 5:34 PM, Roger Binns wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 05/12/2011 01:34 AM, Maciej Fijalkowski wrote: >>>> 32bit pypy works out of the box on valgrind. No idea why 64bit does not > > Are you sure it plays nice with the standard memory allocation functions > (malloc/free)? ?That is unlikely due to use of a GC. ?The doc I wrote shows > what you have to do to get CPython to play nice. Are you using callgrind or using valgrind for memory checks? The former should work, the latter is rather pointless I think, because RPython manages memory on it's own. > > Additionally I need pypy's behaviour to be deterministic. ?I was getting > different behaviour from just running a test versus running under gdb with a > breakpoint in one of my C functions and just hitting continue once it hit! If you have a bug in cpyext somehow (or pypy does, which is also a possibility) you won't get deterministic behavior due to way memory works in C. > > The most promising thing looked to me to be the refcounting GC but that > doesn't compile - the error message implied it had something to do with not > supporting weakrefs. No, never use refcounting GC for anything. What's the problem with say minimark? > > Roger > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > > iEYEARECAAYFAk3L/hQACgkQmOOfHg372QQVRwCdEFJKlDHJzW7V5bvGD57tNyfy > ISMAoL2j4+zO3fM5wxcYTSgoBE63JFU1 > =u/ST > -----END PGP SIGNATURE----- > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > From alex.suhan at gmail.com Thu May 12 20:15:37 2011 From: alex.suhan at gmail.com (=?UTF-8?B?QWxleCDFnnVoYW4=?=) Date: Thu, 12 May 2011 21:15:37 +0300 Subject: [pypy-dev] PHP interpreter and rdict performance Message-ID: <4DCC23C9.3020100@gmail.com> Hello. We're working on a tracing JIT for PHP using PyPy (https://bitbucket.org/asuhan/happy). It's still in its infancy, but it has a bunch of features already in place and beats Zend & HipHop by a large margin in some microbenchmarks -- sieve (and other arithmetical stuff) and lots of tests based on PHPbench. However, we?re struggling with poor performance (2x slowdown compared to Zend) for dictionaries -- especially set operations. The implementation in pypy/rpython/lltypesystem/rdict.py seems perfectly fine at a first glance and we?re relying on it for implementing PHP arrays. pypy-c (built from 43932 changeset) is also about 2x slower than CPython for something like: a = {} t = 0 while t != 1000000: a[t] = t t += 1 While we?re not expecting dictionaries to benefit from JIT magic, it?s a little strange that the ?baseline? performance is not up to par. Is there a way of circumventing this issue? Many thanks for this piece of software and keep up the good work. -- asuhan From arigo at tunes.org Thu May 12 21:29:18 2011 From: arigo at tunes.org (Armin Rigo) Date: Thu, 12 May 2011 21:29:18 +0200 Subject: [pypy-dev] PHP interpreter and rdict performance In-Reply-To: <4DCC23C9.3020100@gmail.com> References: <4DCC23C9.3020100@gmail.com> Message-ID: Hi Alex, On Thu, May 12, 2011 at 8:15 PM, Alex ?uhan wrote: > Hello. We're working on a tracing JIT for PHP using PyPy > (https://bitbucket.org/asuhan/happy). :-) > While we?re not expecting dictionaries to benefit from JIT magic, it?s a > little strange that the ?baseline? performance is not up to par. Is there a > way of circumventing this issue? I cannot reproduce the issue. I tried on Linux, a 32-bit and 64-bit pypy-c's version 1.5, compared to a 32-bit and 64-bit CPython 2.7, and the former is 20%-30% faster than the latter on your example... Could the problem be on the OSX or Windows pypy-c? A bient?t, Armin. From anto.cuni at gmail.com Fri May 13 00:08:20 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 13 May 2011 00:08:20 +0200 Subject: [pypy-dev] JVM backend - status and future work possibilities? In-Reply-To: References: Message-ID: <4DCC5A54.7010500@gmail.com> On 11/05/11 23:02, Micha? Bendowski wrote: > Welcome everyone :) > > I was wondering what is the current status of the JVM backend. [cut] Hello Micha?, Dario, Bo, I'm glad to see that there is interest around the JVM backend :-) Here is a summary of the current status and possible future direction. What works: translation of the basic interpreter to JVM What does not work/it's not there: 1) integration with the JVM classes at the RPython level 2) integration with the JVM classes at application level 3) JIT backend 4) a lot of modules which currently only work with the C backend Point (1) is a blocker for the rest, so it must be done before everything else. The idea, is to be able to instantiate java classes and call java methods from RPython code: the JVM backend will then turn these into the corresponding bytecode (invokespecial/invokevirtual). This is "easy", but you need a deep understanding of the internals of the RPython toolchain to implement this (in particular, the annotation and the rtyping phases). The other requirement is that we want to be able to do JVM call also during testing. As know, RPython is a subset of Python, and you can either translate it to C/JVM/CLI or execute it directly on top of CPython. The latter is what we do all the time when doing testing. Testing on top of CPython is extremely convenient for a large number of reasons which I'm not going to explain because it's late :-) We do not want to loose the possibility of testing on top of CPython, even when our RPython code contains calls to JVM classes. So, we need a way to interface with the JVM from CPython, and this is why I asked Dario to investigate on JPype and JTool. Once step (1) is complete, the other tasks can be done in parallel. ciao, Anto From rogerb at rogerbinns.com Fri May 13 07:47:01 2011 From: rogerb at rogerbinns.com (Roger Binns) Date: Thu, 12 May 2011 22:47:01 -0700 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> <4DBE4F7A.3030109@gmail.com> <4DBF15E0.6080005@rogerbinns.com> <4DC21F58.8000101@rogerbinns.com> <4DCB442F.1080800@rogerbinns.com> <4DCBFE18.3050804@rogerbinns.com> Message-ID: <4DCCC5D5.5020600@rogerbinns.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 >> Are you using callgrind or using valgrind for memory checks? The >> former should work, the latter is rather pointless I think, because >> RPython manages memory on it's own. The point is to use valgrind. >> If you have a bug in cpyext somehow (or pypy does, which is also a >> possibility) you won't get deterministic behavior due to way memory >> works in C. Errr, the way memory works in C is exactly why valgrind does work. >> No, never use refcounting GC for anything. What's the problem with say minimark? That has nothing to do with the issue at hand. To recap the whole thread, I tried porting my CPython extension to pypy. I wanted to ensure I did everything right but there turned out to be no documentation (eg which mailing list is appropriate, common gotchas, how to compile/configure pypy in such a way as to make gdb and valgrind play nice). It was suggested that I write as much of that doc as possible at which point the blanks would be filled in for me. So I wrote it and no blanks have been filled in ... https://docs.google.com/document/pub?id=1Ve9B6Gz5laV63zYFOlb_Lhe8ZBIw2V1756lquott6JI You'll note that for cpython I show how to compile it in such a way as to disable all memory optimisations and recycling so that it works perfectly with valgrind. With pypy my symptoms are that pypy crashes after trying to work on one of my objects that the debugger shows is complete junk. The distributed pypy has no debug symbols. Just compiling pypy following instructions results in function names but no line numbers. Running gdb and just hitting 'continue' on a break point gives very different behaviour than running without gdb. In order for me or anyone else having issues with cpyext we need some hints as to how to proceed. Things that would help greatly: - - How to do a debug build that retains all symbols and line number information - - Some gdb macros to make debugging easier. For example CPython comes with a .gdbinit with nice macros like 'pyo' so you can see what a PyObject * represents - - How to disable all memory optimisations and make it work with valgrind - - How to get deterministic behaviour - ie exactly the same thing happens each time whether you run gdb or not Trying the refcount gc was to address the last two since it should result in objects being freed the moment they are not used and not being dependent on when GC is run or what address spaces it is run over. I don't care what the solutions are, and attempted to find some by myself which didn't work. Trying to debug issues like this when none of the normal tools work is exceptionally difficult especially given the non-deterministic behaviour of pypy with cpyext. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk3MxdEACgkQmOOfHg372QTWcgCgpLcZS1KniZlRCwgHJr/qe8Oo 7Z4AniicHqC892bsFigTL8YMvMTMNlN6 =lHMW -----END PGP SIGNATURE----- From niko at alum.mit.edu Fri May 13 09:29:53 2011 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 13 May 2011 09:29:53 +0200 Subject: [pypy-dev] JVM backend - status and future work possibilities? In-Reply-To: <4DCC5A54.7010500@gmail.com> References: <4DCC5A54.7010500@gmail.com> Message-ID: <4DCCDDF1.7050407@alum.mit.edu> Hello everyone, I just wanted to say that I am certainly available to answer questions about the code that exists. I don't know how much work has been done on it since I last touched it, though. I would be interested in helping out somewhat as well, but at the moment my time is very limited... I'm in the process of finishing up my PhD as well as various other projects. Anyway, I definitely think it'd be great if the JVM backend saw some attention! Niko Antonio Cuni wrote: > On 11/05/11 23:02, Micha? Bendowski wrote: >> Welcome everyone :) >> >> I was wondering what is the current status of the JVM backend. > [cut] > > Hello Micha?, Dario, Bo, > I'm glad to see that there is interest around the JVM backend :-) > > Here is a summary of the current status and possible future direction. > > What works: translation of the basic interpreter to JVM > > What does not work/it's not there: > > 1) integration with the JVM classes at the RPython level > 2) integration with the JVM classes at application level > 3) JIT backend > 4) a lot of modules which currently only work with the C backend > > Point (1) is a blocker for the rest, so it must be done before everything else. > > The idea, is to be able to instantiate java classes and call java methods from > RPython code: the JVM backend will then turn these into the corresponding > bytecode (invokespecial/invokevirtual). This is "easy", but you need a deep > understanding of the internals of the RPython toolchain to implement this (in > particular, the annotation and the rtyping phases). > > The other requirement is that we want to be able to do JVM call also during > testing. As know, RPython is a subset of Python, and you can either translate > it to C/JVM/CLI or execute it directly on top of CPython. The latter is what > we do all the time when doing testing. Testing on top of CPython is extremely > convenient for a large number of reasons which I'm not going to explain > because it's late :-) > > We do not want to loose the possibility of testing on top of CPython, even > when our RPython code contains calls to JVM classes. So, we need a way to > interface with the JVM from CPython, and this is why I asked Dario to > investigate on JPype and JTool. > > Once step (1) is complete, the other tasks can be done in parallel. > > ciao, > Anto > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal at bendowski.pl Fri May 13 15:37:38 2011 From: michal at bendowski.pl (=?UTF-8?Q?Micha=C5=82_Bendowski?=) Date: Fri, 13 May 2011 15:37:38 +0200 Subject: [pypy-dev] JVM backend - status and future work possibilities? In-Reply-To: <4DCCDDF1.7050407@alum.mit.edu> References: <4DCC5A54.7010500@gmail.com> <4DCCDDF1.7050407@alum.mit.edu> Message-ID: Thank you for your support Niko, Antonio and Charles. It seems that the plan for now is to dig into pypy and try to get familiar with the source (and I imagine it's not an easy task). As Antonio mentioned, the first step is probably to work on integration at the RPython level - and we should probably collaborate on that. Later we will have to split probably, as a master's thesis is usually supposed to be an individual contribution. That's not a problem for me right now, since I will be doing an internship during the summer and I won't be able to spend as much time on the project as I would like to. And everything can change during that time :) Just out of curiosity - where are you guys (Bo, Dario) studying? I'm from the University of Warsaw. Cheers, Micha? On Fri, May 13, 2011 at 09:29, Niko Matsakis wrote: > Hello everyone, > > I just wanted to say that I am certainly available to answer questions about > the code that exists.? I don't know how much work has been done on it since > I last touched it, though.? I would be interested in helping out somewhat as > well, but at the moment my time is very limited... I'm in the process of > finishing up my PhD as well as various other projects. Anyway, I definitely > think it'd be great if the JVM backend saw some attention! > > > Niko > > Antonio Cuni wrote: > > On 11/05/11 23:02, Micha? Bendowski wrote: > > Welcome everyone :) > > I was wondering what is the current status of the JVM backend. > > [cut] > > Hello Micha?, Dario, Bo, > I'm glad to see that there is interest around the JVM backend :-) > > Here is a summary of the current status and possible future direction. > > What works: translation of the basic interpreter to JVM > > What does not work/it's not there: > > 1) integration with the JVM classes at the RPython level > 2) integration with the JVM classes at application level > 3) JIT backend > 4) a lot of modules which currently only work with the C backend > > Point (1) is a blocker for the rest, so it must be done before everything > else. > > The idea, is to be able to instantiate java classes and call java methods > from > RPython code: the JVM backend will then turn these into the corresponding > bytecode (invokespecial/invokevirtual). This is "easy", but you need a deep > understanding of the internals of the RPython toolchain to implement this > (in > particular, the annotation and the rtyping phases). > > The other requirement is that we want to be able to do JVM call also during > testing. As know, RPython is a subset of Python, and you can either > translate > it to C/JVM/CLI or execute it directly on top of CPython. The latter is > what > we do all the time when doing testing. Testing on top of CPython is > extremely > convenient for a large number of reasons which I'm not going to explain > because it's late :-) > > We do not want to loose the possibility of testing on top of CPython, even > when our RPython code contains calls to JVM classes. So, we need a way to > interface with the JVM from CPython, and this is why I asked Dario to > investigate on JPype and JTool. > > Once step (1) is complete, the other tasks can be done in parallel. > > ciao, > Anto > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > > From paul.degrandis at gmail.com Fri May 13 17:53:44 2011 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Fri, 13 May 2011 08:53:44 -0700 Subject: [pypy-dev] JVM backend - status and future work possibilities? In-Reply-To: References: <4DCC5A54.7010500@gmail.com> <4DCCDDF1.7050407@alum.mit.edu> Message-ID: A few years back I had started the process of using JPype with PyPy. I most recently started digging into Clojure and JDK 7 ( https://github.com/ohpauleez/soymacchiato), but I now have some free cycles. I'd certainly be up for digging in. Paul 2011/5/13 Micha? Bendowski > Thank you for your support Niko, Antonio and Charles. It seems that > the plan for now is to dig into pypy and try to get familiar with the > source (and I imagine it's not an easy task). As Antonio mentioned, > the first step is probably to work on integration at the RPython level > - and we should probably collaborate on that. Later we will have to > split probably, as a master's thesis is usually supposed to be an > individual contribution. That's not a problem for me right now, since > I will be doing an internship during the summer and I won't be able to > spend as much time on the project as I would like to. And everything > can change during that time :) > > Just out of curiosity - where are you guys (Bo, Dario) studying? I'm > from the University of Warsaw. > > Cheers, > > Micha? > > > > On Fri, May 13, 2011 at 09:29, Niko Matsakis wrote: > > Hello everyone, > > > > I just wanted to say that I am certainly available to answer questions > about > > the code that exists. I don't know how much work has been done on it > since > > I last touched it, though. I would be interested in helping out somewhat > as > > well, but at the moment my time is very limited... I'm in the process of > > finishing up my PhD as well as various other projects. Anyway, I > definitely > > think it'd be great if the JVM backend saw some attention! > > > > > > Niko > > > > Antonio Cuni wrote: > > > > On 11/05/11 23:02, Micha? Bendowski wrote: > > > > Welcome everyone :) > > > > I was wondering what is the current status of the JVM backend. > > > > [cut] > > > > Hello Micha?, Dario, Bo, > > I'm glad to see that there is interest around the JVM backend :-) > > > > Here is a summary of the current status and possible future direction. > > > > What works: translation of the basic interpreter to JVM > > > > What does not work/it's not there: > > > > 1) integration with the JVM classes at the RPython level > > 2) integration with the JVM classes at application level > > 3) JIT backend > > 4) a lot of modules which currently only work with the C backend > > > > Point (1) is a blocker for the rest, so it must be done before everything > > else. > > > > The idea, is to be able to instantiate java classes and call java methods > > from > > RPython code: the JVM backend will then turn these into the corresponding > > bytecode (invokespecial/invokevirtual). This is "easy", but you need a > deep > > understanding of the internals of the RPython toolchain to implement this > > (in > > particular, the annotation and the rtyping phases). > > > > The other requirement is that we want to be able to do JVM call also > during > > testing. As know, RPython is a subset of Python, and you can either > > translate > > it to C/JVM/CLI or execute it directly on top of CPython. The latter is > > what > > we do all the time when doing testing. Testing on top of CPython is > > extremely > > convenient for a large number of reasons which I'm not going to explain > > because it's late :-) > > > > We do not want to loose the possibility of testing on top of CPython, > even > > when our RPython code contains calls to JVM classes. So, we need a way > to > > interface with the JVM from CPython, and this is why I asked Dario to > > investigate on JPype and JTool. > > > > Once step (1) is complete, the other tasks can be done in parallel. > > > > ciao, > > Anto > > _______________________________________________ > > pypy-dev mailing list > > pypy-dev at python.org > > http://mail.python.org/mailman/listinfo/pypy-dev > > > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anto.cuni at gmail.com Fri May 13 18:05:54 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 13 May 2011 18:05:54 +0200 Subject: [pypy-dev] JVM backend - status and future work possibilities? In-Reply-To: References: <4DCC5A54.7010500@gmail.com> <4DCCDDF1.7050407@alum.mit.edu> Message-ID: <4DCD56E2.1000802@gmail.com> On 13/05/11 17:53, Paul deGrandis wrote: > A few years back I had started the process of using JPype with PyPy. I most > recently started digging into Clojure and JDK 7 > (https://github.com/ohpauleez/soymacchiato), but I now have some free cycles. > I'd certainly be up for digging in. Hi Paul, yes indeed, I forgot that there is already a bit of work done, although not much. For those who are interested, the code is in rlib/rjvm.py. So, the first step is to complete this module so that it is possible to call JVM classes without having to interface directly with JPype (or equivalent). The next step would be to teach the RPython toolchain how to translate the calls to rjvm. It is important to get the design of rjvm right, else this step would be very hard, and I'm certainly willing to help whoever is interested in doing it. ciao, Anto From anto.cuni at gmail.com Fri May 13 18:14:35 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 13 May 2011 18:14:35 +0200 Subject: [pypy-dev] JVM backend - status and future work possibilities? In-Reply-To: <5E773F9F-1CDC-4A78-8AFB-9F380BAAA410@headius.com> References: <5E773F9F-1CDC-4A78-8AFB-9F380BAAA410@headius.com> Message-ID: <4DCD58EB.6020203@gmail.com> On 12/05/11 06:39, Charles Oliver Nutter wrote: > FWIW I would be happy to help out with advice and direction, especially if the JVM backend could be rejiggered to take advantage of the new invokedynamic bytecode in Java 7. I have been using it for JRuby and have been very pleased with it. Hello Charles, I think that invokedynamic might be very useful for a potential JVM JIT backend, but not for the static one, because the RPython code translates quite straightforwardly to the "normal" JVM bytecode. About the JIT, I am very interested in hearing about the potential of invokedynamic. For example, I think it could be useful for attaching new bridges to already compiled loops, without the need to recompile the whole loop as the CLI JIT backend does. Two questions: - how fast/slow is an invokedynamic call compared to a local jump? (Assuming that the JVM is able to inline the target method) - are tail calls supported by the JVM, and by invokedynamic in particular? ciao, Anto From paul.degrandis at gmail.com Fri May 13 18:59:47 2011 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Fri, 13 May 2011 09:59:47 -0700 Subject: [pypy-dev] JVM backend - status and future work possibilities? In-Reply-To: <4DCD58EB.6020203@gmail.com> References: <5E773F9F-1CDC-4A78-8AFB-9F380BAAA410@headius.com> <4DCD58EB.6020203@gmail.com> Message-ID: Anto, On your second question, the default JVM does not support TCO, but you can trampoline. There is a JVM version (called DiVinci Machine), that is working to (and can somewhat) support TCO. The DaVinci Machine is where invokedynamic and invokevirtual came from (approved JSRs are part of JDK7 now). Languages like Python and Ruby will see noticeable improvements for invokedynamic, but languages like Clojure will not (since Clojure gets a strict compilation phase before it gets executed). For this reason, my intuition is that (like you) I don't think it's useful for the static JIT, but could be really cool for language hooks thereafter (since the JVM JIT can optimize invokedynamic and invokevirtual calls). On Fri, May 13, 2011 at 9:14 AM, Antonio Cuni wrote: > On 12/05/11 06:39, Charles Oliver Nutter wrote: > > FWIW I would be happy to help out with advice and direction, especially > if the JVM backend could be rejiggered to take advantage of the new > invokedynamic bytecode in Java 7. I have been using it for JRuby and have > been very pleased with it. > > Hello Charles, > > I think that invokedynamic might be very useful for a potential JVM JIT > backend, but not for the static one, because the RPython code translates > quite > straightforwardly to the "normal" JVM bytecode. > > About the JIT, I am very interested in hearing about the potential of > invokedynamic. For example, I think it could be useful for attaching new > bridges to already compiled loops, without the need to recompile the whole > loop as the CLI JIT backend does. > > Two questions: > > - how fast/slow is an invokedynamic call compared to a local jump? > (Assuming > that the JVM is able to inline the target method) > > - are tail calls supported by the JVM, and by invokedynamic in particular? > > ciao, > Anto > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From osadchiy.ilya at gmail.com Fri May 13 19:24:40 2011 From: osadchiy.ilya at gmail.com (Ilya Osadchiy) Date: Fri, 13 May 2011 20:24:40 +0300 Subject: [pypy-dev] Executable stack of pypy-c Message-ID: Hello. When translating pypy 1.5 under Gentoo linux I get binary with executable permission for the stack (so that execute prevention will not be enabled for it). Then if I make a gentoo package for I get annoying notice about it during installation. After some inspection I've found that the problem comes from gcmaptable.s which doesn't have the line '.section .note.GNU-stack,"", at progbits'. (I see that gcmaptable.s is generated "manually" and not by gcc, perhaps that's the reason). 1. Is the executable stack really needed? 2. If it's not needed, can we add the line '.section .note.GNU-stack,"", at progbits' to the end of gcmaptable.s? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From amauryfa at gmail.com Fri May 13 19:52:08 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Fri, 13 May 2011 19:52:08 +0200 Subject: [pypy-dev] Executable stack of pypy-c In-Reply-To: References: Message-ID: Hi, 2011/5/13 Ilya Osadchiy > > Hello. > When translating pypy 1.5 under Gentoo linux I get binary with executable permission for the stack (so that execute prevention will not be enabled for it). Then if I make a gentoo package for I get annoying notice about it during installation. > After some inspection I've found that the problem comes from gcmaptable.s which doesn't have the line '.section??? .note.GNU-stack,"", at progbits'. > (I see that gcmaptable.s is generated "manually" and not by gcc, perhaps that's the reason). > > 1. Is the executable stack really needed? > 2. If it's not needed, can we add the line '.section??? .note.GNU-stack,"", at progbits' to the end of gcmaptable.s? According to http://www.gentoo.org/proj/en/hardened/gnu-stack.xml the preferred incantation is: #if defined(__linux__) && defined(__ELF__) .section .note.GNU-stack,"",%progbits #endif -- Amaury Forgeot d'Arc From cool-rr at cool-rr.com Fri May 13 20:18:08 2011 From: cool-rr at cool-rr.com (cool-RR) Date: Fri, 13 May 2011 20:18:08 +0200 Subject: [pypy-dev] Potential PyPy incompatibility Message-ID: Hey, Here's something I thought about that might be a compatibility problem, not sure if it's a problem in production. When you take a method of a builtin in Pypy, for example `str.join`, it seems to create a new object which is weakreffable, where in CPython it's not. I was just looking at Django's signal mechanism: http://docs.djangoproject.com/en/1.3/topics/signals/#listening-to-signals It creates a weak reference to the signal-handler function, and if it dies, the link is destroyed. Do you think this might be a problem? Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From amauryfa at gmail.com Fri May 13 20:28:50 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Fri, 13 May 2011 20:28:50 +0200 Subject: [pypy-dev] Potential PyPy incompatibility In-Reply-To: References: Message-ID: Hi, 2011/5/13 cool-RR : > Hey, > Here's something I thought about that might be a compatibility problem, not > sure if it's a problem in production. When you take a method of a builtin in > Pypy, for example `str.join`, it seems to create a new object which is > weakreffable, where in CPython it's not. I was just looking at Django's > signal mechanism: > http://docs.djangoproject.com/en/1.3/topics/signals/#listening-to-signals > It creates a weak reference to the signal-handler function, and if it dies, > the link is destroyed. > Do you think this might be a problem? But why would you use an unbound method of a builtin type as a signal handler? Note that "sender" is passed as first argument to the function, whereas i.e. str.join expected a string. -- Amaury Forgeot d'Arc From cool-rr at cool-rr.com Fri May 13 20:44:30 2011 From: cool-rr at cool-rr.com (cool-RR) Date: Fri, 13 May 2011 20:44:30 +0200 Subject: [pypy-dev] Potential PyPy incompatibility In-Reply-To: References: Message-ID: On Fri, May 13, 2011 at 8:28 PM, Amaury Forgeot d'Arc wrote: > Hi, > > 2011/5/13 cool-RR : > > Hey, > > Here's something I thought about that might be a compatibility problem, > not > > sure if it's a problem in production. When you take a method of a builtin > in > > Pypy, for example `str.join`, it seems to create a new object which is > > weakreffable, where in CPython it's not. I was just looking at Django's > > signal mechanism: > > > http://docs.djangoproject.com/en/1.3/topics/signals/#listening-to-signals > > It creates a weak reference to the signal-handler function, and if it > dies, > > the link is destroyed. > > Do you think this might be a problem? > > But why would you use an unbound method of a builtin type > as a signal handler? > > Note that "sender" is passed as first argument to the function, > whereas i.e. str.join expected a string. > > -- > Amaury Forgeot d'Arc > In the case of Django signal handlers it would probably not be a problem in practice, but in other systems it might become a problem so if it can be fixed early (e.g. by making `list.__add__ is list.__add__` True in Pypy as it is in CPython) it might save us some trouble later. Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From drsalists at gmail.com Fri May 13 21:27:34 2011 From: drsalists at gmail.com (Dan Stromberg) Date: Fri, 13 May 2011 12:27:34 -0700 Subject: [pypy-dev] cpyext - compilation, and swig? Message-ID: When I build a swig-generated C extension module against CPython 2.6, and attempt to load it using cpyext, I get: /usr/local/pypy-1.5/bin/pypy >>>> cpyext.load_module('_odirectcmodule.so', 'odirect') Traceback (most recent call last): File "", line 1, in ImportError: unable to load extension module './_odirectcmodule.so': ./_odirectcmodule.so: undefined symbol: PyClass_Type >>>> 1) Is that the right way to load a C extension module using cpyext? 2) Do I need to recompile the module for pypy? 3) Are there known issues with cpyext and swig? 4) Of swig, Cython, hand-coded C, and ctypes, which is the better bet today for reliability with use from PyPy? Thanks! PS: This is, I believe, a module that really does need some C - it does bitwise address arithmetic to get page alignment. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dje.gcc at gmail.com Fri May 13 22:10:01 2011 From: dje.gcc at gmail.com (David Edelsohn) Date: Fri, 13 May 2011 16:10:01 -0400 Subject: [pypy-dev] Performance challenge of mutating class variables Message-ID: I mentioned this issue in a private conversation with Carl and he suggested that I send the details to the mailinglist. The following blog post is a nice summary of a significant performance problem in IronPython's early implementation with good links to the key discussions: http://ironpython-urls.blogspot.com/2009/05/python-jython-and-ironpython.html Robert Smallshire implemented OWL Basic in IronPython and experienced some massive slowdowns relative to CPython and Jython. IronPython developer Dino Diehland investigated the problem and wrote an detailed blog post about the problem and the fix implemented in IronPython. The problem was due to the user program's use of a class variable as a counter, which caused "unfortunate" behavior in the IronPython implementation -- mutating the variable invalidated the cache. PyPy shows a good speedup compared with CPython already, but it is an interesting corner case. The smaller testcase created by Robert might be a good benchmark to add to PyPy Speed Center (http://www.smallshire.org.uk/sufficientlysmall/2009/05/22/ironpython-2-0-and-jython-2-5-performance-compared-to-python-2-5/). - David From amauryfa at gmail.com Fri May 13 22:27:43 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Fri, 13 May 2011 22:27:43 +0200 Subject: [pypy-dev] cpyext - compilation, and swig? In-Reply-To: References: Message-ID: Hi, 2011/5/13 Dan Stromberg : > > When I build a swig-generated C extension module against CPython 2.6, and > attempt to load it using cpyext, I get: > > /usr/local/pypy-1.5/bin/pypy >>>>> cpyext.load_module('_odirectcmodule.so', 'odirect') > Traceback (most recent call last): > ? File "", line 1, in > ImportError: unable to load extension module './_odirectcmodule.so': > ./_odirectcmodule.so: undefined symbol: PyClass_Type >>>>> > > 1) Is that the right way to load a C extension module using cpyext? No, you should just import it, like with CPython. > 2) Do I need to recompile the module for pypy? Yes, because the memory layout of objects is different, and some macros are turned into function calls. And to enforce the difference with CPython, the extension modules are named differently: mymodule.pypy-15.so Normally, it's enough to run "pypy setup.py install" > 3) Are there known issues with cpyext and swig? Some SWIG extension modules are known to work. Normally it should compile and work normally, except if you have custom typemaps and other C code that uses the CPython API. > 4) Of swig, Cython, hand-coded C, and ctypes, which is the better bet today > for reliability with use from PyPy? Cython won't work with pypy because it plays too much with the CPython internals. I don't have an opinion about it, but some of us consider ctypes as the best way to develop extension modules. ctypes will eventually become much faster with pypy, with the JIT compiler emitting directly the machine code to call the function. In any case, only a pure Python implementation can be sped up by the JIT. A C function cannot be traced... At the moment, cpyext is still considered as a (useful) experiment that helps you leverage existing C extensions, but it will always be slow for different reasons, the most obvious one is that the garbabe collector used by pypy moves objects, whereas PyObject* pointers are expected to stay in place... Also, there are probably other directions that we have not yet explored: - implement another C API better suited to pypy - a SWIG backend which emits ctypes code or similar And probably others... -- Amaury Forgeot d'Arc From drsalists at gmail.com Fri May 13 23:37:11 2011 From: drsalists at gmail.com (Dan Stromberg) Date: Fri, 13 May 2011 14:37:11 -0700 Subject: [pypy-dev] cpyext - compilation, and swig? In-Reply-To: References: Message-ID: On Fri, May 13, 2011 at 1:27 PM, Amaury Forgeot d'Arc wrote: > Hi, > > 2) Do I need to recompile the module for pypy? > > Yes, because the memory layout of objects is different, and some > macros are turned into function calls. > And to enforce the difference with CPython, the extension modules are > named differently: mymodule.pypy-15.so > > Normally, it's enough to run "pypy setup.py install" > I don't suppose the path to doing this with "make" has been explored? Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From amauryfa at gmail.com Sat May 14 00:06:22 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Sat, 14 May 2011 00:06:22 +0200 Subject: [pypy-dev] cpyext - compilation, and swig? In-Reply-To: References: Message-ID: 2011/5/13 Dan Stromberg : >> Normally, it's enough to run "pypy setup.py install" > > I don't suppose the path to doing this with "make" has been explored? Do you use a Makefile to compile your extension modules? It will certainly work, if you provide the correct path to pypy/include directory. (and to the import library on Windows) -- Amaury Forgeot d'Arc From darjus at gmail.com Sat May 14 00:10:22 2011 From: darjus at gmail.com (Darjus Loktevic) Date: Fri, 13 May 2011 15:10:22 -0700 Subject: [pypy-dev] cpyext - compilation, and swig? In-Reply-To: References: Message-ID: After looking at SWIG cffi example, it seems that adding basic support for PyPy ctypes should not be hard (no C++ support of course). I might play with it on the weekend. Darjus On Fri, May 13, 2011 at 1:27 PM, Amaury Forgeot d'Arc wrote: > Hi, > > 2011/5/13 Dan Stromberg : >> >> When I build a swig-generated C extension module against CPython 2.6, and >> attempt to load it using cpyext, I get: >> >> /usr/local/pypy-1.5/bin/pypy >>>>>> cpyext.load_module('_odirectcmodule.so', 'odirect') >> Traceback (most recent call last): >> ? File "", line 1, in >> ImportError: unable to load extension module './_odirectcmodule.so': >> ./_odirectcmodule.so: undefined symbol: PyClass_Type >>>>>> >> >> 1) Is that the right way to load a C extension module using cpyext? > > No, you should just import it, like with CPython. > >> 2) Do I need to recompile the module for pypy? > > Yes, because the memory layout of objects is different, and some > macros are turned into function calls. > And to enforce the difference with CPython, the extension modules are > named differently: mymodule.pypy-15.so > > Normally, it's enough to run "pypy setup.py install" > >> 3) Are there known issues with cpyext and swig? > > Some SWIG extension modules are known to work. > Normally it should compile and work normally, > except if you have custom typemaps and other C > code that uses the CPython API. > >> 4) Of swig, Cython, hand-coded C, and ctypes, which is the better bet today >> for reliability with use from PyPy? > > Cython won't work with pypy because it plays too much with the CPython > internals. > > I don't have an opinion about it, but some of us consider ctypes as the best way > to develop extension modules. ctypes will eventually become much > faster with pypy, > with the JIT compiler emitting directly the machine code to call the function. > In any case, only a pure Python implementation can be sped up by the JIT. > A C function cannot be traced... > > At the moment, cpyext is still considered as a (useful) experiment that helps > you leverage existing C extensions, but it will always be slow for different > reasons, the most obvious one is that the garbabe collector used by pypy > moves objects, whereas PyObject* pointers are expected to stay in place... > > Also, there are probably other directions that we have not yet explored: > - implement another C API better suited to pypy > - a SWIG backend which emits ctypes code or similar > And probably others... > > -- > Amaury Forgeot d'Arc > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > From amauryfa at gmail.com Sat May 14 00:41:32 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Sat, 14 May 2011 00:41:32 +0200 Subject: [pypy-dev] cpyext - compilation, and swig? In-Reply-To: References: Message-ID: 2011/5/14 Darjus Loktevic : > After looking at SWIG cffi example, it seems that adding basic support > for PyPy ctypes should not be hard (no C++ support of course). I might > play with it on the weekend. Nice! But note that SWIG cffi can also wrap C++ classes... -- Amaury Forgeot d'Arc From drsalists at gmail.com Sat May 14 01:35:20 2011 From: drsalists at gmail.com (Dan Stromberg) Date: Fri, 13 May 2011 16:35:20 -0700 Subject: [pypy-dev] cpyext - compilation, and swig? In-Reply-To: References: Message-ID: On Fri, May 13, 2011 at 3:06 PM, Amaury Forgeot d'Arc wrote: > 2011/5/13 Dan Stromberg : > >> Normally, it's enough to run "pypy setup.py install" > > > > I don't suppose the path to doing this with "make" has been explored? > > Do you use a Makefile to compile your extension modules? > It will certainly work, if you provide the correct path to > pypy/include directory. > (and to the import library on Windows) > Yes, usually. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From william.leslie.ttg at gmail.com Sat May 14 06:06:22 2011 From: william.leslie.ttg at gmail.com (William ML Leslie) Date: Sat, 14 May 2011 14:06:22 +1000 Subject: [pypy-dev] cpyext - compilation, and swig? In-Reply-To: References: Message-ID: On 14 May 2011 06:27, Amaury Forgeot d'Arc wrote: > Also, there are probably other directions that we have not yet explored: > - implement another C API better suited to pypy > - a SWIG backend which emits ctypes code or similar > And probably others... Not to mention cppyy, which might be a good fit for SWIG. Has anyone tried to use cppyy recently? -- William Leslie From arigo at tunes.org Sat May 14 11:28:22 2011 From: arigo at tunes.org (Armin Rigo) Date: Sat, 14 May 2011 11:28:22 +0200 Subject: [pypy-dev] Potential PyPy incompatibility In-Reply-To: References: Message-ID: Hi, On Fri, May 13, 2011 at 8:44 PM, cool-RR wrote: > In the case of Django signal handlers it would probably not be a problem in > practice, but in other systems it might become a problem so if it can be > fixed early (e.g. by making `list.__add__ is list.__add__` True in Pypy as > it is in CPython) it might save us some trouble later. That's not possible. PyPy uses regular unbound methods for 'str.join', so like with any unbound method, 'x.m is x.m' is false. It's an irregularity of CPython that we already discussed here some time ago. A bient?t, Armin. From arigo at tunes.org Sat May 14 11:41:38 2011 From: arigo at tunes.org (Armin Rigo) Date: Sat, 14 May 2011 11:41:38 +0200 Subject: [pypy-dev] Performance challenge of mutating class variables In-Reply-To: References: Message-ID: Hi David, On Fri, May 13, 2011 at 10:10 PM, David Edelsohn wrote: > The problem was due to the user program's use of a class > variable as a counter, which caused "unfortunate" behavior in the > IronPython implementation -- mutating the variable invalidated the > cache. PyPy has indeed mostly the same behavior, but it is still faster than CPython on the example you give. It is because you don't do anything with the Nodes. If we modify the example (http://paste.pocoo.org/show/388541/) to read all '_children' attributes after all Nodes have been created, then PyPy is very slow. In this example it is 4x slower than CPython when called with 2000000. Thanks for pushing us in trying to find a solution for this problem :-) A bient?t, Armin. From arigo at tunes.org Sat May 14 14:39:21 2011 From: arigo at tunes.org (Armin Rigo) Date: Sat, 14 May 2011 14:39:21 +0200 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: <4DCCC5D5.5020600@rogerbinns.com> References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> <4DBE4F7A.3030109@gmail.com> <4DBF15E0.6080005@rogerbinns.com> <4DC21F58.8000101@rogerbinns.com> <4DCB442F.1080800@rogerbinns.com> <4DCBFE18.3050804@rogerbinns.com> <4DCCC5D5.5020600@rogerbinns.com> Message-ID: Hi Roger, On Fri, May 13, 2011 at 7:47 AM, Roger Binns wrote: >>> Are you using callgrind or using valgrind for memory checks? The >>> former should work, the latter is rather pointless I think, because >>> RPython manages memory on it's own. > > The point is to use valgrind. I think the general problem is that you are trying to approach debugging PyPy like you approach debugging a C-written program (say, CPython). You come with a number of expectations of being able to do certain things, which don't work well or at all in this case. It is a bit like, say, wanting to debug various memory issues in a Java program by running the Java VM in gdb. It may eventually work, but it's a *lot* of pain, and you need to be very familiar with how the VM works internally. I know that gdb is essential to make CPython C extensions work. I believe that it's also a limitation of cpyext the fact that it makes gdb'ing through the PyPy-compiled C extensions so painful. One thing we could add to cpyext is a reference count checker: a special mode in which cpyext doesn't immediately free the CPython objects whose refcount drops to zero, but instead checks for a while if the object is still used, and cleanly complain if it is. This can be added to the RPython source code of cpyext; it's not something that should be debugged in gdb, for the reasons above. If nevertheless you want to use C-level debugging tools, then it's going to be painful: > - - How to do a debug build that retains all symbols and line number information That part is easy: you need to go in the C sources produced in /tmp/usession-*/testing_1, and type "make lldebug". Maybe you want to disable "obmalloc.c", as you'd do in CPython; I think it's some other predefined target in the Makefile. > - - Some gdb macros to make debugging easier. ?For example CPython comes with > a .gdbinit with nice macros like 'pyo' so you can see what a PyObject * > represents This is impossible in general, because the C structures corresponding to objects changes depending on e.g. whether you included some special compilation options or not. It would be possible, though, to come with some macros that work in the common case. This might require additional hacks to fix the C name of some helper functions, so that they can be called by the macros. In other words it's something that someone may come up with at some point (there was some work in that direction), but it's work. > - - How to disable all memory optimisations and make it work with valgrind That's not possible. The minimark GC is not a "memory optimisation"; it's a completely different approach to garbage collection than, say, CPython's reference counting. > - - How to get deterministic behaviour - ie exactly the same thing happens > each time whether you run gdb or not I don't know the details, but there are several factors that make it impossible. The most important one is, again, the fact that the minimark GC triggers collections at times that look random. I don't see how to fix that. > Trying the refcount gc was to address the last two since it should result in > objects being freed the moment they are not used and not being dependent on > when GC is run or what address spaces it is run over. The refcount GC included in PyPy is not suited to run a complete pypy-c. It's there for historical reasons, and it's still used by a number of tests, but it doesn't support weakrefs for example, nor the GCREF type, which is essential for the JIT. I fear there is no more way to get a "completely deterministic" behavior in gdb for pypy-c than there is for a Java VM, for the same reasons. What we are missing is a set of tools that let you locate and debug issues with CPython C extension modules, but not necessarily based on gdb. A bient?t, Armin. From rogerb at rogerbinns.com Sat May 14 16:24:00 2011 From: rogerb at rogerbinns.com (Roger Binns) Date: Sat, 14 May 2011 07:24:00 -0700 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> <4DBE4F7A.3030109@gmail.com> <4DBF15E0.6080005@rogerbinns.com> <4DC21F58.8000101@rogerbinns.com> <4DCB442F.1080800@rogerbinns.com> <4DCBFE18.3050804@rogerbinns.com> <4DCCC5D5.5020600@rogerbinns.com> Message-ID: <4DCE9080.9030108@rogerbinns.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/14/2011 05:39 AM, Armin Rigo wrote: > I think the general problem is that you are trying to approach > debugging PyPy like you approach debugging a C-written program (say, > CPython). Note that I am not trying to debug pypy itself but an extension written in C compiled with cpyext. > It is a > bit like, say, wanting to debug various memory issues in a Java > program by running the Java VM in gdb. It is like trying to debug JNI issues, not Java program issues. To debug JNI you do use a regular C based debugger like gdb. Some JVM also have a checking mode for JNI calls: http://publib.boulder.ibm.com/infocenter/javasdk/v5r0/index.jsp?topic=/com.ibm.java.doc.diagnostics.50/html/jni_debug.html > One thing > we could add to cpyext is a reference count checker: a special mode in > which cpyext doesn't immediately free the CPython objects whose > refcount drops to zero, but instead checks for a while if the object > is still used, and cleanly complain if it is. This can be added to > the RPython source code of cpyext; it's not something that should be > debugged in gdb, for the reasons above. I'm all for a checking mode. Note that you'll still have to provide some way of interacting the with the debugger - it would be rather pointless to emit a message saying "pyobject misuse at address 0x12345678" and then exit as it would need to be established who allocated it and who reused it after death. > If nevertheless you want to use C-level debugging tools, then it's > going to be painful: *Any* debugging tools would be fine, but given C extensions are full of C code you may as well make using C tools on that C code possible. >> - - Some gdb macros to make debugging easier. For example CPython comes with >> a .gdbinit with nice macros like 'pyo' so you can see what a PyObject * >> represents > > This is impossible in general, because the C structures corresponding > to objects changes depending on e.g. whether you included some special > compilation options or not. It would be possible, though, to come > with some macros that work in the common case. This might require > additional hacks to fix the C name of some helper functions, so that > they can be called by the macros. In other words it's something that > someone may come up with at some point (there was some work in that > direction), but it's work. The exact same issues apply to CPython. CPython exports a helper function named _PyObject_Dump(PyObject *) which then calls the internal Python machinery (str etc). 'pyo' is the only macro I use and is very necessary because you have PyObject* all over the place and need some idea of what they are. >> - - How to disable all memory optimisations and make it work with valgrind > > That's not possible. The minimark GC is not a "memory optimisation"; > it's a completely different approach to garbage collection than, say, > CPython's reference counting. It is actually possible. All valgrind needs to know is which areas of memory are in use and which aren't. By far the easiest way is to devolve into C library calls of malloc and free. That is why I mentioned the ref counting GC as it would be malloc and free calls at the end of the day. Since pypy doesn't have a functioning "dumb" memory mode the only alternative is to add calls to valgrind. It has a header you can use that generates no side effect instruction sequences when the program is run normally and tells valgrind things when run under valgrind. See http://valgrind.org/docs/manual/manual-core-adv.html For example the memory allocation code would need to call VALGRIND_MALLOCLIKE_BLOCK for each allocated chunk of memory and the GC would need to call VALGRIND_FREELIKE_BLOCK on each freed chunk. >> - - How to get deterministic behaviour - ie exactly the same thing happens >> each time whether you run gdb or not > > I don't know the details, but there are several factors that make it > impossible. The most important one is, again, the fact that the > minimark GC triggers collections at times that look random. I don't > see how to fix that. I'd be very happy having the GC be triggered at every possible point and do a complete collection. That is the best for a checking mode since it means the time between when something is no longer used and when it is collected will be as short as possible. You can probably trigger it before and after every cpyext wrapped call. > What we are > missing is a set of tools that let you locate and debug issues with > CPython C extension modules, but not necessarily based on gdb. Indeed. At the moment my code works perfectly under CPython, has 99.6% test coverage, is valgrind clean under CPython and only broke the refcount rules for one internal debug method used for fault injection that is only used in special builds. The latter was fairly easily diagnosed and fixed. Under pypy I'm having all sorts of problems including a pypy crash, nonsensical behaviour and just using gdb changing program behaviour. Until pypy is perfect I won't be the only one affected by this! Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk3OkHgACgkQmOOfHg372QQlBgCghjQw1kwzd9KLc1XXNlISPnkk ZY0An0SkMpm3vS7OfknkGtYRJrTvXCgf =y4Oe -----END PGP SIGNATURE----- From arigo at tunes.org Sat May 14 17:20:27 2011 From: arigo at tunes.org (Armin Rigo) Date: Sat, 14 May 2011 17:20:27 +0200 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: <4DCE9080.9030108@rogerbinns.com> References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> <4DBE4F7A.3030109@gmail.com> <4DBF15E0.6080005@rogerbinns.com> <4DC21F58.8000101@rogerbinns.com> <4DCB442F.1080800@rogerbinns.com> <4DCBFE18.3050804@rogerbinns.com> <4DCCC5D5.5020600@rogerbinns.com> <4DCE9080.9030108@rogerbinns.com> Message-ID: Hi Roger, Ah, sorry. I think I got mistaken in my previous mail. The point is that you are debugging *only* a C extension, and not interested in debugging any of the rest of PyPy. Then all we said about GCs makes no sense: the PyObject* within PyPy are not handled by PyPy's GC. They are normally malloc()ed and free()d --- except that, maybe for no good reason any more, we redirect malloc() and free() to obmalloc.c, which is very similar to CPython's PyObject_Malloc. You disable it with "make no_obmalloc", and then they are just normal malloc()ed and free()d objects from the C point of view. This is independent of the GC, which anyway only handles the objects that are in native RPython format. Similarly, you don't want a gdb macro that nicely prints any RPython object. You want instead a macro that nicely prints a PyObject* from cpyext. Sorry, I missed this. And indeed, you can probably just call PyString_AsString(PyObject_Repr(x)) from gdb to get it. In fact, the same gdb macro that works on CPython has chances to work on PyPy --- or it would if we had _PyObject_Dump() in cpyext. Note that the objects that C extension modules see are "placeholder" PyObjects, that indirectly contain a reference to the "real" RPython object. There are no memory optimisations that should interfere with PyObjects; for example, no free list. I'm pretty sure that the troubles you get are simply due to the placeholders being freed too early. The difference with CPython is that the reference count in the placeholder only counts references from the C extension, and not the references that would, in CPython, be taken from other parts of the interpreter. That's why placeholder PyObjects can go away earlier in PyPy than normal PyObjects in CPython. That's the case for which adding a debugging mode in PyPy would be useful. A bient?t, Armin. From parlar at gmail.com Sat May 14 22:20:55 2011 From: parlar at gmail.com (Jay Parlar) Date: Sat, 14 May 2011 16:20:55 -0400 Subject: [pypy-dev] Trouble building pypy Message-ID: I've been trying on-and-off to build PyPy for a few days now, and keep running into various issues. I was hoping someone on the list could give me some pointers as to what I'm doing wrong. I'm working on an i7 iMac, latest Snow Leopard with 12GB of RAM. I used the official 1.5 release to try to build revision 2650f0a670c0. I ran the command 'pypy translate.py --opt=jit targetpypystandalone.py' to start the build. A few times near the beginning the pypy process would fail and I'd get the OS X crash reporter dialog ("pypy quit unexpectedly), but the whole translation process continued on. I put the final part of the terminal output at https://gist.github.com/972588 Any help would be much appreciated. I bought the extra RAM specifically to try to make building PyPy a feasible process :) Jay P. From arigo at tunes.org Sat May 14 22:49:55 2011 From: arigo at tunes.org (Armin Rigo) Date: Sat, 14 May 2011 22:49:55 +0200 Subject: [pypy-dev] Trouble building pypy In-Reply-To: References: Message-ID: Hi Jay, On Sat, May 14, 2011 at 10:20 PM, Jay Parlar wrote: > I've been trying on-and-off to build PyPy for a few days now, and keep > running into various issues. I was hoping someone on the list could > give me some pointers as to what I'm doing wrong. As usual it looks like some obscure sub-platform-specific detail; I love C. These errors you get seem strange: [platform:Error] "_libintl_gettext", referenced from: [platform:Error] _pypy_g_gettext__arrayPtr_star_1 in ccHLYwGr.o The C sources (from /var/folders/7+/7+jU+2VeFbq7ekX49vUWQk+++TQ/-Tmp-/usession-default-2/testing_1) contain a function called pypy_g_gettext__arrayPtr_star_1(), and this function should contain a call to gettext(), which comes from the library libintl. However, in some cases, libintl.h contains "#define gettext libintl_gettext". I can only imagine that for some reason this line is enabled in the .h file, but your actual libintl.dynlib does not actually define this symbol, but just gettext(). You should also check that the Makefile in the above directory really contains "-lintl". But you'll have to go around finding more about this bug on your own, I fear, because this is all I can guess from a bit of Googling around... A bient?t, Armin. From santagada at gmail.com Sat May 14 22:51:03 2011 From: santagada at gmail.com (Leonardo Santagada) Date: Sat, 14 May 2011 17:51:03 -0300 Subject: [pypy-dev] Trouble building pypy In-Reply-To: References: Message-ID: Maybe you are using macports? If I get that error there is a problem finding gettext 64bits. IIRC pypy on osx doesn't use gettext at all. You can try to install a universal (or at least 64bit) gettext like the one from rudix[1][2], but I think your problem is in your python, try to use the apple provided one. I'm lurking at the #pypy channel on freenode and I can try to help. [1] http://rudix.org/ [2] http://code.google.com/p/rudix/downloads/detail?name=gettext-0.18.1.1-1.dmg&can=2&q=gettext On Sat, May 14, 2011 at 5:20 PM, Jay Parlar wrote: > I've been trying on-and-off to build PyPy for a few days now, and keep > running into various issues. I was hoping someone on the list could > give me some pointers as to what I'm doing wrong. > > I'm working on an i7 iMac, latest Snow Leopard with 12GB of RAM. I > used the official 1.5 release to try to build revision 2650f0a670c0. I > ran the command 'pypy translate.py --opt=jit targetpypystandalone.py' > to start the build. A few times near the beginning the pypy process > would fail and I'd get the OS X crash reporter dialog ("pypy quit > unexpectedly), but the whole translation process continued on. > > I put the final part of the terminal output at https://gist.github.com/972588 > > Any help would be much appreciated. I bought the extra RAM > specifically to try to make building PyPy a feasible process :) > > Jay P. > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > -- Leonardo Santagada From darjus at gmail.com Sun May 15 00:02:47 2011 From: darjus at gmail.com (Darjus Loktevic) Date: Sat, 14 May 2011 15:02:47 -0700 Subject: [pypy-dev] cpyext - compilation, and swig? In-Reply-To: References: Message-ID: So, while waiting for my wife in the beauty salon, got something basic working. ======= In [1]: import example In [2]: example.get_time() Out[2]: 'Sat May 14 14:58:35 2011\n' ======= I think it'll take me a few weeks to get it to the point of being really usable (need to add many more things, docs, etc.), but it should be very useful for PyPy and it seems C++ is doable as well. Darjus On Fri, May 13, 2011 at 3:41 PM, Amaury Forgeot d'Arc wrote: > 2011/5/14 Darjus Loktevic : >> After looking at SWIG cffi example, it seems that adding basic support >> for PyPy ctypes should not be hard (no C++ support of course). I might >> play with it on the weekend. > > Nice! > But note that SWIG cffi can also wrap C++ classes... > > -- > Amaury Forgeot d'Arc > From parlar at gmail.com Sun May 15 02:31:28 2011 From: parlar at gmail.com (Jay Parlar) Date: Sat, 14 May 2011 20:31:28 -0400 Subject: [pypy-dev] Trouble building pypy In-Reply-To: References: Message-ID: On Sat, May 14, 2011 at 4:51 PM, Leonardo Santagada wrote: > Maybe you are using macports? > > If I get that error there is a problem finding gettext 64bits. IIRC > pypy on osx doesn't use gettext at all. > > You can try to install a universal (or at least 64bit) gettext like > the one from rudix[1][2], but I think your problem is in your python, > try to use the apple provided one. I'm lurking at the #pypy channel on > freenode and I can try to help. > > [1] http://rudix.org/ > [2] http://code.google.com/p/rudix/downloads/detail?name=gettext-0.18.1.1-1.dmg&can=2&q=gettext I'm not running macports or fink, or anything similar. The Python I used to build it was the 1.5 binary release of PyPy, which I'd figure would be fine. My main goal right now is to dig into the PyPy internals a little, try to learn something about how it all works. I spoke with Alex Gaynor on Twitter a bit, and he said that I can do that just by running the PyPy tests, no need to do a full translation. So I think for now I'll just forget about trying to get the translation working on my system, and start playing with the tests. Cheers! Jay P. From rogerb at rogerbinns.com Sun May 15 04:44:24 2011 From: rogerb at rogerbinns.com (Roger Binns) Date: Sat, 14 May 2011 19:44:24 -0700 Subject: [pypy-dev] cpyext: Detecting pypy and other issues In-Reply-To: References: <4DBD247B.7040102@rogerbinns.com> <4DBD310C.9090806@rogerbinns.com> <4DBE3956.90409@rogerbinns.com> <4DBE4F7A.3030109@gmail.com> <4DBF15E0.6080005@rogerbinns.com> <4DC21F58.8000101@rogerbinns.com> <4DCB442F.1080800@rogerbinns.com> <4DCBFE18.3050804@rogerbinns.com> <4DCCC5D5.5020600@rogerbinns.com> <4DCE9080.9030108@rogerbinns.com> Message-ID: <4DCF3E08.8080602@rogerbinns.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/14/2011 08:20 AM, Armin Rigo wrote: > Ah, sorry. I think I got mistaken in my previous mail. The point is > that you are debugging *only* a C extension, and not interested in > debugging any of the rest of PyPy. Correct. More accurately I have a C extension that works perfectly with CPython including very extensive testing and being valgrind clean. However with pypy things are a complete mess. > Then all we said about GCs makes no sense The point of mentioning GCs was to get deterministic behaviour and hopefully to get valgrind working. > You disable it > with "make no_obmalloc", and then they are just normal malloc()ed and > free()d objects from the C point of view. Would you be able to add a target to the Makefile whose purpose is to make a good pypy for C extension debugging. ie no obmalloc, plentiful debug symbols, ideally doing a GC run immediately before and after every call to cpyext wrapped functions. > This is independent of the > GC, which anyway only handles the objects that are in native RPython > format. The GC matters in terms of being deterministic. Here are 3 examples of problems I have encountered when running my test suite (pure python code in a separate file being a subclass of unittest with lots of test* methods): - - My C level object has a member that is initialised with PyList_New(0). A while later a crash happens because it is garbage (a ref count in the tens of millions when it will never go above 1 and is Py_CLEAR'ed in the destructor). - - One early test calls a method implemented in C with an integer parameter. The pyarg_parse wants a string and hence should raise a type error but doesn't. In gdb I want to confirm what cpyext is passing in hence a request for a 'pyo' macro or similar for gdb. - - If I use gdb in the above and immediately hit 'continue' (ie not doing anything else) then pypy fails the test suite on the grounds that particular unittest method it was just executing doesn't exist > or it would if we had _PyObject_Dump() in cpyext. _PyObject_Dump() also spews the string to stdout IIRC. Should I create a ticket asking for this? > Note that the objects that C extension modules see are "placeholder" > PyObjects, that indirectly contain a reference to the "real" RPython > object. gdb only shows two members for a PyObject - a refcount and a type but no pointer to the RPython object. Having just those two fields isn't useful for debugging hence the need for a 'pyo' or similar gdb macro so you know what is hiding behind the PyObject. > I'm pretty sure that the > troubles you get are simply due to the placeholders being freed too > early. The item is allocated in an object initialiser and is Py_CLEAR'ed in the destructor. The destructor being run multiple times could cause a problem. It is a list of weakrefs so weakref handling having issues could also grief. I'm very sure the code is correct, but certainly believe that it could be outwitted if the garbage collector ran at inopportune times. Using valgrind would tell me when the PyObject part is having issues but it would also be nice if the rest of pypy played nice with valgrind as I mentioned using the macros in the previous email as that would then make it clear who exactly was responsible for any allocations, overwrites and frees of memory that could end up at the same spot. > That's the case for which > adding a debugging mode in PyPy would be useful. It is far preferable for the tool to analyse itself and complain at the first opportunity rather than play happy go lucky. I certainly believe that many existing extensions will hit issues be they due to bugs in the extensions, bugs in the CPython documentation and bugs in pypy. It is better to have a way of proactively finding them versus corrupting memory or crashing as that just makes pypy look bad. Roger -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk3PPgQACgkQmOOfHg372QQZRQCgm5Pe2iE6FOBdSk7SF4dad7A5 EnIAoN8CsJK399q4PPjMDt/2msKTHBDm =6ved -----END PGP SIGNATURE----- From cfbolz at gmx.de Sat May 14 20:15:57 2011 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sat, 14 May 2011 20:15:57 +0200 Subject: [pypy-dev] Performance challenge of mutating class variables In-Reply-To: References: Message-ID: <4DCEC6DD.8050601@gmx.de> On 05/14/2011 11:41 AM, Armin Rigo wrote: > Hi David, > > On Fri, May 13, 2011 at 10:10 PM, David Edelsohn wrote: >> The problem was due to the user program's use of a class >> variable as a counter, which caused "unfortunate" behavior in the >> IronPython implementation -- mutating the variable invalidated the >> cache. > > PyPy has indeed mostly the same behavior, but it is still faster than > CPython on the example you give. It is because you don't do anything > with the Nodes. If we modify the example > (http://paste.pocoo.org/show/388541/) to read all '_children' > attributes after all Nodes have been created, then PyPy is very slow. > In this example it is 4x slower than CPython when called with 2000000. > > Thanks for pushing us in trying to find a solution for this problem :-) Did you try this with PyPy 1.5 or a recent nightly? I thought that the new type __dict__ implementation should have fixed the problem? Carl Friedrich From arigo at tunes.org Sun May 15 17:21:55 2011 From: arigo at tunes.org (Armin Rigo) Date: Sun, 15 May 2011 17:21:55 +0200 Subject: [pypy-dev] Performance challenge of mutating class variables In-Reply-To: <4DCEC6DD.8050601@gmx.de> References: <4DCEC6DD.8050601@gmx.de> Message-ID: Hi Carl Friedrich, On Sat, May 14, 2011 at 8:15 PM, Carl Friedrich Bolz wrote: > Did you try this with PyPy 1.5 or a recent nightly? I thought that the new > type __dict__ implementation should have fixed the problem? Indeed, with a recent PyPy things are much better. My example still takes 1.5x time the time it takes on CPython 2.7, though... A bient?t, Armin. From lkcl at lkcl.net Sun May 15 19:11:27 2011 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sun, 15 May 2011 18:11:27 +0100 Subject: [pypy-dev] Performance challenge of mutating class variables In-Reply-To: References: <4DCEC6DD.8050601@gmx.de> Message-ID: apologies, folks, but did somebody change the settings recently on this list? i suddenly start receiving messages when i am subscribed as "not to receive messages" - whoever has changed the options - without my permission - please could they undo what they have done, so that my wishes are respected and i remain subscribed to this list *without* receiving messages? many thanks, l. From arigo at tunes.org Sun May 15 19:29:33 2011 From: arigo at tunes.org (Armin Rigo) Date: Sun, 15 May 2011 19:29:33 +0200 Subject: [pypy-dev] Performance challenge of mutating class variables In-Reply-To: References: <4DCEC6DD.8050601@gmx.de> Message-ID: Hi Luke, On Sun, May 15, 2011 at 7:11 PM, Luke Kenneth Casson Leighton wrote: > apologies, folks, but did somebody change the settings recently on > this list? This list has moved to python.org. Sorry about the settings of the subscribers, which have not all been transferred. I re-disabled mails for you, but if anyone else has a similar problem, please visit http://mail.python.org/mailman/listinfo/pypy-dev . A bient?t, Armin. From lkcl at lkcl.net Sun May 15 19:39:58 2011 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sun, 15 May 2011 18:39:58 +0100 Subject: [pypy-dev] Performance challenge of mutating class variables In-Reply-To: References: <4DCEC6DD.8050601@gmx.de> Message-ID: On Sun, May 15, 2011 at 6:29 PM, Armin Rigo wrote: > Hi Luke, > > On Sun, May 15, 2011 at 7:11 PM, Luke Kenneth Casson Leighton > wrote: >> apologies, folks, but did somebody change the settings recently on >> this list? > > This list has moved to python.org. i noticed that - i thought it was there all along, because even the archives are there (which is impressive) thanks armin. l. From qbproger at gmail.com Sun May 15 20:36:36 2011 From: qbproger at gmail.com (Joe) Date: Sun, 15 May 2011 14:36:36 -0400 Subject: [pypy-dev] sqlite warnings Message-ID: I ran a script I had that used SQLite with storm, and saw a bunch of RuntimeWarnings about args and return times. I took some time to fix them up, and ran the sqlite tests to fix any remaining ones. Joe -------------- next part -------------- A non-text attachment was scrubbed... Name: sqlite_warnings.patch Type: application/octet-stream Size: 5062 bytes Desc: not available URL: From anto.cuni at gmail.com Mon May 16 15:30:41 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Mon, 16 May 2011 15:30:41 +0200 Subject: [pypy-dev] sqlite warnings In-Reply-To: References: Message-ID: <4DD12701.6010703@gmail.com> On 15/05/11 20:36, Joe wrote: > I ran a script I had that used SQLite with storm, and saw a bunch of > RuntimeWarnings about args and return times. I took some time to fix > them up, and ran the sqlite tests to fix any remaining ones. thank you! I applied the patch in revision a0b3a8cef2c4 ciao, Anto From holger at merlinux.eu Mon May 16 15:35:42 2011 From: holger at merlinux.eu (holger krekel) Date: Mon, 16 May 2011 13:35:42 +0000 Subject: [pypy-dev] new bugs.pypy.org / ssl warning Message-ID: <20110516133542.GF7415@merlinux.eu> Hi all, the pypy issue tracker has moved from the good old codespeak host to a new one. If you go to one of the old URLs you should be transparently redirected to https://bugs.pypy.org You should be able to use your original login, however please note: - if you used your codespeak credentials and don'T know your actual roundup password you may need to trigger a password reset via "Lost your Login?" - you'll get an SSL warning because we use a self-signed certificate (the IP is shared with some other domains). FYI the cert is from "merlinux" and has this SHA1 Fingerprint: 1D:EF:49:8D:77:BE:45:B6:16:1E:79:BC:3D:2C:81:67:CE:3B:86:D8 - The tracker email address is now tracker at bugs.pypy.org in case you are using filters. - there maybe some more problems lurking, please report here or to me privately. many thanks to Ronny Pfannschmidt who majorly helped with the migration of the pypy tracker. It is now running on roundup-1.4 instead of an ancient version. This is good preparation for when we potentially/ eventually move to python.org / PSF infrastructure. For now, the tracker remains hosted courtesy of my little company http://merlinux.eu. best, holger From arigo at tunes.org Tue May 17 13:24:31 2011 From: arigo at tunes.org (Armin Rigo) Date: Tue, 17 May 2011 13:24:31 +0200 Subject: [pypy-dev] Performance challenge of mutating class variables In-Reply-To: References: <4DCEC6DD.8050601@gmx.de> Message-ID: Hi David, On Sun, May 15, 2011 at 5:21 PM, Armin Rigo wrote: >> Did you try this with PyPy 1.5 or a recent nightly? I thought that the new >> type __dict__ implementation should have fixed the problem? > > Indeed, with a recent PyPy things are much better. ?My example > still takes 1.5x time the time it takes on CPython 2.7, though... Well, I cannot reproduce this number... I have no clue how I got it :-( The original code you posted is actually not relevant to the JIT: it contains no loop. The fact is that it is still 2-3x faster on PyPy than CPython, for GC reasons: our GC is better at handling quickly growing heaps, and the allocated objects are much smaller anyway. The code I posted used to be really slow in pypy-1.5, because it was trying again and again to JIT it. Now we fixed the issue, and it is again much faster than CPython (this time, about 10x). A bient?t, Armin. From fijall at gmail.com Tue May 17 13:43:38 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 17 May 2011 13:43:38 +0200 Subject: [pypy-dev] Performance challenge of mutating class variables In-Reply-To: References: <4DCEC6DD.8050601@gmx.de> Message-ID: On Tue, May 17, 2011 at 1:24 PM, Armin Rigo wrote: > Hi David, > > On Sun, May 15, 2011 at 5:21 PM, Armin Rigo wrote: >>> Did you try this with PyPy 1.5 or a recent nightly? I thought that the new >>> type __dict__ implementation should have fixed the problem? >> >> Indeed, with a recent PyPy things are much better. ?My example >> still takes 1.5x time the time it takes on CPython 2.7, though... > > Well, I cannot reproduce this number... I have no clue how I got it :-( > > The original code you posted is actually not relevant to the JIT: it > contains no loop. ?The fact is that it is still 2-3x faster on PyPy > than CPython, for GC reasons: our GC is better at handling quickly > growing heaps, and the allocated objects are much smaller anyway. > > The code I posted used to be really slow in pypy-1.5, because it was > trying again and again to JIT it. ?Now we fixed the issue, and it is > again much faster than CPython (this time, about 10x). Might be CPython 2.6 vs CPython 2.7? There were significant GC changes in 2.7, especially when it comes to O(n^2) behavior. > > > A bient?t, > > Armin. > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > From arigo at tunes.org Tue May 17 13:57:59 2011 From: arigo at tunes.org (Armin Rigo) Date: Tue, 17 May 2011 13:57:59 +0200 Subject: [pypy-dev] Performance challenge of mutating class variables In-Reply-To: References: <4DCEC6DD.8050601@gmx.de> Message-ID: Hi Maciej, On Tue, May 17, 2011 at 1:43 PM, Maciej Fijalkowski wrote: > Might be CPython 2.6 vs CPython 2.7? There were significant GC changes > in 2.7, especially when it comes to O(n^2) behavior. I must have messed up my measures. It's true that Python 2.7 is twice as fast as Python 2.6 in this particular example, but PyPy is still much faster than both. A bient?t, Armin. From fijall at gmail.com Tue May 17 14:11:33 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 17 May 2011 14:11:33 +0200 Subject: [pypy-dev] Performance challenge of mutating class variables In-Reply-To: References: <4DCEC6DD.8050601@gmx.de> Message-ID: On Tue, May 17, 2011 at 1:57 PM, Armin Rigo wrote: > Hi Maciej, > > On Tue, May 17, 2011 at 1:43 PM, Maciej Fijalkowski wrote: >> Might be CPython 2.6 vs CPython 2.7? There were significant GC changes >> in 2.7, especially when it comes to O(n^2) behavior. > > I must have messed up my measures. ?It's true that Python 2.7 is twice > as fast as Python 2.6 in this particular example, but PyPy is still > much faster than both. Which reminds me that we should *really* start running python 2.7 as a baseline python > > > A bient?t, > > Armin. > From anto.cuni at gmail.com Tue May 17 14:13:56 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 17 May 2011 14:13:56 +0200 Subject: [pypy-dev] Performance challenge of mutating class variables In-Reply-To: References: <4DCEC6DD.8050601@gmx.de> Message-ID: <4DD26684.5010303@gmail.com> On 17/05/11 14:11, Maciej Fijalkowski wrote: > Which reminds me that we should *really* start running python 2.7 as a > baseline python for this, it should be "enough" to upgrade ubuntu on tannit, which comes with python2.7. Unless we decide that we want to stay with the LTS release. ciao, Anto From pypy at pocketnix.org Tue May 17 14:46:21 2011 From: pypy at pocketnix.org (Da_Blitz) Date: Tue, 17 May 2011 12:46:21 +0000 Subject: [pypy-dev] _pypy_wait.py fixes Message-ID: <20110517124621.GD14145@pocketnix.org> Hi when calling wait3 or wait4 in the os module a ctypes error about return types is printed. a quick patch is included bellow to silence the message --- Da_Blitz diff -r eaba7adf5188 lib_pypy/_pypy_wait.py --- a/lib_pypy/_pypy_wait.py Thu Apr 28 11:46:10 2011 +0200 +++ b/lib_pypy/_pypy_wait.py Tue May 17 22:43:29 2011 +1000 @@ -8,8 +8,10 @@ c_wait3 = libc.wait3 c_wait3.argtypes = [POINTER(c_int), c_int, POINTER(_struct_rusage)] +c_wait3.restype = c_int c_wait4 = libc.wait4 +c_wait4.restype = c_int c_wait4.argtypes = [c_int, POINTER(c_int), c_int, POINTER(_struct_rusage)] From naylor.b.david at gmail.com Wed May 18 20:01:31 2011 From: naylor.b.david at gmail.com (David Naylor) Date: Wed, 18 May 2011 20:01:31 +0200 Subject: [pypy-dev] PyPy platforms/architectures Message-ID: Hi, I am busy writing a port for pypy on FreeBSD. I have the basics done but I have a slight issue with the package list. It appears pypy creates some files based on the architecture name [1] however the names it uses for those files differ to what FreeBSD uses [2]. I've mapped the obvious name difference (i386->x86_32, amd64->x86_64) however I am not sure what pypy uses for the other architects. Also it appears FreeBSD uses "powerpc" for ppc (even though [2] says otherwise). Please can you clarify the architecture names. Regards, David [1] Under lib_pypy/ctypes_config_cache [2] http://www.freebsd.org/platforms/ From fijall at gmail.com Wed May 18 21:46:29 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 18 May 2011 21:46:29 +0200 Subject: [pypy-dev] PyPy platforms/architectures In-Reply-To: References: Message-ID: On Wed, May 18, 2011 at 8:01 PM, David Naylor wrote: > Hi, > > I am busy writing a port for pypy on FreeBSD. ?I have the basics done but > I have a slight issue with the package list. ?It appears pypy creates some > files based on the architecture name [1] however the names it uses for those > files differ to what FreeBSD uses [2]. > > I've mapped the obvious name difference (i386->x86_32, amd64->x86_64) however > I am not sure what pypy uses for the other architects. ?Also it appears FreeBSD > uses "powerpc" for ppc (even though [2] says otherwise). PyPy's officially supported platforms are x86_32 and x86_64 (and arm coming soon), others are at least untested. The ctypes config cache is mostly for not having mismatches between different platforms running on the same machine, they can be named whatever, just as long as they're different for each platform. > > Please can you clarify the architecture names. > > Regards, > > David > > > [1] Under lib_pypy/ctypes_config_cache > [2] http://www.freebsd.org/platforms/ > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > From naylor.b.david at gmail.com Thu May 19 17:55:16 2011 From: naylor.b.david at gmail.com (David Naylor) Date: Thu, 19 May 2011 17:55:16 +0200 Subject: [pypy-dev] PyPy compile memory requirements Message-ID: Hi, I'm sure you all know that translating pypy requires a large amount of RAM. It has been my experience that GCC also uses a large amount of RAM when compiling, about 4G. Is it possible to get python/pypy to stop once it has written the source files and then to run that build process separately, thus avoiding having both pypy and gcc in memory at the same time? This is on FreeBSD/amd64: # gcc -v Using built-in specs. Target: amd64-undermydesk-freebsd Configured with: FreeBSD/amd64 system compiler Thread model: posix gcc version 4.2.2 20070831 prerelease [FreeBSD] I've observed similar behavior using other versions of gcc. Clang does not exhibit this problem. Regards, David From fijall at gmail.com Thu May 19 18:00:06 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 19 May 2011 18:00:06 +0200 Subject: [pypy-dev] PyPy compile memory requirements In-Reply-To: References: Message-ID: On Thu, May 19, 2011 at 5:55 PM, David Naylor wrote: > Hi, > > I'm sure you all know that translating pypy requires a large amount of RAM. > > It has been my experience that GCC also uses a large amount of RAM when > compiling, about 4G. ?Is it possible to get python/pypy to stop once > it has written > the source files and then to run that build process separately, thus avoiding > having both pypy and gcc in memory at the same time? This is probably due to some GCC bug (I remember the same thing occuring on ubuntu a while ago). If the only goal is to avoid doing those 2 at the same time, do translate.py --source and then go to the /tmp/usession-*/testing_1 directory and type make Note that it should nicely work with swap (since the python process is not running any more when gcc is) > > This is on FreeBSD/amd64: > # gcc -v > Using built-in specs. > Target: amd64-undermydesk-freebsd > Configured with: FreeBSD/amd64 system compiler > Thread model: posix > gcc version 4.2.2 20070831 prerelease [FreeBSD] > > I've observed similar behavior using other versions of gcc. ?Clang does not > exhibit this problem. > > Regards, > > David > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > From arigo at tunes.org Thu May 19 18:34:16 2011 From: arigo at tunes.org (Armin Rigo) Date: Thu, 19 May 2011 18:34:16 +0200 Subject: [pypy-dev] PyPy compile memory requirements In-Reply-To: References: Message-ID: Hi, On Thu, May 19, 2011 at 6:00 PM, Maciej Fijalkowski wrote: > If the only goal is to avoid doing those 2 at the same time, do > translate.py --source and then go to the /tmp/usession-*/testing_1 > directory and type make We should decide if it would make sense or not to just do os.execv("make"), which would have the same effect. The only drawback I can think of would be that you wouldn't get the nice summary and timings at the end of translation --- which we can maybe re-obtain anyway by hacking a bit :-) A bient?t, Armin. From Ronny.Pfannschmidt at gmx.de Thu May 19 20:30:50 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Thu, 19 May 2011 20:30:50 +0200 Subject: [pypy-dev] PyPy compile memory requirements In-Reply-To: References: Message-ID: <1305829850.422.105.camel@Klappe2> On Thu, 2011-05-19 at 18:34 +0200, Armin Rigo wrote: > Hi, > > On Thu, May 19, 2011 at 6:00 PM, Maciej Fijalkowski wrote: > > If the only goal is to avoid doing those 2 at the same time, do > > translate.py --source and then go to the /tmp/usession-*/testing_1 > > directory and type make > > We should decide if it would make sense or not to just do > os.execv("make"), which would have the same effect. The only drawback > I can think of would be that you wouldn't get the nice summary and > timings at the end of translation --- which we can maybe re-obtain > anyway by hacking a bit :-) why not run the translation in a subprocess to begin with? > > > A bient?t, > > Armin. > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From gabomdq at gmail.com Fri May 20 01:58:16 2011 From: gabomdq at gmail.com (Gabriel Jacobo) Date: Thu, 19 May 2011 20:58:16 -0300 Subject: [pypy-dev] Properties under RPython Message-ID: Hi All! I have a quick question for the RPython gurus, are properties supported? I thought from reading pypy tests and some mentions in IRC logs that they should work (plus I haven't found any of the RPython rules explicitly forbidding them), but I'm hitting a dead end trying to use them. If someone confirms that they at least should work in theory, I'll post again with more details. Thanks for your help. -- Gabriel. -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Fri May 20 01:59:49 2011 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 19 May 2011 18:59:49 -0500 Subject: [pypy-dev] Properties under RPython In-Reply-To: References: Message-ID: 2011/5/19 Gabriel Jacobo : > Hi All! I have a quick question for the RPython gurus, are properties > supported? No. > I thought from reading pypy tests and some mentions in IRC logs that they Tests are not RPython. -- Regards, Benjamin From anto.cuni at gmail.com Fri May 20 09:10:38 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 20 May 2011 09:10:38 +0200 Subject: [pypy-dev] Properties under RPython In-Reply-To: References: Message-ID: <4DD613EE.30003@gmail.com> On 20/05/11 01:59, Benjamin Peterson wrote: >> I thought from reading pypy tests and some mentions in IRC logs that they > > Tests are not RPython. well, not all of them. But e.g., the tests in pypy/rpython/test do contain rpython code. As a general rule, to check if something is supported by rpython, you should check that directory, and in particular the code *inside* the functions being passed to self.interpret. ciao, Anto From arigo at tunes.org Fri May 20 10:07:28 2011 From: arigo at tunes.org (Armin Rigo) Date: Fri, 20 May 2011 10:07:28 +0200 Subject: [pypy-dev] PyPy compile memory requirements In-Reply-To: <1305829850.422.105.camel@Klappe2> References: <1305829850.422.105.camel@Klappe2> Message-ID: Hi Ronny, On Thu, May 19, 2011 at 8:30 PM, Ronny Pfannschmidt wrote: > why not run the translation in a subprocess to begin with? It doesn't change much, except that it would break half of the non-standard use cases of translate.py, like --fork-before. Armin. From gabomdq at gmail.com Fri May 20 17:19:57 2011 From: gabomdq at gmail.com (Gabriel Jacobo) Date: Fri, 20 May 2011 12:19:57 -0300 Subject: [pypy-dev] Properties under RPython In-Reply-To: References: Message-ID: <201105201219.57737.gabomdq@gmail.com> On Jue 19 May 2011 20:59:49 Benjamin Peterson escribi?: > 2011/5/19 Gabriel Jacobo : > > Hi All! I have a quick question for the RPython gurus, are properties > > supported? > > No. Ok, thanks for the answer. How would you grade the difficulty in implementing them (Considering I have plenty of Python experience but almost none with PyPy or RPython)? Any pointers as to how to implement them? Are they a desired feature for RPython? Thanks again, Gabriel. From benjamin at python.org Fri May 20 18:44:15 2011 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 20 May 2011 11:44:15 -0500 Subject: [pypy-dev] Properties under RPython In-Reply-To: <201105201219.57737.gabomdq@gmail.com> References: <201105201219.57737.gabomdq@gmail.com> Message-ID: 2011/5/20 Gabriel Jacobo : > On Jue 19 May 2011 20:59:49 Benjamin Peterson escribi?: >> 2011/5/19 Gabriel Jacobo : >> > Hi All! I have a quick question for the RPython gurus, are properties >> > supported? >> >> No. > > Ok, thanks for the answer. How would you grade the difficulty in implementing > them (Considering I have plenty of Python experience but almost none with PyPy > or RPython)? Any pointers as to how to implement them? Are they a desired > feature for RPython? Generally, we try to avoid adding new feature to the RPython language. Usually, there's a simpler way to achieve what you want. -- Regards, Benjamin From gabomdq at gmail.com Fri May 20 21:36:07 2011 From: gabomdq at gmail.com (Gabriel Jacobo) Date: Fri, 20 May 2011 16:36:07 -0300 Subject: [pypy-dev] Properties under RPython In-Reply-To: References: <201105201219.57737.gabomdq@gmail.com> Message-ID: <201105201636.07136.gabomdq@gmail.com> On Vie 20 May 2011 13:44:15 Benjamin Peterson escribi?: > 2011/5/20 Gabriel Jacobo : > > On Jue 19 May 2011 20:59:49 Benjamin Peterson escribi?: > >> 2011/5/19 Gabriel Jacobo : > >> > Hi All! I have a quick question for the RPython gurus, are properties > >> > supported? > >> > >> No. > > > > Ok, thanks for the answer. How would you grade the difficulty in > > implementing them (Considering I have plenty of Python experience but > > almost none with PyPy or RPython)? Any pointers as to how to implement > > them? Are they a desired feature for RPython? > > Generally, we try to avoid adding new feature to the RPython language. > Usually, there's a simpler way to achieve what you want. I have a good chunk of python code that uses properties all around, and I wanted to see if I can run it through the pypy machinery and eventually come the other side with a javascript end product (via RPythonic). Do you have any suggestions as to best solve this situation? (Preprocessing the code somehow perhaps?) Thanks, Gabriel. From benjamin at python.org Fri May 20 21:47:54 2011 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 20 May 2011 14:47:54 -0500 Subject: [pypy-dev] Properties under RPython In-Reply-To: <201105201636.07136.gabomdq@gmail.com> References: <201105201219.57737.gabomdq@gmail.com> <201105201636.07136.gabomdq@gmail.com> Message-ID: 2011/5/20 Gabriel Jacobo : > On Vie 20 May 2011 13:44:15 Benjamin Peterson escribi?: >> 2011/5/20 Gabriel Jacobo : >> > On Jue 19 May 2011 20:59:49 Benjamin Peterson escribi?: >> >> 2011/5/19 Gabriel Jacobo : >> >> > Hi All! I have a quick question for the RPython gurus, are properties >> >> > supported? >> >> >> >> No. >> > >> > Ok, thanks for the answer. How would you grade the difficulty in >> > implementing them (Considering I have plenty of Python experience but >> > almost none with PyPy or RPython)? Any pointers as to how to implement >> > them? Are they a desired feature for RPython? >> >> Generally, we try to avoid adding new feature to the RPython language. >> Usually, there's a simpler way to achieve what you want. > > I have a good chunk of python code that uses properties all around, and I > wanted to see if I can run it through the pypy machinery and eventually come > the other side with a javascript end product (via RPythonic). Do you have any > suggestions as to best solve this situation? (Preprocessing the code somehow > perhaps?) Uh, the javascript backend doesn't really work anymore. -- Regards, Benjamin From gabomdq at gmail.com Fri May 20 21:51:22 2011 From: gabomdq at gmail.com (Gabriel Jacobo) Date: Fri, 20 May 2011 16:51:22 -0300 Subject: [pypy-dev] Properties under RPython In-Reply-To: References: <201105201636.07136.gabomdq@gmail.com> Message-ID: <201105201651.22405.gabomdq@gmail.com> On Vie 20 May 2011 16:47:54 Benjamin Peterson escribi?: > 2011/5/20 Gabriel Jacobo : > > On Vie 20 May 2011 13:44:15 Benjamin Peterson escribi?: > >> 2011/5/20 Gabriel Jacobo : > >> > On Jue 19 May 2011 20:59:49 Benjamin Peterson escribi?: > >> >> 2011/5/19 Gabriel Jacobo : > >> >> > Hi All! I have a quick question for the RPython gurus, are > >> >> > properties supported? > >> >> > >> >> No. > >> > > >> > Ok, thanks for the answer. How would you grade the difficulty in > >> > implementing them (Considering I have plenty of Python experience but > >> > almost none with PyPy or RPython)? Any pointers as to how to implement > >> > them? Are they a desired feature for RPython? > >> > >> Generally, we try to avoid adding new feature to the RPython language. > >> Usually, there's a simpler way to achieve what you want. > > > > I have a good chunk of python code that uses properties all around, and I > > wanted to see if I can run it through the pypy machinery and eventually > > come the other side with a javascript end product (via RPythonic). Do > > you have any suggestions as to best solve this situation? (Preprocessing > > the code somehow perhaps?) > > Uh, the javascript backend doesn't really work anymore. I'm not using the javascript backend, I mentioned I'm using RPythonic which goes with the LLVM->Emscripten->Javascript route. No ideas on a property workaround? From mohyiddeen at yahoo.com Fri May 20 23:23:22 2011 From: mohyiddeen at yahoo.com (Mohyiddeen Othman) Date: Fri, 20 May 2011 14:23:22 -0700 (PDT) Subject: [pypy-dev] List Slicing Performance Drop Message-ID: <437154.75121.qm@web121808.mail.ne1.yahoo.com> Hi, I've been recently doing Project Euler puzzles using Python, and have recently started to use PyPy to increase performance. However I've come across something (no pun intended) puzzling. I made this script for Euler #50. http://pastebin.com/VZ12Nsci It calculates all the primes under 1M first, then looks through them for a solution. The first part (finding primes) is blazing fast under PyPy (as expected). But in the second part PyPy lags behind. CPython managed it in 183s, while PyPy did it in 498s. I'm assuming it has something to do with the list slicing? -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Fri May 20 23:51:03 2011 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 20 May 2011 16:51:03 -0500 Subject: [pypy-dev] Properties under RPython In-Reply-To: <201105201651.22405.gabomdq@gmail.com> References: <201105201636.07136.gabomdq@gmail.com> <201105201651.22405.gabomdq@gmail.com> Message-ID: 2011/5/20 Gabriel Jacobo : > On Vie 20 May 2011 16:47:54 Benjamin Peterson escribi?: >> 2011/5/20 Gabriel Jacobo : >> > On Vie 20 May 2011 13:44:15 Benjamin Peterson escribi?: >> >> 2011/5/20 Gabriel Jacobo : >> >> > On Jue 19 May 2011 20:59:49 Benjamin Peterson escribi?: >> >> >> 2011/5/19 Gabriel Jacobo : >> >> >> > Hi All! I have a quick question for the RPython gurus, are >> >> >> > properties supported? >> >> >> >> >> >> No. >> >> > >> >> > Ok, thanks for the answer. How would you grade the difficulty in >> >> > implementing them (Considering I have plenty of Python experience but >> >> > almost none with PyPy or RPython)? Any pointers as to how to implement >> >> > them? Are they a desired feature for RPython? >> >> >> >> Generally, we try to avoid adding new feature to the RPython language. >> >> Usually, there's a simpler way to achieve what you want. >> > >> > I have a good chunk of python code that uses properties all around, and I >> > wanted to see if I can run it through the pypy machinery and eventually >> > come the other side with a javascript end product (via RPythonic). Do >> > you have any suggestions as to best solve this situation? (Preprocessing >> > the code somehow perhaps?) >> >> Uh, the javascript backend doesn't really work anymore. > > I'm not using the javascript backend, I mentioned I'm using RPythonic which > goes with the LLVM->Emscripten->Javascript route. No ideas on a property > workaround? You could rewrite to have getters like get_my_attr or some such. Using RPython for random projects is not really supported... -- Regards, Benjamin From pypy at pocketnix.org Sat May 21 11:13:58 2011 From: pypy at pocketnix.org (Da_Blitz) Date: Sat, 21 May 2011 09:13:58 +0000 Subject: [pypy-dev] Stackless work Message-ID: <20110521091358.GA23907@pocketnix.org> Hi keeping this one short and simple I am interested in using JIT enabled stackless with pypy (obviously a feature that doesn't exist). i have been playing with the coroutines interface (which i am finding nicer to work with than tasklets) but don't have the skill to pull of the stackless work it has been suggested that pypy devs are willing to work on paid for features. so if anyone is interested in doing work on stackless and being paid for it please feel free to get in touch of me so we can discuss it (either on irc or via email) there isnt a deadline for compleation of this feature, just me wanting to throw a firecracker at someone to at the very least get it started Da_Blitz From anto.cuni at gmail.com Sat May 21 12:22:23 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Sat, 21 May 2011 12:22:23 +0200 Subject: [pypy-dev] JVM backend - status and future work possibilities? In-Reply-To: References: <4DCC5A54.7010500@gmail.com> Message-ID: <4DD7925F.9090905@gmail.com> On 16/05/11 17:10, Bo Chen wrote: > Anto, Hello Bo, (I've CCed pypy-dev again, please click "reply all" to make sure the list is included) > Is there any advice on reading the source code of the pypy? that's a hard question, it depends what is your goal. My experience is that if you have a clear goal and start by writing a failing test, then it's easier to follow what's going on in that particular case. Personally, I love to put pdb.set_trace() here and there to watch the execution of the code in the pieces I'm interested in. > And here is my question, the JVM backend is to invoke java classes in pypy or > translate the python binary to java binary, or both? The JVM backend translates RPython source code into JVM bytecode (actually, it generates a textual form of the JVM bytecode, which is then assembled by jasmin). Currently, you cannot create/call Java entities from RPython code, but as I explained in the earlier emails, this is a prerequisite feature before doing the rest of the work. > There is a prototype use JPype, is this mean the JVM backend turn the java > binary code into the python's. no. JPype allows to call Java code from CPython, but there is no translation of java into python involved. Note that JPype would be used only for tests, and not for the actual translation. This is because the RPython code can be either run on top of cpython (for testing) or translated to jvm code. JPype (or equivalent) is needed for the first case For the second case, JPype does not play any role. What it's needed is to teach the RPython toolchain how to deal with Java classes. Remind that the JVM backend already emits JVM bytecode, so it's "enough" to teach the RPython toolchain to transform e.g. "rjvm.System.out.println" into the corresponding invokestatic bytecode. > And only the JIT complier will transfer the python to java. no JIT is involved at this point. I think it's better to keep the JIT out of this particular discussion, to avoid confusion. > And the Point(1) would be not only just use the JPype, but also need > implement totally? see my explanation above, hoping that it answer your question. ciao, Anto From wlavrijsen at lbl.gov Sun May 22 04:54:08 2011 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Sat, 21 May 2011 19:54:08 -0700 (PDT) Subject: [pypy-dev] help with translation (error in JIT code) Message-ID: Hi, trying to translate reflex-support branch, which was recently move up to 1.5, I'm stuck at the error below. Any ideas? Thanks! Best regards, Wim [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 306, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/translator/driver.py", line 809, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/translator/driver.py", line 286, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/translator/driver.py", line 397, in task_pyjitpl_lltype [translation:ERROR] backend_name=self.config.translation.jit_backend, inline=True) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/jit/metainterp/warmspot.py", line 46, in apply_jit [translation:ERROR] **kwds) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/jit/metainterp/warmspot.py", line 196, in __init__ [translation:ERROR] self.codewriter.make_jitcodes(verbose=verbose) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/jit/codewriter/codewriter.py", line 72, in make_jitcodes [translation:ERROR] self.transform_graph_to_jitcode(graph, jitcode, verbose) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/jit/codewriter/codewriter.py", line 41, in transform_graph_to_jitcode [translation:ERROR] transform_graph(graph, self.cpu, self.callcontrol, portal_jd) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/jit/codewriter/jtransform.py", line 25, in transform_graph [translation:ERROR] t.transform(graph) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/jit/codewriter/jtransform.py", line 39, in transform [translation:ERROR] self.optimize_block(block) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/jit/codewriter/jtransform.py", line 61, in optimize_block [translation:ERROR] oplist = self.rewrite_operation(op) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/jit/codewriter/jtransform.py", line 196, in rewrite_operation [translation:ERROR] return rewrite(self, op) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/jit/codewriter/jtransform.py", line 275, in rewrite_op_direct_call [translation:ERROR] return getattr(self, 'handle_%s_call' % kind)(op) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/jit/codewriter/jtransform.py", line 323, in handle_residual_call [translation:ERROR] calldescr = self.callcontrol.getcalldescr(op) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/jit/codewriter/call.py", line 208, in getcalldescr [translation:ERROR] assert NON_VOID_ARGS == [T for T in ARGS if T is not lltype.Void] [translation:ERROR] AssertionError -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From glbrt.chen at gmail.com Sun May 22 17:00:42 2011 From: glbrt.chen at gmail.com (Bo Chen) Date: Sun, 22 May 2011 23:00:42 +0800 Subject: [pypy-dev] JVM backend - status and future work possibilities? In-Reply-To: <4DD7925F.9090905@gmail.com> References: <4DCC5A54.7010500@gmail.com> <4DD7925F.9090905@gmail.com> Message-ID: Anto, Thanks for your patient. I will read the code of RPython first. Bo Chen On Sat, May 21, 2011 at 6:22 PM, Antonio Cuni wrote: > On 16/05/11 17:10, Bo Chen wrote: > > Anto, > > Hello Bo, > (I've CCed pypy-dev again, please click "reply all" to make sure the list > is > included) > > > Is there any advice on reading the source code of the pypy? > > that's a hard question, it depends what is your goal. My experience is that > if > you have a clear goal and start by writing a failing test, then it's easier > to > follow what's going on in that particular case. > > Personally, I love to put pdb.set_trace() here and there to watch the > execution of the code in the pieces I'm interested in. > > > And here is my question, the JVM backend is to invoke java classes in > pypy or > > translate the python binary to java binary, or both? > > The JVM backend translates RPython source code into JVM bytecode (actually, > it > generates a textual form of the JVM bytecode, which is then assembled by > jasmin). > > Currently, you cannot create/call Java entities from RPython code, but as I > explained in the earlier emails, this is a prerequisite feature before > doing > the rest of the work. > > > There is a prototype use JPype, is this mean the JVM backend turn the > java > > binary code into the python's. > > no. JPype allows to call Java code from CPython, but there is no > translation > of java into python involved. > > Note that JPype would be used only for tests, and not for the actual > translation. This is because the RPython code can be either run on top of > cpython (for testing) or translated to jvm code. JPype (or equivalent) is > needed for the first case > > For the second case, JPype does not play any role. What it's needed is to > teach the RPython toolchain how to deal with Java classes. Remind that the > JVM backend already emits JVM bytecode, so it's "enough" to teach the > RPython > toolchain to transform e.g. "rjvm.System.out.println" into the > corresponding > invokestatic bytecode. > > > And only the JIT complier will transfer the python to java. > > no JIT is involved at this point. I think it's better to keep the JIT out > of > this particular discussion, to avoid confusion. > > > And the Point(1) would be not only just use the JPype, but also need > > implement totally? > > see my explanation above, hoping that it answer your question. > > ciao, > Anto > -------------- next part -------------- An HTML attachment was scrubbed... URL: From naylor.b.david at gmail.com Sun May 22 23:04:00 2011 From: naylor.b.david at gmail.com (David Naylor) Date: Sun, 22 May 2011 23:04:00 +0200 Subject: [pypy-dev] [freebsd] pypy-1.5.0 fails to compile sandbox Message-ID: Hi, I'm trying to compile pypy with the sandbox option however it fails with: cc -pthread -Wl,--export-dynamic,--version-script=../dynamic-symbols-0 -o pypy-c testing_1.o structimpl.o nonfuncnodes.o nonfuncnodes_1.o nonfuncnodes_2.o nonfuncnodes_3.o nonfuncnodes_4.o nonfuncnodes_5.o nonfuncnodes_6.o nonfuncnodes_7.o nonfuncnodes_8.o nonfuncnodes_9.o nonfuncnodes_10.o nonfuncnodes_11.o nonfuncnodes_12.o nonfuncnodes_13.o nonfuncnodes_14.o nonfuncnodes_15.o nonfuncnodes_16.o nonfuncnodes_17.o nonfuncnodes_18.o nonfuncnodes_19.o nonfuncnodes_20.o nonfuncnodes_21.o nonfuncnodes_22.o nonfuncnodes_23.o nonfuncnodes_24.o nonfuncnodes_25.o nonfuncnodes_26.o implement.o implement_1.o implement_2.o implement_3.o implement_4.o implement_5.o implement_6.o implement_7.o implement_8.o implement_9.o implement_10.o implement_11.o implement_12.o implement_13.o implement_14.o implement_15.o implement_16.o implement_17.o implement_18.o implement_19.o implement_20.o implement_21.o implement_22.o implement_23.o implement_24.o implement_25.o implement_26.o implement_27.o implement_28.o implement_29.o implement_30.o implement_31.o implement_32.o implement_33.o implement_34.o implement_35.o implement_36.o implement_37.o implement_38.o implement_39.o implement_40.o implement_41.o implement_42.o implement_43.o implement_44.o implement_45.o implement_46.o implement_47.o implement_48.o implement_49.o implement_50.o implement_51.o implement_52.o implement_53.o implement_54.o implement_55.o implement_56.o implement_57.o implement_58.o implement_59.o implement_60.o profiling.o debug_print.o ../module_cache/module_0.o ../module_cache/module_1.o -lintl -lm /usr/bin/ld: cannot find -lintl *** Error code 1 Under FreeBSD libintl is installed under /usr/local/lib, which requires -L/usr/local/lib for the compiler to find it. That path is included in the platform/freebsd.py code and I set PYPY_LOCALBASE=/usr/local. If this has been fixed in trunk, please could you point me to the revision that fixed it. Regards, David From wlavrijsen at lbl.gov Mon May 23 03:38:17 2011 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Sun, 22 May 2011 18:38:17 -0700 (PDT) Subject: [pypy-dev] help with translation (error in JIT code) In-Reply-To: References: Message-ID: Hi, replying to myself: it was a pointer of char passed through some rpython code to an external C function that was declared with a pointer of void for the argument list. Fixed now. :) Best regards, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From fijall at gmail.com Mon May 23 09:49:10 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 23 May 2011 09:49:10 +0200 Subject: [pypy-dev] [pypy-commit] pypy default: Move the loop in contains() into a seperate function, so it can be inlined if it doesn't hit the fallback case. In-Reply-To: <20110521030849.BA7AB8205C@wyvern.cs.uni-duesseldorf.de> References: <20110521030849.BA7AB8205C@wyvern.cs.uni-duesseldorf.de> Message-ID: We should do such stuff automatically - it's getting annoying. On Sat, May 21, 2011 at 5:08 AM, alex_gaynor wrote: > Author: Alex Gaynor > Branch: > Changeset: r44339:5809a5ace634 > Date: 2011-05-20 22:19 -0500 > http://bitbucket.org/pypy/pypy/changeset/5809a5ace634/ > > Log: ? ?Move the loop in contains() into a seperate function, so it can be > ? ? ? ?inlined if it doesn't hit the fallback case. > From anto.cuni at gmail.com Mon May 23 10:11:35 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Mon, 23 May 2011 10:11:35 +0200 Subject: [pypy-dev] List Slicing Performance Drop In-Reply-To: <437154.75121.qm@web121808.mail.ne1.yahoo.com> References: <437154.75121.qm@web121808.mail.ne1.yahoo.com> Message-ID: <4DDA16B7.8020401@gmail.com> On 20/05/11 23:23, Mohyiddeen Othman wrote: > Hi, > > I've been recently doing Project Euler puzzles using Python, and have recently > started to use PyPy to increase performance. However I've come across > something (no pun intended) puzzling. I made this script for Euler #50. > > http://pastebin.com/VZ12Nsci > > It calculates all the primes under 1M first, then looks through them for a > solution. The first part (finding primes) is blazing fast under PyPy (as > expected). But in the second part PyPy lags behind. CPython managed it in > 183s, while PyPy did it in 498s. I'm assuming it has something to do with the > list slicing? I cannot reproduce, for me pypy is faster on both pieces of code: viper ~ $ pypy-c /tmp/euler.py ('All primes below', 1000000, 'found') 0.43 Max sequence is: .... .... ('It contains', 543, 'elements') ('The sum is', 997651) 72.61 viper ~ $ python /tmp/euler.py ('All primes below', 1000000, 'found') 4.58 Max sequence is: .... .... ('It contains', 543, 'elements') ('The sum is', 997651) 128.81 Could you tell us more about your environment, like which OS, cpu, 32/64 bit and the exact version of pypy/cpython you used to do the benchmark please? ciao, Anto From andrewfr_ice at yahoo.com Mon May 23 16:11:59 2011 From: andrewfr_ice at yahoo.com (Andrew Francis) Date: Mon, 23 May 2011 07:11:59 -0700 (PDT) Subject: [pypy-dev] Tealets Re: Stackless work Message-ID: <113102.33326.qm@web120702.mail.ne1.yahoo.com> Hi Da_Blitz: --- On Sat, 5/21/11, Da_Blitz wrote: > keeping this one short and simple > > I am interested in using JIT enabled stackless with pypy > (obviously a > feature that doesn't exist). i have been playing with the > coroutines > interface (which i am finding nicer to work with than > tasklets) but > don't have the skill to pull of the stackless work > > it has been suggested that pypy devs are willing to work on > paid for > features. so if anyone is interested in doing work on > stackless and > being paid for it please feel free to get in touch of me so > we can > discuss it (either on irc or via email) > > there isnt a deadline for compleation of this feature, just > me wanting > to throw a firecracker at someone to at the very least get > it started Well Da_Blitz, a few months ago, there was talk on creating what I believe Christian told me are "tealets"(I think Armin came up with the term and Maciej the concept). This would be a quick way to get the JIT to work with greenlets under the hood. This would not be as powerful as if one could get the stackless transform to work with the JIT (i.e., no pickling). I wanted to help and I started to look at how greenlets work. Also at Pycon, Christian was gracious in explaining how based Stackless Python works. My own knowledge of PyPY is still really sketchy. Right now my own stackless.py efforts have been in a creating a new concurrency construct called "join patterns." Cheers, Andrew From cfbolz at gmx.de Mon May 23 16:51:56 2011 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Mon, 23 May 2011 16:51:56 +0200 Subject: [pypy-dev] Tealets Re: Stackless work In-Reply-To: <113102.33326.qm@web120702.mail.ne1.yahoo.com> References: <113102.33326.qm@web120702.mail.ne1.yahoo.com> Message-ID: <4DDA748C.3040901@gmx.de> On 05/23/2011 04:11 PM, Andrew Francis wrote: > Hi Da_Blitz: > > --- On Sat, 5/21/11, Da_Blitz wrote: > >> keeping this one short and simple >> >> I am interested in using JIT enabled stackless with pypy (obviously >> a feature that doesn't exist). i have been playing with the >> coroutines interface (which i am finding nicer to work with than >> tasklets) but don't have the skill to pull of the stackless work >> >> it has been suggested that pypy devs are willing to work on paid >> for features. so if anyone is interested in doing work on stackless >> and being paid for it please feel free to get in touch of me so we >> can discuss it (either on irc or via email) >> >> there isnt a deadline for compleation of this feature, just me >> wanting to throw a firecracker at someone to at the very least get >> it started > > Well Da_Blitz, a few months ago, there was talk on creating what I > believe Christian told me are "tealets"(I think Armin came up with > the term and Maciej the concept). This would be a quick way to get > the JIT to work with greenlets under the hood. This would not be as > powerful as if one could get the stackless transform to work with the > JIT (i.e., no pickling). The last bit is no longer true. I started refactoring pickling on the branch unpickle-frames so that they can work also with tealets. Carl Friedrich From anto.cuni at gmail.com Mon May 23 21:34:39 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Mon, 23 May 2011 21:34:39 +0200 Subject: [pypy-dev] PyPy Genova-Pegli Post-EuroPython Sprint June 27 - July 2 2011 Message-ID: <4DDAB6CF.9040509@gmail.com> PyPy Genova-Pegli Post-EuroPython Sprint June 27 - July 2 2011 ============================================================== The next PyPy sprint will be in Genova-Pegli, Italy, the week after EuroPython (which is in Florence, about 3h away by train). This is a fully public sprint: newcomers and topics other than those proposed below are welcome. ------------------------------ Goals and topics of the sprint ------------------------------ * Now that we have released 1.5, the sprint itself is going to be mainly working on fixing issues reported by various users. Possible topics include, but are not limited to: - fixing issues in the bug tracker - improve cpyext, the C-API compatibility layer, to support more extension modules - finish/improve/merge jitypes2, the branch which makes ctypes JIT friendly - general JIT improvements - improve our tools, like the jitviewer or the buildbot infrastructure - make your favorite module/application working on PyPy, if it doesn't yet * Of course this does not prevent people from showing up with a more precise interest in mind If there are newcomers, we will gladly give introduction talks. * Since we are almost on the beach, we can take one day off for summer relaxation and/or tourist visits nearby :-). ----------- Exact times ----------- The work days should be 27 June - 2 July 2011. People may arrive on the 26th already and/or leave on the 3rd. ----------------------- Location & Accomodation ----------------------- Both the sprint venue and the lodging will be at Albergo Puppo[1] in Genova-Pegli, Italy. Pegli is a nice and peaceful little quarter of Genova, and the hotel is directly on the beach, making it a perfect place for those who want to enjoy the sea in the middle of the Italian summer, as a quick search on Google Images shows[2] :-) [1] http://www.albergopuppo.com/inglese/index.htm [2] http://images.google.com/images?q=genova%20pegli The place has a good ADSL Internet connexion with wireless installed. You can of course arrange your own lodging anywhere but I definitely recommend lodging there too. Please *confirm* that you are coming so that we can adjust the reservations as appropriate. The prices are as follows, and they include breakfast and a parking place for the car, in case you need it: - single room: 70 ? - double room: 95 ? - triple room: 105 ? Please register by hg: http://bitbucket.org/pypy/extradoc/src/default/sprintinfo/genova-pegli-2011/people.txt or on the pypy-dev mailing list if you do not yet have check-in rights: http://mail.python.org/mailman/listinfo/pypy-dev In case you want to share a room with someone else but you don't know who, please let us know (either by writing it directly in people.txt or by writing on the mailing list) and we will try to arrange it. From anto.cuni at gmail.com Mon May 23 21:38:01 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Mon, 23 May 2011 21:38:01 +0200 Subject: [pypy-dev] PyPy Genova-Pegli Post-EuroPython Sprint June 27 - July 2 2011 In-Reply-To: <4DDAB6CF.9040509@gmail.com> References: <4DDAB6CF.9040509@gmail.com> Message-ID: <4DDAB799.4000207@gmail.com> On 23/05/11 21:34, Antonio Cuni wrote: > Please register by hg: > > > http://bitbucket.org/pypy/extradoc/src/default/sprintinfo/genova-pegli-2011/people.txt > sorry, the link above is broken: https://bitbucket.org/pypy/extradoc/src/extradoc/sprintinfo/genova-pegli-2011/people.txt ciao, Anto From andrewfr_ice at yahoo.com Mon May 23 23:14:12 2011 From: andrewfr_ice at yahoo.com (Andrew Francis) Date: Mon, 23 May 2011 14:14:12 -0700 (PDT) Subject: [pypy-dev] Tealets Re: Stackless work Message-ID: <606602.99065.qm@web120708.mail.ne1.yahoo.com> Hi Da_Blitz: --- On Mon, 5/23/11, Da_Blitz wrote: > i have seen some tealets work but no documentation from > what i have overheard it seems like stackless should be posible if > someone devoes quite a bit of effort to it, I just googled and I see work has been done. Too cool! > got? anymore info on this join pattern, i am hoping/assuming its not a fork >and join to offload to other cores as i am more concerned > about having huge amounts of threads/tasklets Join Patterns come out of JoCAML and Polyphonic C#. They are a concurrency mechanism. Hopefully, Join Patterns will allow a Stackless programmer to do stuff like # block until all nine reindeer channels are ready to receive, OR # any three of Santa's too many to count army of elves OR # a timeout occurs reindeer = [CUPID, DASHER, PRANCER, BLITZEN, DANCER, COMET, VIXEN, \ ? ? ? ? ? ? DONNER,RUDOLPH] results = stackless.select(joinPattern(reindeer, joinPattern(elves, 3), ? ? ? ? ? ? ? ? ? ? ? ? ? timeout) if results == reindeer: ???for result in reindeer: ? ? ???# read data from each reindeer ? ? ???..... right now join patterns and stackless.select() are not official parts of C based Stackless Python or stackless.py Cheers, Andrew From andrewfr_ice at yahoo.com Mon May 23 23:16:10 2011 From: andrewfr_ice at yahoo.com (Andrew Francis) Date: Mon, 23 May 2011 14:16:10 -0700 (PDT) Subject: [pypy-dev] Tealets Re: Stackless work In-Reply-To: <4DDA748C.3040901@gmx.de> Message-ID: <469923.35662.qm@web120716.mail.ne1.yahoo.com> Hi Carl: --- On Mon, 5/23/11, Carl Friedrich Bolz wrote: > The last bit is no longer true. I started refactoring > pickling on the branch unpickle-frames so that they can work also with > tealets. Gladly noted. Let me download tealets and see if I can play with it! Cheers, Andrew From wlavrijsen at lbl.gov Tue May 24 07:09:56 2011 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Mon, 23 May 2011 22:09:56 -0700 (PDT) Subject: [pypy-dev] unable to translate with current head of default Message-ID: Hi, running into the following traceback when translating. Any ideas? My machine too old to have certain features? Thanks! Best regards, Wim Lavrijsen [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 306, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/translator/driver.py", line 809, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/translator/driver.py", line 286, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/translator/driver.py", line 397, in task_pyjitpl_lltype [translation:ERROR] backend_name=self.config.translation.jit_backend, inline=True) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/jit/metainterp/warmspot.py", line 50, in apply_jit [translation:ERROR] warmrunnerdesc.finish() [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/jit/metainterp/warmspot.py", line 210, in finish [translation:ERROR] self.annhelper.finish() [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/rpython/annlowlevel.py", line 240, in finish [translation:ERROR] self.finish_annotate() [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/rpython/annlowlevel.py", line 259, in finish_annotate [translation:ERROR] ann.complete_helpers(self.policy) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/annotation/annrpython.py", line 176, in complete_helpers [translation:ERROR] self.complete() [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/annotation/annrpython.py", line 250, in complete [translation:ERROR] self.processblock(graph, block) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/annotation/annrpython.py", line 448, in processblock [translation:ERROR] self.flowin(graph, block) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/annotation/annrpython.py", line 508, in flowin [translation:ERROR] self.consider_op(block.operations[i]) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/annotation/annrpython.py", line 710, in consider_op [translation:ERROR] raise_nicer_exception(op, str(graph)) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/annotation/annrpython.py", line 707, in consider_op [translation:ERROR] resultcell = consider_meth(*argcells) [translation:ERROR] File "<4237-codegen /home/wlav/pypydev/pypy/pypy/annotation/annrpython.py:745>", line 3, in consider_op_getattr [translation:ERROR] return arg.getattr(*args) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/annotation/unaryop.py", line 606, in getattr [translation:ERROR] attrdef = ins.classdef.find_attribute(attr) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/annotation/classdef.py", line 217, in find_attribute [translation:ERROR] return self.locate_attribute(attr).attrs[attr] [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/annotation/classdef.py", line 212, in locate_attribute [translation:ERROR] self.generalize_attr(attr) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/annotation/classdef.py", line 307, in generalize_attr [translation:ERROR] self._generalize_attr(attr, s_value) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/annotation/classdef.py", line 297, in _generalize_attr [translation:ERROR] newattr.mutated(self) [translation:ERROR] File "/home/wlav/pypydev/pypy/pypy/annotation/classdef.py", line 137, in mutated [translation:ERROR] raise NoSuchAttrError(homedef, self.name) [translation:ERROR] NoSuchAttrError': (, 'is_xmm') [translation:ERROR] .. v0 = getattr(loc_0, ('is_xmm')) [translation:ERROR] .. '(pypy.jit.backend.x86.assembler:671)Assembler386._assemble_bootstrap_direct_call' [translation:ERROR] Processing block: [translation:ERROR] block at 150 is a [translation:ERROR] in (pypy.jit.backend.x86.assembler:671)Assembler386._assemble_bootstrap_direct_call [translation:ERROR] containing the following operations: [translation:ERROR] v0 = getattr(loc_0, ('is_xmm')) [translation:ERROR] v1 = is_true(v0) [translation:ERROR] --end-- -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From fijall at gmail.com Tue May 24 09:06:43 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 24 May 2011 09:06:43 +0200 Subject: [pypy-dev] PyPy Genova-Pegli Post-EuroPython Sprint June 27 - July 2 2011 In-Reply-To: <4DDAB799.4000207@gmail.com> References: <4DDAB6CF.9040509@gmail.com> <4DDAB799.4000207@gmail.com> Message-ID: On Mon, May 23, 2011 at 9:38 PM, Antonio Cuni wrote: > On 23/05/11 21:34, Antonio Cuni wrote: >> Please register by hg: >> >> >> http://bitbucket.org/pypy/extradoc/src/default/sprintinfo/genova-pegli-2011/people.txt >> > > sorry, the link above is broken: > https://bitbucket.org/pypy/extradoc/src/extradoc/sprintinfo/genova-pegli-2011/people.txt Numpy can be put into the list of topics > > ciao, > Anto > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > From fijall at gmail.com Tue May 24 08:53:53 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 24 May 2011 08:53:53 +0200 Subject: [pypy-dev] unable to translate with current head of default In-Reply-To: References: Message-ID: On Tue, May 24, 2011 at 7:09 AM, wrote: > Hi, > > running into the following traceback when translating. Any ideas? My machine > too old to have certain features? Thanks! Should be fixed now, sorry From anto.cuni at gmail.com Tue May 24 09:24:19 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 24 May 2011 09:24:19 +0200 Subject: [pypy-dev] PyPy Genova-Pegli Post-EuroPython Sprint June 27 - July 2 2011 In-Reply-To: References: <4DDAB6CF.9040509@gmail.com> <4DDAB799.4000207@gmail.com> Message-ID: <4DDB5D23.9050402@gmail.com> On 24/05/11 09:06, Maciej Fijalkowski wrote: > Numpy can be put into the list of topics right, although I've already sent the announcement and it's too late now to change it. It falls in the category "more precise interest in mind", though. ciao, Anto From wlavrijsen at lbl.gov Wed May 25 00:46:57 2011 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Tue, 24 May 2011 15:46:57 -0700 (PDT) Subject: [pypy-dev] unable to translate with current head of default In-Reply-To: References: Message-ID: Hi Maciej, > On Tue, May 24, 2011 at 7:09 AM, wrote: >> running into the following traceback when translating. Any ideas? My machine >> too old to have certain features? Thanks! > Should be fixed now, sorry cool, now it translates again! Btw., the reason to move to head of default was b/c my translated pypy-c was crashing on the spot. The binary download of 1.5 and older locally translated pypy's didn't do that. Turns out there is something with my history file: the crash happens when it's being loaded by readline: Python 2.7.1 (9113640a83ac+8d950d4e5c98+, May 24 2011, 20:01:17) [PyPy 1.5.0-alpha0 with GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. using my private settings ... Program received signal SIGSEGV, Segmentation fault. 0x08aae11c in pypy_g_OptHeap_emitting_operation () By playing a little, I find that the history file is problematic if it contains 1000 lines or more. Having 999 lines or less is fine either way. Doesn't matter for me now that I know, but it's rather odd. Best regards, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From alex.gaynor at gmail.com Wed May 25 01:01:38 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Tue, 24 May 2011 16:01:38 -0700 Subject: [pypy-dev] unable to translate with current head of default In-Reply-To: References: Message-ID: On Tue, May 24, 2011 at 3:46 PM, wrote: > Hi Maciej, > > On Tue, May 24, 2011 at 7:09 AM, wrote: >> >>> running into the following traceback when translating. Any ideas? My >>> machine >>> too old to have certain features? Thanks! >>> >> Should be fixed now, sorry >> > > cool, now it translates again! > > Btw., the reason to move to head of default was b/c my translated pypy-c > was > crashing on the spot. The binary download of 1.5 and older locally > translated > pypy's didn't do that. > > Turns out there is something with my history file: the crash happens when > it's > being loaded by readline: > > Python 2.7.1 (9113640a83ac+8d950d4e5c98+, May 24 2011, 20:01:17) > [PyPy 1.5.0-alpha0 with GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > using my private settings ... > > Program received signal SIGSEGV, Segmentation fault. > 0x08aae11c in pypy_g_OptHeap_emitting_operation () > > By playing a little, I find that the history file is problematic if it > contains > 1000 lines or more. Having 999 lines or less is fine either way. > > Doesn't matter for me now that I know, but it's rather odd. > > Best regards, > Wim > > -- > WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > Given that 1000 is the default cutoff for the JIT, and the gdb backtrace you have, it's pretty clear that there's a crash during compilation, and that readline loops over the lines in the history file, and hitting that 1000th line triggers the compilation. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From wlavrijsen at lbl.gov Wed May 25 01:16:10 2011 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Tue, 24 May 2011 16:16:10 -0700 (PDT) Subject: [pypy-dev] unable to translate with current head of default In-Reply-To: References: Message-ID: Hi Alex, > Given that 1000 is the default cutoff for the JIT, and the gdb backtrace you > have, it's pretty clear that there's a crash during compilation, and that > readline loops over the lines in the history file, and hitting that 1000th > line triggers the compilation. interesting, and that should then mean that the benchmark would also fail and so it does, in the same place ... Okay, then I can't live with it. :} Any idea how to start debugging this? Thanks, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From anto.cuni at gmail.com Wed May 25 09:25:38 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Wed, 25 May 2011 09:25:38 +0200 Subject: [pypy-dev] unable to translate with current head of default In-Reply-To: References: Message-ID: <4DDCAEF2.5040100@gmail.com> Hi Wim, On 25/05/11 01:16, wlavrijsen at lbl.gov wrote: > Okay, then I can't live with it. :} > Not sure to understand: do you get a crash even now that you merged default? If this is the case, do you get it even with a trunk version of pypy? You don't have to translate it by yourself, you can just download one of the nightly builds: http://buildbot.pypy.org/nightly/trunk/ > Any idea how to start debugging this? since you get the crash during the optimieopt phase, you can try to selectively enable/disable the various optimization passes, but using --jit enable_opts, e.g.: ./pypy-c --jit enable_opts=intbounds:rewrite:virtualize:string:heap:ffi:unroll the line above is equivalent of a plain pypy-c, because it enables all optimizations. Try to remove some of them and see if you still get the crash. ciao, Anto From wlavrijsen at lbl.gov Wed May 25 15:43:14 2011 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Wed, 25 May 2011 06:43:14 -0700 (PDT) Subject: [pypy-dev] unable to translate with current head of default In-Reply-To: <4DDCAEF2.5040100@gmail.com> References: <4DDCAEF2.5040100@gmail.com> Message-ID: Hi Anto, > Not sure to understand: do you get a crash even now that you merged default? yes, that's the case. > If this is the case, do you get it even with a trunk version of pypy? You > don't have to translate it by yourself, you can just download one of the > nightly builds: > http://buildbot.pypy.org/nightly/trunk/ With pypy-c-jit-44414-19bf70a7d70c-linux.tar.bz2, I get no crash. Could it be as silly as having a buggy gcc or other tool used underneath when building? My local gcc is 4.4.1, whereas the above pypy build is 4.4.3. > ./pypy-c --jit enable_opts=intbounds:rewrite:virtualize:string:heap:ffi:unroll > > the line above is equivalent of a plain pypy-c, because it enables all > optimizations. Try to remove some of them and see if you still get the crash. Okay, that sounds like something I can try, but it'll take a bit to run through some of the possible combinations. Thanks, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From arigo at tunes.org Wed May 25 21:39:35 2011 From: arigo at tunes.org (Armin Rigo) Date: Wed, 25 May 2011 21:39:35 +0200 Subject: [pypy-dev] multibytecodec: missing features Message-ID: Hi all, Here are the missing features in multibytecodec: * support for ``errors != "strict"''. * classes MultibyteIncrementalEncoder, MultibyteIncrementalDecoder, MultibyteStreamReader and MultibyteStreamWriter. One reason I didn't implement the classes yet is that I couldn't understand two points in how they are supposed to work. But it seems that there are really two bugs, as I've been pointed to: http://bugs.python.org/issue12100 and http://bugs.python.org/issue12171 . So the question is if we should be bug-compatible with Python 2.7 or if we should instead implement some fixed version. I suppose I'm rather for the fixed version, but I'd like to hear some feedback from people that actually use multibytecodecs. Also, I wouldn't mind if someone would pick up the work and just do it, either the classes or ``errors != "strict"'' :-) A bient?t, Armin. From alex.gaynor at gmail.com Wed May 25 21:46:29 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Wed, 25 May 2011 12:46:29 -0700 Subject: [pypy-dev] multibytecodec: missing features In-Reply-To: References: Message-ID: On Wed, May 25, 2011 at 12:39 PM, Armin Rigo wrote: > Hi all, > > Here are the missing features in multibytecodec: > > * support for ``errors != "strict"''. > > * classes MultibyteIncrementalEncoder, MultibyteIncrementalDecoder, > MultibyteStreamReader and MultibyteStreamWriter. > > One reason I didn't implement the classes yet is that I couldn't > understand two points in how they are supposed to work. But it seems > that there are really two bugs, as I've been pointed to: > http://bugs.python.org/issue12100 and > http://bugs.python.org/issue12171 . So the question is if we should > be bug-compatible with Python 2.7 or if we should instead implement > some fixed version. > > I suppose I'm rather for the fixed version, but I'd like to hear some > feedback from people that actually use multibytecodecs. Also, I > wouldn't mind if someone would pick up the work and just do it, either > the classes or ``errors != "strict"'' :-) > > > A bient?t, > > Armin. > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > Victor has fixed one of these bugs, so at least I don't think we should be "bug compatible" with it. Also, if people like my idea of trying to do a PyPy 1.5.1 right after CPython 2.7.2 (RC1 scheduled for the 28th) it would be easy to start pulling in the stdlib now to not deal with it. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gaynor at gmail.com Thu May 26 02:27:08 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Wed, 25 May 2011 17:27:08 -0700 Subject: [pypy-dev] SmallTuples Message-ID: I have a question about SmallTuples. Why do we have multiple classes, rather than doing the "hack" of mallocing a single block of memory for the entirety of the object, since it is immutable and we can know the needed space up front, CPython uses this "hack" for strs for example. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Thu May 26 05:07:31 2011 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 25 May 2011 22:07:31 -0500 Subject: [pypy-dev] SmallTuples In-Reply-To: References: Message-ID: 2011/5/25 Alex Gaynor : > I have a question about SmallTuples.? Why do we have multiple classes, > rather than doing the "hack" of mallocing a single block of memory for the > entirety of the object, since it is immutable and we can know the needed > space up front, CPython uses this "hack" for strs for example. What do you mean by that? Isn't having multiple classes the RPython equivalent of this hack? -- Regards, Benjamin From alex.gaynor at gmail.com Thu May 26 06:15:53 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Wed, 25 May 2011 21:15:53 -0700 Subject: [pypy-dev] SmallTuples In-Reply-To: References: Message-ID: On Wed, May 25, 2011 at 8:07 PM, Benjamin Peterson wrote: > 2011/5/25 Alex Gaynor : > > I have a question about SmallTuples. Why do we have multiple classes, > > rather than doing the "hack" of mallocing a single block of memory for > the > > entirety of the object, since it is immutable and we can know the needed > > space up front, CPython uses this "hack" for strs for example. > > What do you mean by that? Isn't having multiple classes the RPython > equivalent of this hack? > > > > -- > Regards, > Benjamin > Well, IMO the issue is you can end up with a ton of guard_class that are more precise than you need, plus you have to manually specialize for each length, whereas with the other approach you can just automatically apply it to all tuples. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From william.leslie.ttg at gmail.com Thu May 26 06:23:39 2011 From: william.leslie.ttg at gmail.com (William ML Leslie) Date: Thu, 26 May 2011 14:23:39 +1000 Subject: [pypy-dev] SmallTuples In-Reply-To: References: Message-ID: On 26 May 2011 14:15, Alex Gaynor wrote: > Well, IMO the issue is you can end up with a ton of guard_class that are > more precise than you need, plus you have to manually specialize for each > length, whereas with the other approach you can just automatically apply it > to all tuples. With the class approach, you can store the length on the class, saving some space for each object. -- William Leslie From william.leslie.ttg at gmail.com Thu May 26 07:01:46 2011 From: william.leslie.ttg at gmail.com (William ML Leslie) Date: Thu, 26 May 2011 15:01:46 +1000 Subject: [pypy-dev] SmallTuples In-Reply-To: References: Message-ID: On 26 May 2011 14:23, William ML Leslie wrote: > On 26 May 2011 14:15, Alex Gaynor wrote: >> Well, IMO the issue is you can end up with a ton of guard_class that are >> more precise than you need, plus you have to manually specialize for each >> length, whereas with the other approach you can just automatically apply it >> to all tuples. > > With the class approach, you can store the length on the class, saving > some space for each object. Also possibly relevant: The python tuple is intended to be a heterogeneous data structure, as they are in RPython. The length of a tuple is conceptually part of the type, which means that tuple length will be static in sane code anyway. I don't think this is over-specialisation, because for homogeneous usage of tuples we usually won't be talking about SmallTuples. I guess that it does have potential to blow up, and wonder if we already have a system for deoptimising the truly polymorphic case. -- William Leslie From amauryfa at gmail.com Thu May 26 08:00:50 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 26 May 2011 08:00:50 +0200 Subject: [pypy-dev] SmallTuples In-Reply-To: References: Message-ID: 2011/5/26 William ML Leslie : > The python tuple is intended to be a heterogeneous data structure, as > they are in RPython. The length of a tuple is conceptually part of the > type, which means that tuple length will be static in sane code > anyway. But does this apply to the *args tuple? -- Amaury Forgeot d'Arc From cfbolz at gmx.de Thu May 26 10:22:48 2011 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 26 May 2011 10:22:48 +0200 Subject: [pypy-dev] SmallTuples In-Reply-To: References: Message-ID: <4DDE0DD8.9060604@gmx.de> On 05/26/2011 06:15 AM, Alex Gaynor wrote: > Well, IMO the issue is you can end up with a ton of guard_class that are > more precise than you need, plus you have to manually specialize for > each length, whereas with the other approach you can just automatically > apply it to all tuples. There is a reason why they are disabled by default. It is an unanswered question how much this is a problem in practice. However, you cannot do what you described in your first mail in RPython currently: There are no variable-sized instances. Carl Friedrich From william.leslie.ttg at gmail.com Thu May 26 10:59:15 2011 From: william.leslie.ttg at gmail.com (William ML Leslie) Date: Thu, 26 May 2011 18:59:15 +1000 Subject: [pypy-dev] SmallTuples In-Reply-To: References: Message-ID: On 26 May 2011 16:00, Amaury Forgeot d'Arc wrote: > 2011/5/26 William ML Leslie : >> The python tuple is intended to be a heterogeneous data structure, as >> they are in RPython. The length of a tuple is conceptually part of the >> type, which means that tuple length will be static in sane code >> anyway. > > But does this apply to the *args tuple? Good point - you mean "sometimes, frequently not". In the context we are considering, namely promotion of tuple length for small tuples, I want to pretend the answer is yes. It means that we get call-site specialisation for heterogeneous usage, and ignore it for most homogeneous usage. Heterogeneous *args usage is probably common enough - for example, in wrapping decorators, it is common to accept and pass *args and **kwargs to the wrapped function, and if the length of the tuple were promoted, the pack and unpack could be simplified. I suspect that limiting the discussion to /small/ tuples will take care of the outliers, but benchmarks are a good idea. -- William Leslie From arigo at tunes.org Thu May 26 14:35:39 2011 From: arigo at tunes.org (Armin Rigo) Date: Thu, 26 May 2011 14:35:39 +0200 Subject: [pypy-dev] SmallTuples In-Reply-To: References: Message-ID: Hi, FWIW, they are disabled as Carl Friedrich pointed out; but a way to change them to be more on the safe side would be to have only small tuples of length 2 optimized that way (and maybe 3, although I'm really unsure if they are used often enough to be a big win). That way, even in heterogenous usage we end up with only 2 cases: length 2 and any other length. A bient?t, Armin. From arigo at tunes.org Thu May 26 15:46:19 2011 From: arigo at tunes.org (Armin Rigo) Date: Thu, 26 May 2011 15:46:19 +0200 Subject: [pypy-dev] Properties under RPython In-Reply-To: <201105201636.07136.gabomdq@gmail.com> References: <201105201219.57737.gabomdq@gmail.com> <201105201636.07136.gabomdq@gmail.com> Message-ID: Hi Gabriel, On Fri, May 20, 2011 at 9:36 PM, Gabriel Jacobo wrote: > I have a good chunk of python code that uses properties all around, and I > wanted to see if I can run it through the pypy machinery and eventually come > the other side with a javascript end product (via RPythonic). Do you have any > suggestions as to best solve this situation? (Preprocessing the code somehow > perhaps?) Any random large chunk of Python code is not RPython at all. You may hit the issue of properties right now, but I can assure you that there will be many other issues. "Being RPython" is a property best achieved by writing code from scratch, not by trying to convert existing code. It's a bit like trying to convert a Python program to Java; it's painful, and usually better to just write the Java program from scratch. A bient?t, Armin. From wlavrijsen at lbl.gov Thu May 26 22:36:35 2011 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Thu, 26 May 2011 13:36:35 -0700 (PDT) Subject: [pypy-dev] unable to translate with current head of default In-Reply-To: <4DDCAEF2.5040100@gmail.com> References: <4DDCAEF2.5040100@gmail.com> Message-ID: Anto, > ./pypy-c --jit enable_opts=intbounds:rewrite:virtualize:string:heap:ffi:unroll > > the line above is equivalent of a plain pypy-c, because it enables all > optimizations. Try to remove some of them and see if you still get the crash. there are several combinations that work with the pre-built pypy but not with my local version (except one: if only intbounds is selected, neither works). The largest set of combinations that does work is: intbounds:rewrite:virtualize:ffi Several others don't crash, but raise (and again, not in the pre-built): RPython traceback: File "implement_18.c", line 205492, in optimize_loop File "implement_19.c", line 86345, in _optimize_loop File "implement_19.c", line 202071, in UnrollOptimizer_propagate_all_forward File "implement_19.c", line 248573, in Optimizer_reconstruct_for_next_iteration File "implement_19.c", line 247560, in Optimization_reconstruct_for_next_iteration Fatal RPython error: NotImplementedError Does any of that ring a bell? Changing gcc didn't work. For now, I'll live with it with the lower set of options. It may after all still be something specific to my box which I intend to replace sometime soon. Thanks, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From fijall at gmail.com Thu May 26 22:42:32 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 26 May 2011 22:42:32 +0200 Subject: [pypy-dev] unable to translate with current head of default In-Reply-To: References: <4DDCAEF2.5040100@gmail.com> Message-ID: On Thu, May 26, 2011 at 10:36 PM, wrote: > Anto, > >> ./pypy-c --jit >> enable_opts=intbounds:rewrite:virtualize:string:heap:ffi:unroll >> >> the line above is equivalent of a plain pypy-c, because it enables all >> optimizations. ?Try to remove some of them and see if you still get the >> crash. > > there are several combinations that work with the pre-built pypy but not > with > my local version (except one: if only intbounds is selected, neither works). > > The largest set of combinations that does work is: > > ?intbounds:rewrite:virtualize:ffi > > Several others don't crash, but raise (and again, not in the pre-built): > > RPython traceback: > ?File "implement_18.c", line 205492, in optimize_loop > ?File "implement_19.c", line 86345, in _optimize_loop > ?File "implement_19.c", line 202071, in > UnrollOptimizer_propagate_all_forward > ?File "implement_19.c", line 248573, in > Optimizer_reconstruct_for_next_iteration > ?File "implement_19.c", line 247560, in > Optimization_reconstruct_for_next_iteration > Fatal RPython error: NotImplementedError > > Does any of that ring a bell? This is relatively known and yet not fixed :) some operations are unacceptable by backend and removed by optimizations, which in turn lead to crashes if those optimizations are disabled. > > Changing gcc didn't work. For now, I'll live with it with the lower set of > options. It may after all still be something specific to my box which I > intend > to replace sometime soon. > > Thanks, > ? ? Wim > -- > WLavrijsen at lbl.gov ? ?-- ? ?+1 (510) 486 6411 ? ?-- ? ?www.lavrijsen.net > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > From wlavrijsen at lbl.gov Thu May 26 23:06:10 2011 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Thu, 26 May 2011 14:06:10 -0700 (PDT) Subject: [pypy-dev] unable to translate with current head of default In-Reply-To: References: <4DDCAEF2.5040100@gmail.com> Message-ID: Hi Maciej, > This is relatively known and yet not fixed :) some operations are > unacceptable by backend and removed by optimizations, which in turn > lead to crashes if those optimizations are disabled. ah. :) Thanks. I'll live with it. Best regards, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From arigo at tunes.org Sat May 28 09:19:34 2011 From: arigo at tunes.org (Armin Rigo) Date: Sat, 28 May 2011 09:19:34 +0200 Subject: [pypy-dev] [freebsd] pypy-1.5.0 fails to compile sandbox In-Reply-To: References: Message-ID: Hi, On Sun, May 22, 2011 at 11:04 PM, David Naylor wrote: > If this has been fixed in trunk, please could you point me to the > revision that fixed it. Sorry for the delay. We don't usually test much on FreeBSD, but I've found the source of the bug: it's not adding the -L/usr/local option in the final Makefile. Should be fixed by revision e69c2c3e4988. A bient?t, Armin. From arigo at tunes.org Sat May 28 09:27:43 2011 From: arigo at tunes.org (Armin Rigo) Date: Sat, 28 May 2011 09:27:43 +0200 Subject: [pypy-dev] [freebsd] pypy-1.5.0 fails to compile sandbox In-Reply-To: References: Message-ID: Re-hi, On Sat, May 28, 2011 at 9:19 AM, Armin Rigo wrote: > Sorry for the delay. ?We don't usually test much on FreeBSD, but I've > found the source of the bug: it's not adding the -L/usr/local option > in the final Makefile. ?Should be fixed by revision e69c2c3e4988. Bah, I wrote this fix before writing a proper test, and of course it was bogus. See 06948d81bf2b. A bient?t, Armin. From arigo at tunes.org Sat May 28 09:49:07 2011 From: arigo at tunes.org (Armin Rigo) Date: Sat, 28 May 2011 09:49:07 +0200 Subject: [pypy-dev] unable to translate with current head of default In-Reply-To: References: <4DDCAEF2.5040100@gmail.com> Message-ID: Hi Wim, As a summary, can you tell us if there is something wrong with the trunk version of PyPy, as compiled by your GCC? In this case it's a bug in the trunk version, which we have to fix, because we don't want to support only half the GCC configurations out there. Or does the bug only ever occur in your own branch? In this case it's likely to be a bug that, although pre-existing, only shows up in the particular configuration of your branch. It would also be something we have to fix, but it's harder for us to get to the crash. It's hard to just give general guidelines for how to debug such a crash, particularly because all levels of PyPy are involved. There is no real other option than "gdb pypy-c", which requires knowledge of how all levels play together in order to give the C source we get. A bient?t, Armin. From nvetoshkin at naumen.ru Sat May 28 21:05:39 2011 From: nvetoshkin at naumen.ru (nvetoshkin at naumen.ru) Date: Sat, 28 May 2011 14:05:39 -0500 Subject: [pypy-dev] =?utf-8?q?Oracle_Express_to_test_RPythonic_cx=5FOracle?= Message-ID: Hi! On friday I tried to translate pypy with cx_Oracle module enabled but due to errors (cause iterpreter has changed since someone did test the module) - I failed. Armin suggested to start running tests on a regular basis. To do that we need to have sdk + client libs + database itself installed on a test host. I have login/password on oracle site and downloaded client and server: http://dl.dropbox.com/u/30662774/oracle/oracle-xe-client_10.2.0.1-1.0_i386.deb http://dl.dropbox.com/u/30662774/oracle/oracle-xe-universal_10.2.0.1-1.0_i386.deb Original links (and chksums) are available here: http://www.oracle.com/technetwork/database/express-edition/downloads/102xelinsoft-102048.html Vetoshkin Nikita. From naylor.b.david at gmail.com Sun May 29 16:53:36 2011 From: naylor.b.david at gmail.com (David Naylor) Date: Sun, 29 May 2011 16:53:36 +0200 Subject: [pypy-dev] [freebsd] pypy core dumps on unit tests (AssertionError) Message-ID: Hi, I've run the unit tests for pypy and pypy-stackless on FreeBSD and at some stage the tests causes pypy(-stackless) to core dump. Here are the results: # pypy1.5 -c "from test import testall" test_capi internal test_L_code internal test_broken_memoryview RPython traceback: File "implement_3.c", line 63565, in _PyObject_New File "implement_3.c", line 65132, in _PyObject_NewVar File "implement_3.c", line 31701, in from_ref Fatal RPython error: AssertionError Abort (core dumped) # pypy-stackless1.5 -c "from test import testall" test_capi internal test_L_code internal test_broken_memoryview RPython traceback: File "implement_1.c", line 241860, in _PyObject_NewVar File "implement_1.c", line 165468, in from_ref Fatal RPython error: AssertionError Abort (core dumped) Is there any other debug information needed? And how can I help fix this? Also, how would one run the unit tests under sandbox? Regards, David From wlavrijsen at lbl.gov Sun May 29 19:11:21 2011 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Sun, 29 May 2011 10:11:21 -0700 (PDT) Subject: [pypy-dev] unable to translate with current head of default In-Reply-To: References: <4DDCAEF2.5040100@gmail.com> Message-ID: Hi Armin, > As a summary, can you tell us if there is something wrong with the > trunk version of PyPy, as compiled by your GCC? I've tried different versions of GCC (4.4 and 4.5): in both cases do I get a pypy-c that crashes once the jit is invoked. It was just a guess, and it turns out not to be related to the version of GCC (or associated tools). > In this case it's a > bug in the trunk version, which we have to fix, because we don't want > to support only half the GCC configurations out there. Or does the > bug only ever occur in your own branch? I did a fresh hg clone from bitbucket to make sure it's nothing local. I find that the trunk on my machine does NOT crash, but the branch does from that same fresh clone does. There are only a few changes that are in trunk but not yet in the branch (I did a recent merge). Also, the runtime of the branch is somewhat different, as it is linked to one more library (libReflex.so). I'll update the branch once more; easy to test if that makes a difference. Also, I'll try once more on the branch with cppyy disabled. That should be identical to trunk. Thanks, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From wlavrijsen at lbl.gov Mon May 30 03:55:03 2011 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Sun, 29 May 2011 18:55:03 -0700 (PDT) Subject: [pypy-dev] unable to translate with current head of default In-Reply-To: References: <4DDCAEF2.5040100@gmail.com> Message-ID: Hi Armin, okay, something has been fixed over the past couple of days: the readline now works! :) I'll move up to that then. My benchmark, however, does not, so no cigar just yet ... Taking again the list of Anto for the jit options, this is the longest that I can run successfully: intbounds:rewrite:virtualize:string:ffi If I add "unroll" I get the NotImplementedError that Maciej already explained. If I add "heap", I get the segfault. Note that the timing of the benchmark is very good. :) I.e. heap and unroll don't actually do much for my benchmark. Of course, I can't actually run the benchmark with trunk, since I need module cppyy for that. Hence, the crash in the benchmark may be due to pbs in my own rpython (as opposed to whatever caused the issues with readline). Thanks, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From daniel.wagner.ml at googlemail.com Mon May 30 18:04:30 2011 From: daniel.wagner.ml at googlemail.com (Daniel Wagner) Date: Mon, 30 May 2011 11:04:30 -0500 Subject: [pypy-dev] PyROOT with PyPy Message-ID: Dear PyPy-Team, I just found out how awesome PyPy is what a great speed-up I can achieve by using it (factor of 5.0x to 50.0x). I was able to remove a lot of my Numpy dependencies in my scripts but unfortunately I need PyROOT for some. My question is now: Is it possible to use PyROOT with PyPy? If not, what needs to be done to get it working? I'd like to contribute as much as possible to get it done. Greetings, Daniel From wlavrijsen at lbl.gov Mon May 30 19:28:47 2011 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Mon, 30 May 2011 10:28:47 -0700 (PDT) Subject: [pypy-dev] PyROOT with PyPy In-Reply-To: References: Message-ID: Hi Daniel, > I was able to remove a lot of my Numpy dependencies in my scripts but > unfortunately I need PyROOT for some. My question is now: Is it possible > to use PyROOT with PyPy? no, too many hacks into CPython internals for things like lazy loading of classes to have this to work with cpyext. > If not, what needs to be done to get it working? Well, there's a branch, reflex-support: https://bitbucket.org/pypy/pypy/src/dc3b3a380dcd that I'm working on. Is not quite far enough along for it to be useful, and not on the level of PyROOT by far. However, I've been taking more time for it recently, and so development is slogging along. At this pace, it'll be useful within a couple of months (there's a sprint planned in July, which I expect to be the game changer). Next to the above, for it to replace PyROOT, a wrapper on CINT would be needed (currently, there's a thin layer on Reflex). If the amount of code going into PyROOT is limited (e.g. only creating/plotting of histograms), that could be given priority. The future is clang/llvm btw. ROOT itself should have that out by Summer next year, or so I hear. Note that there are a plethora of reasons for doing things this way, rather than making PyROOT working with cpyext. For one, since you care about it, speed is a lot better b/c cppyy is not a black box to the JIT. > I'd like to contribute as much as possible to get it done. Care to help work on cppyy? Best regards, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From arigo at tunes.org Mon May 30 20:04:21 2011 From: arigo at tunes.org (Armin Rigo) Date: Mon, 30 May 2011 20:04:21 +0200 Subject: [pypy-dev] Pull requests and other stuff from bitbucket Message-ID: Hi all, hi Anto, On Bitbucket, the user PyPy has still an e-mail address that goes only to Antonio. It's used at least by the pull requests and fork notifications. It seems that we'd like to see them more widely. Antonio, would it make sense to change that address to arrive to the pypy-commit list? A bient?t, Armin. From anto.cuni at gmail.com Mon May 30 20:38:46 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Mon, 30 May 2011 20:38:46 +0200 Subject: [pypy-dev] Pull requests and other stuff from bitbucket In-Reply-To: References: Message-ID: <4DE3E436.9050808@gmail.com> On 30/05/11 20:04, Armin Rigo wrote: > Hi all, hi Anto, > > On Bitbucket, the user PyPy has still an e-mail address that goes only > to Antonio. It's used at least by the pull requests and fork > notifications. It seems that we'd like to see them more widely. > Antonio, would it make sense to change that address to arrive to the > pypy-commit list? Yes, I confirm that I receive pull/fork notifications and yes, I'd like that other people would be able to receive them. However, changing the address to pypy-dev or pypy-commit does not work, because someone could request a password reset and the corresponding email will be publicly visible. I opened an issue at bitbucket specifically for this: https://bitbucket.org/site/master/issue/2638/allow-users-to-specify-an-email-address-as In the mean time, we could maybe change the address to pypy-z: still not perfect, but at least more people would see the notifications. What do you think? ciao, Anto From fijall at gmail.com Mon May 30 23:25:02 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 30 May 2011 23:25:02 +0200 Subject: [pypy-dev] Pull requests and other stuff from bitbucket In-Reply-To: <4DE3E436.9050808@gmail.com> References: <4DE3E436.9050808@gmail.com> Message-ID: On Mon, May 30, 2011 at 8:38 PM, Antonio Cuni wrote: > On 30/05/11 20:04, Armin Rigo wrote: >> Hi all, hi Anto, >> >> On Bitbucket, the user PyPy has still an e-mail address that goes only >> to Antonio. ?It's used at least by the pull requests and fork >> notifications. ?It seems that we'd like to see them more widely. >> Antonio, would it make sense to change that address to arrive to the >> pypy-commit list? > > Yes, I confirm that I receive pull/fork notifications and yes, I'd like that > other people would be able to receive them. > > However, changing the address to pypy-dev or pypy-commit does not work, > because someone could request a password reset and the corresponding email > will be publicly visible. > > I opened an issue at bitbucket specifically for this: > https://bitbucket.org/site/master/issue/2638/allow-users-to-specify-an-email-address-as > > In the mean time, we could maybe change the address to pypy-z: still not > perfect, but at least more people would see the notifications. ?What do you think? pypy-z sounds like a good solution for now. I also think armin should use @python.org mail adresses, because otherwise they mess up with people's filters ;-) From romain.py at gmail.com Mon May 30 23:31:51 2011 From: romain.py at gmail.com (Romain Guillebert) Date: Mon, 30 May 2011 23:31:51 +0200 Subject: [pypy-dev] [GSoC] Blog post regarding the Cython backend aiming PyPy Message-ID: <20110530213151.GA26763@ubuntu> Hi I've posted and article on my blog that explains what I've done during the community bonding period and the first week of the Google Summer of Code : http://rguillebert.blogspot.com/2011/05/cython-backend-aiming-pypy-week-1.html Cheers Romain From romain.py at gmail.com Mon May 30 23:18:20 2011 From: romain.py at gmail.com (Romain Guillebert) Date: Mon, 30 May 2011 23:18:20 +0200 Subject: [pypy-dev] [GSoC] Blog post regarding the Cython backend aiming PyPy Message-ID: <20110530211820.GA25753@ubuntu> Hi I've posted and article on my blog that explains what I've done during the community bonding period and the first week of the Google Summer of Code : http://rguillebert.blogspot.com/2011/05/cython-backend-aiming-pypy-week-1.html Cheers Romain From romain.py at gmail.com Mon May 30 23:40:11 2011 From: romain.py at gmail.com (Romain Guillebert) Date: Mon, 30 May 2011 23:40:11 +0200 Subject: [pypy-dev] [GSoC] Blog post regarding the Cython backend aiming PyPy Message-ID: <20110530214011.GB26763@ubuntu> Hi I've posted and article on my blog that explains what I've done during the community bonding period and the first week of the Google Summer of Code : http://rguillebert.blogspot.com/2011/05/cython-backend-aiming-pypy-week-1.html Cheers Romain From fijall at gmail.com Tue May 31 09:13:55 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 31 May 2011 09:13:55 +0200 Subject: [pypy-dev] List Slicing Performance Drop In-Reply-To: <4DDA16B7.8020401@gmail.com> References: <437154.75121.qm@web121808.mail.ne1.yahoo.com> <4DDA16B7.8020401@gmail.com> Message-ID: Anto: what version of python did you use? There can be 2.6 vs 2.7 issue On Mon, May 23, 2011 at 10:11 AM, Antonio Cuni wrote: > On 20/05/11 23:23, Mohyiddeen Othman wrote: >> Hi, >> >> I've been recently doing Project Euler puzzles using Python, and have recently >> started to use PyPy to increase performance. However I've come across >> something (no pun intended) puzzling. I made this script for Euler #50. >> >> http://pastebin.com/VZ12Nsci >> >> It calculates all the primes under 1M first, then looks through them for a >> solution. The first part (finding primes) is blazing fast under PyPy (as >> expected). But in the second part PyPy lags behind. CPython managed it in >> 183s, while PyPy did it in 498s. I'm assuming it has something to do with the >> list slicing? > > I cannot reproduce, for me pypy is faster on both pieces of code: > > viper ~ $ pypy-c /tmp/euler.py > ('All primes below', 1000000, 'found') > 0.43 > Max sequence is: > .... > .... > ('It contains', 543, 'elements') > ('The sum is', 997651) > 72.61 > > > viper ~ $ python /tmp/euler.py > ('All primes below', 1000000, 'found') > 4.58 > Max sequence is: > .... > .... > ('It contains', 543, 'elements') > ('The sum is', 997651) > 128.81 > > > Could you tell us more about your environment, like which OS, cpu, 32/64 bit > and the exact version of pypy/cpython you used to do the benchmark please? > > ciao, > Anto > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > http://mail.python.org/mailman/listinfo/pypy-dev > From anto.cuni at gmail.com Tue May 31 09:58:59 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 31 May 2011 09:58:59 +0200 Subject: [pypy-dev] Pull requests and other stuff from bitbucket In-Reply-To: References: <4DE3E436.9050808@gmail.com> Message-ID: <4DE49FC3.7050804@gmail.com> On 30/05/11 23:25, Maciej Fijalkowski wrote: > pypy-z sounds like a good solution for now. Ok, I did it. ciao, Anto From anto.cuni at gmail.com Tue May 31 10:07:08 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 31 May 2011 10:07:08 +0200 Subject: [pypy-dev] List Slicing Performance Drop In-Reply-To: References: <437154.75121.qm@web121808.mail.ne1.yahoo.com> <4DDA16B7.8020401@gmail.com> Message-ID: <4DE4A1AC.8070605@gmail.com> On 31/05/11 09:13, Maciej Fijalkowski wrote: > Anto: what version of python did you use? There can be 2.6 vs 2.7 issue please don't do top posting, it's very confusing on mailing lists :-( Anyway, the results below are with python2.7. python2.6 is slightly slower, but not much. I confirm that pypy is faster on my machine. ciao, Anto > On Mon, May 23, 2011 at 10:11 AM, Antonio Cuni wrote: >> On 20/05/11 23:23, Mohyiddeen Othman wrote: >>> Hi, >>> >>> I've been recently doing Project Euler puzzles using Python, and have recently >>> started to use PyPy to increase performance. However I've come across >>> something (no pun intended) puzzling. I made this script for Euler #50. >>> >>> http://pastebin.com/VZ12Nsci >>> >>> It calculates all the primes under 1M first, then looks through them for a >>> solution. The first part (finding primes) is blazing fast under PyPy (as >>> expected). But in the second part PyPy lags behind. CPython managed it in >>> 183s, while PyPy did it in 498s. I'm assuming it has something to do with the >>> list slicing? >> >> I cannot reproduce, for me pypy is faster on both pieces of code: >> >> viper ~ $ pypy-c /tmp/euler.py >> ('All primes below', 1000000, 'found') >> 0.43 >> Max sequence is: >> .... >> .... >> ('It contains', 543, 'elements') >> ('The sum is', 997651) >> 72.61 >> >> >> viper ~ $ python /tmp/euler.py >> ('All primes below', 1000000, 'found') >> 4.58 >> Max sequence is: >> .... >> .... >> ('It contains', 543, 'elements') >> ('The sum is', 997651) >> 128.81 >> >> >> Could you tell us more about your environment, like which OS, cpu, 32/64 bit >> and the exact version of pypy/cpython you used to do the benchmark please? >> >> ciao, >> Anto >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> http://mail.python.org/mailman/listinfo/pypy-dev >> From markflorisson88 at gmail.com Tue May 31 12:11:42 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Tue, 31 May 2011 12:11:42 +0200 Subject: [pypy-dev] [Cython] [GSoC] Blog post regarding the Cython backend aiming PyPy In-Reply-To: References: <20110530213151.GA26763@ubuntu> Message-ID: On 30 May 2011 23:31, Romain Guillebert wrote: > Hi > > I've posted and article on my blog that explains what I've done during > the community bonding period and the first week of the Google Summer of > Code : > http://rguillebert.blogspot.com/2011/05/cython-backend-aiming-pypy-week-1.html > > Cheers > Romain > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > http://mail.python.org/mailman/listinfo/cython-devel > Cool. Would it be useful to always generate wrapper functions for extern functions with numeric argument types? E.g. this is valid Cython code: cdef extern from "foo.h": ? ?ctypedef unsigned int size_t ? ?size_t func_with_typedef_arg(size_t a) So here the ctypedef for size_t is unsigned int, which is only valid in C for some platforms/architectures. So perhaps a wrapper function could solve that issue: int __pyx_wrapper_func_with_typedef_arg(int a) { ? ?/* some bounds and sign checking code here ? */ ? ?return func_with_typedef(a); } Because then you always know that calling it with argtypes = [c_int] and restype = c_int is valid. (BTW, it's also valid to declare something as a function which is actually a macro, so I suppose you always need wrappers for functions.) Do you already have an idea how to handle struct type arguments? Those are often also incomplete... but perhaps I'm geting too far ahead, I don't think we're quite there yet. I suppose you could also place this ABI burden on the user (for now), as does ctypes. As for the linking stuff, perhaps it's a good idea to look at http://wiki.cython.org/enhancements/distutils_preprocessing (currently down unfortunately, but in google cache). Then you can list libraries like this: '# distutils: libraries = spam eggs' at the top of the file. Cheers, Mark From arigo at tunes.org Tue May 31 16:46:30 2011 From: arigo at tunes.org (Armin Rigo) Date: Tue, 31 May 2011 16:46:30 +0200 Subject: [pypy-dev] [freebsd] pypy core dumps on unit tests (AssertionError) In-Reply-To: References: Message-ID: Hi David, On Sun, May 29, 2011 at 4:53 PM, David Naylor wrote: > Is there any other debug information needed? ?And how can I help fix this? test_capi is normally skipped. I get the same crash as you do if I run it on Linux. The way to run tests and skip them if needed is: cd lib-python; python ../pypy/test_all.py > Also, how would one run the unit tests under sandbox? There is no out-of-the-box solution, but I would expect most of them to fail in this context anyway. A bient?t, Armin. From romain.py at gmail.com Tue May 31 17:08:20 2011 From: romain.py at gmail.com (Romain Guillebert) Date: Tue, 31 May 2011 17:08:20 +0200 Subject: [pypy-dev] [Cython] [GSoC] Blog post regarding the Cython backend aiming PyPy In-Reply-To: References: <20110530213151.GA26763@ubuntu> Message-ID: <20110531150820.GA31833@ubuntu> On Tue, May 31, 2011 at 12:08:52PM +0200, mark florisson wrote: > Cool. Would it be useful to always generate wrapper functions for > extern functions with numeric argument types? E.g. this is valid > Cython code: > > cdef extern from "foo.h": > ctypedef unsigned int size_t > size_t func_with_typedef_arg(size_t a) > > So here the ctypedef for size_t is unsigned int, which is only valid > in C for some platforms/architectures. So perhaps a wrapper function > could solve that issue: > > int __pyx_wrapper_func_with_typedef_arg(int a) { > /* some bounds and sign checking code here ? */ > return func_with_typedef(a); > } > > Because then you always know that calling it with argtypes = [c_int] > and restype = c_int is valid. > (BTW, it's also valid to declare something as a function which is > actually a macro, so I suppose you always need wrappers for > functions.) > > Do you already have an idea how to handle struct type arguments? Those > are often also incomplete... but perhaps I'm geting too far ahead, I > don't think we're quite there yet. I suppose you could also place this > ABI burden on the user (for now), as does ctypes. > > As for the linking stuff, perhaps it's a good idea to look at > http://wiki.cython.org/enhancements/distutils_preprocessing (currently > down unfortunately, but in google cache). Then you can list libraries > like this: '# distutils: libraries = spam eggs' at the top of the > file. > > Cheers, > > Mark Hi Mark For the moment, I try to handle the standard case : call a function with C basic types, then add structures and pass by pointer. To deal with the ABI, fijal told me that there's a tool called ctypes_configure, I will investigate this to see what I can get from it. Cheers Romain From mohyiddeen at yahoo.com Tue May 31 17:35:48 2011 From: mohyiddeen at yahoo.com (Mohyiddeen Othman) Date: Tue, 31 May 2011 08:35:48 -0700 (PDT) Subject: [pypy-dev] List Slicing Performance Drop In-Reply-To: <4DE4A1AC.8070605@gmail.com> Message-ID: <352371.1909.qm@web121814.mail.ne1.yahoo.com> I thought I did reply? Must've been mistaken. My bad. Thanks for the reply. I'm running Windows XP (32-bit) on an Intel Core2Duo E6550 2.33 GHz. I'm using CPython 2.7 and PyPy 1.5 I'm begining to think I'm doing something wrong. --- On Tue, 5/31/11, Antonio Cuni wrote: From: Antonio Cuni Subject: Re: [pypy-dev] List Slicing Performance Drop To: "Maciej Fijalkowski" Cc: "Mohyiddeen Othman" , pypy-dev at python.org Date: Tuesday, May 31, 2011, 1:07 AM On 31/05/11 09:13, Maciej Fijalkowski wrote: > Anto: what version of python did you use? There can be 2.6 vs 2.7 issue please don't do top posting, it's very confusing on mailing lists :-( Anyway, the results below are with python2.7. python2.6 is slightly slower, but not much. I confirm that pypy is faster on my machine. ciao, Anto > On Mon, May 23, 2011 at 10:11 AM, Antonio Cuni wrote: >> On 20/05/11 23:23, Mohyiddeen Othman wrote: >>> Hi, >>> >>> I've been recently doing Project Euler puzzles using Python, and have recently >>> started to use PyPy to increase performance. However I've come across >>> something (no pun intended) puzzling. I made this script for Euler #50. >>> >>> http://pastebin.com/VZ12Nsci >>> >>> It calculates all the primes under 1M first, then looks through them for a >>> solution. The first part (finding primes) is blazing fast under PyPy (as >>> expected). But in the second part PyPy lags behind. CPython managed it in >>> 183s, while PyPy did it in 498s. I'm assuming it has something to do with the >>> list slicing? >> >> I cannot reproduce, for me pypy is faster on both pieces of code: >> >> viper ~ $ pypy-c /tmp/euler.py >> ('All primes below', 1000000, 'found') >> 0.43 >> Max sequence is: >> .... >> .... >> ('It contains', 543, 'elements') >> ('The sum is', 997651) >> 72.61 >> >> >> viper ~ $ python /tmp/euler.py >> ('All primes below', 1000000, 'found') >> 4.58 >> Max sequence is: >> .... >> .... >> ('It contains', 543, 'elements') >> ('The sum is', 997651) >> 128.81 >> >> >> Could you tell us more about your environment, like which OS, cpu, 32/64 bit >> and the exact version of pypy/cpython you used to do the benchmark please? >> >> ciao, >> Anto >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> http://mail.python.org/mailman/listinfo/pypy-dev >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From wlavrijsen at lbl.gov Tue May 31 20:46:35 2011 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Tue, 31 May 2011 11:46:35 -0700 (PDT) Subject: [pypy-dev] unable to translate with current head of default In-Reply-To: References: <4DDCAEF2.5040100@gmail.com> Message-ID: Hi, [replying to myself] > okay, something has been fixed over the past couple of days: the readline > now works! :) I'll move up to that then. too fast ... it sometimes works, and when it doesn't, it sometimes does under gdb or valgrind. Sometimes if I start a new shell, it works, then stops working again for another. (Although the downloaded pypy-c's have never crashed, mind.) This would make me think that there's an uninitialized variable somewhere. Is it possible to create such a thing with a buggy RPython construct? That said, the results with gdb or valgrind are always the same, when they do show the crash: Valgrind: ==16631== Process terminating with default action of signal 11 (SIGSEGV) ==16631== Access not within mapped region at address 0x4 ==16631== at 0x8AADB0C: pypy_g_OptHeap_emitting_operation (in /home/wlav/pypydev/pypy/pypy/translator/goal/pypy-c) gdb: 0x8aadb0c : cmp 0x4(%edi),%eax 0x8aadb0f : jl 0x8aadb37 0x8aadb11 : jmp 0x8aadbbf Whereas I'd expect more variation with an uninitialized variable. Valgrind does not report any uninitialized variables either (other than PyObject_Free, which I'll presume is there for the same reason as for CPython (?)): When I select only the jit options that will not give me a segfault and then run under valgrind, this is the only complaint I get (other than PyObject_Free): ==16681== Address 0x4b18010 is 3,352 bytes inside a block of size 4,100 free'd ==16681== at 0x40268A6: free (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) ==16681== by 0x811282C: pypy_g__ll_dict_setitem_lookup_done__DICTPtr_Address_Ad (in /home/wlav/pypydev/pypy/pypy/translator/goal/pypy-c) Thanks, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net