From santagada at gmail.com Thu Jun 5 04:49:57 2008 From: santagada at gmail.com (Leonardo Santagada) Date: Wed, 4 Jun 2008 23:49:57 -0300 Subject: [pypy-dev] Problem on OOType for jspypy Message-ID: <4A239E04-1A9A-4FF1-B9DC-B3877D0B7503@gmail.com> Directly after annotation the targetjsstandalone.py gives an error on ootype. I cant see what might be the problem, and yes the code is translating to c fine. Here I am talking about the new code on a branch, but hopefully when antocuni and others in europe read this the code is going to be merged to trunk (if it is not wait a little for it). the error is: [translation:ERROR] Exception: unexpected prebuilt constant: but the full trace is on: http://paste.pocoo.org/show/62193/ thanks and I will be tomorrow on the channel. -- Leonardo Santagada From anto.cuni at gmail.com Thu Jun 5 11:38:27 2008 From: anto.cuni at gmail.com (Antonio Cuni) Date: Thu, 05 Jun 2008 11:38:27 +0200 Subject: [pypy-dev] Problem on OOType for jspypy In-Reply-To: <4A239E04-1A9A-4FF1-B9DC-B3877D0B7503@gmail.com> References: <4A239E04-1A9A-4FF1-B9DC-B3877D0B7503@gmail.com> Message-ID: <4847B413.6070803@gmail.com> Leonardo Santagada wrote: > Directly after annotation the targetjsstandalone.py gives an error on > ootype. I cant see what might be the problem, and yes the code is > translating to c fine. Here I am talking about the new code on a > branch, but hopefully when antocuni and others in europe read this the > code is going to be merged to trunk (if it is not wait a little for it). > > the error is: > [translation:ERROR] Exception: unexpected prebuilt constant: > > > but the full trace is on: > http://paste.pocoo.org/show/62193/ > > thanks and I will be tomorrow on the channel. staticmethods are not really supported by rpython. I would say they work by chance with lltype, but you shouldn't use them. ciao, Anto From yunmao at gmail.com Sat Jun 7 22:30:51 2008 From: yunmao at gmail.com (Yun Mao) Date: Sat, 7 Jun 2008 16:30:51 -0400 Subject: [pypy-dev] dead loop compilation In-Reply-To: References: Message-ID: <7cffadfa0806071330h5c9b246fs5de6fa765888829d@mail.gmail.com> Hi, I was playing with pypy svn branch, and one of the first functions i tried was the following: def f(): while 1: pass it looks like pypy was trying to evaluate this and has gone dead too. Is this what I suppose to see? thanks, Yun From santagada at gmail.com Sat Jun 7 23:43:56 2008 From: santagada at gmail.com (Leonardo Santagada) Date: Sat, 7 Jun 2008 18:43:56 -0300 Subject: [pypy-dev] dead loop compilation In-Reply-To: <7cffadfa0806071330h5c9b246fs5de6fa765888829d@mail.gmail.com> References: <7cffadfa0806071330h5c9b246fs5de6fa765888829d@mail.gmail.com> Message-ID: <2A44F6E4-696C-41AA-8EBC-902D1DD4DB80@gmail.com> On 07/06/2008, at 17:30, Yun Mao wrote: > Hi, I was playing with pypy svn branch, and one of the first functions > i tried was the following: > > def f(): > while 1: pass > > it looks like pypy was trying to evaluate this and has gone dead too. > Is this what I suppose to see? thanks, I asked about it on the #pypy channel on irc.freenode.net and the answer is that this is a known issue but won't be fixed. The reason for that is that rpython is a tool to write interpreters and that is not a useful feature (to be able to create infinite loops). I ask you to come to the irc channel and chat about what you are trying to do with pypy and someone there will give you some advice. See you there, -- Leonardo Santagada From arigo at tunes.org Mon Jun 9 11:01:12 2008 From: arigo at tunes.org (Armin Rigo) Date: Mon, 9 Jun 2008 11:01:12 +0200 Subject: [pypy-dev] dead loop compilation In-Reply-To: <7cffadfa0806071330h5c9b246fs5de6fa765888829d@mail.gmail.com> References: <7cffadfa0806071330h5c9b246fs5de6fa765888829d@mail.gmail.com> Message-ID: <20080609090112.GA4711@code0.codespeak.net> Hi, On Sat, Jun 07, 2008 at 04:30:51PM -0400, Yun Mao wrote: > Hi, I was playing with pypy svn branch, and one of the first functions > i tried was the following: > > def f(): > while 1: pass > > it looks like pypy was trying to evaluate this and has gone dead too. > Is this what I suppose to see? thanks, We need a bit more context to know where you are trying to put this code. I tried to type it in a "pypy-c" executable -- is that what you have in mind? In "pypy-c" I can interrupt the infinite loop with Ctrl-C, which is what I expected. (To get this result you may need a "pypy-c" translated with the '--allworkingmodules' option, otherwise I'm not sure signal handling is enabled.) If you mean that you tried to put an infinite loop in an RPython function and translate that, then no chance. The translation toolchain is not really good with strange inputs: it only does something sane if the input is sane (which means, an RPython that actually works and was reasonably well tested first). A bientot, Armin From fijall at gmail.com Mon Jun 9 21:05:58 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 9 Jun 2008 21:05:58 +0200 Subject: [pypy-dev] django performance analysis Message-ID: <693bc9ab0806091205u1dc73dafxbbb1784138097604@mail.gmail.com> Analysis of pypy-c --oldstyle runtests.py --settings=settings -v 2 using new _lsprof module These are analysis of a single test suite (test_client). In general, ctypes costs are bigger (due to teardown/setup which takes most of time). http://codespeak.net/~fijal/django-calltree.png Hereby I'll try to rewrite parts of ctypes (especially calling and some basic objects) in RPython. PS. I can upload kcachegrind input file if someone is interested. From fijall at gmail.com Tue Jun 10 07:41:03 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 10 Jun 2008 07:41:03 +0200 Subject: [pypy-dev] spotting pypy segfault Message-ID: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com> I think this is the one about finalizers, because they raise errors, but that might also be something with threads or anything, I don't know. What to do: 1. Have pypy-c with threads (hybrid gc, faassen, threads, allworkingmodules for me, svn v 55678) 2. Check out sqlalchemy svn checkout http://svn.sqlalchemy.org/sqlalchemy/trunk sqlalchemy go inside this directory (sqlalchemy), and run: PYTHONPATH=test gdb --args pypy-c --oldstyle test/ext/orderinglist.py my gdb got confused, but hopefully you'll have better chance. eventually, if this doesn't work (like you have different sqlalchemy installed), do (inside sqlalchemy dir) python2.4 setup.py develop, eventually as root depending on your system conf. Cheers and have fun debugging, fijal From fijall at gmail.com Tue Jun 10 07:50:22 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 10 Jun 2008 07:50:22 +0200 Subject: [pypy-dev] spotting pypy segfault In-Reply-To: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com> References: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com> Message-ID: <693bc9ab0806092250p788958d6q46ba824fb8fd726c@mail.gmail.com> It's a bit worse. It's infinite rec on C level somewhat. How did that happen? A bug in our cycle analysis? Short part from gdb trace is like: #393 0x08cabd32 in pypy_g_W_Super_getattribute (l_self_19118=0xb72f37d8, l_name_136=0x97c3144) at implement_7.c:35032 #394 0x08bcf5c5 in pypy_g_BuiltinActivation_UwS_W_Super_ObjSpace_str__run_ ( l_self_670=0x983ae94, l_scope_w_55=0xb71de804) at implement_6.c:174279 #395 0x081e9510 in pypy_g_BuiltinCode_funcrun (l_self_174=0x980a040, l_func_8=0x97ef6c0, l_args_28=0xb71de7ec) at implement.c:233924 #396 0x0805ff37 in pypy_g_funccall_star2 (l_self_2634=0x97ef6c0, l_stararg0_6=0xb72f37d8, l_stararg1_1=0xb71de7c4) at implement.c:11482 #397 0x0807f7f1 in pypy_g_get_and_call_function_star1 (l_w_descr_13=0x97ef6c0, l_w_obj_846=0xb72f37d8, l_stararg0_10=0xb71de7c4) at implement.c:30312 #398 0x0807dafd in pypy_g__handle_getattribute (l_w_descr_2=0x97ef6c0, l_w_obj_845=0xb72f37d8, l_w_name_60=0xb71de7c4) at implement.c:29216 #399 0x08062baf in pypy_g_getattr (l_w_obj_841=0xb72f37d8, l_w_name_59=0xb71de7c4) at implement.c:13316 #400 0x08cabd32 in pypy_g_W_Super_getattribute (l_self_19118=0xb72f37d8, l_name_136=0x97c3144) at implement_7.c:35032 How did that happen? Cheers, fijal From fijall at gmail.com Tue Jun 10 07:54:42 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 10 Jun 2008 07:54:42 +0200 Subject: [pypy-dev] spotting pypy segfault In-Reply-To: <693bc9ab0806092250p788958d6q46ba824fb8fd726c@mail.gmail.com> References: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com> <693bc9ab0806092250p788958d6q46ba824fb8fd726c@mail.gmail.com> Message-ID: <693bc9ab0806092254h409160bcifbd59f592bb5d38f@mail.gmail.com> Argh! I think I know how that happened. Suppose we have a cycle, which allocates n bytes on each cycle. So on each iteration we check if stack is at least size of n. But let's suppose we have somewhere along the cycle path of length m > n which returns. It means that our stack check might not catch it since we have n bytes left. But we actually need m, boom. (Traceback resembles a bit this situation, on the very top is stuff that is slightly different than repeating pattern). Cheers, fijal From arigo at tunes.org Tue Jun 10 12:19:45 2008 From: arigo at tunes.org (Armin Rigo) Date: Tue, 10 Jun 2008 12:19:45 +0200 Subject: [pypy-dev] spotting pypy segfault In-Reply-To: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com> References: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com> Message-ID: <20080610101944.GA9646@code0.codespeak.net> Hi Maciej, On Tue, Jun 10, 2008 at 07:41:03AM +0200, Maciej Fijalkowski wrote: > PYTHONPATH=test gdb --args pypy-c --oldstyle test/ext/orderinglist.py ImportError: No module named pysqlite2._sqlite Where can we get the current version of sqlite-ctypes? Armin From fijall at gmail.com Tue Jun 10 17:34:24 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 10 Jun 2008 17:34:24 +0200 Subject: [pypy-dev] spotting pypy segfault In-Reply-To: <20080610101944.GA9646@code0.codespeak.net> References: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com> <20080610101944.GA9646@code0.codespeak.net> Message-ID: <693bc9ab0806100834q9e0e4abjf1b19d0bdcb3bda1@mail.gmail.com> http://codespeak.net/svn/user/fijal/hack/pysqlite2/ You need to mangle a bit with sys.path to import this and not something else. On Tue, Jun 10, 2008 at 12:19 PM, Armin Rigo wrote: > Hi Maciej, > > On Tue, Jun 10, 2008 at 07:41:03AM +0200, Maciej Fijalkowski wrote: >> PYTHONPATH=test gdb --args pypy-c --oldstyle test/ext/orderinglist.py > > ImportError: No module named pysqlite2._sqlite > > Where can we get the current version of sqlite-ctypes? > > > Armin > From lac at openend.se Tue Jun 10 19:01:02 2008 From: lac at openend.se (Laura Creighton) Date: Tue, 10 Jun 2008 19:01:02 +0200 Subject: [pypy-dev] A freidn of mine is organising a computer security conference in Austria Message-ID: <200806101701.m5AH12El008424@theraft.openend.se> He's looking for speakers and sponsors. He wanted to know whom we knew at IBM Zurich Security, and anybody else that he should probably invite to speak because they are interesting. So -- the people in Dresden? Anybody have a list of names and emails I could give him? Laura From lalo.martins at gmail.com Wed Jun 11 04:45:56 2008 From: lalo.martins at gmail.com (Lalo Martins) Date: Wed, 11 Jun 2008 02:45:56 +0000 (UTC) Subject: [pypy-dev] Beijing, China Message-ID: This is an open-ended invitation. If any pypy developers or contributors find themselves in Beijing or nearby in the near future, drop me an email so we can arrange a talk or a dinner with the local geeks. We'll also be happy to show you around, help find hotel, and the usual necessities. best, Lalo Martins -- So many of our dreams at first seem impossible, then they seem improbable, and then, when we summon the will, they soon become inevitable. ----- http://lalomartins.info/ GNU: never give up freedom http://www.gnu.org/ From fijall at gmail.com Wed Jun 11 05:25:21 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 11 Jun 2008 05:25:21 +0200 Subject: [pypy-dev] Beijing, China In-Reply-To: References: Message-ID: <693bc9ab0806102025r2a8dedd1se0967b579dfb9e44@mail.gmail.com> Heh :) A bit too late for me, I've been in Beijing last Autumn actually. Thanks very much anyway! From fijall at gmail.com Sun Jun 15 20:19:17 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sun, 15 Jun 2008 20:19:17 +0200 Subject: [pypy-dev] Talks at the EP Message-ID: <693bc9ab0806151119j7a71b1ffr53d4cddd28af3acc@mail.gmail.com> Jacob: can you tell me more or less exactly what your talk will be about? I would like to prepare mine at some point soon and it would be super-cool if they won't overlap. From arigo at tunes.org Mon Jun 16 11:23:55 2008 From: arigo at tunes.org (Armin Rigo) Date: Mon, 16 Jun 2008 11:23:55 +0200 Subject: [pypy-dev] spotting pypy segfault In-Reply-To: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com> References: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com> Message-ID: <20080616092354.GA25540@code0.codespeak.net> Hi Maciej, On Tue, Jun 10, 2008 at 07:41:03AM +0200, Maciej Fijalkowski wrote: > 1. Have pypy-c with threads (hybrid gc, faassen, threads, > allworkingmodules for me, svn v 55678) Just built exactly such a pypy-c, revision 55678. > PYTHONPATH=test gdb --args pypy-c --oldstyle test/ext/orderinglist.py Works for me (well, all 6 tests fail, but it doesn't segfault). I had to tweak sqlalchemy/__init__.py, though, and I strongly suspect that you had to tweak it too but forgot to mention it. I can only guess the problem is some system-specific thread stack size limit. I don't know how to see or change the default, but translator/c/src/thread_pthread.h seems to support a THREAD_STACK_SIZE define, so maybe you can try recompiling the C source with a value like 8388608 and see if it helps. I have http://svn.sqlalchemy.org/sqlalchemy/trunk revision 4840 and http://codespeak.net/svn/user/fijal/pysqlite2 revision 55678. Here are all my local changes: Index: lib/sqlalchemy/__init__.py =================================================================== --- lib/sqlalchemy/__init__.py (revision 4840) +++ lib/sqlalchemy/__init__.py (working copy) @@ -34,7 +34,7 @@ from sqlalchemy.engine import create_engine, engine_from_config __all__ = [ name for name, obj in locals().items() - if not (name.startswith('_') or inspect.ismodule(obj)) ] + if not (name.startswith('_') or name.startswith('$') or inspect.ismodule(obj)) ] __version__ = 'svn' From anto.cuni at gmail.com Mon Jun 16 21:37:07 2008 From: anto.cuni at gmail.com (Antonio Cuni) Date: Mon, 16 Jun 2008 21:37:07 +0200 Subject: [pypy-dev] Vilnius/Post EuroPython PyPy Sprint 10-12th of July Message-ID: <4856C0E3.5080005@gmail.com> ======================================================== Vilnius/Post EuroPython PyPy Sprint 10-12th of July ======================================================== The PyPy team is sprinting at EuroPython again and we invite you to participate in our 3 day long sprint at the conference hotel - Reval Hotel Lietuva. If you plan to attend the sprint we recommend you to listen to the PyPy technical talks (`EuroPython schedule`_) during the conference since it will give you a good overview of the status of development. On the morning of the first sprint day (10th) we will also have a tutorial session for those new to PyPy development. As 3 days is relatively short for a PyPy sprint we suggest to travel back home on the 13th if possible (but it is ok to attend less than 3 days too). ------------------------------ Goals and topics of the sprint ------------------------------ There are many possible and interesting sprint topics to work on - here we list some possible task areas: * try out Python programs and fix them or fix PyPy or fix performance bottlenecks * some JIT improvement work * port the stackless transform to ootypesystem * other interesting stuff that you would like to work on ...;-) ------------ Registration ------------ If you'd like to come, please subscribe to the `pypy-sprint mailing list`_ and drop a note about your interests and post any questions. More organisational information will be sent to that list. Please register by adding yourself on the following list (via svn): http://codespeak.net/svn/pypy/extradoc/sprintinfo/post-ep2008/people.txt or on the pypy-sprint mailing list if you do not yet have check-in rights: http://codespeak.net/mailman/listinfo/pypy-sprint --------------------------------------- Preparation (if you feel it is needed): --------------------------------------- * read the `getting-started`_ pages on http://codespeak.net/pypy * for inspiration, overview and technical status you are welcome to read `the technical reports available and other relevant documentation`_ * please direct any technical and/or development oriented questions to pypy-dev at codespeak.net and any sprint organizing/logistical questions to pypy-sprint at codespeak.net * if you need information about the conference, potential hotels, directions etc we recommend to look at http://www.europython.org. We are looking forward to meet you at the Vilnius Post EuroPython PyPy sprint! The PyPy team .. See also .. .. _getting-started: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html .. _`pypy-sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint .. _`the technical reports available and other relevant documentation`: http://codespeak.net/pypy/dist/pypy/doc/index.html .. _`EuroPython schedule`: http://europython.org/timetable From fijall at gmail.com Wed Jun 18 04:38:29 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 18 Jun 2008 04:38:29 +0200 Subject: [pypy-dev] relative occurency of opcodes Message-ID: <693bc9ab0806171938s4d0bbc76lf33bd69702e0ece3@mail.gmail.com> I just performed simple statistics on hacked pypy-c. It seems that some opcodes are very often repeated by something else. It might make sense to create some super opcodes which combine those two. Here are statistics for translating gcbench: http://codespeak.net/svn/user/fijal/opcodes.txt From arigo at tunes.org Wed Jun 18 17:30:15 2008 From: arigo at tunes.org (Armin Rigo) Date: Wed, 18 Jun 2008 17:30:15 +0200 Subject: [pypy-dev] relative occurency of opcodes In-Reply-To: <693bc9ab0806171938s4d0bbc76lf33bd69702e0ece3@mail.gmail.com> References: <693bc9ab0806171938s4d0bbc76lf33bd69702e0ece3@mail.gmail.com> Message-ID: <20080618153015.GA3530@code0.codespeak.net> Hi Maciej, On Wed, Jun 18, 2008 at 04:38:29AM +0200, Maciej Fijalkowski wrote: > I just performed simple statistics on hacked pypy-c. It seems that > some opcodes are very often repeated by something else. It might make > sense to create some super opcodes which combine those two. Someone did that for CPython already. This is where the PREDICT() macros in Python/ceval.c come from - they are CPython's solution to the same measures. Armin From fijall at gmail.com Fri Jun 20 03:20:52 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Fri, 20 Jun 2008 03:20:52 +0200 Subject: [pypy-dev] changes to pysqlite/ctypes Message-ID: <693bc9ab0806191820x37fe0c94lf7e40bac704fe62e@mail.gmail.com> We implemented few changes, also new features to sqlite-ctypes. Most notably: * Implement create_function without py_object, using closures * Various bugfixes * Accept subclasses of string/unicode * Added support for readahead logic * Thread safety All with tests, or reenabling of tests. However, when I try to use your supplied info how to commit there, it says "authorization failed". Would you rather prefer it in patches or commited to the repo? Cheers, fijal From lac at openend.se Fri Jun 20 05:17:10 2008 From: lac at openend.se (Laura Creighton) Date: Fri, 20 Jun 2008 05:17:10 +0200 Subject: [pypy-dev] From Brett in infrastructure Message-ID: <200806200317.m5K3HAHB004934@theraft.openend.se> Do we have a tool that does this? Laura ------- Forwarded Message Return-Path: infrastructure-bounces+lac=openend.se at python.org Delivery-Date: Fri Jun 20 04:41:53 2008 Return-Path: From: "Brett Cannon" To: infrastructure at python.org Subject: [Infrastructure] How do we move over to svn 1.5 once 2.6/3.0 final are out? Does anyone happen to know if there is any way to use ``svn merge`` with a branch that has been managed with svnmerge.py initially? I would love to get the built-in support once we are out of beta, but I am not sure how to make sure we don't lose all the info we currently have going in branches such as py3k. - -Brett From arigo at tunes.org Fri Jun 20 08:54:24 2008 From: arigo at tunes.org (Armin Rigo) Date: Fri, 20 Jun 2008 08:54:24 +0200 Subject: [pypy-dev] From Brett in infrastructure In-Reply-To: <200806200317.m5K3HAHB004934@theraft.openend.se> References: <200806200317.m5K3HAHB004934@theraft.openend.se> Message-ID: <20080620065424.GA27546@code0.codespeak.net> Hi Laura, On Fri, Jun 20, 2008 at 05:17:10AM +0200, Laura Creighton wrote: > Do we have a tool that does this? Not that I know of. I don't even really know the svnmerge.py he is talking about and we haven't used it in the pypy repository as far as I am aware. Armin From gh at ghaering.de Fri Jun 20 09:37:24 2008 From: gh at ghaering.de (=?UTF-8?B?R2VyaGFyZCBIw6RyaW5n?=) Date: Fri, 20 Jun 2008 09:37:24 +0200 Subject: [pypy-dev] changes to pysqlite/ctypes In-Reply-To: <693bc9ab0806191820x37fe0c94lf7e40bac704fe62e@mail.gmail.com> References: <693bc9ab0806191820x37fe0c94lf7e40bac704fe62e@mail.gmail.com> Message-ID: <485B5E34.8000201@ghaering.de> Maciej Fijalkowski wrote: > We implemented few changes, also new features to sqlite-ctypes. Most notably: > > * Implement create_function without py_object, using closures > * Various bugfixes > * Accept subclasses of string/unicode > * Added support for readahead logic > * Thread safety > > All with tests, or reenabling of tests. > > However, when I try to use your supplied info how to commit there, it > says "authorization failed". I'd like to try it myself. Unfortuantely I don't have access to the encrypted mail I sent to Armin Rigo (with the password) at work. I'll try it this evening. You didn't by any chance use a HTTP instead of a HTTPS URL? Write access is limited to encrypted connections. > Would you rather prefer it in patches or commited to the repo? After setting up another account, I'd like to make sure it works. -- Gerhard From lac at openend.se Fri Jun 20 10:28:51 2008 From: lac at openend.se (Laura Creighton) Date: Fri, 20 Jun 2008 10:28:51 +0200 Subject: [pypy-dev] From Brett in infrastructure In-Reply-To: Message from Armin Rigo of "Fri, 20 Jun 2008 08:54:24 +0200." <20080620065424.GA27546@code0.codespeak.net> References: <200806200317.m5K3HAHB004934@theraft.openend.se> <20080620065424.GA27546@code0.codespeak.net> Message-ID: <200806200828.m5K8SpjE019780@theraft.openend.se> In a message of Fri, 20 Jun 2008 08:54:24 +0200, Armin Rigo writes: >Hi Laura, > >On Fri, Jun 20, 2008 at 05:17:10AM +0200, Laura Creighton wrote: >> Do we have a tool that does this? > >Not that I know of. I don't even really know the svnmerge.py he is >talking about and we haven't used it in the pypy repository as far as I >am aware. > > >Armin Ah, well I was confused. But it turns out so was he, and the tool he wants already comes with svn. Laura From fijall at gmail.com Fri Jun 20 17:50:04 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Fri, 20 Jun 2008 17:50:04 +0200 Subject: [pypy-dev] Default options for pypy Message-ID: <693bc9ab0806200850q7bb230cfha4aa45b6cf78495b@mail.gmail.com> Can I make --gc=hybrid --thread --faassen --allworkingmodules --oldstyle default options for pypy? People get easily confused. From gh at ghaering.de Sat Jun 21 03:39:46 2008 From: gh at ghaering.de (=?UTF-8?B?R2VyaGFyZCBIw6RyaW5n?=) Date: Sat, 21 Jun 2008 03:39:46 +0200 Subject: [pypy-dev] changes to pysqlite/ctypes In-Reply-To: <693bc9ab0806191820x37fe0c94lf7e40bac704fe62e@mail.gmail.com> References: <693bc9ab0806191820x37fe0c94lf7e40bac704fe62e@mail.gmail.com> Message-ID: <485C5BE2.1070108@ghaering.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Maciej Fijalkowski wrote: > We implemented few changes, also new features to sqlite-ctypes. Most notably: > > * Implement create_function without py_object, using closures > * Various bugfixes > * Accept subclasses of string/unicode > * Added support for readahead logic > * Thread safety > > All with tests, or reenabling of tests. > > However, when I try to use your supplied info how to commit there, it > says "authorization failed". > > Would you rather prefer it in patches or commited to the repo? Pushing to the repo works now. I forgot to add you to "allow-push" in addition to setting up HTTP authorization. I just tested it to make sure it *really* works. - -- Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIXFvidIO4ozGCH14RAhjtAJ9ZeMkPggXN/AiklOj0jWUsPSXvYQCgjl8d whqBbkLUjnliWFkXFl3ptsA= =3yIe -----END PGP SIGNATURE----- From fijall at gmail.com Sat Jun 21 20:24:40 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sat, 21 Jun 2008 20:24:40 +0200 Subject: [pypy-dev] Benchmarking django admin app Message-ID: <693bc9ab0806211124y7783fce4i21dc54a9882cdd27@mail.gmail.com> I did some benchmarks for django admin app. It was fairly hard since django is good at eating exceptions (there are like 3 levels that can eat your SystemExit) which makes it hard to correctly exit application. Anyway, it seems that our ctypes is huge problem there, calling of ctypes functions eats 20% of time overall (which is fairly a lot for 1-value table). C-level benchmark does not show anything specific. Overall it's about 3x slower than cpython, which is fairly bad (it uses the same ctypes-based pysqlite). I think we should repeat this exercise after faster ctypes and see how this goes. I don't have any reasonable benchmark to try without DB, since default webpage takes 4ms to create (12 on pypy-c), but it can be some small issue that makes huge difference. (And we don't support builtin benchmarking for pypy) Cheers, fijal From arigo at tunes.org Sun Jun 22 12:12:18 2008 From: arigo at tunes.org (Armin Rigo) Date: Sun, 22 Jun 2008 12:12:18 +0200 Subject: [pypy-dev] Benchmarking django admin app In-Reply-To: <693bc9ab0806211124y7783fce4i21dc54a9882cdd27@mail.gmail.com> References: <693bc9ab0806211124y7783fce4i21dc54a9882cdd27@mail.gmail.com> Message-ID: <20080622101217.GA11935@code0.codespeak.net> Hi Maciej, On Sat, Jun 21, 2008 at 08:24:40PM +0200, Maciej Fijalkowski wrote: > application. Anyway, it seems that our ctypes is huge problem there, > calling of ctypes functions eats 20% of time overall (which is fairly > a lot for 1-value table). Spending 20% of the time in ctypes is a lot, but something else is consuming even more time than ctypes itself if I believe your numbers, because in the good cases we are around 1.4x slower than CPython. So with the admin app, say that CPython takes 1 sec; we would like to take 1.4 sec; we really take 3 sec; we loose 1.6 sec somewhere, including 0.6 sec in ctypes; so we loose 1 sec somewhere else. In other words, it is more urgent to figure out where we loose 1 sec (33% of our time) than trying to reduce ctypes' 20%. A bientot, Armin. From fijall at gmail.com Sun Jun 22 16:51:23 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sun, 22 Jun 2008 16:51:23 +0200 Subject: [pypy-dev] Benchmarking django admin app In-Reply-To: <20080622101217.GA11935@code0.codespeak.net> References: <693bc9ab0806211124y7783fce4i21dc54a9882cdd27@mail.gmail.com> <20080622101217.GA11935@code0.codespeak.net> Message-ID: <693bc9ab0806220751t929e26dm580f18cc2babdba3@mail.gmail.com> Yes, makes sense to dig somewhere deeper. We lack a bit profiling of builtin functions, which makes it a bit tedious to compare two runs. On the other hand ctypes is a single point which is evident to consume a lot. Also, given bigger tables (like in tests) we consume way more time in ctypes than just 20%, this is minimal possible ctypes usage. Cheers, fijal On Sun, Jun 22, 2008 at 12:12 PM, Armin Rigo wrote: > Hi Maciej, > > On Sat, Jun 21, 2008 at 08:24:40PM +0200, Maciej Fijalkowski wrote: >> application. Anyway, it seems that our ctypes is huge problem there, >> calling of ctypes functions eats 20% of time overall (which is fairly >> a lot for 1-value table). > > Spending 20% of the time in ctypes is a lot, but something else is > consuming even more time than ctypes itself if I believe your numbers, > because in the good cases we are around 1.4x slower than CPython. So > with the admin app, say that CPython takes 1 sec; we would like to take > 1.4 sec; we really take 3 sec; we loose 1.6 sec somewhere, including 0.6 > sec in ctypes; so we loose 1 sec somewhere else. > > In other words, it is more urgent to figure out where we loose 1 sec > (33% of our time) than trying to reduce ctypes' 20%. > > > > A bientot, > > Armin. > From fijall at gmail.com Mon Jun 23 18:14:28 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 23 Jun 2008 18:14:28 +0200 Subject: [pypy-dev] changes to pysqlite/ctypes In-Reply-To: <485C5BE2.1070108@ghaering.de> References: <693bc9ab0806191820x37fe0c94lf7e40bac704fe62e@mail.gmail.com> <485C5BE2.1070108@ghaering.de> Message-ID: <693bc9ab0806230914r314d411fr1b47e05f70722679@mail.gmail.com> > > Pushing to the repo works now. I forgot to add you to "allow-push" in > addition to setting up HTTP authorization. I just tested it to make sure > it *really* works. > > - -- Gerhard Thanks a lot. I just pushed our changeset, tell me what do you think. If you think that's fine, we'll probably remove codespeak copy (although we might keep it around just for convinience reasons, ie the fact that one cannot have hg:external in svn) Cheers, fijal From gherman at darwin.in-berlin.de Thu Jun 26 00:05:04 2008 From: gherman at darwin.in-berlin.de (Dinu Gherman) Date: Thu, 26 Jun 2008 00:05:04 +0200 Subject: [pypy-dev] Dotviewer capabilities Message-ID: <54AE7760-E9A4-4DAF-9D05-130ACD4E0660@darwin.in-berlin.de> Hi all, while preparing a talk for EuroPython.org and chatting with Christian Tismer I remembered that PyPy had a nice PyGame GUI for browsing GraphViz .dot files. So I wanted to play with it and made a new SVN checkout (very funky Mandelbrot sets during compilation, BTW... ;-) Then it occured to me that I don't need PyPy to run dotviewer (great, because it's still compiling). So I throwed a few of my .dot files at dotviewer and got slightly surprised to see a few ra- ther basic things not quite working as I'd expect (like rec- tangular and record-like node shapes. I've created the follow- ing sample files to illustrate my findings: http://python.net/~gherman/tmp/sample.dot http://python.net/~gherman/tmp/sample.pdf http://python.net/~gherman/tmp/sample.sample-dotviewer.jpg I'm running PyGame 1.8.0rc5 on Mac OS X 10.5.3, BTW. So, am I running dotviewer in some wrong fashion? Do I need to run it on a different PyGame version or another OS? Or is there some documentation for docviewer and its capabilities, that I haven't found yet? In fact, what I hoped to find in dotviewer is a browser for dot files in which I can fold individual parts of a graph in and out (while GraphViz is, perhaps, recalculating the layout). If dotviewer cannot do that, maybe somebody knows of another browser which can? Thanks and regards, Dinu From santagada at gmail.com Thu Jun 26 00:36:59 2008 From: santagada at gmail.com (Leonardo Santagada) Date: Wed, 25 Jun 2008 19:36:59 -0300 Subject: [pypy-dev] Dotviewer capabilities In-Reply-To: <54AE7760-E9A4-4DAF-9D05-130ACD4E0660@darwin.in-berlin.de> References: <54AE7760-E9A4-4DAF-9D05-130ACD4E0660@darwin.in-berlin.de> Message-ID: On 25/06/2008, at 19:05, Dinu Gherman wrote: > So I throwed a few of my .dot > files at dotviewer and got slightly surprised to see a few ra- > ther basic things not quite working as I'd expect (like rec- > tangular and record-like node shapes. I've created the follow- > ing sample files to illustrate my findings: > > http://python.net/~gherman/tmp/sample.dot > http://python.net/~gherman/tmp/sample.pdf > http://python.net/~gherman/tmp/sample.sample-dotviewer.jpg > > I'm running PyGame 1.8.0rc5 on Mac OS X 10.5.3, BTW. > > So, am I running dotviewer in some wrong fashion? Do I need > to run it on a different PyGame version or another OS? Or is > there some documentation for docviewer and its capabilities, > that I haven't found yet? > > In fact, what I hoped to find in dotviewer is a browser for > dot files in which I can fold individual parts of a graph in > and out (while GraphViz is, perhaps, recalculating the layout). > If dotviewer cannot do that, maybe somebody knows of another > browser which can? I think the dotviewer only has support for the features needed by pypy graphs, so it only supports the features used there and nothing more. I know pypy.rlib.parsing also uses it but what it draws is the same kind of nodes that pypy translator generates. Unfortunately I don't know of any other dot viewers out there (I know omnigraffle opens dot files maybe you should try) but extending the viewer is probably not too hard. -- Leonardo Santagada From cfbolz at gmx.de Thu Jun 26 00:45:13 2008 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 26 Jun 2008 00:45:13 +0200 Subject: [pypy-dev] Dotviewer capabilities In-Reply-To: <54AE7760-E9A4-4DAF-9D05-130ACD4E0660@darwin.in-berlin.de> References: <54AE7760-E9A4-4DAF-9D05-130ACD4E0660@darwin.in-berlin.de> Message-ID: <4862CA79.7050601@gmx.de> Hi Dinu, Dinu Gherman wrote: > while preparing a talk for EuroPython.org and chatting with > Christian Tismer I remembered that PyPy had a nice PyGame GUI > for browsing GraphViz .dot files. > > So I wanted to play with it and made a new SVN checkout (very > funky Mandelbrot sets during compilation, BTW... ;-) Then it > occured to me that I don't need PyPy to run dotviewer (great, > because it's still compiling). So I throwed a few of my .dot > files at dotviewer and got slightly surprised to see a few ra- > ther basic things not quite working as I'd expect (like rec- > tangular and record-like node shapes. I've created the follow- > ing sample files to illustrate my findings: > > http://python.net/~gherman/tmp/sample.dot > http://python.net/~gherman/tmp/sample.pdf > http://python.net/~gherman/tmp/sample.sample-dotviewer.jpg The PyPy dot viewer isn't working well for records, since PyPy hasn't used them so far. I guess adding support would maybe be easy. Apart from the fact that the borders of the record boxes are missing, what else do you think is wrong with the jpg above? It looks very similar to the pdf. > I'm running PyGame 1.8.0rc5 on Mac OS X 10.5.3, BTW. > > So, am I running dotviewer in some wrong fashion? Do I need > to run it on a different PyGame version or another OS? Or is > there some documentation for docviewer and its capabilities, > that I haven't found yet? > > In fact, what I hoped to find in dotviewer is a browser for > dot files in which I can fold individual parts of a graph in > and out (while GraphViz is, perhaps, recalculating the layout). > If dotviewer cannot do that, maybe somebody knows of another > browser which can? The dotviewer is a browser for large dot files. However, it does not support folding on arbitrary graphs, only zooming and navigation (by clicking on nodes, links etc). If you need folding, you need some customization of the dotviewer, which is possible as well. There are even examples of doing that somewhere in PyPy, but I don't remember where :-). Grepping might help. Cheers, Carl Friedrich From marius at pov.lt Thu Jun 26 02:09:55 2008 From: marius at pov.lt (Marius Gedminas) Date: Thu, 26 Jun 2008 03:09:55 +0300 Subject: [pypy-dev] Dotviewer capabilities In-Reply-To: References: <54AE7760-E9A4-4DAF-9D05-130ACD4E0660@darwin.in-berlin.de> Message-ID: <20080626000955.GA4407@fridge.pov.lt> On Wed, Jun 25, 2008 at 07:36:59PM -0300, Leonardo Santagada wrote: > Unfortunately I don't know of any other dot viewers out there (I know > omnigraffle opens dot files maybe you should try) but extending the > viewer is probably not too hard. There's http://code.google.com/p/jrfonseca/wiki/XDot, which barfed on my .dot files that tried to use HSV colours. It also links to http://zvtm.sourceforge.net/zgrviewer.html which I've never tried. My introduction to PyGame was hacking PyPy's dot viewer during the Vilnius sprint... Ahh, those were the times. Marius Gedminas -- Unix is the answer, but only if you phrase the question very carefully. -- Belinda Asbell -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From gherman at darwin.in-berlin.de Thu Jun 26 11:19:39 2008 From: gherman at darwin.in-berlin.de (Dinu Gherman) Date: Thu, 26 Jun 2008 11:19:39 +0200 Subject: [pypy-dev] Dotviewer capabilities In-Reply-To: <4862CA79.7050601@gmx.de> References: <54AE7760-E9A4-4DAF-9D05-130ACD4E0660@darwin.in-berlin.de> <4862CA79.7050601@gmx.de> Message-ID: Carl Friedrich Bolz: >> http://python.net/~gherman/tmp/sample.dot >> http://python.net/~gherman/tmp/sample.pdf >> http://python.net/~gherman/tmp/sample.sample-dotviewer.jpg Sorry, as you probably found out, the last link must be: http://python.net/~gherman/tmp/sample-dotviewer.jpg > The PyPy dot viewer isn't working well for records, since PyPy hasn't > used them so far. I guess adding support would maybe be easy. Apart > from > the fact that the borders of the record boxes are missing, what else > do > you think is wrong with the jpg above? It looks very similar to the > pdf. I understand and I didn't expect records to work, but I thought that rectangles were common enough to support them. ;-) A quick code review shows that GraphRenderer.draw_node_commands() in drawgraph.py ignores the "rect" node shape, hmm... Funny, but the GraphViz docs don't mention a "rect" shape although it seems to work with dot/neato etc. But ok, after changing my stuff to use "box" instead of "rect" dotviewer renders them happily as rects... Marius Gedminas: > There's http://code.google.com/p/jrfonseca/wiki/XDot, which barfed > on my > .dot files that tried to use HSV colours. It also links to > http://zvtm.sourceforge.net/zgrviewer.html which I've never tried. The first is too heavy to install for me on OS X now (PyGTK) and the seconds looks promising (although very javaish) and very capa- ble of zooming, but not of folding/unfolding. > > My introduction to PyGame was hacking PyPy's dot viewer during the > Vilnius sprint... Ahh, those were the times. > Be prepared for a few questions in Vilnius, then! ;-) I think this folding would be a great feature to have, also for PyPy. Does that look like a sprint topic? Thanks, Dinu -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Thu Jun 26 18:48:23 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 26 Jun 2008 18:48:23 +0200 Subject: [pypy-dev] Dotviewer capabilities In-Reply-To: References: <54AE7760-E9A4-4DAF-9D05-130ACD4E0660@darwin.in-berlin.de> <4862CA79.7050601@gmx.de> Message-ID: <693bc9ab0806260948u288fb6f3v4c91e49f267b0af0@mail.gmail.com> On Thu, Jun 26, 2008 at 11:19 AM, Dinu Gherman wrote: > Carl Friedrich Bolz: > > http://python.net/~gherman/tmp/sample.dot > > http://python.net/~gherman/tmp/sample.pdf > > http://python.net/~gherman/tmp/sample.sample-dotviewer.jpg > > Sorry, as you probably found out, the last link must be: > http://python.net/~gherman/tmp/sample-dotviewer.jpg > > The PyPy dot viewer isn't working well for records, since PyPy hasn't > used them so far. I guess adding support would maybe be easy. Apart from > the fact that the borders of the record boxes are missing, what else do > you think is wrong with the jpg above? It looks very similar to the pdf. > > I understand and I didn't expect records to work, but I thought > that rectangles were common enough to support them. ;-) A quick > code review shows that GraphRenderer.draw_node_commands() in > drawgraph.py ignores the "rect" node shape, hmm... Funny, but > the GraphViz docs don't mention a "rect" shape although it seems > to work with dot/neato etc. But ok, after changing my stuff to > use "box" instead of "rect" dotviewer renders them happily as > rects... > Marius Gedminas: > > There's http://code.google.com/p/jrfonseca/wiki/XDot, which barfed on my > .dot files that tried to use HSV colours. It also links to > http://zvtm.sourceforge.net/zgrviewer.html which I've never tried. > > The first is too heavy to install for me on OS X now (PyGTK) and > the seconds looks promising (although very javaish) and very capa- > ble of zooming, but not of folding/unfolding. > > > My introduction to PyGame was hacking PyPy's dot viewer during the > Vilnius sprint... Ahh, those were the times. > > > Be prepared for a few questions in Vilnius, then! ;-) I think this > folding would be a great feature to have, also for PyPy. Does that > look like a sprint topic? > Thanks, > Dinu > Absolutely. From arigo at tunes.org Sat Jun 28 20:20:10 2008 From: arigo at tunes.org (Armin Rigo) Date: Sat, 28 Jun 2008 20:20:10 +0200 Subject: [pypy-dev] pyrepl: cursor blinks Message-ID: <20080628182010.GA4774@code0.codespeak.net> Hi all, When I'm running an interactive pypy-c over an ssh connection I notice that the cursor blinks a bit when I type (it can remain invisible for a fraction of second). Normal readline-style editing doesn't give this result even over laggy connections. I traced it to pypy/lib/pyrepl/unix_console.py lines 95 and 97, which are the control code for hiding and re-showing the cursor, called all the time when refreshing the display. I really don't like this, but of course it is not a concern for people whose terminal's cursor is blinking all the time anyway. Anyway, if I remove these control codes, I get quite a fine experience again. However, I'm sure this is highly dependent on the terminal and I fear that some terminals might display the cursor moving quickly around while refreshing is taking place. If anyone knows of a better solution to avoid both issues, please do tell :-) Otherwise, if you feel like it, could you please try the attached patch in your favourite terminal? No need to re-translate a pypy-c. Tell me what your experience is, running an interactive pypy-c over an ssh connection (preferably laggy). A bientot, Armin. -------------- next part -------------- Index: unix_console.py =================================================================== --- unix_console.py (revision 56155) +++ unix_console.py (working copy) @@ -92,9 +92,9 @@ self.term = term self._bel = _my_getstr("bel") - self._civis = _my_getstr("civis", optional=1) + self._civis = None # _my_getstr("civis", optional=1) self._clear = _my_getstr("clear") - self._cnorm = _my_getstr("cnorm", optional=1) + self._cnorm = None # _my_getstr("cnorm", optional=1) self._cub = _my_getstr("cub", optional=1) self._cub1 = _my_getstr("cub1", 1) self._cud = _my_getstr("cud", 1) From pedronis at openend.se Sat Jun 28 21:38:29 2008 From: pedronis at openend.se (Samuele Pedroni) Date: Sat, 28 Jun 2008 21:38:29 +0200 Subject: [pypy-dev] I reinstated the nightly run of cpython tests against pypy-c Message-ID: <48669335.20503@openend.se> I switched to only test these combinations: 1) hybrid gc thread old-style allworkingmodules the result of a try run for this one can be seen at the usual place: http://www2.openend.se/~pedronis/pypy-c-test/allworkingmodules/summary.html http://www2.openend.se/~pedronis/pypy-c-test/allworkingmodules/56156/index.html although just the % doesn't tell much it seems we had some regressions in some fundamental tests, like test_builtin also our modified test_descr segfaults with this combination/these days it seems 2) stackless old-style allworkingmodules From arigo at tunes.org Sun Jun 29 00:03:54 2008 From: arigo at tunes.org (Armin Rigo) Date: Sun, 29 Jun 2008 00:03:54 +0200 Subject: [pypy-dev] I reinstated the nightly run of cpython tests against pypy-c In-Reply-To: <48669335.20503@openend.se> References: <48669335.20503@openend.se> Message-ID: <20080628220354.GA30425@code0.codespeak.net> Hi Samuele, On Sat, Jun 28, 2008 at 09:38:29PM +0200, Samuele Pedroni wrote: > the result of a try run for this one can be seen at the usual place: > > http://www2.openend.se/~pedronis/pypy-c-test/allworkingmodules/summary.html > http://www2.openend.se/~pedronis/pypy-c-test/allworkingmodules/56156/index.html Great, thanks! > although just the % doesn't tell much it seems we had some regressions > in some fundamental tests, like test_builtin I'll have a look at it when I have the time. > also our modified test_descr segfaults with this combination/these days > it seems This one is shallow. Fixed in r56168. A bientot, Armin From pedronis at openend.se Sun Jun 29 10:38:17 2008 From: pedronis at openend.se (Samuele Pedroni) Date: Sun, 29 Jun 2008 10:38:17 +0200 Subject: [pypy-dev] I reinstated the nightly run of cpython tests against pypy-c In-Reply-To: <20080628220354.GA30425@code0.codespeak.net> References: <48669335.20503@openend.se> <20080628220354.GA30425@code0.codespeak.net> Message-ID: <486749F9.5070706@openend.se> Armin Rigo wrote: > Hi Samuele, > > On Sat, Jun 28, 2008 at 09:38:29PM +0200, Samuele Pedroni wrote: > >> the result of a try run for this one can be seen at the usual place: >> >> http://www2.openend.se/~pedronis/pypy-c-test/allworkingmodules/summary.html >> http://www2.openend.se/~pedronis/pypy-c-test/allworkingmodules/56156/index.html >> > > Great, thanks! > I added a faassen build too: faassen hybrid gc thread old-style allworkingmodules http://www2.openend.se/~pedronis/pypy-c-test/faassen/summary.html seems to have a tiny bit more failures From anto.cuni at gmail.com Mon Jun 30 15:46:17 2008 From: anto.cuni at gmail.com (Antonio Cuni) Date: Mon, 30 Jun 2008 15:46:17 +0200 Subject: [pypy-dev] pyrepl: cursor blinks In-Reply-To: <20080628182010.GA4774@code0.codespeak.net> References: <20080628182010.GA4774@code0.codespeak.net> Message-ID: <4868E3A9.10609@gmail.com> Armin Rigo wrote: > to avoid both issues, please do tell :-) Otherwise, if you feel like > it, could you please try the attached patch in your favourite terminal? my terminal emulator is Terminal, from xfce4. I tried to simulate a laggy connection by uploading a large file while typing at pypy-over-ssh prompt. I didn't notice any difference between the four possible combinations (patch/no patch and laggy/not laggy). ciao, Anto