From fijall at gmail.com Sun Jan 3 10:31:42 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sun, 3 Jan 2010 10:31:42 +0100 Subject: [pypy-dev] RuPy talk is online Message-ID: <693bc9ab1001030131v1c81982cp25a03a3faf3db10@mail.gmail.com> My RuPy talk is available online: http://blip.tv/file/3014306 A sidenote - it was an extremely early talk (8:30 or so), so I'm more asleep than usual :-) Cheers, fijal From ademan555 at gmail.com Thu Jan 7 08:59:03 2010 From: ademan555 at gmail.com (Dan Roberts) Date: Wed, 06 Jan 2010 23:59:03 -0800 Subject: [pypy-dev] Applevel types Message-ID: <1262851143.28722.7.camel@StormEagle> I want to make multiple interpreter-level types appear as a single application-level type. Armin got me started in the right direction, pointing out pypy/objspace/std/{small,}int{type,object}.py to me. This made it clear that sharing a typedef will accomplish this, however in the case of integer objects, only __new__ is registered with the typedef. How are other methods registered for object instances? I assumed that only methods provided to TypeDef would be visible at the application-level on instances, however this must not be the case, are any methods on the interpreter-level classes automatically wrapped? Can anyone provide more complex examples or explain exactly how the int functionality works? Thanks, Dan From amauryfa at gmail.com Thu Jan 7 09:18:29 2010 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 7 Jan 2010 09:18:29 +0100 Subject: [pypy-dev] Applevel types In-Reply-To: <1262851143.28722.7.camel@StormEagle> References: <1262851143.28722.7.camel@StormEagle> Message-ID: Hi, 2010/1/7 Dan Roberts : > ? ?I want to make multiple interpreter-level types appear as a single > application-level type. ?Armin got me started in the right direction, > pointing out pypy/objspace/std/{small,}int{type,object}.py to me. ?This > made it clear that sharing a typedef will accomplish this, however in > the case of integer objects, only __new__ is registered with the > typedef. ?How are other methods registered for object instances? ?I > assumed that only methods provided to TypeDef would be visible at the > application-level on instances, however this must not be the case, are > any methods on the interpreter-level classes automatically wrapped? ?Can > anyone provide more complex examples or explain exactly how the int > functionality works? I think this is because int objects have no methods outside the special slots __float__ __add__ __repr__... which are implemented as MultiMethods: float__Int in intobject.py, for example. The magic that converts these functions to type methods is in pypy.objspace.std.register_all() -- Amaury Forgeot d'Arc From pedronis at openend.se Fri Jan 8 15:18:55 2010 From: pedronis at openend.se (Samuele Pedroni) Date: Fri, 08 Jan 2010 15:18:55 +0100 Subject: [pypy-dev] 1.2 release schedule Message-ID: <4B473ECF.7000807@openend.se> Hello, I will send the following mail to pypy-dev later in the day, please review Hi, here is the tentative schedule for the upcoming 1.2 release: - 19th of January: feature freeze (features under consideration are listed in extradoc/planning/jit.txt with a reference to the release) - 22th-29th of January: release work (stability, documentation and packaging) to produce a release candidate on the 29th - 5th of February: 1.2 release One of the goals of the release is to gather feedback: http://morepypy.blogspot.com/2009/12/planning-next-release-of-pypy.html it would be good if in the period up to finalizing the release people could try already and think of possible ways to stress out bugs from the JIT trying it on small/medium examples (CPython own tests which we use don't tend to loop enough to really use the JIT a lot). regards From pedronis at openend.se Fri Jan 8 15:41:35 2010 From: pedronis at openend.se (Samuele Pedroni) Date: Fri, 08 Jan 2010 15:41:35 +0100 Subject: [pypy-dev] 1.2 release schedule In-Reply-To: <4B473ECF.7000807@openend.se> References: <4B473ECF.7000807@openend.se> Message-ID: <4B47441F.5090403@openend.se> Samuele Pedroni wrote: > Hello, I will send the following mail to pypy-dev later in the day, > please review > of course ignore these lines, the reviewing I was seeking has happened and the result sent From exarkun at twistedmatrix.com Sat Jan 9 23:37:49 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Sat, 09 Jan 2010 22:37:49 -0000 Subject: [pypy-dev] Module hijinx Message-ID: <20100109223749.8677.2120488026.divmod.xquotient.451@localhost.localdomain> Hello, On a PyPy I translated this morning, I'm seeing weird behavior importing a module that messes with sys.modules (not a nice thing to do, but I would still expect it to work): exarkun at boson:/tmp$ ls -lR trickypackage/ trickypackage/: total 4 -rw-r--r-- 1 exarkun exarkun 61 2010-01-09 11:39 foo.py -rw-r--r-- 1 exarkun exarkun 0 2010-01-09 11:40 __init__.py exarkun at boson:/tmp$ cat trickypackage/foo.py import sys sys.modules['trickypackage.foo'] = "Hello, world" exarkun at boson:/tmp$ python Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>from trickypackage import foo >>>foo 'Hello, world' >>> exarkun at boson:/tmp$ ~/Projects/pypy/trunk/pypy/translator/goal/pypy-c-70469 Python 2.5.2 (70469, Jan 09 2010, 15:24:34) [PyPy 1.1.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``PyPy doesn't change the fundamental physics constants'' >>>from trickypackage import foo >>>foo >>> Anyone have any idea why this might happen? The real problem this is causing for me is with the twisted.internet.reactor module, of course. Jean-Paul From exarkun at twistedmatrix.com Sat Jan 9 23:38:28 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Sat, 09 Jan 2010 22:38:28 -0000 Subject: [pypy-dev] Parallel translation? In-Reply-To: <693bc9ab0912050849m3f9441cpf6e6e2b53a08857a@mail.gmail.com> References: <4B192FC9.5040203@lutzhaase.com> <1afaf6160912040836m617685ean15d3fdada5a1d7bc@mail.gmail.com> <4B194455.6000406@gmail.com> <20091205154445.GA18139@code0.codespeak.net> <693bc9ab0912050849m3f9441cpf6e6e2b53a08857a@mail.gmail.com> Message-ID: <20100109223828.8677.116612598.divmod.xquotient.452@localhost.localdomain> On 5 Dec 2009, 04:49 pm, fijall at gmail.com wrote: >On Sat, Dec 5, 2009 at 4:44 PM, Armin Rigo wrote: >>Hi, >> >>On Fri, Dec 04, 2009 at 06:18:13PM +0100, Antonio Cuni wrote: >>>I agree that at this point in time we cannot or don't want to make >>>annotation/rtyping/backend parallelizable, but it should definitely >>>be >>>possible to just pass the -j flag to 'make' in an automatic way. >> >>Of course, that is full of open problems too. ?The main one is that >>each >>gcc process consumes potentially a lot of RAM, so just passing "-j" is >>not a great idea, as all gccs are started in parallel. ?It looks like >>some obscure tweak is needed, like setting -j to a number that depends >>not only on the number of CPUs (as is classically done) but also on >>the >>total RAM of the system... >> >> >>A bientot, >> >>Armin. > >I guess the original idea was to have a translation option that is >passed as -j flag to make, so one can specify what number of jobs he >wants, instead of trying to guess it automatically. I poked around on this front a bit. I couldn't find any code in PyPy which invokes make. I did find pypy.translator.platform.distutils_platform.DistutilsPlatform._build, though. This seems to be where lists of C files are sent for compilation. Is that right? I thought about how to make this parallel. The cheesy solution, of course, would be to start a few threads and have them do the compilation (which should be sufficiently parallel, since it's another process that's doing the actual work). This is a bit complicated by the chdir calls in the code, though. Also, maybe distutils isn't threadsafe. I dunno if I'll think about this any further, but I thought I'd summarize what little I did figure out. Jean-Paul > >Cheers, >fijal >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev From benjamin at python.org Sun Jan 10 00:51:00 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 9 Jan 2010 17:51:00 -0600 Subject: [pypy-dev] Module hijinx In-Reply-To: <20100109223749.8677.2120488026.divmod.xquotient.451@localhost.localdomain> References: <20100109223749.8677.2120488026.divmod.xquotient.451@localhost.localdomain> Message-ID: <1afaf6161001091551o1b259282g5ea083f51dc41100@mail.gmail.com> 2010/1/9 : > Anyone have any idea why this might happen? ?The real problem this is > causing for me is with the twisted.internet.reactor module, of course. Amaury's refactorings may have broken this. I've checked in a fix now. -- Regards, Benjamin From exarkun at twistedmatrix.com Sun Jan 10 04:21:49 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Sun, 10 Jan 2010 03:21:49 -0000 Subject: [pypy-dev] Module hijinx In-Reply-To: <1afaf6161001091551o1b259282g5ea083f51dc41100@mail.gmail.com> References: <20100109223749.8677.2120488026.divmod.xquotient.451@localhost.localdomain> <1afaf6161001091551o1b259282g5ea083f51dc41100@mail.gmail.com> Message-ID: <20100110032149.8677.331083332.divmod.xquotient.453@localhost.localdomain> On 9 Jan, 11:51 pm, benjamin at python.org wrote: >2010/1/9 : >>Anyone have any idea why this might happen? ?The real problem this is >>causing for me is with the twisted.internet.reactor module, of course. > >Amaury's refactorings may have broken this. I've checked in a fix now. Thanks, your fix appears to have worked. Jean-Paul From amauryfa at gmail.com Sun Jan 10 15:49:15 2010 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Sun, 10 Jan 2010 15:49:15 +0100 Subject: [pypy-dev] Parallel translation? In-Reply-To: <20100109223828.8677.116612598.divmod.xquotient.452@localhost.localdomain> References: <4B192FC9.5040203@lutzhaase.com> <1afaf6160912040836m617685ean15d3fdada5a1d7bc@mail.gmail.com> <4B194455.6000406@gmail.com> <20091205154445.GA18139@code0.codespeak.net> <693bc9ab0912050849m3f9441cpf6e6e2b53a08857a@mail.gmail.com> <20100109223828.8677.116612598.divmod.xquotient.452@localhost.localdomain> Message-ID: Hello, 2010/1/9 : > On 5 Dec 2009, 04:49 pm, fijall at gmail.com wrote: >> I guess the original idea was to have a translation option that is >> passed as -j flag to make, so one can specify what number of jobs he >> wants, instead of trying to guess it automatically. > > I poked around on this front a bit. ?I couldn't find any code in PyPy which > invokes make. ?I did find > pypy.translator.platform.distutils_platform.DistutilsPlatform._build, > though. ?This seems to be where lists of C files are sent for compilation. > ?Is that right? PyPy does generate a makefile (gen_makefile() in http://codespeak.net/svn/pypy/trunk/pypy/translator/c/genc.py ) but it is not used in all configurations: In the same file, see the call to execute_makefile(). -Ojit uses the makefile, though. -- Amaury Forgeot d'Arc From exarkun at twistedmatrix.com Sun Jan 10 17:55:22 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Sun, 10 Jan 2010 16:55:22 -0000 Subject: [pypy-dev] os.openpty Message-ID: <20100110165522.8677.302848211.divmod.xquotient.462@localhost.localdomain> Hi all, I just checked in an implementation of os.openpty: https://codespeak.net/viewvc/?view=rev&revision=70477 It's not all that exciting (unless you were missing it before), but it's my first PyPy commit in a while so I thought I'd point it out in case there's anything horribly wrong with it. I did add a test and make sure translation on Linux 32 still works. Hopefully I didn't introduce any problems on any other platforms. Jean-Paul From cfbolz at gmx.de Tue Jan 12 15:17:57 2010 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 12 Jan 2010 15:17:57 +0100 Subject: [pypy-dev] os.openpty In-Reply-To: <20100110165522.8677.302848211.divmod.xquotient.462@localhost.localdomain> References: <20100110165522.8677.302848211.divmod.xquotient.462@localhost.localdomain> Message-ID: <4B4C8495.7070802@gmx.de> Hi Jean-Paul, On 01/10/2010 05:55 PM, exarkun at twistedmatrix.com wrote: > I just checked in an implementation of os.openpty: > > https://codespeak.net/viewvc/?view=rev&revision=70477 > > It's not all that exciting (unless you were missing it before), but it's > my first PyPy commit in a while so I thought I'd point it out in case > there's anything horribly wrong with it. I did add a test and make sure > translation on Linux 32 still works. Hopefully I didn't introduce any > problems on any other platforms. Not that I have a clue what the thing does, but the commit looks fine to me :-). The nightly tests also don't show anything suspicious. Thanks for doing this! Carl Friedrich From fijall at gmail.com Sat Jan 23 02:23:40 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sat, 23 Jan 2010 02:23:40 +0100 Subject: [pypy-dev] profiling thoughts Message-ID: <693bc9ab1001221723t4b8c45bfo96cb08a0c11bb278@mail.gmail.com> As discussed with Samuele & others, we have to decide what to do with profiling, since cProfile information is sort of pointless. Unladen swallow people seem (with success) to interface oprofile, which also has hooks for JVM, so it's already easier. Links: http://oprofile.sourceforge.net/doc/devel/index.html http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/JIT/OProfileJITEventListener.cpp?view=markup any thoughts? Cheers, fijal From fijall at gmail.com Sun Jan 24 20:12:47 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sun, 24 Jan 2010 20:12:47 +0100 Subject: [pypy-dev] New URL for buildbot Message-ID: <693bc9ab1001241112o3dbe41e1p6b892c1a45fdadae@mail.gmail.com> We have much nicer url for buildbot these days: http://buildbot.pypy.org Cheers, fijal From cfbolz at gmx.de Mon Jan 25 15:01:46 2010 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Mon, 25 Jan 2010 15:01:46 +0100 Subject: [pypy-dev] Call for Papers: DYLA @ TOOLS 2010 Message-ID: <4B5DA44A.4070302@gmx.de> ... apologies for cross-posting and multiple copies! 4th Workshop on Dynamic Languages and Applications (DYLA 2010) categories: programming languages, software engineering, computer science When: Jun 28, 2010 Where: Malaga, Spain Submission Deadline: Mar 31, 2010 Notification Due: May 14, 2010 Final Version Due: May 31, 2010 link: http://scg.unibe.ch/wiki/events/dyla2010 Call For Papers The DYLA Workshop series focuses on the revival of dynamic languages. These days, dynamic languages (like Ruby, Python, JavaScript, Lua, etc?) are getting ever more popular. This is a call to arms for academia! We need to explore the future of dynamic languages through its human aspects and technical issues. We also ought to look back and pick up solutions from existing dynamic languages (such as Scheme, Smalltalk, or Self) to be rediscovered and spread around. Goal and Topics The goal of this workshop is to act as a forum where we can discuss dynamic languages and their applications. Topics of interest include any topic relevant in applying and/or supporting dynamic languages: Ruby, Python, Groovy, JavaScript, Lua, Clojure, Lisp, Scheme, Smalltalk, Self, ABCL, Prolog, and many more? Human aspects of dynamic languages, such as - empirical studies about the application of dynamic languages - best practices and patterns specific to dynamic languages - program correctness through unit testing (as opposed to types) - improved or novel IDE support for dynamic languages - use of dynamic features by library & framework developers - scripting of static application with dynamic languages - reverse engineering and analysis of dynamic applications Technical aspects of dynamic languages, such as - what features make a language a dynamic one? - agents, actors, active object, distribution, concurrency and mobility - delegation, prototypes, mix-ins, traits - first-class closures, continuations, environments - reflection and meta-programming - automated reasoning about programs written in dynamic languages - (concurrent/distributed/mobile/aspect) virtual machines - optimization of dynamic languages - multi-paradigm & static/dynamic-marriages - type systems for dynamic languages - (dynamic) aspects for dynamic languages - higher-order objects & messages - ?other exotic dynamic features Submissions The workshop will have a demo-oriented style. The idea is to allow participants to demonstrate new and interesting features and discuss what they feel is relevant for the dynamic language community. Participants need to submit a 2?4 page position paper of their work in ACM, sig-alternate.cls format. At the workshop, participants will be asked to give 10-minute ?lightning demos? of their contributions. Submission page is https://www.easychair.org/login.cgi?conf=dyla10 Organizers - Alexandre Bergel, Univ of Chile, - Carl Friedrich Bolz, Heinrich-Heine-Univ, D?sseldorf, Germany - Simon Denier, INRIA Lille, France - Michael Haupt, HPI Potsdam, Germany - Adrian Kuhn, Univ of Bern, Switzerland Program Committee - Tom Dinkelaker, Technische Univ Darmstadt, Germany - Johan Fabry, Univ of Chile - Matthew Flatt, Univ of Utah, USA - Stephan Herrmann, TU Berlin, Germany - Abram Hindle, Univ of Waterloo, Canada - Kasper Lund, Google, Denmark. - Michael Perscheid, HPI Potsdam, Germany - Rodolfo Toledo, Univ of Chile - Niko Schwarz, Univ of Bern, Switzerland - Peter Sommerlad, HSR Rapperswil, Switzerland - Alessandro Warth, Viewpoints Research Institute, USA - Vadim Zaytsev, Univ of Koblenz, Germany For further information, please visit our website or follow us on twitter - http://bit.ly/dyla2010 - http://twitter.com/dyla2010 Carl Friedrich Bolz From fijall at gmail.com Sun Jan 31 15:31:20 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sun, 31 Jan 2010 15:31:20 +0100 Subject: [pypy-dev] [pypy-svn] r70985 - pypy/branch/multijit In-Reply-To: <20100130145843.3A7DC1680AB@codespeak.net> References: <20100130145843.3A7DC1680AB@codespeak.net> Message-ID: <693bc9ab1001310631k90af4dcj1c7a20a9c6ab61a3@mail.gmail.com> Hello. My proposition would be not to add new features to tracing until we fix bugs. Bugs being segfaults when you try running stuff on top of pypy-c-jit like apptests or translate.py (smalltalk target crashes every few tries). Cheers, fijal On Sat, Jan 30, 2010 at 3:58 PM, wrote: > Author: arigo > Date: Sat Jan 30 15:58:41 2010 > New Revision: 70985 > > Added: > ? pypy/branch/multijit/ > ? ? ?- copied from r70984, pypy/trunk/ > Log: > A branch in which to support having multiple JITs > in the same RPython program. > > _______________________________________________ > pypy-svn mailing list > pypy-svn at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-svn > From arigo at tunes.org Sun Jan 31 17:21:37 2010 From: arigo at tunes.org (Armin Rigo) Date: Sun, 31 Jan 2010 17:21:37 +0100 Subject: [pypy-dev] New URL for buildbot In-Reply-To: <693bc9ab1001241112o3dbe41e1p6b892c1a45fdadae@mail.gmail.com> References: <693bc9ab1001241112o3dbe41e1p6b892c1a45fdadae@mail.gmail.com> Message-ID: <20100131162137.GA31984@code0.codespeak.net> Hi, On Sun, Jan 24, 2010 at 08:12:47PM +0100, Maciej Fijalkowski wrote: > We have much nicer url for buildbot these days: > > http://buildbot.pypy.org For reference, the "older" plots are at: http://codespeak.net/pypy/jitplots.html I mention them again because they give more information: basically you can get the graph of every number printed at the end of a pypy-c-jit run, like how much time was spent in the backend or how often it aborted tracing for various reasons. A bientot, Armin. From arigo at tunes.org Sun Jan 31 17:25:06 2010 From: arigo at tunes.org (Armin Rigo) Date: Sun, 31 Jan 2010 17:25:06 +0100 Subject: [pypy-dev] [pypy-svn] r70985 - pypy/branch/multijit In-Reply-To: <693bc9ab1001310631k90af4dcj1c7a20a9c6ab61a3@mail.gmail.com> References: <20100130145843.3A7DC1680AB@codespeak.net> <693bc9ab1001310631k90af4dcj1c7a20a9c6ab61a3@mail.gmail.com> Message-ID: <20100131162506.GB31984@code0.codespeak.net> Hi Maciej, On Sun, Jan 31, 2010 at 03:31:20PM +0100, Maciej Fijalkowski wrote: > My proposition would be not to add new features to tracing until we > fix bugs. Bugs being segfaults when you try running stuff on top of > pypy-c-jit like apptests or translate.py (smalltalk target crashes > every few tries). Sorry, it's already done :-) We can now translate RPython programs containing several JITting interpreters. There are of course details to fix but basically it just works. A bientot, Armin.