From sylvain.thenault at logilab.fr Thu Mar 1 10:28:59 2007 From: sylvain.thenault at logilab.fr (Sylvain =?iso-8859-1?Q?Th=E9nault?=) Date: Thu, 1 Mar 2007 10:28:59 +0100 Subject: [pypy-dev] pylint 0.17 containing the RPython checker is available Message-ID: <20070301092859.GE12447@crater.logilab.fr> Hi there ! I'm pleased to announce that the new pylint release containing the rpython checker is available on logilab's web site. This first attempt contains the following checks: * unavailable keywords / builtins * multiple inheritance * mixing multiple types * non homogeneous list * global modification * negative slice index * using %r in format string * warn about special method that are not implicitly called By default the rpython checker is deactivated. Activate it using : pylint --rpython-mode -rn ... (-rn is disabling statistics reports) or pylint --enable-checker=rpython ... to get only rpython checks (though in this case you won't be warn about regular errors). Another interesting thing is the rpython dedicated testing framework, testing that checked things are actually not translatable. I've the idea that this may be useful to generate some kind of documentation for features supported by rpython or not, and help spread information when a feature that wasn't supported is introduced in rpython. That's another story though... If you're interested, check pylint/test/test_rpycompilation.py. Please report any false positives you find or things you can think about that would be interesting to check. Regards, -- Sylvain Th?nault LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services Python et calcul scientifique: http://www.logilab.fr/science From fijal at genesilico.pl Sun Mar 4 17:26:08 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 04 Mar 2007 17:26:08 +0100 Subject: [pypy-dev] pypy ./translate.py command completion Message-ID: <45EAF320.2040904@genesilico.pl> I've written small command line completion to ./translate.py from pypy/translator/goal. Note: works only from that directory (not sure how to make in working in general case) http://codespeak.net/svn/user/fijal/completetranslate.py From arigo at tunes.org Mon Mar 5 18:02:54 2007 From: arigo at tunes.org (Armin Rigo) Date: Mon, 5 Mar 2007 18:02:54 +0100 Subject: [pypy-dev] PyPy after EU funding Message-ID: <20070305170254.GA6342@code0.codespeak.net> Hi all, At the Trillke sprint, we discussed some plans about how to structure the post-EU Open Source project period. This resulted on Sunday in a general session of which you find notes here: extradoc/minute/post-eu-structure.txt We are thinking about setting up a group of individuals caring for conceptual project integrity, releases etc. taking an overall project perspective into account. The main goal would be to keep an overall project consistency: given that PyPy is by now a relatively large project, it seems important that we try to keep such an overall view. The issue that we are trying to avoid here is that of developments going in directions that are completely disruptive for other aspects of the project. The initial candidates for the group were discussed and agreed upon at the sprint. Note that the view on the post-EU period is just evolving: we are still quite busy with working towards 1.0 and wrapping up EU reports towards the final review end May in Bruxelles. Feedback, questions, comments welcome! If there are further discussions or needs, we might target e.g. a #pypy-sync meeting early April to refine and conclude on things. Otherwise we will consider the sprint discussion result as the current basis of operation. A bientot, Armin -- e-mail written in common with input from Holger, Samuele, Bea, Lene, etc. From lene at merlinux.de Tue Mar 6 11:37:09 2007 From: lene at merlinux.de (Lene Wagner) Date: Tue, 06 Mar 2007 11:37:09 +0100 Subject: [pypy-dev] Hildesheim report-writing-sprint report Message-ID: <45ED4455.1050904@merlinux.de> Hi pypy-dev! Welcome to a slightly unconventional sprint report (only fair for this slightly unconventional sprint): The first three days of the Trillke-sprint were fully dedicated to EU-reporting, considering that it may be slightly more fun to write them knowing that everybody else is doing the same, and that's what this report is about. The coding sprint report will follow, once the writers will have managed to travel home and recover a bit. Each of the 14 work packages that PyPy promised to do to the EU will conclude with at least one report due in the next month, and the target of the report sprint was to wrap up most of them and even deliver some to the EU already, also to have some spare time for coding works in March. In addition, there were the so-called 'review recommendations' - some first project period feedback by the Commission, requesting that the reports should meet scientific standards, include references and in general be useful for other than filling the EU's archives purposes. So, everybody was busy collecting papers, writing footnotes and consider overall sense-making strategies of the EU-reports, many of the results can be found here: http://codespeak.net/pypy/dist/pypy/doc/index-report.html A big effort went into the D12 report (prototypes, backends) taken care of by Samuele, Arre, Anto, Guido, Armin and overall coordinated by Holger, who also got Niko on the boat for remotely writing a section on the JVM backend. And then there were D02.1, D14.2, D01.2-4, D13.1, D03.1, D09.1 not to speak of the periodic/final EU management related reports and (sigh) the D06.1 report, being worked on by Holger, Anto, Arre, Stephan, Samuele, Lene, Carl Friedrich, Bea, Armin, Niko, Guido, and even the jet-lagged Michael and Richard helped with reviews and consultancy. Carl Friedrich, the very patient report release manager, took care for overall finalization and pdf-ing of reports apart from driving and starting the D06.1 report. Bea, Holger and Lene sat together dicussing ideas for overall activity reporting. On Wednesday, we finally managed to send out a pile of reports to the EU project officer, also presenting a commonly discussed table of planned report deliveries. And then the OLPC laptop came - introduced by Christian bringing it directly from Pycon to the PyPy group - which more or less stopped all report writing in exchange for people gathering around the machine and starting hacking. But luckily it was only Wednesday afternoon when it arrived and the belly- dancers were about to conquer the sprint room anyway :) (Actually, Armin continued working with the OLPC on the breakday and managed to port the Bub-n-Bros client to it, getting one step closer to the PyPy3000 goal of running Bub-n-Bros on every imaginable platform.) Cheers, Lene From micahel at gmail.com Wed Mar 7 15:12:02 2007 From: micahel at gmail.com (Michael Hudson) Date: Wed, 7 Mar 2007 14:12:02 +0000 Subject: [pypy-dev] Hildesheim Coding Sprint Report Message-ID: Hello pypy-dev! This is the report for the second, coding, half of PyPy's **25th** sprint, and the 18th and final sprint of the EU funded period. We are so completely tired that we don't have the energy to write a witty entry, so we'll skip that bit and start with describing the now usual tutorial for those participants who are less familiar with PyPy code base. This time Carl Friedrich was talking to Georg Brandl, an interested CPython developer, and Anders (only the third Anders to be involved in the project!) Sigfridsson, a PhD student at the Interaction Design Centre at the University of Limerick. Anders originally became involved in the project as an observer of the sprint process for his group's research when we sprinted at the University of Limerick, and that was partly why he was here this time, but it seems he found PyPy interesting enough to learn Python in the mean time and participate on the coding less at this sprint (or maybe he thought the view from the sprint trenches would be more revealing!). This sprint has seen many, many small tasks completed. On the first morning, Holger and Armin improved the readline module to the point of being useful -- supporting line editing and history, but not completion -- and hooked it into the interpreter sufficiently that the interactive interpreter and pdb both use it when available. At the same time Richard and Michael were hunting a bug Richard had discovered translating his own code, which was generally referred to as "the rdict bug" but turned out to be a bug in the garbage collector. Carl Friedrich and his band of helpers (mostly Anders, Georg, Alexander) worked on experimental reimplementations of Python lists, one using a theoretically optimal overallocation strategy and another using chunked storage to reduce the cost of list resizing. Sadly both resulted in a measurable slow down. This can be seen as yet more evidence that theory is different from practice... CF's other target for reimplementation at this sprint was strings. With help and moral support from Armin, he reimplemented strings according to the design from the "ropes paper" of Boehm, Atkinson & Plass: http://www.cs.ubc.ca/local/reading/proceedings/spe91-95/spe/vol25/issue12/spe986.pdf The predictable effect was that "typical Python code", written in the knowledge of how strings are implemented in Python today, takes a small (at most 10%) performance hit, but an arguably more natural and naive style of string handling becomes efficient. And some completely crazy code (like hash('a'*sys.maxint)) becomes very fast too... Continuing the theme of making things slower by object reimplementation, Armin supplied an implementation of general dictionaries as a hash table whose collision resolution is via chained lists instead of open addressing. Next! As opposed to the above, Armin and CF implemented caching of app-level character (i.e. strings of length 1) objects, which was a clear win, improving the pystone benchmark by around 10%. There have been many discussions recently about optimizing the lookup of global variables, and during one that took place here about various corner cases, Armin and Carl Friedrich and Samuele removed from PyPy some of the strange things CPython does to determine what the __builtins__ are for the execution of a given frame -- of course, depending on the value of PyPy's five millionth configuration option. Holger and Antonio came up with yet another optimization idea along these lines, which can be found in doc/discussion/chained_getattr.txt. Going back to the first day, Anto and Samuele worked on analyzing why pypy-cli was being reported as 50+ times slower than CPython on the benchmark page: http://tuatara.cs.uni-duesseldorf.de/benchmark.html To do this they wrote some small benchmarks in RPython and stared at some code, but the main problem seems to be that Mono on PowerPC just isn't that good: running pypy-cli using Microsoft's runtime shows performance just 3-4 times slower than pypy-c. After this, they worked on streamlining PyPy's much complained about external function interface (and also broke translation a few times in the process). The last sprint saw the introduction of a more general registry-based interface for external functions, and Samuele and Anto began by moving the math module over to using this interface. This was harder than what had gone before because these functions depend on header files, so some modifications to the C and LLVM backends were necessary. On the last day, Anto made some small improvements to pypy-cli's performance and Samuele made the taint object space translatable. On the first day Georg and Alexander tried to see how fast a PyPy could get if there was no Global Interpreter Lock (GIL). By disabling the GIL and making the exception state thread-local on the genc-level, they could easily get a binary that at least didn't crash if multiple threads were not modifying internal stuff concurrently. Running 4 Pystone instances (from 4 different modules) on this pypy-c let the process use 381% of cpu time, but the resulting figures were disappointing: running the 4 Pystone instances in parallel was less than 25% faster than running them in series, as opposed to being 300% faster in the best case. Both concluded that the garbage collector used (Boehm) is not very well suited for the heavy-duty memory allocation pattern of PyPy in case of multiple threads. After this, they implemented some of Python 2.5's features in the interpreter, in particular support for __index__ and some extensions to string and dict methods. Anders and Anders worked very productively on fixing some of the bugs in PyPy's issue tracker, implementing the -m command line option in pypy-c, much improved handling of EINTR results from syscalls (which makes most difference when pressing ^C on the command line), allowing buffer objects to be passed to socket.send and preventing modifications to builtin types. Holger and Stephan worked in the direction of moving the currently app-level and extremely slow string interpolation code into RPython by separating out the code that analyzes the format string from the code that access the values to be interpolated. Maciej and Guido worked a little on the javascript backend, both generally tidying and improving compatibility with Internet Explorer. Guido should not be allowed to forget saying "I am happy to work with Internet Explorer" during one of the daily status meetings :-) Stephan and Arre worked on fixing the last remaining bugs in the rewrite of the application-level stackless code that Stephan had been working on for some time. Later Stephan joined Armin and Christian in a discussion about the best API to provide for the new "composable coroutine" concept. They feel that the concept is powerful to encompass threads, greenlets, coroutines, threads, syslets and the best way to barbecue ribs. You can read about the basic idea in the "Composability" section of PyPy's stackless documentation: http://codespeak.net/pypy/dist/pypy/doc/stackless.html and further insight is unlikely to be provided by this diagram: http://python.net/crew/mwh/stackless.jpg The basic conclusion was that this is a very nice and natural model for a lot of things, at least once you've whacked your head into the right shape. A task that occupied various people at various times of the sprint was that of benchmarking, the goal being to determine how much effect the object and other optimizations have. Michael had over the previous month or so some written some scaffolding code to allow various benchmarks to be run and the results recorded. At the sprint he added a benchmark using docutils to process 'translation.txt' from pypy's own documentation and Guido added another using his own 'templess' templating system. Holger worked on getting some code written for bzr that makes nice graphs out of benchmark to parse the benchmark data produced by PyPy's benchmark runs. Maciej worked on the lib/distributed code that demonstrates the PyPy's transparent proxies. After a bit of effort, he was able to write a demo that implemented a remote version of pdb by simply creating a traceback object that proxied all operations to a remote process. Michael and Richard spent a day or so on the LLVM backend, which of late hasn't been so much "maintained" as "held together by increasingly large amounts of sticky tape". After some refactoring of the way the backend handled options, they removed a layer of hacks around the issue of FixedSizedArrays and implemented them properly, and also added support for the direct_* pointer operations produced by rctypes. Michael spent some time using Shark, an OS X profiling application, and found some OS specific flags and tweaks that improved the performance of pypy-c on OS X/PPC by around 20%. As readers of pypy-dev will know by now, there were discussions about how PyPy is going to continue after the end of the EU funding period. However, we don't have to summarize them here because we can just link to Armin's mail: http://codespeak.net/pipermail/pypy-dev/2007q1/003577.html Cordiali Saluti, mwh & Carl Friedrich From cfbolz at gmx.de Thu Mar 8 13:13:11 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 08 Mar 2007 13:13:11 +0100 Subject: [pypy-dev] sprint pictures Message-ID: <45EFFDD7.80609@gmx.de> Hi all! Some of the pictures I took during the Trillke sprint can be found here: http://codespeak.net/~cfbolz/hildesheim3-sprint-pictures/ Cheers, Carl Friedrich From lene at merlinux.de Fri Mar 9 10:16:49 2007 From: lene at merlinux.de (Lene Wagner) Date: Fri, 09 Mar 2007 09:16:49 -0000 Subject: [pypy-dev] sprint pictures In-Reply-To: <45EFFDD7.80609@gmx.de> References: <45EFFDD7.80609@gmx.de> Message-ID: <4619F823.7000903@merlinux.de> Carl Friedrich Bolz schrieb: > Hi all! > > Some of the pictures I took during the Trillke sprint can be found here: > > http://codespeak.net/~cfbolz/hildesheim3-sprint-pictures/ you may also want to look at http://codespeak.net/~lene/trillke-sprint-web/Page1.html greetings, Lene From holger at merlinux.de Fri Mar 9 20:37:32 2007 From: holger at merlinux.de (holger krekel) Date: Fri, 9 Mar 2007 20:37:32 +0100 Subject: [pypy-dev] codespeak/pypy environment overview Message-ID: <20070309193732.GU1954@solar.trillke> Hi folks, if you are interested, it'd be great if some people (hi e.g. Alexander! :) could look into a new technical report, describing PyPy's development infrastructure and codespeak.net, particularly if you think about helping with maintaining codespeak.net in the future. I put the current version (mostly written by Guido Wesdorp, CF and me): http://codespeak.net/~hpk/D02.1-deliverablereport.pdf Feel free to give feedback here or privately, there also is a ".txt" version next to it which you may use for sending typo-fixes or refinements. best & thanks, holger -- merlinux GmbH Steinbergstr. 42 31139 Hildesheim http://merlinux.de tel +49 5121 20800 75 (fax 77) From holger at merlinux.de Sat Mar 10 00:41:33 2007 From: holger at merlinux.de (holger krekel) Date: Sat, 10 Mar 2007 00:41:33 +0100 Subject: [pypy-dev] sprint pictures In-Reply-To: <4619F823.7000903@merlinux.de> References: <45EFFDD7.80609@gmx.de> <4619F823.7000903@merlinux.de> Message-ID: <20070309234133.GX1954@solar.trillke> On Mon, Apr 09, 2007 at 10:24 +0200, Lene Wagner wrote: > ... the first posting outside the eu funding timeframe! thanks for taking the pictures :) holger > Carl Friedrich Bolz schrieb: > > Hi all! > > > > Some of the pictures I took during the Trillke sprint can be found here: > > > > http://codespeak.net/~cfbolz/hildesheim3-sprint-pictures/ > > you may also want to look at > > http://codespeak.net/~lene/trillke-sprint-web/Page1.html > > greetings, > > Lene > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -- merlinux GmbH Steinbergstr. 42 31139 Hildesheim http://merlinux.de tel +49 5121 20800 75 (fax 77) From scott+pypy-dev at scottdial.com Tue Mar 13 03:29:35 2007 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Mon, 12 Mar 2007 22:29:35 -0400 Subject: [pypy-dev] GC_REDIRECT_TO_LOCALS breaks Win32 Message-ID: <45F60C8F.1060308@scottdial.com> I spoke to several people on IRC, but I didn't want to let this fall by the wayside. On revision 39896, mwh changed the pre_pre_gc_code generated for all platforms and made the remark in his log entry, "someone should do a windows build, i guess..." So, that someone would be me. My test runner has been broken all week.. I'll save the commentary on the wisdom of such a checkin. If someone really needs an explanation as to why GC_REDIRECT_TO_LOCALS breaks everything on Win32 as it is a pthread-only thing. In the still-in-development version of gc 7.0, it is a workable thing but for now, it just needs to be excluded[1]. Thanks, -Scott [1] http://scottdial.com/pypy-dev/pre_pre_gc_code.diff -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From aurelien.campeas at logilab.fr Tue Mar 13 12:51:19 2007 From: aurelien.campeas at logilab.fr (=?iso-8859-1?Q?Aur=E9lien_Camp=E9as?=) Date: Tue, 13 Mar 2007 12:51:19 +0100 Subject: [pypy-dev] [pypy-svn] r40417 - in pypy/dist/pypy/module/cclp: . constraint In-Reply-To: <20070313094816.F3E91100A7@code0.codespeak.net> References: <20070313094816.F3E91100A7@code0.codespeak.net> Message-ID: <20070313115119.GA31051@crater.logilab.fr> On Tue, Mar 13, 2007 at 10:48:16AM +0100, ale at codespeak.net wrote: > Author: ale > Date: Tue Mar 13 10:48:15 2007 > New Revision: 40417 > > Modified: > pypy/dist/pypy/module/cclp/constraint/constraint.py > pypy/dist/pypy/module/cclp/constraint/domain.py > pypy/dist/pypy/module/cclp/constraint/variable.py > pypy/dist/pypy/module/cclp/variable.py > Log: > Fixes for not demoting methods - still not translatable with --faasen Thanks for the fixes. But please note that I have a moderately big patch just awaiting to be committed that will wipe out a lot of the existing constraint stuff there (since we now rely on cslib to do the constraint propagation job). I'll find some time asap to commit that. Regards, Aur?lien. From lac at openend.se Tue Mar 13 23:32:07 2007 From: lac at openend.se (Laura Creighton) Date: Tue, 13 Mar 2007 23:32:07 +0100 Subject: [pypy-dev] exerpt from mail from John Gilmore. How close are we to doing this? Message-ID: <200703132232.l2DMW73I004433@theraft.openend.se> We're discussing OLPC. > The PyPy project just got its first OLPC. (We're doing optimisation > work for them.) What kind of optimization? One thing I've been looking at for them recently is about dirty pages. A dirty page is a piece of virtual memory that has been written to since the last time it was loaded from mass storage (flash in their case). Since they have no disk drive to swap to, every dirty page will stay resident in memory forever, until its process dies. Whereas any page that isn't dirty can be thrown away and later re-read from mass storage. The dynamic linker was making piles of dirty pages, which "prelink" can avoid. How many pages does Python dirty while it runs? And if the kernel provided a signal that said, "We're running low on memory -- please do what you can to release some!", is there anything that the Python interpreter -- or even higher level Python code -- can usefully do? As far as I know, all the interpreter's objects are heap-allocated and there's no way to ever return ANY of that ram to the kernel. This is the kiss of death for a memory-limited machine. I just didn't want this idea to get lost in the report writing. Laura From lac at openend.se Wed Mar 14 08:29:22 2007 From: lac at openend.se (Laura Creighton) Date: Wed, 14 Mar 2007 08:29:22 +0100 Subject: [pypy-dev] proofreading Message-ID: <200703140729.l2E7TMKR002654@theraft.openend.se> I'm available to do 'check-the-English'ing. What I need is a list of files in the order that you would like me to read them, preferably when they are done with the massive-update-for-content phase. I also need to know how to check out a pypy src tree that has the papers - what I have now is missing that part. Then -- do you just want me to fix as I go, or do you want me to mark suggested changes? When I am not sure of the sense of the thing, should that happen, I will just mark up the confusing bits. (But we won't have any of those, now, will we. :-) ) I want to be able to work on these un-connected to the internet. I will check back in periodically, however. How's that? Laura From cfbolz at gmx.de Wed Mar 14 10:03:28 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 14 Mar 2007 10:03:28 +0100 Subject: [pypy-dev] proofreading In-Reply-To: <200703140729.l2E7TMKR002654@theraft.openend.se> References: <200703140729.l2E7TMKR002654@theraft.openend.se> Message-ID: <45F7BA60.307@gmx.de> Hi Laura, hi all! Laura Creighton wrote: > I'm available to do 'check-the-English'ing. Cool, thanks for offering this! > What I need is a > list of files in the order that you would like me to read them, > preferably when they are done with the massive-update-for-content > phase. I also need to know how to check out a pypy src tree that > has the papers - what I have now is missing that part. The source files are in subdirectories of the svn repo http://codespeak.net/svn/pypy/eu-tracking/deliverable/ (which is not world-readable, though). The reports that have mostly stable content currently are D2.1, D3.1, D12.1. > Then -- do you just want me to fix as I go, or do you want me to > mark suggested changes? When I am not sure of the sense of the > thing, should that happen, I will just mark up the confusing bits. > (But we won't have any of those, now, will we. :-) ) Fix as you go is fine, add XXXs where something is not understandable. > I want to be able to work on these un-connected to the internet. > I will check back in periodically, however. How's that? That's fine. If anybody else (who does not have access to the eu repository) wants to do fixes to the report, please speak up. I will think of a way to make the sources available then. The pdf versions of all the reports can be found here: http://codespeak.net/pypy/dist/pypy/doc/index-report.html Cheers, Carl Friedrich From arigo at tunes.org Wed Mar 14 10:23:14 2007 From: arigo at tunes.org (Armin Rigo) Date: Wed, 14 Mar 2007 10:23:14 +0100 Subject: [pypy-dev] exerpt from mail from John Gilmore. How close are we to doing this? In-Reply-To: <200703132232.l2DMW73I004433@theraft.openend.se> References: <200703132232.l2DMW73I004433@theraft.openend.se> Message-ID: <20070314092314.GA7007@code0.codespeak.net> Hi, On Tue, Mar 13, 2007 at 11:32:07PM +0100, Laura Creighton wrote: > How many pages does Python dirty while it runs? This could be measured. It is an area where PyPy has more flexibility than CPython (what area isn't? :-) I am not sure so this needs to be checked, but I believe that if we built a pypy-c that included preloaded versions of many stdlib modules, we would get a rather bloated executable, but one which is very close to a memory image of the running processes. It means that it would start extremely fast (all modules are already imported!) and all the pages that come from disk would stay clean (all prebuilt objects are the same in RAM and on disk). Multiple processes would also share all this memory instead of having their own version of all the function and code objects from all the modules. Moreover we have no refcounters that dirty all objects all the time. Boehm might do that, but our custom GC could easily be convinced to never write into the tag bits of prebuilt objects. A bientot, Armin. From arigo at tunes.org Thu Mar 15 18:52:44 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 15 Mar 2007 18:52:44 +0100 Subject: [pypy-dev] Google Summer of Code Message-ID: <20070315175244.GA15208@code0.codespeak.net> Hi all, A few PyPy developers have signed up as mentors for the Google Summer of Code. Students interested in a summer project about PyPy (or any Python-related project) should have a look at: http://wiki.python.org/moin/SummerOfCode A bientot, Armin From paul.degrandis at gmail.com Thu Mar 15 20:03:07 2007 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Thu, 15 Mar 2007 15:03:07 -0400 Subject: [pypy-dev] Google Summer of Code In-Reply-To: <20070315175244.GA15208@code0.codespeak.net> References: <20070315175244.GA15208@code0.codespeak.net> Message-ID: <9c0bb8a00703151203g34ae533cxb6ac1cedfbc2e638@mail.gmail.com> Armin, I DEFINITELY want to do PyPy summer of code. Last year I did NMAP, but had to resign for another project I was a apart of, but this summer I have full dedication to a SoC project. Is there anything that would be awesome to have done in PyPy by the end of summer? Would you want me to work on a series of applications of PyPy features (like I started working on awhile ago)? Would you like to see more focus on the JVM backend? Paul On 3/15/07, Armin Rigo wrote: > > Hi all, > > A few PyPy developers have signed up as mentors for the Google Summer of > Code. Students interested in a summer project about PyPy (or any > Python-related project) should have a look at: > > http://wiki.python.org/moin/SummerOfCode > > > A bientot, > > Armin > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.de Thu Mar 15 20:26:29 2007 From: holger at merlinux.de (holger krekel) Date: Thu, 15 Mar 2007 20:26:29 +0100 Subject: [pypy-dev] Google Summer of Code In-Reply-To: <9c0bb8a00703151203g34ae533cxb6ac1cedfbc2e638@mail.gmail.com> References: <20070315175244.GA15208@code0.codespeak.net> <9c0bb8a00703151203g34ae533cxb6ac1cedfbc2e638@mail.gmail.com> Message-ID: <20070315192629.GI4635@solar.trillke> Hi Paul! On Thu, Mar 15, 2007 at 15:03 -0400, Paul deGrandis wrote: > Armin, > > I DEFINITELY want to do PyPy summer of code. Last year I did NMAP, but had > to resign for another project I was a apart of, but this summer I have full > dedication to a SoC project. Is there anything that would be awesome to > have done in PyPy by the end of summer? Would you want me to work on a > series of applications of PyPy features (like I started working on awhile > ago)? Would you like to see more focus on the JVM backend? IMO bringing the JVM backend to produce PyPy interpreters would be a great goal! Niko Matsakis brought it close already with his Summer of PyPy work. In case you haven't seen it already: you may want to take a look into the "D12" report we just published a few days ago - it talks about higher level backends in particular: http://codespeak.net/pypy/extradoc/eu-report/D12.1_H-L-Backends_and_Feature_Prototypes-interim-2007-03-12.pdf An additional goal (if it's otherwise to easy :) would be to bridge into java libraries and objects. Btw, if you have particular feedback on the above report, let us now - we are going to finalize it next week. best & cheers, holger > On 3/15/07, Armin Rigo wrote: > > > >Hi all, > > > >A few PyPy developers have signed up as mentors for the Google Summer of > >Code. Students interested in a summer project about PyPy (or any > >Python-related project) should have a look at: > > > > http://wiki.python.org/moin/SummerOfCode > > > > > >A bientot, > > > >Armin > >_______________________________________________ > >pypy-dev at codespeak.net > >http://codespeak.net/mailman/listinfo/pypy-dev > > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev -- merlinux GmbH Steinbergstr. 42 31139 Hildesheim http://merlinux.de tel +49 5121 20800 75 (fax 77) From anto.cuni at gmail.com Thu Mar 15 22:11:17 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Thu, 15 Mar 2007 22:11:17 +0100 Subject: [pypy-dev] Google Summer of Code In-Reply-To: <20070315175244.GA15208@code0.codespeak.net> References: <20070315175244.GA15208@code0.codespeak.net> Message-ID: <45F9B675.5010606@gmail.com> Hi Armin, hi all! Armin Rigo wrote: > Hi all, > > A few PyPy developers have signed up as mentors for the Google Summer of > Code. Students interested in a summer project about PyPy (or any > Python-related project) should have a look at: > > http://wiki.python.org/moin/SummerOfCode So, outside there is the sun (well, not right now :-)), the temperature is getting warmer and it's already time to think again about summer of code :-). First of all, I don't know who and how many signed up as a mentors: if the PyPy project needs more, I'm available to sign up as a mentor as well. Next, I'm also considering the possibility of applying as a student again: the PhD grant I will receive from 1st of April is very low, so I will likely look for an additional job, and it would be great if the job were pypy-related instead of one of those boring projects I used to do before :-). But before applying I want to be sure not to be unfair, because my PhD is also about PyPy and I don't want to be paid twice for the same work: so my idea would be to do something unrelated to gencli&Co., just to be sure it's something that does not fit in my PhD program. Moreover, I will be well-disposed to withdraw my application if there are many others interesting pypy proposals, because I would not want to steal someone else the opportunity of doing such as great experience and to be involved in the project. What do you think about it? As I said I basically worry to be judged unfair, so I would like to know the opinions of other developers before begin thinking of a proposal (and in that case I guess that #pypy is much more convenient for discussing it :-)). ciao Anto From lac at openend.se Fri Mar 16 10:29:44 2007 From: lac at openend.se (Laura Creighton) Date: Fri, 16 Mar 2007 10:29:44 +0100 Subject: [pypy-dev] page needs update Message-ID: <200703160929.l2G9TiDc008007@theraft.openend.se> http://codespeak.net/pypy/dist/pypy/doc/dev_method.html add post limerick sprints and sprint participants. Having missed most of these, I cannot do it myself, sorry. From micahel at gmail.com Fri Mar 16 11:06:55 2007 From: micahel at gmail.com (Michael Hudson) Date: Fri, 16 Mar 2007 10:06:55 +0000 Subject: [pypy-dev] page needs update In-Reply-To: References: <200703160929.l2G9TiDc008007@theraft.openend.se> Message-ID: Meant to reply to all, sorry! On 16/03/07, Michael Hudson wrote: > On 16/03/07, Laura Creighton wrote: > > http://codespeak.net/pypy/dist/pypy/doc/dev_method.html > > add post limerick sprints and sprint participants. Having missed > > most of these, I cannot do it myself, sorry. > > All the information is on: > > http://codespeak.net/pypy/dist/pypy/doc/sprint-reports.html > > though the list of participants would take a bit of effort to compile. Actually, that's not quite right, as we never wrote a report for Limerick :( And I guess there are all the people.txt pages for the participants... Cheers, mwh From jgustak at gmail.com Fri Mar 16 21:05:36 2007 From: jgustak at gmail.com (Jakub Gustak) Date: Fri, 16 Mar 2007 21:05:36 +0100 Subject: [pypy-dev] Scheme front end - Google Summer of Code Message-ID: I am deeply interested in writing scheme interpreter/front-end as a part of PyPy Google Summer of Code project. As advised on GSoC site i decided to contact to this mailing list to discuss my application. ### begin ### ================================= PyPy Scheme interpreter/front-end ================================= == Intro == PyPy project's main part is not Python interpreter implementation, but its configurable translator. It provides a good way to avoid writing interpreter for every language and platform. More information: http://codespeak.net/pypy/dist/pypy/doc/architecture.html This project aims to write an interpreter for Scheme for the PyPy framework. Scheme as very interesting dynamic language is nice goal to implement in RPython and provide a translator. It's interesting from a academic point of view to see if call/cc can be implemented on top of the primitives the stackless transform provides. This can lead to possibility of implementing first-class continuations (similar to Scheme ones) in Python. == Project Details == Proposed features: I am aiming to implement almost complete Scheme implementation (according to r5rs specification) in RPython More details: * Complete scheme parser (read macro) in RPython including delayed evaluation and quasi-quotations and macros. * Object space for Scheme covering all Scheme data types. * Research on possibility of implementing call/cc continuations using primitives provided by stackless transform. == Project Schedule == I am able to spend 10 hours/week starting from May until the end of July (end of semester in Poland). During the summer I am able to spend 40 hours/week working on a project. By the end of July I would like to complete the parser, without macros, delayed evaluation and continuations. Then focus on Object space to provide semi functional translator/interpreter. Later I would go on with missing features (e.g call/cc on stackless). If all will go fine we will end up with complete Scheme front-end before mid September, although some restrictions are possible. == About Me == I am a third year student at Wroclaw University of Technology (Poland), Computer Science major. I am interested in meta programming and programming languages design. I have experience in both Lisp and Python languages, including some meta programming. I also am currently studying MIT Structure and Implementation of Computer Programs lectures and text book by Harold Abelson, Gerald J. Sussman and Julie Sussman. I wanted to participate in Google Summer of Code coding in Python or Lisp. This subject gives me opportunity to experiment with both of them. I wrote most of my academic projects in Python. I also am currently working in web application based on Django framework. I am participating in algorithmic contest as spoj.pl or opss.safo.biz writing some of my solutions in Common Lisp, discovering functional programming. I also am a active member (actually a chairman) of IT Academic Association (Akademickie Stowarzyszenie Informatyczne) at Wroclaw University of Technology. I also actively promote free and open software at my University, giving lectures about Xen hypervisor and Kernel-based VM, Vim 7 editor, Subversion and Django-project. Some informations about my work (in Polish): http://jamit.pl/wiki/ http://www.linuxacademy.wroc.pl/ http://www.asi.pwr.wroc.pl/ ### end ### I am looking forward for some feedback from mentors. Sincerely, Jakub ?ukasz Gustak From radix at twistedmatrix.com Fri Mar 16 21:19:10 2007 From: radix at twistedmatrix.com (Christopher Armstrong) Date: Fri, 16 Mar 2007 16:19:10 -0400 Subject: [pypy-dev] Scheme front end - Google Summer of Code In-Reply-To: References: Message-ID: <60ed19d40703161319vf524db8oa47962453c7fcece@mail.gmail.com> On 3/16/07, Jakub Gustak wrote: > I am deeply interested in writing scheme interpreter/front-end as a > part of PyPy Google Summer of Code project. Hmm. That's interesting. I've worked for a little while on a scheme-like interpreter called Safelisp; you may want to take a look at it, and please feel free to ask me any questions. It's somewhat blocked at the moment while I finish reading _Lisp in Small Pieces_ :-) Anyway, it's at http://launchpad.net/subol/ -- the "safelisp-rpython" branch in particular (there are also some outstanding branches from Allen Short lying around, which may represent future developments in Safelisp). -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/ From santagada at gmail.com Fri Mar 16 22:30:40 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Fri, 16 Mar 2007 18:30:40 -0300 Subject: [pypy-dev] Scheme front end - Google Summer of Code In-Reply-To: References: Message-ID: <2f2e5f950703161430o6010d953xbb913bd4a56c3796@mail.gmail.com> Man that would be a great idea, I just hope that if you do it I will be able to assist with anything and get some ideas from your code. If he gets chosen who is going to mentor him? christian tismer and cfbolz I suppose. On 3/16/07, Jakub Gustak wrote: > > I am deeply interested in writing scheme interpreter/front-end as a > part of PyPy Google Summer of Code project. > > As advised on GSoC site i decided to contact to this mailing list to > discuss my application. > > ### begin ### > ================================= > PyPy Scheme interpreter/front-end > ================================= > > == Intro == > > PyPy project's main part is not Python interpreter implementation, but > its configurable translator. It provides a good way to avoid writing > interpreter for every language and platform. More information: > http://codespeak.net/pypy/dist/pypy/doc/architecture.html > > This project aims to write an interpreter for Scheme for the PyPy > framework. > Scheme as very interesting dynamic language is nice goal to implement > in RPython and provide a translator. It's interesting from a academic > point of view to see if call/cc can be implemented on top of the > primitives the stackless transform provides. This can lead to > possibility of implementing first-class continuations (similar to > Scheme ones) in Python. > > == Project Details == > > Proposed features: > > I am aiming to implement almost complete Scheme implementation > (according to r5rs specification) in RPython > > More details: > * Complete scheme parser (read macro) in RPython including delayed > evaluation and quasi-quotations and macros. > * Object space for Scheme covering all Scheme data types. > * Research on possibility of implementing call/cc continuations > using > primitives provided by stackless transform. > > == Project Schedule == > I am able to spend 10 hours/week starting from May until the end of > July (end of semester in Poland). During the summer I am able to spend > 40 hours/week working on a project. > By the end of July I would like to complete the parser, without > macros, delayed evaluation and continuations. Then focus on Object > space to provide semi functional translator/interpreter. Later I would > go on with missing features (e.g call/cc on stackless). If all will go > fine we will end up with complete Scheme front-end before mid > September, although some restrictions are possible. > > == About Me == > I am a third year student at Wroclaw University of Technology > (Poland), Computer Science major. I am interested in meta programming > and programming languages design. I have experience in both Lisp and > Python languages, including some meta programming. I also am currently > studying MIT Structure and Implementation of Computer Programs > lectures and text book by Harold Abelson, Gerald J. Sussman and Julie > Sussman. > > I wanted to participate in Google Summer of Code coding in Python or > Lisp. This subject gives me opportunity to experiment with both of > them. > > I wrote most of my academic projects in Python. I also am currently > working in web application based on Django framework. > I am participating in algorithmic contest as spoj.pl or opss.safo.biz > writing some of my solutions in Common Lisp, discovering functional > programming. > > I also am a active member (actually a chairman) of IT Academic > Association (Akademickie Stowarzyszenie Informatyczne) at Wroclaw > University of Technology. I also actively promote free and open > software at my University, giving lectures about Xen hypervisor and > Kernel-based VM, Vim 7 editor, Subversion and Django-project. > > Some informations about my work (in Polish): > http://jamit.pl/wiki/ > http://www.linuxacademy.wroc.pl/ > http://www.asi.pwr.wroc.pl/ > > ### end ### > > I am looking forward for some feedback from mentors. > > Sincerely, > Jakub ?ukasz Gustak > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -- Leonardo Santagada (http://www.lomohomes.com/retype) N?o se preocupe com o que os outros v?o fazer. O melhor jeito de prever o futuro ? inventa-lo. - Alan Kay -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.degrandis at gmail.com Fri Mar 16 22:39:56 2007 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Fri, 16 Mar 2007 17:39:56 -0400 Subject: [pypy-dev] Google Summer of Code In-Reply-To: <45F9B675.5010606@gmail.com> References: <20070315175244.GA15208@code0.codespeak.net> <45F9B675.5010606@gmail.com> Message-ID: <9c0bb8a00703161439n3477d224q8d266ce62b4933dc@mail.gmail.com> I submitted my proposal today! If you guys need me to tweak the application, please let me know! Paul On 3/15/07, Antonio Cuni wrote: > > Hi Armin, hi all! > > Armin Rigo wrote: > > Hi all, > > > > A few PyPy developers have signed up as mentors for the Google Summer of > > Code. Students interested in a summer project about PyPy (or any > > Python-related project) should have a look at: > > > > http://wiki.python.org/moin/SummerOfCode > > So, outside there is the sun (well, not right now :-)), the temperature > is getting warmer and it's already time to think again about summer of > code :-). > > First of all, I don't know who and how many signed up as a mentors: if > the PyPy project needs more, I'm available to sign up as a mentor as well. > > Next, I'm also considering the possibility of applying as a student > again: the PhD grant I will receive from 1st of April is very low, so I > will likely look for an additional job, and it would be great if the > job were pypy-related instead of one of those boring projects I used to > do before :-). > > But before applying I want to be sure not to be unfair, because my PhD > is also about PyPy and I don't want to be paid twice for the same work: > so my idea would be to do something unrelated to gencli&Co., just to be > sure it's something that does not fit in my PhD program. > > Moreover, I will be well-disposed to withdraw my application if there > are many others interesting pypy proposals, because I would not want to > steal someone else the opportunity of doing such as great experience and > to be involved in the project. > > What do you think about it? As I said I basically worry to be judged > unfair, so I would like to know the opinions of other developers before > begin thinking of a proposal (and in that case I guess that #pypy is > much more convenient for discussing it :-)). > > ciao Anto > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From len-l at telus.net Sat Mar 17 00:05:29 2007 From: len-l at telus.net (Lenard Lindstrom) Date: Fri, 16 Mar 2007 16:05:29 -0700 Subject: [pypy-dev] Scheme front end - Google Summer of Code In-Reply-To: References: Message-ID: <45FB22B9.5020608@telus.net> Jakub Gustak wrote: > I am deeply interested in writing scheme interpreter/front-end as a > part of PyPy Google Summer of Code project. > > As advised on GSoC site i decided to contact to this mailing list to > discuss my application. > > ### begin ### > ================================= > PyPy Scheme interpreter/front-end > ================================= > > == Intro == > > PyPy project's main part is not Python interpreter implementation, but > its configurable translator. It provides a good way to avoid writing > interpreter for every language and platform. More information: > http://codespeak.net/pypy/dist/pypy/doc/architecture.html > > This project aims to write an interpreter for Scheme for the PyPy framework. > Scheme as very interesting dynamic language is nice goal to implement > in RPython and provide a translator. It's interesting from a academic > point of view to see if call/cc can be implemented on top of the > primitives the stackless transform provides. This can lead to > possibility of implementing first-class continuations (similar to > Scheme ones) in Python. > > == Project Details == > > Proposed features: > > I am aiming to implement almost complete Scheme implementation > (according to r5rs specification) in RPython > > More details: > * Complete scheme parser (read macro) in RPython including delayed > evaluation and quasi-quotations and macros. > * Object space for Scheme covering all Scheme data types. > * Research on possibility of implementing call/cc continuations using > primitives provided by stackless transform. > > Just curious, but isn't proper tail recursion a consideration when writing a Scheme interpreter? It's not a Python feature. But I can see, though, that proper tail recursion is not needed to implement the proposed features of the "Project Details". -- Lenard Lindstrom From jgustak at gmail.com Sat Mar 17 01:28:54 2007 From: jgustak at gmail.com (Jakub Gustak) Date: Sat, 17 Mar 2007 01:28:54 +0100 Subject: [pypy-dev] Scheme front end - Google Summer of Code In-Reply-To: <45FB22B9.5020608@telus.net> References: <45FB22B9.5020608@telus.net> Message-ID: > Just curious, but isn't proper tail recursion a consideration when > writing a Scheme interpreter? It's not a Python feature. But I can see, > though, that proper tail recursion is not needed to implement the > proposed features of the "Project Details". Indeed, good point. Scheme does not make much sense without proper tail recursion. I should be implemented as well. Thanks for the remark. I would like to collect some more feedback, then think about it during the weekend, and then submit final application. _ Jakub ?ukasz Gustak From sabre at nondot.org Sun Mar 18 00:25:26 2007 From: sabre at nondot.org (Chris Lattner) Date: Sat, 17 Mar 2007 15:25:26 -0800 (PST) Subject: [pypy-dev] PyPy + LLVM In-Reply-To: <20070314135917.GA31254@code0.codespeak.net> References: <20070314135917.GA31254@code0.codespeak.net> Message-ID: On Wed, 14 Mar 2007, Armin Rigo wrote: > Hi Chris, Hey armin, I'm cc'ing pypy-dev, because this is suitable for a broader audience. I'm sorry for the delay in my reply, I have been travelling recently. > After reading your slides from yesterday's presentation, it occurred to > me that you might have a wrong idea about what PyPy is. I'm Sorry about that. The third section of my talk was only loosely based on pypy. Some of it was "inspired" by pypy, some of it was "unification" ideas that I have been thinking for a while, some is based on my own experience with static analysis, and some is just wishful thinking. I'm sorry if it came across as "here is what pypy is doing", that is now what I intended. > double-guessing here, so of course I may be completely off - please bear > with me. What PyPy is definitely not, is a Python compiler. In order > to get to a common ground of discussion, may I suggest the following > references: Right. If I had to do a layman's summary, I would say that pypy provides infrastructure for building interpreters in [r]python. This infrastructure makes it much easier than starting from scratch, e.g. by providing reusable components for language runtimes (like GC's). My talk (available here http://llvm.org/pubs/2007-03-12-BossaLLVMIntro.html ) clearly doesn't describe pypy as it exists today, but I think it describes a place that pypy could get to if the community desired it (In other words, I think the strengths of pypy and of its community both play well into this). The members of the pypy project clearly have experience with type inference, clearly understand dynamic language issues, and clearly understand that the semantics of these languages differ widely, but there are also a lot of commonality. If the pypy community isn't interested, I will approach others, eventually falling back to doing it directly in LLVM if needed. For the record, I have read Brett Cannon's thesis. I don't think his results are applicable for a number of reasons. First, his work was in the context of an interpreter that used type information for optimization. The approach I'm describing uses type information to give substantially more information to a static compiler. The result of this is that the codegen would be dramatically more efficient, and without the overhead of an interpreter loop, this is far more significant than in his evaluation. Also, significantly more type information would be available to the type propagator if you used more aggressive techniques than he did. To me, the much more significant issue with python (and ruby, and others) for integer operations is that they automatically promote to big integers on demand. In practice, this means that the code generated by a static compiler would be optimized for the small case, but then would fallback gracefully in the large case. If you're familiar with X86 asm, I'd expect code for a series of adds to look like this: add EAX, EBX jo recover_1 add EAX, 17 jo recover_2 add EAX, ECX jo recover_3 The idea of this is that (in the common case where the app deals with small integers) you have *exteremely* fast codegen with easily predicatable branches. The recovery code could, for example, package up the registers into real "integer" objects, and then callback into the interpreter to handle the hard case (note that this is very similar to the fast paths in the standard python interpreter, which eventually falls back to calling PyNumber_Add). Floating point code, if type inference is successful, does not need this sort of recovery code (unlike integer code). Other languages (like Javascript) don't have this issue, but they have others (e.g. there are no integers :), only floats. Of course I'm biased, but I think that this sort of code could easily and naturally be generated and optimized by LLVM, if you chose to target it. This would let LLVM do the range analysis required to eliminate the branches (when possible), handle other low level optimization, codegen, etc. You would get extremely high performance code and a very retargettable compiler. Has python even been executed with a sustained performance of two instructions per add? :) -Chris -- http://nondot.org/sabre/ http://llvm.org/ From niko at alum.mit.edu Sun Mar 18 09:39:42 2007 From: niko at alum.mit.edu (Niko Matsakis) Date: Sun, 18 Mar 2007 09:39:42 +0100 Subject: [pypy-dev] Google Summer of Code In-Reply-To: <9c0bb8a00703161439n3477d224q8d266ce62b4933dc@mail.gmail.com> References: <20070315175244.GA15208@code0.codespeak.net> <45F9B675.5010606@gmail.com> <9c0bb8a00703161439n3477d224q8d266ce62b4933dc@mail.gmail.com> Message-ID: <8B7E7CBF-B84E-49B9-A9A5-C97367A603B9@alum.mit.edu> > I submitted my proposal today! > If you guys need me to tweak the application, please let me know! > I would definitely be excited to see it progress. I haven't had much time lately. Before you start working, I can of course help introduce you to the code. Niko From cfbolz at gmx.de Sun Mar 18 11:01:01 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sun, 18 Mar 2007 11:01:01 +0100 Subject: [pypy-dev] Google Summer of Code In-Reply-To: <9c0bb8a00703161439n3477d224q8d266ce62b4933dc@mail.gmail.com> References: <20070315175244.GA15208@code0.codespeak.net> <45F9B675.5010606@gmail.com> <9c0bb8a00703161439n3477d224q8d266ce62b4933dc@mail.gmail.com> Message-ID: <45FD0DDD.4050201@gmx.de> Hi Paul! Paul deGrandis wrote: > I submitted my proposal today! > If you guys need me to tweak the application, please let me know! Would you maybe post the application here too? I think it would be good to have a common discussion about it (not going back and forth in the Google web app, which takes a lot of time and such). Cheers, Carl Friedrich From paul.degrandis at gmail.com Sun Mar 18 11:10:10 2007 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Sun, 18 Mar 2007 06:10:10 -0400 Subject: [pypy-dev] Google Summer of Code In-Reply-To: <45FD0DDD.4050201@gmx.de> References: <20070315175244.GA15208@code0.codespeak.net> <45F9B675.5010606@gmail.com> <9c0bb8a00703161439n3477d224q8d266ce62b4933dc@mail.gmail.com> <45FD0DDD.4050201@gmx.de> Message-ID: <9c0bb8a00703180310h56da0149o68961cf5eb1f0388@mail.gmail.com> Hi Carl, Totally, that makes a lot of time. Here is the abstract... The recent advancements of PyPy have been impressive to say the least. Much work as been done to provide a JVM backend, but it still lacks full completion of finer details. The goal of my summer of code project will be to work at the remaining problems to achieve a level stability that is fitting to be used with JSR-223 (the Java Scripting addition) and Java6. Doing so will allow PyPy interoperability with Java, Scheme, Python, JavaScript, Ruby, and many more languages that implement this JSR specification. Additionally, should time allow, an empirical experiment will be conducted to show the PyPy's affect on developer efficiency in an academic setting using the JVM backend and JSR-223 bridge. The detailed description: Various developers of the Pypy team (and community) have helped get the JVM high-level backend where it is today. As stated in the the PyPy report from March 12th, "There are a number of possible improvements to this system." My plan for this project is to have detailed discussions with each of the developers both separately and together to gather what they feel are the most important improvements. Through a process similar to that of task-selection in Extreme Programming, I'll work on the tasks one by one, contributing to the large test suite of the PyPy system, and improving the JVM backend. Ideally, half way through the process of improving the backend, I would like to begin the process of creating JSR-223 bindings for use in Java6. This would allow the PyPy Objects and Java Object to work together. To achieve any sort of progress in this area, bridging and mapping into Java libraries and objects is necessary. Once this is achieved such benefits as using Swing GUI objects in PyPy should be possible. I plan on seeking much guidance from jython developers (as some of the work on the PyPy project as well). While this might not be completed in the time span allowed for Summer Of Code, it is one feature of PyPy I would like to see carried out and will see through to completion. The JSR-223 specification binding would allow for interoperability between Java, Python, PyPy, Ruby, Scheme, JavaScript and many more languages. Additionally, during the time frame of summer of code, I would like collect empirical data that illustrates PyPy's affect on developer efficiency in an academic research environment. I plan on writing and publishing the results of ALL summer of code work to conferences like ICSM, SCAM, and others. Regards, Paul On 3/18/07, Carl Friedrich Bolz wrote: > > Hi Paul! > > Paul deGrandis wrote: > > I submitted my proposal today! > > If you guys need me to tweak the application, please let me know! > > Would you maybe post the application here too? I think it would be good > to have a common discussion about it (not going back and forth in the > Google web app, which takes a lot of time and such). > > Cheers, > > Carl Friedrich > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nogradi at gmail.com Sun Mar 18 11:35:02 2007 From: nogradi at gmail.com (Daniel Nogradi) Date: Sun, 18 Mar 2007 11:35:02 +0100 Subject: [pypy-dev] Google Summer of Code In-Reply-To: <9c0bb8a00703180310h56da0149o68961cf5eb1f0388@mail.gmail.com> References: <20070315175244.GA15208@code0.codespeak.net> <45F9B675.5010606@gmail.com> <9c0bb8a00703161439n3477d224q8d266ce62b4933dc@mail.gmail.com> <45FD0DDD.4050201@gmx.de> <9c0bb8a00703180310h56da0149o68961cf5eb1f0388@mail.gmail.com> Message-ID: <5f56302b0703180335m7712db46gfd24c6879cbf2bee@mail.gmail.com> Just some comments for anyone who wants to do the mochikit subproject of the JS backend in the hope that someone finds it useful. Some time ago I started this but stopped due to lack of time but a couple of ideas were considered: 1. I contacted Bob Ippolito and he said he would be willing to put function annotation data into the rst documentation of mochikit once we define exactly what kind of annotation data we want. This was not pursued any further but if someone comes up with a good way of doing this it would be very useful for automatically parsing the documentation and creating the python modules on the fly especially if new versions of mochikit come out. 2. How about integrating ElementTree (or etree in python 2.5) so that the DOM can be manipulated that way? The JS backend would need to convert the ElementTree API to the JS DOM API. Cheers, Daniel From arigo at tunes.org Sun Mar 18 13:26:37 2007 From: arigo at tunes.org (Armin Rigo) Date: Sun, 18 Mar 2007 13:26:37 +0100 Subject: [pypy-dev] PyPy + LLVM In-Reply-To: References: <20070314135917.GA31254@code0.codespeak.net> Message-ID: <20070318122637.GA27138@code0.codespeak.net> Hi Chris, On Sat, Mar 17, 2007 at 03:25:26PM -0800, Chris Lattner wrote: > My talk (available here > http://llvm.org/pubs/2007-03-12-BossaLLVMIntro.html ) clearly doesn't > describe pypy as it exists today, but I think it describes a place that > pypy could get to if the community desired it (In other words, I think > the strengths of pypy and of its community both play well into this). This is a FAQ (or I should say a CEPW - Common External Point of View): http://codespeak.net/pypy/dist/pypy/doc/faq.html#can-pypy-compile-normal-python-programs I agree that what you describe could be done, given enough efforts; it may even be that the overall effort would be less than what we alredy invested in PyPy. However, I can't speak for all the people involved with PyPy but I, at least, am not interested in this approach. My own vision is that of non-static, language-independent optimizations. What we are building supports arbitrary changes to the language semantics. For now, changing the semantics is a static process which requires a full rebuild of PyPy and its JIT, which takes about one hour. In the very long term it might be possible to allow such changes to be inserted into a running environment - and to do that, LLVM will most likely be an essential tool. > Has python even been executed with a sustained performance of two > instructions per add? :) Psyco did that for years, yes. A few days ago, PyPy's own JIT did it for trivial examples too. Neither contain a type inference engine. The latter adapts automatically to arbitrary semantic changes. Neither produce optimal register usage: they produce code that requires continuous run-time updates; at the hot spots, the code eventually stabilizes. The LLVM JIT fits in this picture as a second-stage recompiler for when the code in the hot spots looks stable enough. A bientot, Armin From arigo at tunes.org Sun Mar 18 13:43:16 2007 From: arigo at tunes.org (Armin Rigo) Date: Sun, 18 Mar 2007 13:43:16 +0100 Subject: [pypy-dev] Scheme front end - Google Summer of Code In-Reply-To: <45FB22B9.5020608@telus.net> References: <45FB22B9.5020608@telus.net> Message-ID: <20070318124316.GA32530@code0.codespeak.net> Hi Lenard, On Fri, Mar 16, 2007 at 04:05:29PM -0700, Lenard Lindstrom wrote: > Just curious, but isn't proper tail recursion a consideration when > writing a Scheme interpreter? It's not a Python feature. Note that you get proper tail recursion in RPython when using the stackless features. (It might still make sense to write the Scheme interpreter in a non-recursive way, if it's easy enough.) A bientot, Armin From micahel at gmail.com Sun Mar 18 17:43:37 2007 From: micahel at gmail.com (Michael Hudson) Date: Sun, 18 Mar 2007 16:43:37 +0000 Subject: [pypy-dev] Scheme front end - Google Summer of Code In-Reply-To: <20070318124316.GA32530@code0.codespeak.net> References: <45FB22B9.5020608@telus.net> <20070318124316.GA32530@code0.codespeak.net> Message-ID: On 18/03/07, Armin Rigo wrote: > Hi Lenard, > > On Fri, Mar 16, 2007 at 04:05:29PM -0700, Lenard Lindstrom wrote: > > Just curious, but isn't proper tail recursion a consideration when > > writing a Scheme interpreter? It's not a Python feature. > > Note that you get proper tail recursion in RPython when using the > stackless features. (It might still make sense to write the Scheme > interpreter in a non-recursive way, if it's easy enough.) There's also the mildly interesting theoretical question of whether you can use the stackless primitive "yield_current_frame_to_caller" to implement call-with-current-continuation (I suspect not, but would be interesting to see how close you can get). Cheers, mwh From holger at merlinux.de Sun Mar 18 18:29:47 2007 From: holger at merlinux.de (holger krekel) Date: Sun, 18 Mar 2007 18:29:47 +0100 Subject: [pypy-dev] daily #pypy-sync for 1.0 5pm Message-ID: <20070318172947.GH4635@solar.trillke> Hi folks, As you hopefully know by now, we are in the final run up to our long-awaited 1.0 release. Therefore we suggest to do daily #pypy-sync "meetings" until we have the release out and suggest as time frame 5pm-5:30pm (GMT+1) starting from 19th March (Monday). The meetings should focus on tracking, discussing and assigning pending issues, to be found and updated here: extradoc/planning/1.0/TODO.txt Please note that the dense timeplan at the top of the file aims at a release at 26th March, so please try to be generally available for help, testing, determining and fixing problems particularly in your own expertise areas. If you find any critical problems/issues, please warn Michael and/or Holger (the release drivers) about it quickly. Cheers, Michael and Holger From brian at dorseys.org Sun Mar 18 19:59:23 2007 From: brian at dorseys.org (Brian Dorsey) Date: Sun, 18 Mar 2007 11:59:23 -0700 Subject: [pypy-dev] Google Summer of Code In-Reply-To: <45FD0DDD.4050201@gmx.de> References: <20070315175244.GA15208@code0.codespeak.net> <45F9B675.5010606@gmail.com> <9c0bb8a00703161439n3477d224q8d266ce62b4933dc@mail.gmail.com> <45FD0DDD.4050201@gmx.de> Message-ID: <66e877b70703181159n29704c23iaff2bd755ceae560@mail.gmail.com> On 3/18/07, Carl Friedrich Bolz wrote: > Would you maybe post the application here too? I think it would be good > to have a common discussion about it (not going back and forth in the > Google web app, which takes a lot of time and such). While it may take more time, any conversation which happens outside the google app will likely be invisible to many of the PSF mentors. Perhaps someone can at least post a summary of whatever is discussed here to the google mentor site? Take care, -Brian From len-l at telus.net Sun Mar 18 20:21:24 2007 From: len-l at telus.net (Lenard Lindstrom) Date: Sun, 18 Mar 2007 12:21:24 -0700 Subject: [pypy-dev] Scheme front end - Google Summer of Code In-Reply-To: <20070318124316.GA32530@code0.codespeak.net> References: <45FB22B9.5020608@telus.net> <20070318124316.GA32530@code0.codespeak.net> Message-ID: <45FD9134.403@telus.net> Armin Rigo wrote: > Hi Lenard, > > On Fri, Mar 16, 2007 at 04:05:29PM -0700, Lenard Lindstrom wrote: > >> Just curious, but isn't proper tail recursion a consideration when >> writing a Scheme interpreter? It's not a Python feature. >> > > Note that you get proper tail recursion in RPython when using the > stackless features. (It might still make sense to write the Scheme > interpreter in a non-recursive way, if it's easy enough.) > > Hi Armin, Where I am going with this, I guess, is just how much of the PyPy Python interpreter can be reused in writing the Scheme interpreter. By the looks of it most of the machinery is already in place. And the flip side of the the question is how easily would it be to move those modifications back into Python? Of course the Scheme requirements may be incompatible with Python. -- Lenard Lindstrom From alexandre.fayolle at logilab.fr Sun Mar 18 21:09:06 2007 From: alexandre.fayolle at logilab.fr (Alexandre Fayolle) Date: Sun, 18 Mar 2007 21:09:06 +0100 Subject: [pypy-dev] [pypy-svn] r40710 - pypy/dist/pypy/interpreter/astcompiler In-Reply-To: <20070318170933.E4A6B10086@code0.codespeak.net> References: <20070318170933.E4A6B10086@code0.codespeak.net> Message-ID: <20070318200906.GA3893@crater.logilab.fr> On Sun, Mar 18, 2007 at 06:09:33PM +0100, arigo at codespeak.net wrote: > Node.typedef = TypeDef('ASTNode', > __new__ = interp2app(descr_Node_new, unwrap_spec=[ObjSpace, W_Root, int]), > #__repr__ = interp2app(Node.descr_repr, unwrap_spec=['self', ObjSpace] ), > - __repr__ = interp2app(Node.descr_repr, unwrap_spec=['self', ObjSpace] ), > + #__repr__ = interp2app(Node.descr_repr, unwrap_spec=['self', ObjSpace] ), Sorry about this. I did not mean to check in the uncommented version in subversion. -- Alexandre Fayolle LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services Informatique scientifique: http://www.logilab.fr/science Reprise et maintenance de sites CPS: http://www.migration-cms.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 481 bytes Desc: Digital signature URL: From jgustak at gmail.com Mon Mar 19 01:48:41 2007 From: jgustak at gmail.com (Jakub Gustak) Date: Mon, 19 Mar 2007 01:48:41 +0100 Subject: [pypy-dev] Scheme front end - Google Summer of Code In-Reply-To: References: <45FB22B9.5020608@telus.net> Message-ID: > I would like to collect some more feedback, then think about it during > the weekend, and then submit final application. Submited. _ Jakub ?ukasz Gustak From scott+pypy-dev at scottdial.com Wed Mar 21 13:59:34 2007 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Wed, 21 Mar 2007 08:59:34 -0400 Subject: [pypy-dev] jit.timeshifter.test uses a lot of memory Message-ID: <46012C36.1060300@scottdial.com> I'm not sure if there has been a recent change, but I've always noticed that this test suite requires a lot of memory. What has brought my attention to it especially is that I've been awake while my box has been running the test and it brings things to a crawl with swapping :-( I can't say for sure what the memory requirements are but it is clearly more than what a system with 2GiB of RAM can handle. I can say I've seen it using 1.7GiB (right before I cut it off because I needed to get some work done). My first thought is there is something going terribly wrong due to the tests being run under win32, but I don't get any errors indicating that sort of thing is going on. Is this memory usage intended? Is there anything that can be done to reduce it? I'd hate to have to start skipping those tests. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From arigo at tunes.org Wed Mar 21 17:10:25 2007 From: arigo at tunes.org (Armin Rigo) Date: Wed, 21 Mar 2007 17:10:25 +0100 Subject: [pypy-dev] Summer of Code - deadline extended until next Monday Message-ID: <20070321161024.GA19249@code0.codespeak.net> Hi all, The Summer of Code deadline for student application submission is approaching fast! But you've just been given the whole week-end to ponder over it. The deadline has just been pushed out to: Monday night, the 26th (midnight in UTC). A bientot, Armin. From arigo at tunes.org Thu Mar 22 14:38:10 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 22 Mar 2007 14:38:10 +0100 Subject: [pypy-dev] jit.timeshifter.test uses a lot of memory In-Reply-To: <46012C36.1060300@scottdial.com> References: <46012C36.1060300@scottdial.com> Message-ID: <20070322133809.GA19108@code0.codespeak.net> Hi Scott, On Wed, Mar 21, 2007 at 08:59:34AM -0400, Scott Dial wrote: > I'm not sure if there has been a recent change, but I've always noticed > that this test suite requires a lot of memory. Yes, it's a known problem. We are simply adding more and more tests, and they all leave a lot of stuff behind. On the automated nightly tests on wyvern.cs.uni-duesseldorf.de, we run a few directories one test file at a time (instead of the whole directory at a time). See http://codespeak.net/svn/user/arigo/hack/misc/htmlconftest/autotest.py line 51. A bientot, Armin. From scott+pypy-dev at scottdial.com Thu Mar 22 14:59:05 2007 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Thu, 22 Mar 2007 09:59:05 -0400 Subject: [pypy-dev] jit.timeshifter.test uses a lot of memory In-Reply-To: <20070322133809.GA19108@code0.codespeak.net> References: <46012C36.1060300@scottdial.com> <20070322133809.GA19108@code0.codespeak.net> Message-ID: <46028BA9.80807@scottdial.com> Armin Rigo wrote: > Hi Scott, > On Wed, Mar 21, 2007 at 08:59:34AM -0400, Scott Dial wrote: >> I'm not sure if there has been a recent change, but I've always noticed >> that this test suite requires a lot of memory. > > Yes, it's a known problem. We are simply adding more and more tests, > and they all leave a lot of stuff behind. On the automated nightly > tests on wyvern.cs.uni-duesseldorf.de, we run a few directories one test > file at a time (instead of the whole directory at a time). See > http://codespeak.net/svn/user/arigo/hack/misc/htmlconftest/autotest.py > line 51. > Ah, I haven't been staying readily on-top of updates to htmlconftest, but I do have that bit in there except the assumption about the path separator got me. Would you be open to me providing a rather large patch to add support for win32, with the caveat that it perhaps would be very specific to my box? Then we could stay in sync better and there probably is some merit to making htmlconftest portable to win32 *wink* -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From arigo at tunes.org Thu Mar 22 15:32:03 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 22 Mar 2007 15:32:03 +0100 Subject: [pypy-dev] jit.timeshifter.test uses a lot of memory In-Reply-To: <46028BA9.80807@scottdial.com> References: <46012C36.1060300@scottdial.com> <20070322133809.GA19108@code0.codespeak.net> <46028BA9.80807@scottdial.com> Message-ID: <20070322143203.GA25592@code0.codespeak.net> Hi Scott, On Thu, Mar 22, 2007 at 09:59:05AM -0400, Scott Dial wrote: > Ah, I haven't been staying readily on-top of updates to htmlconftest, > but I do have that bit in there except the assumption about the path > separator got me. Oups! Sorry. > Would you be open to me providing a rather large patch > to add support for win32, with the caveat that it perhaps would be very > specific to my box? Then we could stay in sync better and there probably > is some merit to making htmlconftest portable to win32 *wink* Sure, go ahead. Once this is merged, we can think about factoring out the machine-specific dependencies to the new 'autotestconfig.py' module. A bientot, Armin. From arigo at tunes.org Thu Mar 22 15:35:09 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 22 Mar 2007 15:35:09 +0100 Subject: [pypy-dev] Scheme front end - Google Summer of Code In-Reply-To: <45FD9134.403@telus.net> References: <45FB22B9.5020608@telus.net> <20070318124316.GA32530@code0.codespeak.net> <45FD9134.403@telus.net> Message-ID: <20070322143509.GB25592@code0.codespeak.net> Hi Lenard, On Sun, Mar 18, 2007 at 12:21:24PM -0700, Lenard Lindstrom wrote: > Where I am going with this, I guess, is just how much of the PyPy Python > interpreter can be reused in writing the Scheme interpreter. By the > looks of it most of the machinery is already in place. And the flip side > of the the question is how easily would it be to move those > modifications back into Python? Of course the Scheme requirements may be > incompatible with Python. We can think about this, yes. It's a topic that we vaguely thought about a few time: RPython should make it easy to bridge or merge different languages with each other. We never really explored the question more so far, but it sure is interesting. A bientot, Armin From arigo at tunes.org Thu Mar 22 15:40:36 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 22 Mar 2007 15:40:36 +0100 Subject: [pypy-dev] Scheme front end - Google Summer of Code In-Reply-To: References: <45FB22B9.5020608@telus.net> <20070318124316.GA32530@code0.codespeak.net> Message-ID: <20070322144036.GC25592@code0.codespeak.net> Hi Michael, On Sun, Mar 18, 2007 at 04:43:37PM +0000, Michael Hudson wrote: > There's also the mildly interesting theoretical question of whether > you can use the stackless primitive "yield_current_frame_to_caller" to > implement call-with-current-continuation (I suspect not, but would be > interesting to see how close you can get). No, by itself it only gives one-shot continuations. But then there is the frightening topic of the cloning operation of our GC, which can be used to clone whatever objects including such frame chains. This gives full continuations. (For the records, in a stackless and framework-gc build of pypy-c, you get "clonable coroutines" usable from the application-level Python, which I think are a rather nicer way than low-level continuations to express how much of the world you want to copy and how much you want to share.) A bientot, Armin. From scott+pypy-dev at scottdial.com Fri Mar 23 10:45:00 2007 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Fri, 23 Mar 2007 05:45:00 -0400 Subject: [pypy-dev] translator.js.test invokes pygame (AKA, yet more annoyances from Scott) Message-ID: <4603A19C.9090908@scottdial.com> Greetings, There is a bug somewhere in the translator.js.test suite that invokes pygame. I haven't tracked it down but I just thought I'd mention it because I'm sure someone else will know better than me. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From fijal at genesilico.pl Fri Mar 23 10:59:36 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Fri, 23 Mar 2007 10:59:36 +0100 Subject: [pypy-dev] translator.js.test invokes pygame (AKA, yet more annoyances from Scott) In-Reply-To: <4603A19C.9090908@scottdial.com> References: <4603A19C.9090908@scottdial.com> Message-ID: <4603A508.9010004@genesilico.pl> Scott Dial wrote: > Greetings, > > There is a bug somewhere in the translator.js.test suite that invokes > pygame. I haven't tracked it down but I just thought I'd mention it > because I'm sure someone else will know better than me. > > -Scott > > All places should not do this. Hmm... Could you provide more detailed info? Thanks for spotting. Cheers, fijal From scott+pypy-dev at scottdial.com Fri Mar 23 11:03:31 2007 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Fri, 23 Mar 2007 06:03:31 -0400 Subject: [pypy-dev] duplicate extregistry entry Message-ID: <4603A5F3.9090508@scottdial.com> Ever since a few revisions ago, rypthon/extregistry.py has been throwing an assertion: http://scottdial.com/pypytest/41143/failed/annotation.test.test_annmm.py.test_int_w_ann.html I've not located the change which caused this and unfortunately wyvern isn't showing the same problem. However maybe it is obvious to someone what is broke. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From scott+pypy-dev at scottdial.com Fri Mar 23 11:54:24 2007 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Fri, 23 Mar 2007 06:54:24 -0400 Subject: [pypy-dev] translator.js.test invokes pygame (AKA, yet more annoyances from Scott) In-Reply-To: <4603A508.9010004@genesilico.pl> References: <4603A19C.9090908@scottdial.com> <4603A508.9010004@genesilico.pl> Message-ID: <4603B1E0.1080200@scottdial.com> Maciek Fijalkowski wrote: >> There is a bug somewhere in the translator.js.test suite that invokes >> pygame. I haven't tracked it down but I just thought I'd mention it >> because I'm sure someone else will know better than me. >> > All places should not do this. Hmm... Could you provide more detailed > info? Thanks for spotting. > Sorry, I could've been more specific. They appear during translator.js.test.test_main. I decided to do the legwork.. rpython2javascript is the source of the problem. You have conflicting controlflow for determining whether to use pdb or not. The default value is use_pdb=True which will override jsconfig's use_pdb option always. Finally, during an Exception at the bottom debug(driver, use_pdb) will use the function argument and never the jsconfig value. So, jsconfig is basically ignored. I think the use_pdb argument needs to go away. http://scottdial.com/pypy-dev/translator.js.diff -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From scott+pypy-dev at scottdial.com Fri Mar 23 11:55:49 2007 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Fri, 23 Mar 2007 06:55:49 -0400 Subject: [pypy-dev] duplicate extregistry entry In-Reply-To: <4603A5F3.9090508@scottdial.com> References: <4603A5F3.9090508@scottdial.com> Message-ID: <4603B235.8070409@scottdial.com> Scott Dial wrote: > I've not located the change which caused this and unfortunately wyvern > isn't showing the same problem. However maybe it is obvious to someone > what is broke. > I retract this, mwh just fixed this on r41145. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From amauryfa at gmail.com Fri Mar 23 11:57:20 2007 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Fri, 23 Mar 2007 11:57:20 +0100 Subject: [pypy-dev] Compiling pypy with python2.5 Message-ID: Hello, After some work I managed to run the pypy translation using python 2.5. I know that it is not the best moment because of the imminent release, so feel free to put this under your pillow for now. With some changes, I successfully translate pypy-c on Windows (with --gc=framework), and the produced program seems to work. But I did not rerun the entire test suite. I join the patch (too shy to commit it), with some comments: 1/ The struct module was rewritten in python 2.5. Better use the full python implementation provided by pypy. 2/ KeyboardInterrupt is used by RPython; translation of exceptions must now process all classes derived from BaseException. 3/ This one was difficult, and I am not sure of the correction: with 2.5, IMPORT_NAME takes a new parameter to specify the "relative import" level. CPython-2.5 generate pcode with this extra parameter, but pypy-2.4 can ignore it. The hard part is that pyopcode.py must interpret pcode coming from both the pypy ast compiler and CPython .pyc files. Anyway the implementation in ceval.c itself is not clear (it seems to try to be compatible with 2.4 pcode. How can this work?). 4/ 5/ Easy corrections (not specific to 2.5) while trying the --jit option on Windows: platform.machine() is empty on my machine, and codebuf_nt is expected to export a PTR type. Index: pypy/module/array/app_array.py ================================================================ --- pypy/module/array/app_array.py (revision 41035) +++ pypy/module/array/app_array.py (working copy) @@ -24,7 +24,7 @@ array(typecode [, initializer]) -- create a new array """ import sys -from struct import pack, unpack +from pypy.lib.struct import pack, unpack if sys.maxunicode == 65535: UNICODE_SIZE = 2 Index: pypy/annotation/annrpython.py ================================================================ --- pypy/annotation/annrpython.py (revision 41035) +++ pypy/annotation/annrpython.py (working copy) @@ -8,6 +8,7 @@ from pypy.objspace.flow.model import Variable, Constant from pypy.objspace.flow.model import FunctionGraph from pypy.objspace.flow.model import c_last_exception, checkgraph +from py.builtin import BaseException import py log = py.log.Producer("annrpython") py.log.setconsumer("annrpython", ansi_log) @@ -622,7 +623,7 @@ last_exc_value_var = link.last_exc_value # may be None for non-exception link if isinstance(link.exitcase, (types.ClassType, type)) \ - and issubclass(link.exitcase, Exception): + and issubclass(link.exitcase, BaseException): assert last_exception_var and last_exc_value_var last_exc_value_object = self.bookkeeper.valueoftype(link.exitcase) last_exception_object = annmodel.SomeObject() Index: pypy/interpreter/pyopcode.py =================================================================== --- pypy/interpreter/pyopcode.py (revision 41035) +++ pypy/interpreter/pyopcode.py (working copy) @@ -705,6 +705,12 @@ w_modulename = f.getname_w(nameindex) modulename = f.space.str_w(w_modulename) w_fromlist = f.popvalue() + # AFA: Python 2.5 relative import + # we MAY encounter an additional '-1' parameter + if f.valuestackdepth > 0: + w_level = f.peekvalue() + if space.is_true(space.eq(w_level, space.wrap(-1))): + f.popvalue() w_import = f.get_builtin().getdictvalue_w(f.space, '__import__') if w_import is None: raise OperationError(space.w_ImportError, Index: pypy/jit/codegen/detect_cpu.py =================================================================== --- pypy/jit/codegen/detect_cpu.py (revision 41035) +++ pypy/jit/codegen/detect_cpu.py (working copy) @@ -8,10 +8,13 @@ pass def autodetect(): + mach = None try: import platform mach = platform.machine() except ImportError: + pass + if not mach: platform = sys.platform.lower() if platform.startswith('win'): # assume an Intel Windows return 'i386' Index: pypy/jit/codegen/i386/codebuf_nt.py =================================================================== --- pypy/jit/codegen/i386/codebuf_nt.py (revision 41035) +++ pypy/jit/codegen/i386/codebuf_nt.py (working copy) @@ -19,24 +19,24 @@ globals().update(ctypes_platform.configure(CConfig)) # cannot use c_void_p as return value of functions :-( -LPVOID = ctypes.POINTER(ctypes.c_char) +PTR = ctypes.POINTER(ctypes.c_char) VirtualAlloc = ctypes.windll.kernel32.VirtualAlloc -VirtualAlloc.argtypes = [LPVOID, SIZE_T, DWORD, DWORD] -VirtualAlloc.restype = LPVOID +VirtualAlloc.argtypes = [PTR, SIZE_T, DWORD, DWORD] +VirtualAlloc.restype = PTR VirtualProtect = ctypes.windll.kernel32.VirtualProtect -VirtualProtect.argtypes = [LPVOID, SIZE_T, DWORD, ctypes.POINTER(DWORD)] +VirtualProtect.argtypes = [PTR, SIZE_T, DWORD, ctypes.POINTER(DWORD)] VirtualProtect.restype = BOOL VirtualFree = ctypes.windll.kernel32.VirtualFree -VirtualFree.argtypes = [LPVOID, SIZE_T, DWORD] +VirtualFree.argtypes = [PTR, SIZE_T, DWORD] VirtualFree.restype = BOOL # ____________________________________________________________ def alloc(map_size): - res = VirtualAlloc(LPVOID(), map_size, MEM_COMMIT|MEM_RESERVE, + res = VirtualAlloc(PTR(), map_size, MEM_COMMIT|MEM_RESERVE, PAGE_EXECUTE_READWRITE) if not res: raise MemoryError -- Amaury Forgeot d'Arc From scott+pypy-dev at scottdial.com Fri Mar 23 15:11:53 2007 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Fri, 23 Mar 2007 10:11:53 -0400 Subject: [pypy-dev] translator.cli.query bug/patch for win32 Message-ID: <4603E029.8060609@scottdial.com> Under windows, subprocess.Popen will return stdout with newlines of '\r\n' which exec does not like (python code always has '\n' newlines). The solution is to simply turn on universal_newlines. http://scottdial.com/pypy-dev/translator.cli.query.diff -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From scott+pypy-dev at scottdial.com Fri Mar 23 15:29:11 2007 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Fri, 23 Mar 2007 10:29:11 -0400 Subject: [pypy-dev] Compiling pypy with python2.5 In-Reply-To: References: Message-ID: <4603E437.6070206@scottdial.com> Amaury Forgeot d'Arc wrote: > 4/ 5/ Easy corrections (not specific to 2.5) while trying the --jit > option on Windows: platform.machine() is empty on my machine, and > codebuf_nt is expected to export a PTR type. > I noticed this same problem and your patch is virtually identical to mine. I've run the test suite with this and it seems to be a positive change (as in I actually run more tests than I have in the past). So, I endorse this part of the patch. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From arigo at tunes.org Fri Mar 23 21:48:33 2007 From: arigo at tunes.org (Armin Rigo) Date: Fri, 23 Mar 2007 21:48:33 +0100 Subject: [pypy-dev] Compiling pypy with python2.5 In-Reply-To: <4603E437.6070206@scottdial.com> References: <4603E437.6070206@scottdial.com> Message-ID: <20070323204833.GA29982@code0.codespeak.net> Hi, On Fri, Mar 23, 2007 at 10:29:11AM -0400, Scott Dial wrote: > Amaury Forgeot d'Arc wrote: > > 4/ 5/ Easy corrections (not specific to 2.5) while trying the --jit > > option on Windows: platform.machine() is empty on my machine, and > > codebuf_nt is expected to export a PTR type. > > I noticed this same problem and your patch is virtually identical to > mine. I've run the test suite with this and it seems to be a positive > change (as in I actually run more tests than I have in the past). So, I > endorse this part of the patch. Checked this part in, thanks. We should maybe try a couple of Linux translations with the 2.5-relevant changes to see if they basically work there too, and apply them in this case. Amaury, the import from 'pypy.lib.struct' that your patch introduced in an 'app_*' module looks suspicious... I'm not sure why we are not automatically picking up struct.py from pypy/lib/ from app-level code. A bientot, Armin From arigo at tunes.org Fri Mar 23 22:04:39 2007 From: arigo at tunes.org (Armin Rigo) Date: Fri, 23 Mar 2007 22:04:39 +0100 Subject: [pypy-dev] Compiling pypy with python2.5 In-Reply-To: <20070323204833.GA29982@code0.codespeak.net> References: <4603E437.6070206@scottdial.com> <20070323204833.GA29982@code0.codespeak.net> Message-ID: <20070323210439.GA32409@code0.codespeak.net> Hi Amaury, On Fri, Mar 23, 2007 at 09:48:33PM +0100, Armin Rigo wrote: > Amaury, the import from 'pypy.lib.struct' that your patch introduced in > an 'app_*' module looks suspicious... I'm not sure why we are not > automatically picking up struct.py from pypy/lib/ from app-level code. Ah yes, a theory is as follows: geninterp tries to add pypy/lib/ in front of sys.path so that when it executes app_array.py directly on top of CPython the modules over there get found (that's a bit of a hack, really). But in CPython, 'import struct' always pick up the built-in struct module if it is really a built-in and not an extension module. So the hack may fail if we're running on top of a CPython with a built-in struct module. This could be solved with a counter-hack: try adding a file pypy/module/array/struct.py with the content: from pypy.lib.struct import * A bientot, Armin From jacob at strakt.com Sat Mar 24 11:35:42 2007 From: jacob at strakt.com (Jacob =?utf-8?q?Hall=C3=A9n?=) Date: Sat, 24 Mar 2007 11:35:42 +0100 Subject: [pypy-dev] Google SoC extension Message-ID: <200703241135.42305.jacob@strakt.com> Google has extended the application period for SoC projects to the 26th of March, so if there is anyone who dropped sending in an application due to time constraints, you have a second chance. Jacob From arigo at tunes.org Sat Mar 24 11:38:05 2007 From: arigo at tunes.org (Armin Rigo) Date: Sat, 24 Mar 2007 11:38:05 +0100 Subject: [pypy-dev] Compiling pypy with python2.5 In-Reply-To: References: <4603E437.6070206@scottdial.com> <20070323204833.GA29982@code0.codespeak.net> <20070323210439.GA32409@code0.codespeak.net> Message-ID: <20070324103804.GA952@code0.codespeak.net> Hi Amaury, On Fri, Mar 23, 2007 at 10:57:43PM +0100, Amaury Forgeot d'Arc wrote: > the struct module is built-in in CPython2.4; so the hack does work. Ah, I see: the struct module is pure Python in 2.5, so my theory doesn't hold. (Btw by 'built-in' I meant 'not a C extension module in a separate DLL': for the import "semantics" of CPython, built-in modules take precedence over any other module from any kind of external file, whereas C extensions can be hidden by other modules earlier on sys.path.) But that's not the problem here. As you noted the difference seems to be whether the struct module was already imported before or not. I now think that the proper hack is to make the 'import struct' local instead of global, so that geninterp won't try to freeze it in. The current situation is very strange, btw, because our struct module is not geninterp'ed, but the array module is - and it includes a geninterp'ed version of most of the struct module via this global import. Can you tell me if it solves the problem if the 'from struct import xxx' is moved into the fromstring() and tostring() methods? A bientot, Armin. From arigo at tunes.org Sat Mar 24 11:49:41 2007 From: arigo at tunes.org (Armin Rigo) Date: Sat, 24 Mar 2007 11:49:41 +0100 Subject: [pypy-dev] Google SoC extension In-Reply-To: <200703241135.42305.jacob@strakt.com> References: <200703241135.42305.jacob@strakt.com> Message-ID: <20070324104941.GA2146@code0.codespeak.net> Hi Jacob, On Sat, Mar 24, 2007 at 11:35:42AM +0100, Jacob Hall??n wrote: > Google has extended the application period for SoC projects to the 26th of > March, so if there is anyone who dropped sending in an application due to > time constraints, you have a second chance. I said that here already :-) but better twice (three times now :-) than zero. A bientot, Armin From holger at merlinux.de Sat Mar 24 12:28:37 2007 From: holger at merlinux.de (holger krekel) Date: Sat, 24 Mar 2007 12:28:37 +0100 Subject: [pypy-dev] new architecture document Message-ID: <20070324112837.GZ4635@solar.trillke> Hi all, i have a first draft of a pypy/doc/new-architecture.txt I've used a new file so that one can easier check back with the old one, and also because architecture.txt is an often visited web link. I tried to implement the following intentions: * separate cleanly between mission/goals and architecture description * keep status information mostly out (particularly of the mission statement and goals) * Talk about PyPy as two things, the translation framework and the Python Interpreter. * clarify terminology, particulary trying to use terms so that it becomes more obvious that PyPy can be used for implementing other dynamic languages, i.e. cannot only be used to implement Python Interpreters (although the latter is a major goal and also helps the translation framework with its architecture) If we'd like to provide status information we should do that in a different section or document, i think. Well, and I am sure it still has lots of room for fixing, improvement and refinement, please hack away and/or tell me what you think :) best, holger -- merlinux GmbH Steinbergstr. 42 31139 Hildesheim http://merlinux.de tel +49 5121 20800 75 (fax 77) From pedronis at openendsystems.com Mon Mar 26 17:53:19 2007 From: pedronis at openendsystems.com (Samuele Pedroni) Date: Mon, 26 Mar 2007 17:53:19 +0200 Subject: [pypy-dev] making the release branch for 1.0: release/1.0.x Message-ID: <4607EC6F.2080308@openendsystems.com> If possible avoid code changes on dist to avoid confusion, unless they are bug fixes that need to be ported to the branch. We should release at some point tomorrow. Thanks, Samuele Pedroni From simon at arrowtheory.com Mon Mar 26 23:30:15 2007 From: simon at arrowtheory.com (Simon Burton) Date: Mon, 26 Mar 2007 14:30:15 -0700 Subject: [pypy-dev] special methods Message-ID: <20070326143015.dc17c8a0.simon@arrowtheory.com> The attached script is an attempt to get __str__ to work in rpython. After annotation, it looks for str(someinstance), rewriting the block to call __str__. Then the modified blocks are fed back into the annotator. It seems to work. (And now i know a lot more about the internals of the translation process...) I had a think about over-riding consider_op_str. It would seem natural for this code to go there, but the annotator is not really set up to handle rewriting. Why do i not need to manually call the flow operation on the __str__ ? (see the flow_str function) It seems the annotator somehow must re-trigger the flowing operation when it finds that __str__ is being called. thanks, Simon. -------------- next part -------------- A non-text attachment was scrubbed... Name: special.py Type: text/x-python Size: 3557 bytes Desc: not available URL: From pedronis at openendsystems.com Tue Mar 27 23:45:42 2007 From: pedronis at openendsystems.com (Samuele Pedroni) Date: Tue, 27 Mar 2007 23:45:42 +0200 Subject: [pypy-dev] PyPy 1.0: JIT compilers for free (and more) Message-ID: <46099086.6080502@openendsystems.com> ========================================== PyPy 1.0: JIT compilers for free and more ========================================== Welcome to the PyPy 1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! Although still not mature enough for general use, PyPy 1.0 materializes for the first time the full extent of our original vision: - A flexible Python interpreter, written in "RPython": - Mostly unaware of threading, memory and lower-level target platform aspects. - Showcasing advanced interpreter features and prototypes. - Passing core CPython regression tests, translatable to C, LLVM and .NET. - An advanced framework to translate such interpreters and programs: - That performs whole type-inference on RPython programs. - Can weave in threading, memory and target platform aspects. - Has low level (C, LLVM) and high level (CLI, Java, JavaScript) backends. - A **Just-In-Time Compiler generator** able to **automatically** enhance the low level versions of our Python interpreter, leading to run-time machine code that runs algorithmic examples at speeds typical of JITs! Previous releases, particularly the 0.99.0 release from February, already highlighted features of our Python implementation and the abilities of our translation approach but the **new JIT generator** clearly marks a major research result and gives weight to our vision that one can generate efficient interpreter implementations, starting from a description in a high level language. We have prepared several entry points to help you get started: * The main entry point for JIT documentation and status: http://codespeak.net/pypy/dist/pypy/doc/jit.html * The main documentation and getting-started PyPy entry point: http://codespeak.net/pypy/dist/pypy/doc/index.html * Our online "play1" demos showcasing various Python interpreters, features (and a new way to program AJAX applications): http://play1.codespeak.net/ * Our detailed and in-depth Reports about various aspects of the project: http://codespeak.net/pypy/dist/pypy/doc/index-report.html In the next few months we are going to discuss the goals and form of the next stage of development - now more than ever depending on your feedback and contributions - and we hope you appreciate PyPy 1.0 as an interesting basis for greater things to come, as much as we do ourselves! have fun, the PyPy release team, Samuele Pedroni, Armin Rigo, Holger Krekel, Michael Hudson, Carl Friedrich Bolz, Antonio Cuni, Anders Chrigstroem, Guido Wesdorp Maciej Fijalkowski, Alexandre Fayolle and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html What is PyPy? ================================ Technically, PyPy is both a Python interpreter implementation and an advanced compiler, or more precisely a framework for implementing dynamic languages and generating virtual machines for them. The framework allows for alternative frontends and for alternative backends, currently C, LLVM and .NET. For our main target "C", we can can "mix in" different garbage collectors and threading models, including micro-threads aka "Stackless". The inherent complexity that arises from this ambitious approach is mostly kept away from the Python interpreter implementation, our main frontend. PyPy is now also a Just-In-Time compiler generator. The translation framework contains the now-integrated JIT generation technology. This depends only on a few hints added to the interpreter source and should be able to cope with the changes to the interpreter and be generally applicable to other interpreters written using the framework. Socially, PyPy is a collaborative effort of many individuals working together in a distributed and sprint-driven way since 2003. PyPy would not have gotten as far as it has without the coding, feedback and general support from numerous people. Formally, many of the current developers were involved in executing an EU contract with the goal of exploring and researching new approaches to language and compiler development and software engineering. This contract's duration is about to end this month (March 2007) and we are working and preparing the according final review which is scheduled for May 2007. For the future, we are in the process of setting up structures to help maintain conceptual integrity of the project and to discuss and deal with funding opportunities related to further PyPy sprinting and developments. See here for results of the discussion so far: http://codespeak.net/pipermail/pypy-dev/2007q1/003577.html 1.0.0 Feature highlights ============================== Here is a summary list of key features included in PyPy 1.0: - The Just-In-Time compiler generator, now capable of generating the first JIT compiler versions of our Python interpreter: http://codespeak.net/pypy/dist/pypy/doc/jit.html - More Python interpreter optimizations (a CALL_METHOD bytecode, a method cache, rope-based strings), now running benchmarks at around half of CPython's speed (without the JIT): http://codespeak.net/pypy/dist/pypy/doc/interpreter-optimizations.html - The Python interpreter can be translated to .NET and enables interactions with the CLR libraries: http://codespeak.net/pypy/dist/pypy/doc/cli-backend.html http://codespeak.net/pypy/dist/pypy/doc/clr-module.html - Aspect Oriented Programming facilities (based on mutating the Abstract Syntax Tree): http://codespeak.net/pypy/dist/pypy/doc/aspect_oriented_programming.html http://codespeak.net/pypy/extradoc/eu-report/D10.1_Aspect_Oriented_Programming_in_PyPy-2007-03-22.pdf - The JavaScript backend has evolved to a point where it can be used to write AJAX web applications with it. This is still an experimental technique, though. For demo applications which also showcase various generated Python and PROLOG interpreters, see: http://play1.codespeak.net/ - Proxying object spaces and features of our Python interpreter: - Tainting: a 270-line proxy object space tracking and boxing sensitive information within an application. - Transparent proxies: allow the customization of both application and builtin objects from application level code. Now featuring an initial support module (tputil.py) for working with transparent proxies. For a detailed description and discussion of high level backends and Python interpreter features, please see our extensive "D12" report: http://codespeak.net/pypy/extradoc/eu-report/D12.1_H-L-Backends_and_Feature_Prototypes-2007-03-22.pdf Funding partners and organisations ===================================================== PyPy development and activities happen as an open source project and with the support of a consortium partially funded by a 28 month European Union IST research grant for the period from December 2004 to March 2007. The full partners of that consortium are: Heinrich-Heine University (Germany), Open End (Sweden) merlinux GmbH (Germany), tismerysoft GmbH (Germany) Logilab Paris (France), DFKI GmbH (Germany) ChangeMaker (Sweden), Impara (Germany) From mauriceling at gmail.com Wed Mar 28 09:52:27 2007 From: mauriceling at gmail.com (Maurice Ling) Date: Wed, 28 Mar 2007 17:52:27 +1000 Subject: [pypy-dev] Finding Carl Friedrich Bolz thesis Message-ID: <460A1EBB.2040805@acm.org> Hi, Does anyone knows where to get Carl Friedrich Bolz's masters thesis? Thanks Cheers maurice From cfbolz at gmx.de Wed Mar 28 10:22:51 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 28 Mar 2007 10:22:51 +0200 Subject: [pypy-dev] Finding Carl Friedrich Bolz thesis In-Reply-To: <460A1EBB.2040805@acm.org> References: <460A1EBB.2040805@acm.org> Message-ID: <460A25DB.9070301@gmx.de> Hi Maurice! Maurice Ling wrote: > Does anyone knows where to get Carl Friedrich Bolz's masters thesis? I guess I should know :-). It's not written (not even started) yet, to be honest. If you want to know more about the Prolog interpreter, you might be interested in the following draft paper: http://codespeak.net/svn/user/cfbolz/hack/prolog/paper/prolog-in-rpython.pdf (it's not really citeable in a scientific sense, though). My bachelor thesis should be finished in a couple of weeks though. Cheers, Carl Friedrich From faassen at startifact.com Wed Mar 28 18:21:14 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 28 Mar 2007 18:21:14 +0200 Subject: [pypy-dev] PyPy 1.0: JIT compilers for free (and more) In-Reply-To: <46099086.6080502@openendsystems.com> References: <46099086.6080502@openendsystems.com> Message-ID: Samuele Pedroni wrote: > ========================================== > PyPy 1.0: JIT compilers for free and more > ========================================== > > Welcome to the PyPy 1.0 release - a milestone integrating the results > of four years of research, engineering, management and sprinting > efforts, concluding the 28 months phase of EU co-funding! Congrats on the 1.0 release, folks! Regards, Martijn From tismer at stackless.com Wed Mar 28 19:54:35 2007 From: tismer at stackless.com (Christian Tismer) Date: Wed, 28 Mar 2007 19:54:35 +0200 Subject: [pypy-dev] special methods In-Reply-To: <20070326143015.dc17c8a0.simon@arrowtheory.com> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> Message-ID: On 26.03.2007, at 23:30, Simon Burton wrote: > > The attached script is an attempt to get __str__ to work in rpython. > After annotation, it looks for str(someinstance), rewriting the > block to call __str__. > Then the modified blocks are fed back into the annotator. > > It seems to work. (And now i know a lot more about the internals of > the translation process...) > > I had a think about over-riding consider_op_str. It would seem > natural for this code > to go there, but the annotator is not really set up to handle > rewriting. This looks good, congrats! I had a different approach in mind which would work without explicitly adding a pass to the annotator, no idea which idea is more practical. What I wanted to do is a tiny patch to flow space that allows to add little plugins for extension. My plugin would intercept things like str(...) during flowing, and rewrite accordingly. This is quite similar to what you did, but does things earlier. I'm not sure about this yet, but I think when doing these things after annotation, then there might be some trouble with annotating certain things, especially when I think of supporting __add__ and friends. It might be necessary to do this expansion earlier to make annotation work. As I'm writing this, I'm now pretty sure that this is true. Conclusion (should think before writing) My idea (and plan) in general is to add an interceptor plugin to flow space (or a subclass, trying to make tiny additions only that don't break anything). Then all special methods are checked for all operations and the code is inserted, like a preprocessor during flowing. cheers - chris From simon at arrowtheory.com Wed Mar 28 20:19:44 2007 From: simon at arrowtheory.com (Simon Burton) Date: Wed, 28 Mar 2007 11:19:44 -0700 Subject: [pypy-dev] special methods In-Reply-To: References: <20070326143015.dc17c8a0.simon@arrowtheory.com> Message-ID: <20070328111944.47c4538f.simon@arrowtheory.com> On Wed, 28 Mar 2007 19:54:35 +0200 Christian Tismer wrote: > > This looks good, congrats! > > I had a different approach in mind which > would work without explicitly adding a pass to the > annotator, no idea which idea is more practical. > > What I wanted to do is a tiny patch to flow space > that allows to add little plugins for extension. > My plugin would intercept things like str(...) > during flowing, and rewrite accordingly. This is quite > similar to what you did, but does things earlier. Yes, this is what Richard suggested also, but don't we need the annotation ? Otherwise we don't know when to intercept str(...). That's why I do the pass after an annotation. Simon. From tismer at stackless.com Wed Mar 28 20:45:27 2007 From: tismer at stackless.com (Christian Tismer) Date: Wed, 28 Mar 2007 20:45:27 +0200 Subject: [pypy-dev] special methods In-Reply-To: <20070328111944.47c4538f.simon@arrowtheory.com> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> <20070328111944.47c4538f.simon@arrowtheory.com> Message-ID: <45380EF9-D8E6-495B-8A2E-D8E4AC040EEC@stackless.com> On 28.03.2007, at 20:19, Simon Burton wrote: > On Wed, 28 Mar 2007 19:54:35 +0200 > Christian Tismer wrote: > >> >> This looks good, congrats! >> >> I had a different approach in mind which >> would work without explicitly adding a pass to the >> annotator, no idea which idea is more practical. >> >> What I wanted to do is a tiny patch to flow space >> that allows to add little plugins for extension. >> My plugin would intercept things like str(...) >> during flowing, and rewrite accordingly. This is quite >> similar to what you did, but does things earlier. > > Yes, this is what Richard suggested also, but don't we > need the annotation ? Otherwise we don't know when to intercept str > (...). > > That's why I do the pass after an annotation. Well, tracking the argument passed to str, we know when we have an instance, and then we do the magic. ciao - chris From tismer at stackless.com Wed Mar 28 22:38:05 2007 From: tismer at stackless.com (Christian Tismer) Date: Wed, 28 Mar 2007 22:38:05 +0200 Subject: [pypy-dev] special methods In-Reply-To: <20070328111944.47c4538f.simon@arrowtheory.com> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> <20070328111944.47c4538f.simon@arrowtheory.com> Message-ID: <3D49028A-66F9-4276-A74E-F5445C0681E6@stackless.com> On 28.03.2007, at 20:19, Simon Burton wrote: > On Wed, 28 Mar 2007 19:54:35 +0200 > Christian Tismer wrote: > >> >> This looks good, congrats! >> >> I had a different approach in mind which >> would work without explicitly adding a pass to the >> annotator, no idea which idea is more practical. >> >> What I wanted to do is a tiny patch to flow space >> that allows to add little plugins for extension. >> My plugin would intercept things like str(...) >> during flowing, and rewrite accordingly. This is quite >> similar to what you did, but does things earlier. > > Yes, this is what Richard suggested also, but don't we > need the annotation ? Otherwise we don't know when to intercept str > (...). Yes you are right. Well maybe not so right. Is there any object that has no __str__ method? If yes, then this is an RPython syntax error. I thing we *always* can call the object's __str__ method if it is RPython code. What needs to be added is a handler routine. It must be registered as a function that specializes on the type of its argument. This way it will be the right __str__ call for every type. ciao -- chris From len-l at telus.net Wed Mar 28 23:37:03 2007 From: len-l at telus.net (Lenard Lindstrom) Date: Wed, 28 Mar 2007 14:37:03 -0700 Subject: [pypy-dev] Finding Carl Friedrich Bolz thesis In-Reply-To: <460A25DB.9070301@gmx.de> References: <460A1EBB.2040805@acm.org> <460A25DB.9070301@gmx.de> Message-ID: <460ADFFF.5080505@telus.net> Carl Friedrich Bolz wrote: > Hi Maurice! > > Maurice Ling wrote: > > Does anyone knows where to get Carl Friedrich Bolz's masters thesis? > > I guess I should know :-). It's not written (not even started) yet, to > be honest. If you want to know more about the Prolog interpreter, you > might be interested in the following draft paper: > > http://codespeak.net/svn/user/cfbolz/hack/prolog/paper/prolog-in-rpython.pdf > > Nice write-up. It is a good introduction to PyPy. Particularly useful is the description of the stackless transform. -- Lenard Lindstrom From cfbolz at gmx.de Wed Mar 28 23:45:21 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 28 Mar 2007 23:45:21 +0200 Subject: [pypy-dev] Finding Carl Friedrich Bolz thesis In-Reply-To: <460ADFFF.5080505@telus.net> References: <460A1EBB.2040805@acm.org> <460A25DB.9070301@gmx.de> <460ADFFF.5080505@telus.net> Message-ID: <460AE1F1.7050105@gmx.de> Hi Lenard, Lenard Lindstrom wrote: >> http://codespeak.net/svn/user/cfbolz/hack/prolog/paper/prolog-in-rpython.pdf > > Nice write-up. It is a good introduction to PyPy. Particularly useful is > the description of the stackless transform. > Hey, thanks for the feedback (although it is not meant to be an introduction to PyPy at all, but it contains one :-) ). I am generally interested in feedback on this thing. Cheers, Carl Friedrich From tismer at stackless.com Thu Mar 29 01:14:03 2007 From: tismer at stackless.com (Christian Tismer) Date: Thu, 29 Mar 2007 01:14:03 +0200 Subject: [pypy-dev] special methods In-Reply-To: <3D49028A-66F9-4276-A74E-F5445C0681E6@stackless.com> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> <20070328111944.47c4538f.simon@arrowtheory.com> <3D49028A-66F9-4276-A74E-F5445C0681E6@stackless.com> Message-ID: <1A3C9A6E-750A-4620-A3A3-44F174E58CFC@stackless.com> On 28.03.2007, at 22:38, Christian Tismer wrote: > > I thing we *always* can call the object's __str__ method if it > is RPython code. > What needs to be added is a handler routine. It must be registered > as a function that specializes on the type of its argument. This > way it will be the right __str__ call for every type. > > ciao -- chris > correction: I think some special casing is necessary because we don't use the __str__ lookup for builtin types, ASFAIK. The rest probably holds. Maybe I give it a whirl, tomorrow. g'night - chris From arigo at tunes.org Thu Mar 29 14:13:36 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 29 Mar 2007 14:13:36 +0200 Subject: [pypy-dev] pypy/dist => pypy/trunk, re Message-ID: <20070329121335.GA2597@code0.codespeak.net> Hi all, So, time to move the development of PyPy to http://codespeak.net/svn/pypy/trunk and keep http://codespeak.net/svn/pypy/dist for the latest known-stable copy of the trunk? Armin From holger at merlinux.de Thu Mar 29 14:23:00 2007 From: holger at merlinux.de (holger krekel) Date: Thu, 29 Mar 2007 14:23:00 +0200 Subject: [pypy-dev] pypy/dist => pypy/trunk, re In-Reply-To: <20070329121335.GA2597@code0.codespeak.net> References: <20070329121335.GA2597@code0.codespeak.net> Message-ID: <20070329122300.GG16818@solar.trillke> Hi Armin, On Thu, Mar 29, 2007 at 14:13 +0200, Armin Rigo wrote: > So, time to move the development of PyPy to > http://codespeak.net/svn/pypy/trunk and keep > http://codespeak.net/svn/pypy/dist for the > latest known-stable copy of the trunk? yes - it was discussed yesterday on #pypy already a bit. There is the issue that the website/news come from dist but let this not prevent us from just doing the copy now. I also guess that nightly testing should shift to test 'trunk' regularly. holger From cfbolz at gmx.de Thu Mar 29 14:27:24 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 29 Mar 2007 14:27:24 +0200 Subject: [pypy-dev] pypy/dist => pypy/trunk, re In-Reply-To: <20070329122300.GG16818@solar.trillke> References: <20070329121335.GA2597@code0.codespeak.net> <20070329122300.GG16818@solar.trillke> Message-ID: <460BB0AC.3050903@gmx.de> holger krekel wrote: > On Thu, Mar 29, 2007 at 14:13 +0200, Armin Rigo wrote: >> So, time to move the development of PyPy to >> http://codespeak.net/svn/pypy/trunk and keep >> http://codespeak.net/svn/pypy/dist for the >> latest known-stable copy of the trunk? > > yes - it was discussed yesterday on #pypy already > a bit. There is the issue that the website/news > come from dist but let this not prevent us from > just doing the copy now. I also guess that nightly > testing should shift to test 'trunk' regularly. We should be careful to protect pypy-dist for accidental checkins, because otherwise we will certainly have accidents. Cheers, Carl Friedrich From arigo at tunes.org Thu Mar 29 18:00:47 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 29 Mar 2007 18:00:47 +0200 Subject: [pypy-dev] special methods In-Reply-To: <20070326143015.dc17c8a0.simon@arrowtheory.com> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> Message-ID: <20070329160047.GA5567@code0.codespeak.net> Hi Simon, On Mon, Mar 26, 2007 at 02:30:15PM -0700, Simon Burton wrote: > The attached script is an attempt to get __str__ to work in rpython. > After annotation, it looks for str(someinstance), rewriting the block to call __str__. > Then the modified blocks are fed back into the annotator. The "official" approach is quite different. It would involve a consider_op_str() on SomeInstance, as you also thought. It is in some sense harder, but more robust - I certainly wouldn't be happy to check in code in PyPy that adds a rewriting pass in the middle of annotation... For example, your approach only supports direct 'str(x)' calls, which is somehow the easy case - because they can be manually replaced by 'x.__str__()' in the source code anyway - but not indirect cases like 'str([x, y, z])' where the x, y and z have a custom __str__() method. To do this properly you need a consider_op_str() using bookkeeper.emulate_pbc_call(), a lot of patience understanding what rpbc.py is doing, and probably a call to hlinvoke() in the ll_str() of rclass.py... and then the same for the oo type system, if you want to be complete. Argh. All in all I'll stick to the point of view that adding support for special methods in RPython is a very dangerous direction to go: where do you stop? Is __add__() RPython? Is the full logic of __add__() versus __radd__() RPython? A bientot, Armin. From arigo at tunes.org Thu Mar 29 18:08:46 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 29 Mar 2007 18:08:46 +0200 Subject: [pypy-dev] pypy/dist => pypy/trunk, re In-Reply-To: <460BB0AC.3050903@gmx.de> References: <20070329121335.GA2597@code0.codespeak.net> <20070329122300.GG16818@solar.trillke> <460BB0AC.3050903@gmx.de> Message-ID: <20070329160846.GA8848@code0.codespeak.net> Hi Carl, On Thu, Mar 29, 2007 at 02:27:24PM +0200, Carl Friedrich Bolz wrote: > We should be careful to protect pypy-dist for accidental checkins, > because otherwise we will certainly have accidents. Is it possible to have a check-in hook that allows deletions and additions (possibly with history) within pypy/dist, but not changes? In this way we can only 'svn rm && svn cp' things to pypy/dist from some other place like pypy/trunk. Armin From tismer at stackless.com Thu Mar 29 19:16:36 2007 From: tismer at stackless.com (Christian Tismer) Date: Thu, 29 Mar 2007 19:16:36 +0200 Subject: [pypy-dev] special methods In-Reply-To: <20070329160047.GA5567@code0.codespeak.net> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> <20070329160047.GA5567@code0.codespeak.net> Message-ID: <6594B465-2525-4476-9B19-04051840CE11@stackless.com> On 29.03.2007, at 18:00, Armin Rigo wrote: > Hi Simon, > > On Mon, Mar 26, 2007 at 02:30:15PM -0700, Simon Burton wrote: >> The attached script is an attempt to get __str__ to work in rpython. >> After annotation, it looks for str(someinstance), rewriting the >> block to call __str__. >> Then the modified blocks are fed back into the annotator. > > The "official" approach is quite different. It would involve a > consider_op_str() on SomeInstance, as you also thought. It is in some > sense harder, but more robust Agreed, that's the hard but complete way. > For example, your approach only supports direct 'str(x)' > calls, which is somehow the easy case - because they can be manually > replaced by 'x.__str__()' in the source code anyway - but not indirect > cases like 'str([x, y, z])' where the x, y and z have a custom > __str__() > method. I think the idea was exactly to only support the simple cases, where you can manually use x.__str__ I would certainly not go the path to add full support for these things to the annotator. Instead, I would just expand things in a way to support the simple cases, and not as an addition to PyPy's core, but as an add-on. > All in all I'll stick to the point of view that adding support for > special methods in RPython is a very dangerous direction to go: > where do > you stop? Is __add__() RPython? Is the full logic of __add__() > versus > __radd__() RPython? Simple, just a little nicer. I would not support __radd__ at all, but just __add__ and always enforce the same types. This idea is really not about a huge extension, but some simple optional additions that let code look a little more like Python. Shortly put: anything that needs to seriously change the annotator should not be considered. Some syntactic sugar does not hurt. You think even that makes no sense, right? At least it should not hurt... ciao - chris From simon at arrowtheory.com Thu Mar 29 19:18:06 2007 From: simon at arrowtheory.com (Simon Burton) Date: Thu, 29 Mar 2007 10:18:06 -0700 Subject: [pypy-dev] special methods In-Reply-To: <20070329160047.GA5567@code0.codespeak.net> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> <20070329160047.GA5567@code0.codespeak.net> Message-ID: <20070329101806.1f771720.simon@arrowtheory.com> On Thu, 29 Mar 2007 18:00:47 +0200 Armin Rigo wrote: > Hi Simon, > > On Mon, Mar 26, 2007 at 02:30:15PM -0700, Simon Burton wrote: > > The attached script is an attempt to get __str__ to work in rpython. > > After annotation, it looks for str(someinstance), rewriting the block to call __str__. > > Then the modified blocks are fed back into the annotator. > > The "official" approach is quite different. It would involve a > consider_op_str() on SomeInstance, as you also thought. It is in some > sense harder, but more robust - I certainly wouldn't be happy to check > in code in PyPy that adds a rewriting pass in the middle of > annotation... Yes, it does seem to lack delicacy. > For example, your approach only supports direct 'str(x)' > calls, which is somehow the easy case - because they can be manually > replaced by 'x.__str__()' in the source code anyway - but not indirect > cases like 'str([x, y, z])' where the x, y and z have a custom __str__() > method. Yes, i see. String comprehensions don't work either. > To do this properly you need a consider_op_str() using > bookkeeper.emulate_pbc_call(), a lot of patience understanding what > rpbc.py is doing, and probably a call to hlinvoke() in the ll_str() of > rclass.py... and then the same for the oo type system, if you want to > be complete. Argh. Well, thanks for the keywords. I already understand the codebase much better from hacking around with this, so it's not entirely a waste of time if you end up saying "nah that sucks". :) > > All in all I'll stick to the point of view that adding support for > special methods in RPython is a very dangerous direction to go: where do > you stop? Is __add__() RPython? Is the full logic of __add__() versus > __radd__() RPython? What is your concern here ? Does it screw up the JIT, or some other aspect I am missing ? I guess, the full __add__ / __radd__ semantics is a little tricky, and implementing it statically would likely (at least if i tried) produce the kind of brutal code that is perhaps difficult to maintain.. However, just having __str__ support would be really handy. bye for now, Simon. From arigo at tunes.org Thu Mar 29 19:23:37 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 29 Mar 2007 19:23:37 +0200 Subject: [pypy-dev] special methods In-Reply-To: <6594B465-2525-4476-9B19-04051840CE11@stackless.com> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> <20070329160047.GA5567@code0.codespeak.net> <6594B465-2525-4476-9B19-04051840CE11@stackless.com> Message-ID: <20070329172337.GA23453@code0.codespeak.net> Hi Christian, On Thu, Mar 29, 2007 at 07:16:36PM +0200, Christian Tismer wrote: > Shortly put: anything that needs to seriously change the annotator > should not be considered. Some syntactic sugar does not hurt. > > You think even that makes no sense, right? At least it should not > hurt... I think it hurts because it's obscure to describe: "you can write str(x) and x.__str__() will be called, but if you write str([x]) then x.__str__() will not be called"... I'm always open to the possibility that there are use cases where such a hack would be enough, though. A bientot, Armin From micahel at gmail.com Thu Mar 29 19:26:37 2007 From: micahel at gmail.com (Michael Hudson) Date: Thu, 29 Mar 2007 18:26:37 +0100 Subject: [pypy-dev] special methods In-Reply-To: <20070329172337.GA23453@code0.codespeak.net> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> <20070329160047.GA5567@code0.codespeak.net> <6594B465-2525-4476-9B19-04051840CE11@stackless.com> <20070329172337.GA23453@code0.codespeak.net> Message-ID: On 29/03/07, Armin Rigo wrote: > Hi Christian, > > On Thu, Mar 29, 2007 at 07:16:36PM +0200, Christian Tismer wrote: > > Shortly put: anything that needs to seriously change the annotator > > should not be considered. Some syntactic sugar does not hurt. > > > > You think even that makes no sense, right? At least it should not > > hurt... > > I think it hurts because it's obscure to describe: "you can write str(x) > and x.__str__() will be called, but if you write str([x]) then > x.__str__() will not be called"... Well, if you write str([x]) then x.__str__() will certainly not be called :-) Cheers, mwh (str([x]) will call x.__repr__()) (sorry) From arigo at tunes.org Thu Mar 29 19:35:35 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 29 Mar 2007 19:35:35 +0200 Subject: [pypy-dev] special methods In-Reply-To: <20070329101806.1f771720.simon@arrowtheory.com> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> <20070329160047.GA5567@code0.codespeak.net> <20070329101806.1f771720.simon@arrowtheory.com> Message-ID: <20070329173534.GB23453@code0.codespeak.net> Hi Simon, On Thu, Mar 29, 2007 at 10:18:06AM -0700, Simon Burton wrote: > What is your concern here ? Does it screw up the JIT, or some other aspect > I am missing ? No, just the obscurity of these methods: the full Python __add__/__radd__ semantics are more than a little tricky. They are impossible to implement statically, because in order to know which one to call first you need to know the two exact subclasses of the arguments, which you only know at run-time. The RPython approach so far has at least a clear message: no special methods, apart from __init__() and __del__(). I'm not against adding a few of them, to be honest; e.g. __getitem__() would be my favorite. But then they should be fully implemented. For example, I just realized that without the full rtyper solution, your patch can work for str(x) but not for '%s' % (x,), which looks rather inconsistent. A bientot, Armin. From arigo at tunes.org Thu Mar 29 19:39:17 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 29 Mar 2007 19:39:17 +0200 Subject: [pypy-dev] special methods In-Reply-To: References: <20070326143015.dc17c8a0.simon@arrowtheory.com> <20070329160047.GA5567@code0.codespeak.net> <6594B465-2525-4476-9B19-04051840CE11@stackless.com> <20070329172337.GA23453@code0.codespeak.net> Message-ID: <20070329173916.GC23453@code0.codespeak.net> Hi Michael, On Thu, Mar 29, 2007 at 06:26:37PM +0100, Michael Hudson wrote: > Well, if you write str([x]) then x.__str__() will certainly not be called :-) Ah right. Then '%s' % (x,) . So far the rtyper has no notion of the "repr" of an object being something else than its "str", so str([x]) is really the same as '[' + str(x) + ']'. All this stuff would need clean-ups before we go more in the direction of fully supporting custom __str__() methods... A bientot, Armin. From tismer at stackless.com Fri Mar 30 00:01:39 2007 From: tismer at stackless.com (Christian Tismer) Date: Fri, 30 Mar 2007 00:01:39 +0200 Subject: [pypy-dev] special methods In-Reply-To: <20070329172337.GA23453@code0.codespeak.net> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> <20070329160047.GA5567@code0.codespeak.net> <6594B465-2525-4476-9B19-04051840CE11@stackless.com> <20070329172337.GA23453@code0.codespeak.net> Message-ID: <4C4B2B53-7301-4201-A449-7BECF48A4304@stackless.com> On 29.03.2007, at 19:23, Armin Rigo wrote: > Hi Christian, > > On Thu, Mar 29, 2007 at 07:16:36PM +0200, Christian Tismer wrote: >> Shortly put: anything that needs to seriously change the annotator >> should not be considered. Some syntactic sugar does not hurt. >> >> You think even that makes no sense, right? At least it should not >> hurt... > > I think it hurts because it's obscure to describe: "you can write > str(x) > and x.__str__() will be called, but if you write str([x]) then > x.__str__() will not be called"... I agree that we disagree. Very much. Having no solution instead of a partial one for 80% of the common cases is mathematically correct, but nothing to tell users. And I honor and do appreciate them. not yet branching - ly y'rs -- chris From tismer at stackless.com Fri Mar 30 00:14:59 2007 From: tismer at stackless.com (Christian Tismer) Date: Fri, 30 Mar 2007 00:14:59 +0200 Subject: [pypy-dev] special methods In-Reply-To: <20070329173534.GB23453@code0.codespeak.net> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> <20070329160047.GA5567@code0.codespeak.net> <20070329101806.1f771720.simon@arrowtheory.com> <20070329173534.GB23453@code0.codespeak.net> Message-ID: <3A950EFA-9BD8-411D-BA94-222EB48BB9DC@stackless.com> On 29.03.2007, at 19:35, Armin Rigo wrote: > Hi Simon, > > On Thu, Mar 29, 2007 at 10:18:06AM -0700, Simon Burton wrote: >> What is your concern here ? Does it screw up the JIT, or some >> other aspect >> I am missing ? > > No, just the obscurity of these methods: the full Python > __add__/__radd__ semantics are more than a little tricky. And no RPython programmer needs them. > The RPython approach so far has at least a clear message: no special > methods, apart from __init__() and __del__(). I'm not against > adding a > few of them, to be honest; e.g. __getitem__() would be my > favorite. But > then they should be fully implemented. For example, I just realized > that without the full rtyper solution, your patch can work for str(x) > but not for '%s' % (x,), which looks rather inconsistent. And for that reason, you would drop the whole thing, waiting for a complete solution? This is not realistic, it will probably happen, anyway. why do I try this, again -- chris From simon at arrowtheory.com Fri Mar 30 03:25:55 2007 From: simon at arrowtheory.com (Simon Burton) Date: Thu, 29 Mar 2007 18:25:55 -0700 Subject: [pypy-dev] special methods In-Reply-To: <3A950EFA-9BD8-411D-BA94-222EB48BB9DC@stackless.com> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> <20070329160047.GA5567@code0.codespeak.net> <20070329101806.1f771720.simon@arrowtheory.com> <20070329173534.GB23453@code0.codespeak.net> <3A950EFA-9BD8-411D-BA94-222EB48BB9DC@stackless.com> Message-ID: <20070329182555.654a8224.simon@arrowtheory.com> On Fri, 30 Mar 2007 00:14:59 +0200 Christian Tismer wrote: > > On 29.03.2007, at 19:35, Armin Rigo wrote: > > > Hi Simon, > > > > On Thu, Mar 29, 2007 at 10:18:06AM -0700, Simon Burton wrote: > >> What is your concern here ? Does it screw up the JIT, or some > >> other aspect > >> I am missing ? > > > > No, just the obscurity of these methods: the full Python > > __add__/__radd__ semantics are more than a little tricky. > > And no RPython programmer needs them. Well... it's possible to extend the rtyper to do this. Without even touching the pypy codebase. This is what we did with pypy.rpython.numpy . Simon. From simon at arrowtheory.com Fri Mar 30 03:37:44 2007 From: simon at arrowtheory.com (Simon Burton) Date: Thu, 29 Mar 2007 18:37:44 -0700 Subject: [pypy-dev] special methods In-Reply-To: <20070329160047.GA5567@code0.codespeak.net> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> <20070329160047.GA5567@code0.codespeak.net> Message-ID: <20070329183744.d6882bbf.simon@arrowtheory.com> On Thu, 29 Mar 2007 18:00:47 +0200 Armin Rigo wrote: > To do this properly you need a consider_op_str() using > bookkeeper.emulate_pbc_call(), a lot of patience understanding what > rpbc.py is doing, and probably a call to hlinvoke() in the ll_str() of > rclass.py... I am very curious about this suggestively named hlinvoke.. It seems that if we can get the ll_str of rclass to call the low-level-ized version of the __str__ method then we are done. Is this the idea behind hlinvoke ? (... looking now at test_rpbc...) Simon. From lkcl at lkcl.net Fri Mar 30 15:07:16 2007 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Fri, 30 Mar 2007 13:07:16 +0000 (UTC) Subject: [pypy-dev] congratulations on the 1.0 py-py release! Message-ID: <1313377165lkcl@lkcl.net> dear pypy developers, congratulations on reaching the 1.0 release of py-py: the project you are working on is very significant and is also a major achievement that you should be proud of. [today, py-py: tomorrow, the rest of the world ... :) ] i was stunned and delighted to see the just-in-time compiler capabilities and also the mention of "virtual machines". i therefore wanted to make you aware of a project that you should consider absorbing and adapting: a Python "Microkernel" Operating System, called "cleese": http://www.jtauber.com/cleese the developers of cleese managed a proof-of-concept back in 2003 and would love to continue work on it. the concept of an entirely python-esque environment, including a development tool (e.g. http://leo.sf.net) right from the bare metal up just... absolutely tickles me pink :) and - i note on the documentation: the idea of using py-py for embedded systems. how much more "embedded" can you get if the entire OS is cut and out-the-window! good luck, l. From shafranov at gmail.com Fri Mar 30 16:48:39 2007 From: shafranov at gmail.com (Alexander Shafranov) Date: Fri, 30 Mar 2007 18:48:39 +0400 Subject: [pypy-dev] socket and select on windows Message-ID: <74fc943d0703300748u4a56f422ld8422e80d77883c@mail.gmail.com> Hi! It seems, that in 1.0 release, socket and and select modules aren't implemented for windows. Does someone work on implementation? I could give a try.. Alexander. -------------- next part -------------- An HTML attachment was scrubbed... URL: From amauryfa at gmail.com Fri Mar 30 17:48:17 2007 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Fri, 30 Mar 2007 17:48:17 +0200 Subject: [pypy-dev] socket and select on windows In-Reply-To: <74fc943d0703300748u4a56f422ld8422e80d77883c@mail.gmail.com> References: <74fc943d0703300748u4a56f422ld8422e80d77883c@mail.gmail.com> Message-ID: Hello, Alexander Shafranov wrote: > It seems, that in 1.0 release, socket and and select modules aren't > implemented for windows. > Does someone work on implementation? I did a lot of work in this direction several months ago. I went as far as having the module annotated, and most of the tests running. The tough part was the reimplementation of getaddrinfo() and getnameinfo(), which do not exists on many windows platforms. I think it even translates into C. It's been a while, but I think I stopped because of some rough aspects of the rctypes interface (for example: casting pointers looses track of reference counting). Anyway, I'll be happy to share my changes, to have another eye on the stuff... If you agree, I will try to resurrect my work and send you a patch later tonight. > I could give a try.. You're welcome... -- Amaury Forgeot d'Arc From arigo at tunes.org Fri Mar 30 18:25:35 2007 From: arigo at tunes.org (Armin Rigo) Date: Fri, 30 Mar 2007 18:25:35 +0200 Subject: [pypy-dev] special methods In-Reply-To: <20070329183744.d6882bbf.simon@arrowtheory.com> References: <20070326143015.dc17c8a0.simon@arrowtheory.com> <20070329160047.GA5567@code0.codespeak.net> <20070329183744.d6882bbf.simon@arrowtheory.com> Message-ID: <20070330162535.GA25418@code0.codespeak.net> Hi Simon, On Thu, Mar 29, 2007 at 06:37:44PM -0700, Simon Burton wrote: > I am very curious about this suggestively named hlinvoke.. It seems that > if we can get the ll_str of rclass to call the low-level-ized version of the __str__ > method then we are done. Is this the idea behind hlinvoke ? (... looking now at test_rpbc...) Yes. It's used to call back RPython functions from low-level helpers, where it's normally not possible. It's used by objectmodel.rdict() to call the RPython functions that implement the custom key equality and hash. See lltypesystem/rdict.py for an example... You don't get method dispatch for free - you have to call an RPython function, not a method, so you'd need to manually add in the vtable of the class a field for the low-level function pointer to the RPython function. Note also that hlinvoke() is not yet implemented for the ootypesystem. A bientot, Armin. From shafranov at gmail.com Fri Mar 30 18:27:32 2007 From: shafranov at gmail.com (Alexander Shafranov) Date: Fri, 30 Mar 2007 20:27:32 +0400 Subject: [pypy-dev] socket and select on windows In-Reply-To: References: <74fc943d0703300748u4a56f422ld8422e80d77883c@mail.gmail.com> Message-ID: <74fc943d0703300927u5ca7528ey85a39737f13dbe94@mail.gmail.com> Ok, I'll wait for your patch and will try to help. Alexander. On 3/30/07, Amaury Forgeot d'Arc wrote: > > Hello, > > Alexander Shafranov wrote: > > It seems, that in 1.0 release, socket and and select modules aren't > > implemented for windows. > > Does someone work on implementation? > > I did a lot of work in this direction several months ago. > I went as far as having the module annotated, and most of the tests > running. > The tough part was the reimplementation of getaddrinfo() and > getnameinfo(), which do not exists on many windows platforms. > I think it even translates into C. > > It's been a while, but I think I stopped because of some rough aspects > of the rctypes interface (for example: casting pointers looses track > of reference counting). > > Anyway, I'll be happy to share my changes, to have another eye on the > stuff... > If you agree, I will try to resurrect my work and send you a patch > later tonight. > > > I could give a try.. > You're welcome... > > -- > Amaury Forgeot d'Arc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anto.cuni at gmail.com Fri Mar 30 21:14:47 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 30 Mar 2007 21:14:47 +0200 Subject: [pypy-dev] Towards pypy-jvm Message-ID: <460D61A7.2060908@gmail.com> Hi Niko, hi all! I've read in the IRC logs that there has been a bit of discussion about what genjvm still lacks in order to translate pypy. Some weeks ago I also tried to translate pypy-jvm; it seems that the two most important missing features are r_dict and weakrefs. The good news is that with some hacks it's possible to get a pypy version that doesn't make use of r_dict or weakrefs: have a look at this IRC log: http://tismerysoft.de/pypy/irc-logs/pypy/%23pypy.log.20070307 The bad news is that even with those changes, jvm crashed because of a failed assertion, then I gave up. I've no clue what it's going wrong, but maybe it's not something terribly wrong to fix. I hope that this infos can help you in some way :-). ciao Anto From nytrokiss at gmail.com Fri Mar 30 21:48:17 2007 From: nytrokiss at gmail.com (James Matthews) Date: Fri, 30 Mar 2007 12:48:17 -0700 Subject: [pypy-dev] congratulations on the 1.0 py-py release! In-Reply-To: <1313377165lkcl@lkcl.net> References: <1313377165lkcl@lkcl.net> Message-ID: <8a6b8e350703301248x31304f62q721b3bf3a71f06a@mail.gmail.com> Happy to hear i cannot wait to test it out! On 3/30/07, Luke Kenneth Casson Leighton wrote: > > dear pypy developers, > > congratulations on reaching the 1.0 release of py-py: the project > you are working on is very significant and is also a major achievement > that you should be proud of. > > [today, py-py: tomorrow, the rest of the world ... :) ] > > i was stunned and delighted to see the just-in-time compiler > capabilities and also the mention of "virtual machines". > > i therefore wanted to make you aware of a project that you should > consider absorbing and adapting: a Python "Microkernel" Operating > System, called "cleese": http://www.jtauber.com/cleese > > the developers of cleese managed a proof-of-concept back in 2003 and > would love to continue work on it. > > the concept of an entirely python-esque environment, including a > development tool (e.g. http://leo.sf.net) right from the bare metal > up just... absolutely tickles me pink :) > > and - i note on the documentation: the idea of using py-py for > embedded systems. how much more "embedded" can you get if the entire > OS is cut and out-the-window! > > good luck, > > l. > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -- http://www.goldwatches.com/watches.asp?Brand=39 http://www.wazoozle.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From niko at alum.mit.edu Fri Mar 30 23:40:12 2007 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 30 Mar 2007 23:40:12 +0200 Subject: [pypy-dev] Towards pypy-jvm In-Reply-To: <460D61A7.2060908@gmail.com> References: <460D61A7.2060908@gmail.com> Message-ID: > The bad news is that even with those changes, jvm crashed because > of a failed assertion, then I gave up. I've no clue what it's going > wrong, but maybe it's not something terribly wrong to fix. > > I hope that this infos can help you in some way :-). I'll look into it, thanks! I think implementing rdicts and weakrefs would probably be pretty easy, so maybe I will just do that. I've been itching to get back to PyPy-hacking anyhow! Niko From anto.cuni at gmail.com Sat Mar 31 00:38:38 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Sat, 31 Mar 2007 00:38:38 +0200 Subject: [pypy-dev] Towards pypy-jvm In-Reply-To: References: <460D61A7.2060908@gmail.com> Message-ID: <460D916E.8070008@gmail.com> Niko Matsakis wrote: >> The bad news is that even with those changes, jvm crashed because of a >> failed assertion, then I gave up. I've no clue what it's going wrong, >> but maybe it's not something terribly wrong to fix. >> >> I hope that this infos can help you in some way :-). > > I'll look into it, thanks! I think implementing rdicts and weakrefs > would probably be pretty easy, so maybe I will just do that. I've been > itching to get back to PyPy-hacking anyhow! Ok, good to know :-). The last time I checked what java's Hasttable offers and I saw you can't pass to it custom hashing and equality functions, but maybe there is a simple way to do it that I don't know. Btw, good work on genjvm :-). ciao Anto From lac at openend.se Sat Mar 31 11:35:16 2007 From: lac at openend.se (Laura Creighton) Date: Sat, 31 Mar 2007 11:35:16 +0200 Subject: [pypy-dev] Paul Fernhout wants edit-and-continue in python Message-ID: <200703310935.l2V9ZG9m002435@theraft.openend.se> How hard would it be to give it to him? Laura ------- Forwarded Message Return-Path: edu-sig-bounces at python.org Delivery-Date: Sat Mar 31 06:18:19 2007 From: "Paul D. Fernhout" User-Agent: Icedove 1.5.0.9 (X11/20061220) MIME-Version: 1.0 To: "edu-sig at python.org" To clarify, on PataPata, at the moment I am using Jython to prototype a version of a Smalltalk-like system and syntax for the Java (JVM) platform. I am doing this in part to have a system which supports "edit and continue", and also in part just because it is fun. I think Java is a terrible language for a human to use because it is too verbose (among other reasons). Still, after ten years, the JVM is a not-that-terrible platform which has the advantage of hype and installed base and can supply about 1/2 C speeds in a portable way. So Java (or JVM byte codes) isn't that awful as a portable machine language -- and it involves less work for a single programmer to maintain a complex application across multiple platforms than for C. Personally, I like Swing, but then, I come from the world (VisualWorks) where most core Swing designers come from. :-) There is no question in my mind that Jython is a much saner way to write code for the JVM than Java in most situations. And Jython is probably a better choice for most JVM things than most of these JVM languages: http://www.robert-tolksdorf.de/vmlanguages.html (unless you already know one of the other languages or have a problem which has a neatly packaged solution in one of those systems). Still, for anything other than writing arbitrary portable fast code (1/2 C) or code that interfaces easily with Java libraries, Python by itself is probably a more reliable cross platform solution than JVM approaches in many situations. In any case, there is still a lot of Python in my own noodling around, at least for now. :-) And also, while I learned a lot about prototypes and their strengths and weaknesses when writing Python code for PataPata to emulate prototypes in Python, if anything, I developed a better respect for how much you could do quirky things in plain old Python when you wanted to or needed to, thus minimizing some of the need to use prototypes for everything. Still, there are always tempting new projects like "Factor" http://factorcode.org/ which uses FORTH ideals to do amazing things (like with OpenGL) in fairly small footprints and with minimal external dependencies and with fairly few core developer-years of work (though the developers are unusually bright. :-) From the web page: "Factor has an optimizing native compiler, automatic memory management with a generational garbage collector, a powerful collections library, and various advanced language features such as higher-order programming, continuations, and extensible syntax. Factor development is done in an interactive environment with a graphical user interface. There is a single-stepping debugger which can travel backwards in time, and extensive documentation. An easy to use C library interface allows one to call C libraries without writing any glue code in C. Bindings for OpenGL, FreeType, X11, Cocoa, and Windows APIs are provided." (I can hope AK's FONC http://vpri.org/mailman/listinfo/fonc http://www.viewpointsresearch.org/html/writings.htm in a few years will produce something even beyond that, if it succeeds). In any case, I see the JVM, Python, C, FONC, Squeak, and Factor as all possible substrates for running a higher level system on top of. A well designed self-reflective system really should be able to run on all of them with appropriate back ends and interface layers. It just requires a higher level of abstraction than, for example, how PataPata already runs on both top of Python/TK and Jython/Swing. And also a willingness to sacrifice some performance in some situations. :-) Right now, on a 3Ghz AMD64, this new VM in Jython is doing about 25000 message sends a second (and about 30K in Python). This is about 1/1000 the speed of C, but I think when the system reflects on itself and can generate pure Java, I am hoping to have it at Python-ish and Squeak-ish speeds of about 1/20 to 1/100 C overall for loops and calls and sends. But given how fast computers continue to get, I'm not too concerned about performance. Even if it remained at 1/1000 C speed, in ten years, that would mean it would run about the speed of C on today's computers - -- and there are plenty of interesting applications one can run on today's computers. I also have Jython GUI that lets me single step through hand-compiled code. (None of this has been checked in so SourceForge yet though.) From an educational point of view, looking at such a system and changing it, even if the system was never used for any other activity) could help Python-literate people understand more about message passing and VM construction and (abstractly) how a CPU works. I'm certainly learning something from it. :-) I certainly remain completely pro-Python for commercial development. And I remain pro-Python as the language of choice to teach most kids with at the appropriate age. It's a great language. And a great community. Python generally plays nice with other systems (including C and Java) and looks a lot like what most programmers already know, making it an easy "upsell" as a dynamic language compared to, say, Smalltalk or Lisp. http://dictionary.reference.com/browse/upsell And Python has really hit its stride now seeing all the great job postings for people knowledgeable in it to do very interesting work. If I was teaching a kid programming right now, I would recommend Python as the best all-around choice without hesitation, especially as a first language. (Although, for kids who wanted to be computer or electronic engineers, I might still suggest Forth as a best first language, so they learn good factoring habits early on, plus a basic familiarity with low level issues in an interactive environment. But even then, Python should come next, for its practicality. :-) Still, as much as I like Python, I think lack of "edit and continue" in Python is a big issue -- both for advanced users and beginners, especially given how Smalltalk, Ruby, C#, Visual Basic, and other languages have it. Having to rerun Python applications due to typos or to insert print statements is the biggest thorn in my paw when I use Python. As I have said, I think lack of "Edit and continue" probably reduces Python developer productivity to about 1/3 of what it could be (although you can get part of that back by hacking your application to use a reloader window like the one I posted on the Jython user list). Still, even with this glaring handicap relative to other dynamic and static languages like Ruby or VB or C#, Python still usually comes out on top IMHO over many other languages for most common programming tasks for various reasons based on its strengths. Which just shows what an awesome language Python is, still winning the race while putt-putting on two cylinders instead of vrooming by on the six cylinders everyone else has out of the box. Some kinda magic going on there. :-) I started to look into seeing how "edit and continue" could be addressed with Python, but, especially given Guido saying it was "impossible", and looking in the email archives of various lists to see it has been brought up multiple times on comp.lang.python and such and shot down, http://www.google.com/search?hl=en&q=python+%22edit+and+continue%22 it seems it might be easier to build something from scratch which supports it (on top of Java using those libraries) than to try (and fail) to improve Python. I also have always preferred Smalltalk keyword syntax over Python, as it is easily extensible (and easily readable (to me), so there are always forces tugging me towards creating alternatives, which people here will rightfully say stand a snowball's chance of seeing widespread adoption. The bottom line on "Edit and continue" is that making it work properly likely requires deep thinking and understanding about the entire Python runtime system (as well as getting pickup of your changes by various IDE developers), and Python as a system has moved so far along the development curve that coming from a standing start up to an understanding of Python's current internals necessarily beyond that of Guido (who says it is impossible :-) is likely a long hard effort. During which time Python will keep moving. It's not exactly a best first choice of Python internals hacking. Guido is obviously the best person to tackle it, I think; just need to figure out how to motivate him to care about it instead of other cool and important things. Now if I could just get hold of some "first herring" perhaps? :-) http://travel.independent.co.uk/europe/article548699.ece http://en.wikipedia.org/wiki/Herring Still, if a student in the "Summer of Code" (or others) got interested in the project, then I would revisit my personal support for it. I'm sure it's doable -- just a matter of how much of Python's (or Jython's) internals would need to be strewn about the floor before it worked (and an IDE's too, I guess) -- and then how many of those scattered pieces could be fit back together under the hood. The shortest path is modifying an existing IDE (PyDev? IDLE?) and an existing Python or Jython VM so it breaks on any exception before it is thrown (avoiding the likely most "impossible" part of restarting a thrown Python exception), and then providing ways in the IDE where you can specify which modules and exceptions are debugged and which are thrown, and then having an ability to selectively modify a function and reenter it with the same preserved entry state (sort of like a continuation). And of course, you need to then save the change back to the source file. We already have the reloading code; we would just copy over the one modified function. I have poked around some in Jython internals, so for me, adding "edit and continue" to Jython would be the easier possibility than for CPython (but of a more limited audience). If I recall correctly, Jython uses Java exceptions, which are not restartable, so that approach would likely always be be limited to breaking on where they are raised (unless the JVM itself changes). Still, if/when I get the Smalltalk-syntax support on this new base (not much time for it these days for various reasons), I don't see any reason a Python-syntax could not also be added to it (especially using one of the Python-in-Python compilers as the base). So, I fully expect the outcome of what I am doing could support a Python-ish syntax to get edit and continue. But that really is not what we all want, as opposed to the real thing -- Python 2.6 with "edit and continue support". :-) Perhaps someone with experience writing a PEP might want to work with me on at least defining the problem in a way it could be formally submitted? Of course now that I look even harder at people's comments on "edit and continue" I just came across this: http://haacked.com/archive/2006/02/08/OnReligiousWarsinSoftware.aspx listing "Edit and continue" as #5 on "Well Known Religious" wars. :-) Which completely surprises me how people could be against it (and yes, I have read some counter arguments, http://www.codinghorror.com/blog/archives/000507.html but never seen one from someone who had ever actually used the feature extensively, at least when well implemented in a system like Smalltalk, including ones that allow "coding in the debugger" as a development style: http://www.google.com/search?hl=en&q=%22coding+in+the+debugger%22 ) Still, for fixing typos alone, this "edit and continue" feature could save many people (especially beginners) a lot of time with Python IMHO. If Python had it, I'd probably stop thinking about Smalltalk so much. :-) - --Paul Fernhout Michael Tobis wrote: > This is all great stuff! Thanks to all who responded here or in email! > > However so far this all goes to only half of the questions I am trying > to address. > > I'd also like to consider the bad news. At least three important > projects that I know of have abandoned Python in favor of Java or > Squeak: > > 1) Alice > 2) Patapata > 3) Mark Guzdial's CS for artists course/textbook > > It is also plain to see that self-directed web-centric beginners are > more likely to gravitate to Javascript, Actionscript/Flash or PHP. > > What are the roots of these choices? Should we be concerned about > them? If so, is there anything we can and should realistically do > about them? _______________________________________________ Edu-sig mailing list Edu-sig at python.org http://mail.python.org/mailman/listinfo/edu-sig ------- End of Forwarded Message From tjreedy at udel.edu Sat Mar 31 23:12:52 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 31 Mar 2007 17:12:52 -0400 Subject: [pypy-dev] Paul Fernhout wants edit-and-continue in python References: <200703310935.l2V9ZG9m002435@theraft.openend.se> Message-ID: | From: "Paul D. Fernhout" | User-Agent: Icedove 1.5.0.9 (X11/20061220) | I started to look into seeing how "edit and continue" could be addressed | with Python, but, especially given Guido saying it was "impossible", and | looking in the email archives of various lists to see it has been | brought up multiple times on comp.lang.python and such and shot down, | http://www.google.com/search?hl=en&q=python+%22edit+and+continue%22 | it seems it might be easier to build something from scratch which | supports it (on top of Java using those libraries) than to try (and | fail) to improve Python. I also have always preferred Smalltalk keyword | syntax over Python, as it is easily extensible (and easily readable (to | me), so there are always forces tugging me towards creating | alternatives, which people here will rightfully say stand a snowball's | chance of seeing widespread adoption. | | The bottom line on "Edit and continue" is that making it work properly | likely requires deep thinking and understanding about the entire Python | runtime system (as well as getting pickup of your changes by various IDE | developers), and Python as a system has moved so far along the | development curve that coming from a standing start up to an | understanding of Python's current internals necessarily beyond that of | Guido (who says it is impossible :-) is likely a long hard effort. In one of his artima blogs on Pycon2007, Guido says that the OneLaptopPerChild people who attended put a similar request to him and persuaded him to start rethinking hard about improving the situation. So he is at least working on making reload more usefule. I have no idea how far he has gotten in the month since. tjr