From DavidA at ActiveState.com Sat Feb 1 07:24:44 2003 From: DavidA at ActiveState.com (David Ascher) Date: Fri, 31 Jan 2003 22:24:44 -0800 Subject: [pypy-dev] FYI: archive available Message-ID: <3E3B682C.40905@ActiveState.com> FYI: I've had the pypy-dev archives added to the ASPN Mailing list archive. If you care to, you can check it out at: http://aspn.activestate.com/ASPN/Mail/Browse/Threaded/pypy-dev/ -- David Ascher From hpk at trillke.net Sat Feb 1 11:14:53 2003 From: hpk at trillke.net (holger krekel) Date: Sat, 1 Feb 2003 11:14:53 +0100 Subject: [pypy-dev] List Summary Volunteer In-Reply-To: <200301311023.37374.troy@gci.net>; from troy@gci.net on Fri, Jan 31, 2003 at 10:23:37AM -0900 References: <200301311023.37374.troy@gci.net> Message-ID: <20030201111453.L19341@prim.han.de> Hi Troy, [Troy Melhase Fri, Jan 31, 2003 at 10:23:37AM -0900] > Hi Folks: > > I've been lurking on pypy-dev since the original announcement, and I'd like to > help out in some way. I'm not a compiler writer, nor a C coder, but I think > I can write a summary for the list traffic once a week. The "soft" factors for driving a project are often underestimated. That's why i like to do an infrastructure track on the upcoming sprint. > I assume that simply volunteering is enough to be given this responsibility. > Sometime Saturday, I'll summarize the traffic from the last three weeks and > send them to the list and to c.l.p for group review. thanks a lot. I am looking forward to your summary. To make it easily usable it would be good to do it with reST (restructured text) for which you can find good information here: http://docutils.sourceforge.net/#restructuredtext reST is also used by Brett Canon for his nice Python-Dev summaries. I could then incorporate it and index/archive (automatically) on the upcoming web site. greetings, holger From tismer at tismer.com Sun Feb 2 04:24:23 2003 From: tismer at tismer.com (Christian Tismer) Date: Sun, 02 Feb 2003 04:24:23 +0100 Subject: [pypy-dev] List Summary Volunteer In-Reply-To: <200301311023.37374.troy@gci.net> References: <200301311023.37374.troy@gci.net> Message-ID: <3E3C8F67.5060807@tismer.com> Troy Melhase wrote: > Hi Folks: > > I've been lurking on pypy-dev since the original announcement, and I'd like to > help out in some way. I'm not a compiler writer, nor a C coder, but I think > I can write a summary for the list traffic once a week. I really much appreciate that, and I believe that everybody else will, too. This is for sure not easy, and a lot of work! > I assume that simply volunteering is enough to be given this responsibility. > Sometime Saturday, I'll summarize the traffic from the last three weeks and > send them to the list and to c.l.p for group review. > > Comments? Can't do anything but to encourage you, and to wish you luck and stableness. Even re-reading everything thoroughly took be about three hours today. thanks a lot in advance -- chris From logistix at zworg.com Sun Feb 2 04:34:44 2003 From: logistix at zworg.com (logistix) Date: Sat, 1 Feb 2003 19:34:44 -0800 Subject: [pypy-dev] Compiler Benchmarks Message-ID: <3755.1044156884@zworg.com> As I've mentioned before, I've been goofing around with a parser written in pure Python. After getting sick of debugging the old one (like 70 stack frames down) I decided to hand-code one from scratch. I just ran some benchmarks and thought there might be some interest in the results. Something as processor intensive as compiling would also probably be a good benchmark for various incarnations of PyPython vs CPython. I compiled the contents of the Lib directory (but not subdirectories) with the builtin compile, the compiler module's compile, and a hacked compiler module that used my parser on a 866Mhz machine running WinXP. The builtin compile took an average of 2.9 seconds. The compiler module took an average of 105.3 seconds, or 36.3 times as long. The compiler module uses CPython's parser, but other than that it's in pure Python. The hacked compiler module took an average of 177.2 seconds or 61.1 times as long. It has a pure Python module, but relies on tokeize which leaves re as the only C dependancy left. Although I have no desire to write re in python (can you guess why?), I could hand-code a DFA for python tokens if people think there's a need. -logistix From tismer at tismer.com Mon Feb 3 04:11:20 2003 From: tismer at tismer.com (Christian Tismer) Date: Mon, 03 Feb 2003 04:11:20 +0100 Subject: [pypy-dev] List Summary Volunteer In-Reply-To: <200302012316.26289.troy@gci.net> References: <200301311023.37374.troy@gci.net> <3E3C8F67.5060807@tismer.com> <200302012316.26289.troy@gci.net> Message-ID: <3E3DDDD8.1030700@tismer.com> Troy Melhase wrote: ... > It seems as if www.codespeak.net is unavailable -- at least to me. The result > is that I cannot copy message URLs from the archive into the summary. I'll > have to wait until I can reach the server again before I complete the first > one. :( I will send you the complete archive in private email as a .zip archive. If you have Mozilla, it is easy for you to copy the file into your email folder and to restart Mozilla (this *is* necessary). If not, well, you have the "mailbox" package which is able to read the folder. cheers - chris From vlindberg at verio.net Mon Feb 3 18:17:39 2003 From: vlindberg at verio.net (VanL) Date: Mon, 03 Feb 2003 10:17:39 -0700 Subject: [pypy-dev] List Summary Volunteer References: <200301311023.37374.troy@gci.net> Message-ID: <3E3EA433.1040205@verio.net> Hi, I unfortunately did not have time to volunteer to head the whole summary, as you did. However, it seems that it is one of those parallelizable tasks that might benefit from a few more hands. If you wish, I could summarize a one or two particular threads, and email them to you to be included in the overall summary. I think that if a few others might volunteer to write small summaries, then the overall load might be reduced, and a more comprehensive and helpful summary generated. Then again, if you just want to do it, go ahead! You have my gratitude. VanL Troy Melhase wrote: >Hi Folks: > >I've been lurking on pypy-dev since the original announcement, and I'd like to >help out in some way. I'm not a compiler writer, nor a C coder, but I think >I can write a summary for the list traffic once a week. > >I assume that simply volunteering is enough to be given this responsibility. >Sometime Saturday, I'll summarize the traffic from the last three weeks and >send them to the list and to c.l.p for group review. > >Comments? > >-troy > > > >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev > > From troy at gci.net Sun Feb 2 09:16:26 2003 From: troy at gci.net (Troy Melhase) Date: Sat, 01 Feb 2003 23:16:26 -0900 Subject: [pypy-dev] List Summary Volunteer In-Reply-To: <3E3C8F67.5060807@tismer.com> References: <200301311023.37374.troy@gci.net> <3E3C8F67.5060807@tismer.com> Message-ID: <200302012316.26289.troy@gci.net> Hi Chris: > I really much appreciate that, and I believe that > everybody else will, too. This is for sure not easy, > and a lot of work! Aye, after reading the posts to date, I think condensing the ideas correctly will take some thought. My only goals are to be succinct and unobtrusive, for the sole objective of helping in some small way. > Can't do anything but to encourage you, and to wish you > luck and stableness. Even re-reading everything > thoroughly took be about three hours today. I, too, reread everything today. It gave me a renewed sense of excitement about this project. The goals are quite lofty, and I wish all of you the best success. However... It seems as if www.codespeak.net is unavailable -- at least to me. The result is that I cannot copy message URLs from the archive into the summary. I'll have to wait until I can reach the server again before I complete the first one. :( -troy From oren-py-l at hishome.net Sun Feb 2 18:54:34 2003 From: oren-py-l at hishome.net (Oren Tirosh) Date: Sun, 2 Feb 2003 12:54:34 -0500 Subject: [pypy-dev] Re: [Python-Dev] Property syntax In-Reply-To: <200301311821.h0VIL3M10010@odiug.zope.com> References: <200301311628.h0VGSps08054@odiug.zope.com> <20030131165507.GA17116@panix.com> <200301311821.h0VIL3M10010@odiug.zope.com> Message-ID: <20030202175434.GA13189@hishome.net> On Fri, Jan 31, 2003 at 01:21:03PM -0500, Guido van Rossum wrote: > > > That's all fine and dandy, but then I don't understand how the poor > > > implementation of property is going to extract __get__ etc. from the > > > local variables of the function body after executing it. > > > > Sort of. Each keyword can handle the thunk differently. For the > > property keyword, it'd be handled more similarly to a class. In fact, > > class then becomes > > > > def C as class: > > suite > > Um, the whole point of this syntax is that property, synchronized > etc. do *not* have to be keywords -- they are just callables. class C(X, Y): suite <=> def C(X, Y) as type: suite <=> C = type('C', (X, Y), CODE, GLOBALS) The type constructor currently expects a dictionary as third argument. It will need to be modified to do the equivalent of: class_namespace = {} exec CODE in class_namespace, GLOBALS And class_namespace will be initialized to the equivalent dictionary. For functions: def F(args...): suite <=> def F(CODE, GLOBALS, 'F', ARGDEFS) as function: suite <=> F = function(CODE, GLOBALS, 'F', ARGDEFS) CCed to the PyPython list because looking at the creation of function and type objects this way may help bootstrapping - the 'function' and 'type' factories could be written in Python using some kind of more primitive callable construct added as a crutch for PyPython. Oren From troy at gci.net Mon Feb 3 20:13:26 2003 From: troy at gci.net (Troy Melhase) Date: Mon, 03 Feb 2003 10:13:26 -0900 Subject: [pypy-dev] List Summary Volunteer In-Reply-To: <3E3EA433.1040205@verio.net> References: <200301311023.37374.troy@gci.net> <3E3EA433.1040205@verio.net> Message-ID: <200302031013.26464.troy@gci.net> Hi VanL: This is a great idea and I whole-heartedly welcome any contributions from you and other list readers. There is a risk in this, however, and that risk is duplicated effort. If you intend on summarizing one or more threads, please send me a note as early in the week as possible. Thanks! :) -troy P.S. My posts to pypy-dev are bouncing, and I cannot reach the codespeak.net web server. Can you? On Monday 03 February 2003 08:17 am, VanL wrote: > Hi, > > I unfortunately did not have time to volunteer to head the whole > summary, as you did. However, it seems that it is one of those > parallelizable tasks that might benefit from a few more hands. > > If you wish, I could summarize a one or two particular threads, and > email them to you to be included in the overall summary. I think that > if a few others might volunteer to write small summaries, then the > overall load might be reduced, and a more comprehensive and helpful > summary generated. > > Then again, if you just want to do it, go ahead! You have my gratitude. > > VanL From oren-pypy at hishome.net Sun Feb 2 19:26:57 2003 From: oren-pypy at hishome.net (Oren Tirosh) Date: Sun, 2 Feb 2003 13:26:57 -0500 Subject: [pypy-dev] Compiler Benchmarks In-Reply-To: <3755.1044156884@zworg.com> References: <3755.1044156884@zworg.com> Message-ID: <20030202182657.GA38500@hishome.net> On Sat, Feb 01, 2003 at 07:34:44PM -0800, logistix wrote: > I compiled the contents of the Lib directory (but not subdirectories) > with the builtin compile, the compiler module's compile, and a hacked > compiler module that used my parser on a 866Mhz machine running WinXP. > > The builtin compile took an average of 2.9 seconds. > > The compiler module took an average of 105.3 seconds, or 36.3 times as > long. The compiler module uses CPython's parser, but other than that > it's in pure Python. > > The hacked compiler module took an average of 177.2 seconds or 61.1 > times as long. It has a pure Python module, but relies on tokeize > which leaves re as the only C dependancy left. To get a (very rough) estimate of the performance we can expect from a PyPython compiler it would be interesting to know how fast this code runs with psyco. Oren From Gerald.Klix at klix.ch Sun Feb 2 13:27:43 2003 From: Gerald.Klix at klix.ch (Gerald Klix) Date: Sun, 02 Feb 2003 13:27:43 +0100 Subject: [pypy-dev] smalltalk as a model? References: <20030128114006.9C9215ABD2@thoth.codespeak.net> Message-ID: <3E3D0EBF.6090900@klix.ch> Hi, last week, inspired by this list, I started to do a Python in Python implementation, after the Squeak model. Currently I have an object memory with out GC running. From tomorrow onwards I will continue to work on it. cya Gerald Stephan Diehl wrote: > Hello, > > during the last days I got a little bit interested in smalltalk (article in > c't 2/2003). Since smalltalk is already about 30 years old and shares some > similarities to Python, some aspects of smalltalk might ease the effort to > implement Python in Python. > Here are some smalltalk faclets that makes it interesting: > - Everything is an object (even classes and code blocks for example) > - All actions are performed by invoking an object method (even loops) > - there are only 5 keywords and only a couple of "special" characters > - the smalltalk VM is implemented in smalltalk > - smalltalk comes with "batteries included" > > With smalltalk in mind, one could write a kind of ProtoPython that has much > less overhead than the existing Python. > Possible features of a ProtoPython (very incomplete): > - smaller set of reserved words ("print","import",etc. shouldn't be reserved > words) > - no operators, '+','-',... will be methods > - at least the for loop could be a method of a (to be defined) list class. > > Even if this approach is absolutely ridiculous, just looking at smalltalk > might give some ideas/hints how to construct a self describing language. > > Some Links: > > Squeak (open source smalltalk runtime): http://www.squeak.org > > smalltalk 80 :The Language and Its Implementation (contains details about the > VM): http://users.ipa.net/~dwighth/smalltalk/bluebook/bluebook_imp_toc.html > > [Python-Dev] Classes and Metaclasses in Smalltalk: > Guido about a mail from Jim Althoff, inventor of smalltalk metaclasses > http://mail.python.org/pipermail/python-dev/2001-May/014508.html > > smalltalkish python: > http://squeak.cs.uiuc.edu/mail/squeak/msg04577.html > > Stephan > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From hpk at trillke.net Mon Feb 3 22:08:27 2003 From: hpk at trillke.net (holger krekel) Date: Mon, 3 Feb 2003 22:08:27 +0100 Subject: [pypy-dev] Compiler Benchmarks In-Reply-To: <20030202182657.GA38500@hishome.net>; from oren-pypy@hishome.net on Sun, Feb 02, 2003 at 01:26:57PM -0500 References: <3755.1044156884@zworg.com> <20030202182657.GA38500@hishome.net> Message-ID: <20030203220827.E19341@prim.han.de> [Oren Tirosh Sun, Feb 02, 2003 at 01:26:57PM -0500] > On Sat, Feb 01, 2003 at 07:34:44PM -0800, logistix wrote: > > I compiled the contents of the Lib directory (but not subdirectories) > > with the builtin compile, the compiler module's compile, and a hacked > > compiler module that used my parser on a 866Mhz machine running WinXP. > > > > The builtin compile took an average of 2.9 seconds. > > > > The compiler module took an average of 105.3 seconds, or 36.3 times as > > long. The compiler module uses CPython's parser, but other than that > > it's in pure Python. > > > > The hacked compiler module took an average of 177.2 seconds or 61.1 > > times as long. It has a pure Python module, but relies on tokeize > > which leaves re as the only C dependancy left. > > To get a (very rough) estimate of the performance we can expect from > a PyPython compiler it would be interesting to know how fast this code > runs with psyco. I agree that this would be interesting even though the PyPy-compiler might run on a quite different PSYCO. I can imagine modifying the compiler to implement the same restrictions that we might set on the PyPy-Interpreter. I can't wait to begin hacking on this. logistix, is your parser/compiler code available anywhere? holger From logistix at zworg.com Mon Feb 3 23:58:32 2003 From: logistix at zworg.com (logistix) Date: Mon, 3 Feb 2003 14:58:32 -0800 Subject: [pypy-dev] Compiler Benchmarks Message-ID: <10856.1044313112@zworg.com> holger krekel wrote: > > [Oren Tirosh Sun, Feb 02, 2003 at 01:26:57PM -0500] > > On Sat, Feb 01, 2003 at 07:34:44PM -0800, logistix wrote: > > > I compiled the contents of the Lib directory (but not > > > subdirectories) with the builtin compile, the compiler module's > > > compile, and a hacked compiler module that used my parser on a > > > 866Mhz machine running WinXP. > > > > > > The builtin compile took an average of 2.9 seconds. > > > > > > The compiler module took an average of 105.3 seconds, or 36.3 > > > times as long. The compiler module uses CPython's parser, but > > > other than that it's in pure Python. > > > > > > The hacked compiler module took an average of 177.2 seconds or > > > 61.1 times as long. It has a pure Python module, but relies on > > > tokeize which leaves re as the only C dependancy left. > > > > To get a (very rough) estimate of the performance we can expect from > > a PyPython compiler it would be interesting to know how fast this code > > runs with psyco. > > I agree that this would be interesting even though the PyPy-compiler > might run on a quite different PSYCO. I can imagine modifying the > compiler to implement the same restrictions that we might set on the > PyPy-Interpreter. I can't wait to begin hacking on this. > > logistix, is your parser/compiler code available anywhere? > > holger > I just wrote the parser, the compiler code is already in the base distribution. There were a few bugs that turned up in Python2.2, so that's why I didn't run psyco benchmarks. Just using the standard compiler module in 2.2 with psyco didn't yield much of an improvement (108 seconds vs 115). Here's the code if you're interested. As I mentioned, you'll need at least 2.3a1. The instructions to tie it into the compiler modules are in the docstring: http://members.bellatlantic.net/~olsongt/python/pparser.py -logistix P.S. If anyone wants to patch this in and run Tools/Compiler/Regrtest.py against it, results would be appreciated. From hpk at trillke.net Tue Feb 4 01:50:52 2003 From: hpk at trillke.net (holger krekel) Date: Tue, 4 Feb 2003 01:50:52 +0100 Subject: [pypy-dev] Compiler Benchmarks In-Reply-To: <10856.1044313112@zworg.com>; from logistix@zworg.com on Mon, Feb 03, 2003 at 02:58:32PM -0800 References: <10856.1044313112@zworg.com> Message-ID: <20030204015052.H19341@prim.han.de> [logistix Mon, Feb 03, 2003 at 02:58:32PM -0800] > holger krekel wrote: > > [Oren Tirosh Sun, Feb 02, 2003 at 01:26:57PM -0500] > > > To get a (very rough) estimate of the performance we can expect from > > > a PyPython compiler it would be interesting to know how fast this code > > > runs with psyco. > > > > I agree that this would be interesting even though the PyPy-compiler > > might run on a quite different PSYCO. I can imagine modifying the > > compiler to implement the same restrictions that we might set on the > > PyPy-Interpreter. I can't wait to begin hacking on this. > > > > logistix, is your parser/compiler code available anywhere? > > > > holger > > > > I just wrote the parser, the compiler code is already in the base > distribution. of course. > There were a few bugs that turned up in Python2.2, so that's why I > didn't run psyco benchmarks. Just using the standard compiler module > in 2.2 with psyco didn't yield much of an improvement (108 seconds vs > 115). Where is Armin if you need him :-) Here's the code if you're interested. As I mentioned, you'll need at > least 2.3a1. The instructions to tie it into the compiler modules are > in the docstring: > > http://members.bellatlantic.net/~olsongt/python/pparser.py Sure looks interesting. You do a lot of function calls, right? > -logistix > > P.S. If anyone wants to patch this in and run Tools/Compiler/Regrtest.py > against it, results would be appreciated. i ran the Lib/test/test_parser against your module but it choked mostly (on missing totuple methods among other stuff). Regrtest is currently running with no problems. (it takes some time, though :-) If this is really a quite complete parser then pparser.pyc is a lot smaller than the equivalent parsermodule.o. not that this means too much :-) does your pparser work similar to parsermodule.c ? holger From tismer at tismer.com Tue Feb 4 02:51:50 2003 From: tismer at tismer.com (Christian Tismer) Date: Tue, 04 Feb 2003 02:51:50 +0100 Subject: [pypy-dev] smalltalk as a model? In-Reply-To: <3E3D0EBF.6090900@klix.ch> References: <20030128114006.9C9215ABD2@thoth.codespeak.net> <3E3D0EBF.6090900@klix.ch> Message-ID: <3E3F1CB6.5020900@tismer.com> Gerald Klix wrote: > Hi, > last week, inspired by this list, I started to do a Python in Python > implementation, after the Squeak model. Currently I have an object > memory with out GC running. To put your mouth where your code is, please let us know :-)) > From tomorrow onwards I will continue to work on it. How about sharing your experience in the sprint? ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From logistix at zworg.com Tue Feb 4 03:11:49 2003 From: logistix at zworg.com (logistix) Date: Mon, 3 Feb 2003 18:11:49 -0800 Subject: [pypy-dev] Compiler Benchmarks Message-ID: <3376.1044324709@zworg.com> holger krekel wrote: > > > Here's the code if you're interested. As I mentioned, you'll need at > > least 2.3a1. The instructions to tie it into the compiler modules are > > in the docstring: > > > > http://members.bellatlantic.net/~olsongt/python/pparser.py > > Sure looks interesting. You do a lot of function calls, right? > > > -logistix > > > > P.S. If anyone wants to patch this in and run Tools/Compiler/Regrtest.py > > against it, results would be appreciated. > > i ran the Lib/test/test_parser against your module but it choked mostly > (on missing totuple methods among other stuff). > > Regrtest is currently running with no problems. (it takes some time, > though :-) > My Regrtest is locking up on test_pickletest (or something like that) Also saw a bug in simple_stmt. > If this is really a quite complete parser then pparser.pyc > is a lot smaller than the equivalent parsermodule.o. > not that this means too much :-) > > does your pparser work similar to parsermodule.c ? > Actually parsermodule doesn't do any of the work. Here's my non-authorative explaination of how CPython parses, as I'm still figuring it out myself. There's a utility Pgen, which is Python's custom Parser Generator similar to Yacc or Bison. It uses grammar/grammar as input, and generates graminit.c. Graminit.c will hurt your brain if you try to figure out what it's doing (also highlights some of the perils of automatic code generation). Parsetok.c takes the generated grammar in conjunction with some tokenized text, and builds the AST tree based on these two inputs. Parsermodule.c just provides a Python interface to the above internals. The bulk of the code is validation code, so that if you try to compile a hand-built ast, it'll throw an error there instead of blowing up the compiler internals. It's actually been really handy for me to debug the stuff I've been writing. Hand-coding the parser, like I did, is more likely to result in subtile bugs creeping in and is harder to maintain than PGen if your source grammar is changing alot. Right now Python's grammar is reasonably stable so it's not too bad. I previously tried to write a Parser Generator in pure Python, but it had all kinds of bugs and I'm still not entirely comfortable doing serious low-level debugging in Python. I also noticed some comments in PGen that indicated they used a "unique" method of reducing the grammar. From tismer at tismer.com Tue Feb 4 03:14:54 2003 From: tismer at tismer.com (Christian Tismer) Date: Tue, 04 Feb 2003 03:14:54 +0100 Subject: [pypy-dev] Compiler Benchmarks In-Reply-To: <20030202182657.GA38500@hishome.net> References: <3755.1044156884@zworg.com> <20030202182657.GA38500@hishome.net> Message-ID: <3E3F221E.5060603@tismer.com> Oren Tirosh wrote: > On Sat, Feb 01, 2003 at 07:34:44PM -0800, logistix wrote: > >>I compiled the contents of the Lib directory (but not subdirectories) >>with the builtin compile, the compiler module's compile, and a hacked >>compiler module that used my parser on a 866Mhz machine running WinXP. >> >>The builtin compile took an average of 2.9 seconds. >> >>The compiler module took an average of 105.3 seconds, or 36.3 times as >>long. The compiler module uses CPython's parser, but other than that >>it's in pure Python. >> >>The hacked compiler module took an average of 177.2 seconds or 61.1 >>times as long. It has a pure Python module, but relies on tokeize >>which leaves re as the only C dependancy left. > > > To get a (very rough) estimate of the performance we can expect from > a PyPython compiler it would be interesting to know how fast this code > runs with psyco. Current Psyco probably does not give a good estimate what can be achieved. I've had very good and quite bad experiences, yet. Psyco is going to start off from the ground, written in Python, with the experience of its C alpha implementation. I don't put any relevance into measurements, today, and you shouldn't, either. Some misbeliever muttered today, that Armin's brain has shrunk from a planet down to an asteroid, by adopting this PyPy project. Nice joke! I positively did not reply, since I want us to prove him wrong. This time, Armin's mission is not only a singleton planet, but it carries impressingly intelligent life. And we will focus our collective work into creating an incredible outlet of imaginative power. (whow, did I say that? Can somebody please adjust tismer-bot?) According to Stanislav Lem's immortal work, my current project name proposal is "Solaris". If you want to avoid me telling that story in detail on the sprint, be quick and read that book. !-) all the best -- pirx (yes, I do think he is caught in that time-loop trap) (and he still thinks he can change the world. Together we can) -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at tismer.com Tue Feb 4 03:19:25 2003 From: tismer at tismer.com (Christian Tismer) Date: Tue, 04 Feb 2003 03:19:25 +0100 Subject: [pypy-dev] List Summary Volunteer In-Reply-To: <200302031013.26464.troy@gci.net> References: <200301311023.37374.troy@gci.net> <3E3EA433.1040205@verio.net> <200302031013.26464.troy@gci.net> Message-ID: <3E3F232D.8070808@tismer.com> Troy Melhase wrote: > Hi VanL: > > This is a great idea and I whole-heartedly welcome any contributions from you > and other list readers. > > There is a risk in this, however, and that risk is duplicated effort. If you > intend on summarizing one or more threads, please send me a note as early in > the week as possible. You both, please get together, ASAP. I'm very happy that you are trying to get some structure into this bulk of thoughts, claims, recommendations abnd committments. > P.S. > My posts to pypy-dev are bouncing, and I cannot reach the codespeak.net web > server. Can you? I believe it is stable now, after Holger and Jum fixed some hardware problems. Please, go ahead -- chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tdelaney at avaya.com Tue Feb 4 03:28:52 2003 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Tue, 4 Feb 2003 13:28:52 +1100 Subject: [pypy-dev] Compiler Benchmarks Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE07FC0C@au3010avexu1.global.avaya.com> > From: Christian Tismer [mailto:tismer at tismer.com] > > I don't put any relevance into measurements, today, > and you shouldn't, either. > Some misbeliever muttered today, that Armin's brain has > shrunk from a planet down to an asteroid, by adopting this > PyPy project. Nice joke! Hey - I wasn't suggesting that it's because he's working on PyPy. He's in the real world now, so of course his brain is shrinking. That's why I think he needs to work on another doctorate ;) As to whether PyPy would be a suitable body of work ... Tim Delaney From dwhall256 at yahoo.com Wed Feb 5 01:40:05 2003 From: dwhall256 at yahoo.com (Dean Hall) Date: Tue, 4 Feb 2003 16:40:05 -0800 (PST) Subject: [pypy-dev] just joined Message-ID: <20030205004005.44741.qmail@web40501.mail.yahoo.com> Hey guys, I just joined this list and thought I'd introduce myself. I'm Dean. How am I doing so far? I have a paper that I'm presenting at PyCon2003 about PyMite, my from-scratch flyweight python interpreter for 8-bit architectures (under development). The reviewer of the paper recommended that I contact this endeavor to share/exchange ideas. So, here I am. Rundown of PyMite (prepare to lower your expectations): Supports Integers, Strings, Lists, Dicts and Tuples [NOT float, long, complex]. Supports the useful 60+% of Python 2.0's bytecodes. Supports modules, classes, methods, functions [but not (yet) inheritance, varargs, or default args]. It's stackless; has a FAST inline native code interface; fits in 15KB of code on an Atmel atMega103 8-bit processor; runs fast enough to not be a pain. On the todo list: hybrid mark/sweep/coloring GC, exceptions. Source not yet available. I'll post it on SF by PyCon. It'll be open source, but I'm going to be very strict about code submissions because my size constraints are insanely small. Splinter development is OK by me (but I doubt anyone else is interested). !!Dean __________________________________________________ Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now. http://mailplus.yahoo.com From hpk at trillke.net Wed Feb 5 10:19:18 2003 From: hpk at trillke.net (holger krekel) Date: Wed, 5 Feb 2003 10:19:18 +0100 Subject: [pypy-dev] just joined In-Reply-To: <20030205004005.44741.qmail@web40501.mail.yahoo.com>; from dwhall256@yahoo.com on Tue, Feb 04, 2003 at 04:40:05PM -0800 References: <20030205004005.44741.qmail@web40501.mail.yahoo.com> Message-ID: <20030205101918.X19341@prim.han.de> Hello Dean! [Dean Hall Tue, Feb 04, 2003 at 04:40:05PM -0800] > Hey guys, > > I just joined this list and thought I'd introduce > myself. I'm Dean. How am I doing so far? > > I have a paper that I'm presenting at PyCon2003 about > PyMite, my from-scratch flyweight python interpreter > for 8-bit architectures (under development). The > reviewer of the paper recommended that I contact this > endeavor to share/exchange ideas. So, here I am. > > Rundown of PyMite (prepare to lower your > expectations): > Supports Integers, Strings, Lists, Dicts and Tuples > [NOT float, long, complex]. Supports the useful 60+% > of Python 2.0's bytecodes. Supports modules, classes, > methods, functions [but not (yet) inheritance, > varargs, or default args]. It's stackless; has a FAST > inline native code interface; fits in 15KB of code on > an Atmel atMega103 8-bit processor; runs fast enough > to not be a pain. > On the todo list: hybrid mark/sweep/coloring GC, > exceptions. > > Source not yet available. I'll post it on SF by > PyCon. It'll be open source, but I'm going to be very > strict about code submissions because my size > constraints are insanely small. Splinter development > is OK by me (but I doubt anyone else is interested). No! I am pretty sure there is interest here. We will be doing a one-week Sprint from 17th to 23th of February and would appreciate looking into your source code and experimenting with it. At least i would. One of the features of OpenSource development is "unplanned cooperation" and i have seen companies as well as individual people beeing surprised by it. cheers, holger From stephan.diehl at gmx.net Wed Feb 5 11:04:41 2003 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Wed, 5 Feb 2003 11:04:41 +0100 Subject: [pypy-dev] smalltalk as a model? In-Reply-To: <3E3D0EBF.6090900@klix.ch> References: <20030128114006.9C9215ABD2@thoth.codespeak.net> <3E3D0EBF.6090900@klix.ch> Message-ID: <20030205100410.1C4715ABD7@thoth.codespeak.net> On Sunday 02 February 2003 13:27, you wrote: > Hi, > last week, inspired by this list, I started to do a Python in Python > implementation, after the Squeak model. Currently I have an object > memory with out GC running. > > From tomorrow onwards I will continue to work on it. Would you mind sharing what you have done so far? Cheers Stephan > > cya > Gerald > From troy at gci.net Thu Feb 6 08:39:48 2003 From: troy at gci.net (Troy Melhase) Date: Wed, 05 Feb 2003 22:39:48 -0900 Subject: [pypy-dev] pypy-dev List Summary - 06 Jan 2003 to 31 Jan 2003 Message-ID: <200302052239.48561.troy@gci.net> This is a summary of the discussions on the pypy-dev mailing list between 06 January and 31 January 2003. The full archive of the list can be found here: http://www.codespeak.net/pipermail/pypy-dev/ Bootstrapping ------------- Holger Krekel starts the discussion in earnest with a message regarding how the project came about, and how to get Minimal Python[1] bootstrapped. Robin Becker suggests implementing the ctypes module. Thomas Heller posts a working version using libffi, and the possibility of using ffcall is also mentioned. Christian Tismer would rather use the essential ideas of ctypes, rather than the module itself, to keep Minimal Python small: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000005.html The build environment is discussed in this thread; CPython, SCons, make and friends, and Builtool are mentioned. C Core, Macros -------------- Florian Schulze asks about plans for the C core. Christian says most objects will be borrowed at first. Andrew McGregor brings up macros, and a lively discussion ensues. The consensus is that macros have already been rejected by the BDFL, and implementing them would conflict with the project goal of maintaining conformance with the language definition: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000010.html Ouroboros --------- Damien Morton suggests using the ouroboros as the basis for the project logo. In his words: "In mythology, the Ouroboros is any image of a snake, worm, serpent, or dragon biting its own tail." He posts links and descriptions, and Holger wants to add a bit of cuteness and fun: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000017.html Intellectual Property --------------------- David Ascher wants the issues of IP and license resolved earlier rather than later. He suggests the IP be assigned to the PSF, and that the license be PSF or AFL: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000042.html Road Map --------- Bengt Richter suggests a project road map, and Guido responds with a quick list of Python-already-implemented-in-Python. Christian thanks him, and suggests that Python code devoid of trickery is more suitable to Psyco. Armin Rigo agrees: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000047.html You tricksters know who you are! PyVM ---- Paul Swartz posts a link to his PyVM, a Python Bytecode interpreter. http://www.codespeak.net/pipermail/pypy-dev/2003q1/000048.html Project Name ------------ Many suggestions for an official project name were posted to the list throughout the month. In lieu of listing them all (and surely omitting some), the summary is that there is no official name yet. "Minimal Python" and "PyPython" are used commonly. Compile.c ---------- Holger wonders aloud how to remove the need for compile.c, and talks about a multi-stage VM: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000097.html Psyco in Python --------------- Edward K. Ream asks how Psyco will fit in Minimal Python. Michael Hudson and Christian do their best to answer him, and mention the tantalizing possibility of having Psyco re-implemented in Python: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000091.html Lessons from Experience ----------------------- Rocco Moretti talks about his experience trying to write a Python interpreter in Python. He mentions the difficulty distinguishing between host and target, and that use of Exceptions in CPython gave him trouble. Christian advocates the generated-C approach, and Armin believes in separating application from interpreter Exceptions: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000104.html Low-Level Data Type Representation ---------------------------------- Christian, Samuele Pedroni and Paul Rubin discuss various approaches on representing low-level data types in at the much-higher Python level. Paul suggests tagged representation, Christian expresses the need to keep the types easily deduced by the interpreter, and Samuele suggests the boxed values approach. Guido chimes in with his first-hand knowledge of tagged representation in the ABC language: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000121.html From PyPy to Psyco ------------------ Armin revisits the first goal of the project, writing a Python interpreter in Python. From there, he suggests, the "interesting things" are possible. This interpreter can be statically analyzed, and from the analysis will come a minimal compiled interpreter, a bytecode checker, and Psyco: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000125.html Edward K. Ream responds with his interpretations and suggestions for making Armin's tools available sooner. Ported Psyco ------------ The subject of Psyco on non-x86 hardware comes up, and Armin says the newer code will be flexible and easy to port: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000218.html Leo Outlines and Questions -------------------------- Edward K. Ream posts links to Leo outlines he's made for Psyco and for a C tool suite. http://www.codespeak.net/pipermail/pypy-dev/2003q1/000137.html Edward also suggests that Psyco can't be faster than compiled C, and Armin reminds him that it's a matter of perspective. Agreement is quickly reached that the point isn't relevant to the project: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000153.html More Questions -------------- Edward is full of questions, and Armin has answers for him. Armin casts more light on Psyco, and his illumination deserves special note: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000155.html Bengt Richter brings up the idea of check-pointing Psyco code, and he and Edward discuss storing the code in .pyp files. Armin likes the idea, and thinks it might be done outside of Psyco: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000162.html Restricted Language ------------------- Christian poses many questions regarding the initial restricted language. Discussion turns quickly to types: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000160.html Armin clarifies the issue by drawing a distinction between Python at the interpreter-level and Python at the application-level: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000211.html Later in the thread, Scott David Daniels advises watching for boundary conditions, and advocates a check-pointing system: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000203.html Armin revisits the need for a different approach to exception handling: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000221.html Implementation Language Questions ---------------------------------- Stephan Diehl asks why the implementation could not be written in Objective C. Christian responds that the idea is to remove C, not to replace it: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000167.html Translating CPython Source -------------------------- Christian speaks from experience when he says translating CPython source and also maintaining concurrency with it is a loosing proposition. As an alternative, he suggests reusing portions of a C compiler package to provide a level of automated translation of CPython source: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000188.html Edward advises changing perspective such that the problem goes away. Thomas Heller suggests that folks pick up a CPython module or two and translate them into Python. Holger believes maintaining two sources, CPython and PyPython would be a nightmare, and Armin mentions Stackless and Jython as examples of the inherent difficulty of implementing Python from translated CPython. Armin believes the translation could be automatic for most changes to CPython given an association between custom .py files and the CPython source. Microthreads ------------ Sebastien Pierre asks if microthreads will be incorporated, and Armin says they would be easy enough, given a high-level description of the interpreter: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000219.html Minimal Sprint -------------- Armin asks if there are definite dates for the first sprint. Holger suggests the 17th to the 23rd of February, and lists a possible location in Hildesheim: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000237.html Holger later announces the Minimal Python Sprint. He cites the dates above, and the goals of the sprint: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000263.html C Emissions ----------- Logistix asks for clarification regarding the progression from byte code to C to assembly. Armin describes the two processes discussed so far: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000252.html CPython - PyPython ------------------ Holger recaps the topics decisions to date made regarding translating CPython sources. The Python language isn't to be modified, CPython takes lead wherever possible, and clean abstractions lead his list: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000239.html __builtin__, in Python ---------------------- Scott Fenton posts his replacement for the __builtin__ module: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000246.html The discussion turns to Scott's implementation of ord() and chr(), with Armin pointing out that these would be supplied to the interpreter, not by it: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000254.html Scott later says that his implementation wasn't meant to be definitive: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000291.html Bengt Richter tries to factor out some of the differences between type representation, and Scott and Samuele Pedroni discuss untyped bit vectors: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000319.html Ctypes News ----------- Thomas Heller posts news about the Ctypes module, noting that argument conversion was rewritten, and that libffi is integrated: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000310.html Python Bytecode from... GCC? ---------------------------- Thomas Fanslau asks generating Python bytecode from GCC would be possible, and Nathan Heagy says it would be difficult: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000259.html Compiler Package Notes ---------------------- Logistix posts his notes on the compiler package. Michael Hudson confirms most of his impressions: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000277.html Jeremy Hylton posts with his status on the package, and summarizes the work left to be done: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000286.html Jeremy wonders if Minimal Python would need to generate CPython-compatible bytecode, and Christian thinks it may not: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000288.html Armin posts an interesting solution to Jeremy's stack-depth computation problem. His solution is the main interpreter loop of Python in Python: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000302.html Miasma: x86 machine Code Generation in Python ---------------------------------------------- Darius Bacon posts his x86 code generator, and Holger asks him to explain the challenges and solutions: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000274.html Object-Derived -------------- Scott Fenton wonders how to implement classmethod, thinking it is only applicable to subtypes of object. Christian corrects the misunderstanding and suggests coding for old and new-style classes: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000292.html Christian also proposes an extension to the new class system. His extension would have __slots__ indicate their type and allow types to derive from nothing. Builtin Types ------------- Bengt Richter posts his ideas on "describing C structures in a canonical abstract way". Darius Bacon recognizes this as "First-Class Data-type Representation in SchemeXerox". Thomas Heller mentions these ideas are already present in ctypes: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000298.html Christian thinks of using Ctypes as the first target of a types implementation in Python. Samuele Pedroni brings up the issue of abstracting use of PyTypeObject and the interpreter internal inheritance and lookup mechanisms: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000313.html The issue of using Ctypes as the basis of a type system in Python is debated. Agreement is reached that the type system must describe types, but implementing the system is not urgent: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000351.html Holger posts his set of restrictions for the pypy-python-interpreter. Armin clarifies some of them and posts his additions: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000360.html __builtin__, Again ------------------- Scott Fenton posts his updated version of __builtin__: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000331.html The working URL for Scott?s code comes a few messages later: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000333.html Holger thinks having this code pass the equivalent CPython tests would be nice, and that scripting doc string generation would be easy: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000336.html Pedro Rodriguez adds his Python implementation of the dir() builtin: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000343.html Bits with Types --------------- Bengt Richter posts initial ideas on a generic approach to evaluating machine code: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000355.html Modeled from Smalltalk ---------------------- Stephan Diehl suggests looking at Smalltalk and how it has been implemented in Smalltalk: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000356.html List Archive ------------ David Ascher sends a notice that ASPN is hosting an archive of the pypy-dev list: http://www.codespeak.net/pipermail/pypy-dev/2003q1/000371.html whew! -troy From florian.proff.schulze at gmx.net Thu Feb 6 18:27:29 2003 From: florian.proff.schulze at gmx.net (Florian Schulze) Date: Thu, 06 Feb 2003 18:27:29 +0100 Subject: [pypy-dev] Re: pypy-dev List Summary - 06 Jan 2003 to 31 Jan 2003 In-Reply-To: <200302052239.48561.troy@gci.net> References: <200302052239.48561.troy@gci.net> Message-ID: On Wed, 05 Feb 2003 22:39:48 -0900, Troy Melhase wrote: I think that was a nice summary. -- Florian Schulze From hpk at trillke.net Thu Feb 6 18:56:45 2003 From: hpk at trillke.net (holger krekel) Date: Thu, 6 Feb 2003 18:56:45 +0100 Subject: [pypy-dev] pypy-dev List Summary - 06 Jan 2003 to 31 Jan 2003 In-Reply-To: <200302052239.48561.troy@gci.net>; from troy@gci.net on Wed, Feb 05, 2003 at 10:39:48PM -0900 References: <200302052239.48561.troy@gci.net> Message-ID: <20030206185645.L1836@prim.han.de> [Troy Melhase Wed, Feb 05, 2003 at 10:39:48PM -0900] > This is a summary of the discussions on the pypy-dev mailing list between 06 > January and 31 January 2003. The full archive of the list can be found here: > > http://www.codespeak.net/pipermail/pypy-dev/ thanks a lot, Troy. Even docutils could do something with it :-) It will be on the the pypy-site which is going to be announced in the next hours. Reading the summary i noticed that we really have a lot of loose ends but i am probably not the first one to notice that. The upcoming sprint will certainly help to understand what we are trying to do. These summaries are going to be a great help for remembering and keeping up with what happens. regards, holger From troy at gci.net Thu Feb 6 21:15:46 2003 From: troy at gci.net (Troy Melhase) Date: Thu, 06 Feb 2003 11:15:46 -0900 Subject: [pypy-dev] pypy-dev List Summary - 06 Jan 2003 to 31 Jan 2003 In-Reply-To: <20030206185645.L1836@prim.han.de> References: <200302052239.48561.troy@gci.net> <20030206185645.L1836@prim.han.de> Message-ID: <200302061115.46260.troy@gci.net> apologies * 10000 I had no idea that the text would contain look like that in some mail readers. Next time, I'll do the whole thing in emacs. Or better yet, pico. :) I did run the text thru docutils, and it formatted the titles and links as I had hoped. Did you have something different in mind for the formatting? -troy On Thursday 06 February 2003 08:56 am, holger krekel wrote: > [Troy Melhase Wed, Feb 05, 2003 at 10:39:48PM -0900] > > > This is a summary of the discussions on the pypy-dev mailing list between > > 06 January and 31 January 2003. The full archive of the list can be > > found here: > > > > http://www.codespeak.net/pipermail/pypy-dev/ > > thanks a lot, Troy. Even docutils could do something with it :-) > It will be on the the pypy-site which is going to be announced in the > next hours. > > Reading the summary i noticed that we really have a lot of loose > ends but i am probably not the first one to notice that. > The upcoming sprint will certainly help to understand what we > are trying to do. > > These summaries are going to be a great help for remembering and > keeping up with what happens. > > regards, > > holger From hpk at trillke.net Thu Feb 6 21:42:43 2003 From: hpk at trillke.net (holger krekel) Date: Thu, 6 Feb 2003 21:42:43 +0100 Subject: [pypy-dev] pypy-dev List Summary - 06 Jan 2003 to 31 Jan 2003 In-Reply-To: <200302061115.46260.troy@gci.net>; from troy@gci.net on Thu, Feb 06, 2003 at 11:15:46AM -0900 References: <200302052239.48561.troy@gci.net> <20030206185645.L1836@prim.han.de> <200302061115.46260.troy@gci.net> Message-ID: <20030206214243.P1836@prim.han.de> [Troy Melhase Thu, Feb 06, 2003 at 11:15:46AM -0900] > apologies * 10000 > > I had no idea that the text would contain look like that in some mail readers. > Next time, I'll do the whole thing in emacs. Or better yet, pico. :) i am wondering * 10 * 10 I didn't mean 'loose ends' in a syntactic but semantic way. Or am i not getting something? > I did run the text thru docutils, and it formatted the titles and links as I > had hoped. Did you have something different in mind for the formatting? no. But I have to integrate docutils-css into my general-css for the pypy-site. That's coming. everything-is-fine-ly y'rs, holger From roccomoretti at netscape.net Fri Feb 7 04:00:45 2003 From: roccomoretti at netscape.net (Rocco Moretti) Date: Thu, 06 Feb 2003 22:00:45 -0500 Subject: [pypy-dev] pypy-dev List Summary - 06 Jan 2003 to 31 Jan 2003 Message-ID: <330312BA.67D7F253.9ADE5C6A@netscape.net> holger krekel wrote: >Reading the summary i noticed that we really have a lot of loose >ends but i am probably not the first one to notice that. Would you (or anyone else, for that matter) mind posting a list to start people thinking/discussing? We don't need to have answers yet - knowing the right question is 80% of the battle. -Rocco __________________________________________________________________ The NEW Netscape 7.0 browser is now available. Upgrade now! http://channels.netscape.com/ns/browsers/download.jsp Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/ From hpk at trillke.net Fri Feb 7 04:39:21 2003 From: hpk at trillke.net (holger krekel) Date: Fri, 7 Feb 2003 04:39:21 +0100 Subject: [pypy-dev] ann: website eventually Message-ID: <20030207043921.S1836@prim.han.de> Hello, finally after quite some hacking here is the first incarnation of our web site: http://codespeak.net/pypy I hope it is stable although i did quite some things to moinmoin, roundup and mailman to make them all adhere to a consistent CSS among other stuff. The site currently focuses on the upcoming sprint but can easily be extended with new pages. Troy's summary is found under the 'lists' menu link. I hope you enjoy the new user interface to moinmoin although the UserPreferences aka login-screen is still the old one. Follow the advice on the "moin" entry page to cope with it. regards, holger -- "you can have it fast, cheap or high quality. pick two." (from the whiteboard at infrae, nl) From paoloinvernizzi at dmsware.com Fri Feb 7 09:36:02 2003 From: paoloinvernizzi at dmsware.com (Paolo Invernizzi) Date: Fri, 7 Feb 2003 09:36:02 +0100 Subject: [pypy-dev] ann: website eventually References: <20030207043921.S1836@prim.han.de> Message-ID: <003301c2ce83$f2c72410$7200a8c0@dmsware.com> > finally after quite some hacking here is the first > incarnation of our web site: Very good job! Congratulation! And I love the logo idea ;-) Paolo From arigo at tunes.org Fri Feb 7 11:38:27 2003 From: arigo at tunes.org (Armin Rigo) Date: Fri, 7 Feb 2003 02:38:27 -0800 (PST) Subject: [pypy-dev] pypy-dev List Summary - 06 Jan 2003 to 31 Jan 2003 In-Reply-To: <330312BA.67D7F253.9ADE5C6A@netscape.net>; from roccomoretti@netscape.net on Thu, Feb 06, 2003 at 10:00:45PM -0500 References: <330312BA.67D7F253.9ADE5C6A@netscape.net> Message-ID: <20030207103827.59E134AAC@bespin.org> Hello Rocco, On Thu, Feb 06, 2003 at 10:00:45PM -0500, Rocco Moretti wrote: > >Reading the summary i noticed that we really have a lot of loose > >ends but i am probably not the first one to notice that. > > Would you (or anyone else, for that matter) mind posting a list > to start people thinking/discussing? I plan to go through Troy's summaries and identify the main ones that either reached some conclusion "we should do the thing like that" or leaved several such possible conclusions opened. Then I will make corresponding Wiki pages and encourage people to post code snippets and experiments there. I will also gather links to such experiments that have already been posted in pypy-dev and put them there. My motivation is also that I have myself a couple of such experiments that I would like to share :-) bare with me if it reflects my very own point of view only -- better yet, add your own pages. It's a Wiki! A bient?t, Armin. From hpk at trillke.net Fri Feb 7 13:04:19 2003 From: hpk at trillke.net (holger krekel) Date: Fri, 7 Feb 2003 13:04:19 +0100 Subject: [pypy-dev] pypy-dev List Summary - 06 Jan 2003 to 31 Jan 2003 In-Reply-To: <20030207103827.59E134AAC@bespin.org>; from arigo@tunes.org on Fri, Feb 07, 2003 at 02:38:27AM -0800 References: <330312BA.67D7F253.9ADE5C6A@netscape.net> <20030207103827.59E134AAC@bespin.org> Message-ID: <20030207130419.W1836@prim.han.de> [Armin Rigo Fri, Feb 07, 2003 at 02:38:27AM -0800] > Hello Rocco, > > On Thu, Feb 06, 2003 at 10:00:45PM -0500, Rocco Moretti wrote: > > >Reading the summary i noticed that we really have a lot of loose > > >ends but i am probably not the first one to notice that. > > > > Would you (or anyone else, for that matter) mind posting a list > > to start people thinking/discussing? > > I plan to go through Troy's summaries and identify the main ones that either > reached some conclusion "we should do the thing like that" or leaved several > such possible conclusions opened. Then I will make corresponding Wiki pages > and encourage people to post code snippets and experiments there. I will > also gather links to such experiments that have already been posted in > pypy-dev and put them there. Cool. I am too busy organizing these days to seriously do this. > My motivation is also that I have myself a couple of such experiments that I > would like to share :-) bare with me if it reflects my very own point of view > only -- better yet, add your own pages. It's a Wiki! But note, that i'd like the typtical Wiki fate: lots of outdated and badly maintained pages. We want to keep this Wiki clean and proper :-) I have done quite a bit to improve the wiki-experience and hope that we don't treat it as "just another wiki". There is lots of room for improvement. For example we can define Macros with which we can reference Issues (in the roundup tracker) or directly reference files in the upcoming repository. We have a server instance of subversion running which i am going to test out some more in the next days. cheers, holger From hpk at trillke.net Fri Feb 7 14:10:27 2003 From: hpk at trillke.net (holger krekel) Date: Fri, 7 Feb 2003 14:10:27 +0100 Subject: [pypy-dev] pypy-dev List Summary - 06 Jan 2003 to 31 Jan 2003 In-Reply-To: <20030207130419.W1836@prim.han.de>; from hpk@trillke.net on Fri, Feb 07, 2003 at 01:04:19PM +0100 References: <330312BA.67D7F253.9ADE5C6A@netscape.net> <20030207103827.59E134AAC@bespin.org> <20030207130419.W1836@prim.han.de> Message-ID: <20030207141027.X1836@prim.han.de> [holger krekel Fri, Feb 07, 2003 at 01:04:19PM +0100] > [Armin Rigo Fri, Feb 07, 2003 at 02:38:27AM -0800] > > Hello Rocco, > > > > On Thu, Feb 06, 2003 at 10:00:45PM -0500, Rocco Moretti wrote: > > > >Reading the summary i noticed that we really have a lot of loose > > > >ends but i am probably not the first one to notice that. > > > > > > Would you (or anyone else, for that matter) mind posting a list > > > to start people thinking/discussing? > > > > I plan to go through Troy's summaries and identify the main ones that either > > reached some conclusion "we should do the thing like that" or leaved several > > such possible conclusions opened. Then I will make corresponding Wiki pages > > and encourage people to post code snippets and experiments there. I will > > also gather links to such experiments that have already been posted in > > pypy-dev and put them there. > > Cool. I am too busy organizing these days to seriously do this. > > > My motivation is also that I have myself a couple of such experiments that I > > would like to share :-) bare with me if it reflects my very own point of view > > only -- better yet, add your own pages. It's a Wiki! > > But note, that i'd like the typtical Wiki fate: lots of outdated and badly ^^^^^^^^^^^^ "to avoid" is missing here, of course :-) holger From nathanh at zu.com Fri Feb 7 17:11:45 2003 From: nathanh at zu.com (Nathan Heagy) Date: Fri, 07 Feb 2003 10:11:45 -0600 Subject: [pypy-dev] ann: website eventually In-Reply-To: <003301c2ce83$f2c72410$7200a8c0@dmsware.com> Message-ID: Speaking of graphics and what-not, I am a graphic designer as well as a programmer so if you'd like any help with graphics, design or web stuff let me know. On Friday, February 7, 2003, at 02:36 AM, Paolo Invernizzi wrote: >> finally after quite some hacking here is the first >> incarnation of our web site: > > Very good job! Congratulation! > And I love the logo idea ;-) > > Paolo > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > > -- Nathan Heagy phone:306.653.4747 fax:306.653.4774 http://www.zu.com From hpk at trillke.net Fri Feb 7 17:33:28 2003 From: hpk at trillke.net (holger krekel) Date: Fri, 7 Feb 2003 17:33:28 +0100 Subject: [pypy-dev] ann: website eventually In-Reply-To: ; from nathanh@zu.com on Fri, Feb 07, 2003 at 10:11:45AM -0600 References: <003301c2ce83$f2c72410$7200a8c0@dmsware.com> Message-ID: <20030207173328.E1836@prim.han.de> [Nathan Heagy Fri, Feb 07, 2003 at 10:11:45AM -0600] > Speaking of graphics and what-not, I am a graphic designer as well as a > programmer so if you'd like any help with graphics, design or web stuff > let me know. Sure I will. Thanks! Right where i live i have good contacts to some designers, too so we will definitely get something better than python.org if i dare say that :-) The Python.org site is a bit overloaded and sometimes (they shuffle images) has these titanic-sinking py th on. org logo which is really questionable, isn't it? holger From Tobias.Oberstein at gmx.de Fri Feb 7 19:09:48 2003 From: Tobias.Oberstein at gmx.de (Tobias Oberstein) Date: Fri, 7 Feb 2003 19:09:48 +0100 Subject: [pypy-dev] multiple isolated VMs in a single process Message-ID: Are there any design plans regarding threading architecture yet? I particular, are you planning to support having multiple interpreters in one process _without_ sharing objects (no free threading but VM-wise locking). Supporting this would require early design decisions like - no global data for the interpreter/VM - no global data in object implementations - but what about extension modules? many have global data .. isn't it? Also, in an C embedding API, will the VM-state be take from thread-local storage or passed into every function by the embedding application via an opaque pointer? I'm thinking of embedding a dynamic language into an multithreaded OODBMS. The different VMs within the server process would share data via objects instantiated from a special extension class that wraps up the OODBMS. Synchronisation on these objects is taken care of by the OODBMS. No need to share objects local to each VM. Sadly, as I was told, fixing above issues for CPython would be hard like shit. It's so sad that global data has not been avoided from the very beginning in the CPython. Tobias From hpk at trillke.net Fri Feb 7 19:49:59 2003 From: hpk at trillke.net (holger krekel) Date: Fri, 7 Feb 2003 19:49:59 +0100 Subject: [pypy-dev] multiple isolated VMs in a single process In-Reply-To: ; from Tobias.Oberstein@gmx.de on Fri, Feb 07, 2003 at 07:09:48PM +0100 References: Message-ID: <20030207194959.K1836@prim.han.de> [Tobias Oberstein Fri, Feb 07, 2003 at 07:09:48PM +0100] > Are there any design plans regarding threading architecture yet? good question. > I particular, are you planning to support having multiple > interpreters in one process _without_ sharing objects (no free > threading but VM-wise locking). Supporting this would require > early design decisions like > > - no global data for the interpreter/VM > - no global data in object implementations > - but what about extension modules? many have global data .. > > isn't it? Also, in an C embedding API, will the VM-state be > take from thread-local storage or passed into every function > by the embedding application via an opaque pointer? IMO we should defer this specific question to a later point. I don't expect a C-API to PyPython any time soon. We probably will go for a python-only system first. At least i wouldn't consider backward-compatibility to CPython's C-API a mandatory feature. We might as well run on ObjC for that matter :-) > I'm thinking of embedding a dynamic language into an > multithreaded OODBMS. The different VMs within the server process > would share data via objects instantiated from a special > extension class that wraps up the OODBMS. Synchronisation on > these objects is taken care of by the OODBMS. No need to share > objects local to each VM. A valid use case, IMO. although some people would say that you might use ZODB which does it with multiple processes. To me the biggest point for using processes is stability. It does come at a cost, though. You have to deal with "copies" of objects and only have "shared" objects at a higher level. This might suit your needs, though. > Sadly, as I was told, fixing above issues for CPython would > be hard like shit. It's so sad that global data has not been > avoided from the very beginning in the CPython. I am sure we will try to avoid global data structures. We are all good and nice programmers, aren't we? :-) Tobias, would you mind going to our issue-tracker http://codespeak.net/issues/pypy/ registering yourself and posting your "issue" as a "wishlist" item? This way we are sure not to loose valuable suggestions and use cases. Better yet, join the upcoming sprint right next door :-) thanks, holger From Tobias.Oberstein at gmx.de Fri Feb 7 21:11:30 2003 From: Tobias.Oberstein at gmx.de (Tobias Oberstein) Date: Fri, 7 Feb 2003 21:11:30 +0100 Subject: AW: [pypy-dev] multiple isolated VMs in a single process In-Reply-To: <20030207194959.K1836@prim.han.de> Message-ID: Hello Holger, > > Are there any design plans regarding threading architecture yet? > > good question. > > > I particular, are you planning to support having multiple > > interpreters in one process _without_ sharing objects (no free > > threading but VM-wise locking). Supporting this would require > > early design decisions like > > > > - no global data for the interpreter/VM > > - no global data in object implementations > > - but what about extension modules? many have global data .. > > > > isn't it? Also, in an C embedding API, will the VM-state be > > take from thread-local storage or passed into every function > > by the embedding application via an opaque pointer? > > IMO we should defer this specific question to a later point. I don't > expect a C-API to PyPython any time soon. We probably will go for > a python-only system first. At least i wouldn't consider > backward-compatibility to CPython's C-API a mandatory feature. Just my view: CPython API compatibility - I don't care. Any C API which exposes the VM - probably I would care. Think about it: a fast, compact, portable Python VM with good multithreading support and good integration capabilities could be a real competitor to CPython in the embedding world. On the other hand, how likely is it that PyPython becomes a threat to "stand-alone" Python any time soon? OK, I've just read a bit about performance stuff and PyCo - thats of course cool. Anyway, I submitted a "wish" item for this issue also;) http://codespeak.net/issues/pypy/issue4 > We might as well run on ObjC for that matter :-) > > > I'm thinking of embedding a dynamic language into an > > multithreaded OODBMS. The different VMs within the server process > > would share data via objects instantiated from a special > > extension class that wraps up the OODBMS. Synchronisation on > > these objects is taken care of by the OODBMS. No need to share > > objects local to each VM. > > A valid use case, IMO. although some people would say > that you might use ZODB which does it with multiple > processes. To me the biggest point for using processes > is stability. It does come at a cost, though. You have > to deal with "copies" of objects and only have "shared" > objects at a higher level. This might suit your needs, though. It imposes a big performance hit, since every call on an object from a persistable class must go over IPC, unless the object is cached in the process. But then again all issues of synchronization (be it pessimistic via locking or optimistic via multi-versioning) arise. I don't like it. Regarding the stability issue: agreed when speaking of components hacked in C/C++. But if I have components in Python, then I don't see any problems unless my VM is buggy of course. > > > Sadly, as I was told, fixing above issues for CPython would > > be hard like shit. It's so sad that global data has not been > > avoided from the very beginning in the CPython. > > I am sure we will try to avoid global data structures. > We are all good and nice programmers, aren't we? :-) > > Tobias, would you mind going to our issue-tracker > > http://codespeak.net/issues/pypy/ > > registering yourself and posting your "issue" as a > "wishlist" item? This way we are sure not to loose > valuable suggestions and use cases. Done: http://codespeak.net/issues/pypy/issue3 Greets, Tobias > > Better yet, join the upcoming sprint right next door :-) > > thanks, > > holger > From hpk at trillke.net Fri Feb 7 21:44:16 2003 From: hpk at trillke.net (holger krekel) Date: Fri, 7 Feb 2003 21:44:16 +0100 Subject: [pypy-dev] multiple isolated VMs in a single process In-Reply-To: ; from Tobias.Oberstein@gmx.de on Fri, Feb 07, 2003 at 09:11:30PM +0100 References: <20030207194959.K1836@prim.han.de> Message-ID: <20030207214416.N1836@prim.han.de> Hello Tobias, thanks for putting in the issues. some commments inline ... [Tobias Oberstein] > Just my view: CPython API compatibility - I don't care. Any C API > which exposes the VM - probably I would care. If you have a PyPy-VM-API then we want to have it stable which places a burden (but might be worth it). > Think about it: a fast, compact, portable Python VM with good multithreading > support and good integration capabilities could be a real competitor to > CPython > in the embedding world. On the other hand, how likely is it that PyPython > becomes > a threat to "stand-alone" Python any time soon? We are not yet at marketing issues :-) > > > I'm thinking of embedding a dynamic language into an > > > multithreaded OODBMS. The different VMs within the server process > > > would share data via objects instantiated from a special > > > extension class that wraps up the OODBMS. Synchronisation on > > > these objects is taken care of by the OODBMS. No need to share > > > objects local to each VM. > > > > A valid use case, IMO. although some people would say > > that you might use ZODB which does it with multiple > > processes. To me the biggest point for using processes > > is stability. It does come at a cost, though. You have > > to deal with "copies" of objects and only have "shared" > > objects at a higher level. This might suit your needs, though. > > It imposes a big performance hit, since every call on an object > from a persistable class must go over IPC, unless the object is > cached in the process. Which is the whole point of ZEO/RPC. ZODB doesn't offer an RPC mechanism between python objects (IIRC). > But then again all issues of synchronization > (be it pessimistic via locking or optimistic via multi-versioning) > arise. I don't like it. I see your point but there are locking issues with multithreading, too. > Regarding the stability issue: agreed when speaking of components > hacked in C/C++. But if I have components in Python, then I don't > see any problems unless my VM is buggy of course. sure but if there are IO problems in some tasks then it's nice to be able to 'signal' them in a clean way. Threads don't mix well with asynchronous signals. > > I am sure we will try to avoid global data structures. > > We are all good and nice programmers, aren't we? :-) > > > > Tobias, would you mind going to our issue-tracker > > > > http://codespeak.net/issues/pypy/ > > > > registering yourself and posting your "issue" as a > > "wishlist" item? This way we are sure not to loose > > valuable suggestions and use cases. > > Done: http://codespeak.net/issues/pypy/issue3 cool. thanks. But you didn't answer the Sprint question :-) i presume you can't make time on such a short notice. regards, holger From Tobias.Oberstein at gmx.de Fri Feb 7 22:33:11 2003 From: Tobias.Oberstein at gmx.de (Tobias Oberstein) Date: Fri, 7 Feb 2003 22:33:11 +0100 Subject: AW: [pypy-dev] multiple isolated VMs in a single process In-Reply-To: <20030207214416.N1836@prim.han.de> Message-ID: Hello Holger, > > > > I'm thinking of embedding a dynamic language into an > > > > multithreaded OODBMS. The different VMs within the server process > > > > would share data via objects instantiated from a special > > > > extension class that wraps up the OODBMS. Synchronisation on > > > > these objects is taken care of by the OODBMS. No need to share > > > > objects local to each VM. > > > > > > A valid use case, IMO. although some people would say > > > that you might use ZODB which does it with multiple > > > processes. To me the biggest point for using processes > > > is stability. It does come at a cost, though. You have > > > to deal with "copies" of objects and only have "shared" > > > objects at a higher level. This might suit your needs, though. > > > > It imposes a big performance hit, since every call on an object > > from a persistable class must go over IPC, unless the object is > > cached in the process. > > Which is the whole point of ZEO/RPC. ZODB doesn't offer an > RPC mechanism between python objects (IIRC). > > > But then again all issues of synchronization > > (be it pessimistic via locking or optimistic via multi-versioning) > > arise. I don't like it. > > I see your point but there are locking issues with multithreading, > too. Yes, but it's solved (hopefully) by a robust OODBMS. By the way, speaking of "robust", probably I should just reveal a bit of background info why I'm so nuts about this stuff (it's not strictly on topic and got long, but eh ..): In 2000, SAP AG open-sourced a enterprise grade RDBMS (it runs R/3 so it's no toy stuff): http://www.sapdb.org What few people know is the fact that SAP also (yet unofficially) released the "LiveCache"-technology as open-source. LiveCache is a combination of an OODBMS integrated on top of SAP DB (which makes SAP DB a _hybrid_ database) plus a COM-like run-time environment that makes it possible to call components implemented in C++ from database client sessions just like any other stored procedure. Components are run within the address space (in-process) of LiveCache (which is multi-threaded) for reason of maximum performance. SAP currently ship LiveCache as the basis for SAP-APO, which is part of SAP's supply-chain mgt. system. They developed it, because APO must perform in real-time. Ah, and there are LiveCache installations on SMPs with 64 GB Ram and > 20 CPUs. I hope this all explains my interest: "Python embedded in LiveCache" I bet this could be a real killer app! Real enterprise grade integrated application server/OODBMS with modern high-level dynamic language. Btw: I'm NOT employed by SAP or in any way related with them. > > > Regarding the stability issue: agreed when speaking of components > > hacked in C/C++. But if I have components in Python, then I don't > > see any problems unless my VM is buggy of course. > > sure but if there are IO problems in some tasks then it's nice > to be able to 'signal' them in a clean way. Threads don't > mix well with asynchronous signals. The only IO I'd like to do is via persisted objects. It's pretty much a "sandbox"-Python I'm thinking of. More radically: "import" should look in the database for code, not in the filesystem. Maybe I'm better off taking a minimal subset of CPython and tweaking it until it fits. > But you didn't answer the Sprint question :-) > i presume you can't make time on such a short notice. Problem is, instead of writing postings I should learn for the tests I have to do in the coming 2 months (I'm studying math) .. well, I'm an addict geek. Anyway, PyPy is way cool and I guess the Sprint will be great! Greets, Tobias From troy at gci.net Sun Feb 9 09:03:46 2003 From: troy at gci.net (Troy Melhase) Date: Sat, 08 Feb 2003 23:03:46 -0900 Subject: [pypy-dev] pypy-dev List Summary - 01 Feb 2003 to 08 Feb 2003 Message-ID: <200302082303.46292.troy@gci.net> This is a summary of the discussions on the pypy-dev mailing list between 01 February and 08 February 2003. The full archive of the list can be found here: http://codespeak.net/pipermail/pypy-dev/ Compiler Benchmarks ------------------- Logistix posts benchmarks for a pure Python compile module: http://codespeak.net/pipermail/pypy-dev/2003q1/000374.html Oren Tirosh suggests running the same benchmarks with Psyco. Holger Krekel reminds us that the Psyco used in the Minimal project will be a very different beast. Christian Tismer agrees that current performance gains are not indicators of what is possible. Logistix later posts a link to his code, and provides some details: http://codespeak.net/pipermail/pypy-dev/2003q1/000383.html PyMite ------ Dean Hall introduces the PyMite interpreter, "my from-scratch flyweight python interpreter for 8-bit architectures." The source is not available yet, but will be before his presentation at PyCon2003: http://codespeak.net/pipermail/pypy-dev/2003q1/000390.html Web Site -------- Holger sends a note that the project web site is ready for use: http://codespeak.net/pipermail/pypy-dev/2003q1/000399.html The site currently focuses on the upcoming sprint, and has the obligatory wiki and list summaries. The site link is: http://codespeak.net/pypy Multiple, Isolated VMs in a Single Process --------------------------------------- Tobias Oberstein ask about design plans for a threading architecture. He asks out of interest for having multiple interpreters within a process without the need for shared objects. Tobias describes what he sees as design decisions: http://codespeak.net/pipermail/pypy-dev/2003q1/000406.html Holger believes the question should be deferred until the initial PyPython interpreter has a C API, and asks Tobias to create a wish list item: http://codespeak.net/pipermail/pypy-dev/2003q1/000407.html Minimally, -troy From arigo at tunes.org Sun Feb 9 19:40:37 2003 From: arigo at tunes.org (Armin Rigo) Date: Sun, 9 Feb 2003 10:40:37 -0800 (PST) Subject: [pypy-dev] Wiki content Message-ID: <20030209184037.86F2D1F37@bespin.org> Hello everybody, I have put some content into the Wiki. It is some kind of a technical summary of all the aspects we discussed so far -- a very subjective one, anyway. Any different opinion is of course welcome. And if I missed your pet subject, I give you my apologizes (but after you receive them, go ahead and add it to the Wiki :-) http://codespeak.net/moin/pypy/moin.cgi/PyPythonProjects I inserted a couple of links to my own experiments in the Wiki pages: * a Python-interpreter-in-Python toy that I used as bytecode checker http://codespeak.net/moin/pypy/moin.cgi/BytecodeInterpreter * a custom (incomplete) C parser http://codespeak.net/moin/pypy/moin.cgi/LibraryMaintenance * an x86 assembler/disassembler http://codespeak.net/moin/pypy/moin.cgi/RuntimeSystemJit A bient?t, Armin. From Tobias.Oberstein at gmx.de Sun Feb 9 20:28:44 2003 From: Tobias.Oberstein at gmx.de (Tobias Oberstein) Date: Sun, 9 Feb 2003 20:28:44 +0100 Subject: [pypy-dev] reusing existing work: Parrot In-Reply-To: Message-ID: more by accidence, I came across the following stuff http://www.parrotcode.org which might be of interest when it comes to - reusing existing work in PyPy - focusing PyPy (whatever this means) and concentrating manpower .. Parrot will be the new VM written from scratch for Perl6. The developers claim that its generally suitable for dynamically typed languages - which fits Python as well. Also it is designed as an abstract register machine, which they claim will provide better performance. Btw.: JVMs and .NET CLI are stack-based VMs optimized for statically typed languages. Regarding targeting Parrot from Python, there has been some work done: http://www.amk.ca/conceit/parrot.html As I understand it, the main thing lacking is a "port" of Python objects (dicts,..) to Parrot, where there are "PMCs" which are very similar to PyObject. I have no stakes in either PyPy or Parrot - it's just my two pence that it could pay to not reinvent another VM, but port as much modules/objects to pure Python (which is all about PyPy as I understood) and write a Python->Parott Bytecode emitter plus complement with appropriate PMCs. Greets, Tobias From lalo at laranja.org Mon Feb 10 15:58:38 2003 From: lalo at laranja.org (Lalo Martins) Date: Mon, 10 Feb 2003 12:58:38 -0200 Subject: [pypy-dev] reusing existing work: Parrot In-Reply-To: References: Message-ID: <20030210145838.GB17348@laranja.org> Sorry, but quoting a bad-taste sticker, Parrot is as useful to Pypy as a bycicle is useful to a fish. Pypy is *not* about porting modules to pure-python, it's about having the *whole* implementation, including VM, written in pure-python. (You know you're answering a FAQ when the message is smaller than your signature...) []s, |alo +---- -- Those who trade freedom for security lose both and deserve neither. -- http://www.laranja.org/ mailto:lalo at laranja.org pgp key: http://www.laranja.org/pessoal/pgp Eu jogo RPG! (I play RPG) http://www.eujogorpg.com.br/ GNU: never give up freedom http://www.gnu.org/ From jeremy at zope.com Mon Feb 10 21:28:05 2003 From: jeremy at zope.com (Jeremy Hylton) Date: 10 Feb 2003 15:28:05 -0500 Subject: AW: [pypy-dev] multiple isolated VMs in a single process In-Reply-To: References: Message-ID: <1044908885.19408.4.camel@slothrop.zope.com> On Fri, 2003-02-07 at 16:33, Tobias Oberstein wrote: > > > > A valid use case, IMO. although some people would say > > > > that you might use ZODB which does it with multiple > > > > processes. To me the biggest point for using processes > > > > is stability. It does come at a cost, though. You have > > > > to deal with "copies" of objects and only have "shared" > > > > objects at a higher level. This might suit your needs, though. This is a bit off-topic, but I wanted to clarify about ZODB. ZODB does allow a multi-threaded application to use multiple database connections. Each connection is associated with a thread and gets in independent / isolated view of the database. This approach has the costs you mention -- multiple copies of objects in particular -- but sharing is a bit cheaper because method calls replace IPC. Jeremy From Tobias.Oberstein at gmx.de Tue Feb 11 00:00:46 2003 From: Tobias.Oberstein at gmx.de (Tobias Oberstein) Date: Tue, 11 Feb 2003 00:00:46 +0100 Subject: AW: AW: [pypy-dev] multiple isolated VMs in a single process In-Reply-To: <1044908885.19408.4.camel@slothrop.zope.com> Message-ID: > On Fri, 2003-02-07 at 16:33, Tobias Oberstein wrote: > > > > > A valid use case, IMO. although some people would say > > > > > that you might use ZODB which does it with multiple > > > > > processes. To me the biggest point for using processes > > > > > is stability. It does come at a cost, though. You have > > > > > to deal with "copies" of objects and only have "shared" > > > > > objects at a higher level. This might suit your needs, though. > > This is a bit off-topic, but I wanted to clarify about ZODB. ZODB does > allow a multi-threaded application to use multiple database > connections. Each connection is associated with a thread and gets in > independent / isolated view of the database. Just to be sure: last time I read into ZODB I learned that ZODB does multi-versioning with optimistic concurrency control at the object level. Associated with every database connection is an object cache (to provide the isolated view). So on object access, data has to be copied (in-process) from the storage core to the object cache of the specific database connection. > This approach has the > costs you mention -- multiple copies of objects in particular -- but > sharing is a bit cheaper because method calls replace IPC. Right, but if you want to scale on SMPs (which, I admit, is not the most common case and certainly debatable on its own) situation, the often heard advice is to use ZEO, which i) requires some form of IPC (for ZODB<->ZEO, is there anything like shared-memory connection? It uses TCP/IP, even if both ZEO and ZODB's are on the same host, isn't it?) ii) uses broadcasts from ZEO to all the ZODBs to invalidate touched objects, which doesn't scale for write intensive apps If one does not use ZEO, but runs ZODB as a single process multi-threaded process on the other hand, scaling on SMPs will be limited by the global interpreter lock of CPython. Of course, this is an implementation issue with CPython. Fixing the GIL issue with CPython once and forever would not be too hard (see attached idea sketch). However as I was told, fixing the GIL issue would have to get enough votes to become core. I'm not feeling in a position to collect them in the Python community .. guess I'm an outsider on all measures;) So I moved on looking for other means. Enter PyPy. There's a chance that PyPy does not repeat design flaws like relying heavily on global state and stuff. Great. Currently I'm even considering completely different VMs like from the Mono project or from the Portable.NET project. I want to intergrate a VM with a OODBMS and do that right. That's _my_ focus. Tobias -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: PyWorld_PEP.txt URL: From pyth at devel.trillke.net Tue Feb 11 16:11:10 2003 From: pyth at devel.trillke.net (holger krekel) Date: Tue, 11 Feb 2003 16:11:10 +0100 Subject: [pypy-dev] Scons? Message-ID: <20030211161110.E1706@prim.han.de> Hello, i have been following the SCons effort for quite some time without seriously deploying it. I know that Skip Montanero did some tests with the CPython-sources and said he liked it. Is anybody here on the list who knows enough about SCons to tell what the current state of it is? And if it would suite the CPython-build process? Anybody tried anything? If we want to do it i imagine that we could contribute it to CPython if there is interest. Yes, i know there is a bootstrapping problem (as always) but that's secondary IMO. holger From tismer at tismer.com Wed Feb 12 00:51:09 2003 From: tismer at tismer.com (Christian Tismer) Date: Wed, 12 Feb 2003 00:51:09 +0100 Subject: [pypy-dev] reusing existing work: Parrot In-Reply-To: <20030210145838.GB17348@laranja.org> References: <20030210145838.GB17348@laranja.org> Message-ID: <3E498C6D.8040002@tismer.com> Lalo Martins wrote: > Sorry, but quoting a bad-taste sticker, Parrot is as useful to Pypy as a > bycicle is useful to a fish. True, but for a different reason: We can't target at a VM that isn't readily developed. So if we have to fix the bicycle very much before biking, we might consider to build our own unicycle, instead. :-) > Pypy is *not* about porting modules to > pure-python, it's about having the *whole* implementation, including VM, > written in pure-python. This is right, but not the full story. In the end, we need the little hamster who is driving the wheel. This can be a small VM in C, generated assembly, generated C code, ..., anything that finally actually does the computation. We do not intend to depend on an existing VM, after all, but this will be self-contained. However you look at it, there will be at least a small C kernel for bootstrapping, at first. Maybe that will vanish, too, given that we also code a code generator in Python. More to come on the sprint. Need to talk. ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From vlindberg at verio.net Wed Feb 12 01:00:27 2003 From: vlindberg at verio.net (VanL) Date: Tue, 11 Feb 2003 17:00:27 -0700 Subject: [pypy-dev] Updates from the sprint? Message-ID: <3E498E9B.9040002@verio.net> For those of us who are unable to get to Germany, is there any way to get updates from the sprint? These don't have to be cooked or processed -- we can do that, and the sprinters can sprint. But being able to follow what goes on would be very interesting. If I could get all I would wish for, we could get: 1. Some sort of recording of the talks given each evening. 2. Presentation materials 3. The code snapshot from the previous night. Would this be possible? Van From tismer at tismer.com Wed Feb 12 01:38:05 2003 From: tismer at tismer.com (Christian Tismer) Date: Wed, 12 Feb 2003 01:38:05 +0100 Subject: [pypy-dev] Updates from the sprint? In-Reply-To: <3E498E9B.9040002@verio.net> References: <3E498E9B.9040002@verio.net> Message-ID: <3E49976D.6050003@tismer.com> VanL wrote: > For those of us who are unable to get to Germany, is there any way to > get updates from the sprint? These don't have to be cooked or processed > -- we can do that, and the sprinters can sprint. But being able to > follow what goes on would be very interesting. I think we are going to put a daily update into the Wiki. Dinu Gherman is also probably writing an article, which might have an outlet for those not in the sprint. > If I could get all I would wish for, we could get: > > 1. Some sort of recording of the talks given each evening. > 2. Presentation materials > 3. The code snapshot from the previous night. > > Would this be possible? I think so. Making the materials available should be no problem. The code will be put into the versioning system every day. This will probably be public read, or mirrored if bandwidth is an issue. Thanks for the suggestion! Webcams-might-be-overdone - ly y'rs -- chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From bokr at oz.net Wed Feb 12 07:28:32 2003 From: bokr at oz.net (Bengt Richter) Date: Tue, 11 Feb 2003 22:28:32 -0800 Subject: [pypy-dev] Updates from the sprint? In-Reply-To: <3E498E9B.9040002@verio.net> Message-ID: <5.0.2.1.1.20030211222215.00a675f0@mail.oz.net> At 17:00 2003-02-11 -0700, VanL wrote: >For those of us who are unable to get to Germany, is there any way to get updates from the sprint? These don't have to be cooked or processed -- we can do that, and the sprinters can sprint. But being able to follow what goes on would be very interesting. > >If I could get all I would wish for, we could get: > >1. Some sort of recording of the talks given each evening. mp3's would be neat. (Remind people to say their names when they speak, so we can tell). It wouldn't have to be edited much. >2. Presentation materials >3. The code snapshot from the previous night. > >Would this be possible? I hope so ;-) Regards, Bengt Richter From tanzer at swing.co.at Wed Feb 12 07:22:34 2003 From: tanzer at swing.co.at (Christian Tanzer) Date: Wed, 12 Feb 2003 07:22:34 +0100 Subject: [pypy-dev] Scons? In-Reply-To: Your message of "Tue, 11 Feb 2003 16:11:10 +0100." <20030211161110.E1706@prim.han.de> Message-ID: holger krekel wrote: > i have been following the SCons effort for quite some > time without seriously deploying it. I know that > Skip Montanero did some tests with the CPython-sources > and said he liked it. > > Is anybody here on the list who knows enough about > SCons to tell what the current state of it is? We have been using SCons instead of Make for a while now and been quite satisfied. Works great for compiling C/C++, running LaTeX, using McMillan's installer, and doing other stuff. > And if it would suite the CPython-build process? I don't see why not. Haven't used it for that, though. -- Christian Tanzer tanzer at swing.co.at Glasauergasse 32 Tel: +43 1 876 62 36 A-1130 Vienna, Austria Fax: +43 1 877 66 92 From gherman at darwin.in-berlin.de Wed Feb 12 09:37:18 2003 From: gherman at darwin.in-berlin.de (Dinu Gherman) Date: Wed, 12 Feb 2003 09:37:18 +0100 Subject: [pypy-dev] Updates from the sprint? In-Reply-To: <5.0.2.1.1.20030211222215.00a675f0@mail.oz.net> Message-ID: <327343FB-3E65-11D7-BF45-00039345C610@darwin.in-berlin.de> Bengt Richter: > mp3's would be neat. (Remind people to say their names when they > speak, so we can tell). It wouldn't have to be edited much. Or live AV-streaming, maybe? :-) Dinu -- Dinu C. Gherman ...................................................................... "It is by the goodness of God that in our country we have these three unspeakably precious things: freedom of speech, freedom of conscience, and the prudence to practice neither." (Mark Twain) From hpk at trillke.net Wed Feb 12 12:33:33 2003 From: hpk at trillke.net (holger krekel) Date: Wed, 12 Feb 2003 12:33:33 +0100 Subject: [pypy-dev] Updates from the sprint? In-Reply-To: <3E498E9B.9040002@verio.net>; from vlindberg@verio.net on Tue, Feb 11, 2003 at 05:00:27PM -0700 References: <3E498E9B.9040002@verio.net> Message-ID: <20030212123332.W3033@prim.han.de> [VanL Tue, Feb 11, 2003 at 05:00:27PM -0700] > For those of us who are unable to get to Germany, is there any way to > get updates from the sprint? These don't have to be cooked or processed > -- we can do that, and the sprinters can sprint. But being able to > follow what goes on would be very interesting. > > If I could get all I would wish for, we could get: > > 1. Some sort of recording of the talks given each evening. > 2. Presentation materials > 3. The code snapshot from the previous night. > > Would this be possible? maybe. I am not sure i can organize this up-front but if some of the participants are willing to do it, sure. regards, holger From steven_shaw at iprimus.com.au Wed Feb 12 16:12:41 2003 From: steven_shaw at iprimus.com.au (Steven Shaw) Date: Thu, 13 Feb 2003 01:12:41 +1000 Subject: [pypy-dev] generating C References: <20030210145838.GB17348@laranja.org> <3E498C6D.8040002@tismer.com> Message-ID: <040b01c2d2a9$30da15e0$9e3832d2@z> Why not generate C code, compile it and dynamically load it? This is the technique used for Goo http://googoogaga.org/ Pretty sure this technique is used for gcl, too. http://www.gnu.org/software/gcl/ This allows you to take advantage of the best C compiler optimisations on the platform you are targeting. Also, it means you don't have to have a bunch of jitter guys who try to keep up with the breadth of cpus and their evolution. When I was comparing CL implementations recently, I thought that clisp or cmucl would be the best (free ones) because they have native compilation. However, I found that cmucl didn't have support for PIII or PIV. Therefore gcl which generates C seemed like the best option. Steve. From tismer at tismer.com Wed Feb 12 17:28:39 2003 From: tismer at tismer.com (Christian Tismer) Date: Wed, 12 Feb 2003 17:28:39 +0100 Subject: [pypy-dev] generating C In-Reply-To: <040b01c2d2a9$30da15e0$9e3832d2@z> References: <20030210145838.GB17348@laranja.org> <3E498C6D.8040002@tismer.com> <040b01c2d2a9$30da15e0$9e3832d2@z> Message-ID: <3E4A7637.1070407@tismer.com> Steven Shaw wrote: > Why not generate C code, compile it and dynamically load it? I expect that you have read the relevant posts, and you know that one target is C code generation. So the question is about the "dynamic", right? > This is the technique used for Goo http://googoogaga.org/ > Pretty sure this technique is used for gcl, too. > http://www.gnu.org/software/gcl/ > > This allows you to take advantage of the best C compiler optimisations on > the platform you are targeting. Also, it means you don't have to have a > bunch of jitter guys who try to keep up with the breadth of cpus and their > evolution. When I was comparing CL implementations recently, I thought that > clisp or cmucl would be the best (free ones) because they have native > compilation. However, I found that cmucl didn't have support for PIII or > PIV. Therefore gcl which generates C seemed like the best option. I think this is an additional patho to go, although firing a compiler and creating a dll dynamically might be not as tiny and quick as directly generating machine code, like Psyco does today. I think to put something into the Wiki about code generation targets. ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From lac at strakt.com Wed Feb 12 20:47:09 2003 From: lac at strakt.com (Laura Creighton) Date: Wed, 12 Feb 2003 20:47:09 +0100 Subject: [pypy-dev] Is anybody planning to give a talk on PyPy for OSCON? Message-ID: <200302121947.h1CJl9dM021777@ratthing-b246.strakt.com> This is precisely the sort of thing that Nat Tarkington said he wanted for the Python track -- new community efforts, new implementations. Who is going to OSCON here besides me? Laura Creighton From tismer at tismer.com Thu Feb 13 03:07:50 2003 From: tismer at tismer.com (Christian Tismer) Date: Thu, 13 Feb 2003 03:07:50 +0100 Subject: [pypy-dev] Is anybody planning to give a talk on PyPy for OSCON? In-Reply-To: <200302121947.h1CJl9dM021777@ratthing-b246.strakt.com> References: <200302121947.h1CJl9dM021777@ratthing-b246.strakt.com> Message-ID: <3E4AFDF6.3020209@tismer.com> Laura Creighton wrote: > This is precisely the sort of thing that Nat Tarkington said > he wanted for the Python track -- new community efforts, new > implementations. > > Who is going to OSCON here besides me? I'm to PyCon already, cannot do them both. -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From lac at strakt.com Thu Feb 13 08:43:56 2003 From: lac at strakt.com (Laura Creighton) Date: Thu, 13 Feb 2003 08:43:56 +0100 Subject: [pypy-dev] Is anybody planning to give a talk on PyPy for OSCON? In-Reply-To: Message from Christian Tismer of "Thu, 13 Feb 2003 03:07:50 +0100." <3E4AFDF6.3020209@tismer.com> References: <200302121947.h1CJl9dM021777@ratthing-b246.strakt.com> <3E4AFDF6.3020209@tismer.com> Message-ID: <200302130743.h1D7hujL025068@ratthing-b246.strakt.com> In a message of Thu, 13 Feb 2003 03:07:50 +0100, Christian Tismer writes: >Laura Creighton wrote: >> This is precisely the sort of thing that Nat Tarkington said >> he wanted for the Python track -- new community efforts, new >> implementations. >> >> Who is going to OSCON here besides me? > >I'm to PyCon already, cannot do them both. > >-- >Christian Tismer :^) >Mission Impossible 5oftware : Have a break! Take a ride on Python's >Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ >14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ >work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 >PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 > whom do you want to sponsor today? http://www.stackless.com/ > Okay, does anybody mind if I submit a proposal to OSCON, 'a report on the Sprint and the project'? If nobody minds, anything you want to make sure I put in the proposal? Laura From chux at houston.rr.com Thu Feb 13 16:41:11 2003 From: chux at houston.rr.com (Charles Crain) Date: Thu, 13 Feb 2003 09:41:11 -0600 Subject: [pypy-dev] Scons? Message-ID: <000001c2d376$5ce37f50$1401a8c0@technobill> (in response to Holger Krekel's question about SCons) I am on the SCons development team, and can answer any questions you have about using SCons for PyPython. It's still "technically" in Alpha, which by our definition simply means it is not feature complete. However, the features it does have are extremely stable and well-tested...we have a very large automated test suite, a rigorous testing process, and a growing user community. We make releases about once a month (we try to anyway), and we are theoretically only a handful of releases away from a first final version. We'd be very entusiastic about being included in a high-profile (and technically really cool) project like PyPython. Please let us know if we can do anything for you. Feel free to email me or post to the scons-users email list (information at www.scons.org). -Charles From arigo at tunes.org Thu Feb 13 17:31:20 2003 From: arigo at tunes.org (Armin Rigo) Date: Thu, 13 Feb 2003 17:31:20 +0100 Subject: [pypy-dev] Object model Message-ID: <20030213163120.GA31338@magma.unil.ch> Hello everybody, I have been experimenting a bit about that distinction I tried to draw between "application-level" and "interpreter-level" objects. I still think the idea to be essential but I "reorganized" my thoughts a little bit. Now I believe in following the idea that CPython's main loop does not assume anything at all about the PyObjects it manipulates, but does every operation via calls to library functions like PyNumber_Add(). All it does is moving PyObjects around, mainly between the stack and the locals. Similarily, our own main loop should not assume anything about the objects it stores in the stack and the locals, not even that they have some well-known methods. Instead, we would provide it with the equivalent of PyNumber_Add() & co. Now the fun part of it is that there is no reason to use only one fixed set of functions to do the operations. There are several things we want to be able to do with our interpreter, and these will probably correspond to different ways of "seeing" what an object is. We might try several ways to implement the objects, for example trying different implementations for what the application will always see as a list. And even, we might want to have several concurrent "object spaces" existing at the same time, e.g. to implement the idea of multiple VMs in the same process. Thus it seems natural to define a class ObjectSpace, that defines methods like PyNumber_Add() and PySequence_GetItem(). Each frame would have an ''objectspace'' attribute that holds the space in which it runs. (Maybe shorter, more Pythonic names are better, e.g. add() and getitem().) Thus an object space is: * a way to implement objects; * from the interpreter's point of view, it is the "library" needed to do any operation; * from the application's point of view, it is essentially invisible. Methods: * add(x,y), getitem(x,y), etc. for the operations, that take two "objects" (black boxes for the interpreter) and returns an "object" (black box too). * type(x), taking a black box object and returning a black box object as well. * wrap(a), taking a *normal* object and putting it into a black box. Its what the interpreter does for a LOAD_CONST, for example: it has a real Python object and wants to send it to the object space. For example, if the object space has a custom implementation "class MyList" for lists, then wrap([1,2,3]) should create a MyList instance. * unwrap(x) is the inverse operation. Used by the interpreter in the rare cases where it really needs to observe the object. For example, for a conditional jump, after it obtained the truth-value with (say) a call to the truth() method, it must pull the result out of the object space to know whether it is really False or True. For example, the straightforward object space that just uses real Python objects to implement themselves would be defined like that: class BorrowingObjectSpace(ObjectSpace): def wrap(self, x): return x def unwrap(self, x): return x def add(self, x, y): return x+y def type(self, x): return type(x) ... Cool things can be imagined with other kinds of object spaces. We may even think about a distributed interpreter, running on one machine with object spaces that actually reside on other machines! With this perspective, wrap(x) sends x to the remote machine and returns a handle to it, and unwrap(x) downloads from the remote machine the object whose handle is x. We may even have several concurrently-running object spaces that could communicate, e.g. running a frame in a different object space and marshalling arguments and the return value! Looks like a cool point of view, doesn't it ? A bient?t, Armin. From hpk at trillke.net Thu Feb 13 17:56:15 2003 From: hpk at trillke.net (holger krekel) Date: Thu, 13 Feb 2003 17:56:15 +0100 Subject: [pypy-dev] Scons? In-Reply-To: <000001c2d376$5ce37f50$1401a8c0@technobill>; from chux@houston.rr.com on Thu, Feb 13, 2003 at 09:41:11AM -0600 References: <000001c2d376$5ce37f50$1401a8c0@technobill> Message-ID: <20030213175615.F3033@prim.han.de> Hello Charles, [Charles Crain Thu, Feb 13, 2003 at 09:41:11AM -0600] > (in response to Holger Krekel's question about SCons) > > I am on the SCons development team, and can answer any questions you > have about using SCons for PyPython. It's still "technically" in Alpha, > which by our definition simply means it is not feature complete. > However, the features it does have are extremely stable and > well-tested...we have a very large automated test suite, a rigorous > testing process, and a growing user community. that sounds great. > We make releases about once a month (we try to anyway), and we are > theoretically only a handful of releases away from a first final > version. We probably need a couple of more releases for our first final :-) > We'd be very entusiastic about being included in a high-profile (and > technically really cool) project like PyPython. Please let us know if > we can do anything for you. Feel free to email me or post to the > scons-users email list (information at www.scons.org). I suggest that some sprint-people try to supplant SCons on current CPython with our own architecture in mind. I haven't really planned details yet because i am too busy in physical organization. However, it would be very good if some of the SCons and PyPy people could communicate next week to help if there are problems. Thanks a lot for your offer and cooperation! As we plan to run on top of CPython first it makes a lot of sense to have SCons drive the CPython build and thus get rid of make/configure dependencies. It's also a good release goal for our sprint. The real challenge here is to have a scheme for taking current CPython-cvs as an "upstream" and have a system for keeping in sync what we take out of CPython into our CPython-PyPy. cheers, holger From mwh at python.net Thu Feb 13 17:51:32 2003 From: mwh at python.net (Michael Hudson) Date: Thu, 13 Feb 2003 16:51:32 +0000 Subject: [pypy-dev] Re: Object model References: <20030213163120.GA31338@magma.unil.ch> Message-ID: <2mof5gazu3.fsf@starship.python.net> Armin Rigo writes: [cool stuff] > class BorrowingObjectSpace(ObjectSpace): > > def wrap(self, x): > return x > > def unwrap(self, x): > return x > > def add(self, x, y): > return x+y > > def type(self, x): > return type(x) "from operator import *" could go here? [...] > Looks like a cool point of view, doesn't it ? Yes! Brain a bit too fried to be sure, but I think so, anyway. Cheers, M. -- SPIDER: 'Scuse me. [scuttles off] ZAPHOD: One huge spider. FORD: Polite though. -- The Hitch-Hikers Guide to the Galaxy, Episode 11 From vlindberg at verio.net Thu Feb 13 18:48:46 2003 From: vlindberg at verio.net (VanL) Date: Thu, 13 Feb 2003 10:48:46 -0700 Subject: [pypy-dev] Object model References: <20030213163120.GA31338@magma.unil.ch> Message-ID: <3E4BDA7E.5060409@verio.net> I am trying to make sure I understand this correctly... Please bear with me if am am being slow. I come from more of a hardware than software background, so I am just tring to apply the concepts to what I know. I think you are envisioning a model interpreter that works (more or less) like the CPU works in a computer. CPUs, especially RISCy ones, usually work with essentially three things: 1. Operators (like iadd, fadd, imul, fdiv, or, xor, etc) 2. Registers 3. Immediate values In normal operation, the CPU has no idea what the values in two registers refer to, all it knows is that it has been told to add the contents of register A with register B and store the result in register C. I see this as analogous to your "black box" concept: The registers/pyobjects are the black boxes and all the main loop has to do is stuff the black boxes into the correct functional units as fast as it can. The immediate values of a CPU would be analogous to your unwrapped objects -- if I understand, you would wrap them in a black box before stuffing them into the functional units for purposes of making the functional units more specialized. If this is a correct interpretation, then that would suggest that a lot of the traditional "hardware" optimization techniques might be able to be used. However, the compiler would also need to be pretty smart to correctly order instructions. It also might suggest a bootstrapping procedure: 1. Start with the bytecode, rather than the compiler. Treat the byte code as "machine code" that we execute and CPython as the compiler. 2. Put together the load and dispatch logic in python. There may need to be a function decomposition step, where "big" bytecode instructions are split apart into simpler, single-step functions. Create the "functional units" so that initially they use CPython to actually perform the operation. 3. Port over the functional units so that they create and execute machine code, rather than relying on CPython to do the work. When all the functional units are ported over, PyPy v 1.0 is complete. This also would allow things like trace caches and conditional execution.... Using a register-based python vm might also speed the way to a native-code compiler. However, I thing that this could be made quite fast. Once again, pardon me if this is a wild misinterpretation of what you were saying; as I said, I am a hardware guy who is just starting to get into compiler theory. VanL Armin Rigo wrote: >Hello everybody, > >I have been experimenting a bit about that distinction I tried to draw between >"application-level" and "interpreter-level" objects. I still think the idea >to be essential but I "reorganized" my thoughts a little bit. > >Now I believe in following the idea that CPython's main loop does not assume >anything at all about the PyObjects it manipulates, but does every operation >via calls to library functions like PyNumber_Add(). All it does is moving >PyObjects around, mainly between the stack and the locals. Similarily, our >own main loop should not assume anything about the objects it stores in the >stack and the locals, not even that they have some well-known methods. >Instead, we would provide it with the equivalent of PyNumber_Add() & co. > >Now the fun part of it is that there is no reason to use only one fixed set of >functions to do the operations. There are several things we want to be able >to do with our interpreter, and these will probably correspond to different >ways of "seeing" what an object is. We might try several ways to implement >the objects, for example trying different implementations for what the >application will always see as a list. And even, we might want to have >several concurrent "object spaces" existing at the same time, e.g. to >implement the idea of multiple VMs in the same process. > >Thus it seems natural to define a class ObjectSpace, that defines methods like >PyNumber_Add() and PySequence_GetItem(). Each frame would have an >''objectspace'' attribute that holds the space in which it runs. (Maybe >shorter, more Pythonic names are better, e.g. add() and getitem().) > >Thus an object space is: > > * a way to implement objects; > > * from the interpreter's point of view, it is the "library" needed to do any >operation; > > * from the application's point of view, it is essentially invisible. > >Methods: > > * add(x,y), getitem(x,y), etc. for the operations, that take two "objects" >(black boxes for the interpreter) and returns an "object" (black box too). > > * type(x), taking a black box object and returning a black box object as >well. > > * wrap(a), taking a *normal* object and putting it into a black box. Its >what the interpreter does for a LOAD_CONST, for example: it has a real Python >object and wants to send it to the object space. For example, if the object >space has a custom implementation "class MyList" for lists, then wrap([1,2,3]) >should create a MyList instance. > > * unwrap(x) is the inverse operation. Used by the interpreter in the rare >cases where it really needs to observe the object. For example, for a >conditional jump, after it obtained the truth-value with (say) a call to the >truth() method, it must pull the result out of the object space to know >whether it is really False or True. > >For example, the straightforward object space that just uses real Python >objects to implement themselves would be defined like that: > >class BorrowingObjectSpace(ObjectSpace): > > def wrap(self, x): > return x > > def unwrap(self, x): > return x > > def add(self, x, y): > return x+y > > def type(self, x): > return type(x) > ... > >Cool things can be imagined with other kinds of object spaces. We may even >think about a distributed interpreter, running on one machine with object >spaces that actually reside on other machines! With this perspective, wrap(x) >sends x to the remote machine and returns a handle to it, and unwrap(x) >downloads from the remote machine the object whose handle is x. > >We may even have several concurrently-running object spaces that could >communicate, e.g. running a frame in a different object space and marshalling >arguments and the return value! > > >Looks like a cool point of view, doesn't it ? > > >A bient?t, > >Armin. >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev > > From logistix at zworg.com Thu Feb 13 20:46:45 2003 From: logistix at zworg.com (logistix) Date: Thu, 13 Feb 2003 11:46:45 -0800 Subject: [pypy-dev] Object model Message-ID: <8893.1045165605@zworg.com> VanL wrote: > I think you are envisioning a model interpreter that works (more or > less) like the CPU works in a computer. > > CPUs, especially RISCy ones, usually work with essentially three things: > > 1. Operators (like iadd, fadd, imul, fdiv, or, xor, etc) > 2. Registers > 3. Immediate values > > > It also might suggest a bootstrapping procedure: > > 1. Start with the bytecode, rather than the compiler. Treat the byte > code as "machine code" that we execute and CPython as the compiler. A lot of the stuff he's talking about here is based on the existing Python Interpreter. It runs stack-based bytecode, where the stack simply contains pointers to PyObjects. It's also a little wierd because it turns functions and stuff into code objects that get thrown on the stack instead of trying to represent everything as a monolithic block of memory where you would JSR to a memory address to call a function. Then these code objects can have their own embedded code objects. Since you seem to have a good low-level understanding of things, checking out the documentation for the 'dis' module will show you all the existing "assembly" instructions. ceval.c has the actual implementation of the interpreter loop. From hpk at trillke.net Thu Feb 13 21:32:30 2003 From: hpk at trillke.net (holger krekel) Date: Thu, 13 Feb 2003 21:32:30 +0100 Subject: [pypy-dev] Is anybody planning to give a talk on PyPy for OSCON? In-Reply-To: <200302121947.h1CJl9dM021777@ratthing-b246.strakt.com>; from lac@strakt.com on Wed, Feb 12, 2003 at 08:47:09PM +0100 References: <200302121947.h1CJl9dM021777@ratthing-b246.strakt.com> Message-ID: <20030213213230.I3033@prim.han.de> [Laura Creighton Wed, Feb 12, 2003 at 08:47:09PM +0100] > This is precisely the sort of thing that Nat Tarkington said > he wanted for the Python track -- new community efforts, new > implementations. > > Who is going to OSCON here besides me? I might go but it's impossible to say now. It's quite an expensive and faraway-from-here trip. Anyway, I am fine with you doing a proposal. Have the people actually doing the presentation to be on the proposal? If so then put me on although it's less than 50% that i will make it. And remember: flexibility and simplicity first, then performance :-) holger From Bram at moolenaar.net Thu Feb 13 21:33:53 2003 From: Bram at moolenaar.net (Bram Moolenaar) Date: Thu, 13 Feb 2003 21:33:53 +0100 Subject: [pypy-dev] Scons? Message-ID: <200302132033.h1DKXr514419@moolenaar.net> Holger Krekel wrote (to Charles Crain): > As we plan to run on top of CPython first it makes a lot > of sense to have SCons drive the CPython build and thus > get rid of make/configure dependencies. It's also a good > release goal for our sprint. The real challenge here is > to have a scheme for taking current CPython-cvs as an > "upstream" and have a system for keeping in > sync what we take out of CPython into our CPython-PyPy. As an alternative, you might want to look into A-A-P. Like SCons it is a Python based build system. The main difference is that SCons uses a file with Python syntax while A-A-P uses a syntax that looks more like a Makefile. In A-A-P you can use Python script just about everywhere, but the dependencies are specified like in a Makefile. One of the advantages is that you can specify a list of files without quotes or commas (you do need to use quotes when a file name contains a blank). Like SCons, A-A-P is in an alpha stage. SCons has more support for compilers and languages. A-A-P has support for uploading, downloading and CVS. I'm currently working on a GUI IDE for A-A-P. Most info can be found on the web site: http://www.A-A-P.org/ Have a look at the examples: http://www.A-A-P.org/examples.html Ask me if you have further questions. -- hundred-and-one symptoms of being an internet addict: 225. You sign up for free subscriptions for all the computer magazines /// Bram Moolenaar -- Bram at Moolenaar.net -- http://www.Moolenaar.net \\\ /// Creator of Vim - Vi IMproved -- http://www.Vim.org \\\ \\\ Project leader for A-A-P -- http://www.A-A-P.org /// \\\ Help AIDS victims, buy at Amazon -- http://ICCF.nl/click1.html /// From pedronis at bluewin.ch Thu Feb 13 22:26:37 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Thu, 13 Feb 2003 22:26:37 +0100 Subject: [pypy-dev] Object model References: <20030213163120.GA31338@magma.unil.ch> Message-ID: <02a701c2d3a6$97aca0a0$6d94fea9@newmexico> From: "Armin Rigo" > Hello everybody, > > I have been experimenting a bit about that distinction I tried to draw between > "application-level" and "interpreter-level" objects. I still think the idea > to be essential but I "reorganized" my thoughts a little bit. > > Now I believe in following the idea that CPython's main loop does not assume > anything at all about the PyObjects it manipulates, but does every operation > via calls to library functions like PyNumber_Add(). All it does is moving > PyObjects around, mainly between the stack and the locals. Similarily, our > own main loop should not assume anything about the objects it stores in the > stack and the locals, not even that they have some well-known methods. > Instead, we would provide it with the equivalent of PyNumber_Add() & co. > > Now the fun part of it is that there is no reason to use only one fixed set of > functions to do the operations. There are several things we want to be able > to do with our interpreter, and these will probably correspond to different > ways of "seeing" what an object is. We might try several ways to implement > the objects, for example trying different implementations for what the > application will always see as a list. And even, we might want to have > several concurrent "object spaces" existing at the same time, e.g. to > implement the idea of multiple VMs in the same process. > > Thus it seems natural to define a class ObjectSpace, that defines methods like > PyNumber_Add() and PySequence_GetItem(). Each frame would have an > ''objectspace'' attribute that holds the space in which it runs. (Maybe > shorter, more Pythonic names are better, e.g. add() and getitem().) > > Thus an object space is: > > * a way to implement objects; > > * from the interpreter's point of view, it is the "library" needed to do any > operation; > > * from the application's point of view, it is essentially invisible. > > Methods: > > * add(x,y), getitem(x,y), etc. for the operations, that take two "objects" > (black boxes for the interpreter) and returns an "object" (black box too). > > * type(x), taking a black box object and returning a black box object as > well. > > * wrap(a), taking a *normal* object and putting it into a black box. Its > what the interpreter does for a LOAD_CONST, for example: it has a real Python > object and wants to send it to the object space. For example, if the object > space has a custom implementation "class MyList" for lists, then wrap([1,2,3]) > should create a MyList instance. > > * unwrap(x) is the inverse operation. Used by the interpreter in the rare > cases where it really needs to observe the object. For example, for a > conditional jump, after it obtained the truth-value with (say) a call to the > truth() method, it must pull the result out of the object space to know > whether it is really False or True. > > For example, the straightforward object space that just uses real Python > objects to implement themselves would be defined like that: > > class BorrowingObjectSpace(ObjectSpace): > > def wrap(self, x): > return x > > def unwrap(self, x): > return x > > def add(self, x, y): > return x+y > > def type(self, x): > return type(x) > ... > > Cool things can be imagined with other kinds of object spaces. We may even > think about a distributed interpreter, running on one machine with object > spaces that actually reside on other machines! With this perspective, wrap(x) > sends x to the remote machine and returns a handle to it, and unwrap(x) > downloads from the remote machine the object whose handle is x. > > We may even have several concurrently-running object spaces that could > communicate, e.g. running a frame in a different object space and marshalling > arguments and the return value! > > > Looks like a cool point of view, doesn't it ? > I think it is even a necessary abstraction in some form (In a mail I suggested something like this but in the form of set of functions, not an object), because the bytecode / main loop is an artifact of _some_ possible Python implementations not all, and now implementing the abstraction as policy pattern increase flexibility. OTOH it would be a pity if the result is to avoid tackling the problem of describing the Python semantics relative to objects in Python in an __execution substrate (at least partially) indipendent__ way. IOW if the result is that we get different ObjectSpace definitions for each possible target (C,OCaml,...) without any implementation sharing among them, we both get maximal freedom but also maximal duplication. regards. From lac at strakt.com Thu Feb 13 22:38:57 2003 From: lac at strakt.com (Laura Creighton) Date: Thu, 13 Feb 2003 22:38:57 +0100 Subject: [pypy-dev] Is anybody planning to give a talk on PyPy for OSCON? In-Reply-To: Message from holger krekel of "Thu, 13 Feb 2003 21:32:30 +0100." <20030213213230.I3033@prim.han.de> References: <200302121947.h1CJl9dM021777@ratthing-b246.strakt.com> <20030213213230.I3033@prim.han.de> Message-ID: <200302132138.h1DLcvjL028470@ratthing-b246.strakt.com> In a message of Thu, 13 Feb 2003 21:32:30 +0100, holger krekel writes: >[Laura Creighton Wed, Feb 12, 2003 at 08:47:09PM +0100] >> This is precisely the sort of thing that Nat Tarkington said >> he wanted for the Python track -- new community efforts, new >> implementations. >> >> Who is going to OSCON here besides me? > >I might go but it's impossible to say now. It's quite >an expensive and faraway-from-here trip. > >Anyway, I am fine with you doing a proposal. Have the >people actually doing the presentation to be on the proposal? >If so then put me on although it's less than 50% that >i will make it. And remember: flexibility and simplicity >first, then performance :-) > > holger No, anybody can be on the proposal. We can put the entire membership of pypy-dev on, for instance. The person who makes the proposal doesn't have to be the person or persons who make the paper, or even that does the presenting. The only thing is the deadline for the proposal is Feb 15th, and what I need is for it to be ok with people here that if I make one, and if it gets accepted, that some paper will happen which Jacob or I could present. Other people are, of course, welcome to do the presenting, if they are attending. Laura From hpk at trillke.net Fri Feb 14 01:30:47 2003 From: hpk at trillke.net (holger krekel) Date: Fri, 14 Feb 2003 01:30:47 +0100 Subject: [pypy-dev] Scons? In-Reply-To: <200302132033.h1DKXr514419@moolenaar.net>; from Bram@moolenaar.net on Thu, Feb 13, 2003 at 09:33:53PM +0100 References: <200302132033.h1DKXr514419@moolenaar.net> Message-ID: <20030214013047.M3033@prim.han.de> Hi Bram, [Bram Moolenaar Thu, Feb 13, 2003 at 09:33:53PM +0100] > Holger Krekel wrote (to Charles Crain): > > > As we plan to run on top of CPython first it makes a lot > > of sense to have SCons drive the CPython build and thus > > get rid of make/configure dependencies. It's also a good > > release goal for our sprint. The real challenge here is > > to have a scheme for taking current CPython-cvs as an > > "upstream" and have a system for keeping in > > sync what we take out of CPython into our CPython-PyPy. > > As an alternative, you might want to look into A-A-P. Like SCons it is > a Python based build system. The main difference is that SCons uses > a file with Python syntax while A-A-P uses a syntax that looks more like > a Makefile. In A-A-P you can use Python script just about everywhere, > but the dependencies are specified like in a Makefile. One of the > advantages is that you can specify a list of files without quotes or > commas (you do need to use quotes when a file name contains a blank). Now we even have two choices. Didn't know that there are two build systems although A-A-P rings some bells. > Like SCons, A-A-P is in an alpha stage. SCons has more support for > compilers and languages. A-A-P has support for uploading, downloading > and CVS. I'm currently working on a GUI IDE for A-A-P. I guess a GUI is not of primary importance to us but i am not sure if the number compilers and languages is important. We'll try to go with subversion but a-a-p (what a name, btw :-) could certainly be adapted to it. > Most info can be found on the web site: http://www.A-A-P.org/ > Have a look at the examples: http://www.A-A-P.org/examples.html > Ask me if you have further questions. Nice website, especially your tools collection! And a-a-p also seems to have an advantage regarding documentation. I give both projects a closer look although i probably like to work with SCons first because i know it somewhat and don't care about makefile syntax too much. But this also depends on the people attending the Sprint and their preferences. Thanks for the pointers! holger From roccomoretti at netscape.net Fri Feb 14 03:50:39 2003 From: roccomoretti at netscape.net (Rocco Moretti) Date: Thu, 13 Feb 2003 21:50:39 -0500 Subject: [pypy-dev] Object model Message-ID: <76DCCBF7.12349F4C.9ADE5C6A@netscape.net> Armin Rigo wrote: >Now I believe in following the idea that CPython's main loop does not assume >anything at all about the PyObjects it manipulates, but does every operation >via calls to library functions like PyNumber_Add(). ?All it does is moving >PyObjects around, mainly between the stack and the locals. ?Similarily, our >own main loop should not assume anything about the objects it stores in the >stack and the locals, not even that they have some well-known methods. >Instead, we would provide it with the equivalent of PyNumber_Add() & co. I was originally going to say I thought the general-dispatch-function and the standard-object-method-interface options were practically equivalent, but I changed my mind. The reason lies along the concept of multi-methods. (I think that's the right term - functions are polymorphic not just on their first argument (the parent object), but on all arguments.) This may not make much difference on most operations, but it facilitates arithmatic conversion. If we're adding two numbers, we can handle the conversion code in the general Py_NumberAdd routine, instead of spreading it out to the functions implementing float and integers and rationals etc. ... > * wrap(a), taking a *normal* object and putting it into a black box. ?Its >what the interpreter does for a LOAD_CONST, for example: it has a real Python >object and wants to send it to the object space. ?For example, if the object >space has a custom implementation "class MyList" for lists, then wrap([1,2,3]) >should create a MyList instance. I wonder whether wrap() and unwrap() are really nessasary - they seem to blur the distinction between interpreter-level and application-object- implementation-level. It seems to me that the LOAD_CONST issue woold be better dealt with by having the compiler/function loader create the black box objects themselves, instead of there being a run-time translation between interpreter-level objects and application-level objects. > * unwrap(x) is the inverse operation. ?Used by the interpreter in the rare >cases where it really needs to observe the object. ?For example, for a >conditional jump, after it obtained the truth-value with (say) a call to the >truth() method, it must pull the result out of the object space to know >whether it is really False or True. But how would we handle it if we wanted to redefine what was considered "true"? A generic unwrap couldn't tell what the unwrapped object is being used for. A better way of implementing it would be to look at the specific cases when the interpreter machinery actually cares about the value of the black box object. In the case of conditional branching, it might be best to define a standard function in the ObjectSpace which takes a black-box application-level object and returns an interpreter-level object representing the truth value. (Not a generally "unwrapped" object, specifically 0 or 1 - or rather interpreter-level True and False, depending on how compatible we want to make the the PyPython codebase.) >Looks like a cool point of view, doesn't it ? Very much so, and it interfaces well with the concept of being able to hot-swap bytecodes. (Not only can you change the bytecode semantics of the processor, you can also change the object semantics.) But-first-we-need-a-working-implementation... -Rocco __________________________________________________________________ The NEW Netscape 7.0 browser is now available. Upgrade now! http://channels.netscape.com/ns/browsers/download.jsp Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/ From vlindberg at verio.net Fri Feb 14 05:09:15 2003 From: vlindberg at verio.net (VanL) Date: Thu, 13 Feb 2003 21:09:15 -0700 Subject: [pypy-dev] Object model In-Reply-To: <76DCCBF7.12349F4C.9ADE5C6A@netscape.net> References: <76DCCBF7.12349F4C.9ADE5C6A@netscape.net> Message-ID: <3E4C6BEB.2030007@verio.net> It seems like an appropriate time to ask this: I know that we want to preserve language semantics. However, are bytecode-level semantics similarly to be preserved, especially because bytecode isn't always portable between CPython versions? More specifically, is a stack-based VM sacrosanct? The traditional argument against register-based systems has been that they usually increase compiled code size. However, storage and bandwidth are becoming cheaper, and I think we could take advantage of "hardware" tricks like conditional execution and mulitple dispatch to significantly increase speed. In fact, we could take advantage of the fact that "memory" accesses are just as cheap as "register" accesses for the PyPy VM and even use a memory-memory architecture. That gives the highest code density -- even higher than stack-based bytecode -- and still allows most "hardware" tricks. Finally, It allows for a nice abstraction of various parts of the execution engine - the "Integer Functional Unit" could be highly optimized, because it would always know that it was dealing with integers. We wouldn't be stuck with machine types, though, we could have a "String Functional Unit" just as easily. VanL From paoloinvernizzi at dmsware.com Fri Feb 14 09:04:29 2003 From: paoloinvernizzi at dmsware.com (Paolo Invernizzi) Date: Fri, 14 Feb 2003 09:04:29 +0100 Subject: R: [pypy-dev] Object model In-Reply-To: <3E4C6BEB.2030007@verio.net> Message-ID: > In fact, we could take advantage of the fact that "memory" > accesses are just as cheap as "register" accesses for the PyPy VM and > even use a memory-memory architecture. That gives the highest code > density -- even higher than stack-based bytecode -- and still allows > most "hardware" tricks. I think this is a major point for archiving a better VM. I dont think we are stucked with stack-based VM, we can experiment different road... I think that different VM implementations are a possibility, and we, at least, can let the user (or the optimizer!) choose which one is better for a certain application, or even part of application if in that case the problem is speed and not memory footprint. Paolo Invernizzi From Bram at moolenaar.net Fri Feb 14 10:42:59 2003 From: Bram at moolenaar.net (Bram Moolenaar) Date: Fri, 14 Feb 2003 10:42:59 +0100 Subject: [pypy-dev] Scons? In-Reply-To: <20030214013047.M3033@prim.han.de> Message-ID: <200302140942.h1E9gxt01418@moolenaar.net> Holger - > > Like SCons, A-A-P is in an alpha stage. SCons has more support for > > compilers and languages. A-A-P has support for uploading, downloading > > and CVS. I'm currently working on a GUI IDE for A-A-P. > > I guess a GUI is not of primary importance to us but i am not > sure if the number compilers and languages is important. We'll > try to go with subversion but a-a-p (what a name, btw :-) could > certainly be adapted to it. If I'm not mistaking then subversion is similar to CVS (no locking). It should not be difficult to take the CVS support implementation and adjust it for subversion. It just requires someone that knows subversion to implement this. > > Most info can be found on the web site: http://www.A-A-P.org/ > > Have a look at the examples: http://www.A-A-P.org/examples.html > > Ask me if you have further questions. > > Nice website, especially your tools collection! And a-a-p also > seems to have an advantage regarding documentation. > > I give both projects a closer look although i probably like > to work with SCons first because i know it somewhat and don't > care about makefile syntax too much. But this also depends on the > people attending the Sprint and their preferences. It's a free choice for free software! - Bram -- hundred-and-one symptoms of being an internet addict: 234. You started college as a chemistry major, and walk out four years later as an Internet provider. /// Bram Moolenaar -- Bram at Moolenaar.net -- http://www.Moolenaar.net \\\ /// Creator of Vim - Vi IMproved -- http://www.Vim.org \\\ \\\ Project leader for A-A-P -- http://www.A-A-P.org /// \\\ Help AIDS victims, buy at Amazon -- http://ICCF.nl/click1.html /// From arigo at tunes.org Fri Feb 14 18:05:27 2003 From: arigo at tunes.org (Armin Rigo) Date: Fri, 14 Feb 2003 09:05:27 -0800 (PST) Subject: [pypy-dev] Re: Object model In-Reply-To: <2mof5gazu3.fsf@starship.python.net>; from mwh@python.net on Thu, Feb 13, 2003 at 04:51:32PM +0000 References: <20030213163120.GA31338@magma.unil.ch> <2mof5gazu3.fsf@starship.python.net> Message-ID: <20030214170527.B2D354C3A@bespin.org> Hello Michael, On Thu, Feb 13, 2003 at 04:51:32PM +0000, Michael Hudson wrote: > > class BorrowingObjectSpace(ObjectSpace): > > .... > > def add(self, x, y): > > return x+y > > "from operator import *" could go here? Almost. There is that extra 'self' argument. Also, I didn't tell you the whole truth: I still think we must catch exceptions at that point. So it would be more like that: def add(self, x, y): try: return x+y except: self.handle_python_exception() Armin From arigo at tunes.org Fri Feb 14 18:05:34 2003 From: arigo at tunes.org (Armin Rigo) Date: Fri, 14 Feb 2003 09:05:34 -0800 (PST) Subject: [pypy-dev] Object model In-Reply-To: <3E4BDA7E.5060409@verio.net>; from vlindberg@verio.net on Thu, Feb 13, 2003 at 10:48:46AM -0700 References: <20030213163120.GA31338@magma.unil.ch> <3E4BDA7E.5060409@verio.net> Message-ID: <20030214170534.01E1F4C3B@bespin.org> Hello VanL, On Thu, Feb 13, 2003 at 10:48:46AM -0700, VanL wrote: > I think you are envisioning a model interpreter that works (more or > less) like the CPU works in a computer. Thanks for pointing it out! The analogy is indeed interesting, and seems coherent. Armin From arigo at tunes.org Fri Feb 14 18:05:34 2003 From: arigo at tunes.org (Armin Rigo) Date: Fri, 14 Feb 2003 09:05:34 -0800 (PST) Subject: [pypy-dev] Object model In-Reply-To: <02a701c2d3a6$97aca0a0$6d94fea9@newmexico>; from pedronis@bluewin.ch on Thu, Feb 13, 2003 at 10:26:37PM +0100 References: <20030213163120.GA31338@magma.unil.ch> <02a701c2d3a6$97aca0a0$6d94fea9@newmexico> Message-ID: <20030214170534.DDE684C38@bespin.org> Hello Samuele, On Thu, Feb 13, 2003 at 10:26:37PM +0100, Samuele Pedroni wrote: > OTOH it would be a pity if the result is to avoid tackling the problem of > describing the Python semantics relative to objects in Python in an __execution > substrate (at least partially) indipendent__ way. Right, but then we can put a multi-method-dispatch mecanism in some abstract subclass of the abstract base ObjectSpace class. It would be thought to capture the low-level Python dispatch semantics. Then the concrete object spaces are instances of subclasses which may, but need not, subclass this "MultiDispatchObjectSpace". (phew! 'guess we'll need a shorter name :-) For example, the BorrowingObjectSpace I showed in my e-mail need not subclass MultiDispatchObjectSpace. > IOW if the result is that we get different ObjectSpace definitions for each > possible target (C,OCaml,...) without any implementation sharing among them, we > both get maximal freedom but also maximal duplication. That's why I was thinking about a class hierarchy in the first place. Object spaces may be more or less implementation-dependent; at worst we can isolate code common to several implementations into a common superclass. But I am sure we can do even better than that by basing implementations on common concepts, like "raw block of memory", inductive types, or Python classes with typed slots. A bient?t, Armin. From arigo at tunes.org Fri Feb 14 18:05:37 2003 From: arigo at tunes.org (Armin Rigo) Date: Fri, 14 Feb 2003 09:05:37 -0800 (PST) Subject: [pypy-dev] Object model In-Reply-To: <76DCCBF7.12349F4C.9ADE5C6A@netscape.net>; from roccomoretti@netscape.net on Thu, Feb 13, 2003 at 09:50:39PM -0500 References: <76DCCBF7.12349F4C.9ADE5C6A@netscape.net> Message-ID: <20030214170537.7732A4C3C@bespin.org> Hello Rocco, On Thu, Feb 13, 2003 at 09:50:39PM -0500, Rocco Moretti wrote: > I was originally going to say I thought the general-dispatch-function and > the standard-object-method-interface options were practically equivalent, > but I changed my mind. > > The reason lies along the concept of multi-methods. That was the point. A more minor point is that the methods add() etc. now automatically have a hidden "self" argument, which can be used to determine the context in which we are working (more specifically, the object space, but then from the object space we might come back to the currently executing frame). > I wonder whether wrap() and unwrap() are really nessasary - they seem to > blur the distinction between interpreter-level and application-object- > implementation-level. I think that the generality of these methods might be very useful in the future. If instead we tried to identify all the spots in the interpreter that require a specific treatment that could be done in generality with wrap() or unwrap(), then we would be more dependent on the current Python VM and bytecodes. I admit there are also drawbacks, but I would rather say that there are mainly optimization drawbacks. We can always later add a "hint" optional parameter to wrap() and unwrap() to let the ObjectSpace know for which particular reason the method is called. The wrap()/unwrap() symmetry is also nice because it allows us to define a "default" implementation for all ObjectSpace methods: def add(self, x, y): x0, y0 = self.unwrap(x0), self.unwrap(y0) z0 = x0 + y0 return self.wrap(z0) If you think about unwrap()/wrap() as respectively downloading/uploading the object through a network, it's not the fastest implementation, but it works! > It seems to me that the LOAD_CONST issue woold be better dealt with by > having the compiler/function loader create the black box objects > themselves, instead of there being a run-time translation between > interpreter-level objects and application-level objects. I don't think it's so clear. LOAD_CONST is not the only place where we would need wrap(). The bytecode interpreter may catch a RuntimeError or MemoryError, for example, and then it would want to send it to the application. As for LOAD_CONST, it's not clear where (interpreter- or object-space) the constants are best stored. Think about implementing Pyrex, the Python-to-C translator: it could be done with a PyrexObjectSpace whose add() method would only emit the following line of C code: v3 = PyNumber_Add(v1, v2); and the "black box" objects would only store the name of the C variable, not an actual value. If you have an explicit wrap() operation, then whenever a LOAD_CONST is seen, PyrexObjectSpace emits C code like: v5 = PyInt_FromLong(123); If on the other hand the LOAD_CONST is invisible to the object space, then PyrexObjectSpace must pre-build all the constants and put them into global C variables, whose names are transmitted to the pypy main loop. It might be a good optimization to do so, but then it might not always be the case (e.g. if we are targetting machine code instead of C and don't have so many registers available). With an explicit wrap() you can choose to "cache" all or some constants in global variables, or not. Without the wrap() you are forced to "cache" them all. > > * unwrap(x) is the inverse operation. > > But how would we handle it if we wanted to redefine what was > considered "true"? A generic unwrap couldn't tell what the unwrapped > object is being used for. Yes, that's right. It's actually the most difficult operation; for example, in PyrexObjectSpace, you cannot actually unwrap any object because it will later be in some variable of your emitted C code, but not now. I still think that we should try to provide a generic unwrap(), because it is essential for "network" object spaces where it represents downloading, or for Psyco where it represents suspending compilation, running the code, waiting for the execution point to reach that point, and "pulling" the actual value out of the registers. ObjectSpaces may only partially implement wrap()/unwrap(), i.e. it would be acceptable for them to fail to wrap or unwrap a value. For example, it can be expected that PyrexObjectSpace can only wrap "literal constants", like integers, strings, and floats (by emitting a call to PyXxx_FromYyy()). Conversely, it can only unwrap objects returned by its own truth() method, where it knows that it must be either False or True. (It still doesn't know whether it is False or True, but it can *duplicate* the caller frame and returns False once and True once, so that both paths will be tried.) The point of this lengthly explanation is to show that the difficulty does not lie directly in "unwrap() vs. specialized-versions-of-it()", because both have the same problem in some ObjectSpaces. I expect the same problem to show up in any specialized version of unwrap(). The rest is optimization hints. A bient?t, Armin. From Bram at moolenaar.net Thu Feb 13 21:33:53 2003 From: Bram at moolenaar.net (Bram Moolenaar) Date: Thu, 13 Feb 2003 21:33:53 +0100 Subject: [pypy-dev] Scons? Message-ID: <200302132033.h1DKXr514419@moolenaar.net> Holger Krekel wrote (to Charles Crain): > As we plan to run on top of CPython first it makes a lot > of sense to have SCons drive the CPython build and thus > get rid of make/configure dependencies. It's also a good > release goal for our sprint. The real challenge here is > to have a scheme for taking current CPython-cvs as an > "upstream" and have a system for keeping in > sync what we take out of CPython into our CPython-PyPy. As an alternative, you might want to look into A-A-P. Like SCons it is a Python based build system. The main difference is that SCons uses a file with Python syntax while A-A-P uses a syntax that looks more like a Makefile. In A-A-P you can use Python script just about everywhere, but the dependencies are specified like in a Makefile. One of the advantages is that you can specify a list of files without quotes or commas (you do need to use quotes when a file name contains a blank). Like SCons, A-A-P is in an alpha stage. SCons has more support for compilers and languages. A-A-P has support for uploading, downloading and CVS. I'm currently working on a GUI IDE for A-A-P. Most info can be found on the web site: http://www.A-A-P.org/ Have a look at the examples: http://www.A-A-P.org/examples.html Ask me if you have further questions. -- hundred-and-one symptoms of being an internet addict: 225. You sign up for free subscriptions for all the computer magazines /// Bram Moolenaar -- Bram at Moolenaar.net -- http://www.Moolenaar.net \\\ /// Creator of Vim - Vi IMproved -- http://www.Vim.org \\\ \\\ Project leader for A-A-P -- http://www.A-A-P.org /// \\\ Help AIDS victims, buy at Amazon -- http://ICCF.nl/click1.html /// From rxe at ukshells.co.uk Fri Feb 14 23:42:06 2003 From: rxe at ukshells.co.uk (Richard Emslie) Date: Fri, 14 Feb 2003 22:42:06 +0000 (GMT) Subject: [pypy-dev] Memory management Message-ID: Hi all, This is my first post. I have been enjoying lurking away here, reading the various threads has been interesting despite having to occasionally peel my brain off the frying pan ;-) I have a naive question which I hope some kind sole will help me out on. I have been waiting for memory management (lower level than garbage collecting / ref counting) to be discussed. I'm probably barking up the wrong tree - but a "pypy python" object has to be resident in memory somehow - and this needs a physical representation and heap allocation / deallocation presumably? Perhaps this is a post-bootstrap issue as we can piggy back on the existing CPython infra-structure during development - but as far as I can reason this will have to be ultimately handled by a minimal statically compiled python which we do we do use in bootstrapping our pypy python? Is this true? Apologies if this has already been discussed - or I am plain misunderstanding how things will work. :-) Good luck with the sprint! Thanks, Richard From tismer at tismer.com Sat Feb 15 06:19:57 2003 From: tismer at tismer.com (Christian Tismer) Date: Sat, 15 Feb 2003 06:19:57 +0100 Subject: [pypy-dev] Memory management In-Reply-To: References: Message-ID: <3E4DCDFD.5000909@tismer.com> Richard Emslie wrote: > Hi all, > > This is my first post. I have been enjoying lurking away here, reading > the various threads has been interesting despite having to occasionally > peel my brain off the frying pan ;-) A very familiar feeling to me ;-) > I have a naive question which I hope some kind sole will help me out on. > I have been waiting for memory management (lower level than garbage > collecting / ref counting) to be discussed. I'm probably barking up the > wrong tree - but a "pypy python" object has to be resident in memory > somehow - and this needs a physical representation and heap allocation / > deallocation presumably? Sure it needs that. > Perhaps this is a post-bootstrap issue as we can piggy back on the > existing CPython infra-structure during development - but as far as I can > reason this will have to be ultimately handled by a minimal statically > compiled python which we do we do use in bootstrapping our pypy python? > Is this true? Well, we are not intending to re-implement the GC in Python, yet. Instead, we assume it exists or we don't need it. Garbage collection is very visible in the current implementation. You see Py_INCREF and Py_(X)DECREF in many places right now. This is there by tradition, introduced by early optimization. It is also a draw-back. By doing explicit refcount handling, the source code gets cluttered, and it is hard to be optmized. This is a matter of having to be explicit in C. Also, it is mixed with optimization issues. Both should appear in PyPy, nowhere! Some of the well-known C++ wrappers around the Python C library handle this is a rigurous way: Every newly used reference increses the refcound, and later-on removes it by a method of the destructor of an automatic variable. This is an approach, but not very sensible. One of PyPy's intentions is to get stuff like that onto another level of abstraction. A function with a parameter should not need to spell an extra reference to the parameter, but it should tell the RTL that it need this parameter to stay alive while it needs it. This might have the same implementation, but is very different from an algorithmic view. The point is to formulate this function's need, regardless if the continued existance of the parameter is implemented by refcounts or something else. The point is to retract from being this explicit, to a position where we can express the requirement of this parameter to be existant long enough. *Then*, we have the freedom to choose how this can be implemented and be rendered into code. Simply saying "increase the refcount" is bad, since it carries very many assumptions about the object model and how things are implemented. Exactly to remove as much as possible of that from a generic implementation is one target of this project. When you are programming in Python, do you think of refcounts? Most probably not. This is so, since the Python interpreter simply adds a reference to any object that you reference, and removes one if you drop it. This isn't optimial, like the C source shows: It tries to save refcounts in many cases. But this is a thing the Python programmer doesn't see. It is, anyway, far from optimum to have an interpreter act this way. If it knew when an obkect needs to stay alive and when not, it could handle this (implemented by refcounts or in a couple of other ways) much more efficiently. The C source suffers from this: When the programmer knows that (s)he needs an object, (s)he increfs the objects, performs the operation and then decrefs it. This is about telling what to do. Instead of telling the system what the need is, the user has to spell what to do and how to do this. So what you see in the C code is a contribution to the limitation of this language: You cannot tell it how things should be done and how objects should be treated, but you have to verbosely implement it. This exactly is what PyPy will try to avoid. In PyPy, I will be able to spell that I need this parameter object to be alive from now on, until I don't need it any longer. The implementation level will have quite some different view of this, but it has to adhere to my presumtions. It is free to implement it with simple refcounting (and it will do so fore quite some while for sure), but not forced to. I'd also like to spell "I don't need this object anly longer, kill it ASAP, please", or "I'm done with it" but don't care about its existance. This cannot be spelled by explicit implementations like with Py_(IN|DE)CREF, because these don't distinguish between earliest deconstruction and really-not caring, and they also dictate the way how object existance is implemented. > Apologies if this has already been discussed - or I am plain > misunderstanding how things will work. :-) It wasn't so explicit, yet, and it was a nice chance to elaborate on the topic a bit more. This text has probably a number of rpetitions, since it was written at 6:00 in the morning. Please DECREF them before reading :-) sincerely -- chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From hpk at trillke.net Sat Feb 15 09:46:06 2003 From: hpk at trillke.net (holger krekel) Date: Sat, 15 Feb 2003 09:46:06 +0100 Subject: [pypy-dev] Memory management In-Reply-To: <3E4DCDFD.5000909@tismer.com>; from tismer@tismer.com on Sat, Feb 15, 2003 at 06:19:57AM +0100 References: <3E4DCDFD.5000909@tismer.com> Message-ID: <20030215094606.U3033@prim.han.de> [Christian Tismer Sat, Feb 15, 2003 at 06:19:57AM +0100] > Richard Emslie wrote: > > I have a naive question which I hope some kind sole will help me out on. > > I have been waiting for memory management (lower level than garbage > > collecting / ref counting) to be discussed. I'm probably barking up the > > wrong tree - but a "pypy python" object has to be resident in memory > > somehow - and this needs a physical representation and heap allocation / > > deallocation presumably? > > Sure it needs that. > > > Perhaps this is a post-bootstrap issue as we can piggy back on the > > existing CPython infra-structure during development - but as far as I can > > reason this will have to be ultimately handled by a minimal statically > > compiled python which we do we do use in bootstrapping our pypy python? > > Is this true? > > Well, we are not intending to re-implement the GC in Python, > yet. Instead, we assume it exists or we don't need it. Especially since we don't have anybody really intimate with GC issues, i guess. > Garbage collection is very visible in the current implementation. > You see Py_INCREF and Py_(X)DECREF in many places right now. > This is there by tradition, introduced by early optimization. > It is also a draw-back. By doing explicit refcount handling, > the source code gets cluttered, and it is hard to be optmized. I agree that we would like to avoid INCREF/DECREF everywhere. > One of PyPy's intentions is to get stuff like that > onto another level of abstraction. A function with > a parameter should not need to spell an extra reference > to the parameter, but it should tell the RTL that it need > this parameter to stay alive while it needs it. > This might have the same implementation, but is very > different from an algorithmic view. Hmm, this i don't understand. It sounds much like current INCREF/DECREF which means exactly this: "i want to use this object"/"i don't want to use it anymore". Objects can be shared in many places so i don't yet see how your "tell the runtime that it needs this paramater to stay alive while it needs it" beeing different from current CPython's scheme. Can you give a more concrete example how this would work and/or be spelled? Or anyone else? cheers, holger From hpk at trillke.net Sat Feb 15 09:58:51 2003 From: hpk at trillke.net (holger krekel) Date: Sat, 15 Feb 2003 09:58:51 +0100 Subject: [pypy-dev] downtime codespeak/pypy Message-ID: <20030215095851.X3033@prim.han.de> Hello, we plan to transfer the codespeak-computer which hosts the pypy site to the sprint-place. This gives us 100 MBit access to our repository among other conveniencies. So expect some downtimes on early sunday afternoon after which the DNS-switch should be completed. mail to the pypy-dev list should be automatically deferred until the host is online again. Of course it would be nicer to have a second computer which could take-over without downtime but Jens-Uwe didn't manage to get something like this done, yet. regards, holger From hpk at trillke.net Sat Feb 15 10:05:03 2003 From: hpk at trillke.net (holger krekel) Date: Sat, 15 Feb 2003 10:05:03 +0100 Subject: [pypy-dev] repository access Message-ID: <20030215100502.Y3033@prim.han.de> Hi again, we plan to use subversion for managing our code. If you want to participate from the outside or otherwise share/contribute code please try to setup a subversion client. I've heard that installing it is not too obvious from the http://subversion.tigris.org website so if you succeed installing it on your system, success stories are certainly welcome. For people using Gentoo Linux i've have success doing an ebuild on "Neon" and subsequently "subversion" which appears to work. The real hard part is to install the server side but this already works and a test server is reachable at http://codespeak.net:8080/ (subversion roots are urls and WebDav/DeltaX is the client/server network protocol). holger From lac at strakt.com Sat Feb 15 10:20:32 2003 From: lac at strakt.com (Laura Creighton) Date: Sat, 15 Feb 2003 10:20:32 +0100 Subject: [pypy-dev] repository access In-Reply-To: Message from holger krekel of "Sat, 15 Feb 2003 10:05:03 +0100." <20030215100502.Y3033@prim.han.de> References: <20030215100502.Y3033@prim.han.de> Message-ID: <200302150920.h1F9KWjL004317@ratthing-b246.strakt.com> Windows users who would like to connect to subversion through internet explorer can try TortoiseSVN http://tortoisesvn.tigris.org/ I haven't used that myself, but I have seen its big brother TortoiseCVS in action http://www.tortoisecvs.org/ and it makes CVS one heck of a lot easier on Windows. Laura Creighton From arigo at tunes.org Sat Feb 15 10:36:26 2003 From: arigo at tunes.org (Armin Rigo) Date: Sat, 15 Feb 2003 01:36:26 -0800 (PST) Subject: [pypy-dev] Object model In-Reply-To: <20030214170537.7732A4C3C@bespin.org>; from arigo@tunes.org on Fri, Feb 14, 2003 at 09:05:37AM -0800 References: <76DCCBF7.12349F4C.9ADE5C6A@netscape.net> <20030214170537.7732A4C3C@bespin.org> Message-ID: <20030215093626.591574C4B@bespin.org> Hello Rocco, Mmh, my previous e-mail was quite lengthy. Sorry for that, I guess I was thinking aloud... I was seduced in the first place by the symmetry and generality of wrap()/unwrap() (whose name came from the already-cited paper "Representing Type Information in Dynamically Typed Languages", http://citeseer.nj.nec.com/gudeman93representing.html). One of the problems with wrap()/unwrap() is when they are applied to containers. As they are, they have to "translate" the whole structure recursively, which is often not a good idea. On the other hand, if we define more specialized purpose-oriented methods, we loose opportunities given by wrap()/unwrap() to let particular routines send or examine an arbitrary piece of data. I'm thinking about type-based dispatch, for example, where you have to query the type of an object with the appropriate ObjectSpace method, and then unwrap() this type. I don't know which one is better. Maybe both are useful. I guess we will have to try and see. Hopefully, trying both should be easy in the early phases. A bient?t, Armin. From arigo at tunes.org Sat Feb 15 10:36:27 2003 From: arigo at tunes.org (Armin Rigo) Date: Sat, 15 Feb 2003 01:36:27 -0800 (PST) Subject: [pypy-dev] Memory management In-Reply-To: <20030215094606.U3033@prim.han.de>; from hpk@trillke.net on Sat, Feb 15, 2003 at 09:46:06AM +0100 References: <3E4DCDFD.5000909@tismer.com> <20030215094606.U3033@prim.han.de> Message-ID: <20030215093627.401E64C26@bespin.org> Hello Holger, On Sat, Feb 15, 2003 at 09:46:06AM +0100, holger krekel wrote: > > Well, we are not intending to re-implement the GC in Python, > > yet. Instead, we assume it exists or we don't need it. > > Especially since we don't have anybody really intimate with > GC issues, i guess. Not only so. We really want to isolate all memory implementation issues to some clearly-separated level, and I don't think an advanced GC is one of the first things to worry about. As Christian pointed out, a first possible implementation will just stuff tons of INCREF/DECREF everywhere. When this works we can think about being more subtle by optimizing the memory-implementation-level. > > One of PyPy's intentions is to get stuff like that > > onto another level of abstraction. A function with > > a parameter should not need to spell an extra reference > > to the parameter, but it should tell the RTL that it need > > this parameter to stay alive while it needs it. > > This might have the same implementation, but is very > > different from an algorithmic view. > > Hmm, this i don't understand. It sounds much like current > INCREF/DECREF which means exactly this: "i want to use > this object"/"i don't want to use it anymore". Yes, but the calls to Py_INCREF() and Py_DECREF() have to be algorithmically present in the C code. It works well if you need an object for some precise range of lines in your C function, but it is not so fine with respect to anything else. Take parameter passing: the Python C API doc states for each function if the function returns a new ref or not, and if arguments are decref'ed or not. This should be formally declared. We have the same problem for containers: any object holding pointers to other objects -- do they hold a reference or not? This is not explicitely told in the declaration of the C structures. There are a lot of programming languages which only have a GC and no refcounts, but I know about one (Mercury) in which the compiler is also able to track which variables hold uniquely referenced objects and which variables could hold shared values. We can even specify that a given function argument must contain a uniquely referenced object (i.e. an object that no one else can possibly know about). This lets the function do mutating operations on immutable objects (cf. _PyString_Resize()). BTW in Mercury *all* objects are immutable, but the language is still efficient because the compiler can often emit code that mutates them under-the-hood using these techniques. Armin From arigo at tunes.org Sat Feb 15 12:29:00 2003 From: arigo at tunes.org (Armin Rigo) Date: Sat, 15 Feb 2003 03:29:00 -0800 (PST) Subject: [pypy-dev] repository access In-Reply-To: <20030215100502.Y3033@prim.han.de>; from hpk@trillke.net on Sat, Feb 15, 2003 at 10:05:03AM +0100 References: <20030215100502.Y3033@prim.han.de> Message-ID: <20030215112900.6FDA74C39@bespin.org> Hello everybody, On Sat, Feb 15, 2003 at 10:05:03AM +0100, holger krekel wrote: > client. I've heard that installing it is not too obvious > from the http://subversion.tigris.org website so if you > succeed installing it on your system, success stories > are certainly welcome. In my opinion, they have a bootstrapping problem. The easy instructions tell you to download a source tarball, compile it, and use it to checkout the latest sources from their own repository. Fine. The sources are 6.7MB (my poor modem line), which is not a problem per se. But once you have built your client, you realize that the instructions tell you to use the damn thing to checkout the whole source tree into a *new* directory -- 6.7MB again! It won't download just the differences because the original tarball was not a checked-out working directory in the first place :-(( When you must download 6.7MB and trash them immediately, I say the bootstrapping procedure is bad bad bad :-( Let's hope pypy will eventually bootstrap without asking users to download CPython first! This said, for Linux the instructions aren't that complicated: http://svn.collab.net/repos/svn/trunk/INSTALL (section II. A. 1, Bootstrapping from a Tarball). Here is also a direct link to the current archive: http://subversion.tigris.org/files/documents/15/2699/subversion-r4503.tar.gz I had to fix a compilation problem because I have an old 'expat' installed (version 1.1). 'configure' correctly detects it, but a line is missing in subversion/include/svn_xml.h (add it at the beginning of the file if 'make' fails with an error message about that file): #include "svn_private_config.h" Armin From hpk at trillke.net Sat Feb 15 13:16:26 2003 From: hpk at trillke.net (holger krekel) Date: Sat, 15 Feb 2003 13:16:26 +0100 Subject: [pypy-dev] repository access In-Reply-To: <20030215112900.6FDA74C39@bespin.org>; from arigo@tunes.org on Sat, Feb 15, 2003 at 03:29:00AM -0800 References: <20030215100502.Y3033@prim.han.de> <20030215112900.6FDA74C39@bespin.org> Message-ID: <20030215131626.I3033@prim.han.de> [Armin Rigo Sat, Feb 15, 2003 at 03:29:00AM -0800] > Hello everybody, > > On Sat, Feb 15, 2003 at 10:05:03AM +0100, holger krekel wrote: > > client. I've heard that installing it is not too obvious > > from the http://subversion.tigris.org website so if you > > succeed installing it on your system, success stories > > are certainly welcome. > > In my opinion, they have a bootstrapping problem. The easy instructions tell > you to download a source tarball, compile it, and use it to checkout the > latest sources from their own repository. Fine. The sources are 6.7MB (my > poor modem line), which is not a problem per se. But once you have built your > client, you realize that the instructions tell you to use the damn thing to > checkout the whole source tree into a *new* directory -- 6.7MB again! It > won't download just the differences because the original tarball was not a > checked-out working directory in the first place :-(( > > When you must download 6.7MB and trash them immediately, I say the > bootstrapping procedure is bad bad bad :-( Let's hope pypy will eventually > bootstrap without asking users to download CPython first! thanks for pointing this out :-) I want to have a self-contained distro of CPython/PyPy. Hopefully we get this done during the sprint (with all the upstreaming features we need) holger From tismer at tismer.com Sat Feb 15 19:42:28 2003 From: tismer at tismer.com (Christian Tismer) Date: Sat, 15 Feb 2003 19:42:28 +0100 Subject: [pypy-dev] Memory management In-Reply-To: <20030215094606.U3033@prim.han.de> References: <3E4DCDFD.5000909@tismer.com> <20030215094606.U3033@prim.han.de> Message-ID: <3E4E8A14.1080007@tismer.com> holger krekel wrote: > [Christian Tismer Sat, Feb 15, 2003 at 06:19:57AM +0100] ... >>One of PyPy's intentions is to get stuff like that >>onto another level of abstraction. A function with >>a parameter should not need to spell an extra reference >>to the parameter, but it should tell the RTL that it need >>this parameter to stay alive while it needs it. >>This might have the same implementation, but is very >>different from an algorithmic view. > > > Hmm, this i don't understand. It sounds much like current > INCREF/DECREF which means exactly this: "i want to use > this object"/"i don't want to use it anymore". Objects > can be shared in many places so i don't yet see how your > "tell the runtime that it needs this paramater to stay > alive while it needs it" beeing different from > current CPython's scheme. Can you give a more concrete > example how this would work and/or be spelled? Or > anyone else? Simple. You don't want to INC/DECREF, but you want to spell how the lines of your code increase or decrease the liveness of objects. "I need it to be alive from here to here", maybe. Obe approach would be to mimick the automatic variables of C++. An implementation *could* be that the initialiser of an automated local variable increases the refcount of the object that is attached, and that it decrefs again, when the scope of the function is left. This is *one* implementation. Whatever we implement, if it is possible to switch from refcounts to a gc without changing our implementation, then we have made a huge improvement. I was about to explain this further, but now I see Armin's post, it says exactly what I wanted to say. See 'ya tomorrow -- chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at tismer.com Sat Feb 15 19:56:31 2003 From: tismer at tismer.com (Christian Tismer) Date: Sat, 15 Feb 2003 19:56:31 +0100 Subject: [pypy-dev] repository access In-Reply-To: <20030215131626.I3033@prim.han.de> References: <20030215100502.Y3033@prim.han.de> <20030215112900.6FDA74C39@bespin.org> <20030215131626.I3033@prim.han.de> Message-ID: <3E4E8D5F.5060005@tismer.com> holger krekel wrote: > [Armin Rigo Sat, Feb 15, 2003 at 03:29:00AM -0800] > >>Hello everybody, >> >>On Sat, Feb 15, 2003 at 10:05:03AM +0100, holger krekel wrote: >> >>>client. I've heard that installing it is not too obvious >>>from the http://subversion.tigris.org website so if you >>>succeed installing it on your system, success stories >>>are certainly welcome. >> >>In my opinion, they have a bootstrapping problem. The easy instructions tell >>you to download a source tarball, compile it, and use it to checkout the >>latest sources from their own repository. Fine. The sources are 6.7MB (my >>poor modem line), which is not a problem per se. But once you have built your >>client, you realize that the instructions tell you to use the damn thing to >>checkout the whole source tree into a *new* directory -- 6.7MB again! It >>won't download just the differences because the original tarball was not a >>checked-out working directory in the first place :-(( >> >>When you must download 6.7MB and trash them immediately, I say the >>bootstrapping procedure is bad bad bad :-( Let's hope pypy will eventually >>bootstrap without asking users to download CPython first! What I read about installing subversion really makes me wonder whether it would be easier for everybody to stick with plain old CVS? I use it all day and could work immediately. > thanks for pointing this out :-) I want to have a self-contained distro > of CPython/PyPy. Hopefully we get this done during the sprint (with > all the upstreaming features we need) Yes, that would be a nice goal. ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From hpk at trillke.net Sat Feb 15 20:39:25 2003 From: hpk at trillke.net (holger krekel) Date: Sat, 15 Feb 2003 20:39:25 +0100 Subject: [pypy-dev] downtime codespeak/pypy In-Reply-To: <20030215095851.X3033@prim.han.de>; from hpk@trillke.net on Sat, Feb 15, 2003 at 09:58:51AM +0100 References: <20030215095851.X3033@prim.han.de> Message-ID: <20030215203925.K3033@prim.han.de> [holger krekel Sat, Feb 15, 2003 at 09:58:51AM +0100] > Hello, > > we plan to transfer the codespeak-computer which > hosts the pypy site to the sprint-place. This > gives us 100 MBit access to our repository among > other conveniencies. So expect some downtimes > on early sunday afternoon after which the DNS-switch > should be completed. mail to the pypy-dev list should > be automatically deferred until the host is online again. > > Of course it would be nicer to have a second computer > which could take-over without downtime but Jens-Uwe > didn't manage to get something like this done, yet. Oh, sorry! Big excuse to Jens-Uwe! Of course, he didn't know anything about this plan with a secondary and _I_ didn't get this going before the sprint. I reedited this mail and sent out eventually sent out this bogus sentence! holger From hpk at trillke.net Sat Feb 15 20:44:33 2003 From: hpk at trillke.net (holger krekel) Date: Sat, 15 Feb 2003 20:44:33 +0100 Subject: [pypy-dev] repository access In-Reply-To: <3E4E8D5F.5060005@tismer.com>; from tismer@tismer.com on Sat, Feb 15, 2003 at 07:56:31PM +0100 References: <20030215100502.Y3033@prim.han.de> <20030215112900.6FDA74C39@bespin.org> <20030215131626.I3033@prim.han.de> <3E4E8D5F.5060005@tismer.com> Message-ID: <20030215204433.M3033@prim.han.de> Hi Christian, [Christian Tismer Sat, Feb 15, 2003 at 07:56:31PM +0100] > holger krekel wrote: > > [Armin Rigo Sat, Feb 15, 2003 at 03:29:00AM -0800] > > > >>Hello everybody, > >> > >>On Sat, Feb 15, 2003 at 10:05:03AM +0100, holger krekel wrote: > >> > >>>client. I've heard that installing it is not too obvious > >>>from the http://subversion.tigris.org website so if you > >>>succeed installing it on your system, success stories > >>>are certainly welcome. > >> > >>In my opinion, they have a bootstrapping problem. The easy instructions tell > >>you to download a source tarball, compile it, and use it to checkout the > >>latest sources from their own repository. Fine. The sources are 6.7MB (my > >>poor modem line), which is not a problem per se. But once you have built your > >>client, you realize that the instructions tell you to use the damn thing to > >>checkout the whole source tree into a *new* directory -- 6.7MB again! It > >>won't download just the differences because the original tarball was not a > >>checked-out working directory in the first place :-(( > >> > >>When you must download 6.7MB and trash them immediately, I say the > >>bootstrapping procedure is bad bad bad :-( Let's hope pypy will eventually > >>bootstrap without asking users to download CPython first! > > What I read about installing subversion really makes > me wonder whether it would be easier for everybody > to stick with plain old CVS? I use it all day and > could work immediately. If we don't try it at the sprint it is likely to be postponed for a long time, i guess. If installation is the only serious barrier then we should still try it IMO. We can switch anytime to cvs if we face serious difficulties. cheers, holger From tismer at tismer.com Sat Feb 15 22:03:08 2003 From: tismer at tismer.com (Christian Tismer) Date: Sat, 15 Feb 2003 22:03:08 +0100 Subject: [pypy-dev] repository access In-Reply-To: <20030215204433.M3033@prim.han.de> References: <20030215100502.Y3033@prim.han.de> <20030215112900.6FDA74C39@bespin.org> <20030215131626.I3033@prim.han.de> <3E4E8D5F.5060005@tismer.com> <20030215204433.M3033@prim.han.de> Message-ID: <3E4EAB0C.7020104@tismer.com> holger krekel wrote: ... [CVS] > If we don't try it at the sprint it is likely to be > postponed for a long time, i guess. If installation is > the only serious barrier then we should still try it IMO. > We can switch anytime to cvs if we face serious difficulties. So I guess it is better for me not to waste time tonight and do this on Monday, together with people who know how to do it. cheers - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From hpk at trillke.net Sat Feb 15 22:22:09 2003 From: hpk at trillke.net (holger krekel) Date: Sat, 15 Feb 2003 22:22:09 +0100 Subject: [pypy-dev] repository access In-Reply-To: <3E4EAB0C.7020104@tismer.com>; from tismer@tismer.com on Sat, Feb 15, 2003 at 10:03:08PM +0100 References: <20030215100502.Y3033@prim.han.de> <20030215112900.6FDA74C39@bespin.org> <20030215131626.I3033@prim.han.de> <3E4E8D5F.5060005@tismer.com> <20030215204433.M3033@prim.han.de> <3E4EAB0C.7020104@tismer.com> Message-ID: <20030215222209.T3033@prim.han.de> [Christian Tismer Sat, Feb 15, 2003 at 10:03:08PM +0100] > holger krekel wrote: > ... > [CVS] > > > If we don't try it at the sprint it is likely to be > > postponed for a long time, i guess. If installation is > > the only serious barrier then we should still try it IMO. > > We can switch anytime to cvs if we face serious difficulties. > > So I guess it is better for me not to waste > time tonight and do this on Monday, together > with people who know how to do it. yip, that should be fine. For people coming later or "participating from the outside" it probably can't hurt to try it out as soon as possible. While you are at it, you may want to have an IRC-client as the sprinters may be present on some IRC channel during next week. More details when we have them. holger From hpk at trillke.net Sat Feb 15 22:57:01 2003 From: hpk at trillke.net (holger krekel) Date: Sat, 15 Feb 2003 22:57:01 +0100 Subject: [pypy-dev] Runtime information from tests Message-ID: <20030215225701.W3033@prim.han.de> Hello, the following idea has been circulating in my mind quite a bit. I'll put this to discussion on the sprint but you may want to comment here on the list, too. I think that we should generally use our (unit-) tests in combination with our flexible upcoming interpreter to gather type and other information at *runtime*. This dynamically gathered information can later be used to generate C, ObjC or other source code. Or for optimizations debugging and whatnot. So we wouldn't generate from a static Parse-Tree only but almost always in combination with the runtime information we gather during executing the tests. Thus we can use the tests not only for producing a robust code base but for generating code, specializations and optimizations of all kinds. I hope that with this approach we can avoid many "hints" which we might otherwise be inclined to hardcode. Better invest the time in good tests. For this to work we should have a scheme which makes it easy to get from a unittests to its correlating function/method/class/module and vice versa. Although not strictly neccessary i'd like to be able to navigate in both directions at runtime, also. and-how-do-we-test-the-tests-ly y'rs, holger From roccomoretti at netscape.net Sun Feb 16 00:30:20 2003 From: roccomoretti at netscape.net (Rocco Moretti) Date: Sat, 15 Feb 2003 18:30:20 -0500 Subject: [pypy-dev] Object model Message-ID: <328257BC.7369ACF6.9ADE5C6A@netscape.net> Armin Rigo wrote: >One of the problems with wrap()/unwrap() is when they are applied to >containers. ?As they are, they have to "translate" the whole structure >recursively, which is often not a good idea. > >On the other hand, if we define more specialized purpose-oriented methods, we >loose opportunities given by wrap()/unwrap() to let particular routines send >or examine an arbitrary piece of data. ?I'm thinking about type-based >dispatch, for example, where you have to query the type of an object with the >appropriate ObjectSpace method, and then unwrap() this type. > >I don't know which one is better. ?Maybe both are useful. ?I guess we will >have to try and see. ?Hopefully, trying both should be easy in the early >phases. Now that I think of it, there is is no reason *not* to have both. As much as feasable, we can define special-purpose functions to aid in optimization and separation of concerns (e.g. have the truth() converter), but also have the generic functions (e.g. unwrap()) so that there is maximum flexibility available if we choose to use it. If we allow each to not work for some cases, having both can incorporate either extreme (just have the other function always fail). We can get the best of all worlds. -Rocco __________________________________________________________________ The NEW Netscape 7.0 browser is now available. Upgrade now! http://channels.netscape.com/ns/browsers/download.jsp Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/ From jum at anubis.han.de Sun Feb 16 13:54:52 2003 From: jum at anubis.han.de (Jens-Uwe Mager) Date: Sun, 16 Feb 2003 13:54:52 +0100 Subject: [pypy-dev] repository access In-Reply-To: <20030215112900.6FDA74C39@bespin.org> References: <20030215100502.Y3033@prim.han.de> <20030215112900.6FDA74C39@bespin.org> Message-ID: <20030216125452.GI1068@ANUBIS> On Sat, Feb 15, 2003 at 03:29 -0800, Armin Rigo wrote: > In my opinion, they have a bootstrapping problem. The easy instructions tell > you to download a source tarball, compile it, and use it to checkout the > latest sources from their own repository. Fine. The sources are 6.7MB (my > poor modem line), which is not a problem per se. But once you have built your > client, you realize that the instructions tell you to use the damn thing to > checkout the whole source tree into a *new* directory -- 6.7MB again! It > won't download just the differences because the original tarball was not a > checked-out working directory in the first place :-(( > > When you must download 6.7MB and trash them immediately, I say the > bootstrapping procedure is bad bad bad :-( Let's hope pypy will eventually > bootstrap without asking users to download CPython first! The bootstrap is not that complicated for the normal user, only folks that want to participate in svn development are required to go through the hoops of bootstrapping. If you are simply using a remote repository, download the bootstrap tarball, compile it and you are all set. Only if you want to stay current with svn development you need to check out the tree again with svn and compile that. -- Jens-Uwe Mager From lac at strakt.com Sun Feb 16 13:58:20 2003 From: lac at strakt.com (Laura Creighton) Date: Sun, 16 Feb 2003 13:58:20 +0100 Subject: [pypy-dev] about testing Message-ID: <200302161258.h1GCwKnM013435@ratthing-b246.strakt.com> This just out from the gang at Twisted Matrix. Build-bot, an automatic testing service. Produces results like this: http://twistedmatrix.com/~warner.twistd/ I think it is really cool. I wonder if we could adapt it to our purposes. Laura From jum at anubis.han.de Mon Feb 17 13:08:24 2003 From: jum at anubis.han.de (Jens-Uwe Mager) Date: Mon, 17 Feb 2003 13:08:24 +0100 Subject: [pypy-dev] downtime codespeak/pypy In-Reply-To: <20030215095851.X3033@prim.han.de> References: <20030215095851.X3033@prim.han.de> Message-ID: <20030217120824.GM1068@ANUBIS> On Sat, Feb 15, 2003 at 09:58 +0100, holger krekel wrote: > we plan to transfer the codespeak-computer which > hosts the pypy site to the sprint-place. This > gives us 100 MBit access to our repository among > other conveniencies. So expect some downtimes > on early sunday afternoon after which the DNS-switch > should be completed. mail to the pypy-dev list should > be automatically deferred until the host is online again. We indeed did have problems get the mail up and running again as the sprint location is using a very particular firewall configuration. I hope this gets through now and the mailing list works again. -- Jens-Uwe Mager From troy at gci.net Mon Feb 17 20:11:16 2003 From: troy at gci.net (Troy Melhase) Date: Mon, 17 Feb 2003 10:11:16 -0900 Subject: [pypy-dev] pypy-dev List Summary - 09 Feb 2003 to 16 Feb 2003 Message-ID: <200302171011.17012.troy@gci.net> This is a summary of the discussions on the pypy-dev mailing list between 09 February and 16 February 2003. The full archive of the list can be found here: http://codespeak.net/pipermail/pypy-dev/ ZODB and Threads ---------------- In a continuation from last weeks thread discussing multiple VMs in a single process, Jeremy Hylton says multi-threaded ZODB applications can use multiple connections. His response prompts Tobias Oberstein to share his belief that the GIL is the real issue. He suggests a replacement as well: http://codespeak.net/pipermail/pypy-dev/2003q1/000416.html Wiki Content ------------ Armin Rigo posts links to summaries he has created in the wiki: http://codespeak.net/pipermail/pypy-dev/2003q1/000412.html Reusing Parrot -------------- Tobias Oberstein brings up his recent discovery of the Parrot VM: http://codespeak.net/pipermail/pypy-dev/2003q1/000413.html Lalo Martins thinks Parrot isn't relevent because it would only provide a different VM. Christian Tismer concurs that it wouldn't be useful, instead because the VM isn't yet finished: http://codespeak.net/pipermail/pypy-dev/2003q1/000418.html Generating C ------------ Steven Shaw suggests that Minimal Python could generate C code, compile it, and load it at runtime. He cites Goo and GCL as examples: http://codespeak.net/pipermail/pypy-dev/2003q1/000425.html Chris responds by saying C code is already a backend target, and that Psyco would probably be faster: http://codespeak.net/pipermail/pypy-dev/2003q1/000426.html Scons ----- Holger Krekel asks the list readers for opinions on Scons. Chris says he has been quite satisfied with it: http://codespeak.net/pipermail/pypy-dev/2003q1/000422.html Charles Crain, a Scons developer, says Scons is "'technically' in alpha" but extremely stable and well-tested: http://codespeak.net/pipermail/pypy-dev/2003q1/000430.html Bram Moolenaar, lead developer of A-A-P, suggests A-A-P as an alternative and lists the similarities and differences between the two: http://codespeak.net/pipermail/pypy-dev/2003q1/000449.html Sprint Updates -------------- VanL asks if sprint materials can be made available during the sprint. He asks for recordings of the talks, presentation materials, and nightly code snapshots: http://codespeak.net/pipermail/pypy-dev/2003q1/000419.html Chris says daily updates will go in the wiki, that the code will be available, and that Dinu Gherman will probably write an article: http://codespeak.net/pipermail/pypy-dev/2003q1/000420.html Bengt Richter asks for mp3s, and Dinu suggests AV streams: http://codespeak.net/pipermail/pypy-dev/2003q1/000423.html PyPy at OSCON ------------- Laura Creighton asks the list readers if anyone will be presenting or speaking at OSCON: http://codespeak.net/pipermail/pypy-dev/2003q1/000427.html Chris says he's going to PyCON and can't attend both. Holger says he doesn't know if he's going and suggests Laura do the presentation: http://codespeak.net/pipermail/pypy-dev/2003q1/000436.html Laura proposes a report on the PyPy sprint and the project: http://codespeak.net/pipermail/pypy-dev/2003q1/000429.html Object Model ------------ Armin discusses his reorganized thoughts regarding the distinction between "application-level" and "interpreter-level" objects. He now believes in following the lead of the CPython main loop by not assuming anything about the objects upon which it operates. He suggests implementing an 'ObjectSpace' class and making its instances attributes of frame objects. These 'objectspace' attributes would be invisible to the application and would define the "library" needed to perform the operations of the interpreter. The objectspaces would define 'add', 'getitem', and more, in addition to 'wrap' and 'unwrap' methods: http://codespeak.net/pipermail/pypy-dev/2003q1/000431.html Michael Hudson likes the idea, and Armin explains that exceptions need to be caught in the objectspaces: http://codespeak.net/pipermail/pypy-dev/2003q1/000445.html VanL asks for clarification by drawing analogies from his hardware background, and Armin thinks his analogy is interesting and coherent: http://codespeak.net/pipermail/pypy-dev/2003q1/000434.html Samuele Pedroni thinks the freedom provided by this approach also means increased duplication: http://codespeak.net/pipermail/pypy-dev/2003q1/000438.html Armin believes this can be mitigated by a multi-method-dispatch mechanism in an abstract class: http://codespeak.net/pipermail/pypy-dev/2003q1/000447.html Rocco Moretti wonders if 'wrap' and 'unwrap' would be necessary, and Armin makes compelling arguments for them despite the drawbacks. Armin likes the symmetry between 'wrap' and 'unwrap' for a 'default' implementation, and thinks the two methods could be the basis of network objectspaces: http://codespeak.net/pipermail/pypy-dev/2003q1/000448.html Armin follows up to his own post by suggesting type-based dispatch may be useful, and that both approaches should be tried: http://codespeak.net/pipermail/pypy-dev/2003q1/000456.html VanL asks if bytecode semantics are to be preserved with the same rigor as language semantics. He wonders if a register-based VM could replace the stack-based approach. Paolo Invernizzi believes different VM implementations are possible: http://codespeak.net/pipermail/pypy-dev/2003q1/000443.html Memory Management ----------------- Richard Emslie broaches the subject of memory management. He wonders how PyPy will handle bytes at a lower level: http://codespeak.net/pipermail/pypy-dev/2003q1/000450.html Chris responds that the intent is to not reimplement the the CPython GC, as it is overly visible in CPython. He lists the ways it encumbers the C source and describes alternate object-referencing semantics: http://codespeak.net/pipermail/pypy-dev/2003q1/000451.html Holger asks for clarification on the different algorithmic view of INCREF and DECREF, and Armin uses parameter passing as an example: http://codespeak.net/pipermail/pypy-dev/2003q1/000457.html Chris says INCREF and DECREF aren't wanted, but instead a method to increase and decrease the "liveliness" of an object. He thinks one approach may be C-style automatic variables, but reminds that it's only one of many possible implementations: http://codespeak.net/pipermail/pypy-dev/2003q1/000460.html Down Time --------- Holger sends notice that codespeak.net, the PyPy site and lists will be down while the hardware is moved to the sprint location: http://codespeak.net/pipermail/pypy-dev/2003q1/000453.html Subverted --------- Holger announces Subversion as the planned source-control platform for PyPy: http://codespeak.net/pipermail/pypy-dev/2003q1/000454.html Armin points out that Subversion must be downloaded twice to bootstrap itself, and hopes that PyPy will be more efficient: http://codespeak.net/pipermail/pypy-dev/2003q1/000458.html Chris asks if CVS would be a better choice, but Holger thinks Subversion should be tried for the sprint: http://codespeak.net/pipermail/pypy-dev/2003q1/000463.html Laura Creighton adds that Internet Explorer(tm) users can connect to the Tigris SVN: http://codespeak.net/pipermail/pypy-dev/2003q1/000455.html Minimally, -troy From arman at twinsun.com Tue Feb 18 20:11:10 2003 From: arman at twinsun.com (Arman Bostani) Date: Tue, 18 Feb 2003 11:11:10 -0800 Subject: [pypy-dev] Ideas on pypy architecture (ala tismer) Message-ID: <3E52854E.7000503@twinsun.com> A long time ago I worked on a Prolog compiler which had the following design. * The system had an abstract machine specification which is analogous to the Python byte code instructions. * There was a second, very low-level instruction set (called progol) which was essentially a set of macros from which assembly code was generated. To port to a new architecture, new mappings for progol instructions were defined. * The system was able to compile the abstract machine instructions directly to progol. Also, a Prolog interpreter was written in progol. Now I think Pypy might benefit from a similar architecture. My suggestion is as follows: * Examine the entire C code base and come up with a low-level instruction set (let's call it pynoth). As above, this instruction set would be just a bunch of macros but probably translated to C rather than assembly. * Write a semi-automated converter (ala tismer) to translate the existing C code into pynoth. Perhaps, if the pynoth code can retain comments and look semi-decent, the Python gods will accept to use it rather than the existing C code base. If not, then we'd have to retranslate new/modified sources to pynoth as python changes. * Develop a psyco-style code generator which generates pynoth instead of assembly. * Over time, we can get rid of the hand-hacked pynoth code and rewrite much of the system in python. -arman -- Arman Bostani 360 N. Sepulveda Blvd. #2055 Twin Sun Inc. El Segundo, CA 90245 arman at twinsun.com Tel: (310) 524-1800, Fax: (310) 524-1818 From sismex01 at hebmex.com Tue Feb 18 20:49:06 2003 From: sismex01 at hebmex.com (sismex01 at hebmex.com) Date: Tue, 18 Feb 2003 13:49:06 -0600 Subject: [pypy-dev] Ideas on pypy architecture (ala tismer) Message-ID: > From: Arman Bostani [mailto:arman at twinsun.com] > Sent: Tuesday, February 18, 2003 1:11 PM > > A long time ago I worked on a Prolog compiler which had the following > design. > > * The system had an abstract machine specification which is analogous > to the Python byte code instructions. > > * There was a second, very low-level instruction set (called progol) > which was essentially a set of macros from which assembly code was > generated. To port to a new architecture, new mappings for progol > instructions were defined. > > * The system was able to compile the abstract machine instructions > directly to progol. Also, a Prolog interpreter was written in > progol. > > Now I think Pypy might benefit from a similar architecture. My > suggestion is as follows: > > * Examine the entire C code base and come up with a low-level > instruction set (let's call it pynoth). As above, this instruction > set would be just a bunch of macros but probably translated to C > rather than assembly. > > * Write a semi-automated converter (ala tismer) to translate the > existing C code into pynoth. Perhaps, if the pynoth code > can retain > comments and look semi-decent, the Python gods will accept > to use it > rather than the existing C code base. If not, then we'd have to > retranslate new/modified sources to pynoth as python changes. > > * Develop a psyco-style code generator which generates pynoth instead > of assembly. > > * Over time, we can get rid of the hand-hacked pynoth code and rewrite > much of the system in python. > > -arman > > -- > Arman Bostani 360 N. Sepulveda Blvd. #2055 > Twin Sun Inc. El Segundo, CA 90245 > arman at twinsun.com Tel: (310) 524-1800, Fax: (310) 524-1818 > Hello all, I've been lurking around this list for a while, I hope I don't make a complete fool of myself :-) I have only a single question: Why compile to C, when that language has a number of shortcomings which could be eliminated by picking a different target? By "shortcomings", I'm referring to certain aspects of generated code which become inaccesible, simply because C does not facilitate access to those aspects. One of the main aspects I'm talking about is the machine stack. How do you "continue" to another function? How do you inspect the return stack to gain knowledge from the calling sequence to arrive to a certain execution point? I've been playing a bit (emphasis on "playing", nothing serious yet) with forth, and the language itself provides easy access to the machine stack, without having to resort to strange contortions or inline assembly. Continuations and co-routines are written in some 20 lines (simple implementations), without requiring access to mind-bending hacks. Of course, many might say that Forth is *like* inline assembly in some aspects, but the kernel may well be hidden from sight and yet you still retain access to those aspects, normally hidden in C. I'm not advocating Forth as an intermediate language, it's plenty wierd, mostly I'm questioning C's use. Any comments on the following are plenty appreciated. Thanks. -gustavo From tismer at tismer.com Tue Feb 18 23:09:11 2003 From: tismer at tismer.com (Christian Tismer) Date: Tue, 18 Feb 2003 23:09:11 +0100 Subject: [pypy-dev] Object model In-Reply-To: <3E4C6BEB.2030007@verio.net> References: <76DCCBF7.12349F4C.9ADE5C6A@netscape.net> <3E4C6BEB.2030007@verio.net> Message-ID: <3E52AF07.5010100@tismer.com> VanL wrote: > It seems like an appropriate time to ask this: > > I know that we want to preserve language semantics. However, are > bytecode-level semantics similarly to be preserved, especially because > bytecode isn't always portable between CPython versions? Having our bytecode interpreter serves different goals. One is to have CPython available as a compiler, immediately. Another one is to understand the Python engine better. By re-implementing in a different language, but with similar semantics, we learn everything about it. Furthermore, we try to get this huge amount of open issues smaller. The few things we don't change now is the language and the bytecodes. After having made our way though this, we have a very well understanding of the implementation and all the issues it handles. We are by no means saying that byte VMs are the best way to go. There are other ways which have their advantages. At first, we want to go this way, which looks short and achievable. When that works, we can change the target. ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From DavidA at ActiveState.com Wed Feb 19 00:03:41 2003 From: DavidA at ActiveState.com (David Ascher) Date: Tue, 18 Feb 2003 15:03:41 -0800 Subject: [pypy-dev] about testing In-Reply-To: <200302161258.h1GCwKnM013435@ratthing-b246.strakt.com> References: <200302161258.h1GCwKnM013435@ratthing-b246.strakt.com> Message-ID: <3E52BBCD.50905@ActiveState.com> Laura Creighton wrote: > This just out from the gang at Twisted Matrix. > Build-bot, an automatic testing service. > Produces results like this: http://twistedmatrix.com/~warner.twistd/ > I think it is really cool. I wonder if we could adapt it to our > purposes. Just FYI, Trent Mick and I are presenting a talk about a 'competing' solution called 'smoke'. I look forward to hearing about BuildBot at PyCon so that we can hopefully get the best of both approaches, possibly even blending the two code bases. We'll see. --david From andreas at trillke.net Wed Feb 19 01:58:35 2003 From: andreas at trillke.net (Andreas Friedge) Date: Wed, 19 Feb 2003 01:58:35 +0100 (CET) Subject: [pypy-dev] first code Message-ID: <1098.10.1.2.30.1045616315.squirrel@squirrel.trillke.net> first real code can be seen now :) take a look around in the subversion repository with your favorite webbrowser: * http://codespeak.net:8080/svn/pypy/trunk/src/sandbox/interpreter/opcodes.py object space(s) provides implementation of functionality at interpreter level with some mind boggeling help from pure python code at application level that becomes loaded with an dynamic mechanism as seen with various applicationfile.call(...) like in def PRINT_ITEM_TO this dynamically loaded code resides in corresponding file with extension '.app.py' http://codespeak.net:8080/svn/pypy/trunk/src/sandbox/interpreter/opcodes.app.py * application level objects handled directly in the interpreter are (or better should be) consistently denoted with prefix 'w_'. objects that need to be handed over between the two distinct levels become wrapped/unwrapped * opcode imlpementations in opcade.py only raise OperationError(). Application level exceptions must be properly constructed with the wrap mechanism see def UNPACK_SEQUENCE(f, itemcount): * next steps: BorrowingObjectSpace: Object space that is implemented using a subset of python (no nested scopes,...) the interpreter main loopm, frames and maybe a module? have fun andreas From hpk at trillke.net Wed Feb 19 02:43:47 2003 From: hpk at trillke.net (holger krekel) Date: Wed, 19 Feb 2003 02:43:47 +0100 Subject: [pypy-dev] new website / subversion Message-ID: <20030219024347.M16259@prim.han.de> Hello, just a quick note that we updated our website http://codespeak.net/pypy on the second sprint day. There are some visible (content) changes but the real change is that the complete website including the issue-tracker, our modified MoinMoin, the navigation and whatnot is now controled by subversion. We have some more changes pending such as a generated Wiki-Skeleton for an annotated CPython source tree. Here the idea is that we can gather some knowledge about functions/C-files to support our PyPy-Interpreter/Python-modules implementation that reimplement CPython functionality. If you find bugs, please use our Issue-Tracker where you have to register yourself, first. http://codespeak.net/issues/pypy/ Obviously, we currently have two tracks (at least) going on in the sprint namely the infrastructure and the interpreter track. More general discussions take place all the time (especially at breakfast and dinner), of course. It's hard to summarize those so i leave that to others. good night, holger From hpk at trillke.net Fri Feb 21 12:45:29 2003 From: hpk at trillke.net (holger krekel) Date: Fri, 21 Feb 2003 12:45:29 +0100 Subject: [pypy-dev] updates Message-ID: <20030221124529.U16259@prim.han.de> hello, just updated the website a bit. Apparrently we can't really make the time to post detailed informations about the sprint here. We are having a lot of live discussions and coding and it's hard enough to track them even if you are here at the sprint. Also it's unclear for me if there is enough interest to hear any details from the sprint apart from what we publish on the website (under e.g. Design-Considerations or the subversion-repository). regards, holger From hpk at trillke.net Sun Feb 23 17:11:02 2003 From: hpk at trillke.net (holger krekel) Date: Sun, 23 Feb 2003 17:11:02 +0100 Subject: [pypy-dev] sprint is over, pypy starts Message-ID: <20030223171102.A11666@prim.han.de> hello pypy-dev, the sprint is over. It was a pretty intensive and what we think successful week. It proved to be impossible to do daily reports or communicate much with the outside world, sorry. It was hard enough to keep up with what everybody was doing. Watching Twin Peaks in the night didn't help much, either :^) The most interesting result is a somewhat running interactive pypy interpreter which is pretty satisfying. It turned out that the "objectspace" design makes a great lot of sense. The concept of dispatching operations on objects to a "library" is already in CPython but PyPy makes this very explicit. There also is an explicit distinction between interpreter-level and application level objects. Whereever possible we work in good old python application space. The interpreter runs with "normal" python, too, but it is even more more explicit in its code and naming. The PyPy-Interpreter only runs on the "trivial" objectspace (dispatching directly to CPython) but we had good progress on doing a "std" objectspace that reimplements all the CPython Types in Python. The "intobject.py" is pretty complete and gives a good example of a conversion from C-Python to Py-Python. When the StdObjSpace is more complete we are eager to generate all kinds of code from it. The first target may be regenerating a python interpreter/core in C-CPython. Those of you familiar with the ceval.c and friends will recognize strong similarities with CPython in http://codespeak.net:8080/svn/pypy/trunk/src/pypy/interpreter/pyframe.py Best of all, we did'nt have big naming, license, PEP308 or coding-style discussions. It's pypy, MIT-license, drop it and http://codespeak.net:8080/svn/pypy/trunk/doc/coding-style.txt This coding-style document was agreed on in much less than one hour which must be a world record, i guess :-) Also we now have "subversion" serving almost all documents and configurations of our project (including the website). We got the server python svn-bindings running so we can directly script all layers of subversion in the future. We refactored parts of our Wiki-code (MoinMoin-based) to have its pages also subversion controled. Although it's not complete you can already edit wiki-pages locally on your machine. I expect website updates to have to wait a bit, though, as most of the sprinters need to catch up with their other activities and slow down a bit. While it is impossible to post a summary of the discussions i guess that these discussions will influence the future development of pypy quite a bit and reflect to the pypy-dev list, of course. Last but not least, we (kind of) fixed another sprint between 20th and 24th of June 2003 right before EuroPython (which is between 25th to 27th). We may do an in-between sprint at beginning of May in Sweden. so much for my little report, have fun, holger From jum at anubis.han.de Mon Feb 24 13:09:35 2003 From: jum at anubis.han.de (Jens-Uwe Mager) Date: Mon, 24 Feb 2003 13:09:35 +0100 Subject: [pypy-dev] Subversion downtime to implement issue 10 Message-ID: <20030224120935.GC5376@ANUBIS> I will take the subversion server down starting at 14:00 local time for about one hour to implement the following issue: http://codespeak.net/issues/pypy/issue10 This will also move the repository to the data partition, so if you still have file urls in scripts please make sure that these are changed to /data/svn for the main repository or /data/test-svn for the personal and test repositories. -- Jens-Uwe Mager From mwh at python.net Mon Feb 24 13:47:31 2003 From: mwh at python.net (Michael Hudson) Date: Mon, 24 Feb 2003 12:47:31 +0000 Subject: [pypy-dev] Re: Subversion downtime to implement issue 10 References: <20030224120935.GC5376@ANUBIS> Message-ID: <2mfzqdg80s.fsf@starship.python.net> Jens-Uwe Mager writes: > I will take the subversion server down starting at 14:00 local time for > about one hour to implement the following issue: > > http://codespeak.net/issues/pypy/issue10 Attempting to access that URL leads to this error message where hpk's change test should be: ERROR reading file: msg17 Possibly an access right configuration problem. [Errno 2] No such file or directory: '/www/codespeak.net/issues/pypy/db/files/msg17' any ideas? Cheers, M. -- Never meddle in the affairs of NT. It is slow to boot and quick to crash. -- Stephen Harris -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From jum at anubis.han.de Mon Feb 24 14:08:32 2003 From: jum at anubis.han.de (Jens-Uwe Mager) Date: Mon, 24 Feb 2003 14:08:32 +0100 Subject: [pypy-dev] Re: Subversion downtime to implement issue 10 In-Reply-To: <2mfzqdg80s.fsf@starship.python.net> References: <20030224120935.GC5376@ANUBIS> <2mfzqdg80s.fsf@starship.python.net> Message-ID: <20030224130832.GD5376@ANUBIS> On Mon, Feb 24, 2003 at 12:47 +0000, Michael Hudson wrote: > Jens-Uwe Mager writes: > > > I will take the subversion server down starting at 14:00 local time for > > about one hour to implement the following issue: > > > > http://codespeak.net/issues/pypy/issue10 > > Attempting to access that URL leads to this error message where hpk's > change test should be: > > ERROR reading file: msg17 > Possibly an access right configuration problem. > [Errno 2] No such file or directory: '/www/codespeak.net/issues/pypy/db/files/msg17' > > any ideas? Holger appears to be working on that, I still have that issue in mail form so I attach it. -- Jens-Uwe Mager -------------- next part -------------- An embedded message was scrubbed... From: holger krekel Subject: [issue10] hpk Date: Sun, 23 Feb 2003 15:23:06 +0000 Size: 1635 URL: From jum at anubis.han.de Mon Feb 24 15:08:59 2003 From: jum at anubis.han.de (Jens-Uwe Mager) Date: Mon, 24 Feb 2003 15:08:59 +0100 Subject: [pypy-dev] Subversion downtime to implement issue 10 In-Reply-To: <20030224120935.GC5376@ANUBIS> References: <20030224120935.GC5376@ANUBIS> Message-ID: <20030224140859.GA8224@ANUBIS> On Mon, Feb 24, 2003 at 13:09 +0100, Jens-Uwe Mager wrote: > I will take the subversion server down starting at 14:00 local time for > about one hour to implement the following issue: > > http://codespeak.net/issues/pypy/issue10 > > This will also move the repository to the data partition, so if you > still have file urls in scripts please make sure that these are changed > to /data/svn for the main repository or /data/test-svn for the personal > and test repositories. OK, the move is completed and what was previously under the URL: http://codespeak.net:8080/svn/pypy/trunk/ is now under: http://codespeak.net:8080/svn/trunk/pypy/ -- Jens-Uwe Mager From hpk at trillke.net Mon Feb 24 16:18:12 2003 From: hpk at trillke.net (holger krekel) Date: Mon, 24 Feb 2003 16:18:12 +0100 Subject: [pypy-dev] Subversion downtime to implement issue 10 In-Reply-To: <20030224140859.GA8224@ANUBIS>; from jum@anubis.han.de on Mon, Feb 24, 2003 at 03:08:59PM +0100 References: <20030224120935.GC5376@ANUBIS> <20030224140859.GA8224@ANUBIS> Message-ID: <20030224161812.H11666@prim.han.de> [Jens-Uwe Mager Mon, Feb 24, 2003 at 03:08:59PM +0100] > On Mon, Feb 24, 2003 at 13:09 +0100, Jens-Uwe Mager wrote: > > > I will take the subversion server down starting at 14:00 local time for > > about one hour to implement the following issue: > > > > http://codespeak.net/issues/pypy/issue10 > > > > This will also move the repository to the data partition, so if you > > still have file urls in scripts please make sure that these are changed > > to /data/svn for the main repository or /data/test-svn for the personal > > and test repositories. > > OK, the move is completed and what was previously under the URL: > > http://codespeak.net:8080/svn/pypy/trunk/ > > is now under: > > http://codespeak.net:8080/svn/trunk/pypy/ Oh. didn't we want to leave the external url unmodified? I think it is better to have the old scheme /svn/pypy/trunk persist even with the new storage location. Otherwise different root-level projects would share "trunk" and branches and so on which is not a good thing. (jens-uwe knows this: ) subversion's trunk/branch/tags/vendor directories are for convenience only and follow no special rules. They are useful when thinking in cvs's terms (which make some sense , of course) So i am all for having separate hierachies for every project in codespeak's subversion repository. cheers, holger From hpk at trillke.net Mon Feb 24 16:29:15 2003 From: hpk at trillke.net (holger krekel) Date: Mon, 24 Feb 2003 16:29:15 +0100 Subject: [pypy-dev] Re: Subversion downtime to implement issue 10 In-Reply-To: <20030224130832.GD5376@ANUBIS>; from jum@anubis.han.de on Mon, Feb 24, 2003 at 02:08:32PM +0100 References: <20030224120935.GC5376@ANUBIS> <2mfzqdg80s.fsf@starship.python.net> <20030224130832.GD5376@ANUBIS> Message-ID: <20030224162915.J11666@prim.han.de> [Jens-Uwe Mager Mon, Feb 24, 2003 at 02:08:32PM +0100] > On Mon, Feb 24, 2003 at 12:47 +0000, Michael Hudson wrote: > > > Jens-Uwe Mager writes: > > > > > I will take the subversion server down starting at 14:00 local time for > > > about one hour to implement the following issue: > > > > > > http://codespeak.net/issues/pypy/issue10 > > > > Attempting to access that URL leads to this error message where hpk's > > change test should be: > > > > ERROR reading file: msg17 > > Possibly an access right configuration problem. > > [Errno 2] No such file or directory: '/www/codespeak.net/issues/pypy/db/files/msg17' > > > > any ideas? > > Holger appears to be working on that, I still have that issue in mail > form so I attach it. Jens-Uwe is right, i was putting the issue-tracker into subversion and moved stuff around. the issue-tracker should now be ok, again. i copied over the old messages. there might still be some problems lurking so keep me updated. holger From arigo at tunes.org Mon Feb 24 17:44:13 2003 From: arigo at tunes.org (Armin Rigo) Date: Mon, 24 Feb 2003 08:44:13 -0800 (PST) Subject: [pypy-dev] stdobjspace status Message-ID: <20030224164413.51465501B@bespin.org> Hello everybody, I've been progressing on putting the pieces together and will check-in my changes as soon as possible. The stdobjspace is getting ready to be tested together with the interpreter, but is not quite there yet. If someone (Michael?) can figure out a good solution for the END_FINALLY problem, I'd love to hear it. The problem is that in a try:except: block, when we enter the except: part, there are some control paths that reach the final END_FINALLY opcode (re-raising the exception) and some that don't (caught exception). It makes it hard to know when we must remove from the blockstack the description of the potentially re-raisable exception. I was also thinking about the explicit multimethod registration in the standard object space. For example, intobject.py contains the following code: def int_int_add(space, w_int1, w_int2): ... StdObjSpace.add.register(int_int_add, W_IntObject, W_IntObject) def int_int_sub(space, w_int1, w_int2): ... StdObjSpace.sub.register(int_int_sub, W_IntObject, W_IntObject) def int_int_mul(space, w_int1, w_int2): ... StdObjSpace.mul.register(int_int_mul, W_IntObject, W_IntObject) I am proposing that it could be written with some more declarative idiom abusing the 'class' keyword, something along these lines: class __multi__(W_IntObject, W_IntObject): def add(w_int1, w_int2, space): ... def sub(w_int1, w_int2, space): ... def mul(w_int1, w_int2, space): ... It looks much nicer, I think, and it addresses a point raised during the sprint (by Stephan if I remember right) about making the xxxobject.py files less dependent on the actual name of their object space. Indeed, the 'class __multi__' code no longer mentions StdObjSpace. This can be implemented using an original dispatching algorithm that has been as far as I know invented for the Slate language (http://slate.tunes.org/). The implementation of 'class __multi__(X, Y)' is defined to store the function objects defined in the body -- here add(), sub(), mul() -- into *both* the X and the Y class, into special-named attributes. Performing dispatch to 'space.add(x,y)' is done by looking into the class of 'x' and the class of 'y' to see if we can find the same function object in both. More specifically, after a definition like class __multi__(X, Y): def f(x, y, extra): ... we would have the function object 'f()' stored into each of the lists 'X.__dispatch_on_arg_1__' and 'Y.__dispatch_on_arg_2__'. Then a call like 'f(x,y,extra)' looks up the types of 'x' and 'y' and searches for a function object that would be present both in 'x.__class__.__dispatch_on_arg_1__' and 'y.__class__.__dispatch_on_arg_2__'. What is nice with this approach is the removal of the need for a central "repository" of functions, role which was currently held by StdObjSpace.add. BTW the new order of the arguments ('space' at the end) is such that single-dispatch is a particular case of Python's original dispatch semantics, with the argument we dispatch on being first (althought that original semantic is also extended by the delegation stuff, e.g. delegating from 'int' to 'long' for methods that fail to be implemented by 'int' for overflow reasons). A bient?t, Armin. From jum at anubis.han.de Mon Feb 24 18:05:11 2003 From: jum at anubis.han.de (Jens-Uwe Mager) Date: Mon, 24 Feb 2003 18:05:11 +0100 Subject: [pypy-dev] Subversion downtime to implement issue 10 In-Reply-To: <20030224161812.H11666@prim.han.de> References: <20030224120935.GC5376@ANUBIS> <20030224140859.GA8224@ANUBIS> <20030224161812.H11666@prim.han.de> Message-ID: <20030224170511.GB8224@ANUBIS> On Mon, Feb 24, 2003 at 16:18 +0100, holger krekel wrote: > [Jens-Uwe Mager Mon, Feb 24, 2003 at 03:08:59PM +0100] > > On Mon, Feb 24, 2003 at 13:09 +0100, Jens-Uwe Mager wrote: > > > > > I will take the subversion server down starting at 14:00 local time for > > > about one hour to implement the following issue: > > > > > > http://codespeak.net/issues/pypy/issue10 > > > > > > This will also move the repository to the data partition, so if you > > > still have file urls in scripts please make sure that these are changed > > > to /data/svn for the main repository or /data/test-svn for the personal > > > and test repositories. > > > > OK, the move is completed and what was previously under the URL: > > > > http://codespeak.net:8080/svn/pypy/trunk/ > > > > is now under: > > > > http://codespeak.net:8080/svn/trunk/pypy/ > > Oh. didn't we want to leave the external url unmodified? > > I think it is better to have the old scheme > > /svn/pypy/trunk > > persist even with the new storage location. Otherwise > different root-level projects would share "trunk" and branches > and so on which is not a good thing. > > (jens-uwe knows this: ) subversion's trunk/branch/tags/vendor > directories are for convenience only and follow no special rules. > They are useful when thinking in cvs's terms (which make some sense , > of course) > > So i am all for having separate hierachies for every project in > codespeak's subversion repository. Oops, yes. I forgot that discussion, so I have again shuffled things around (this time in the repository only) so the old URLs continue to work. So you to find the src repository under: http://codespeak.net:8080/svn/pypy/trunk/ According to our style guidelines I have also created the following directories: http://codespeak.net:8080/svn/pypy/branch/ A directory holding branches of the mainline trunk code. http://codespeak.net:8080/svn/pypy/tag/ A directory holding tagged versions, e.g. releases. The imported cpython tree lives under: http://codespeak.net:8080/svn/vendor/cpython/ in particular the directory: http://codespeak.net:8080/svn/vendor/cpython/current/ corresponds to a recent head of the cpython cvs. -- Jens-Uwe Mager From arigo at tunes.org Mon Feb 24 18:05:33 2003 From: arigo at tunes.org (Armin Rigo) Date: Mon, 24 Feb 2003 09:05:33 -0800 (PST) Subject: [pypy-dev] IRC chat Message-ID: <20030224170533.E12664F06@bespin.org> Hello again, I think the idea of setting up an IRC channel has been mentioned several times here and at the sprint, but as far as I know nothing concrete occurred. So unless somebody would prefer another server I suggest irc.freenode.net:6667, #pypy. A bient?t, Armin. From hpk at trillke.net Mon Feb 24 18:10:35 2003 From: hpk at trillke.net (holger krekel) Date: Mon, 24 Feb 2003 18:10:35 +0100 Subject: [pypy-dev] IRC chat In-Reply-To: <20030224170533.E12664F06@bespin.org>; from arigo@tunes.org on Mon, Feb 24, 2003 at 09:05:33AM -0800 References: <20030224170533.E12664F06@bespin.org> Message-ID: <20030224181035.M11666@prim.han.de> [Armin Rigo Mon, Feb 24, 2003 at 09:05:33AM -0800] > Hello again, > > I think the idea of setting up an IRC channel has been mentioned several times > here and at the sprint, but as far as I know nothing concrete occurred. So > unless somebody would prefer another server I suggest irc.freenode.net:6667, > #pypy. i just joined but how do we make this channel permanent? Armin, if you can then please join, so we can discuss this online :-) holger From tismer at tismer.com Mon Feb 24 18:51:18 2003 From: tismer at tismer.com (Christian Tismer) Date: Mon, 24 Feb 2003 18:51:18 +0100 Subject: [pypy-dev] stdobjspace status In-Reply-To: <20030224164413.51465501B@bespin.org> References: <20030224164413.51465501B@bespin.org> Message-ID: <3E5A5B96.7020809@tismer.com> Armin Rigo wrote: ... > I was also thinking about the explicit multimethod registration in the > standard object space. For example, intobject.py contains the following code: > > def int_int_add(space, w_int1, w_int2): > ... > StdObjSpace.add.register(int_int_add, W_IntObject, W_IntObject) ... > I am proposing that it could be written with some more declarative idiom > abusing the 'class' keyword, something along these lines: > > class __multi__(W_IntObject, W_IntObject): > def add(w_int1, w_int2, space): > ... > def sub(w_int1, w_int2, space): > ... > def mul(w_int1, w_int2, space): > ... Hmm, "deriving" from the same class twice doesn't look very clean, and space at the end, and no "self", hmm hmm. > It looks much nicer, I think, and it addresses a point raised during the > sprint (by Stephan if I remember right) about making the xxxobject.py files > less dependent on the actual name of their object space. Indeed, the 'class > __multi__' code no longer mentions StdObjSpace. I agree that this is an advantage. ... > More specifically, after a definition like > > class __multi__(X, Y): > def f(x, y, extra): > ... > > we would have the function object 'f()' stored into each of the lists > 'X.__dispatch_on_arg_1__' and 'Y.__dispatch_on_arg_2__'. Then a call like > 'f(x,y,extra)' looks up the types of 'x' and 'y' and searches for a function > object that would be present both in 'x.__class__.__dispatch_on_arg_1__' and > 'y.__class__.__dispatch_on_arg_2__'. What is nice with this approach is the > removal of the need for a central "repository" of functions, role which was > currently held by StdObjSpace.add. Well, I see that class __multi__ should have a special meta-class, in order to behave as you like? Or is it just a placeholder, interpreted by something else? > BTW the new order of the arguments ('space' at the end) is such that > single-dispatch is a particular case of Python's original dispatch semantics, > with the argument we dispatch on being first (althought that original semantic > is also extended by the delegation stuff, e.g. delegating from 'int' to 'long' > for methods that fail to be implemented by 'int' for overflow reasons). I don't get why spae at the end is an advantage? Just for not pretending to be something like "self"? Please give me an example how I would change my intobject implementation. Do you mean that instead of the single functions, I should put together such a __multi__ class? The current way of doing things was not so bad, since we had a defined interface which operations exist. How should it be done without StdObjSpace.add.register(int_int_add, W_IntObject, W_IntObject) Also, how about sticking with one agreed interface for, say, at least two weeks and have more objects implemented? I'm not happy to change tested code all over again. At least, with the current frequency of design changes, I'm a bit reluctant to do anything on IntObject. ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From lalo at laranja.org Tue Feb 25 12:12:06 2003 From: lalo at laranja.org (Lalo Martins) Date: Tue, 25 Feb 2003 08:12:06 -0300 Subject: [pypy-dev] IRC chat In-Reply-To: <20030224181035.M11666@prim.han.de> References: <20030224170533.E12664F06@bespin.org> <20030224181035.M11666@prim.han.de> Message-ID: <20030225111206.GJ19543@laranja.org> On Mon, Feb 24, 2003 at 06:10:35PM +0100, holger krekel wrote: > > > > I think the idea of setting up an IRC channel has been mentioned several times > > here and at the sprint, but as far as I know nothing concrete occurred. So > > unless somebody would prefer another server I suggest irc.freenode.net:6667, > > #pypy. > > i just joined but how do we make this channel permanent? Armin, if you can > then please join, so we can discuss this online :-) It may be that Pypy didn't yet reach the point in development where my skill and experience may be useful; however, I'm a very experienced IRC user and works requires that I stay on IRC all the time I can (and when I'm not, I leave my client connected, with the nickname "lalo[out]"). Therefore I volunteer to run the IRC channel. I took the liberty of registering the channel. If the project leaders who want the operator flag would send me their nicknames, I can set them up with ChanServ. (But you need to register your nickname first with NickServ. Look for me in the channel if you don't know how to do that.) []s, |alo +---- -- Those who trade freedom for security lose both and deserve neither. -- http://www.laranja.org/ mailto:lalo at laranja.org pgp key: http://www.laranja.org/pessoal/pgp Eu jogo RPG! (I play RPG) http://www.eujogorpg.com.br/ GNU: never give up freedom http://www.gnu.org/ From mwh at python.net Tue Feb 25 15:04:51 2003 From: mwh at python.net (Michael Hudson) Date: Tue, 25 Feb 2003 14:04:51 +0000 Subject: [pypy-dev] Re: stdobjspace status References: <20030224164413.51465501B@bespin.org> Message-ID: <2mu1ese9rw.fsf@starship.python.net> Armin Rigo writes: > The stdobjspace is getting ready to be tested together with the interpreter, > but is not quite there yet. If someone (Michael?) can figure out a good > solution for the END_FINALLY problem, I'd love to hear it. The problem is > that in a try:except: block, when we enter the except: part, there are some > control paths that reach the final END_FINALLY opcode (re-raising the > exception) and some that don't (caught exception). It makes it hard to know > when we must remove from the blockstack the description of the potentially > re-raisable exception. Is this the problem that we sometimes push interpreter level objects onto the applications's stack? I'm not sure we have much freedom here unless we fiddle the compiled bytecodes. But I'm still a little fuzzy on the issues. Cheers, M. -- Unfortunately, nigh the whole world is now duped into thinking that silly fill-in forms on web pages is the way to do user interfaces. -- Erik Naggum, comp.lang.lisp From tismer at tismer.com Tue Feb 25 18:24:29 2003 From: tismer at tismer.com (Christian Tismer) Date: Tue, 25 Feb 2003 18:24:29 +0100 Subject: [pypy-dev] Re: stdobjspace status In-Reply-To: <2mu1ese9rw.fsf@starship.python.net> References: <20030224164413.51465501B@bespin.org> <2mu1ese9rw.fsf@starship.python.net> Message-ID: <3E5BA6CD.6080806@tismer.com> Michael Hudson wrote: ... > Is this the problem that we sometimes push interpreter level objects > onto the applications's stack? I'm not sure we have much freedom here > unless we fiddle the compiled bytecodes. But I'm still a little fuzzy > on the issues. Oh, do we do that? The CPython equivalent would be to push plain integers, mixed with real object addresses :-) I think we should better not do this, 'cause these two worlds are meant to be absolutely incompatible without wrappers. It might make sense if I extend the set of restricted objects so we can use them for the interpreter. For example, I could write a derived list type, which is only able to hold either wrapped objects, or None, which would act line NULL in C. Such restricted objects would also make it very clear to a compiler, what code to create. cheers - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From mwh at python.net Tue Feb 25 18:27:56 2003 From: mwh at python.net (Michael Hudson) Date: Tue, 25 Feb 2003 17:27:56 +0000 Subject: [pypy-dev] Re: stdobjspace status References: <20030224164413.51465501B@bespin.org> <2mu1ese9rw.fsf@starship.python.net><3E5BA6CD.6080806@tismer.com> Message-ID: <2mof50e0df.fsf@starship.python.net> Christian Tismer writes: > Michael Hudson wrote: > ... > >> Is this the problem that we sometimes push interpreter level objects >> onto the applications's stack? I'm not sure we have much freedom here >> unless we fiddle the compiled bytecodes. But I'm still a little fuzzy >> on the issues. > > Oh, do we do that? At the moment, yes. Well, occasionally. > The CPython equivalent would be to push plain integers, > mixed with real object addresses :-) > I think we should better not do this, 'cause these > two worlds are meant to be absolutely incompatible > without wrappers. Yes! It's a bug. But neither Armin nor myself have found the brain cells to sort this one out yet. Cheers, M. -- If comp.lang.lisp *is* what vendors are relying on to make or break Lisp sales, that's more likely the problem than is the effect of any one of us on such a flimsy marketing strategy... -- Kent M Pitman, comp.lang.lisp From tismer at tismer.com Tue Feb 25 19:03:38 2003 From: tismer at tismer.com (Christian Tismer) Date: Tue, 25 Feb 2003 19:03:38 +0100 Subject: [pypy-dev] Re: stdobjspace status In-Reply-To: <2mof50e0df.fsf@starship.python.net> References: <20030224164413.51465501B@bespin.org> <2mu1ese9rw.fsf@starship.python.net><3E5BA6CD.6080806@tismer.com> <2mof50e0df.fsf@starship.python.net> Message-ID: <3E5BAFFA.3080305@tismer.com> Michael Hudson wrote: > Christian Tismer writes: > > >>Michael Hudson wrote: >>... >> >> >>>Is this the problem that we sometimes push interpreter level objects >>>onto the applications's stack? I'm not sure we have much freedom here >>>unless we fiddle the compiled bytecodes. But I'm still a little fuzzy >>>on the issues. >> >>Oh, do we do that? ... > Yes! It's a bug. But neither Armin nor myself have found the brain > cells to sort this one out yet. Hey, this is then a very good reason to implement a restricted array object. You just plug it into the frame as its stack, and it will crash when it feels abused. :-) ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From stephan.diehl at gmx.net Wed Feb 26 11:12:03 2003 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Wed, 26 Feb 2003 11:12:03 +0100 Subject: [pypy-dev] Base Object library (was: stdobjspace status) In-Reply-To: <3E5BAFFA.3080305@tismer.com> References: <20030224164413.51465501B@bespin.org> <2mof50e0df.fsf@starship.python.net> <3E5BAFFA.3080305@tismer.com> Message-ID: <20030226101130.3E52A5B2C1@thoth.codespeak.net> >> [...] > > Hey, this is then a very good reason to implement > a restricted array object. You just plug it into > the frame as its stack, and it will crash when > it feels abused. :-) > > ciao - chris I don't know if this is in line with Christians comment, but I'd like to make a kind of proposal. Until now, I'm very uncomfortable with the way we are implementing the stdobjspace. I have the feeling (there it is, this kind of discussion shouldn't be based on "feelings") that our implementation is somewhat circular, since we are using real Python objects to base our implementation of PyPython objects on. I'd rather have the notion of an external library our internal building blocks are coming from. To make it more clear what I mean, I'll describe, what I have in mind with "numbers". During the last few days, there was (is) an discussion on c.l.p about float arithmetic and some problems, one can run into. In that discussion, the following paper was mentioned: http://www2.hursley.ibm.com/decimal/decarith.html It describes a number implementation that is compatible with "floats" and "ints". To make it short, a number is a triple (sign,coefficient,exponent) whereas the exponent is a base 10 exponent. 0.25 thus could be written as (+,25,-2) The paper goes on in describing all the operations, exceptions, etc. that are needed to implement a number library based on that number model. The way, Python numbers are implemented at the moment, is to take the low level C "numbers" and wrap them as good as it gets and add a lot of code where things are missing. I'd rather take a usefull number implementation and just wrap that (and translate only the exceptions, errors). In order to implement such a number scheme, we'd need of course some notion of bit fields. This boils down to: 1. define the most basic building blocks like bit fields + corresponding memory management. 2. define numbers, lists, strings, dicts, etc. on top of 1. Please note that these objects have nothing to do with the objspace we are defining, but are used as the building blocks and are never exposed outside the objspace. Does this make sense? Cheers Stephan From stephan.diehl at gmx.net Wed Feb 26 11:28:30 2003 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Wed, 26 Feb 2003 11:28:30 +0100 Subject: [pypy-dev] svn and issue tracking doesn't work Message-ID: <20030226102754.CF54E5B2C1@thoth.codespeak.net> codespeak.net doesn't work as expected. codespeak.net:8080/svn doesn't respond at all. With the issue tracking, I can log in but not create a new issue (or have a look at the existing ones). Cheers Stephan From hpk at trillke.net Wed Feb 26 12:09:36 2003 From: hpk at trillke.net (holger krekel) Date: Wed, 26 Feb 2003 12:09:36 +0100 Subject: [pypy-dev] svn and issue tracking doesn't work In-Reply-To: <20030226102754.CF54E5B2C1@thoth.codespeak.net>; from stephan.diehl@gmx.net on Wed, Feb 26, 2003 at 11:28:30AM +0100 References: <20030226102754.CF54E5B2C1@thoth.codespeak.net> Message-ID: <20030226120935.C11666@prim.han.de> [Stephan Diehl Wed, Feb 26, 2003 at 11:28:30AM +0100] > codespeak.net doesn't work as expected. > codespeak.net:8080/svn doesn't respond at all. > With the issue tracking, I can log in but not create a new issue (or have a > look at the existing ones). Well, issue tracking works for me. what exactly doesn't work for you? Are you using exactly http://codespeak.net/issues/pypy/ ? I am using Konqueror and Mozilla which both work. I noticed too that subversion doesn't answer requests so jum and/or me will investigate. It did work this morning. holger From stephan.diehl at gmx.net Wed Feb 26 12:34:22 2003 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Wed, 26 Feb 2003 12:34:22 +0100 Subject: [pypy-dev] svn and issue tracking doesn't work In-Reply-To: <20030226120935.C11666@prim.han.de> References: <20030226102754.CF54E5B2C1@thoth.codespeak.net> <20030226120935.C11666@prim.han.de> Message-ID: <20030226113350.12D2D5B2CF@thoth.codespeak.net> On Wednesday 26 February 2003 12:09, you wrote: > [Stephan Diehl Wed, Feb 26, 2003 at 11:28:30AM +0100] > > > codespeak.net doesn't work as expected. > > codespeak.net:8080/svn doesn't respond at all. > > With the issue tracking, I can log in but not create a new issue (or have > > a look at the existing ones). > > Well, issue tracking works for me. what exactly > doesn't work for you? Are you using exactly > > http://codespeak.net/issues/pypy/ > > ? I am using Konqueror and Mozilla which both work. I just realized that Mozilla works, but not Opera 6.11 (on Linux) With Opera (just in case you want to chase this), if I go for example to http://codespeak.net/issues/issue8 I get a 404 returncode Very strange. Stephan [...] > > holger From hpk at trillke.net Wed Feb 26 12:44:14 2003 From: hpk at trillke.net (holger krekel) Date: Wed, 26 Feb 2003 12:44:14 +0100 Subject: [pypy-dev] svn and issue tracking doesn't work In-Reply-To: <200302261134.MAA22624@prim.merlinux.de>; from stephan.diehl@gmx.net on Wed, Feb 26, 2003 at 12:34:22PM +0100 References: <20030226102754.CF54E5B2C1@thoth.codespeak.net> <20030226120935.C11666@prim.han.de> <200302261134.MAA22624@prim.merlinux.de> Message-ID: <20030226124414.D11666@prim.han.de> [Stephan Diehl Wed, Feb 26, 2003 at 12:34:22PM +0100] > On Wednesday 26 February 2003 12:09, you wrote: > > [Stephan Diehl Wed, Feb 26, 2003 at 11:28:30AM +0100] > > > > > codespeak.net doesn't work as expected. > > > codespeak.net:8080/svn doesn't respond at all. > > > With the issue tracking, I can log in but not create a new issue (or have > > > a look at the existing ones). > > > > Well, issue tracking works for me. what exactly > > doesn't work for you? Are you using exactly > > > > http://codespeak.net/issues/pypy/ > > > > ? I am using Konqueror and Mozilla which both work. > > I just realized that Mozilla works, but not Opera 6.11 (on Linux) > > With Opera (just in case you want to chase this), if I go for example to > http://codespeak.net/issues/issue8 this url is wrong. when i click there i get http://codespeak.net/issues/pypy/issue4 which works. > Very strange. indeed :-) holger From jum at anubis.han.de Wed Feb 26 12:56:58 2003 From: jum at anubis.han.de (Jens-Uwe Mager) Date: Wed, 26 Feb 2003 12:56:58 +0100 Subject: [pypy-dev] svn and issue tracking doesn't work In-Reply-To: <20030226120935.C11666@prim.han.de> References: <20030226102754.CF54E5B2C1@thoth.codespeak.net> <20030226120935.C11666@prim.han.de> Message-ID: <20030226115658.GE3820@ANUBIS> On Wed, Feb 26, 2003 at 12:09 +0100, holger krekel wrote: > I noticed too that subversion doesn't answer requests > so jum and/or me will investigate. It did work this > morning. I have no idea what happened, but I had to do an svnadmin recover on the database. Fortunately after having used db_archive to properly clean up the Berkeley DB transaction log files this runs in a few seconds instead of multiple hours. I do now run a nightly script to run db_archive as well as doing an incremental dump to a text file. -- Jens-Uwe Mager From stephan.diehl at gmx.net Wed Feb 26 13:09:09 2003 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Wed, 26 Feb 2003 13:09:09 +0100 Subject: [pypy-dev] svn and issue tracking doesn't work In-Reply-To: <20030226124414.D11666@prim.han.de> References: <20030226102754.CF54E5B2C1@thoth.codespeak.net> <200302261134.MAA22624@prim.merlinux.de> <20030226124414.D11666@prim.han.de> Message-ID: <20030226120836.71F2D5B2C1@thoth.codespeak.net> [...] > > > > With Opera (just in case you want to chase this), if I go for example to > > http://codespeak.net/issues/issue8 > > this url is wrong. That's probably the case. What I should have mentioned, of cause, is that I didn't enter that URL by hand, but is a link on http://codespeak.net/issues/pypy (when using Opera) To be more precise: the page contains the link and for some strange reason, Opera "forgets" the pypy part of the complete url. Anyway, since Mozilla is working, I'm happy and everybody else is probably as well. Stephan > > when i click there i get > > http://codespeak.net/issues/pypy/issue4 > > which works. > > > Very strange. > > indeed :-) > > holger From rodrigobamboo at terra.com.br Wed Feb 26 13:17:40 2003 From: rodrigobamboo at terra.com.br (Rodrigo B. de Oliveira) Date: Wed, 26 Feb 2003 09:17:40 -0300 Subject: [pypy-dev] svn and issue tracking doesn't work References: <20030226102754.CF54E5B2C1@thoth.codespeak.net><200302261134.MAA22624@prim.merlinux.de> <20030226124414.D11666@prim.han.de> <20030226120836.71F2D5B2C1@thoth.codespeak.net> Message-ID: <003801c2dd91$0f724a60$a30aa1c8@rodrigobamboo.com> > [...] > > > > > > With Opera (just in case you want to chase this), if I go for example to > > > http://codespeak.net/issues/issue8 > > > > this url is wrong. > > That's probably the case. What I should have mentioned, of cause, is that I > didn't enter that URL by hand, but is a link on > http://codespeak.net/issues/pypy (when using Opera) > To be more precise: the page contains the link and for some > strange reason, Opera "forgets" the pypy part of the complete url. > > Anyway, since Mozilla is working, I'm happy and everybody else is probably as > well. > Just letting you know that the same problem happens to IE 6. Rodrigo From anthony at interlink.com.au Wed Feb 26 15:00:08 2003 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 27 Feb 2003 01:00:08 +1100 Subject: [pypy-dev] svn and issue tracking doesn't work In-Reply-To: <20030226120836.71F2D5B2C1@thoth.codespeak.net> Message-ID: <200302261400.h1QE09c02904@localhost.localdomain> >>> Stephan Diehl wrote > That's probably the case. What I should have mentioned, of cause, is that I > didn't enter that URL by hand, but is a link on > http://codespeak.net/issues/pypy (when using Opera) Try going, instead, to http://codespeak.net/issues/pypy/ (note trailing /) -- Anthony Baxter It's never too late to have a happy childhood. From stephan.diehl at gmx.net Wed Feb 26 15:54:43 2003 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Wed, 26 Feb 2003 15:54:43 +0100 Subject: [pypy-dev] svn and issue tracking doesn't work In-Reply-To: <200302261400.h1QE09c02904@localhost.localdomain> References: <200302261400.h1QE09c02904@localhost.localdomain> Message-ID: <20030226145409.C96C45A7E6@thoth.codespeak.net> On Wednesday 26 February 2003 15:00, you wrote: > >>> Stephan Diehl wrote > > > > That's probably the case. What I should have mentioned, of cause, is that > > I didn't enter that URL by hand, but is a link on > > http://codespeak.net/issues/pypy (when using Opera) > > Try going, instead, to http://codespeak.net/issues/pypy/ > (note trailing /) That's a good one. I just love these browser/webserver specialities :-) I'm sure there is an apache configuration for this kind of behaviour. Thanks for the info Stephan From hpk at trillke.net Wed Feb 26 15:54:55 2003 From: hpk at trillke.net (holger krekel) Date: Wed, 26 Feb 2003 15:54:55 +0100 Subject: [pypy-dev] svn and issue tracking doesn't work In-Reply-To: <003801c2dd91$0f724a60$a30aa1c8@rodrigobamboo.com>; from rodrigobamboo@terra.com.br on Wed, Feb 26, 2003 at 09:17:40AM -0300 References: <20030226102754.CF54E5B2C1@thoth.codespeak.net><200302261134.MAA22624@prim.merlinux.de> <20030226124414.D11666@prim.han.de> <20030226120836.71F2D5B2C1@thoth.codespeak.net> <003801c2dd91$0f724a60$a30aa1c8@rodrigobamboo.com> Message-ID: <20030226155455.H11666@prim.han.de> [Rodrigo B. de Oliveira Wed, Feb 26, 2003 at 09:17:40AM -0300] > > [...] > > > > > > > > With Opera (just in case you want to chase this), if I go for example > to > > > > http://codespeak.net/issues/issue8 > > > > > > this url is wrong. > > > > That's probably the case. What I should have mentioned, of cause, is that > I > > didn't enter that URL by hand, but is a link on > > http://codespeak.net/issues/pypy (when using Opera) > > To be more precise: the page contains the link and for > some > > strange reason, Opera "forgets" the pypy part of the complete url. > > > > Anyway, since Mozilla is working, I'm happy and everybody else is probably > as > > well. > > > > Just letting you know that the same problem happens to IE 6. does this problem persist with the correc url? Never forget the trailing '/' http://codespeak.net/issues/pypy/ ^^^ otherwise you will get strange effects like wrong urls etc. The problem is known with roundup but i am not sure it is fixed. holger From tismer at tismer.com Wed Feb 26 16:34:57 2003 From: tismer at tismer.com (Christian Tismer) Date: Wed, 26 Feb 2003 16:34:57 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <200302261009.h1QA9Jc23764@trixie.triqs.com> References: <20030224164413.51465501B@bespin.org> <2mof50e0df.fsf@starship.python.net> <3E5BAFFA.3080305@tismer.com> <200302261009.h1QA9Jc23764@trixie.triqs.com> Message-ID: <3E5CDEA1.7080401@tismer.com> Stephan Diehl wrote: ... > This boils down to: > 1. define the most basic building blocks like bit fields + corresponding > memory management. > 2. define numbers, lists, strings, dicts, etc. on top of 1. This is actually my plan. It just takes more time. > Please note that these objects have nothing to do with the objspace we are > defining, but are used as the building blocks and are never exposed outside > the objspace. > > Does this make sense? Yes, makes sense. For the time being, my r_int objects for instance are derived from true integers, but they are *meant* to denote simple, internal integers. Their behavior is modelled this way, and a compiler is supposed to recognize r_int and generate the primitive code instead of object calls. In that sense, I'm already using primitive types. Well, the nice thing is, that these primitive types can be used both in objectspace and outside of it. Sure, they have to be not intermixed, and the meaning of r_int in objectspace is clearly that we want these objects to be replaced by primitives. Nevertheless, r_int can be imported from application space, becoming an application object, now. This is another level, "other world", but still the r_int objects make sense, since a compiler on application space (like Psyco) can now drive concluions of the usage of r_int, too. So the meaning of the primitive types is always to provide a way down to the really implementable layers, regardless in which level of world you just happen to be. It even makes sense if you run PyPy on PyPy which is run on CPython, etc. ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From rodrigobamboo at terra.com.br Wed Feb 26 17:11:51 2003 From: rodrigobamboo at terra.com.br (Rodrigo B. de Oliveira) Date: Wed, 26 Feb 2003 13:11:51 -0300 Subject: [pypy-dev] svn and issue tracking doesn't work References: <20030226102754.CF54E5B2C1@thoth.codespeak.net><200302261134.MAA22624@prim.merlinux.de> <20030226124414.D11666@prim.han.de> <20030226120836.71F2D5B2C1@thoth.codespeak.net> <003801c2dd91$0f724a60$a30aa1c8@rodrigobamboo.com> <20030226155455.H11666@prim.han.de> Message-ID: <001e01c2ddb1$c643d7c0$a30aa1c8@rodrigobamboo.com> It works fine with the trailing '/', thanks. Rodrigo ----- Original Message ----- From: "holger krekel" To: "Rodrigo B. de Oliveira" Cc: "Stephan Diehl" ; "holger krekel" ; Sent: Wednesday, February 26, 2003 11:54 AM Subject: Re: [pypy-dev] svn and issue tracking doesn't work > [Rodrigo B. de Oliveira Wed, Feb 26, 2003 at 09:17:40AM -0300] >> Just letting you know that the same problem happens to IE 6. > > does this problem persist with the correc url? Never forget > the trailing '/' > > http://codespeak.net/issues/pypy/ > ^^^ > > otherwise you will get strange effects like wrong urls etc. > The problem is known with roundup but i am not sure it is > fixed. > > holger > From stephan.diehl at gmx.net Wed Feb 26 17:42:35 2003 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Wed, 26 Feb 2003 17:42:35 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <3E5CDEA1.7080401@tismer.com> References: <20030224164413.51465501B@bespin.org> <200302261009.h1QA9Jc23764@trixie.triqs.com> <3E5CDEA1.7080401@tismer.com> Message-ID: <20030226164202.478BB5B919@thoth.codespeak.net> On Wednesday 26 February 2003 16:34, you wrote: > Stephan Diehl wrote: > ... > > > This boils down to: > > 1. define the most basic building blocks like bit fields + corresponding > > memory management. > > 2. define numbers, lists, strings, dicts, etc. on top of 1. > > This is actually my plan. It just takes more time. > > > Please note that these objects have nothing to do with the objspace we > > are defining, but are used as the building blocks and are never exposed > > outside the objspace. > > > > Does this make sense? > > Yes, makes sense. Puh, so the week was not completely lost on me :-) > For the time being, my r_int objects for instance > are derived from true integers, but they are *meant* > to denote simple, internal integers. Their > behavior is modelled this way, and a compiler is > supposed to recognize r_int and generate the primitive > code instead of object calls. In that sense, I'm > already using primitive types. At the moment, the constructor of W_IntObject looks like: def __init__(w_self, intval): w_self.intval = r_int(intval) At the end, this should look like: def __init__(w_self, rintval): w_self.intval = rintval where rintval is already the restricted intval. Am I right here? [...] > > So the meaning of the primitive types is always > to provide a way down to the really implementable > layers, regardless in which level of world you > just happen to be. It even makes sense if you run > PyPy on PyPy which is run on CPython, etc. I take it then that in the end, we get something like the following: ######################## # PyPy Interpreter # ######################## # StdObjSpace # ######################## # basic/primitive Types # ######################## These basic types have nothing to do at all with the StdObjSpace and are implemented at first in Python (for convinience) and later probably in C or Assembler. The ObjSpace just has an interface to these basic types. So, in the floatobject module for example, I'd rather define the float_float_add function like this (I left out the errorchecking): ------------------------------------------------------------------------- import basicfloat def float_float_add(space, w_float1, w_float2): x = w_float1.floatval y = w_float2.floatval z = basicfloat.add(x,y) return W_FloatObject(z) ------------------------------------------------------------------------- instead of ------------------------------------------------------------------------- def float_float_add(space, w_float1, w_float2): x = w_float1.floatval y = w_float2.floatval z = x + y return W_FloatObject(z) -------------------------------------------------------------------------- The most important thing seems to me to define content and interface of the basic types. I'd suggest one module "basictypes" that defines the number and string definitions and operations. Apart from that (or together?) we'll need a general bitfield (bytefield?) and then, on top of that, the list and dictionary types. Cheers Stephan > > ciao - chris From tismer at tismer.com Wed Feb 26 18:33:39 2003 From: tismer at tismer.com (Christian Tismer) Date: Wed, 26 Feb 2003 18:33:39 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <200302261639.h1QGdoc26254@trixie.triqs.com> References: <20030224164413.51465501B@bespin.org> <200302261009.h1QA9Jc23764@trixie.triqs.com> <3E5CDEA1.7080401@tismer.com> <200302261639.h1QGdoc26254@trixie.triqs.com> Message-ID: <3E5CFA73.3070609@tismer.com> Stephan Diehl wrote: ... > At the moment, the constructor of W_IntObject looks like: > > def __init__(w_self, intval): > w_self.intval = r_int(intval) > > At the end, this should look like: > > def __init__(w_self, rintval): > w_self.intval = rintval > > where rintval is already the restricted intval. > Am I right here? May be, not sure, I only want to make sure that self.intval is an r_int. >>So the meaning of the primitive types is always >>to provide a way down to the really implementable >>layers, regardless in which level of world you >>just happen to be. It even makes sense if you run >>PyPy on PyPy which is run on CPython, etc. > > > I take it then that in the end, we get something like the following: > > ######################## > # PyPy Interpreter # > ######################## > # StdObjSpace # > ######################## > # basic/primitive Types # > ######################## > > These basic types have nothing to do at all with the StdObjSpace and are > implemented at first in Python (for convinience) and later probably in C or > Assembler. Yes, but don't get me wrong: I will not rewrite them in C, but a code generator will do that. SO, actually, most of the work is done after writing it in Python. > The ObjSpace just has an interface to these basic types. > So, in the floatobject module for example, I'd rather define the > float_float_add function like this (I left out the errorchecking): > > ------------------------------------------------------------------------- > import basicfloat > > def float_float_add(space, w_float1, w_float2): > x = w_float1.floatval > y = w_float2.floatval > z = basicfloat.add(x,y) > > return W_FloatObject(z) > ------------------------------------------------------------------------- > > instead of > ------------------------------------------------------------------------- > def float_float_add(space, w_float1, w_float2): > x = w_float1.floatval > y = w_float2.floatval > z = x + y > > return W_FloatObject(z) > -------------------------------------------------------------------------- Why that? You can do that, but I don't see the point. This is why I defined r_int as derived from true ints: I want to keep the simple notation of "+". Why should I use a special method? If there is a special method, then it can be found in the class definition of r_int. > The most important thing seems to me to define content and interface of the > basic types. > I'd suggest one module "basictypes" that defines the number and string > definitions and operations. Apart from that (or together?) we'll need a > general bitfield (bytefield?) and then, on top of that, the list and > dictionary types. Yes. That is what I hope to do very soon. ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From stephan.diehl at gmx.net Wed Feb 26 19:33:37 2003 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Wed, 26 Feb 2003 19:33:37 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <3E5CFA73.3070609@tismer.com> References: <20030224164413.51465501B@bespin.org> <200302261639.h1QGdoc26254@trixie.triqs.com> <3E5CFA73.3070609@tismer.com> Message-ID: <20030226183302.9AAB15B910@thoth.codespeak.net> On Wednesday 26 February 2003 18:33, you wrote: > Stephan Diehl wrote: > ... > > > At the moment, the constructor of W_IntObject looks like: > > > > def __init__(w_self, intval): > > w_self.intval = r_int(intval) > > > > At the end, this should look like: > > > > def __init__(w_self, rintval): > > w_self.intval = rintval > > > > where rintval is already the restricted intval. > > Am I right here? > > May be, not sure, I only want to make sure > that self.intval is an r_int. I guess that depends a little on who'll create the W_ Objects. (I have no idea yet) > [...] > > > > These basic types have nothing to do at all with the StdObjSpace and are > > implemented at first in Python (for convinience) and later probably in C > > or Assembler. > > Yes, but don't get me wrong: > I will not rewrite them in C, but > a code generator will do that. > SO, actually, most of the work is done after > writing it in Python. No, my memory is not that bad :-) I just wanted to emphasize, that the basic objects are in some way coming from somewhere else. It just doesn't matter if they were compiled from Python or implemented in some completely different language. > > > The ObjSpace just has an interface to these basic types. > > So, in the floatobject module for example, I'd rather define the > > float_float_add function like this (I left out the errorchecking): > > > > ------------------------------------------------------------------------- > > import basicfloat > > > > def float_float_add(space, w_float1, w_float2): > > x = w_float1.floatval > > y = w_float2.floatval > > z = basicfloat.add(x,y) > > > > return W_FloatObject(z) > > ------------------------------------------------------------------------- > > > > instead of > > ------------------------------------------------------------------------- > > def float_float_add(space, w_float1, w_float2): > > x = w_float1.floatval > > y = w_float2.floatval > > z = x + y > > > > return W_FloatObject(z) > > ------------------------------------------------------------------------- > >- > > Why that? > You can do that, but I don't see the point. The point (at least for me) is to distinguish the basic types from normal python in an optical way. I find it difficult enough not to get an headache over all these different levels of implementation and what kind of operation is allowed. With the explicit library call I'd try to keep my confusion about this at bay. > This is why I defined r_int as derived from > true ints: I want to keep the simple notation > of "+". > Why should I use a special method? > If there is a special method, then > it can be found in the class definition > of r_int. > > > The most important thing seems to me to define content and interface of > > the basic types. > > I'd suggest one module "basictypes" that defines the number and string > > definitions and operations. Apart from that (or together?) we'll need a > > general bitfield (bytefield?) and then, on top of that, the list and > > dictionary types. > > Yes. That is what I hope to do very soon. I'll finish the floatobject module then and try to implement the number library I mentioned in my initial mail about that. > > ciao - chris Have fun Stephan From tismer at tismer.com Wed Feb 26 19:46:12 2003 From: tismer at tismer.com (Christian Tismer) Date: Wed, 26 Feb 2003 19:46:12 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <200302261639.h1QGdoc26254@trixie.triqs.com> References: <20030224164413.51465501B@bespin.org> <200302261009.h1QA9Jc23764@trixie.triqs.com> <3E5CDEA1.7080401@tismer.com> <200302261639.h1QGdoc26254@trixie.triqs.com> Message-ID: <3E5D0B74.7030709@tismer.com> Stephan Diehl wrote: Once again on this: > def __init__(w_self, rintval): > w_self.intval = rintval > > where rintval is already the restricted intval. > Am I right here? As said, I'm not sure if rintval will be an r_int already, it depends on the rest of the implementation which is quite a bit foggy to me, still. Maybe I define intval as a slot, and give it a setter method that always enforces the primitive r_int type. Also there will be a basic notation of primitive arrays. They might be built using the array module, but I think I like it better to implement it using derived list objects, with a restricted type attribute. And then, these objects are meant to describe just plain memory arrays, no Python objects. The are, but they don't behave like that. For instance, r_objectarray would be a list, which is restricted that it only can hold references to W_Object, or None which acts as NULL. r_objectarray would also override Python operators like "+" and "*", since these don't exist in RPython's arrays: An array can only be created once, with a fixed size. There is no concatenation or repetition. Instead, this has to be built upon that, in terms of the primitive arrays. So, W_ListObject would have some fields like self.objarray # r_objectarray self.maxlen # r_uint self.actlen # r_uint It has been suggested to not use maxlen, since we could use len(self.objarray), but I believe this is wrong to do. If we are modelling primitive arrays, then they don't support len() at all. Now, for example, the definition of concatenation for our new lists could read like so: class r_range(object): def __init__(self, *args): self.xr = xrange(*args) def __getitem__(self, idx): return r_int(self.xr[idx]) class W_ListObject(W_Object): __slots__ = ["objarray", "maxlen", "actlen"] def __init__(self, lng, data=None): self.objarray = newobjarray(lng) if data: for i in r_range(lng): self.objarray[i] = data[i] self.actlen = lng else: self.actlen = 0 self.maxlen = lng def list_list_add(space, w_lis1, w_lis2): lng1 = w_lis1.actlen lng2 = w_lis2.actlen lng = lng1 + lng2 res = W_ListObject(lng) for i in r_range(lng1): res.objarray[i] = w_lis1.objarray[i] for i in r_range(lng1, lng): res.objarray[i] = w_lis2.objarray[i] res.actlen = lng return res Maybe this is just very rough, but give a little an idea how things can work. Wile we are at it, Stephan, you wrote the longobject.py . You are probably aware of the fact that this implementation is slightly not what we need? You coded, for instance: def long_long_sub(space, w_long1, w_long2): x = w_long1.longval y = w_long2.longval try: z = x - y # <== ick! We don't have that! except Error,e: raise OperationError(Error, e) return W_LongObject(z) This is ok, if you rely on the existance of long objects. But for a real implementation, we need to use integer arrays like in longobject.c, and implement everything by hand. So how to fill longobject.py with real life? I think, we first need some basic structure like r_integerarray or maybe even better r_uintarray, and based upon that, we must build every and all the necessary loops. Anybody wants to try it? ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From hpk at trillke.net Wed Feb 26 20:16:59 2003 From: hpk at trillke.net (holger krekel) Date: Wed, 26 Feb 2003 20:16:59 +0100 Subject: [pypy-dev] codespeak downtime thursday 1pm-3pm (gmt+1) Message-ID: <20030226201659.O11666@prim.han.de> hello, please expect a downtime from 1pm to 3pm (gmt+1) tommorow. the server hosting the pypy project will move to a faster network location thanks to the nice people from Helios Software GmbH in Hannover. We will try to change the DNS addresses before moving the computer so that all the caches have time to adapt. Note, though, that Mozilla apparently *never* times out DNS-address resolutions. holger From stephan.diehl at gmx.net Wed Feb 26 20:22:04 2003 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Wed, 26 Feb 2003 20:22:04 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <3E5D0B74.7030709@tismer.com> References: <20030224164413.51465501B@bespin.org> <200302261639.h1QGdoc26254@trixie.triqs.com> <3E5D0B74.7030709@tismer.com> Message-ID: <20030226192132.0F8845B910@thoth.codespeak.net> [...] > > Wile we are at it, Stephan, > you wrote the longobject.py . > You are probably aware of the fact that > this implementation is slightly not what > we need? Yes, I'm aware. If it is already anything, then only a proxy. My initial idea was, of course, just to build the interface of the objects first --- so they can be already used as long as everything is in an CPython environment --- and provide the "real" implementation later. > > You coded, for instance: > > def long_long_sub(space, w_long1, w_long2): > x = w_long1.longval > y = w_long2.longval > try: > z = x - y # <== ick! We don't have that! > except Error,e: > raise OperationError(Error, e) > return W_LongObject(z) > > This is ok, if you rely on the existance of > long objects. But for a real implementation, > we need to use integer arrays like in > longobject.c, and implement everything by > hand. That's actually the place, where I'd really do it different to the CPython implementation. I'd factor out the r_longs to a library (as discussed several times in this thread) and would indeed write: ----------------------------------------------------------------------------------------- import basiclong # <== or something similar def long_long_sub(space, w_long1, w_long2): x = w_long1.longval y = w_long2.longval try: z = basiclong.sub(x , y) # except Error,e: raise OperationError(Error, e) return W_LongObject(z) --------------------------------------------------------------------------------------- As far as I'm concerned, the implementation of the basic/restricted types and the implementation of the W_XXXObjects should be clearly separated. > > So how to fill longobject.py with real life? > I think, we first need some basic structure > like r_integerarray or maybe even better > r_uintarray, and based upon that, we must > build every and all the necessary loops. > > Anybody wants to try it? > > ciao - chris Anyway, we are probably not too far apart in our opinion about this topic :-) Cheers Stephan From tismer at tismer.com Wed Feb 26 20:33:02 2003 From: tismer at tismer.com (Christian Tismer) Date: Wed, 26 Feb 2003 20:33:02 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <200302261919.h1QJJJc10226@trixie.triqs.com> References: <20030224164413.51465501B@bespin.org> <200302261639.h1QGdoc26254@trixie.triqs.com> <3E5D0B74.7030709@tismer.com> <200302261919.h1QJJJc10226@trixie.triqs.com> Message-ID: <3E5D166E.1010405@tismer.com> Stephan Diehl wrote: ... > That's actually the place, where I'd really do it different to the CPython > implementation. I'd factor out the r_longs to a library (as discussed several > times in this thread) and would indeed write: > ----------------------------------------------------------------------------------------- > import basiclong # <== or something similar > def long_long_sub(space, w_long1, w_long2): > x = w_long1.longval > y = w_long2.longval > try: > z = basiclong.sub(x , y) # > except Error,e: > raise OperationError(Error, e) > return W_LongObject(z) > --------------------------------------------------------------------------------------- I don't remember that this was discussed, or I misunderstood all the time. What library should it be which provides the longs? Maybe this works for longs, but what is then up with say, dictionaries? > As far as I'm concerned, the implementation of the basic/restricted types and > the implementation of the W_XXXObjects should be clearly separated. What do you mean? I can't see where I mixed them up. The ones are the building blocks of the others. ... > Anyway, we are probably not too far apart > in our opinion about this topic :-) Well, just this one: I don't understand the library idea. How do we get to a minimal, self-contained Python if we don't implement everything ourselves? ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From roccomoretti at netscape.net Thu Feb 27 04:23:01 2003 From: roccomoretti at netscape.net (Rocco Moretti) Date: Wed, 26 Feb 2003 22:23:01 -0500 Subject: [pypy-dev] Wiki Change Tracking Message-ID: <17035A26.3BB1634F.9ADE5C6A@netscape.net> I think there might be an issue with keeping track of Wiki page changes via RecentChanges (http://codespeak.net/moin/pypy/moin.cgi/RecentChanges). When I have gone in there recently, I've only seen the changes for the current day. There is no history past that. I suspect it may be an error, as in the past RecentChanges has given me a few weeks worth of changes. Is there some cron job or interaction with the subversion server which is causing changes older than 24 hours to be dropped? -Rocco __________________________________________________________________ The NEW Netscape 7.0 browser is now available. Upgrade now! http://channels.netscape.com/ns/browsers/download.jsp Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/ From paoloinvernizzi at dmsware.com Thu Feb 27 09:30:40 2003 From: paoloinvernizzi at dmsware.com (Paolo Invernizzi) Date: Thu, 27 Feb 2003 09:30:40 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <3E5D0B74.7030709@tismer.com> Message-ID: -----Original Message----- Christian Tismer wrote: >As said, I'm not sure if rintval will be >an r_int already, it depends on the rest >of the implementation which is quite a >bit foggy to me, still. >Maybe I define intval as a slot, and give >it a setter method that always enforces >the primitive r_int type. Actual W_IntObject looks like: def __init__(w_self, intval): w_self.intval = r_int(intval) -- versus -- def __init__(w_self, rintval): w_self.intval = rintval Someone can help me to clarify my mind *I'm a bit confused*...?? W_IntObject is the implementation of *python int type*, a wrapped object (a). the *rint* is one of the possible *concrete* type that can be handled by the W_IntObject in its job of implementing the python int (b). Pypy must emit C/Assembler/ObjC/...restricted-python??.. In other words... is not the *W_IntObject with r_int* one of the *possible* choice that pypy can choose for representing (to target code emission!) a plain python int (c)? Instead of r_int, for example, I can choose to use a tagged-rapresentation of an int, and write a t_int that check the overflow and so problems (and where to check arithmetics?)...(d). So (d) lead to *how/where* replace the r_int with the t_int. Actual version suggest that this is done implementing different W_IntObject (with the same interface! All are python *int* object); second version of the init suggest that this is done *outside*, and that the W_IntObject is a *neutral* implementation of the W_IntObject dispate whatever the *compiler/optimizer/developer-of-pypy* decided to use as concrete *intval* (e). I think we need some idea and prototipe of the code *above* W_IntObject (Armin help ;) The compiler do something like.... ??? - Ok, folk! - Let's analize that application! - Umm in this hot-part of the program I can *try* to specialize and use a *W_IntObject with tagged int*! - Let's recompile this chunk of the app using *W_IntObject with t_int*, a pure python implementation of tagged int. - Simply *recompile* the hot-part (is all python! Not to do Armin tricks for just-in-time assembler compilation!) and check if all is ok! - No overflow, no problems, ok! - Emit *real* C/Assembler/Whatever tagged repr. What part of the my personal brainstorm is wrong? ;-( Paolo Invernizzi From arigo at tunes.org Thu Feb 27 11:44:35 2003 From: arigo at tunes.org (Armin Rigo) Date: Thu, 27 Feb 2003 11:44:35 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <3E5D0B74.7030709@tismer.com> References: <20030224164413.51465501B@bespin.org> <200302261009.h1QA9Jc23764@trixie.triqs.com> <3E5CDEA1.7080401@tismer.com> <200302261639.h1QGdoc26254@trixie.triqs.com> <3E5D0B74.7030709@tismer.com> Message-ID: <20030227104435.GA30494@magma.unil.ch> Hello Christian, Whoow, I am afraid this thread is going in some wrong direction! As Paolo points out, this is getting confused. There are already several different levels at which things are working, and Stephan and you are adding yet another one. I don't say this is necessarily wrong, but things are not very clear. The W_IntObject class itself is supposed to be one possible implementation of Python's integers, described in a high-level point of view. The whole point is in that word "high-level". We (or at least I) want the objspace/std/* files to be translatable to whatever actual low-level language that we like, and not only C -- this is why the bitfield metaphor is not necessarily a good thing. When you are talking about r_int and similarly r_float or even r_long wrappers, you want to make it clear that you mean a "machine-level" plain object. But actually that was the whole point of RPython. The C translation of all the interpreter-level code will produce machine-level "int"s. In other words there is no deep reason to use a special wrapping mecanism for ints, floats, lists, whatever, to implement the W_XxxObject classes, because we will need *exactly the same* low-level objects to implement just *everything else* in the interpreter! For example, when the interpreter manipulates the current bytecode position, it manipulates what looks like Python integers and strings --- opcode = ord(self.bytecode[nextposition]) nextposition += 1 --- but we want this code to be translated into efficient C code, too. The least thing we want is to have to rewrite *all* this code with r_ints! So the *only* point of r_ints, as far as I can tell, is to have explicit control over bit width and overflow detection. There is no built-in Python type "integer with n bits" that generates the correct OverflowErrors. And there is no unsigned integer type, hence the need for r_uint. But apart from that, just use plain integers and it is fine! That is the whole purpose of writing our pypy in Python, isn't it? Creating r_long, r_float, r_list... looks like we are reinventing our own language. The confusion that may have led to this point is that we never explicitely said what objects we would allow in RPython. Remember, RPython is supposed to be Python with a few restrictions whose goal is *only* to ease the job of type inference and later translation. But the *existing* high-level constructs of Python must be kept! If we don't use them, we are back again at the C level and there is no point in writing pypy in Python. A couple of more specific points now... On Wed, Feb 26, 2003 at 07:46:12PM +0100, Christian Tismer wrote: > So, W_ListObject > would have some fields like > self.objarray # r_objectarray > self.maxlen # r_uint > self.actlen # r_uint > > It has been suggested to not use maxlen, since > we could use len(self.objarray), but I believe > this is wrong to do. If we are modelling primitive > arrays, then they don't support len() at all. I feel exactly the opposite. Just *use* real Python lists all the way, including most of their normal operations. This is nice because they describe high-level operations. In my view a W_ListObject would have only two fields: * self.objarray # a real Python list of wrapped items * self.length # the actual list length with self.length <= len(self.objarray). The self.objarray list grows by more than one item at a time, to amortize the cost of adding a single element at a time, just like what the current CPython implementation does. All this works fine, is normal pure Python code, and can be expected to translate to an efficient C implementation in which lists are just translated into malloc'ed blocks of PyObject* pointers. The point is that we still have a nice Pythonic high-level implementation of W_ListObject but nevertheless can translate it into an efficient low-level implementation. Another point I would like to raise is that we don't have to give types to every variable in pypy, like enforcing that a slot contains an r_int. This is just unPythonic. *All* types are expected to be computed during type inference. So don't even say that W_ListObject.objarray is a list of wrapped objects or Nones -- this can be figured out automatically. ..from Paolo's reply: > In other words... is not the *W_IntObject with r_int* one of the *possible* > choice that pypy can choose for representing (to target code emission!) a > plain python int (c)? > Instead of r_int, for example, I can choose to use a tagged-rapresentation > of an int, and write a t_int that check the overflow and so problems (and > where to check arithmetics?)...(d). This is right. Consider for example the case of *two* different implementations (that would both appear as having the 'int' type to users). Say that one is like Christian's W_IntObject+r_int, and the other one can only encode small, tagged integers. The choice to use one or both representations in an actual C implementation must be made by the RPython-to-C translator, and not in the object space. For example, if we want to produce something very similar to the current CPython, then we have no use for small, tagged integers. The question is thus, "how do we express things to allow for this?" Similarily, we may provide different implementations for lists, dictionaries, whatever; we may even consider that Python's "long" type is an unneeded hack, for long integers could be just another implementation for the 'int' type, which goes very much in the direction that Python seems to go with the recent automatic conversions of overflows to longs. The original intent of classes like W_IntObject was "one class, one implementation", and I think that we must stick to that idea because these classes are what are used for the multiple dispatch routines. I don't have a clear and complete answer for the rest of the question "how do we express things to allow for this?". I hope that this e-mail has clarified some points. Disagreement is welcome. I apologize to Christian and Stephan because it seems that we might have to reorganize the xxxobject.py sources, althought I'm not sure yet how. In an effort to go in that direction I'd like to add that nothing has been done yet about: * built-in methods (like list.append); the StdObjSpace.xxx.register() trick only works for built-in operators * non-built-in operators and methods, e.g. implementing something like long_long_add in application-space (longobject_app.py). A bient?t, Armin. From hpk at trillke.net Thu Feb 27 12:00:02 2003 From: hpk at trillke.net (holger krekel) Date: Thu, 27 Feb 2003 12:00:02 +0100 Subject: [pypy-dev] Wiki Change Tracking In-Reply-To: <17035A26.3BB1634F.9ADE5C6A@netscape.net>; from roccomoretti@netscape.net on Wed, Feb 26, 2003 at 10:23:01PM -0500 References: <17035A26.3BB1634F.9ADE5C6A@netscape.net> Message-ID: <20030227120002.Y11666@prim.han.de> [Rocco Moretti Wed, Feb 26, 2003 at 10:23:01PM -0500] > I think there might be an issue with keeping track of Wiki page changes via RecentChanges (http://codespeak.net/moin/pypy/moin.cgi/RecentChanges). > When I have gone in there recently, I've only seen the changes for the current day. There is no history past that. true. > I suspect it may be an error, as in the past RecentChanges has given me a few weeks worth of changes. hmmm. we are in the process of refactoring the wiki-code to tie into subversion. The goal is that you can edit *and* rename wiki pages locally. In the end, the Wiki/subversion way is to be used for documenting the project. I suspect that moinmoin's current way of computing the RecentChanges (which is a macro) is currently broken. > Is there some cron job or interaction with the subversion server which is causing changes older than 24 hours to be dropped? there used to be a cron-job but it is not running right now. After the server move today i look into this history-issue and try to fix it. holger From pypy-issues at codespeak.net Thu Feb 27 12:05:18 2003 From: pypy-issues at codespeak.net (holger krekel) Date: Thu, 27 Feb 2003 11:05:18 +0000 Subject: [pypy-dev] [issue13] RecentChanges is broken in the wiki Message-ID: <1046343918.18.0.272614105958.issue@codespeak.net> New submission from holger krekel : Rocco reported that the RecentChanges page is currently broken. it only shows the history of the past few days. ---------- assignedto: hpk messages: 24 nosy: hpk, rocco priority: bug status: unread title: RecentChanges is broken in the wiki topic: wiki __________________________________________________ PyPython issue tracker http://codespeak.net/issues/pypy/issue13 __________________________________________________ From pedronis at bluewin.ch Thu Feb 27 11:55:42 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Thu, 27 Feb 2003 11:55:42 +0100 Subject: [pypy-dev] question References: <20030224164413.51465501B@bespin.org><200302261009.h1QA9Jc23764@trixie.triqs.com> <3E5CDEA1.7080401@tismer.com><200302261639.h1QGdoc26254@trixie.triqs.com> <3E5D0B74.7030709@tismer.com> <20030227104435.GA30494@magma.unil.ch> Message-ID: <013c01c2de4e$dc64bcc0$6d94fea9@newmexico> I admit for now I have just skimmed over the code. I can more or less see how something like classic-style classes can be implemented. Have you already thought on how to implement new-style classes and in particular built-in types subclassing like in: class myint(int): ... e.g. will the implemented multi-dispatching intepreted statically at PyPy static translation time ... or should the final low-level runtime support it dynamically? regards. From pypy-issues at codespeak.net Thu Feb 27 12:12:29 2003 From: pypy-issues at codespeak.net (holger krekel) Date: Thu, 27 Feb 2003 11:12:29 +0000 Subject: [pypy-dev] [issue14] issues go to pypy-dev Message-ID: <1046344348.77.0.923218363496.issue@codespeak.net> New submission from holger krekel : As requested on the sprint pypy-dev now gets notification of "new issues". With some new "detector" code for roundup i configured the pypy-issue tracker so that new issues will get sent to the pypy-dev list. The list itself is not "nosy" and everyone has to decide to "nosy in" on specific issues. Don't reply to the new post but follow the web-link in the posting. NEVER send mail to both the issue-tracker and the mailing list. This might currently lead to the implosion of the universe, who knows :-) I'll try to improve the way that mailing lists and issue-trackers interact and will communicate with the roundup developers about this. ---------- messages: 25 nosy: hpk priority: feature status: testing title: issues go to pypy-dev __________________________________________________ PyPython issue tracker http://codespeak.net/issues/pypy/issue14 __________________________________________________ From paoloinvernizzi at dmsware.com Thu Feb 27 13:00:12 2003 From: paoloinvernizzi at dmsware.com (Paolo Invernizzi) Date: Thu, 27 Feb 2003 13:00:12 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <20030227104435.GA30494@magma.unil.ch> Message-ID: > -----Original Message----- > From: Armin Rigo > *All* types are expected to be computed during type inference. > So don't even say that W_ListObject.objarray is a list of wrapped > objects or Nones -- this can be figured out automatically. > Consider for example the case of *two* different implementations > (that would both appear as having the 'int' type to users). > Say that one is like Christian's W_IntObject+r_int, and the other > one can only encode small, tagged integers. > The choice to use one or both representations in an actual C > implementation must be made by the RPython-to-C translator, and > not in the object space. > The original intent of classes like W_IntObject was "one class, one > implementation", and I think that we must stick to that idea because these > classes are what are used for the multiple dispatch routines. Can you point out some idea about the *work* of the RPython-to-C translator? I can only imagine what I pointed out in the last email... - The RPython-to-C translator check the source code of the application for *static* *compile-time* optimization. - The RPython-to-C translator run the application under some objectspace. If I've understood well the idea, the application is the pypy-interpreter PLUS the application... - It performs introspection and optimize changing on-the-fly things. Some W_XXXObject is replaced with other, probably more *restricted or suited to the target the specific code emission*, ex the W_IntObject+r_int. Donno if this phase is to be done, I guess is an *additional* layer but remembering the discussion about the stack dept calculation, this can be the way to tell if all in the application is ok and aid the compiler... - loop analysis on even-more-restricted source - Finally emit C code for that specialized-parts, for example code that mimics current CPython functions for pypy-part of the interpreter, or code that recall the specialization done by psyco. Paolo Invernizzi From arigo at tunes.org Thu Feb 27 14:10:55 2003 From: arigo at tunes.org (Armin Rigo) Date: Thu, 27 Feb 2003 14:10:55 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: References: <20030227104435.GA30494@magma.unil.ch> Message-ID: <20030227131055.GA32657@magma.unil.ch> Hello Paolo, On Thu, Feb 27, 2003 at 01:00:12PM +0100, Paolo Invernizzi wrote: > Can you point out some idea about the *work* of the RPython-to-C translator? Your description is reasonable, althought I can imagine that we will start with only the following objective in mind: being able to re-emit C code that looks like CPython. The code written in RPython ("R"estricted) is only the interpreter-level stuff (and not absolutely everything, e.g. multi-dispatch should probably be rewritten by hand in C depending on the low-level object representation). I was targetting only this part for the RPython-to-C translator. In other words, this would (at first) be a completely static transformation of the pypy source to C. This would be application-independent, giving something that can be easily redistributed as an alternative to CPython. Armin From paoloinvernizzi at dmsware.com Thu Feb 27 14:35:33 2003 From: paoloinvernizzi at dmsware.com (Paolo Invernizzi) Date: Thu, 27 Feb 2003 14:35:33 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <20030227131055.GA32657@magma.unil.ch> Message-ID: > -----Original Message----- > From: Armin Rigo > I was targetting only this part for the RPython-to-C translator. > In other words, this would (at first) be a completely static > transformation of the pypy source to C. > This would be application-independent, giving something that can > be easily redistributed as an alternative to CPython. This is the case when the "application" is the pypyinterpreter, but conceptually, if the RPython-to-C translator works, it can translate every "application" written in RPython, or I'm missing something? Last question, I promise ;) I have not understood if *completely static* means that the code of pypy is analized *without* running pypy and the C sources are emitted. Or is it necessary for the translator to *execute* it for collecting information? Thanks a lot Armin for the kind responses! (and forget my bad english!) Paolo From arigo at tunes.org Thu Feb 27 14:42:59 2003 From: arigo at tunes.org (Armin Rigo) Date: Thu, 27 Feb 2003 14:42:59 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: References: <20030227131055.GA32657@magma.unil.ch> Message-ID: <20030227134259.GC32657@magma.unil.ch> Hello Paolo, On Thu, Feb 27, 2003 at 02:35:33PM +0100, Paolo Invernizzi wrote: > This is the case when the "application" is the pypyinterpreter, but > conceptually, if the RPython-to-C translator works, it can translate every > "application" written in RPython, or I'm missing something? Yes, that's right. > I have not understood if *completely static* means that the code of pypy is > analized *without* running pypy and the C sources are emitted. Or is it > necessary for the translator to *execute* it for collecting information? It depends on your point of view. I would say that from some point of view it is just a static external transformation. From some other point of view, this analysis will certainly be implemented by actually *running* the pypy interpreter with a special object space, whose purpose is to collect a trace of everything going on and every variable's types, and at the end write C code. A bient?t, Armin. From arigo at tunes.org Thu Feb 27 14:52:36 2003 From: arigo at tunes.org (Armin Rigo) Date: Thu, 27 Feb 2003 14:52:36 +0100 Subject: [pypy-dev] Re: stdobjspace status In-Reply-To: <2mu1ese9rw.fsf@starship.python.net> References: <20030224164413.51465501B@bespin.org> <2mu1ese9rw.fsf@starship.python.net> Message-ID: <20030227135236.GD32657@magma.unil.ch> Hello Michael, On Tue, Feb 25, 2003 at 02:04:51PM +0000, Michael Hudson wrote: > (...END_FINALLY...) > > Is this the problem that we sometimes push interpreter level objects > onto the applications's stack? I'm not sure we have much freedom here > unless we fiddle the compiled bytecodes. But I'm still a little fuzzy > on the issues. I think I came up with a reasonable solution. If you read the Python docs on sys.exc_info() (http://www.python.org/doc/current/lib/module-sys.html#l2h-246) you see a description that makes natural the introduction of a new stack, the "exception stack", of which sys.exc_info() would return the top-most item. Then I think that we can use this execution-context-global (not frame-dependent) stack to store the current exceptions. The END_FINALLY opcode can work by inspecting this stack to know if there is an exception to restore or not. It is also cleaner than both CPython, which abuses the valuestack to store the exception stuff, and the current pypy, were we abuse the blockstack for this purpose. A bient?t, Armin. From arigo at tunes.org Thu Feb 27 15:01:58 2003 From: arigo at tunes.org (Armin Rigo) Date: Thu, 27 Feb 2003 15:01:58 +0100 Subject: [pypy-dev] Test suite for basic types Message-ID: <20030227140158.GE32657@magma.unil.ch> Hello everybody, I found in the standard Python lib a test, namely test_builtin, which tests all built-in functions. Is anybody aware of a similar test for the basic built-in types, testing ints, longs, lists, dicts, tuples, strings...? And every opcode? If not, this would probably be a good subproject. Besides, it is something that could be contributed back to CPython. It could also be turned into a much more accurate benchmark than that horrible PyStone ! A bient?t, Armin. From arigo at tunes.org Thu Feb 27 15:14:08 2003 From: arigo at tunes.org (Armin Rigo) Date: Thu, 27 Feb 2003 15:14:08 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <20030227104435.GA30494@magma.unil.ch> References: <20030224164413.51465501B@bespin.org> <200302261009.h1QA9Jc23764@trixie.triqs.com> <3E5CDEA1.7080401@tismer.com> <200302261639.h1QGdoc26254@trixie.triqs.com> <3E5D0B74.7030709@tismer.com> <20030227104435.GA30494@magma.unil.ch> Message-ID: <20030227141408.GA1066@magma.unil.ch> Hello Christian, hello Stephan, Please, don't take my previous e-mail personally, I know that you have been working quite hard on the {int,float}objects. It was planned from the ground up that we would discover new issues and new ways of seeing them as the project kicks off. Also, it doesn't mean at all that anything you did will be trashed; it just means that (I feel) it is time to pause a bit and think about how, with the insight given by your code, we can build a coherent thing. At worst this will mean changing the interface to something more declarative than the StdObjSpace.xxx.register() calls -- if at all. A bient?t, Armin. From arigo at tunes.org Thu Feb 27 15:35:23 2003 From: arigo at tunes.org (Armin Rigo) Date: Thu, 27 Feb 2003 15:35:23 +0100 Subject: [pypy-dev] Release something? Message-ID: <20030227143523.GB1066@magma.unil.ch> Hello everybody, One more e-mail to remind you that one of the goal of the sprint was to release something at the end of the week. Did you realize that the interpreter (with the trivial object space) already works nicely enough to run the following: # python interactive.py Python 2.2.2 (#2, Nov 14 2002, 17:24:26) [GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.68mdk)] in pypy PyPyConsole / TrivialObjSpace >>> import dis >>> dis.dis(dis.dis) I particularly enjoy the slooooooowness of the resulting disassembly dump :-) Do you think, just for the fun, that this would be worth a "release"? It would show the "outside world" that we actually got something "serious" done. Of course "serious" needs quote, given the speed of the result, but at this point everybody should know that it was planned, and that we are dreaming about top speeds involving some rather obscure translations (probably invoking strange beasts from the theory of Computer Science). A bient?t, Armin. From stephan.diehl at gmx.net Thu Feb 27 15:42:03 2003 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Thu, 27 Feb 2003 15:42:03 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <20030227141408.GA1066@magma.unil.ch> References: <20030224164413.51465501B@bespin.org> <20030227104435.GA30494@magma.unil.ch> <20030227141408.GA1066@magma.unil.ch> Message-ID: <20030227144128.0798E5BA27@thoth.codespeak.net> Hi Armin, On Thursday 27 February 2003 15:14, you wrote: > Hello Christian, hello Stephan, > > Please, don't take my previous e-mail personally, I know that you have been > working quite hard on the {int,float}objects. Don't worry. This whole discussion show at least that nothing is set already and freah ideas are still welcome :-) > It was planned from the > ground up that we would discover new issues and new ways of seeing them as > the project kicks off. Also, it doesn't mean at all that anything you did > will be trashed; it just means that (I feel) it is time to pause a bit and > think about how, with the insight given by your code, we can build a > coherent thing. At the moment, nothing needs to be trashed at all, since the "hard-to-implement" types are not implemented yet. > At worst this will mean changing the interface to > something more declarative than the StdObjSpace.xxx.register() calls -- if > at all. That shouldn't be a problem as well (could be done by a good editor :-) Nevertheless, I still don't see that my way woundn't be a valid one. As far as I've understood the issues so far, the implementation details of the W_XXXObjects are just that: details. The interpreter just sees them as black boxes with behaviour. The only place where we need to be carefull is the instanciation (and that was not discussed yet). The point of compiling RPython into C is a complete independent discussion. Cheers Stephan > > > A bient?t, > > Armin. > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From stephan.diehl at gmx.net Thu Feb 27 15:59:22 2003 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Thu, 27 Feb 2003 15:59:22 +0100 Subject: [pypy-dev] Release something? In-Reply-To: <20030227143523.GB1066@magma.unil.ch> References: <20030227143523.GB1066@magma.unil.ch> Message-ID: <20030227145847.102AA5BA27@thoth.codespeak.net> On Thursday 27 February 2003 15:35, you wrote: > Hello everybody, > > One more e-mail to remind you that one of the goal of the sprint was to > release something at the end of the week. > > Did you realize that the interpreter (with the trivial object space) > already works nicely enough to run the following: No, actually not. This is brilliant. > > # python interactive.py > Python 2.2.2 (#2, Nov 14 2002, 17:24:26) > [GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.68mdk)] in pypy > PyPyConsole / TrivialObjSpace > > >>> import dis > >>> dis.dis(dis.dis) > > I particularly enjoy the slooooooowness of the resulting disassembly dump > :-) > > Do you think, just for the fun, that this would be worth a "release"? It > would show the "outside world" that we actually got something "serious" > done. Of course "serious" needs quote, given the speed of the result, but > at this point everybody should know that it was planned, and that we are > dreaming about top speeds involving some rather obscure translations > (probably invoking strange beasts from the theory of Computer Science). Well, at least it should be make known (but then, all the important people are reading this list anyway :-) Stephan > > > A bient?t, > > Armin. > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From hpk at trillke.net Thu Feb 27 16:16:43 2003 From: hpk at trillke.net (holger krekel) Date: Thu, 27 Feb 2003 16:16:43 +0100 Subject: [pypy-dev] Release something? In-Reply-To: <20030227143523.GB1066@magma.unil.ch>; from arigo@tunes.org on Thu, Feb 27, 2003 at 03:35:23PM +0100 References: <20030227143523.GB1066@magma.unil.ch> Message-ID: <20030227161643.E11666@prim.han.de> [Armin Rigo Thu, Feb 27, 2003 at 03:35:23PM +0100] > Hello everybody, > > One more e-mail to remind you that one of the goal of the sprint was to > release something at the end of the week. we released something: urls into our subversion tree :-) > Did you realize that the interpreter (with the trivial object space) already > works nicely enough to run the following: > > # python interactive.py > Python 2.2.2 (#2, Nov 14 2002, 17:24:26) > [GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.68mdk)] in pypy > PyPyConsole / TrivialObjSpace > >>> import dis > >>> dis.dis(dis.dis) > > I particularly enjoy the slooooooowness of the resulting disassembly dump :-) me, too :-) > Do you think, just for the fun, that this would be worth a "release"? It > would show the "outside world" that we actually got something "serious" > done. Of course "serious" needs quote, given the speed of the result, but at > this point everybody should know that it was planned, and that we are dreaming > about top speeds involving some rather obscure translations (probably invoking > strange beasts from the theory of Computer Science). before doing a release we should clean up any stale documentation in the wiki. Other than that a pypy-0.1 is feasible within a week's range IMO. Also the exception tracebacks are already pretty nice in that they separate interpreter and application level origins thus easing debugging. btw, I am working on nested scopes and a script that invokes all unit-tests. Might be nice for the release. Also can't we make the trivial's objspace "wrap" and "unwrap" actually do something like def wrap(self, value): return Box(value) def unwrap(self, w_value): return w_value.value with class Box: def __init__(self, value): self.value = value ? I just tried but got errors which i am investigating. They may indicate incorrect wrap/unwrap arithmetic. holger From mwh at python.net Thu Feb 27 16:19:15 2003 From: mwh at python.net (Michael Hudson) Date: Thu, 27 Feb 2003 15:19:15 +0000 Subject: [pypy-dev] Re: Test suite for basic types References: <20030227140158.GE32657@magma.unil.ch> Message-ID: <2md6ld7nv0.fsf@starship.python.net> Armin Rigo writes: > Hello everybody, > > I found in the standard Python lib a test, namely test_builtin, which tests > all built-in functions. Is anybody aware of a similar test for the basic > built-in types, testing ints, longs, lists, dicts, tuples, strings...? You're looking for test_types. > And every opcode? No idea. > If not, this would probably be a good subproject. Besides, it is something > that could be contributed back to CPython. Yes. > It could also be turned into a much more accurate benchmark than > that horrible PyStone ! Accurate in what sense? Cheers, M. -- Gullible editorial staff continues to post links to any and all articles that vaguely criticize Linux in any way. -- Reason #4 for quitting slashdot today, from http://www.cs.washington.edu/homes/klee/misc/slashdot.html From lac at strakt.com Thu Feb 27 16:28:47 2003 From: lac at strakt.com (Laura Creighton) Date: Thu, 27 Feb 2003 16:28:47 +0100 Subject: [pypy-dev] Release something? In-Reply-To: Message from Armin Rigo of "Thu, 27 Feb 2003 15:35:23 +0100." <20030227143523.GB1066@magma.unil.ch> References: <20030227143523.GB1066@magma.unil.ch> Message-ID: <200302271528.h1RFSljL029921@ratthing-b246.strakt.com> In a message of Thu, 27 Feb 2003 15:35:23 +0100, Armin Rigo writes: >Hello everybody, > >One more e-mail to remind you that one of the goal of the sprint was to >release something at the end of the week. > >Did you realize that the interpreter (with the trivial object space) already >works nicely enough to run the following: > ># python interactive.py >Python 2.2.2 (#2, Nov 14 2002, 17:24:26) >[GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.68mdk)] in pypy >PyPyConsole / TrivialObjSpace >>>> import dis >>>> dis.dis(dis.dis) > >I particularly enjoy the slooooooowness of the resulting disassembly dump > :-) > >Do you think, just for the fun, that this would be worth a "release"? It >would show the "outside world" that we actually got something "serious" >done. Of course "serious" needs quote, given the speed of the result, >but at this point everybody should know that it was planned, and that >we are dreaming about top speeds involving some rather obscure translations >(probably invoking strange beasts from the theory of Computer Science). > > >A bient?t, > >Armin. >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev I think that this is a great plan. Plus we didn't promise to be fast, only elegant -(well, simple and flexible). If we can get the idea of object spaces and the idea of multiple dispatches across, then that should be good enough for, say, the article(s) that Cameron Laird wants to write about us. It will make a nice talk for Python conferences as well. Laura From arigo at tunes.org Thu Feb 27 16:53:46 2003 From: arigo at tunes.org (Armin Rigo) Date: Thu, 27 Feb 2003 16:53:46 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <20030227144128.5EA76502E@bespin.org> References: <20030224164413.51465501B@bespin.org> <20030227104435.GA30494@magma.unil.ch> <20030227141408.GA1066@magma.unil.ch> <20030227144128.5EA76502E@bespin.org> Message-ID: <20030227155346.GC1066@magma.unil.ch> Hello Stephan, hello Laura, Thanks Laura for pointing out the potential language problems. Your point of view on Compliant Object Spaces is very helpful. It seems to show that we will want different object spaces that share quite a lot of source files, e.g. two object spaces which are the same except for their list implementation. Moreover, some of these implementations might be easier to translate into efficient C code and some others might be better suited for translation to some other language. What we may thus need is a way to define several object spaces from the same set of files. The std/ directory would then contain all files that implement a bit of the "compliant" behavior for some specific kind of objects; it could contain e.g. both the standard and Stephan's decimal implementation of floats. What is left to be determined is thus how the objects are instantiated, as Stephan pointed out. This currently takes place in the class ObjSpace, in the methods wrap(), newlist(), newtuple(), newdict(), etc. Then what we will need at some point is to have several different subclasses of baseobjspace.ObjSpace in several files, possibly all inside std/, which import and instanciate different W_XxxObject classes. To make an object space which is the same as another one expect for its implementation of floats, just override its wrap() method. Sounds cool! A bient?t, Armin. From arigo at tunes.org Thu Feb 27 16:56:12 2003 From: arigo at tunes.org (Armin Rigo) Date: Thu, 27 Feb 2003 16:56:12 +0100 Subject: [pypy-dev] Re: Test suite for basic types In-Reply-To: <2md6ld7nv0.fsf@starship.python.net> References: <20030227140158.GE32657@magma.unil.ch> <2md6ld7nv0.fsf@starship.python.net> Message-ID: <20030227155612.GD1066@magma.unil.ch> Hello Michael, On Thu, Feb 27, 2003 at 03:19:15PM +0000, Michael Hudson wrote: > You're looking for test_types. Thanks! > > It could also be turned into a much more accurate benchmark than > > that horrible PyStone ! > > Accurate in what sense? More representative of real Python code. If we do n list operations, m integer operations, p dict operations, q class operations, for suitable values of n, m, p and q, then it would probably be much more representative than this Pascal-like PyStone which just never tests most of Python's very common types and built-in functions (like 'len'). A bient?t, Armin. From mwh at python.net Thu Feb 27 16:59:48 2003 From: mwh at python.net (Michael Hudson) Date: Thu, 27 Feb 2003 15:59:48 +0000 Subject: [pypy-dev] Re: Release something? References: <20030227143523.GB1066@magma.unil.ch> <20030227161643.E11666@prim.han.de> Message-ID: <2m7kbl7lzf.fsf@starship.python.net> holger krekel writes: > Also can't we make the trivial's objspace "wrap" and "unwrap" > actually do something like > > def wrap(self, value): > return Box(value) > def unwrap(self, w_value): > return w_value.value > > with > > class Box: > def __init__(self, value): > self.value = value > > ? I think that should be "slightlylesstrivialobjspace.py" or something. > I just tried but got errors which i am investigating. > They may indicate incorrect wrap/unwrap arithmetic. Quite likely there's some of that. You also have to thoroughly rewrite trivial.py. Cheers, M. -- Every day I send overnight packages filled with rabid weasels to people who use frames for no good reason. -- The Usenet Oracle, Oracularity #1017-1 From mwh at python.net Thu Feb 27 17:05:26 2003 From: mwh at python.net (Michael Hudson) Date: Thu, 27 Feb 2003 16:05:26 +0000 Subject: [pypy-dev] Re: Test suite for basic types References: <20030227140158.GE32657@magma.unil.ch> <2md6ld7nv0.fsf@starship.python.net> <20030227155612.GD1066@magma.unil.ch> Message-ID: <2m1y1t7lq1.fsf@starship.python.net> Armin Rigo writes: > Hello Michael, > > On Thu, Feb 27, 2003 at 03:19:15PM +0000, Michael Hudson wrote: >> You're looking for test_types. > > Thanks! No problem. >> > It could also be turned into a much more accurate benchmark than >> > that horrible PyStone ! >> >> Accurate in what sense? > > More representative of real Python code. I don't think test_types is very representative either... > If we do n list operations, m integer operations, p dict operations, > q class operations, for suitable values of n, m, p and q, then it > would probably be much more representative than this Pascal-like > PyStone which just never tests most of Python's very common types > and built-in functions (like 'len'). You know about mal's pybench? It's more quantative that pystone, at least. Cheers, M. -- You can remove divmod() when I'm dead. Before then, it stays. I'm sure all will agree that's a reasonable compromise. -- Tim Peters negotiating on python-dev From mwh at python.net Thu Feb 27 17:01:38 2003 From: mwh at python.net (Michael Hudson) Date: Thu, 27 Feb 2003 16:01:38 +0000 Subject: [pypy-dev] Re: stdobjspace status References: <20030224164413.51465501B@bespin.org> <2mu1ese9rw.fsf@starship.python.net> <20030227135236.GD32657@magma.unil.ch> Message-ID: <2m4r6p7lwd.fsf@starship.python.net> Armin Rigo writes: > Hello Michael, > > On Tue, Feb 25, 2003 at 02:04:51PM +0000, Michael Hudson wrote: >> (...END_FINALLY...) >> >> Is this the problem that we sometimes push interpreter level objects >> onto the applications's stack? I'm not sure we have much freedom here >> unless we fiddle the compiled bytecodes. But I'm still a little fuzzy >> on the issues. > > I think I came up with a reasonable solution. If you read the Python docs on > sys.exc_info() (http://www.python.org/doc/current/lib/module-sys.html#l2h-246) > you see a description that makes natural the introduction of a new stack, the > "exception stack", of which sys.exc_info() would return the top-most item. > Then I think that we can use this execution-context-global (not > frame-dependent) stack to store the current exceptions. The END_FINALLY > opcode can work by inspecting this stack to know if there is an exception to > restore or not. Sounds reasonable. How does this interact with 3-arg raise? I'm feeling a bit dumb today. Cheers, M. -- > I wouldn't want to live without readline, but some of the > things it does call for the application of thumbscrews. -- me on python-dev From stephan.diehl at gmx.net Thu Feb 27 17:23:44 2003 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Thu, 27 Feb 2003 17:23:44 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <20030227155346.GC1066@magma.unil.ch> References: <20030224164413.51465501B@bespin.org> <20030227144128.5EA76502E@bespin.org> <20030227155346.GC1066@magma.unil.ch> Message-ID: <20030227162308.4BC225BA27@thoth.codespeak.net> On Thursday 27 February 2003 16:53, you wrote: > Hello Stephan, hello Laura, > > Thanks Laura for pointing out the potential language problems. Is there a possibility that Lauras mail didn't end up on the list? I couldn't find it. > > Your point of view on Compliant Object Spaces is very helpful. It seems to > show that we will want different object spaces that share quite a lot of > source files, e.g. two object spaces which are the same except for their > list implementation. Moreover, some of these implementations might be > easier to translate into efficient C code and some others might be better > suited for translation to some other language. > > What we may thus need is a way to define several object spaces from the > same set of files. The std/ directory would then contain all files that > implement a bit of the "compliant" behavior for some specific kind of > objects; it could contain e.g. both the standard and Stephan's decimal > implementation of floats. What is left to be determined is thus how the > objects are instantiated, as Stephan pointed out. This currently takes > place in the class ObjSpace, in the methods wrap(), newlist(), newtuple(), > newdict(), etc. Then what we will need at some point is to have several > different subclasses of baseobjspace.ObjSpace in several files, possibly > all inside std/, which import and instanciate different W_XxxObject > classes. To make an object space which is the same as another one expect > for its implementation of floats, just override its wrap() method. This basicly sounds like what I had in mind (but couldn't express very well :-) About object instantiation: Where excatly is the textual representation of a python object turned into an object? Is it done by the parser? (for example "12345L" --> 12345L) Stephan > > Sounds cool! > > > A bient?t, > > Armin. > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From hpk at trillke.net Thu Feb 27 17:33:23 2003 From: hpk at trillke.net (holger krekel) Date: Thu, 27 Feb 2003 17:33:23 +0100 Subject: [pypy-dev] Re: Release something? In-Reply-To: <2m7kbl7lzf.fsf@starship.python.net>; from mwh@python.net on Thu, Feb 27, 2003 at 03:59:48PM +0000 References: <20030227143523.GB1066@magma.unil.ch> <20030227161643.E11666@prim.han.de> <2m7kbl7lzf.fsf@starship.python.net> Message-ID: <20030227173323.F11666@prim.han.de> [Michael Hudson Thu, Feb 27, 2003 at 03:59:48PM +0000] > holger krekel writes: > > > Also can't we make the trivial's objspace "wrap" and "unwrap" > > actually do something like > > > > def wrap(self, value): > > return Box(value) > > def unwrap(self, w_value): > > return w_value.value > > > > with > > > > class Box: > > def __init__(self, value): > > self.value = value > > > > ? > > I think that should be "slightlylesstrivialobjspace.py" or something. > > > I just tried but got errors which i am investigating. > > They may indicate incorrect wrap/unwrap arithmetic. > > Quite likely there's some of that. You also have to thoroughly > rewrite trivial.py. jep, just fiddled with it. seems more involved than expected. so the trivial space remains trivial for the time being. holger From lac at strakt.com Thu Feb 27 17:18:56 2003 From: lac at strakt.com (Laura Creighton) Date: Thu, 27 Feb 2003 17:18:56 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: Message from Stephan Diehl of "Thu, 27 Feb 2003 17:23:44 +0100." <20030227162308.4BC225BA27@thoth.codespeak.net> References: <20030224164413.51465501B@bespin.org> <20030227144128.5EA76502E@bespin.org> <20030227155346.GC1066@magma.unil.ch> <20030227162308.4BC225BA27@thoth.codespeak.net> Message-ID: <200302271618.h1RGIvjL030244@ratthing-b246.strakt.com> In a message of Thu, 27 Feb 2003 17:23:44 +0100, Stephan Diehl writes: >On Thursday 27 February 2003 16:53, you wrote: >> Hello Stephan, hello Laura, >> >> Thanks Laura for pointing out the potential language problems. > >Is there a possibility that Lauras mail didn't end up on the list? I coul >dn't >find it. Yes. I wrote this, but only sent it to Armin because I thought it was too long and maybe it was wrong too. It is Still too long. Apologies. Laura Is it possible that this misunderstanding is all about naming? I am getting the impression that Stephan and Christian are looking for some basic fundamental types, the foundation of the system, to build things out of. And Armin is saying that there aren't any types, just behaviours of object spaces. This is strongly reminding me of what happened to the physicists when they started looking for 'the elementary building blocks of matter'. Once you get to sub-atomic particles, you end up with a mess to think about. When Dirac discovered the positron, and anti-matter, suddenly you could no longer think of things as being made out of smaller things any more. A hydrogen atom is usually a proton and an electron. But it might temporarily be 1 proton, 2 electrons, and one positron. This made all the physicists go -eep-. What's Hydrogen? Something that behaves like Hydrogen, it seems :-). Time passes, and quantum dynamics is created/discovered. But the deep problem remains. Werner Heisenberg calls this the 'problem of physical content vs mathematical form'. He said that the problem is embedded in the language we use to think. Maybe we have a language problem too. We shouldn't have called it the StdObjSpace. Instead we should have called it the C_like_familiar_ObjSpace. It represents one way to implement an Object Space, and it has the desirable property that it is like the one we know. But we could eventually make as many of these as we like (though only a few of them are a good idea now, or we will confuse everybody even more than we are confusing ourselves now.) But I am now thiking of an ObjectSpace as, not a thing, but a collection of behaviours. And you can specify the behaviours. In particular, we can specify what behaviour is required in an Compliant Object Space, where Compliant is now defined as 'runs C python code and gets the same result as C python'. All of the spaces that do this will be equal to each other in some sense, though some will be more useful, and fun than others. We could make 2 that are exactly the same except for, for instance, the list implementation, which is one way to test that we have our ideas ok. Or one that does integers the way that we are used to, and one that does them this new way that Stephan proposes. Then, we can call the set of all Object Spaces that are Compliant, the StdObjSpaces.( <-- see the s) . But the idea is not to make too many of them, because what we need is a proof of concept that our idea of breaking out the object spaces into their own idea, and dispatching on that works, not the best StdObjSpace implementation. The best one will be determined by things like 'can Armin and Christian make it go _really fast_, which is something we aren't supposed to be thinking about now. (Though its hard not to sometimes. :-) ) (We can then make non-compliant objectspaces if we like. I mustn't think about that either :-). Thus what we need are experimental specifications, in the form of unit tests, to measure whether a given objectspace is 'standard' or not -- in other words, does it comply with C Python. We aren't looking for better bits to build our space out of, we are looking for better ways to know if our space produces the correct behaviours. Laura From stephan.diehl at gmx.net Thu Feb 27 18:00:47 2003 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Thu, 27 Feb 2003 18:00:47 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <200302271618.h1RGIvjL030244@ratthing-b246.strakt.com> References: <20030224164413.51465501B@bespin.org> <20030227162308.4BC225BA27@thoth.codespeak.net> <200302271618.h1RGIvjL030244@ratthing-b246.strakt.com> Message-ID: <20030227170011.A4AD55BA27@thoth.codespeak.net> > > Yes. I wrote this, but only sent it to Armin because I thought it was > too long and maybe it was wrong too. It is Still too long. Apologies. It's neither too long nor wrong. Anyway, if anybody would write only something if it's supposed to be the absolute thruth, we might run into some severe problems :-) The exciting thing about this whole project is the fact that it's not set yet and things can be discussed. > > Laura > > Is it possible that this misunderstanding is all about naming? I am > getting the impression that Stephan and Christian are looking for some > basic fundamental types, the foundation of the system, to build things > out of. And Armin is saying that there aren't any types, just > behaviours of object spaces. Yes, that's it. [...] > > Maybe we have a language problem too. We shouldn't have called it the > StdObjSpace. Instead we should have called it the > C_like_familiar_ObjSpace. After all, StdObjSpace seems fine since it should contain something that behaves like standard python objects. > Thus what we need are experimental specifications, in the form of > unit tests, to measure whether a given objectspace is 'standard' or > not -- in other words, does it comply with C Python. We aren't looking > for better bits to build our space out of, we are looking for better > ways to know if our space produces the correct behaviours. Definatelly. At least the unittests for intobject and floatobject are doing exactly this (but then this is easy, because both xxxobjects are just wrappers of Pythons originals). We probably have to resort to string representations of the results and compare them with Python originals. > > Laura Stephan From arigo at tunes.org Thu Feb 27 18:01:02 2003 From: arigo at tunes.org (Armin Rigo) Date: Thu, 27 Feb 2003 18:01:02 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <20030227162309.7C8CC5176@bespin.org> References: <20030224164413.51465501B@bespin.org> <20030227144128.5EA76502E@bespin.org> <20030227155346.GC1066@magma.unil.ch> <20030227162309.7C8CC5176@bespin.org> Message-ID: <20030227170102.GB2872@magma.unil.ch> Hello Stephan, On Thu, Feb 27, 2003 at 05:23:44PM +0100, Stephan Diehl wrote: > About object instantiation: > Where excatly is the textual representation of a python object turned into an > object? Is it done by the parser? (for example "12345L" --> 12345L) Damn. I don't know. If we don't include any 'long' type among our allowed RPython types (and we probably don't), then we will have to rethink the way code objects are currently completely put into interpreter-space level. This cannot work if this code object contains constants of unknown types! So it seems that even code objects must be object-space dependent. They would be compiled by the application-level 'compiler' package inside of a given object space, and not be 'extractible' from there. The interpreter would only unwrap the bytecode string, but not the tuple of constants. If on the other hand we would prefer completely portable code objects, then they must contain "portable constant literals", including longs, and wrap() must support at least these constants. This could be nice, too, because we could then have object-space-independent bytecode repositories (like .pyc files). But then we must include in RPython the notion of longs -- at least their existence, not necessarily any specific operation on them. The same about complex numbers. Tough choice. Armin From stephan.diehl at gmx.net Thu Feb 27 18:42:00 2003 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Thu, 27 Feb 2003 18:42:00 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <20030227170102.GB2872@magma.unil.ch> References: <20030224164413.51465501B@bespin.org> <20030227162309.7C8CC5176@bespin.org> <20030227170102.GB2872@magma.unil.ch> Message-ID: <20030227174124.E34885B34B@thoth.codespeak.net> Hallo Armin, On Thursday 27 February 2003 18:01, you wrote: > Hello Stephan, > > On Thu, Feb 27, 2003 at 05:23:44PM +0100, Stephan Diehl wrote: > > About object instantiation: > > Where excatly is the textual representation of a python object turned > > into an object? Is it done by the parser? (for example "12345L" --> > > 12345L) > > Damn. I don't know. > If we don't include any 'long' type among our allowed > RPython types (and we probably don't), then we will have to rethink the way > code objects are currently completely put into interpreter-space level. > This cannot work if this code object contains constants of unknown types! Maybe, then RPython was not a very good idea in the first place. > > So it seems that even code objects must be object-space dependent. They > would be compiled by the application-level 'compiler' package inside of a > given object space, and not be 'extractible' from there. The interpreter > would only unwrap the bytecode string, but not the tuple of constants. > > If on the other hand we would prefer completely portable code objects, then > they must contain "portable constant literals", including longs, and wrap() > must support at least these constants. This could be nice, too, because we > could then have object-space-independent bytecode repositories (like .pyc > files). But then we must include in RPython the notion of longs -- at > least their existence, not necessarily any specific operation on them. The > same about complex numbers. Would it help if the bytecode just holds the string representation of an object? I guess the lexer gives down the string plus type information. maybe somthing like ('long','12345L'). If this were taken as it is into the bytecode, the interpreter then could run the instantiation on the fly. But then, the question is, if the ObjSpaces we are talking about could live in Application Space? The interpreter just requires some Object types to be present in order to function. In order to be used as an internal type, the XXXObjects just have to comply to some interface and know how to create oneselfs from a string (and give the compiler some regexp so it can be known). (Hmm, this is probably a little bit too off target) Anyway, this stuff must not be decided now. > > Tough choice. > > > Armin Cheers Stephan > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From tismer at tismer.com Thu Feb 27 20:05:12 2003 From: tismer at tismer.com (Christian Tismer) Date: Thu, 27 Feb 2003 20:05:12 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <20030227104435.GA30494@magma.unil.ch> References: <20030224164413.51465501B@bespin.org> <200302261009.h1QA9Jc23764@trixie.triqs.com> <3E5CDEA1.7080401@tismer.com> <200302261639.h1QGdoc26254@trixie.triqs.com> <3E5D0B74.7030709@tismer.com> <20030227104435.GA30494@magma.unil.ch> Message-ID: <3E5E6168.1080005@tismer.com> Armin Rigo wrote: ... > The W_IntObject class itself is supposed to be one possible implementation of > Python's integers, described in a high-level point of view. The whole point > is in that word "high-level". We (or at least I) want the objspace/std/* > files to be translatable to whatever actual low-level language that we like, > and not only C -- this is why the bitfield metaphor is not necessarily a good > thing. We probably don't need bitfields now. > When you are talking about r_int and similarly r_float or even r_long > wrappers, you want to make it clear that you mean a "machine-level" plain > object. The is no r_float, since floats are just machine-level. There is also no r_long, since there are no longs at all. > But actually that was the whole point of RPython. The C translation > of all the interpreter-level code will produce machine-level "int"s. In other > words there is no deep reason to use a special wrapping mecanism for ints, > floats, lists, whatever, to implement the W_XxxObject classes, because we will > need *exactly the same* low-level objects to implement just *everything else* > in the interpreter! For example, when the interpreter manipulates the current > bytecode position, it manipulates what looks like Python integers and strings > --- > > opcode = ord(self.bytecode[nextposition]) > nextposition += 1 > > --- but we want this code to be translated into efficient C code, too. The > least thing we want is to have to rewrite *all* this code with r_ints! Right, and I didn't hear me saying so. I just need the r_ints to make the emulation behave correctly, when we are running it in CPython! > So the *only* point of r_ints, as far as I can tell, is to have explicit > control over bit width and overflow detection. There is no built-in Python > type "integer with n bits" that generates the correct OverflowErrors. And > there is no unsigned integer type, hence the need for r_uint. But apart from > that, just use plain integers and it is fine! That is the whole purpose of > writing our pypy in Python, isn't it? Creating r_long, r_float, r_list... > looks like we are reinventing our own language. Forget about r_long and r_float, I can't remeber that we talked about that, at least I didn't think of it. r_list is something where I'm not sure, because I don't believe that we can create all code by pure magic. Somebody has to write an implementation of the future list implementation (which has nothing to do with r_list but to use it as a building block). But I believe it is bad to use regular lists and use things like append(), or list1 + list2. This is not primitive and neither available in the C or Assembly targets, so why may we assume they exist? Only for that reason, I wanted to create a subtype of lists, which is just not capable to do certain things. > The confusion that may have led to this point is that we never explicitely > said what objects we would allow in RPython. Remember, RPython is supposed to > be Python with a few restrictions whose goal is *only* to ease the job of type > inference and later translation. But the *existing* high-level constructs of > Python must be kept! If we don't use them, we are back again at the C level > and there is no point in writing pypy in Python. Then you should become clearer about what exactly we want to use from Python. ... >>So, W_ListObject >>would have some fields like >> self.objarray # r_objectarray >> self.maxlen # r_uint >> self.actlen # r_uint >> >>It has been suggested to not use maxlen, since >>we could use len(self.objarray), but I believe >>this is wrong to do. If we are modelling primitive >>arrays, then they don't support len() at all. > > > I feel exactly the opposite. Just *use* real Python lists all the way, > including most of their normal operations. This is nice because they describe > high-level operations. In my view a W_ListObject would have only two fields: > > * self.objarray # a real Python list of wrapped items > * self.length # the actual list length > > with self.length <= len(self.objarray). The self.objarray list grows by more > than one item at a time, to amortize the cost of adding a single element at a > time, just like what the current CPython implementation does. Sounds inconsequent to me. On the one hand you use the length attribute, so the thing cannot be translated into a pointer to a plain memory area, but it is more. On the other hand, you repeat the C implementation of piecewise allocation. Where do you set the fence? > All this works > fine, is normal pure Python code, and can be expected to translate to an > efficient C implementation in which lists are just translated into malloc'ed > blocks of PyObject* pointers. The point is that we still have a nice Pythonic > high-level implementation of W_ListObject but nevertheless can translate it > into an efficient low-level implementation. Being explicit about actual length, but implicit about maximum length is a bit arbitrary for me. I thought to describe a piece of memory by using a list. It is a piece of memory, that malloc returns. So I thought I need to model length as well. If I don't model length, but rely on a code generator to figure this out, well, then I have the problem that this code generator does not exist yet, but I want to be able to generate code, as a proof of concept. For that reason, I liked an implementation that is so basic, that even I could write a code generator, without the need for any magic that I don't understand. Kinda bottom-up way, maybe this is the wrong way, but I need to be convinced. At the moment, I feel completely blocked. > Another point I would like to raise is that we don't have to give types to > every variable in pypy, like enforcing that a slot contains an r_int. This is > just unPythonic. *All* types are expected to be computed during type > inference. So don't even say that W_ListObject.objarray is a list of wrapped > objects or Nones -- this can be figured out automatically. You make the one false assumption that all the code is correct. The reason why I'd like to restrict the objarray, or especially the array used as the stack in frames, is better debugging. I was just told about the fact that we currently sometimes have "wrong" objects on the stack. Exactly that would be found easily, if I provide a modified array that checks for exactly that. I can't find how debugging is unpythonic. > ..from Paolo's reply: > > >>In other words... is not the *W_IntObject with r_int* one of the *possible* >>choice that pypy can choose for representing (to target code emission!) a >>plain python int (c)? >>Instead of r_int, for example, I can choose to use a tagged-rapresentation >>of an int, and write a t_int that check the overflow and so problems (and >>where to check arithmetics?)...(d). This is a misunderstanding. I wasn't about using integer objects at all, the r_int emulated what will be a machine integer, later. This isn't about tagging. I don't thing to use integer /objects/ for implementing lists, for instance. I surely mean all integer objects to be turned into machine words. Again, this is probably since I thought that in /std/ we do what's needed for generatong C code. > This is right. Consider for example the case of *two* different > implementations (that would both appear as having the 'int' type to users). > Say that one is like Christian's W_IntObject+r_int, and the other one can only > encode small, tagged integers. The choice to use one or both representations > in an actual C implementation must be made by the RPython-to-C translator, and > not in the object space. For example, if we want to produce something very > similar to the current CPython, then we have no use for small, tagged > integers. The question is thus, "how do we express things to allow for this?" I had the impression that the "std" objects space is meant to be the one to be translated into C. Therefore I was trying to write directly translatable code. But I see this goal disappearing... What I wanted to do is a 'real' implementation of basic types, whith 'real' algorithms, taken from the C sources, with some simplifications and clarifications by using Python, but not by using Python objects. If this is not the place to put this, then I'd like to ask for the place for that. > Similarily, we may provide different implementations for lists, dictionaries, > whatever; we may even consider that Python's "long" type is an unneeded hack, > for long integers could be just another implementation for the 'int' type, > which goes very much in the direction that Python seems to go with the recent > automatic conversions of overflows to longs. I agree. So are the implementations like W_IntObject implementations, or just interface definitions? ... > In an effort to go in that direction I'd like to add that nothing has been > done yet about: > > * built-in methods (like list.append); the StdObjSpace.xxx.register() trick > only works for built-in operators > * non-built-in operators and methods, e.g. > implementing something like long_long_add in application-space > (longobject_app.py). Yes. This doesn't happen, since we have no coherent design how this should be done. - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at tismer.com Thu Feb 27 20:22:31 2003 From: tismer at tismer.com (Christian Tismer) Date: Thu, 27 Feb 2003 20:22:31 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <20030227170102.GB2872@magma.unil.ch> References: <20030224164413.51465501B@bespin.org> <20030227144128.5EA76502E@bespin.org> <20030227155346.GC1066@magma.unil.ch> <20030227162309.7C8CC5176@bespin.org> <20030227170102.GB2872@magma.unil.ch> Message-ID: <3E5E6577.2020109@tismer.com> Hi Armin, > On Thu, Feb 27, 2003 at 05:23:44PM +0100, Stephan Diehl wrote: > >>About object instantiation: >>Where excatly is the textual representation of a python object turned into an >>object? Is it done by the parser? (for example "12345L" --> 12345L) > > > Damn. I don't know. If we don't include any 'long' type among our allowed > RPython types (and we probably don't), then we will have to rethink the way > code objects are currently completely put into interpreter-space level. This > cannot work if this code object contains constants of unknown types! Maybe I'm ignorant -- didn't read all of the implementation, yet, but why is this a problem? I don't think that RPython has anything to do with Python constants and Python code objects, and of course I think that creating code objects should happen in user space. Then, the problem vanishes. > So it seems that even code objects must be object-space dependent. They would > be compiled by the application-level 'compiler' package inside of a given > object space, and not be 'extractible' from there. The interpreter would only > unwrap the bytecode string, but not the tuple of constants. I can't follow. Maybe the physical layout of structures is different, depending of the object space, but from the Python view, they are all the same. If you marshal them, they are the same. Now think of marshalled code objects, which you unmarshal from the application-level. All the marshalled contants will of course be turned into objects in the current application space, however that is implemented. Do we really use code objects from the "C" level Python? This was a temporary hack, as I understood. Of course cou can use them as a template, but I'm assuming that we would transform them for the "upper" world, before executing. > If on the other hand we would prefer completely portable code objects, then > they must contain "portable constant literals", including longs, and wrap() > must support at least these constants. This could be nice, too, because we > could then have object-space-independent bytecode repositories (like .pyc > files). But then we must include in RPython the notion of longs -- at least > their existence, not necessarily any specific operation on them. The same > about complex numbers. I still see no reason why a long needs to exist in RPython at all. Please, give me a hint :-) ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From hpk at trillke.net Thu Feb 27 20:50:08 2003 From: hpk at trillke.net (holger krekel) Date: Thu, 27 Feb 2003 20:50:08 +0100 Subject: [pypy-dev] compiling "constants" (was: Base Object library (was: ...)) In-Reply-To: <20030227170102.GB2872@magma.unil.ch>; from arigo@tunes.org on Thu, Feb 27, 2003 at 06:01:02PM +0100 References: <20030224164413.51465501B@bespin.org> <20030227144128.5EA76502E@bespin.org> <20030227155346.GC1066@magma.unil.ch> <20030227162309.7C8CC5176@bespin.org> <20030227170102.GB2872@magma.unil.ch> Message-ID: <20030227205008.H11666@prim.han.de> [stephan] > About object instantiation: > Where excatly is the textual representation of a python object turned into an > object? Is it done by the parser? (for example "12345L" --> 12345L) It's the compiler and not the parser that turns "12345L" into a Python object. It executes "eval" on atom_number and atom_string symbols (see compiler/transformer.py in the standard lib). [armin] > Damn. I don't know. If we don't include any 'long' type among our allowed > RPython types (and we probably don't), then we will have to rethink the way > code objects are currently completely put into interpreter-space level. This > cannot work if this code object contains constants of unknown types! In CPython, the compiler package just uses the C-level "eval" to implicitely get ready-made "constant" Python-objects. Thus it avoids the problem of computing the constants by effectively handing it to its C-equivalent "compile.c" . If we want to have a pure python compiler then we need to break the cycle (computing a constant requires computing a constant). So for PyPy, the compiler package cannot use eval. It should make a try-except dance trying to apply "int", "float", "long" to atom_numbers and "str", "unicode" to atom_strings. These are builtin types we have to implement anyway and they will dispatch to the appropriate objectspace internally. Thus the compiler could stay clean of infinite recursion and provide already wrapped "constant" objects. So all in all, the compiler package is the right place to fix the "interpreter doesn't want to know types of constants" problem. Maybe the above sketched fix might even find its way into Python-2.3 as it also speeds up and un-tricks the compiling process :-) greetings, holger From tismer at tismer.com Thu Feb 27 20:55:43 2003 From: tismer at tismer.com (Christian Tismer) Date: Thu, 27 Feb 2003 20:55:43 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <200302271618.h1RGIvjL030244@ratthing-b246.strakt.com> References: <20030224164413.51465501B@bespin.org> <20030227144128.5EA76502E@bespin.org> <20030227155346.GC1066@magma.unil.ch> <20030227162308.4BC225BA27@thoth.codespeak.net> <200302271618.h1RGIvjL030244@ratthing-b246.strakt.com> Message-ID: <3E5E6D3F.1030505@tismer.com> Laura Creighton wrote: [snipped almost all, totally agreed] > The best one > will be determined by things like 'can Armin and Christian make it go > _really fast_, which is something we aren't supposed to be thinking > about now. (Though its hard not to sometimes. :-) ) My problem is simply that we are too abstract for my taste, yet. I'd like to build abstraction with concrete implementation at the same time, because I need a proof of concept, early. What I need is one long line from the very upper levels, which I like very much, down to the primitive code that *really* implements it in some way not les efficient that CPython. I need t get my "environment definitions", to be able to work on that ground level. But I agree that we shouldn't stick with only repeating CPython, so there is a need for different abstraction levels, which are maybe missing in the current picture. cheers - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From hpk at trillke.net Thu Feb 27 21:28:53 2003 From: hpk at trillke.net (holger krekel) Date: Thu, 27 Feb 2003 21:28:53 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <3E5E6577.2020109@tismer.com>; from tismer@tismer.com on Thu, Feb 27, 2003 at 08:22:31PM +0100 References: <20030224164413.51465501B@bespin.org> <20030227144128.5EA76502E@bespin.org> <20030227155346.GC1066@magma.unil.ch> <20030227162309.7C8CC5176@bespin.org> <20030227170102.GB2872@magma.unil.ch> <3E5E6577.2020109@tismer.com> Message-ID: <20030227212853.J11666@prim.han.de> [Christian Tismer Thu, Feb 27, 2003 at 08:22:31PM +0100] > Hi Armin, > > > On Thu, Feb 27, 2003 at 05:23:44PM +0100, Stephan Diehl wrote: > > > >>About object instantiation: > >>Where excatly is the textual representation of a python object turned into an > >>object? Is it done by the parser? (for example "12345L" --> 12345L) > > > > > > Damn. I don't know. If we don't include any 'long' type among our allowed > > RPython types (and we probably don't), then we will have to rethink the way > > code objects are currently completely put into interpreter-space level. This > > cannot work if this code object contains constants of unknown types! > > Maybe I'm ignorant -- didn't read all of the > implementation, yet, but why is this a problem? > I don't think that RPython has anything to do > with Python constants and Python code objects, > and of course I think that creating code objects > should happen in user space. Then, the problem vanishes. Agreed. Creating code objects at application-level needs some tweaks, though (see my other post, same thread). > > So it seems that even code objects must be object-space dependent. They would > > be compiled by the application-level 'compiler' package inside of a given > > object space, and not be 'extractible' from there. The interpreter would only > > unwrap the bytecode string, but not the tuple of constants. > > I can't follow. Maybe the physical layout of structures > is different, depending of the object space, but > from the Python view, they are all the same. > If you marshal them, they are the same. Now think of > marshalled code objects, which you unmarshal from > the application-level. All the marshalled contants > will of course be turned into objects in the current > application space, however that is implemented. Marshaled (pypy-) code objects would need to serialize objectspace dependent types. Maybe we should require all objectspaces to implement interoperable serialization. This would also provide a means to transfer objects from one objectspace to another (and to CPython). However, I am a bit uneasy about tying pypy too deep to CPython's codeobject layout. We are likely to want to extend/modify it the future anyway. So maybe we shouldn't take CPython-comptability down to the code object . It's the "heart of gold", produced by the compiler and driving the interpreter. These are areas where we want maximum flexibility and thus shouldn't think too much from CPython's code object which really is more of an implementation detail than anything else. just my 2ec, holger From hpk at trillke.net Thu Feb 27 21:39:19 2003 From: hpk at trillke.net (holger krekel) Date: Thu, 27 Feb 2003 21:39:19 +0100 Subject: [pypy-dev] the truth about pypy Message-ID: <20030227213919.L11666@prim.han.de> Hi, everybody knows, that google is the modern "oracle" of today. So i asked http://www.googlism.com which uses google to find out the truth about something (usually a person). So i asked what "pypy" really is. The one and only answer was "pypy is the optimal initiator sequence" any questions? :-) holger From roccomoretti at netscape.net Fri Feb 28 03:42:19 2003 From: roccomoretti at netscape.net (Rocco Moretti) Date: Thu, 27 Feb 2003 21:42:19 -0500 Subject: [pypy-dev] Release something? Message-ID: <0000AE05.3C7C7342.9ADE5C6A@netscape.net> Armin Rigo wrote: >Hello everybody, > >One more e-mail to remind you that one of the goal of the sprint was to >release something at the end of the week. In regards to release, we should keep in mind that *anyone* who wants to look at pypy can get subversion and check out the most recent version - so in a sense we have a continual release process. So the concept of a "release" only makes sense in regards to people who don't want to bother downloading svn or tracking changes - i.e. non- developers. Therefore it only makes sense to "release" pypy when it is worthwhile and interesting in and of itself. (Which may be now.) Therefore, I submit for approval the following (tentative) release milestone guidelines: ------------------ 0.0.1 - pypy passes all of CPython's regression tests under the trivial object space. (On the appropriate version of CPython.) 0.1.0 - pypy passes all of CPython's regression tests under a StandardObject Space. (Regardless of host Python.) 1.0.0 - pypy passes all of Python's (note generality) regression tests, and is stand-alone (needs no host Python implementation) -- Note that these are just suggestions to start discussion - feel free to ignore them. -Rocco __________________________________________________________________ The NEW Netscape 7.0 browser is now available. Upgrade now! http://channels.netscape.com/ns/browsers/download.jsp Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/ From tismer at tismer.com Fri Feb 28 06:19:06 2003 From: tismer at tismer.com (Christian Tismer) Date: Fri, 28 Feb 2003 06:19:06 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <20030227212853.J11666@prim.han.de> References: <20030224164413.51465501B@bespin.org> <20030227144128.5EA76502E@bespin.org> <20030227155346.GC1066@magma.unil.ch> <20030227162309.7C8CC5176@bespin.org> <20030227170102.GB2872@magma.unil.ch> <3E5E6577.2020109@tismer.com> <20030227212853.J11666@prim.han.de> Message-ID: <3E5EF14A.4040203@tismer.com> holger krekel wrote: [tismer] >>I don't think that RPython has anything to do >>with Python constants and Python code objects, >>and of course I think that creating code objects >>should happen in user space. Then, the problem vanishes. > > > Agreed. Creating code objects at application-level needs > some tweaks, though (see my other post, same thread). Saw that post -- sure, I do think the same. ... > Marshaled (pypy-) code objects would need to serialize > objectspace dependent types. No, I don't think so. Why should they serialize objectspace dependant types? Al the ncessary types are available through userspace, however they are implemented in objectspace. So they should serialize their userspace equivalent, nothing else. exactly that can recreated from userspace, however the objectspace looks like. > Maybe we should require all > objectspaces to implement interoperable serialization. > This would also provide a means to transfer objects from > one objectspace to another (and to CPython). The common external interface is how objects are pickles/marshalled. This is certainly independant from the implementation, and also the reason why I took it as an example to clarify matters. > However, I am a bit uneasy about tying pypy too deep to > CPython's codeobject layout. We are likely to want to > extend/modify it the future anyway. So maybe we shouldn't > take CPython-comptability down to the code object . If we want to borrow code objects and run them in our interpreter, as stated from the beginning, we have to be able to swallow them. This is an absolute must. Requiring compatibility with our objectspace is an absolute mustnot. We need to read the sored structure and create our equivalent object, whatever that is. > It's the "heart of gold", produced by the compiler and > driving the interpreter. These are areas where we want > maximum flexibility and thus shouldn't think too much > from CPython's code object which really is more of > an implementation detail than anything else. lease have a look at the code object from the implementation's view. There is nothing specific which should not be in our implementation. If we have problems to marshal our code objects, since they are object space depandent, then we have very wrong concept about object space. "Object space" is an implementation detail, IMHO. Python's code serialization does not have this problem. Why should we? ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at tismer.com Fri Feb 28 06:20:42 2003 From: tismer at tismer.com (Christian Tismer) Date: Fri, 28 Feb 2003 06:20:42 +0100 Subject: [pypy-dev] Release something? In-Reply-To: <0000AE05.3C7C7342.9ADE5C6A@netscape.net> References: <0000AE05.3C7C7342.9ADE5C6A@netscape.net> Message-ID: <3E5EF1AA.4000105@tismer.com> Rocco Moretti wrote: > Armin Rigo wrote: > > >>Hello everybody, >> >>One more e-mail to remind you that one of the goal of the sprint was to >>release something at the end of the week. > > > In regards to release, we should keep in mind that *anyone* who wants to > look at pypy can get subversion and check out the most recent > version - so in a sense we have a continual release process. > > So the concept of a "release" only makes sense in regards to people who > don't want to bother downloading svn or tracking changes - i.e. non- > developers. Therefore it only makes sense to "release" pypy when it is > worthwhile and interesting in and of itself. (Which may be now.) > > Therefore, I submit for approval the following (tentative) release > milestone guidelines: > ------------------ > 0.0.1 - pypy passes all of CPython's regression tests under the trivial > object space. (On the appropriate version of CPython.) > 0.1.0 - pypy passes all of CPython's regression tests under a > StandardObject Space. (Regardless of host Python.) > 1.0.0 - pypy passes all of Python's (note generality) regression tests, > and is stand-alone (needs no host Python implementation) Good advice! thanks -- chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From paoloinvernizzi at dmsware.com Fri Feb 28 09:17:14 2003 From: paoloinvernizzi at dmsware.com (Paolo Invernizzi) Date: Fri, 28 Feb 2003 09:17:14 +0100 Subject: [pypy-dev] Re: Base Object library (was: stdobjspace status) In-Reply-To: <3E5E6168.1080005@tismer.com> Message-ID: Christian Write... > If I don't model length, but rely on a code generator > to figure this out, well, then I have the problem > that this code generator does not exist yet, but I want > to be able to generate code, as a proof of concept. > For that reason, I liked an implementation that is > so basic, that even I could write a code generator, > without the need for any magic that I don't understand. > > Kinda bottom-up way, maybe this is the wrong way, > but I need to be convinced. At the moment, I feel > completely blocked. This is what I was trying to clarify in my mind. I cannot figure out what is pertinence of the code generator and what is pertinence of "how we implement app-level python int in W_IntObject". IMHO we need at least some piece/ideas of this code-generator, to have a concrete case to discuss. And, I'm not convinced is a good idea to re-implement all CPython in pypy, passing all tests, release something and still have no code for C generation... the risk is to have to rewrite tons of code for adapting it to future requirement of the generator... > Again, this is probably since I thought that in /std/ we > do what's needed for generatong C code. Yup! My impression is that we are inverting the effort... if the generator is VERY smart I can write very pythonic code in implementations of W_XXXObj and compiler stuff, and delegate the analysis to it. If is pretty dumb, We can help it, with RPython for example but I think is time clarify this point. > What I wanted to do is a 'real' implementation of > basic types, whith 'real' algorithms, taken from > the C sources, with some simplifications and > clarifications by using Python, but not by using > Python objects. I think the problem how much pythonic must be or must be not the algorithms for generator taste... Paolo Invernizzi From arigo at tunes.org Fri Feb 28 20:53:46 2003 From: arigo at tunes.org (Armin Rigo) Date: Fri, 28 Feb 2003 11:53:46 -0800 (PST) Subject: [pypy-dev] stdobjspace summary? Message-ID: <20030228195346.7D61C5261@bespin.org> Hello everybody, As I won't read my mail during the whole next week I thought it might be helpful if I attempted some kind of summary. It collects numerous ideas posted here (recently or not) and a bit of personal feelings. I hope it might help in getting some distance over what we are actually trying to do with our "standard object space". I am afraid it is all very vague, but should be kept in mind before we actually design something new there. The std/ directory is the "space of compliant Python object implementations", as opposed to non-compliant object spaces which do some more funny things while the interpreter follows the bytecode. It collects various implementations for the same user types, which are based on some lower-level abstractions. The abstractions that are used are explicitely described in each file, for example: N-bits or machine-sized signed or unsigned integers; memory blocks with explicit management; bitfield objects; various hints (e.g. "this implementation should not be used with refcounts because it has many circularities"); or even which "RPython level" we use (if we develop more and more complex translators), including what basic type operations we allow. Then each low-level abstraction itself has a reference implementation in Python, like Christian's r_int; this should be put in a file that can easily be associated to alternate implementations of the same concept, the ones that can be used by the translator (like the description of the C 'int' type and the tedious complexities of overflow detection in C). These low-level implementations can also explicitely depend on other low-level implementations: for example, "C lists with stored length" could be implemented in term of malloc-style memory blocks -- as opposed to "lists as Java arrays" which don't go lower-level because the length is always stored in Java arrays. "Above" all these files, we have several compliant object spaces, whose purpose is to link a selection of implementations together. For testing purposes object spaces should be easy to build and run in Python, thanks to the Python reference implementations of low-level abstractions. On the other hand, the "C object space" links some or all the files which are (indirectly) based on C-implemented low-level abstractions only; this is what the translator will input. Which files an object space depends on should be specified declaratively, in such a way that we can actually use this information to implement the "wrap", "newlist", "newstring",... methods of the object space. These are the functions that build W_XxxObject instances of the basic types from scratch. Let me stress again that the maximum flexibility seems to be to allow each concept to be implemented in possibly several ways above lower-level concepts, including new ones if the existing ones don't suit you. The arguments on the mailing list show nicely enough that there is just no single better way to implement things, so we should just allow them all to coexist[1]. After that, *other* files are responsible for putting the pieces together -- explicitely (listing the files to include), implicitely (automatically searching for implementations that meet some requirements), or even dynamically at run-time to optimize the performances (not easy, but at least it should be *possible*)! [1] is a departure from one of Python's moto: "there is one and only one obvious way to do something". Well, I assume that I won't shock people around here if I say I never liked this one :-) A bient?t, Armin.