From santagada at gmail.com Thu Oct 5 07:47:12 2006 From: santagada at gmail.com (Leonardo Santagada) Date: Thu, 5 Oct 2006 02:47:12 -0300 Subject: [pypy-dev] PyParser Message-ID: <2f2e5f950610042247y61b6225ep3df5aaf7b1755770@mail.gmail.com> As some of you already know I will start to make the javascript interpreter. I am thinking about using the pyparser module that is in pypy, does someone knows if it is still working? Some people on the #pypy were talking about a modified ply that generates RPython, if someones knows about that also would be nice. Also the documentation of the pyparser module seems to not be in a good shape, I could do something about it if it is desired/allowed :) -- Leonardo Santagada (http://www.lomohomes.com/retype) N?o se preocupe com o que os outros v?o fazer. O melhor jeito de prever o futuro ? inventa-lo. - Alan Kay From jdoda at sympatico.ca Fri Oct 6 18:28:11 2006 From: jdoda at sympatico.ca (Jonathan Doda) Date: Fri, 06 Oct 2006 12:28:11 -0400 Subject: [pypy-dev] benchmarking pypy with pybench Message-ID: <4526841B.4040206@sympatico.ca> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - From what I understand, neither pystone nor richards are very representative of idiomatic python, but pybench is (as much as is possible for a synthetic benchmark, in any case). So, I thought it would be interesting to see how pypy compared to cpython using pybench. Pybench 2.0 was used, downloaded from the python.org svn repo. Pybench reports details of the machine and python installation using the various functions in the platform module, but these functions don't appear to work in pypy, so the reporting was disabled. The UnicodeProperties test also had to be disabled due to pypy not having a unicodedata module. All other tests ran successfully, and pybench was otherwise unmodified. pypy revision 32952 was used, but pypy was translated using the default options, so I imagine these results could be improved on. The tests are run ten times, the first group of three columns shows the best times, and the second group shows the average times. Within each group, the first column is the pypy time, the second the python 2.4.3 time, and the third is the ratio. I hope these results are of interest, and if anyone has any advice on how to improve the test setup, or translation time options I should try, please let me know. Jonathan Doda Test minimum average - ------------------------------------------------------------------------ BuiltinFunctionCalls: 343ms 178ms 1.93 368ms 179ms 2.06 BuiltinMethodLookup: 554ms 161ms 3.44 567ms 174ms 3.26 CompareFloats: 285ms 172ms 1.66 295ms 173ms 1.71 CompareFloatsIntegers: 1492ms 171ms 8.73 1576ms 171ms 9.22 CompareIntegers: 410ms 160ms 2.56 425ms 160ms 2.66 CompareInternedStrings: 414ms 164ms 2.52 421ms 165ms 2.55 CompareLongs: 251ms 137ms 1.83 261ms 138ms 1.89 CompareStrings: 279ms 145ms 1.92 293ms 146ms 2.01 CompareUnicode: 315ms 145ms 2.17 320ms 146ms 2.19 ConcatStrings: 2230ms 201ms 11.09 2251ms 209ms 10.77 ConcatUnicode: 5515ms 209ms 26.39 5570ms 213ms 26.15 CreateInstances: 953ms 180ms 5.29 1000ms 189ms 5.29 CreateNewInstances: 712ms 165ms 4.32 746ms 166ms 4.49 CreateStringsWithConcat: 1067ms 175ms 6.10 1078ms 178ms 6.06 CreateUnicodeWithConcat: 905ms 167ms 5.42 919ms 175ms 5.25 DictCreation: 661ms 136ms 4.86 686ms 136ms 5.04 DictWithFloatKeys: 811ms 191ms 4.25 826ms 192ms 4.30 DictWithIntegerKeys: 482ms 154ms 3.13 493ms 155ms 3.18 DictWithStringKeys: 584ms 143ms 4.08 592ms 144ms 4.11 ForLoops: 898ms 150ms 5.99 905ms 152ms 5.95 IfThenElse: 388ms 135ms 2.87 396ms 136ms 2.91 ListSlicing: 423ms 106ms 3.99 454ms 107ms 4.24 NestedForLoops: 1137ms 176ms 6.46 1141ms 178ms 6.41 NormalClassAttribute: 559ms 156ms 3.58 567ms 157ms 3.61 NormalInstanceAttribute: 935ms 143ms 6.54 945ms 144ms 6.56 PythonFunctionCalls: 1048ms 133ms 7.88 1057ms 134ms 7.89 PythonMethodCalls: 1275ms 168ms 7.59 1285ms 169ms 7.60 Recursion: 1599ms 185ms 8.64 1607ms 186ms 8.64 SecondImport: 685ms 137ms 5.00 694ms 138ms 5.03 SecondPackageImport: 602ms 146ms 4.12 618ms 149ms 4.15 SecondSubmoduleImport: 987ms 179ms 5.51 996ms 182ms 5.47 SimpleComplexArithmetic: 745ms 197ms 3.78 750ms 201ms 3.73 SimpleDictManipulation: 668ms 151ms 4.42 670ms 153ms 4.38 SimpleFloatArithmetic: 396ms 172ms 2.30 407ms 182ms 2.24 SimpleIntFloatArithmetic: 363ms 131ms 2.77 376ms 133ms 2.83 SimpleIntegerArithmetic: 367ms 132ms 2.78 377ms 133ms 2.83 SimpleListManipulation: 384ms 121ms 3.17 399ms 123ms 3.24 SimpleLongArithmetic: 651ms 144ms 4.52 660ms 145ms 4.55 SmallLists: 371ms 140ms 2.65 372ms 142ms 2.62 SmallTuples: 2122ms 148ms 14.34 2144ms 150ms 14.29 SpecialClassAttribute: 545ms 154ms 3.54 573ms 155ms 3.70 SpecialInstanceAttribute: 939ms 182ms 5.16 971ms 184ms 5.28 StringMappings: 517ms 170ms 3.04 539ms 172ms 3.13 StringPredicates: 491ms 160ms 3.07 518ms 168ms 3.08 StringSlicing: 994ms 154ms 6.45 1039ms 160ms 6.49 TryExcept: 424ms 156ms 2.72 444ms 157ms 2.83 TryRaiseExcept: 666ms 182ms 3.66 687ms 184ms 3.73 TupleSlicing: 521ms 163ms 3.20 547ms 165ms 3.32 UnicodeMappings: 389ms 168ms 2.32 403ms 170ms 2.37 UnicodePredicates: 433ms 152ms 2.85 449ms 157ms 2.86 UnicodeSlicing: 1874ms 180ms 10.41 1922ms 186ms 10.33 Totals: 42658ms 8125ms 5.25 43595ms 8261ms 5.28 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) iD8DBQFFJoQbWEgeeLQ8a74RAh36AJ9J0MwiaaM6aqAnvQidrTCCeSyKMQCfWvRZ vyz4YzrY0oEYzaAgBd9Ny+0= =Qr6G -----END PGP SIGNATURE----- From cfbolz at gmx.de Fri Oct 6 20:15:49 2006 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 06 Oct 2006 20:15:49 +0200 Subject: [pypy-dev] benchmarking pypy with pybench In-Reply-To: <4526841B.4040206@sympatico.ca> References: <4526841B.4040206@sympatico.ca> Message-ID: <45269D55.3080307@gmx.de> Hi Jonathan! Thanks for doing this, we used pybench a while ago but didn't pursue if further. This is indeed very interesting! Jonathan Doda wrote: > - From what I understand, neither pystone nor richards are very > representative of idiomatic python, but pybench is (as much as is > possible for a synthetic benchmark, in any case). So, I thought it would > be interesting to see how pypy compared to cpython using pybench. > > Pybench 2.0 was used, downloaded from the python.org svn repo. Pybench > reports details of the machine and python installation using the various > functions in the platform module, but these functions don't appear to > work in pypy, so the reporting was disabled. The UnicodeProperties test > also had to be disabled due to pypy not having a unicodedata module. All > other tests ran successfully, and pybench was otherwise unmodified. PyPy has a unicodedata module, it is not compiled in by default, though. It's not clear to me why the platform module does not work, though. > pypy revision 32952 was used, but pypy was translated using the default > options, so I imagine these results could be improved on. > > The tests are run ten times, the first group of three columns shows the > best times, and the second group shows the average times. Within each > group, the first column is the pypy time, the second the python 2.4.3 > time, and the third is the ratio. > > I hope these results are of interest, and if anyone has any advice on > how to improve the test setup, or translation time options I should try, > please let me know. You could try to pass in some translation-options that try to optimize common string operations and small integers: python translate.py targetpypystandalone.py --objspace-std-withsmallint --objspace-std-withstrslice --objspace-std-withstrjoin Other interesting options are --objspace-std-withstrdict (to enable optimizations for dictionaries with string keys), --usemodules=unicodedata (to enable the unicode data module). The options are mutually compatible, so you can do a super-build with all of them (which unfortunately increases executable size quite a bit). Cheers, Carl Friedrich Bolz From simon at arrowtheory.com Sun Oct 8 21:50:20 2006 From: simon at arrowtheory.com (Simon Burton) Date: Sun, 8 Oct 2006 12:50:20 -0700 Subject: [pypy-dev] py.test bug Message-ID: <20061008125020.72f527e0.simon@arrowtheory.com> Traceback (most recent call last): File "/home/simon/bin/py.test", line 4, in ? py.test.cmdline.main() File "/home/simon/local/pypy/py/test/cmdline.py", line 17, in main failures = session.main(args) File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 34, in main return super(TerminalSession, self).main(args) File "/home/simon/local/pypy/py/test/session.py", line 52, in main self.footer(colitems) File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 167, in footer self.failures() File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 262, in failures self.repr_failure(colitem, outcome) File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 280, in repr_failure handler(item, excinfo, traceback) File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 297, in repr_failure_tblong firstsourceline = entry.getfirstlinesource() AttributeError: 'TracebackEntry' object has no attribute 'getfirstlinesource' Simon. From fijal at genesilico.pl Sun Oct 8 22:55:44 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 08 Oct 2006 22:55:44 +0200 Subject: [pypy-dev] py.test bug In-Reply-To: <20061008125020.72f527e0.simon@arrowtheory.com> References: <20061008125020.72f527e0.simon@arrowtheory.com> Message-ID: <452965D0.8070800@genesilico.pl> Simon Burton wrote: >Traceback (most recent call last): > File "/home/simon/bin/py.test", line 4, in ? > py.test.cmdline.main() > File "/home/simon/local/pypy/py/test/cmdline.py", line 17, in main > failures = session.main(args) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 34, in main > return super(TerminalSession, self).main(args) > File "/home/simon/local/pypy/py/test/session.py", line 52, in main > self.footer(colitems) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 167, in footer > self.failures() > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 262, in failures > self.repr_failure(colitem, outcome) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 280, in repr_failure > handler(item, excinfo, traceback) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 297, in repr_failure_tblong > firstsourceline = entry.getfirstlinesource() >AttributeError: 'TracebackEntry' object has no attribute 'getfirstlinesource' > > >Simon. >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev > > You seem to have old version of py.test. Please update to svn trunk. From fijal at genesilico.pl Sun Oct 8 22:59:19 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 08 Oct 2006 22:59:19 +0200 Subject: [pypy-dev] py.test bug In-Reply-To: <20061008125020.72f527e0.simon@arrowtheory.com> References: <20061008125020.72f527e0.simon@arrowtheory.com> Message-ID: <452966A7.7080309@genesilico.pl> Simon Burton wrote: >Traceback (most recent call last): > File "/home/simon/bin/py.test", line 4, in ? > py.test.cmdline.main() > File "/home/simon/local/pypy/py/test/cmdline.py", line 17, in main > failures = session.main(args) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 34, in main > return super(TerminalSession, self).main(args) > File "/home/simon/local/pypy/py/test/session.py", line 52, in main > self.footer(colitems) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 167, in footer > self.failures() > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 262, in failures > self.repr_failure(colitem, outcome) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 280, in repr_failure > handler(item, excinfo, traceback) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 297, in repr_failure_tblong > firstsourceline = entry.getfirstlinesource() >AttributeError: 'TracebackEntry' object has no attribute 'getfirstlinesource' > > >Simon. >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev > > Ugh. That seems that I'm the one with old version of py.test :-( From simon at arrowtheory.com Mon Oct 9 01:48:09 2006 From: simon at arrowtheory.com (Simon Burton) Date: Sun, 8 Oct 2006 16:48:09 -0700 Subject: [pypy-dev] py.test bug In-Reply-To: <452966A7.7080309@genesilico.pl> References: <20061008125020.72f527e0.simon@arrowtheory.com> <452966A7.7080309@genesilico.pl> Message-ID: <20061008164809.099459a3.simon@arrowtheory.com> On Sun, 08 Oct 2006 22:59:19 +0200 Maciek Fijalkowski wrote: > > > > Ugh. That seems that I'm the one with old version of py.test :-( I think i fixed it.. (it's been commited) Simon. From anthony at interlink.com.au Mon Oct 9 12:42:07 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon, 9 Oct 2006 20:42:07 +1000 Subject: [pypy-dev] benchmarking pypy with pybench In-Reply-To: <45269D55.3080307@gmx.de> References: <4526841B.4040206@sympatico.ca> <45269D55.3080307@gmx.de> Message-ID: <200610092042.09163.anthony@interlink.com.au> On Saturday 07 October 2006 04:15, Carl Friedrich Bolz wrote: > PyPy has a unicodedata module, it is not compiled in by default, though. > It's not clear to me why the platform module does not work, though. 'platform.py' also doesn't work with IronPython. I posted a patch on SF for this - www.python.org/sf/1563842 There's another patch with pybench fixes for IronPython that might be related. Don't have the patch# right now. Making it work with pypy is probably worthwhile, too. Someone working on the project might want to add a note to the patch. -- Anthony Baxter It's never too late to have a happy childhood. From arigo at tunes.org Tue Oct 10 00:52:17 2006 From: arigo at tunes.org (Armin Rigo) Date: Tue, 10 Oct 2006 00:52:17 +0200 Subject: [pypy-dev] py.test bug In-Reply-To: <20061008164809.099459a3.simon@arrowtheory.com> References: <20061008125020.72f527e0.simon@arrowtheory.com> <452966A7.7080309@genesilico.pl> <20061008164809.099459a3.simon@arrowtheory.com> Message-ID: <20061009225217.GA14831@code0.codespeak.net> Hi Simon, On Sun, Oct 08, 2006 at 04:48:09PM -0700, Simon Burton wrote: > > Ugh. That seems that I'm the one with old version of py.test :-( > > I think i fixed it.. (it's been commited) Sorry for the trouble, I forgot to checkin code.py together with the rest of my changes. A bientot, Armin From jacob at strakt.com Wed Oct 11 01:53:04 2006 From: jacob at strakt.com (Jacob =?iso-8859-1?q?Hall=E9n?=) Date: Wed, 11 Oct 2006 01:53:04 +0200 Subject: [pypy-dev] Slight worries Message-ID: <200610110153.07659.jacob@strakt.com> I try to follow the discussions on IRC regularly, and I have some worries after what I have seen in the last couple of weeks. There seem to be too many problems with broken checkins. The translation breaks, tests stop working etc. While I understand that running tests takes an enormous amount of time, it worries me that we have to fix things on the trunk so very often. I think this eats up time that could be better spent. If people need to leave testing for automated tests, there ought to be a less disruptive way of doing this. Of course I may be wrong, and this is really the optimal way of moving forward, but the I would like to be reassured that it is that way. A second thing that is bothering me is that there does not seem to be a canonical list of software that our development system is designed to work with. I feel that this is important. It is difficult enough to make things work with a specific set of tools. Currently we should be making sure that things work with one set of tools and the generalization to various different versions of pygame, grapviz, gcc and whatnot can happen later. I know this is not quite attainable, since preferred versions of the various tools don't work on all platforms, but I hope you see the core point. Trying to make things work with all possible combinations of peripheral software will just generate lots and lots of work. We need a public stance on what we support. Jacob Hall?n From exarkun at divmod.com Wed Oct 11 02:41:17 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 10 Oct 2006 20:41:17 -0400 Subject: [pypy-dev] Slight worries In-Reply-To: <200610110153.07659.jacob@strakt.com> Message-ID: <20061011004117.1717.1169292187.divmod.quotient.73138@ohm> On Wed, 11 Oct 2006 01:53:04 +0200, Jacob Hall?n wrote: >I try to follow the discussions on IRC regularly, and I have some worries >after what I have seen in the last couple of weeks. > >There seem to be too many problems with broken checkins. The translation >breaks, tests stop working etc. While I understand that running tests takes >an enormous amount of time, it worries me that we have to fix things on the >trunk so very often. I think this eats up time that could be better spent. > >If people need to leave testing for automated tests, there ought to be a less >disruptive way of doing this. > >Of course I may be wrong, and this is really the optimal way of moving >forward, but the I would like to be reassured that it is that way. I'd like to echo this sentement. It's really amazing how much progress the PyPy project has made over the last year or so, but it's frustrating to never be sure if updating to the latest revision is going to eliminate my ability to do anything with it. The translation process is currently immensely expensive, so I can certainly understand that people want to commit to trunk without doing extensive testing (I can imagine such testing taking up an entire day or more in some cases). But as well all know, the introduction of regressions into trunk comes with costs too. I'm not going to suggest any specific course of action, because I'm confident everyone involved with the PyPy project can recognize this problem and consider ways to address it. Howeverr, if there's anything I can do to help come up with or enact a solution, I offer what assistance I can. For starters, if it would be helpful, I may be able to offer some hardware to run the automated test suite. This aside, PyPy has come a long way, and everyone involved should be very proud of what you guys have created. I have confidence that these growing pains will soon be behind the project. > >A second thing that is bothering me is that there does not seem to be a >canonical list of software that our development system is designed to work >with. I feel that this is important. It is difficult enough to make things >work with a specific set of tools. Currently we should be making sure that >things work with one set of tools and the generalization to various different >versions of pygame, grapviz, gcc and whatnot can happen later. I know this is >not quite attainable, since preferred versions of the various tools don't >work on all platforms, but I hope you see the core point. Trying to make >things work with all possible combinations of peripheral software will just >generate lots and lots of work. We need a public stance on what we support. > >Jacob Hall?n Jean-Paul From fijal at genesilico.pl Thu Oct 12 19:58:29 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Thu, 12 Oct 2006 19:58:29 +0200 Subject: [pypy-dev] [pypy-svn] r33236 - pypy/dist/pypy/objspace/test In-Reply-To: <20061012172440.DAAAB1007C@code0.codespeak.net> References: <20061012172440.DAAAB1007C@code0.codespeak.net> Message-ID: <452E8245.1090601@genesilico.pl> auc at codespeak.net wrote: >Author: auc >Date: Thu Oct 12 19:24:38 2006 >New Revision: 33236 > >Modified: > pypy/dist/pypy/objspace/test/test_logicobjspace.py >Log: >some needed adjustement (most notably, removed explicit scheduling calls) > > >Modified: pypy/dist/pypy/objspace/test/test_logicobjspace.py >============================================================================== >--- pypy/dist/pypy/objspace/test/test_logicobjspace.py (original) >+++ pypy/dist/pypy/objspace/test/test_logicobjspace.py Thu Oct 12 19:24:38 2006 >@@ -6,6 +6,11 @@ > # we might be called from _test_logic_build > # if not, check your paths > >+try: >+ is_interpreted() >+except: >+ def is_interpreted(): return True >+ > > > try: from pypy.conftest import gettestobjspace from py.test import skip except ImportError: pass # we might be called from _test_logic_build # if not, check your paths try: is_interpreted() except: def is_interpreted(): return True This is really cool kind of code. If we cannot import pypy, # check your paths is in comment (no print, nothing). If we cannot import py.test, we probably won't have is_interpreted, so if is_interpreted(): py.test.skip("dsa") will just break. And simply calling py.test test_logicobjspace.py fails. (It should skip if there are no blablabla available). This attempts of having broken tests makes py.test work harder (because I always have to check if tests are broken by default or not). From aurelien.campeas at logilab.fr Fri Oct 13 09:43:41 2006 From: aurelien.campeas at logilab.fr (=?iso-8859-1?Q?Aur=E9lien_Camp=E9as?=) Date: Fri, 13 Oct 2006 09:43:41 +0200 Subject: [pypy-dev] [pypy-svn] r33236 - pypy/dist/pypy/objspace/test In-Reply-To: <452E8245.1090601@genesilico.pl> References: <20061012172440.DAAAB1007C@code0.codespeak.net> <452E8245.1090601@genesilico.pl> Message-ID: <20061013074341.GA19980@crater.logilab.fr> On Thu, Oct 12, 2006 at 07:58:29PM +0200, Maciek Fijalkowski wrote: > auc at codespeak.net wrote: > > >Author: auc > >Date: Thu Oct 12 19:24:38 2006 > >New Revision: 33236 > > > >Modified: > > pypy/dist/pypy/objspace/test/test_logicobjspace.py > >Log: > >some needed adjustement (most notably, removed explicit scheduling calls) > > > > > >Modified: pypy/dist/pypy/objspace/test/test_logicobjspace.py > >============================================================================== > >--- pypy/dist/pypy/objspace/test/test_logicobjspace.py (original) > >+++ pypy/dist/pypy/objspace/test/test_logicobjspace.py Thu Oct 12 19:24:38 2006 > >@@ -6,6 +6,11 @@ > > # we might be called from _test_logic_build > > # if not, check your paths > > > >+try: > >+ is_interpreted() > >+except: > >+ def is_interpreted(): return True > >+ > > > > > > > try: > from pypy.conftest import gettestobjspace > from py.test import skip > except ImportError: > pass > # we might be called from _test_logic_build > # if not, check your paths > > try: > is_interpreted() > except: > def is_interpreted(): return True > > This is really cool kind of code. If we cannot import pypy, # check your > paths is in comment (no print, nothing). If we cannot import py.test, we > probably won't have is_interpreted, so if is_interpreted(): Wrong. Ok this deserves an explanation : whenever we want to test a pypy logic build, using _test_logic_build.py, is_interpreted is defined and returns False ... (yes it is not too explicit nor pretty). > py.test.skip("dsa") will just break. And simply calling py.test > test_logicobjspace.py fails. (It should skip if there are no blablabla > available). What are you trying to do ? > > This attempts of having broken tests makes py.test work harder (because > I always have to check if tests are broken by default or not). > Please change the tone of your remarks. From niko at alum.mit.edu Fri Oct 13 11:45:49 2006 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 13 Oct 2006 11:45:49 +0200 Subject: [pypy-dev] CLI tests on Mac OS X Message-ID: Hey, I am wondering whether anyone has had success running the CLI tests on Mac OS X? I am having a problem where the tests seem to lock up in random places. I have tried letting them run all night without any progress. In each case, "top" reveals that the mono process is sucking up 60-80% of my CPU time, and if I kill the mono process then the test fails but the test process continues. When I run the tests on my linux machine, it seems to work --- however, since I use the mac for most of my development, it is a bit of pain to move the patches back and forth for testing, especially if new files are involved. I have mono installed through DarwinPorts, and it claims to be version 1.1.16.1. This is on Mac OS X 10.4, and using Python 2.4 (also from DarwinPorts). Has anyone else seen similar behavior or know a solution? thanks! Niko From cfbolz at gmx.de Fri Oct 13 13:02:50 2006 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 13 Oct 2006 13:02:50 +0200 Subject: [pypy-dev] [pypy-svn] r33249 - in pypy/dist/pypy/translator: cli jvm jvm/src oosupport In-Reply-To: <20061013104610.BCC32100BD@code0.codespeak.net> References: <20061013104610.BCC32100BD@code0.codespeak.net> Message-ID: <452F725A.2000005@gmx.de> Hi Niko niko at codespeak.net wrote: > Author: niko > Date: Fri Oct 13 12:46:09 2006 > New Revision: 33249 > > Added: > pypy/dist/pypy/translator/jvm/ > pypy/dist/pypy/translator/jvm/generator.py > pypy/dist/pypy/translator/jvm/genjvm.py > pypy/dist/pypy/translator/jvm/opcodes.py > pypy/dist/pypy/translator/jvm/src/ > pypy/dist/pypy/translator/jvm/src/PyPy.java > pypy/dist/pypy/translator/jvm/types.py > Modified: > pypy/dist/pypy/translator/cli/function.py > pypy/dist/pypy/translator/cli/metavm.py > pypy/dist/pypy/translator/oosupport/metavm.py > Log: > first stabs at a jvm backend --- doesn't run or anything yet, but contains > at least a plausible implementation of the scalar operations > > also refactor a few things from cli into oosupport (mostly deleting duplicates > like getfield and setfield) and add a few (possibly oververbose :) comments. > changes to cli are very minor, but ran cli tests in any case and received > no regressions. [snip] I know that it is not much fun to do it this early in the life of a backend, but please try to write some tests. I will probably be annoying but you will get payoffs very quickly and thinking about how to test something (ideally before you start coding) helps you understand the upcoming tasks. Can't really comment on the technical quality of your checkin since I don't know any Java (nor much of the ootypesystem, for that matter :-). "Untested code is not working" Cheers, Carl Friedrich From niko at alum.mit.edu Fri Oct 13 13:07:53 2006 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 13 Oct 2006 13:07:53 +0200 Subject: [pypy-dev] [pypy-svn] r33249 - in pypy/dist/pypy/translator: cli jvm jvm/src oosupport In-Reply-To: <452F725A.2000005@gmx.de> References: <20061013104610.BCC32100BD@code0.codespeak.net> <452F725A.2000005@gmx.de> Message-ID: <324B6388-8878-432F-8BFB-27D8C6E1E709@alum.mit.edu> > I know that it is not much fun to do it this early in the life of a > backend, but please try to write some tests. I will probably be > annoying > but you will get payoffs very quickly and thinking about how to test > something (ideally before you start coding) helps you understand the > upcoming tasks. Agreed. Actually, my next goal was to write start writing tests for the code I had written so far, but I thought I would check-in what I had done already before I started them. Niko From fijal at genesilico.pl Sat Oct 21 13:35:25 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sat, 21 Oct 2006 13:35:25 +0200 Subject: [pypy-dev] py.test distributed Message-ID: <453A05FD.8090307@genesilico.pl> I'm gathering issues that you're all missing in py.test distributed. There is as well LSession (--session=L) which by default runs locally boxed version of tests (with segfault catching, applevel output catching and such). From cfbolz at gmx.de Sat Oct 21 23:52:39 2006 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sat, 21 Oct 2006 23:52:39 +0200 Subject: [pypy-dev] range list implementation Message-ID: <453A96A7.5080005@gmx.de> Hi all! Just a few words about the range list implementation I wrote during the last few days (when I was fed up with the config branch...). There is a new list implementation W_RangeListObject. The idea is that calling range will not immediately allocate the whole list but only store start, stop and step into the range list object. The range list object behaves like a perfectly normal list to the user. For some operations the range list is 'forced', that is the full list is generated. This is especially the case for operations that mutate the list. The following operations on the range list keep it in its non-forced memory-saving form: * iteration * len * getitem * getitem with a slice (which returns a new range list) * iter * repr * reverse * sort The reverse and sort method are admittedly somewhat useless (who constructs a range and sorts it?) but were easy to do. It would be possible to add support for more operations, but I doubt that it would be useful. I have not timed the result extensively, but you can pull tricks like that: >>>> import sys >>>> r = range(sys.maxint) >>>> len(r) 2147483647 >>>> iter(r) >>>> print r[1000:2000:30] [1000, 1030, 1060, 1090, 1120, 1150, 1180, 1210, 1240, 1270, 1300, 1330, 1360, 1390, 1420, 1450, 1480, 1510, 1540, 1570, 1600, 1630, 1660, 1690, 1720, 1750, 1780, 1810, 1840, 1870, 1900, 1930, 1960, 1990] Some very trivial timings: 2.877 seconds CPython versus 0.026 seconds pypy-c-withrangelist for the following snippet which is of course written in a way to showcase the advantages of range lists. Of course CPython uses a lot more memory too :-) import time t1 = time.time() result = [] for i in range(10000): result.append(range(i)) t2 = time.time() print t2 - t1 It remains to be seen what sort of benefits this gives for realistic applications. Cheers, Carl Friedrich From len-l at telus.net Sun Oct 22 01:43:57 2006 From: len-l at telus.net (Lenard Lindstrom) Date: Sat, 21 Oct 2006 16:43:57 -0700 Subject: [pypy-dev] range list implementation In-Reply-To: <453A96A7.5080005@gmx.de> Message-ID: <453A4E4D.22498.969B45@len-l.telus.net> On 21 Oct 2006 at 23:52, Carl Friedrich Bolz wrote: > Hi all! > > Just a few words about the range list implementation I wrote during the > last few days (when I was fed up with the config branch...). There is a > new list implementation W_RangeListObject. The idea is that calling > range will not immediately allocate the whole list but only store start, > stop and step into the range list object. The range list object behaves > like a perfectly normal list to the user. For some operations the range > list is 'forced', that is the full list is generated. This is especially > the case for operations that mutate the list. The following operations > on the range list keep it in its non-forced memory-saving form: > > * iteration > * len > * getitem > * getitem with a slice (which returns a new range list) > * iter > * repr > * reverse > * sort > Early versions of the xrange type supported slicing. Slicing was dropped from xrange in Python 2.2. As I recall reading somewhere there just wasn't enough interest to justify the cost of maintaining the C code. But maybe it makes more sense when coded in Python. Lenard Lindstrom From fijal at genesilico.pl Sun Oct 22 14:10:56 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 22 Oct 2006 14:10:56 +0200 Subject: [pypy-dev] [pypy-svn] r33528 - in pypy/dist/pypy/translator: jvm jvm/src jvm/test oosupport In-Reply-To: <20061021233702.32CB210070@code0.codespeak.net> References: <20061021233702.32CB210070@code0.codespeak.net> Message-ID: <453B5FD0.70101@genesilico.pl> niko at codespeak.net wrote: >Author: niko >Date: Sun Oct 22 01:36:59 2006 >New Revision: 33528 > >Added: > pypy/dist/pypy/translator/jvm/__init__.py > pypy/dist/pypy/translator/jvm/option.py > pypy/dist/pypy/translator/jvm/test/ > pypy/dist/pypy/translator/jvm/test/__init__.py > pypy/dist/pypy/translator/jvm/test/runtest.py > pypy/dist/pypy/translator/jvm/test/test_bool.py > pypy/dist/pypy/translator/jvm/typesystem.py >Removed: > pypy/dist/pypy/translator/jvm/types.py >Modified: > pypy/dist/pypy/translator/jvm/ (props changed) > pypy/dist/pypy/translator/jvm/conftest.py > pypy/dist/pypy/translator/jvm/database.py > pypy/dist/pypy/translator/jvm/generator.py > pypy/dist/pypy/translator/jvm/genjvm.py > pypy/dist/pypy/translator/jvm/node.py > pypy/dist/pypy/translator/jvm/opcodes.py > pypy/dist/pypy/translator/jvm/src/PyPy.java > pypy/dist/pypy/translator/oosupport/metavm.py >Log: >further work towards working jvm tests --- this is an intermediate >check-in, still evolving the JVM TypeSystem and database. > > > >Added: pypy/dist/pypy/translator/jvm/__init__.py >============================================================================== > >Modified: pypy/dist/pypy/translator/jvm/conftest.py >============================================================================== >--- pypy/dist/pypy/translator/jvm/conftest.py (original) >+++ pypy/dist/pypy/translator/jvm/conftest.py Sun Oct 22 01:36:59 2006 >@@ -13,6 +13,10 @@ > ## help='View the graphs before they are generated'), > Option('--wd', action='store_true', dest='wd', default=False, > help='Output to current directory instead of /tmp'), >+ Option('--noassemble', action='store_true', dest="noasm", default=False, >+ help="don't assemble jasmin files"), >+ Option('--norun', action='store_true', dest="norun", default=False, >+ help="don't run the compiled executable"), > Option('--package', action='store', dest='package', default='pypy', > help='Package to output generated classes into') > #Option('--opt', action='XXX', dest='YYY', default=DEF, help='HELP') > >Modified: pypy/dist/pypy/translator/jvm/database.py >============================================================================== >--- pypy/dist/pypy/translator/jvm/database.py (original) >+++ pypy/dist/pypy/translator/jvm/database.py Sun Oct 22 01:36:59 2006 >@@ -1,11 +1,85 @@ > """ >-For now, a simple worklist abstraction. >+The database tracks which graphs have already been generated, and maintains >+a worklist. It also contains a pointer to the type system. It is passed >+into every node for generation along with the generator. > """ >+from cStringIO import StringIO >+from pypy.rpython.ootypesystem import ootype >+from pypy.translator.jvm.typesystem import jvm_method_desc >+from pypy.translator.jvm import node >+import pypy.translator.jvm.generator as jvmgen >+import pypy.translator.jvm.typesystem as jvmtypes > > class Database: >- def __init__(self): >- self._pending = [] >+ def __init__(self, ts): >+ # Public attributes: >+ self.type_system = ts >+ >+ # Private attributes: >+ self._classes = {} # Maps ootype class objects to node.Class objects >+ self._counter = 0 # Used to create unique names >+ self._pending = [] # Worklist >+ >+ def _make_unique_name(self, nm): >+ cnt = self._counter >+ self._counter += 1 >+ return nm + "_" + str(cnt) + "_" >+ >+ def get_class_for(self, ooclass): >+ """ Given an OOTypeSystem Instance object representing a user >+ defined class (ooclass), returns a node.Class object representing >+ its jvm counterpart. """ >+ >+ # Create class object if it does not already exist: >+ if ooclass in self._classes: >+ return self._classes[ooclass] >+ clname = self._make_unique_name(ooclass._name) >+ clobj = self._classes[ooclass] = node.Class(clname) >+ >+ # Add fields: >+ for fieldnm, (fieldty, fielddef) in ooclass._fields.iteritems(): >+ if ftype is ootype.Void: continue >+ fieldnm = self._make_unique_name(fieldnm) >+ fieldty = self.type_system.ootype_to_jvm(ftype) >+ clobj.add_field(fieldty, fieldnm) # TODO --- fielddef?? >+ >+ # Add methods: >+ for mname, mimpl in ooclass._methods.iteritems(): >+ if not hasattr(mimpl, 'graph'): >+ # Abstract method >+ TODO >+ else: >+ # if the first argument's type is not a supertype of >+ # this class it means that this method this method is >+ # not really used by the class: don't render it, else >+ # there would be a type mismatch. >+ args = m_meth.graph.getargs() >+ SELF = args[0].concretetype >+ if not ootype.isSubclass(ooclass, SELF): continue >+ mobj = _method_for_graph(clobj, False, mimpl.graph) >+ clobj.add_method(mobj) >+ >+ return clobj >+ >+ def _method_for_graph(self, classobj, is_static, graph): > >+ """ >+ Creates a node.Function object for a particular graph. Adds the >+ method to 'classobj', which should be a node.Class object. >+ """ >+ >+ # Build up a func object >+ func_name = self._make_unique_name(graph.name) >+ argtypes = [arg.concretetype for arg in graph.getargs() >+ if arg.concretetype is not ootype.Void] >+ jargtypes = [self.type_system.ootype_to_jvm(argty) >+ for argty in argtypes] >+ rettype = graph.getreturnvar().concretetype >+ jrettype = self.type_system.ootype_to_jvm(rettype) >+ funcobj = self._translated[cachekey] = node.Function( >+ classobj, func_name, jargtypes, jrettype, graph, is_static) >+ return funcobj >+ > def pending_node(self, node): > self._pending.append(node) > >@@ -14,6 +88,3 @@ > > def pop(self): > return self._pending.pop() >- >- def method_for_graph(self, graph): >- > >Modified: pypy/dist/pypy/translator/jvm/generator.py >============================================================================== >--- pypy/dist/pypy/translator/jvm/generator.py (original) >+++ pypy/dist/pypy/translator/jvm/generator.py Sun Oct 22 01:36:59 2006 >@@ -1,6 +1,7 @@ >+import os # >+from pypy.objspace.flow import model as flowmodel > from pypy.translator.oosupport.metavm import Generator >- >-__all__ = ['JasminGenerator'] >+from pypy.translator.jvm.typesystem import JvmType > > # ___________________________________________________________________________ > # JVM Opcode Flags: >@@ -8,11 +9,11 @@ > # Indicates certain properties of each opcode. Used mainly for debugging > # assertions > >-NOFLAGS = 0 >-BRANCH = 1 # Opcode is a branching opcode (implies a label argument) >-INTARG = 2 # Opcode has an integer argument >-CONST = 6 # Opcode has specialized variants (implies INTARG) >-INVOKE = 8 # Opcode is some kind of method invocation >+NOFLAGS = 0 >+BRANCH = 1 # Opcode is a branching opcode (implies a label argument) >+INTARG = 2 # Opcode has an integer argument >+CONSTSPEC = 6 # Opcode has specialized variants (implies INTARG) >+INVOKE = 8 # Opcode is some kind of method invocation > > # ___________________________________________________________________________ > # JVM Opcodes: >@@ -27,6 +28,48 @@ > """ > self.flags = flags > self.jvmstr = jvmstr >+ >+class OpcodeFamily(object): >+ """ >+ Many opcodes in JVM have variants that depend on the type of the >+ operands; for example, one must choose the correct ALOAD, ILOAD, >+ or DLOAD depending on whether one is loading a reference, integer, >+ or double variable respectively. Each instance of this class >+ defines one 'family' of opcodes, such as the LOAD family shown >+ above, and produces Opcode objects specific to a particular type. >+ """ >+ def __init__(self, flags, suffix): >+ """ >+ flags is a set of flags (see above) that describe opcode >+ jvmstr is the name for jasmin printouts >+ """ >+ self.flags = flags >+ self.suffix = suffix >+ self.cache = {} >+ >+ def _o(self, prefix): >+ try: >+ return self.cache[prefix] >+ except KeyError: >+ self.cache[prefix] = obj = Opcode(self.flags, prefix+self.suffix) >+ return obj >+ >+ def for_type(self, argtype): >+ """ Returns a customized opcode of this family appropriate to >+ 'argtype', a JvmType object. """ >+ >+ # These are always true: >+ if self[0] == 'L': return self._o("A") # Objects >+ if self[0] == '[': return self._o("A") # Arrays >+ if self == 'I': return self._o("I") # Integers >+ if self == 'J': return self._o("L") # Integers >+ if self == 'D': return self._o("D") # Doubles >+ if self == 'C': return self._o("C") # Characters >+ if self == 'B': return self._o("B") # Bytes >+ if self == 'V': return self._o("") # Void [used by RETURN] >+ >+ # TODO --- extend? etc >+ unimplemented > > # Define the opcodes for IFNE, IFEQ, IFLT, IF_ICMPLT, etc. The IFxx > # variants compare a single integer arg against 0, and the IF_ICMPxx >@@ -48,11 +91,11 @@ > > # Other opcodes > GOTO = Opcode(BRANCH, 'goto') >-ICONST = Opcode(CONST, 'iconst') >-DCONST_0 = Opcode(CONST, 'dconst_0') >-DCONST_1 = Opcode(CONST, 'dconst_1') >-LCONST_0 = Opcode(CONST, 'lconst_0') >-LCONST_1 = Opcode(CONST, 'lconst_1') >+ICONST = Opcode(CONSTSPEC, 'iconst') >+DCONST_0 = Opcode(NOFLAGS, 'dconst_0') >+DCONST_1 = Opcode(NOFLAGS, 'dconst_1') >+LCONST_0 = Opcode(NOFLAGS, 'lconst_0') >+LCONST_1 = Opcode(NOFLAGS, 'lconst_1') > GETFIELD = Opcode(NOFLAGS, 'getfield') > PUTFIELD = Opcode(NOFLAGS, 'putfield') > GETSTATIC = Opcode(NOFLAGS, 'getstatic') >@@ -66,16 +109,56 @@ > IDIV = Opcode(NOFLAGS, 'idiv') > IREM = Opcode(NOFLAGS, 'irem') > IAND = Opcode(NOFLAGS, 'iand') >+IOR = Opcode(NOFLAGS, 'ior') > ISHL = Opcode(NOFLAGS, 'ishl') > ISHR = Opcode(NOFLAGS, 'ishr') >+IUSHR = Opcode(NOFLAGS, 'iushr') > DCMPG = Opcode(NOFLAGS, 'dcmpg') > DCMPL = Opcode(NOFLAGS, 'dcmpl') > NOP = Opcode(NOFLAGS, 'nop') > I2D = Opcode(NOFLAGS, 'i2d') > I2L = Opcode(NOFLAGS, 'i2l') >+D2I= Opcode(NOFLAGS, 'd2i') >+L2I = Opcode(NOFLAGS, 'l2i') >+ATHROW = Opcode(NOFLAGS, 'athrow') >+DNEG = Opcode(NOFLAGS, 'dneg') >+DADD = Opcode(NOFLAGS, 'dadd') >+DSUB = Opcode(NOFLAGS, 'dsub') >+DMUL = Opcode(NOFLAGS, 'dmul') >+DDIV = Opcode(NOFLAGS, 'ddiv') >+DREM = Opcode(NOFLAGS, 'drem') >+LNEG = Opcode(NOFLAGS, 'lneg') >+LADD = Opcode(NOFLAGS, 'ladd') >+LSUB = Opcode(NOFLAGS, 'lsub') >+LMUL = Opcode(NOFLAGS, 'lmul') >+LDIV = Opcode(NOFLAGS, 'ldiv') >+LREM = Opcode(NOFLAGS, 'lrem') >+LAND = Opcode(NOFLAGS, 'land') >+LOR = Opcode(NOFLAGS, 'lor') >+LXOR = Opcode(NOFLAGS, 'lxor') >+LSHL = Opcode(NOFLAGS, 'lshl') >+LSHR = Opcode(NOFLAGS, 'lshr') >+LUSHR = Opcode(NOFLAGS, 'lushr') >+ >+# Loading/storing local variables >+LOAD = OpcodeFamily(INTARG, "load") >+STORE = OpcodeFamily(INTARG, "store") >+RETURN = OpcodeFamily(NOFLAGS, "return") >+ >+# Loading/storing from arrays >+# *NOTE*: This family is characterized by the type of the ELEMENT, >+# not the type of the ARRAY. >+# >+# Also: here I break from convention by naming the objects ARRLOAD >+# rather than ALOAD, even though the suffix is 'aload'. This is to >+# avoid confusion with the ALOAD opcode. >+ARRLOAD = OpcodeFamily(NOFLAGS, "aload") >+ARRSTORE = OpcodeFamily(NOFLAGS, "astore") > > # ___________________________________________________________________________ > # Helper Method Information >+# >+# These are used by code outside of this module as well. > > class Method(object): > def __init__(self, classnm, methnm, desc, opcode=INVOKESTATIC): >@@ -95,15 +178,40 @@ > PYPYUINTTODOUBLE = Method('pypy.PyPy', 'uint_to_double', '(I)D') > PYPYDOUBLETOUINT = Method('pypy.PyPy', 'double_to_uint', '(D)I') > PYPYLONGBITWISENEGATE = Method('pypy.PyPy', 'long_bitwise_negate', '(L)L') >+PYPYARRAYTOLIST = Method('pypy.PyPy', 'array_to_list', >+ '([Ljava/lang/Object;)Ljava/util/List;') >+PYPYSTRTOINT = Method('pypy.PyPy', 'str_to_int', >+ '([Ljava/lang/String;)I') >+PYPYSTRTOUINT = Method('pypy.PyPy', 'str_to_uint', >+ '([Ljava/lang/String;)I') >+PYPYSTRTOLONG = Method('pypy.PyPy', 'str_to_long', >+ '([Ljava/lang/String;)J') >+PYPYSTRTOULONG = Method('pypy.PyPy', 'str_to_ulong', >+ '([Ljava/lang/String;)J') >+PYPYSTRTOBOOL = Method('pypy.PyPy', 'str_to_bool', >+ '([Ljava/lang/String;)B') >+PYPYSTRTODOUBLE = Method('pypy.PyPy', 'str_to_double', >+ '([Ljava/lang/String;)D') >+PYPYSTRTOCHAR = Method('pypy.PyPy', 'str_to_char', >+ '([Ljava/lang/String;)C') > > class JVMGenerator(Generator): > > """ Base class for all JVM generators. Invokes a small set of '_' > methods which indicate which opcodes to emit; these can be >- translated by a subclass into Jasmin assembly, binary output, etc.""" >+ translated by a subclass into Jasmin assembly, binary output, etc. >+ Must be inherited from to specify a particular output format; >+ search for the string 'unimplemented' to find the methods that >+ must be overloaded. """ >+ >+ def __init__(self, type_system): >+ self.type_system = type_system > > # __________________________________________________________________ > # JVM specific methods to be overloaded by a subclass >+ # >+ # If the name does not begin with '_', it will be called from >+ # outside the generator. > > def begin_class(self, classnm): > """ >@@ -114,64 +222,213 @@ > def end_class(self): > unimplemented > >- def begin_function(self, funcname, argtypes, static=False): >+ def add_field(self, fname, ftype): >+ """ >+ fname --- name of the field (a string) >+ ftype --- JvmType for the field >+ """ >+ # TODO --- should fdesc be an ootype?? >+ unimplemented >+ >+ def begin_function(self, funcname, argvars, argtypes, rettype, >+ static=False): > """ > funcname --- name of the function >- argtypes --- types of each argument (in what format??) >+ argvars --- list of objects passed to load() that represent arguments; >+ should be in order, or () if load() will not be used >+ argtypes --- JvmType for each argument >+ rettype --- JvmType for the return value > static --- keyword, if true then a static func is generated >+ >+ This function also defines the scope for variables passed to >+ load()/store(). > """ >- unimplemented >+ # Compute the indicates of each argument in the local variables >+ # for the function. Note that some arguments take up two slots >+ # depending on their type [this is compute by type_width()] >+ self.next_offset = 0 >+ self.local_vars = {} >+ for idx, var in enumerate(argvars): >+ self.local_vars[var] = self.next_offset >+ self.next_offset += argtypes[idx].type_width() >+ # Prepare a map for the local variable indices we will add >+ # Let the subclass do the rest of the work; note that it does >+ # not need to know the argvars parameter, so don't pass it >+ self._begin_function(funcname, argtypes, rettype, static) >+ >+ def _begin_function(self, funcname, argtypes, rettype, static): >+ """ >+ Main implementation of begin_function. The begin_function() >+ does some generic handling of args. >+ """ >+ unimplemented > > def end_function(self): >+ del self.next_offset >+ del self.local_vars >+ self._end_function() >+ >+ def _end_function(self): >+ unimplemented >+ >+ def mark(self, lbl): >+ """ Marks the point that a label indicates. """ > unimplemented > >- def _unique_label(self, desc): >+ def _instr(self, opcode, *args): >+ """ Emits an instruction with the given opcode and arguments. >+ The correct opcode and their types depends on the opcode. """ >+ unimplemented >+ >+ def return_val(self, vartype): >+ """ Returns a value from top of stack of the JvmType 'vartype' """ >+ self._instr(RETURN.for_type(vartype)) >+ >+ def load_jvm_var(self, vartype, varidx): >+ """ Loads from jvm slot #varidx, which is expected to hold a value of >+ type vartype """ >+ self._instr(LOAD.for_type(vartype), varidx) >+ >+ def store_jvm_var(self, vartype, varidx): >+ """ Loads from jvm slot #varidx, which is expected to hold a value of >+ type vartype """ >+ self._instr(STORE.for_type(vartype), varidx) >+ >+ def load_from_array(self, elemtype): >+ """ Loads something from an array; the result will be of type 'elemtype' >+ (and hence the array is of type 'array_of(elemtype)'), where >+ 'elemtype' is a JvmType. Assumes that the array ref and index are >+ already pushed onto stack (in that order). """ >+ self._instr(ARRLOAD.for_type(elemtype)) >+ >+ def store_to_array(self, elemtype): >+ """ Stores something into an array; the result will be of type >+ 'elemtype' (and hence the array is of type >+ 'array_of(elemtype)'), where 'elemtype' is a JvmType. Assumes >+ that the array ref, index, and value are already pushed onto >+ stack (in that order).""" >+ self._instr(ARRLOAD.for_type(elemtype)) >+ >+ def unique_label(self, desc, mark=False): > """ Returns an opaque, unique label object that can be passed an >- argument for branching opcodes, or the _mark instruction. >+ argument for branching opcodes, or the mark instruction. > > 'desc' should be a comment describing the use of the label. > It is for decorative purposes only and should be a valid C >- identifier.""" >+ identifier. >+ >+ 'mark' --- if True, then also calls self.mark() with the new lbl """ > labelnum = len(self._labels) > self._labels.append(desc) >- return ('Label', labelnum) >+ res = ('Label', labelnum) >+ if mark: >+ self.mark(res) >+ return res >+ >+ # __________________________________________________________________ >+ # Exception Handling > >- def _mark(self, lbl): >- """ Marks the point that a label indicates. """ >- unimplemented >+ def begin_try(self): >+ """ >+ Begins a try/catch region. Must be followed by a call to end_try() >+ after the code w/in the try region is complete. >+ """ >+ self.begintrylbl = self.unique_label("begin_try", mark=True) > >- def _instr(self, opcode, *args): >- """ Emits an instruction with the given opcode and arguments. >- The correct opcode and their types depends on the opcode. """ >- unimplemented >+ def end_try(self): >+ """ >+ Ends a try/catch region. Must be followed immediately >+ by a call to begin_catch(). >+ """ >+ self.endtrylbl = self.unique_label("end_try", mark=True) >+ >+ def begin_catch(self, excclsty): >+ """ >+ Begins a catch region corresponding to the last try; there can >+ be more than one call to begin_catch, in which case the last >+ try region is reused. >+ 'excclsty' --- a JvmType for the class of exception to be caught >+ """ >+ catchlbl = self.unique_label("catch") >+ self.mark(catchlbl, mark=True) >+ self.try_catch_region( >+ excclsty, self.begintrylbl, send.endtrylbl, catchlbl) > >+ def end_catch(self): >+ """ >+ Ends a catch region. >+ (Included for CLI compatibility) >+ """ >+ return >+ >+ def try_catch_region(self, excclsty, trystartlbl, tryendlbl, catchlbl): >+ """ >+ Indicates a try/catch region: >+ 'excclsty' --- a JvmType for the class of exception to be caught >+ 'trystartlbl', 'tryendlbl' --- labels marking the beginning and end >+ of the try region >+ 'catchlbl' --- label marking beginning of catch region >+ """ >+ unimplemented >+ > # __________________________________________________________________ > # Generator methods and others that are invoked by MicroInstructions > # > # These translate into calls to the above methods. > > def emit(self, instr, *args): >- """ 'instr' in our case must be the name of another method, or >- a JVM opcode (as named above) """ >- >- if hasattr(self, instr): >+ """ 'instr' in our case must be either a string, in which case >+ it is the name of a method to invoke, or an Opcode/Method >+ object (defined above).""" >+ >+ if isinstance(instr, str): > return getattr(self, instr)(*args) >- >- glob = globals() >- if instr in glob: >- val = glob[instr] >- if isinstance(val, Opcode): >- self._instr(glob[opcode], *args) >- else if isinstance(val, Method): >- val.invoke(self) > >- assert False >+ if isinstance(instr, Opcode): >+ return self._instr(instr, *args) >+ >+ if isinstance(instr, Method): >+ return instr.invoke(self) >+ >+ raise Exception("Unknown object in call to emit(): "+repr(instr)) >+ >+ def _var_data(self, v): >+ # Determine java type: >+ jty = self.type_system.ootype_to_jvm(v.concretetype) >+ # Determine index in stack frame slots: >+ # note that arguments and locals can be treated the same here >+ if v in self.local_vars: >+ idx = self.local_vars[v] >+ else: >+ idx = self.local_vars[v] = self.next_offset >+ self.next_offset += jty.type_width() >+ return jty, idx > > def load(self, v): >- unimplemented >+ if isinstance(v, flowmodel.Variable): >+ jty, idx = _var_data(v) >+ return self.load_jvm_var(jty, idx) >+ >+ if isinstance(v, flowmodel.Constant): >+ # TODO: Refactor and complete this code? Maybe more like cli code? >+ if TYPE is ootype.Void: >+ pass >+ elif TYPE is ootype.Bool: >+ self._instr(ICONST, int(value)) >+ elif TYPE is ootype.Char or TYPE is ootype.UniChar: >+ self._instr(ICONST, ord(value)) >+ elif isinstance(value, CDefinedIntSymbolic): >+ self._instr(ICONST, DEFINED_INT_SYMBOLICS[value.expr]) >+ elif TYPE in (ootype.Signed, ootype.Unsigned): >+ self._instr(ICONST, value) # handle Unsigned better! >+ >+ raise Exception('Unexpected type for v in load(): '+v) > > def store(self, v): >- unimplemented >+ if isinstance(v, flowmodel.Variable): >+ jty, idx = _var_data(v) >+ return self.store_jvm_var(jty, idx) >+ raise Exception('Unexpected type for v in store(): '+v) > > def set_field(self, concretetype, value): > self._instr(SETFIELD, concretetype, value) >@@ -185,6 +442,13 @@ > # __________________________________________________________________ > # Methods invoked directly by strings in jvm/opcode.py > >+ def goto(self, lbl): >+ self._instr(GOTO, lbl) >+ >+ def throw(self): >+ """ Throw the object from top of the stack as an exception """ >+ self._instr(ATHROW) >+ > def iabs(self): > MATHIABS.invoke(self) > >@@ -196,6 +460,10 @@ > self._instr(ICONST, -1) > self._instr(IXOR) > >+ def goto_if_true(self, label): >+ """ Jumps if the top of stack is true """ >+ self._instr(IFNE, label) >+ > ##### Comparison methods > > def _compare_op(self, cmpopcode): >@@ -206,14 +474,14 @@ > instruction equals zero]. Consumes as many operands from the > stack as the cmpopcode consumes, typically 1 or 2. > """ >- midlbl = self._unique_label() >- endlbl = self._unique_label() >+ midlbl = self.unique_label('cmpop') >+ endlbl = self.unique_label('cmpop') > self._instr(cmpopcode, midlbl) > self._instr(ICONST, 0) > self._instr(GOTO, endlbl) >- self._mark(midlbl) >+ self.mark(midlbl) > self._instr(ICONST, 1) >- self._mark(endlbl) >+ self.mark(endlbl) > > logical_not = lambda self: self._compare_op(IFEQ) > equals_zero = logical_not >@@ -271,4 +539,72 @@ > ulong_greater_equals = lambda self: self._ulong_compare_op(IFGE) > > class JasminGenerator(JVMGenerator): >- pass >+ >+ def __init__(self, outdir, package): >+ self.outdir = outdir >+ >+ def begin_class(self, classnm): >+ """ >+ classnm --- full Java name of the class (i.e., "java.lang.String") >+ """ >+ >+ iclassnm = classnm.replace('.', '/') >+ jfile = "%s/%s.j" % (self.outdir, iclassnm) >+ >+ try: >+ jdir = jfile[:jfile.rindex('/')] >+ os.makedirs(jdir) >+ except OSError: pass >+ self.out = open(jfile, 'w') >+ >+ # Write the JasminXT header >+ self.out.write(".bytecode XX\n") >+ #self.out.write(".source \n") >+ self.out.write(".class public %s\n" % iclassnm) >+ self.out.write(".super java/lang/Object\n") # ? >+ >+ def end_class(self): >+ self.out.close() >+ self.out = None >+ >+ def add_field(self, fname, fdesc): >+ # TODO --- Signature for generics? >+ # TODO --- these must appear before methods, do we want to buffer >+ # them up to allow out of order calls to add_field()? >+ assert isinstance(fdesc, JvmType) >+ self.out.write('.field public %s %s\n' % (fname, fdesc)) >+ >+ def _begin_function(self, funcname, argtypes, rettype, static): >+ # Throws clause? Only use RuntimeExceptions? >+ kw = ['public'] >+ if static: kw.append('static') >+ self.out.write('.method %s %s (%s)%s\n' % ( >+ funcname, " ".join(kw), >+ "".join(argtypes), rettype)) >+ >+ def _end_function(self): >+ self.out.write('.end method\n') >+ >+ def mark(self, lbl): >+ """ Marks the point that a label indicates. """ >+ _, lblnm = lbl >+ assert _ == "Label" >+ self.out.write(' %s:\n' % lblnm) >+ >+ def _instr(self, opcode, *args): >+ jvmstr = opcode.jvmstr >+ >+ # Hack: this should be somewhere else, just not sure where yet >+ if opcode.flags & CONSTSPEC: >+ if args[0] == -1: >+ jvmstr += "_m1" >+ elif args[0] >= 0 and args[0] <= 5: >+ jvmstr += "_%d" % args[0] >+ >+ self.out.write(' %s %s\n' % ( >+ jvmstr, " ".join([str(s) for s in args]))) >+ >+ def try_catch_region(self, excclsty, trystartlbl, tryendlbl, catchlbl): >+ self.out.write(' .catch %s from %s to %s using %s\n' % ( >+ excclsty.int_class_name(), trystartlbl, tryendlbl, catchlbl)) >+ > >Modified: pypy/dist/pypy/translator/jvm/genjvm.py >============================================================================== >--- pypy/dist/pypy/translator/jvm/genjvm.py (original) >+++ pypy/dist/pypy/translator/jvm/genjvm.py Sun Oct 22 01:36:59 2006 >@@ -11,7 +11,9 @@ > from pypy.translator.jvm.generator import JasminGenerator > from pypy.translator.jvm.option import getoption > from pypy.translator.jvm.database import Database >+from pypy.translator.jvm.typesystem import JvmTypeSystem > from pypy.translator.jvm.log import log >+from pypy.translator.jvm.node import EntryPoint > > class JvmError(Exception): > """ Indicates an error occurred in the JVM runtime """ >@@ -26,9 +28,13 @@ > > For those interested in the location of the files, the following > attributes exist: >- tmpdir --- root directory from which all files can be found >- javadir --- the directory containing *.java >- classdir --- the directory where *.class will be generated >+ tmpdir --- root directory from which all files can be found (py.path obj) >+ javadir --- the directory containing *.java (py.path obj) >+ classdir --- the directory where *.class will be generated (py.path obj) >+ package --- a string with the name of the package (i.e., 'java.util') >+ >+ The following attributes also exist to find the state of the sources: >+ compiled --- True once the sources have been compiled successfully > """ > > def __init__(self, tmpdir, package): >@@ -39,11 +45,12 @@ > """ > self.tmpdir = tmpdir > self.package = package >+ self.compiled = False > > # Compute directory where .java files are > self.javadir = self.tmpdir > for subpkg in package.split('.'): >- self.srcdir = os.path.join(self.srcdir, subpkg) >+ self.javadir = self.javadir.join(subpkg) > > # Compute directory where .class files should go > self.classdir = self.javadir >@@ -51,12 +58,14 @@ > def compile(self): > """ > Compiles the .java sources into .class files, ready for execution. >+ Raises a JvmError if compilation fails. > """ > javac = getoption('javac') >- javafiles = [f for f in os.listdir(self.javadir) >+ javafiles = [f for f in self.javadir.listdir() > if f.endswith('.java')] > res = subprocess.call([javac] + javafiles) > if res: raise JvmError('Failed to compile!') >+ else: self.compiled = True > > def execute(self, args): > """ >@@ -64,6 +73,7 @@ > output as a string. The 'args' are provided as arguments, > and will be converted to strings. > """ >+ assert self.compiled > strargs = [str(a) for a in args] > cmd = [getoption('java'), '%s.Main' % self.package] > pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE).stdout >@@ -81,12 +91,12 @@ > t = TranslationContext() > ann = t.buildannotator() > ann.build_types(func, annotation) >- t.buildrtype(type_system="ootype").specialize() >+ t.buildrtyper(type_system="ootype").specialize() > main_graph = t.graphs[0] > if getoption('view'): t.view() > if getoption('wd'): tmpdir = py.path.local('.') > else: tmpdir = udir >- jvm = GenJvm(tmpdir, t) >+ jvm = GenJvm(tmpdir, t, entrypoint=EntryPoint(main_graph, True)) > return jvm.generate_source() > > class GenJvm(object): >@@ -104,19 +114,18 @@ > 'entrypoint' --- if supplied, an object with a render method > """ > self.jvmsrc = JvmGeneratedSource(tmpdir, getoption('package')) >- self.db = Database() >+ self.type_system = JvmTypeSystem() >+ self.db = Database(self.type_system) > if entrypoint: > self.db.pending_node(entrypoint) >+ else: >+ self.db.pending_node(EntryPoint(translator.graphs[0], False)) > > def generate_source(self): > """ Creates the sources, and returns a JvmGeneratedSource object > for manipulating them """ > generator = self._create_generator() > >- # Deal with entry point >- if not self.db.len_pending(): >- # XXX default entry point >- > # Drain worklist > n = 0 > while self.db.len_pending(): >@@ -129,13 +138,12 @@ > (n, total, n*100.0/total)) > > # Return the source object once we have finished >- generator.all_done() > return self.jvmsrc > > def _create_generator(self): > """ Creates and returns a Generator object according to the > configuration. Right now, however, there is only one kind of > generator: JasminGenerator """ >- return JasminGenerator(self.jvmsrc.javadir) >+ return JasminGenerator(self.jvmsrc.javadir, self.jvmsrc.package) > > > >Modified: pypy/dist/pypy/translator/jvm/node.py >============================================================================== >--- pypy/dist/pypy/translator/jvm/node.py (original) >+++ pypy/dist/pypy/translator/jvm/node.py Sun Oct 22 01:36:59 2006 >@@ -1,3 +1,16 @@ >+""" >+Rendering nodes for the JVM. I suspect that a lot of this could be >+made to be common between CLR and JVM. >+""" >+ >+ >+from pypy.rpython.lltypesystem import lltype >+from pypy.rpython.ootypesystem import ootype >+from pypy.translator.jvm.typesystem import jStringArray, jVoid, jThrowable >+from pypy.translator.jvm.typesystem import jvm_for_class >+import pypy.translator.jvm.generator as jvmgen >+from pypy.translator.jvm.opcodes import opcodes >+ > class Node(object): > def render(self, db, generator): > unimplemented >@@ -10,22 +23,256 @@ > testing (see __init__) > """ > >- def __init__(self, graph): >+ def __init__(self, graph, expandargs): > """ > 'graph' --- The initial graph to invoke from main() >+ 'expandargs' --- controls whether the arguments passed to main() >+ are passed as a list, or expanded to match each argument to the graph >+ >+ The 'expandargs' option deserves explanation: >+ >+ it will be false for a standalone build, because in that >+ case we want to convert the String[] array that main() receives >+ into a corresponding python List of string objects. >+ >+ it will (generally) be true when compiling individual >+ functions, in which case we might be compiling an entry >+ point with a signature like (a:int,b:float) in which case >+ argv[1] should be converted to an integer, and argv[2] >+ should be converted to a float. > """ > self.graph = graph >+ self.expand_arguments = expandargs > pass > >+ # XXX --- perhaps this table would be better placed in typesystem.py >+ # so as to constrain the knowledge of lltype and ootype >+ _type_conversion_methods = { >+ ootype.Signed:jvmgen.PYPYSTRTOINT, >+ ootype.Unsigned:jvmgen.PYPYSTRTOUINT, >+ lltype.SignedLongLong:jvmgen.PYPYSTRTOLONG, >+ lltype.UnsignedLongLong:jvmgen.PYPYSTRTOULONG, >+ ootype.Bool:jvmgen.PYPYSTRTOBOOL, >+ ootype.Float:jvmgen.PYPYSTRTODOUBLE, >+ ootype.Char:jvmgen.PYPYSTRTOCHAR >+ } >+ > def render(self, db, gen): > gen.begin_class('pypy.Main') >- gen.begin_function('main', 'String[]', static=True) >+ gen.begin_function('main', (), [jStringArray], jVoid, static=True) > >- # XXX --- handle arguments somehow! (will probably need some options) >+ # Handle arguments: >+ if self.expand_arguments: >+ # Convert each entry into the array to the desired type by >+ # invoking an appropriate helper function on each one >+ for i, arg in enumerate(self.graph.getargs()): >+ gen.emit(jvmgen.ICONST, i) >+ gen.emit(self._type_conversion_methods[arg.concretetype]) >+ else: >+ # Convert the array of strings to a List as the >+ # python method expects >+ arg0 = self.graph.getargs()[0] >+ assert isinstance(arg0.concretetype, ootype.List), str(arg0.concretetype) >+ assert arg0._ITEMTYPE is ootype.String >+ gen.load_jvm_var(0) >+ gen.emit(jvmgen.PYPYARRAYTOLIST) > > # Generate a call to this method >- db.method_for_graph(self.graph).invoke(gen) >+ gen.emit(db.method_for_graph(self.graph, static=True)) > > gen.end_function() > gen.end_class() > >+class Function(object): >+ >+ """ Represents a function to be emitted. *Note* that it is not a >+ descendant of Node: it cannot be entered into the database >+ worklist. This is because in Java, all functions occur w/in a >+ class: therefore classes as a whole must be placed on the >+ worklist. """ >+ >+ def __init__(self, classobj, name, jargtypes, jrettype, graph, is_static): >+ """ >+ classobj: the Class object this is a part of (even static >+ functions have a class) >+ name: the name of the function >+ jargtypes: JvmType of each argument >+ jrettype: JvmType this function returns >+ graph: the graph representing the body of the function >+ is_static: boolean flag indicate whether func is static (!) >+ """ >+ self.classnm = classnm >+ self.name = name >+ self.graph = graph >+ self.jargtypes = jargtypes >+ self.jrettype = jrettype >+ self.is_static = is_static >+ >+ def method(self): >+ """ Returns a jvmgen.Method that can invoke this function """ >+ if self.is_static: opcode = jvmgen.INVOKESTATIC >+ else: opcode = jvmgen.INVOKEVIRTUAL >+ mdesc = jvm_method_desc(self.jargtypes, self.jrettype) >+ return jvmgen.Method(classnm, self.func_name, mdesc, opcode=opcode) >+ >+ def render_func(self, db, gen): >+ if getattr(self.graph.func, 'suggested_primitive', False): >+ assert False, 'Cannot render a suggested_primitive' >+ >+ # Prepare argument lists for begin_function call >+ jargvars = [] >+ jargtypes = [] >+ for arg in self.graph.getargs(): >+ if arg.concretetype is ootype.Void: continue >+ jargvars.append(arg) >+ jargtypes.append(db.type_system.ootype_to_jvm(arg.concretetype)) >+ >+ # Determine return type >+ jrettype = db.type_system.ootype_to_jvm( >+ self.graph.getreturnvar().concretetype) >+ >+ # Start the function definition >+ gen.begin_function(self.name, jargvars, jargtypes, jrettype, >+ static=self.is_static) >+ >+ # Go through each block and create a label for it; if the >+ # block will be used to catch an exception, add a second label >+ # to catch_labels >+ block_labels = {} >+ #catch_labels = {} >+ for ctr, block in enumerate(graph.iterblocks()): >+ blbl = gen.unique_label('Block_'+ctr) >+ block_labels[block] = blbl >+ >+ ## Go through the blocks we may toss exceptions to >+ #if block.exitswitch == flowmodel.c_last_exception: >+ # for link in block.exits: >+ # if link.exitcase is None: continue # return >+ # if link.target not in catch_labels: >+ # catch_labels[link.target] = gen.unique_label('catch') >+ >+ # Iterate through the blocks and generate code for them >+ return_blocks = [] >+ for block in graph.iterblocks(): >+ >+ # Mark the beginning of the block, render all the ops, and >+ # then mark the end. >+ gen.mark(block_labels[block][0]) >+ >+ # Determine whether the last oper in this block may raise an exc >+ handle_exc = (block.exitswitch == flowmodel.c_last_exception) >+ >+ # Render the operations; create labels for a try/catch >+ # region around the last operation >+ if block.operations: >+ for op in block.operations[:-1]: >+ self._render_op(op) >+ if handle_exc: trybeglbl = gen.unique_label('try', mark=True) >+ self._render_op(block.operations[-1]) >+ if handle_exc: tryendlbl = gen.unique_label('try', mark=True) >+ >+ # Handle 'return' blocks: in this case, we return the >+ # variable specified >+ if self._is_return_block(block): >+ return_var = block.inputargs[0] >+ return_ty = ootype_to_jvm(return_var.concretetype) >+ if return_var.concretetype is not Void: >+ self.load(return_var) >+ gen.return_val(return_ty) >+ >+ # Handle 'raise' blocks: in this case, we just throw the >+ # variable specified >+ if self._is_raise_block(block): >+ exc = block.inputargs[1] >+ self.load(exc) >+ gen.throw() >+ >+ if handle_exc: >+ # search for the "default" block to be executed when >+ # no exception is raised >+ for link in block.exits: >+ if link.exitcase is None: >+ self._copy_link_vars(gen, link) >+ gen.goto(block_labels[link.target]) >+ >+ # TODO: proper exception handling; we may not want to >+ # use the same model as CLR >+ else: >+ # no exception handling, determine correct link to follow >+ for link in block.exits: >+ self._copy_link_vars(gen, link) >+ target_label = block_labels[link.target] >+ if link.exitcase is None or link is block.exits[-1]: >+ gen.goto(target_label) >+ else: >+ assert type(link.exitcase is bool) >+ assert block.exitswitch is not None >+ gen.load(block.exitswitch) >+ gen.goto_if_true(target_label) >+ >+ gen.end_function() >+ >+ def _render_op(self, op): >+ instr_list = opcodes.get(op.opname, None) >+ assert getoption('nostop') or instr_list is not None >+ if instr_list: instr_list.render(self, op) >+ >+ def _copy_link_vars(self, gen, link): >+ target = link.target >+ for to_load, to_store in zip(link.args, target.inputargs): >+ if to_load.concretetype is not Void: >+ gen.load(to_load) >+ gen.store(to_store) >+ >+ def _is_return_block(self, block): >+ return (not block.exits) and len(block.inputargs) == 1 >+ >+ def _is_raise_block(self, block): >+ return (not block.exits) and len(block.inputargs) == 2 >+ >+class Class(Node): >+ >+ """ Represents a class to be emitted. Note that currently, classes >+ are emitted all in one shot, not piecemeal. """ >+ >+ def __init__(self, name): >+ """ >+ 'name' should be a fully qualified Java class name like >+ "java.lang.String" >+ """ >+ self.name = name >+ self.fields = [] >+ self.methods = {} # Maps graph -> Function >+ self.rendered = False >+ >+ def jvm_type(self): >+ return jvm_for_class(self.name) >+ >+ def add_field(self, fieldty, fieldnm): >+ """ Creates a new field in this with type 'fieldty' (a >+ JvmType) and with the name ;fieldnm; (a String). Must be called >+ before render().""" >+ assert not self.rendered >+ self.fields.append((fieldty, fieldnm)) >+ >+ def has_method_for(self, graph): >+ return graph in self.methods >+ >+ def add_method(self, func): >+ """ Creates a new method in this class, represented by the >+ Function object 'func'. Must be called before render(); >+ intended to be invoked by the database.""" >+ assert not self.rendered >+ self.methods[func.graph] = func >+ >+ def render(self, db, gen): >+ self.rendered = True >+ gen.begin_class(self.name) >+ >+ for fieldty, fieldnm in self.fields: >+ gen.add_field(fieldty, fieldnm) >+ >+ for method in self.methods.values(): >+ method.render_func(db, gen) >+ >+ gen.end_class(self.name) > >Modified: pypy/dist/pypy/translator/jvm/opcodes.py >============================================================================== >--- pypy/dist/pypy/translator/jvm/opcodes.py (original) >+++ pypy/dist/pypy/translator/jvm/opcodes.py Sun Oct 22 01:36:59 2006 >@@ -5,41 +5,48 @@ > > """ > >-from pypy.translator.cli.metavm import Call, CallMethod, RuntimeNew, \ >- IndirectCall, GetField, SetField, CastTo, OOString, DownCast, NewCustomDict,\ >- CastWeakAdrToPtr, MapException >-from pypy.translator.oosupport.metavm import PushArg, PushAllArgs, StoreResult, InstructionList,\ >- New >-from pypy.translator.cli.cts import WEAKREF >+from pypy.translator.oosupport.metavm import \ >+ PushArg, PushAllArgs, StoreResult, InstructionList, New, DoNothing >+import pypy.translator.jvm.generator as jvmgen >+ >+def _check_zer(op): >+ # TODO >+ return op >+ >+def _check_ovf(op): >+ # TODO >+ return op > >+# This table maps the opcodes to micro-ops for processing them. >+# It is post-processed by a function to be found below. > opcodes = { > # __________ object oriented operations __________ >- 'new': [New], >- 'runtimenew': [RuntimeNew], >- 'oosetfield': [SetField], >- 'oogetfield': [GetField], >- 'oosend': [CallMethod], >- 'ooupcast': DoNothing, >- 'oodowncast': [DownCast], >- 'oois': 'ceq', >- 'oononnull': [PushAllArgs, 'ldnull', 'ceq']+Not, >- 'instanceof': [CastTo, 'ldnull', 'cgt.un'], >- 'subclassof': [PushAllArgs, 'call bool [pypylib]pypy.runtime.Utils::SubclassOf(class [mscorlib]System.Type, class[mscorlib]System.Type)'], >- 'ooidentityhash': [PushAllArgs, 'callvirt instance int32 object::GetHashCode()'], >- 'oohash': [PushAllArgs, 'callvirt instance int32 object::GetHashCode()'], >- 'oostring': [OOString], >- 'ooparse_int': [PushAllArgs, 'call int32 [pypylib]pypy.runtime.Utils::OOParseInt(string, int32)'], >- 'oonewcustomdict': [NewCustomDict], >- >- 'same_as': DoNothing, >- 'hint': [PushArg(0), StoreResult], >- 'direct_call': [Call], >- 'indirect_call': [IndirectCall], >- >- 'cast_ptr_to_weakadr': [PushAllArgs, 'newobj instance void class %s::.ctor(object)' % WEAKREF], >- 'cast_weakadr_to_ptr': [CastWeakAdrToPtr], >- 'gc__collect': 'call void class [mscorlib]System.GC::Collect()', >- 'resume_point': Ignore, >+ #'new': [New], >+ #'runtimenew': [RuntimeNew], >+ #'oosetfield': [SetField], >+ #'oogetfield': [GetField], >+ #'oosend': [CallMethod], >+ #'ooupcast': DoNothing, >+ #'oodowncast': [DownCast], >+ #'oois': 'ceq', >+ #'oononnull': [PushAllArgs, 'ldnull', 'ceq']+Not, >+ #'instanceof': [CastTo, 'ldnull', 'cgt.un'], >+ #'subclassof': [PushAllArgs, 'call bool [pypylib]pypy.runtime.Utils::SubclassOf(class [mscorlib]System.Type, class[mscorlib]System.Type)'], >+ #'ooidentityhash': [PushAllArgs, 'callvirt instance int32 object::GetHashCode()'], >+ #'oohash': [PushAllArgs, 'callvirt instance int32 object::GetHashCode()'], >+ #'oostring': [OOString], >+ #'ooparse_int': [PushAllArgs, 'call int32 [pypylib]pypy.runtime.Utils::OOParseInt(string, int32)'], >+ #'oonewcustomdict': [NewCustomDict], >+ # >+ #'same_as': DoNothing, >+ #'hint': [PushArg(0), StoreResult], >+ #'direct_call': [Call], >+ #'indirect_call': [IndirectCall], >+ # >+ #'cast_ptr_to_weakadr': [PushAllArgs, 'newobj instance void class %s::.ctor(object)' % WEAKREF], >+ #'cast_weakadr_to_ptr': [CastWeakAdrToPtr], >+ #'gc__collect': 'call void class [mscorlib]System.GC::Collect()', >+ #'resume_point': Ignore, > > # __________ numeric operations __________ > >@@ -56,126 +63,132 @@ > 'unichar_ne': 'not_equals', > > 'int_is_true': 'not_equals_zero', >- 'int_neg': 'INEG', >+ 'int_neg': jvmgen.INEG, > 'int_neg_ovf': None, # How to handle overflow? > 'int_abs': 'iabs', > 'int_abs_ovf': _check_ovf('iabs'), > 'int_invert': 'bitwise_negate', > >- 'int_add': 'IADD', >- 'int_sub': 'ISUB', >- 'int_mul': 'IMUL', >- 'int_floordiv': 'IDIV', >- 'int_floordiv_zer': _check_zer('IDIV'), >- 'int_mod': 'IREM', >+ 'int_add': jvmgen.IADD, >+ 'int_sub': jvmgen.ISUB, >+ 'int_mul': jvmgen.IMUL, >+ 'int_floordiv': jvmgen.IDIV, >+ 'int_floordiv_zer': _check_zer(jvmgen.IDIV), >+ 'int_mod': jvmgen.IREM, > 'int_lt': 'less_than', > 'int_le': 'less_equals', > 'int_eq': 'equals', > 'int_ne': 'not_equals', > 'int_gt': 'greater_than', > 'int_ge': 'greater_equals', >- 'int_and': 'IAND', >- 'int_or': 'IOR', >- 'int_lshift': 'ISHL', >- 'int_rshift': 'ISHR', >- 'int_xor': 'IXOR', >- 'int_add_ovf': _check_ovf('IADD'), >- 'int_sub_ovf': _check_ovf('ISUB'), >- 'int_mul_ovf': _check_ovf('IMUL'), >- 'int_floordiv_ovf': 'IDIV', # these can't overflow! >- 'int_mod_ovf': 'IREM', >+ 'int_and': jvmgen.IAND, >+ 'int_or': jvmgen.IOR, >+ 'int_lshift': jvmgen.ISHL, >+ 'int_rshift': jvmgen.ISHR, >+ 'int_xor': jvmgen.IXOR, >+ 'int_add_ovf': _check_ovf(jvmgen.IADD), >+ 'int_sub_ovf': _check_ovf(jvmgen.ISUB), >+ 'int_mul_ovf': _check_ovf(jvmgen.IMUL), >+ 'int_floordiv_ovf': jvmgen.IDIV, # these can't overflow! >+ 'int_mod_ovf': jvmgen.IREM, > 'int_lt_ovf': 'less_than', > 'int_le_ovf': 'less_equals', > 'int_eq_ovf': 'equals', > 'int_ne_ovf': 'not_equals', > 'int_gt_ovf': 'greater_than', > 'int_ge_ovf': 'greater_equals', >- 'int_and_ovf': 'IAND', >- 'int_or_ovf': 'IOR', >+ 'int_and_ovf': jvmgen.IAND, >+ 'int_or_ovf': jvmgen.IOR, > >- 'int_lshift_ovf': _check_ovf('ISHL'), >- 'int_lshift_ovf_val': _check_ovf('ISHL'), # VAL?? >+ 'int_lshift_ovf': _check_ovf(jvmgen.ISHL), >+ 'int_lshift_ovf_val': _check_ovf(jvmgen.ISHL), # VAL?? > >- 'int_rshift_ovf': 'ISHR', # these can't overflow! >- 'int_xor_ovf': 'IXOR', >- 'int_floordiv_ovf_zer': _check_zer('IDIV'), >- 'int_mod_ovf_zer': _check_zer('IREM'), >+ 'int_rshift_ovf': jvmgen.ISHR, # these can't overflow! >+ 'int_xor_ovf': jvmgen.IXOR, >+ 'int_floordiv_ovf_zer': _check_zer(jvmgen.IDIV), >+ 'int_mod_ovf_zer': _check_zer(jvmgen.IREM), > > 'uint_is_true': 'not_equals_zero', > 'uint_invert': 'bitwise_negate', > >- 'uint_add': 'IADD', >- 'uint_sub': 'ISUB', >- 'uint_mul': 'IMUL', >- 'uint_div': 'IDIV', # valid? >+ 'uint_add': jvmgen.IADD, >+ 'uint_sub': jvmgen.ISUB, >+ 'uint_mul': jvmgen.IMUL, >+ 'uint_div': jvmgen.IDIV, # valid? > 'uint_truediv': None, # TODO >- 'uint_floordiv': 'IDIV', # valid? >- 'uint_mod': 'IREM', # valid? >+ 'uint_floordiv': jvmgen.IDIV, # valid? >+ 'uint_mod': jvmgen.IREM, # valid? > 'uint_lt': 'u_less_than', > 'uint_le': 'u_less_equals', > 'uint_eq': 'u_equals', > 'uint_ne': 'u_not_equals', > 'uint_gt': 'u_greater_than', > 'uint_ge': 'u_greater_equals', >- 'uint_and': 'IAND', >- 'uint_or': 'IOR', >- 'uint_lshift': 'ISHL', >- 'uint_rshift': 'IUSHR', >- 'uint_xor': 'IXOR', >- >- 'float_is_true': [PushAllArgs, 'DCONST_0', 'dbl_not_equals'], >- 'float_neg': 'DNEG', >+ 'uint_and': jvmgen.IAND, >+ 'uint_or': jvmgen.IOR, >+ 'uint_lshift': jvmgen.ISHL, >+ 'uint_rshift': jvmgen.IUSHR, >+ 'uint_xor': jvmgen.IXOR, >+ >+ 'float_is_true': [PushAllArgs, >+ jvmgen.DCONST_0, >+ 'dbl_not_equals'], >+ 'float_neg': jvmgen.DNEG, > 'float_abs': 'dbl_abs', > >- 'float_add': 'DADD', >- 'float_sub': 'DSUB', >- 'float_mul': 'DMUL', >- 'float_truediv': 'DDIV', >- 'float_mod': 'DREM', # use Math.IEEEremainder? >+ 'float_add': jvmgen.DADD, >+ 'float_sub': jvmgen.DSUB, >+ 'float_mul': jvmgen.DMUL, >+ 'float_truediv': jvmgen.DDIV, >+ 'float_mod': jvmgen.DREM, # use Math.IEEEremainder? > 'float_lt': 'dbl_less_than', > 'float_le': 'dbl_less_equals', > 'float_eq': 'dbl_equals', > 'float_ne': 'dbl_not_equals', > 'float_gt': 'dbl_greater_than', > 'float_ge': 'dbl_greater_equals', >- 'float_floor': 'MATHFLOOR', >- 'float_fmod': 'DREM', # DREM is akin to fmod() in C >+ 'float_floor': jvmgen.MATHFLOOR, >+ 'float_fmod': jvmgen.DREM, # DREM is akin to fmod() in C > >- 'llong_is_true': [PushAllArgs, 'LCONST_0', 'long_not_equals'], >- 'llong_neg': 'LNEG', >- 'llong_neg_ovf': _check_ovf('LNEG'), >- 'llong_abs': 'MATHLABS', >- 'llong_invert': 'PYPYLONGBITWISENEGATE', >- >- 'llong_add': 'LADD', >- 'llong_sub': 'LSUB', >- 'llong_mul': 'LMUL', >- 'llong_div': 'LDIV', >+ 'llong_is_true': [PushAllArgs, >+ jvmgen.LCONST_0, >+ 'long_not_equals'], >+ 'llong_neg': jvmgen.LNEG, >+ 'llong_neg_ovf': _check_ovf(jvmgen.LNEG), >+ 'llong_abs': jvmgen.MATHLABS, >+ 'llong_invert': jvmgen.PYPYLONGBITWISENEGATE, >+ >+ 'llong_add': jvmgen.LADD, >+ 'llong_sub': jvmgen.LSUB, >+ 'llong_mul': jvmgen.LMUL, >+ 'llong_div': jvmgen.LDIV, > 'llong_truediv': None, # TODO >- 'llong_floordiv': 'LDIV', >- 'llong_mod': 'LREM', >+ 'llong_floordiv': jvmgen.LDIV, >+ 'llong_mod': jvmgen.LREM, > 'llong_lt': 'long_less_than', > 'llong_le': 'long_less_equals', > 'llong_eq': 'long_equals', > 'llong_ne': 'long_not_equals', > 'llong_gt': 'long_greater_than', > 'llong_ge': 'long_greater_equals', >- 'llong_and': 'LAND', >- 'llong_or': 'LOR', >- 'llong_lshift': 'LSHL', >- 'llong_rshift': 'LSHR', >- 'llong_xor': 'LXOR', >- >- 'ullong_is_true': [PushAllArgs, 'LCONST_0', 'long_not_equals'], >- 'ullong_invert': 'PYPYLONGBITWISENEGATE', >- >- 'ullong_add': 'LADD', >- 'ullong_sub': 'LSUB', >- 'ullong_mul': 'LMUL', >- 'ullong_div': 'LDIV', # valid? >+ 'llong_and': jvmgen.LAND, >+ 'llong_or': jvmgen.LOR, >+ 'llong_lshift': jvmgen.LSHL, >+ 'llong_rshift': jvmgen.LSHR, >+ 'llong_xor': jvmgen.LXOR, >+ >+ 'ullong_is_true': [PushAllArgs, >+ jvmgen.LCONST_0, >+ 'long_not_equals'], >+ 'ullong_invert': jvmgen.PYPYLONGBITWISENEGATE, >+ >+ 'ullong_add': jvmgen.LADD, >+ 'ullong_sub': jvmgen.LSUB, >+ 'ullong_mul': jvmgen.LMUL, >+ 'ullong_div': jvmgen.LDIV, # valid? > 'ullong_truediv': None, # TODO >- 'ullong_floordiv': 'LDIV', # valid? >- 'ullong_mod': 'LREM', # valid? >+ 'ullong_floordiv': jvmgen.LDIV, # valid? >+ 'ullong_mod': jvmgen.LREM, # valid? > 'ullong_lt': 'ulong_less_than', > 'ullong_le': 'ulong_less_equals', > 'ullong_eq': 'ulong_equals', >@@ -189,19 +202,24 @@ > # trick. > 'cast_bool_to_int': DoNothing, > 'cast_bool_to_uint': DoNothing, >- 'cast_bool_to_float': [PushAllArgs, 'not_equals_zero', 'I2D'], >+ 'cast_bool_to_float': [PushAllArgs, 'not_equals_zero', jvmgen.I2D], > > 'cast_char_to_int': DoNothing, > 'cast_unichar_to_int': DoNothing, > 'cast_int_to_char': DoNothing, > 'cast_int_to_unichar': DoNothing, > 'cast_int_to_uint': DoNothing, >- 'cast_int_to_float': 'I2D', >- 'cast_int_to_longlong': 'I2L', >+ 'cast_int_to_float': jvmgen.I2D, >+ 'cast_int_to_longlong': jvmgen.I2L, > 'cast_uint_to_int': DoNothing, >- 'cast_uint_to_float': PYPYUINTTODOUBLE, >- 'cast_float_to_int': 'D2I', >- 'cast_float_to_uint': PYPYDOUBLETOUINT, >- 'truncate_longlong_to_int': 'L2I', >+ 'cast_uint_to_float': jvmgen.PYPYUINTTODOUBLE, >+ 'cast_float_to_int': jvmgen.D2I, >+ 'cast_float_to_uint': jvmgen.PYPYDOUBLETOUINT, >+ 'truncate_longlong_to_int': jvmgen.L2I, > > } >+ >+for opc in opcodes: >+ val = opcodes[opc] >+ if not isinstance(val, list): >+ val = [PushAllArgs, val] > >Added: pypy/dist/pypy/translator/jvm/option.py >============================================================================== >--- (empty file) >+++ pypy/dist/pypy/translator/jvm/option.py Sun Oct 22 01:36:59 2006 >@@ -0,0 +1,4 @@ >+from pypy.translator.jvm.conftest import option >+ >+def getoption(name): >+ return getattr(option, name) > >Modified: pypy/dist/pypy/translator/jvm/src/PyPy.java >============================================================================== >--- pypy/dist/pypy/translator/jvm/src/PyPy.java (original) >+++ pypy/dist/pypy/translator/jvm/src/PyPy.java Sun Oct 22 01:36:59 2006 >@@ -1,12 +1,35 @@ > package pypy; > >+import java.util.List; >+import java.util.ArrayList; >+ >+/** >+ * Class with a number of utility routines. >+ */ > public class PyPy { > >+ /** >+ * Compares two unsigned integers (value1 and value2) and returns >+ * a value greater than, equal to, or less than zero if value 1 is >+ * respectively greater than, equal to, or less than value2. The >+ * idea is that you can do the following: >+ * >+ * Call uint_cmp(value1, value2) >+ * IFLT ... // jumps if value1 < value2 >+ * IFEQ ... // jumps if value1 == value2 >+ * IFGT ... // jumps if value1 > value2 >+ * etc >+ */ > public static int uint_cmp(int value1, int value2) { >- final int VALUE2BIGGER = -1; > final int VALUE1BIGGER = 1; >+ final int VALUE2BIGGER = -1; > final int EQUAL = 0; > >+ if (((value1 | value2) & Integer.MIN_VALUE) == 0) { >+ // neither is negative, presumably the common case >+ return value1 - value2; >+ } >+ > if (value1 == value2) > return EQUAL; > >@@ -23,13 +46,8 @@ > return VALUE1BIGGER; > } > } >- else if (value2 < 0) { >- // value1 is not neg, value2 is neg >- return VALUE2BIGGER; >- } >- >- if (value1 > value2) >- return VALUE1BIGGER; >+ >+ // value1 is not neg, value2 is neg > return VALUE2BIGGER; > } > >@@ -92,4 +110,76 @@ > public static long long_bitwise_negate(long value) { > return ~value; > } >+ >+ public static List array_to_list(Object[] array) { >+ List l = new ArrayList(); >+ for (Object o : array) { >+ l.add(o); >+ } >+ return l; >+ } >+ >+ public static int str_to_int(String s) { >+ try { >+ return Integer.parseInt(s); >+ } catch (NumberFormatException fe) { >+ throw new RuntimeException(fe); >+ } >+ } >+ >+ public static int str_to_uint(String s) { >+ try { >+ long l = Long.parseLong(s); >+ if (l < Integer.MAX_VALUE) >+ return l; >+ int lowerword = l & 0xFFFF; >+ int upperword = l >> 16; >+ return lowerword + (upperword << 16); >+ } catch (NumberFormatException fe) { >+ throw new RuntimeException(fe); >+ } >+ } >+ >+ public static long str_to_long(String s) { >+ try { >+ return Long.parseLong(s); >+ } catch (NumberFormatException fe) { >+ throw new RuntimeException(fe); >+ } >+ } >+ >+ public static long str_to_ulong(String s) { >+ // oh bother >+ throw new RuntimeException("TODO--- str to ulong"); >+ } >+ >+ public static boolean str_to_bool(String s) { >+ // not sure what are considered valid boolean values... >+ // let's be very accepting and take both strings and numbers >+ if (s.equalsIgnoreCase("true")) >+ return true; >+ if (s.equalsIgnoreCase("false")) >+ return false; >+ >+ try { >+ int i = Integer.parseInt(s); >+ return i != 0; >+ } catch (NumberFormatException ex) { >+ throw new RuntimeException(ex); >+ } >+ } >+ >+ public static double str_to_double(String s) { >+ try { >+ return Double.parseDouble(s); >+ } catch (NumberFormatException ex) { >+ throw new RuntimeException(ex); >+ } >+ } >+ >+ public static char str_to_char(String s) { >+ if (s.length() != 1) >+ throw new RuntimeException("String not single character: '"+s+"'"); >+ return s.charAt(0); >+ } > } >\ No newline at end of file > >Added: pypy/dist/pypy/translator/jvm/test/__init__.py >============================================================================== > >Added: pypy/dist/pypy/translator/jvm/test/runtest.py >============================================================================== >--- (empty file) >+++ pypy/dist/pypy/translator/jvm/test/runtest.py Sun Oct 22 01:36:59 2006 >@@ -0,0 +1,119 @@ >+import os >+import platform >+ >+import py >+from py.compat import subprocess >+from pypy.tool.udir import udir >+from pypy.rpython.test.tool import BaseRtypingTest, OORtypeMixin >+from pypy.rpython.lltypesystem.lltype import typeOf >+from pypy.rpython.ootypesystem import ootype >+from pypy.annotation.model import lltype_to_annotation >+from pypy.translator.translator import TranslationContext >+from pypy.translator.jvm.genjvm import generate_source_for_function >+from pypy.translator.jvm.option import getoption >+ >+FLOAT_PRECISION = 8 >+ >+# CLI duplicate >+class StructTuple(tuple): >+ def __getattr__(self, name): >+ if name.startswith('item'): >+ i = int(name[len('item'):]) >+ return self[i] >+ else: >+ raise AttributeError, name >+ >+# CLI duplicate >+class OOList(list): >+ def ll_length(self): >+ return len(self) >+ >+ def ll_getitem_fast(self, i): >+ return self[i] >+ >+# CLI duplicate >+class ExceptionWrapper: >+ def __init__(self, class_name): >+ self.class_name = class_name >+ >+ def __repr__(self): >+ return 'ExceptionWrapper(%s)' % repr(self.class_name) >+ >+# CLI could-be duplicate >+class JvmGeneratedSourceWrapper(object): >+ def __init__(self, gensrc): >+ """ gensrc is an instance of JvmGeneratedSource """ >+ self.gensrc = gensrc >+ >+ def __call__(self, *args): >+ if not self.gensrc.compiled: >+ py.test.skip("Assembly disabled") >+ >+ if getoption('norun'): >+ py.test.skip("Execution disabled") >+ >+ resstr = self.gensrc.execute(args) >+ res = eval(resstr) >+ if isinstance(res, tuple): >+ res = StructTuple(res) # so tests can access tuple elements with .item0, .item1, etc. >+ elif isinstance(res, list): >+ res = OOList(res) >+ return res >+ >+class JvmTest(BaseRtypingTest, OORtypeMixin): >+ def __init__(self): >+ self._func = None >+ self._ann = None >+ self._jvm_src = None >+ >+ def _compile(self, fn, args, ann=None): >+ if ann is None: >+ ann = [lltype_to_annotation(typeOf(x)) for x in args] >+ if self._func is fn and self._ann == ann: >+ return JvmGeneratedSourceWrapper(self._jvm_src) >+ else: >+ self._func = fn >+ self._ann = ann >+ self._jvm_src = generate_source_for_function(fn, ann) >+ if not getoption('noasm'): >+ self._jvm_src.compile() >+ return JvmGeneratedSourceWrapper(self._jvm_src) >+ >+ def _skip_win(self, reason): >+ if platform.system() == 'Windows': >+ py.test.skip('Windows --> %s' % reason) >+ >+ def interpret(self, fn, args, annotation=None): >+ py.test.skip("jvm tests don't work yet") >+ src = self._compile(fn, args, annotation) >+ res = src(*args) >+ if isinstance(res, ExceptionWrapper): >+ raise res >+ return res >+ >+ def interpret_raises(self, exception, fn, args): >+ import exceptions # needed by eval >+ try: >+ self.interpret(fn, args) >+ except ExceptionWrapper, ex: >+ assert issubclass(eval(ex.class_name), exception) >+ else: >+ assert False, 'function did not raise any exception at all' >+ >+ def float_eq(self, x, y): >+ return round(x, FLOAT_PRECISION) == round(y, FLOAT_PRECISION) >+ >+ def ll_to_string(self, s): >+ return s >+ >+ def ll_to_list(self, l): >+ return l >+ >+ def class_name(self, value): >+ return value.class_name.split(".")[-1] >+ >+ def is_of_instance_type(self, val): >+ return isinstance(val, InstanceWrapper) >+ >+ def read_attr(self, obj, name): >+ py.test.skip('read_attr not supported on genjvm tests') > >Added: pypy/dist/pypy/translator/jvm/test/test_bool.py >============================================================================== >--- (empty file) >+++ pypy/dist/pypy/translator/jvm/test/test_bool.py Sun Oct 22 01:36:59 2006 >@@ -0,0 +1,7 @@ >+import py >+from pypy.translator.jvm.test.runtest import JvmTest >+from pypy.rpython.test.test_rbool import BaseTestRbool >+ >+class TestJvmBool(JvmTest, BaseTestRbool): >+ pass >+ > >Added: pypy/dist/pypy/translator/jvm/typesystem.py >============================================================================== >--- (empty file) >+++ pypy/dist/pypy/translator/jvm/typesystem.py Sun Oct 22 01:36:59 2006 >@@ -0,0 +1,157 @@ >+""" >+Translation between PyPy ootypesystem and JVM type system. >+ >+Here are some tentative non-obvious decisions: >+ >+Signed scalar types mostly map as is. >+ >+Unsigned scalar types are a problem; the basic idea is to store them >+as signed values, but execute special code when working with them. Another >+option would be to use classes, or to use the "next larger" type and remember to use appropriate modulos. The jury is out on >+this. Another idea would be to add a variant type system that does >+not have unsigned values, and write the required helper and conversion >+methods in RPython --- then it could be used for multiple backends. >+ >+Python strings are mapped to byte arrays, not Java Strings, since >+Python strings are really sets of bytes, not unicode code points. >+Jury is out on this as well; this is not the approach taken by cli, >+for example. >+ >+Python Unicode strings, on the other hand, map directly to Java Strings. >+ >+WeakRefs can hopefully map to Java Weak References in a straight >+forward fashion. >+ >+Collections can hopefully map to Java collections instances. Note >+that JVM does not have an idea of generic typing at its lowest level >+(well, they do have signature attributes, but those don't really count >+for much). >+ >+""" >+from pypy.rpython.lltypesystem import lltype, llmemory >+from pypy.rpython.ootypesystem import ootype >+from pypy.translator.jvm.option import getoption >+from pypy.translator.jvm.log import log >+ >+class JvmType(str): >+ """ >+ The class we use to represent JVM types; it is just a string with >+ the JVM type descriptor at the moment. Using JvmType allows us to >+ use isinstance, however. The grammar for type descriptors can be >+ read about here: >+ http://java.sun.com/docs/books/vmspec/2nd-edition/html/ClassFile.doc.html >+ """ >+ def is_scalar(self): >+ return self[0] != 'L' and self[0] != '[' >+ def is_reference(self): >+ return not self.is_scalar() >+ def is_array(self): >+ return self[0] == '[' >+ def int_class_name(self): >+ """ Converts a descriptor like Ljava/lang/Object; to >+ java/lang/Object """ >+ assert self[0] == 'L' and self[-1] == ';' >+ return self[1:-1] >+ def type_width(self): >+ """ Returns number of JVM words this type takes up. JVM words >+ are a theoretically abstract quantity that basically >+ represents 32 bits; so most types are 1, but longs and doubles >+ are 2. """ >+ if self[0] == 'J' or self[0] == 'D': >+ return 2 >+ return 1 >+ >+# JVM type functions >+ >+def jvm_array_of(jtype): >+ """ Returns a JvmType representing an array of 'jtype', which must be >+ another JvmType """ >+ assert isinstance(jtype, JvmType) >+ return JvmType('['+str(jtype)) >+ >+def jvm_for_class(classnm): >+ """ Returns a JvmType representing a particular class 'classnm', which >+ should be a fully qualified java class name (i.e., 'java.lang.String') """ >+ return JvmType('L%s;' % classnm.replace('.','/')) >+ >+# Common JVM types >+jVoid = JvmType('V') >+jInt = JvmType('I') >+jLong = JvmType('J') >+jBool = JvmType('Z') >+jDouble = JvmType('D') >+jByte = JvmType('B') >+jByteArray = jvm_array_of(jByte) >+jChar = JvmType('C') >+jThrowable = jvm_for_class('java.lang.Throwable') >+jObject = jvm_for_class('java.lang.Object') >+jString = jvm_for_class('java.lang.String') >+jStringArray = jvm_array_of(jString) >+jArrayList = jvm_for_class('java.util.ArrayList') >+jHashMap = jvm_for_class('java.util.HashMap') >+jIterator = jvm_for_class('java.util.Iterator') >+jClass = jvm_for_class('java.lang.Class') >+jStringBuilder = jvm_for_class('java.lang.StringBuilder') >+ >+# Map from OOType to an internal JVM type descriptor >+_lltype_to_jvm = { >+ ootype.Void: jVoid, >+ ootype.Signed: jInt, >+ ootype.Unsigned: jInt, >+ lltype.SignedLongLong: jLong, >+ lltype.UnsignedLongLong: jLong, >+ ootype.Bool: jBool, >+ ootype.Float: jDouble, >+ ootype.Char: jByte, >+ ootype.UniChar: jChar, >+ ootype.String: jByteArray, >+ ootype.ROOT: jObject, >+ >+ # We may want to use PyPy wrappers here later: >+ llmemory.WeakGcAddress: jObject, # XXX >+ ootype.StringBuilder: jStringBuilder, >+ ootype.Class: jClass, >+ ootype.List: jArrayList, >+ ootype.Dict: jHashMap, >+ ootype.DictItemsIterator:jIterator >+ } >+ >+# Method descriptor construction >+def jvm_method_desc(argtypes, rettype): >+ """ A Java method has a descriptor, which is a string specified >+ its argument and return types. This function converts a list of >+ argument types (JvmTypes) and the return type (also a JvmType), >+ into one of these descriptor strings. """ >+ return "(%s)%s" % ("".join(argtypes), rettype) >+ >+class JvmTypeSystem(object): >+ >+ """ This object translates between the OOTypeSystem and JVM type >+ descriptors. """ >+ >+ def enforce_jvm(self, typ): >+ if isinstance(typ, JvmType): >+ return typ >+ return self.ootype_to_jvm(typ) >+ >+ def ootype_to_jvm(self, oot): >+ """ Returns an instance of JvmType corresponding to the given >+ OOType """ >+ >+ # Check the easy cases >+ if oot in _lltype_to_jvm: >+ return _lltype_to_jvm[oot] >+ >+ # Now handle the harder ones >+ if isinstance(oot, lltype.Ptr) and isinstance(t.TO, lltype.OpaqueType): >+ return jObject >+ if isinstance(oot, ootype.Instance): >+ return XXX >+ if isinstance(oot, ootype.Record): >+ return XXX >+ if isinstance(oot, ootype.StaticMethod): >+ return XXX >+ >+ # Uh-oh >+ unhandled_case >+ > >Modified: pypy/dist/pypy/translator/oosupport/metavm.py >============================================================================== >--- pypy/dist/pypy/translator/oosupport/metavm.py (original) >+++ pypy/dist/pypy/translator/oosupport/metavm.py Sun Oct 22 01:36:59 2006 >@@ -98,6 +98,10 @@ > > def __call__(self, *args): > return self.render(*args) >+ >+class _DoNothing(MicroInstruction): >+ def render(self, generator, op): >+ pass > > class PushArg(MicroInstruction): > """ Pushes a given operand onto the stack. """ >@@ -227,3 +231,4 @@ > SetField = _SetField() > GetField = _GetField() > DownCast = _DownCast() >+DoNothing = _DoNothing() >_______________________________________________ >pypy-svn mailing list >pypy-svn at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-svn > > Ugh. You've broken a lot of stuff by just copying conftest.py from cli. Basically you cannot do that, because both are loaded at the same time (the reason for that is unclear) and options are conflicting. From fijal at genesilico.pl Sun Oct 22 14:38:21 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 22 Oct 2006 14:38:21 +0200 Subject: [pypy-dev] [pypy-svn] r33528 - in pypy/dist/pypy/translator: jvm jvm/src jvm/test oosupport In-Reply-To: <20061021233702.32CB210070@code0.codespeak.net> References: <20061021233702.32CB210070@code0.codespeak.net> Message-ID: <453B663D.3040108@genesilico.pl> As well as some of your commits have broken JS tests. I'm not totally convinced that my attempt (returning void) is better than your (not returning void), but please at least run JS tests and check if nothing is broken. If you brake something, but you're totally convinced that I'm the one who should fix something please at least contact me, I would be happy to sort things out. From fijal at genesilico.pl Mon Oct 23 00:26:04 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Mon, 23 Oct 2006 00:26:04 +0200 Subject: [pypy-dev] ECMAScript and ML Message-ID: <453BEFFC.2020704@genesilico.pl> Reposted from ES4 discussion (about ECMAScript): > We seem to have chosen ML (OCaml with call/cc, probably) as the meta- > language for the ES4 spec. This is hot of the presses, but it looks > like the right choice, and any change that moves away from it will > take unlikely effort and a better candidate language. We were faced > with the arduous task of inventing our own sound meta-language, and > it started looking like ML with customizations. > This means we will have a reference implementation, with all the > software engineering overhead that implies. But it will be an > implementation whose goal is clarity and soundness, not space or time > performance. And with some work, as Cormac Flanagan points out, the > reference code (or probably a subset of it) could be used with Coq to > do automated proofs. > We will make the reference implementation available in due course, as > open source. We will welcome contributors (Hi, Nicolas! ;-). I think it's quite interesting how it influences our attempt to write down JS interpreter. From niko at alum.mit.edu Mon Oct 23 09:49:28 2006 From: niko at alum.mit.edu (Niko Matsakis) Date: Mon, 23 Oct 2006 09:49:28 +0200 Subject: [pypy-dev] [pypy-svn] r33528 - in pypy/dist/pypy/translator: jvm jvm/src jvm/test oosupport In-Reply-To: <453B663D.3040108@genesilico.pl> References: <20061021233702.32CB210070@code0.codespeak.net> <453B663D.3040108@genesilico.pl> Message-ID: <316816B7-1417-4CD0-9259-BFA9ECF0A961@alum.mit.edu> > As well as some of your commits have broken JS tests. I'm not totally > convinced that my attempt (returning void) is better than your (not > returning void), but please at least run JS tests and check if nothing > is broken. If you brake something, but you're totally convinced > that I'm > the one who should fix something please at least contact me, I > would be > happy to sort things out. My sincere apologies! I was trying to be careful not to break things, but evidently not careful enough. I will make sure to run the JS tests in the future --- I didn't in the past because I assumed I did not have the required software. Regarding the change to get_ and set_field to make it ignore Void arguments, I ported that from the CLI code --- I mentioned it on IRC and people thought it was a good idea. I don't really understand too well whether it is necessary or not, but I am afraid that your change might break the CLI tests, as previously they relied upon their own version of the GetField MetaVM op which *did* ignore the Void operations. Again, my apologies for breaking things. :) Niko From fijal at genesilico.pl Mon Oct 23 11:50:01 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Mon, 23 Oct 2006 11:50:01 +0200 Subject: [pypy-dev] [pypy-svn] r33528 - in pypy/dist/pypy/translator: jvm jvm/src jvm/test oosupport In-Reply-To: <316816B7-1417-4CD0-9259-BFA9ECF0A961@alum.mit.edu> References: <20061021233702.32CB210070@code0.codespeak.net> <453B663D.3040108@genesilico.pl> <316816B7-1417-4CD0-9259-BFA9ECF0A961@alum.mit.edu> Message-ID: <453C9049.6050602@genesilico.pl> Niko Matsakis wrote: >> As well as some of your commits have broken JS tests. I'm not totally >> convinced that my attempt (returning void) is better than your (not >> returning void), but please at least run JS tests and check if nothing >> is broken. If you brake something, but you're totally convinced that >> I'm >> the one who should fix something please at least contact me, I would be >> happy to sort things out. > > > My sincere apologies! I was trying to be careful not to break > things, but evidently not careful enough. I will make sure to run > the JS tests in the future --- I didn't in the past because I assumed > I did not have the required software. > > Regarding the change to get_ and set_field to make it ignore Void > arguments, I ported that from the CLI code --- I mentioned it on IRC > and people thought it was a good idea. I don't really understand too > well whether it is necessary or not, but I am afraid that your change > might break the CLI tests, as previously they relied upon their own > version of the GetField MetaVM op which *did* ignore the Void > operations. > > Again, my apologies for breaking things. :) > > > Niko Sorry for being too agressive :-] Actually I do agree with you and I'll make JS working that way. Just please inform me each time you change something which breaks tests explicitely. Cheers, Fijal From cfbolz at gmx.de Mon Oct 23 11:59:00 2006 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Mon, 23 Oct 2006 11:59:00 +0200 Subject: [pypy-dev] [pypy-svn] r33528 - in pypy/dist/pypy/translator: jvm jvm/src jvm/test oosupport In-Reply-To: <453C9049.6050602@genesilico.pl> References: <20061021233702.32CB210070@code0.codespeak.net> <453B663D.3040108@genesilico.pl> <316816B7-1417-4CD0-9259-BFA9ECF0A961@alum.mit.edu> <453C9049.6050602@genesilico.pl> Message-ID: <453C9264.4070200@gmx.de> Maciek Fijalkowski wrote: > Niko Matsakis wrote: > >>> As well as some of your commits have broken JS tests. I'm not totally >>> convinced that my attempt (returning void) is better than your (not >>> returning void), but please at least run JS tests and check if nothing >>> is broken. If you brake something, but you're totally convinced that >>> I'm >>> the one who should fix something please at least contact me, I would be >>> happy to sort things out. >> >> My sincere apologies! I was trying to be careful not to break >> things, but evidently not careful enough. I will make sure to run >> the JS tests in the future --- I didn't in the past because I assumed >> I did not have the required software. >> >> Regarding the change to get_ and set_field to make it ignore Void >> arguments, I ported that from the CLI code --- I mentioned it on IRC >> and people thought it was a good idea. I don't really understand too >> well whether it is necessary or not, but I am afraid that your change >> might break the CLI tests, as previously they relied upon their own >> version of the GetField MetaVM op which *did* ignore the Void >> operations. >> >> Again, my apologies for breaking things. :) I think it happened to everybody at one point. > Sorry for being too agressive :-] Actually I do agree with you and I'll > make JS working that way. Just please inform me each time you change > something which breaks tests explicitely. Well, actually your fix broke the cli tests :-). Cheers, Carl Friedrich From fijal at genesilico.pl Mon Oct 23 12:01:48 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Mon, 23 Oct 2006 12:01:48 +0200 Subject: [pypy-dev] [pypy-svn] r33528 - in pypy/dist/pypy/translator: jvm jvm/src jvm/test oosupport In-Reply-To: <453C9264.4070200@gmx.de> References: <20061021233702.32CB210070@code0.codespeak.net> <453B663D.3040108@genesilico.pl> <316816B7-1417-4CD0-9259-BFA9ECF0A961@alum.mit.edu> <453C9049.6050602@genesilico.pl> <453C9264.4070200@gmx.de> Message-ID: <453C930C.5090009@genesilico.pl> Carl Friedrich Bolz wrote: >Maciek Fijalkowski wrote: > > >>Niko Matsakis wrote: >> >> >> >>>>As well as some of your commits have broken JS tests. I'm not totally >>>>convinced that my attempt (returning void) is better than your (not >>>>returning void), but please at least run JS tests and check if nothing >>>>is broken. If you brake something, but you're totally convinced that >>>>I'm >>>>the one who should fix something please at least contact me, I would be >>>>happy to sort things out. >>>> >>>> >>>My sincere apologies! I was trying to be careful not to break >>>things, but evidently not careful enough. I will make sure to run >>>the JS tests in the future --- I didn't in the past because I assumed >>>I did not have the required software. >>> >>>Regarding the change to get_ and set_field to make it ignore Void >>>arguments, I ported that from the CLI code --- I mentioned it on IRC >>>and people thought it was a good idea. I don't really understand too >>>well whether it is necessary or not, but I am afraid that your change >>>might break the CLI tests, as previously they relied upon their own >>>version of the GetField MetaVM op which *did* ignore the Void >>>operations. >>> >>>Again, my apologies for breaking things. :) >>> >>> > >I think it happened to everybody at one point. > > > > >>Sorry for being too agressive :-] Actually I do agree with you and I'll >>make JS working that way. Just please inform me each time you change >>something which breaks tests explicitely. >> >> > >Well, actually your fix broke the cli tests :-). > >Cheers, > >Carl Friedrich > > Yeah, I know. Didn't know that CLI uses oosupport, I've fixed all by now I guess. My apologies this time ;-) From jeff at taupro.com Tue Oct 31 07:19:14 2006 From: jeff at taupro.com (Jeff Rush) Date: Tue, 31 Oct 2006 00:19:14 -0600 Subject: [pypy-dev] Invitation to Present at PyCon 2007 Message-ID: <4546EAE2.7010005@taupro.com> The deadline for submitting a talk for PyCon 2007, to be held Feb 23-25 in Addison (Dallas), Texas is upon us. I've been looking through the database of submitted talk proposals and I don't see anything related to PyPy. I would like to invite the PyPy community to present on some aspect of their project, as they did at PyCon 2006 and hopefully make some new converts. The deadline for talk submission is midnight Oct 31. To give a talk, go to the following page, create a speaker account and provide a brief proposal. http://us.pycon.org/TX2007/CallForProposals We're also collecting ideas on talks in general on the following wiki page, in case anyone is looking for inspiration. http://us.pycon.org/TX2007/TalkIdeas And there will be presentations on the Parrot VM, with which a comparison against PyPy would be quite interesting. I hope to see many of you in Dallas! Jeff Rush Co-Chair PyCon 2007