From hpk at trillke.net Sun May 1 00:58:36 2005 From: hpk at trillke.net (holger krekel) Date: Sun, 1 May 2005 00:58:36 +0200 Subject: [pypy-dev] latest test results! Message-ID: <20050430225836.GH22996@solar.trillke.net> Hi py-dev, at http://codespeak.net/~hpk/pypy-testresult/ you'll find continously updated test results from running CPython's regression test suite against PyPy. Each time when someone checks in test results from a local test run (see Armins earlier mail on how to do that), the above web page will be updated. Feel free to participate. It would be nice if we would now categorize CPython's tests into e.g. 'language features', 'extension modules' and 'platform specific'. We are obviously more interested to get the language feature regression tests fully working. cheers, holger From arigo at tunes.org Sun May 1 13:55:57 2005 From: arigo at tunes.org (Armin Rigo) Date: Sun, 1 May 2005 12:55:57 +0100 Subject: [pypy-dev] A saner directory structure Message-ID: <20050501115557.GA26559@vicky.ecs.soton.ac.uk> Hi all, Sorry for Yet Another directory structure reorganization. This one shouldn't have too much effects. Its purpose was to clean up the mess of pypy/lib/, which prevented most tests from running at all. Now, we have: dist/pypy/lib/ Used ONLY for complete pure Python reimplementation of modules. The test2 subdirectory just contains regular tests for these. These tests run on top of CPython, and are meant to check that the reimplementations are correct, independently of PyPy. dist/lib-python/2.3.4/ The unmodified CPython library (used to be lib-python-2.3.4). dist/lib-python/modified-2.3.4/ The files and tests that we have modified from the CPython library. PyPy runs with sys.path set to [... lib-python/modified-2.3.4, lib-python/2.3.4, ...] so that the modified-2.3.4 directory shadows the 2.3.4 one. There is a 'test' package in modified-2.3.4; all the files put there shadow the corresponding file from 2.3.4/test. (We use the standard pkgutil module to give the illusion of a 'test' package that contains all files from both modified-2.3.4/test and 2.3.4/test). The place to check out the 'result' directory for tests have been moved to 'dist/testresult'. Just run py.test in lib-python/2.3.4/test and it will tell you more precisely. You might have to remove the old lib-python-2.3.4 directory by hand: svn will leave it behind because you probably have .pyc files in there (and possibly the previous check-out of the 'result' directory). A bient?t, Armin. From jacob at strakt.com Sun May 1 17:02:20 2005 From: jacob at strakt.com (Jacob Hallen) Date: Sun, 1 May 2005 17:02:20 +0200 Subject: [pypy-dev] Pypy talk at Europython Message-ID: <200505011702.20621.jacob@strakt.com> I assume that we will want to have a Pypy presentation at Europython. Official last day for submissions is today, though I will be extending the submission time for a few days due to some technical problems with the submission software, and some technical problems with Strakts Internet connection during the weekend. Who will make the submission? Jacob From hpk at trillke.net Sun May 1 17:14:50 2005 From: hpk at trillke.net (holger krekel) Date: Sun, 1 May 2005 17:14:50 +0200 Subject: [pypy-dev] Pypy talk at Europython In-Reply-To: <200505011702.20621.jacob@strakt.com> References: <200505011702.20621.jacob@strakt.com> Message-ID: <20050501151450.GJ22996@solar.trillke.net> Hi Jacob, On Sun, May 01, 2005 at 17:02 +0200, Jacob Hallen wrote: > I assume that we will want to have a Pypy presentation at Europython. > > Official last day for submissions is today, though I will be extending the > submission time for a few days due to some technical problems with the > submission software, and some technical problems with Strakts Internet > connection during the weekend. > > Who will make the submission? Armin is scheduled to prepare the submission which will be a talk mainly about pypy's evolving low level code generation backends (LLVM and C in particular). Although i am now wondering if we shouldn't also say a few words about the then-to-be-released first PyPy version ... We discussed that at least Armin and Carl, and quite possibly Samuele and Christian (and maybe me with an introduction, talking a few words about the release) will be giving the talk. That probably means we wouldn't mind getting a 60 minute or at least a 45 minute slot. cheers, holger From ac at strakt.com Mon May 2 15:08:50 2005 From: ac at strakt.com (=?ISO-8859-1?Q?Anders_Chrigstr=F6m?=) Date: Mon, 02 May 2005 15:08:50 +0200 Subject: [pypy-dev] new unicode checkins / announcing branches In-Reply-To: <20050430070251.GV22996@solar.trillke.net> References: <20050429220011.A8BBA27BB7@code1.codespeak.net> <20050430070251.GV22996@solar.trillke.net> Message-ID: <42762662.7040208@strakt.com> holger krekel wrote: > i noticed that both of you are doing unicode related > checkins. Could you make sure that you are not > duplicating efforts? Will do. > In general, i think it's a good idea to advertise > new branches (like the nice "non-fake-unicode" one) > on pypy-dev. Better redundant than sorry. I appologize for what confusion my clam-like behaviour has caused. I will be more verbose in the future. The named branch is where I currently develop the unicode type and other unicode related stuff (e.g. the unicodedata module). I plan on having this work complete enough for merging back into the trunk this week or early next week. > Um, and does one of you have a nice nickname? :-) You can use 'Arre' for me. Happy hacking, /Arre From arigo at tunes.org Tue May 3 20:02:40 2005 From: arigo at tunes.org (Armin Rigo) Date: Tue, 3 May 2005 19:02:40 +0100 Subject: [pypy-dev] Builtin-in functions Message-ID: <20050503180240.GA27116@vicky.ecs.soton.ac.uk> Hi all, Builtin-in functions differ from Python function in that you can't bind the former as methods. We are discovering more and more of cases like that: class Stuff: defaultComparison = cmp where self.defaultComparison(x,y) is expected to work. Instead, in PyPy, cmp is then called with three arguments: self, x, y. The problem is the number of cases we keep finding all around. I think that Guido said it was an internal undefined detail, but clearly a lot of people depend on it, so we're stuck. Samuele came up with a nice proposal: the attribute loading code of LazyModule (i.e. our internal modules) is a natural place to turn our functions into some kind of built-in functions. It will turn exactly the function into built-in ones and leave alone, e.g., the methods of class complex -- which need to be boundable to their complex instance! The proposal is thus to wrap functions into a which behaves mostly like the original one, but doesn't have a __get__(). A bientot, Armin From hpk at trillke.net Tue May 3 21:22:49 2005 From: hpk at trillke.net (holger krekel) Date: Tue, 3 May 2005 21:22:49 +0200 Subject: [pypy-dev] Builtin-in functions In-Reply-To: <20050503180240.GA27116@vicky.ecs.soton.ac.uk> References: <20050503180240.GA27116@vicky.ecs.soton.ac.uk> Message-ID: <20050503192249.GF22996@solar.trillke.net> Hi Armin, On Tue, May 03, 2005 at 19:02 +0100, Armin Rigo wrote: > Builtin-in functions differ from Python function in that you can't bind the > former as methods. ... > > Samuele came up with a nice proposal: the attribute loading code of LazyModule > (i.e. our internal modules) is a natural place to turn our functions into some > kind of built-in functions. ... Good idea! > The proposal is thus to wrap functions into a which > behaves mostly like the original one, but doesn't have a __get__(). And i see you already hacked this, nice. For all the others: PyPy is now passing more than 80% of the core python language tests now, see here: http://codespeak.net/~hpk/pypy-testresult/ cheers, holger From hpk at trillke.net Wed May 4 14:55:02 2005 From: hpk at trillke.net (holger krekel) Date: Wed, 4 May 2005 14:55:02 +0200 Subject: [pypy-dev] Re: [pypy-svn] r11817 - in pypy/dist/pypy/lib: . test2 In-Reply-To: <20050502193231.42F4527B84@code1.codespeak.net> References: <20050502193231.42F4527B84@code1.codespeak.net> Message-ID: <20050504125502.GM22996@solar.trillke.net> Hi Anders, also for the sake of others a public comment on your checkin. On Mon, May 02, 2005 at 21:32 +0200, ale at codespeak.net wrote: > Author: ale > Date: Mon May 2 21:32:31 2005 > New Revision: 11817 > > Added: > pypy/dist/pypy/lib/test2/test_codeccallbacks.py This is not the right place to add modified CPython's regression tests anymore. Modified regression tests should go to lib-python/modified-2.3.4/test/ and they should be copied from the non-modified 2.3.4 directory instead of simply adding them, btw. This allows us to trace the history of the modified file much better (a simple 'svn log test_codeccallbacks.py' shows where it came from. The pypy/lib/test2 directory should only contain interpreter level tests that can load our pypy/lib implementations simply like so: from pypy.lib import _codecs # this should just work when # run with test_all.py / # py.test def test_pytest_style(): assert ... # test functionality at CPython-level # to make sure it basically works The advantage of writing tests at this level is that they run fast because they run directly on CPython. You should never copy regression (or unittest-style) tests to lib/test2, though. If you have any questions, don't hesitate to grab any of us on #pypy or here on pypy-dev. cheers, holger From hpk at trillke.net Wed May 4 22:27:48 2005 From: hpk at trillke.net (holger krekel) Date: Wed, 4 May 2005 22:27:48 +0200 Subject: [pypy-dev] importing modules from pypy/lib In-Reply-To: <20050504125502.GM22996@solar.trillke.net> References: <20050502193231.42F4527B84@code1.codespeak.net> <20050504125502.GM22996@solar.trillke.net> Message-ID: <20050504202748.GS22996@solar.trillke.net> On Wed, May 04, 2005 at 14:55 +0200, holger krekel wrote: > from pypy.lib import _codecs # this should just work when > # run with test_all.py / > # py.test This is actually currently not true, but should be true in my opinion. Armin, i see you have done some hack in pypy/lib/test2/support.py to import modules from pypy/lib. Is this still warranted? cheers, holger From arigo at tunes.org Wed May 4 23:04:12 2005 From: arigo at tunes.org (Armin Rigo) Date: Wed, 4 May 2005 22:04:12 +0100 Subject: [pypy-dev] importing modules from pypy/lib In-Reply-To: <20050504202748.GS22996@solar.trillke.net> References: <20050502193231.42F4527B84@code1.codespeak.net> <20050504125502.GM22996@solar.trillke.net> <20050504202748.GS22996@solar.trillke.net> Message-ID: <20050504210412.GA24972@vicky.ecs.soton.ac.uk> Hi Holger, On Wed, May 04, 2005 at 10:27:48PM +0200, holger krekel wrote: > (...) Armin, i see you have done some hack > in pypy/lib/test2/support.py to import modules from > pypy/lib. Is this still warranted? No, you're right. We could 'import pypy.lib.imp' and avoid the problem that 'import imp' always get the built-in module and never imp.py in CPython. Armin From hpk at trillke.net Sat May 7 10:17:36 2005 From: hpk at trillke.net (holger krekel) Date: Sat, 7 May 2005 10:17:36 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12035 - in pypy/dist/pypy: interpreter objspace/std In-Reply-To: <20050506221216.6A04E27B5C@code1.codespeak.net> References: <20050506221216.6A04E27B5C@code1.codespeak.net> Message-ID: <20050507081736.GJ22996@solar.trillke.net> Hi Armin, Hi Samuele, On Sat, May 07, 2005 at 00:12 +0200, arigo at codespeak.net wrote: > Author: arigo > Date: Sat May 7 00:12:16 2005 > New Revision: 12035 > > Modified: > pypy/dist/pypy/interpreter/function.py > pypy/dist/pypy/interpreter/pycode.py > pypy/dist/pypy/objspace/std/dicttype.py > pypy/dist/pypy/objspace/std/fake.py > pypy/dist/pypy/objspace/std/floattype.py > pypy/dist/pypy/objspace/std/inttype.py > pypy/dist/pypy/objspace/std/listtype.py > pypy/dist/pypy/objspace/std/longtype.py > pypy/dist/pypy/objspace/std/objecttype.py > pypy/dist/pypy/objspace/std/slicetype.py > pypy/dist/pypy/objspace/std/stringtype.py > pypy/dist/pypy/objspace/std/tupletype.py > pypy/dist/pypy/objspace/std/typetype.py > Log: > Replace the calls to xyz.__init__() with known calls to > XyzClass.__init__(xyz). Avoids annotator surprizes. When doing such (and other similarly non-obvious) changes could you at least add a few lines in the RPython documentation trying to pin down the "restriction"? cheers, holger From lac at strakt.com Sun May 8 02:06:33 2005 From: lac at strakt.com (Laura Creighton) Date: Sun, 08 May 2005 02:06:33 +0200 Subject: [pypy-dev] oops, thought that chekcings automatically forwarded replied to pypy-dev Message-ID: <200505080006.j4806XlA030216@theraft.strakt.com> See my questions about hpk's checkin laura ------- Forwarded Message In a message of Sat, 07 May 2005 21:36:18 +0200, hpk at codespeak.net writes: >Author: hpk >Date: Sat May 7 21:36:17 2005 >New Revision: 12054 > >Modified: > py/dist/py/test/collect.py >Log: >don't iterate over empty names > > >Modified: py/dist/py/test/collect.py >========================================================================= >===== >--- py/dist/py/test/collect.py (original) >+++ py/dist/py/test/collect.py Sat May 7 21:36:17 2005 >@@ -48,7 +48,7 @@ > Directory = py.test.Config.getvalue('Directory', pkgpath) > current = Directory(pkgpath) > #print "pkgpath", pkgpath >- names = fspath.relto(pkgpath).split(fspath.sep) >+ names = filter(None, fspath.relto(pkgpath).split(fspath.sep)) > for name in names: > current = current.join(name) > assert current, "joining %r resulted in None!" % (names,) >_______________________________________________ >py-svn mailing list >py-svn at codespeak.net >http://codespeak.net/mailman/listinfo/py-svn This seems to fail the 'straightforwardness test' .... any real reason we need filter there, or am i just lost? Laura ------- End of Forwarded Message From arigo at tunes.org Sun May 8 08:02:42 2005 From: arigo at tunes.org (Armin Rigo) Date: Sun, 8 May 2005 08:02:42 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12037 - pypy/dist/lib-python/modified-2.3.4/test In-Reply-To: <20050506223412.85B2827B5C@code1.codespeak.net> References: <20050506223412.85B2827B5C@code1.codespeak.net> Message-ID: <20050508060242.GA29180@code1.codespeak.net> Hi Anders, On Sat, May 07, 2005 at 12:34:12AM +0200, ale at codespeak.net wrote: > Modified: pypy/dist/lib-python/modified-2.3.4/test/test_codeccallbacks.py > ============================================================================== > +import sys, htmlentitydefs, unicodedata > +sys.path.insert(0,r'd:\projects\pypy_co') > +sys.path.insert(0,r'd:\projects\pypy_co\pypy\lib') > +sys.path.insert(0,r'd:\projects\pypy_co\lib-python\modified-2.3.4') > +sys.path.insert(0,r'd:\projects\pypy_co\lib-python\2.3.4') > +from pypy.lib import _codecs > +sys.modules['_codecs'] = _codecs > +from pypy.lib import encodings > +sys.modules['encodings'] = encodings > +from pypy.lib import codecs > +sys.modules['codecs'] = codecs > +reload(encodings) > +reload(codecs) > +assert codecs == encodings.codecs Hard-coded paths? Hum. Sorry for the confusion: although PyPy's own sys.path includes this 'pypy/lib' directory, importing '_codecs' doesn't find the file pypy/lib/_codecs.py because it is special-cased in pypy/module/sys2/state.py. Holger just removed it from there, so now you should be able to just import _codecs normally. In general, importing the pypy package at app-level is likely to make things crash. A few days ago I moved a similar piece of code out of the way because it confused the translation process. Mea culpa; I should have signalled the problem and proposed a solution. (On the other hand, feel free to ask on pypy-dev when you get such issues :-) A bientot, Armin. From arigo at tunes.org Sun May 8 08:24:10 2005 From: arigo at tunes.org (Armin Rigo) Date: Sun, 8 May 2005 08:24:10 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12035 - in pypy/dist/pypy: interpreter objspace/std In-Reply-To: <20050507081736.GJ22996@solar.trillke.net> References: <20050506221216.6A04E27B5C@code1.codespeak.net> <20050507081736.GJ22996@solar.trillke.net> Message-ID: <20050508062410.GB29180@code1.codespeak.net> Hi Holger, On Sat, May 07, 2005 at 10:17:36AM +0200, holger krekel wrote: > > Log: > > Replace the calls to xyz.__init__() with known calls to > > XyzClass.__init__(xyz). Avoids annotator surprizes. > > When doing such (and other similarly non-obvious) changes > could you at least add a few lines in the RPython documentation > trying to pin down the "restriction"? It is difficult to describe in general terms. It is not an RPython restriction, so to say. The log message should have put things in context: it is only above the strange explicit-instance-creation points that we have here and there in PyPy, e.g. in Module.descr_module__new__(): module = space.allocate_instance(Module, w_subtype) module.__init__(space, space.wrap('?'), space.w_None) return space.wrap(module) The first line instantiates the class Module without actually calling __init__(). Its purpose is that if w_subtype is a user subclass of the , allocate_instance() actually returns a fresh instance of UserModule instead. Anyway, the __init__ must be called explicitely. (This is the only place where we call __init__() explicitely.) The problem here, that is clear in hindsight, is that the annotator assumes that 'module' can be an instance of any interp-level subclass of Module -- as it be indeed: UserModule is a (automatically-defined) subclass of Module, so 'module' can be an instance of one of these two classes. However, it cannot be an instance of LazyModule, although LazyModule is also an interp-level subclass of Module. This, the annotator cannot know. The problem is that LazyModule has a different __init__() with a completely different signature. This makes the annotator unhappy (it crashes, complaining that we're calling a method with the wrong number of arguments). Hence the fix (this one done some time ago already) to call instead explicitely the correct method: Module.__init__(module, space, space.wrap('?'), space.w_None) Yesterday's fix was simply to apply the same fix systematically in all places that use space.allocate_instance(), to avoid future problems when more interp-level subclasses are introduced (as we did for BuiltinFunction subclassing Function). RPython is not easy to define simply, but -- more importantly -- it's not easy at all to figure out when we actually followed RPython rules or not, as this case shows. The annotator needs to generate much better error messages than it does currently. A bientot, Armin. From hpk at trillke.net Sun May 8 09:18:31 2005 From: hpk at trillke.net (holger krekel) Date: Sun, 8 May 2005 09:18:31 +0200 Subject: [pypy-dev] oops, thought that chekcings automatically forwarded replied to pypy-dev In-Reply-To: <200505080006.j4806XlA030216@theraft.strakt.com> References: <200505080006.j4806XlA030216@theraft.strakt.com> Message-ID: <20050508071831.GD5609@solar.trillke.net> Hi Laura, On Sun, May 08, 2005 at 02:06 +0200, Laura Creighton wrote: > See my questions about hpk's checkin confusion, confusion :-) Actually the below checkin relates to py-dev not pypy-dev. and to answer your question: filter(None, somelist) is a nice idiom to filter out false arguments such as None or empty strings. Please restart/repost this thread at py-dev if there is a need. cheers, holger > ------- Forwarded Message > > In a message of Sat, 07 May 2005 21:36:18 +0200, hpk at codespeak.net writes: > >Author: hpk > >Date: Sat May 7 21:36:17 2005 > >New Revision: 12054 > > > >Modified: > > py/dist/py/test/collect.py > >Log: > >don't iterate over empty names > > > > > >Modified: py/dist/py/test/collect.py > >========================================================================= > >===== > >--- py/dist/py/test/collect.py (original) > >+++ py/dist/py/test/collect.py Sat May 7 21:36:17 2005 > >@@ -48,7 +48,7 @@ > > Directory = py.test.Config.getvalue('Directory', pkgpath) > > current = Directory(pkgpath) > > #print "pkgpath", pkgpath > >- names = fspath.relto(pkgpath).split(fspath.sep) > >+ names = filter(None, fspath.relto(pkgpath).split(fspath.sep)) > > for name in names: > > current = current.join(name) > > assert current, "joining %r resulted in None!" % (names,) > >_______________________________________________ > >py-svn mailing list > >py-svn at codespeak.net > >http://codespeak.net/mailman/listinfo/py-svn > > This seems to fail the 'straightforwardness test' .... > any real reason we need filter there, or am i just lost? > > Laura > > ------- End of Forwarded Message > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From hpk at trillke.net Sun May 8 09:49:17 2005 From: hpk at trillke.net (holger krekel) Date: Sun, 8 May 2005 09:49:17 +0200 Subject: [pypy-dev] do we still need to generate applevel _exceptions.py? Message-ID: <20050508074917.GE5609@solar.trillke.net> Hi pypy-dev, We have pypy/lib/_exceptions.py which is a generated applevel definition and gets translated to pypy/module/exceptionsinterp.py by Christian fine geninterp mechanism. Now to fix some remaining compliancy bugs it seems easy to fix the applevel exceptions definition unless there is a need to still be able to regenerate that as well. I'd guess the latter is not neccessary anymore so is it fine to just fix pypy/lib/_exceptions.py and regenerate? And what was the incantation again to regenerate the exceptionsinterp.py again (did i miss some piece of documentation somewhere)? cheers, holger From arigo at tunes.org Sun May 8 11:26:59 2005 From: arigo at tunes.org (Armin Rigo) Date: Sun, 8 May 2005 11:26:59 +0200 Subject: [pypy-dev] Compiler Message-ID: <20050508092659.GA30755@code1.codespeak.net> Hi Ludovic, hi all, I had a look at recparser, and how to integrate it into PyPy. Ideally, it can be exported as the 'parser' module by adding a line to interpreter/baseobjspace.py (see the commented-out line about the other 'parser'). A few comments about the interface file pyparser.py (this should be put in some documentation...): * applevel() requires obscure tweaking about the 'import compiler' statement, the prevent the whole compiler package to be dragged in and compiled by PyPy (which may be what we want later, but for now it just doesn't work, I expect). I checked that in. * the interpleveldef exports a class, 'STType'. I added another hack in lazymodule.py to make that work. Basically, the interp-level exports had to be wrapped objects, or functions -- which get wrapped automatically. Types now also get wrapped automatically. Previously, you'd have needed an interpleveldef like 'STType': 'space.gettypeobject(pyparser.STType.typedef)' which fishes the typedef (i.e. the definition of the app-level type) corresponding to the class STType, and asks the space to build a real app-level type object for it. At the moment, with the above changes, it appears to work rather nicely (at least the few exported methods). But we cannot feed the parse tuples to the pure Python compiler package because the latter expect tuples with line number information, and as far as I see you're always generating tuples without. It seems that you're collecting the information already so it should not be difficult to fix. The next step would be to integrate it so that it is used by the built-ins, like compile(). There is a new abstraction, class Compiler, in pypy.interpreter.compiler. Its purpose is to be subclassed by concrete compilers; currently there is only CPythonCompiler, which cheats and calls compile() at interpreter-level. I guess that it should be possible to create another subclass that uses recparser and the pure Python compiler package to do its job, or even a generic PythonCompiler that uses whatever built-in 'parser' module is available, and then the pure Python compiler package. All of PyPy ends up using the compiler instance is stored in the current execution context whenever it needs to compile source code (including at the interactive prompt). Finally, a quick look over the recparser sources shows a few constructs that are clearly not "RPython", i.e. too dynamic. We need to think a bit and see how to address the issue. About RPython: http://codespeak.net/pypy/index.cgi?doc/coding-style.html#restricted-python Before we actually try to perform type inference on recparser, it's a bit hard to know if there are type problems or not. It is often the case that even when we write code knowing that it should be RPython we oversee some subtle typing problem. I'll give it a try, I guess (this is done by enabling the recparser module in baseobjspace as hinted above, running "dist/goal/translate_pypy.py targetpypy", and trying to make sense out of the obscure assertion errors and enormous flow graphs we get...) For now, a problematic feature that is obvious is the visitor pattern that you use extensively. It's definitely a great pattern, but not one that immediately applies to C- or Java-like languages. I'm not saying that you should rewrite all of recparser; more that we need to find a trick to implement visitor patterns without the getattr() with a computed attribute name. Possibly something along these lines: class MyVisitor: def visit_name1(self, node): ... def visit_name2(self, node): ... # this can be computed by a for loop instead: VISIT_MAP = {'name1': visit_name1, 'name2': visit_name2, } class Node: def visit(self, visitor): visit_meth = visitor.VISIT_MAP[self.name] visit_meth(visitor, self) The difference with the getattr() case is that the operation that replaces it, a getitem on a constant dictionary, has a reasonable C-level equivalent, namely a (precomputed) hash table lookup. That's it for now. Don't hesitate to ask if I'm not making sense, or for more help about integration issues. I am aware that it is some kind of guesswork at the moment. Just feel free to post to pypy-dev. A bientot, Armin. From arigo at tunes.org Sun May 8 12:35:08 2005 From: arigo at tunes.org (Armin Rigo) Date: Sun, 8 May 2005 12:35:08 +0200 Subject: [pypy-dev] Re: applevel _exceptions.py Message-ID: <20050508103508.GA31881@code1.codespeak.net> Hi Holger, Christian should know better than me if we'll want to regenerate _exceptions.py in the future. About your second question, I guess that it's time to remove exceptionsinterp.py from svn and use the more general compile-and-cache mecanisms by now, which has the obvious advantage that we don't have to fish for the incantation (which can be found in the svn log)... Armin From mwh at python.net Sun May 8 15:36:30 2005 From: mwh at python.net (Michael Hudson) Date: Sun, 08 May 2005 14:36:30 +0100 Subject: [pypy-dev] Re: [pypy-svn] r12044 - in pypy/dist/pypy/objspace/std: . test In-Reply-To: <20050506231633.DA1F027B5C@code1.codespeak.net> (pedronis@codespeak.net's message of "Sat, 7 May 2005 01:16:33 +0200 (CEST)") References: <20050506231633.DA1F027B5C@code1.codespeak.net> Message-ID: <2mwtq9vnz5.fsf@starship.python.net> pedronis at codespeak.net writes: > + assert sorted(d[0].keys()) == ['__dict__','__metaclass__','__module__'] So we require Python 2.4 now? Cheers, mwh -- The above comment may be extremely inflamatory. For your protection, it has been rot13'd twice. -- the signature of "JWhitlock" on slashdot From pedronis at strakt.com Sun May 8 16:26:36 2005 From: pedronis at strakt.com (Samuele Pedroni) Date: Sun, 08 May 2005 16:26:36 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12044 - in pypy/dist/pypy/objspace/std: . test In-Reply-To: <2mwtq9vnz5.fsf@starship.python.net> References: <20050506231633.DA1F027B5C@code1.codespeak.net> <2mwtq9vnz5.fsf@starship.python.net> Message-ID: <427E219C.4020408@strakt.com> Michael Hudson wrote: > pedronis at codespeak.net writes: > > >>+ assert sorted(d[0].keys()) == ['__dict__','__metaclass__','__module__'] > > > So we require Python 2.4 now? > no, the plan I think is to switch after the first release. That test is a PyPy own app-level test (right now we don't run them on top of CPython and in fact some of them would not pass there), PyPy already defines sorted on top of Python2.3 too: PyPy in StdObjSpace on top of Python 2.3 (startupttime: 1.74 secs) >>>> sorted(['a', 'c', 'b']) ['a', 'b', 'c'] >>>> From mwh at python.net Sun May 8 17:12:42 2005 From: mwh at python.net (Michael Hudson) Date: Sun, 08 May 2005 16:12:42 +0100 Subject: [pypy-dev] Re: [pypy-svn] r12044 - in pypy/dist/pypy/objspace/std: . test References: <20050506231633.DA1F027B5C@code1.codespeak.net> <2mwtq9vnz5.fsf@starship.python.net> <427E219C.4020408@strakt.com> Message-ID: <2msm0xvjit.fsf@starship.python.net> Samuele Pedroni writes: > Michael Hudson wrote: >> pedronis at codespeak.net writes: >> >>>+ assert sorted(d[0].keys()) == ['__dict__','__metaclass__','__module__'] >> So we require Python 2.4 now? >> > > no, the plan I think is to switch after the first release. > > That test is a PyPy own app-level test Duh. Should have spotted that. Sorry for the noise :) Cheers, mwh -- 31. Simplicity does not precede complexity, but follows it. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From tismer at stackless.com Sun May 8 20:35:50 2005 From: tismer at stackless.com (Christian Tismer) Date: Sun, 08 May 2005 20:35:50 +0200 Subject: [pypy-dev] do we still need to generate applevel _exceptions.py? In-Reply-To: <20050508074917.GE5609@solar.trillke.net> References: <20050508074917.GE5609@solar.trillke.net> Message-ID: <427E5C06.8040900@stackless.com> holger krekel wrote: > Hi pypy-dev, > > We have pypy/lib/_exceptions.py which is a generated applevel > definition and gets translated to pypy/module/exceptionsinterp.py > by Christian fine geninterp mechanism. Now to fix some remaining > compliancy bugs it seems easy to fix the applevel exceptions definition > unless there is a need to still be able to regenerate that as well. > I'd guess the latter is not neccessary anymore so is it fine > to just fix pypy/lib/_exceptions.py and regenerate? And what > was the incantation again to regenerate the exceptionsinterp.py > again (did i miss some piece of documentation somewhere)? I think we can continue to generate the exceptions. For migration to future versions, it finds subtle argument additions quite nicely. The only complicancy problems that I see are the __str__ methods. There are just a few to be implemented by hand. A mechanism to supply these methods is inside the generator. As an example there is already one such method implemented, see the end of pypy/tool/_enumerate_exceptions.py About interpleveling the things, I agree with Armin that this should be cache-generated, soon. Will look into that. ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From hpk at trillke.net Sun May 8 22:07:54 2005 From: hpk at trillke.net (holger krekel) Date: Sun, 8 May 2005 22:07:54 +0200 Subject: [pypy-dev] do we still need to generate applevel _exceptions.py? In-Reply-To: <427E5C06.8040900@stackless.com> References: <20050508074917.GE5609@solar.trillke.net> <427E5C06.8040900@stackless.com> Message-ID: <20050508200754.GU5609@solar.trillke.net> Hi Christian, On Sun, May 08, 2005 at 20:35 +0200, Christian Tismer wrote: > holger krekel wrote: > >We have pypy/lib/_exceptions.py which is a generated applevel > >definition and gets translated to pypy/module/exceptionsinterp.py > >by Christian fine geninterp mechanism. Now to fix some remaining > >compliancy bugs it seems easy to fix the applevel exceptions definition > >unless there is a need to still be able to regenerate that as well. > >I'd guess the latter is not neccessary anymore so is it fine > >to just fix pypy/lib/_exceptions.py and regenerate? And what > >was the incantation again to regenerate the exceptionsinterp.py > >again (did i miss some piece of documentation somewhere)? > > I think we can continue to generate the exceptions. For migration > to future versions, it finds subtle argument additions quite nicely. Right. But after we move to 2.4 do you expect many changes to track? > The only complicancy problems that I see are the __str__ methods. > There are just a few to be implemented by hand. > A mechanism to supply these methods is inside the generator. > As an example there is already one such method implemented, > see the end of pypy/tool/_enumerate_exceptions.py thanks, i hope i used it correctly. > About interpleveling the things, I agree with Armin that > this should be cache-generated, soon. Will look into that. yes, that would be great. In addition i wouldn't mind if we allow to modify the __str__ methods directly in the applevel file at least while staying with a minor python version, it's also more straightforward to test via lib/test2 then. thanks & cheers, holger From arigo at tunes.org Sun May 8 23:01:41 2005 From: arigo at tunes.org (Armin Rigo) Date: Sun, 8 May 2005 23:01:41 +0200 Subject: [pypy-dev] Re: applevel _exceptions.py Message-ID: <20050508210141.GA29604@code1.codespeak.net> Hi Holger, > In addition i wouldn't mind if we allow to modify the __str__ methods > directly in the applevel file at least while staying with a minor python > version, it's also more straightforward to test via lib/test2 then. In both case, i.e. whether we write __str__ directly in lib/ or generate them from elsewhere into there, it's possible (and a good idea) to write a test in lib/test2/. A bientot, Armin. From hpk at trillke.net Sun May 8 23:11:07 2005 From: hpk at trillke.net (holger krekel) Date: Sun, 8 May 2005 23:11:07 +0200 Subject: [pypy-dev] Re: applevel _exceptions.py In-Reply-To: <20050508210141.GA29604@code1.codespeak.net> References: <20050508210141.GA29604@code1.codespeak.net> Message-ID: <20050508211107.GW5609@solar.trillke.net> Hi Armin, On Sun, May 08, 2005 at 23:01 +0200, Armin Rigo wrote: > > In addition i wouldn't mind if we allow to modify the __str__ methods > > directly in the applevel file at least while staying with a minor python > > version, it's also more straightforward to test via lib/test2 then. > > In both case, i.e. whether we write __str__ directly in lib/ or generate > them from elsewhere into there, it's possible (and a good idea) to write > a test in lib/test2/. indeed, and i just commited it, forgot it last time. Of course, you have to invoke "tool/_enum*.py" and "mv .. .." each time before you can test a change (unless you are developing in lib/_exceptions.py and move the code to _enum*.py later but while doing that i accidentally destroyed my first impl :-) greetings, holger From pedronis at strakt.com Mon May 9 00:29:19 2005 From: pedronis at strakt.com (Samuele Pedroni) Date: Mon, 09 May 2005 00:29:19 +0200 Subject: [pypy-dev] incomplete builtin modules policy In-Reply-To: <20050508211107.GW5609@solar.trillke.net> References: <20050508210141.GA29604@code1.codespeak.net> <20050508211107.GW5609@solar.trillke.net> Message-ID: <427E92BF.1060901@strakt.com> Hi, we had failures re-appear for core tests because of some inprogress but still incomplete reimplementation of builtin modules got exposed. This is a problem because we can be changing things for example about the object model (objspace/std...) that are more related to the current core goal and we would like as few as possible unrelated failures to make it easy to spot failures produced by related changes and in general to track problems down. As things are now implemented if a module _codecs or binascii is in pypy/lib it will be used even if the corresponding module is listed as needing faking in sys2/state.py, that means pypy/lib takes precedence over faking listing. Given that they cause unrelated failures both binascii and _codecs need likely to be deactived by renaming them. The question now is whether we want a mechanism such that locally it is possible to enable (e.g. even under the renamed name), so to be able to easely do testing for them for people working on them. One possibility is to restore the old model where faking list take precedence and have support for a module that can put in place locally under module/sys2 but is not checked in and that will be imported and can redefine, substract from the faking list. The other possibility is to have such a module defines things like if 'binascii' is requested to be imported use 'inprogress_binascii' instead. Do we need such a feature, what approach would people prefer? regards. From hpk at trillke.net Mon May 9 10:00:13 2005 From: hpk at trillke.net (holger krekel) Date: Mon, 9 May 2005 10:00:13 +0200 Subject: [pypy-dev] incomplete builtin modules policy In-Reply-To: <427E92BF.1060901@strakt.com> References: <20050508210141.GA29604@code1.codespeak.net> <20050508211107.GW5609@solar.trillke.net> <427E92BF.1060901@strakt.com> Message-ID: <20050509080012.GB5609@solar.trillke.net> Hi Samuele, On Mon, May 09, 2005 at 00:29 +0200, Samuele Pedroni wrote: > Hi, we had failures re-appear for core tests because of some inprogress > but still incomplete reimplementation of builtin modules got exposed. > This is a problem because we can be changing things for example about > the object model (objspace/std...) that are more related to the current > core goal and we would like as few as possible unrelated failures to > make it easy to spot failures produced by related changes and in general > to track problems down. > > As things are now implemented if a module _codecs or binascii is in > pypy/lib it will be used even if the corresponding module is listed as > needing faking in sys2/state.py, that means pypy/lib takes precedence > over faking listing. What about adding a '--unstable' switch both to pypy and to py.test which would cause pypy to load modules from pypy/lib/ no matter what and otherwise (by default) would load stable/faked modules as specified in some list or dictionary? Prefixes like 'inprogress_' or 'unstable_' are not that nice because that makes testing from lib/test2 slightly inconvenient although that's not too big a deal, probably. On a side note, it might make sense to generally advise for our pypy/lib reimplementations to convert the according CPython regression tests to the py.test model (with Laura's conversion tool) because they would run fast and nice on CPython level. holger From Ludovic.Aubry at logilab.fr Mon May 9 12:59:01 2005 From: Ludovic.Aubry at logilab.fr (Ludovic Aubry) Date: Mon, 9 May 2005 12:59:01 +0200 Subject: [pypy-dev] Re: Compiler In-Reply-To: <20050508092659.GA30755@code1.codespeak.net> References: <20050508092659.GA30755@code1.codespeak.net> Message-ID: <20050509105901.GB4296@logilab.fr> Hi, On Sun, May 08, 2005 at 11:26:59AM +0200, Armin Rigo wrote: > Hi Ludovic, hi all, > > I had a look at recparser, and how to integrate it into PyPy. Ideally, > it can be exported as the 'parser' module by adding a line to > interpreter/baseobjspace.py (see the commented-out line about the other > 'parser'). Yes we've been experiencing with that already. > A few comments about the interface file pyparser.py (this > should be put in some documentation...): > > * applevel() requires obscure tweaking about the 'import compiler' > statement, the prevent the whole compiler package to be dragged in and > compiled by PyPy (which may be what we want later, but for now it just > doesn't work, I expect). I checked that in. maybe this will solve the problem we're seeing when trying to compile parse trees generated from either parsers. > * the interpleveldef exports a class, 'STType'. I added another hack in > lazymodule.py to make that work. Basically, the interp-level exports > had to be wrapped objects, or functions -- which get wrapped > automatically. Types now also get wrapped automatically. Previously, > you'd have needed an interpleveldef like > > 'STType': 'space.gettypeobject(pyparser.STType.typedef)' > > which fishes the typedef (i.e. the definition of the app-level type) > corresponding to the class STType, and asks the space to build a real > app-level type object for it. ok > At the moment, with the above changes, it appears to work rather nicely > (at least the few exported methods). But we cannot feed the parse > tuples to the pure Python compiler package because the latter expect > tuples with line number information, and as far as I see you're always > generating tuples without. It seems that you're collecting the > information already so it should not be difficult to fix. > > The next step would be to integrate it so that it is used by the > built-ins, like compile(). There is a new abstraction, class Compiler, > in pypy.interpreter.compiler. Its purpose is to be subclassed by > concrete compilers; currently there is only CPythonCompiler, which > cheats and calls compile() at interpreter-level. I guess that it should > be possible to create another subclass that uses recparser and the pure > Python compiler package to do its job, or even a generic PythonCompiler > that uses whatever built-in 'parser' module is available, and then the > pure Python compiler package. > > All of PyPy ends up using the compiler instance is stored in the current > execution context whenever it needs to compile source code (including at > the interactive prompt). > > > Finally, a quick look over the recparser sources shows a few constructs > that are clearly not "RPython", i.e. too dynamic. We need to think a > bit and see how to address the issue. About RPython: > http://codespeak.net/pypy/index.cgi?doc/coding-style.html#restricted-python > > Before we actually try to perform type inference on recparser, it's a > bit hard to know if there are type problems or not. It is often the > case that even when we write code knowing that it should be RPython we > oversee some subtle typing problem. I'll give it a try, I guess (this > is done by enabling the recparser module in baseobjspace as hinted > above, running "dist/goal/translate_pypy.py targetpypy", and trying to > make sense out of the obscure assertion errors and enormous flow graphs > we get...) > For now, a problematic feature that is obvious is the visitor pattern > that you use extensively. It's definitely a great pattern, but not one > that immediately applies to C- or Java-like languages. I'm not saying > that you should rewrite all of recparser; more that we need to find a > trick to implement visitor patterns without the getattr() with a > computed attribute name. Possibly something along these lines: > > class MyVisitor: > def visit_name1(self, node): > ... > def visit_name2(self, node): > ... > > # this can be computed by a for loop instead: > VISIT_MAP = {'name1': visit_name1, > 'name2': visit_name2, > } > > class Node: > def visit(self, visitor): > visit_meth = visitor.VISIT_MAP[self.name] > visit_meth(visitor, self) > > The difference with the getattr() case is that the operation that > replaces it, a getitem on a constant dictionary, has a reasonable > C-level equivalent, namely a (precomputed) hash table lookup. sure, I discussed that with Hoelger already, thing is the visitor isn't used for parsing but only by the EBNFParser which parses the python grammar file and turn it into a tree of grammar object This should be called only at startup time. I must say I am not sure whether the following call in recparser/__init__.py: PYTHON_PARSER = pythonutil.python_grammar() really is called at bootstrap time ? anyway, at this time PYTHON_PARSER is a static tree of objects representing the grammar and for now the parsing is done by providing a 'builder' object to the match method of the tree (in fact there are several subtrees, one for each grammar targets) > > That's it for now. Don't hesitate to ask if I'm not making sense, or > for more help about integration issues. I am aware that it is some kind > of guesswork at the moment. Just feel free to post to pypy-dev. > > > A bientot, > > Armin. > -- Ludovic Aubry LOGILAB, Paris (France). http://www.logilab.com http://www.logilab.fr http://www.logilab.org From arigo at tunes.org Mon May 9 16:32:30 2005 From: arigo at tunes.org (Armin Rigo) Date: Mon, 9 May 2005 16:32:30 +0200 Subject: [pypy-dev] Re: Compiler In-Reply-To: <20050509105901.GB4296@logilab.fr> References: <20050508092659.GA30755@code1.codespeak.net> <20050509105901.GB4296@logilab.fr> Message-ID: <20050509143230.GA24376@code1.codespeak.net> Hi Ludovic, (Replying here for reference, repeating our IRC conversation) On Mon, May 09, 2005 at 12:59:01PM +0200, Ludovic Aubry wrote: > > The difference with the getattr() case is that the operation that > > replaces it, a getitem on a constant dictionary, has a reasonable > > C-level equivalent, namely a (precomputed) hash table lookup. > sure, I discussed that with Hoelger already, thing is the visitor isn't > used for parsing but only by the EBNFParser which parses the python > grammar file and turn it into a tree of grammar object > This should be called only at startup time. Then there is no need to change this source, indeed. > I must say I am not sure whether the following call in recparser/__init__.py: > PYTHON_PARSER = pythonutil.python_grammar() > really is called at bootstrap time ? Yes, "bootstrap time" is actually a long time. All .py modules are imported at bootstrap time. Their 'def' and 'class' statements are all executed at bootstrap time -- defining new functions and classes is forbidden in RPython. It means that you can actually put any strange full Python code in the global "top-level" code of any module. The above line can go in the globals of the most convenient module, not only __init__.py. A bientot, Armin. From arigo at tunes.org Mon May 9 22:00:41 2005 From: arigo at tunes.org (Armin Rigo) Date: Mon, 9 May 2005 22:00:41 +0200 Subject: [pypy-dev] Just so you know... Message-ID: <20050509200041.GA28031@code1.codespeak.net> Hi all, Samuele and me have been working on a better approach for GenC, something less hackish. It seems that we have a nice model by now, but it's still a bit experimental. The description of this approach requires more than I want to write down right now; we're starting a motivation / documentation / TODO, in the following folder: http://codespeak.net/svn/user/pedronis/llrpy/ The prototype is in ll.py, the text in overview.txt. The idea is that we'll send a more useful e-mail to pypy-dev once overview.txt gets completed and expanded. In the meantime you can try to make sense out of that if you feel like it :-) (Note that it's not Yet Another Completely New paradigm; there are a lot of ideas that have already been floating around. But it seems that we are starting to put these ideas into a consistent whole now. Also, the approach should be able to reuse a lot of the infrastructure that we already have now -- flow graph, annotator, etc.: it's unlikely that we will need yet another large subdirectory of pypy :-) A bientot, Samuele & Armin From tismer at stackless.com Tue May 10 04:22:33 2005 From: tismer at stackless.com (Christian Tismer) Date: Tue, 10 May 2005 04:22:33 +0200 Subject: [pypy-dev] Re: applevel _exceptions.py In-Reply-To: <20050508211107.GW5609@solar.trillke.net> References: <20050508210141.GA29604@code1.codespeak.net> <20050508211107.GW5609@solar.trillke.net> Message-ID: <42801AE9.9040003@stackless.com> holger krekel wrote: ... > indeed, and i just commited it, forgot it last time. Of course, you > have to invoke "tool/_enum*.py" and "mv .. .." each time before you can > test a change (unless you are developing in lib/_exceptions.py and move > the code to _enum*.py later but while doing that i accidentally > destroyed my first impl :-) That's of course a very bad problem with generated source, into which I've stepped several times... ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Tue May 10 04:32:19 2005 From: tismer at stackless.com (Christian Tismer) Date: Tue, 10 May 2005 04:32:19 +0200 Subject: [pypy-dev] Magic problems Message-ID: <42801D33.7060108@stackless.com> Hi, during my wrestling with the ovfcheck "macro", I came along a couple of problems, which are unrelated but nevertheless noteworthy. I discovered, that when testall.py ../*dunno*/test_geninterp.py is run, the generated code does not expose any exception branches!!! This is probably due to the fact that some magic module makes all exceptions into AssertionError subclasses. But these are removed by simplify.py ! I was unable to follow the magic logic of this, but the pygame view was at least able to show me that we really have a problem here. You can try this by writing a small function call, which usually should expose at least one simple exception outlet. It won't, but becomes a straight sequence of simple blocks. Nice, anyway, but unfortunately Python world is not that nice :-) This problem seems to be local to geninterp.py, which since a while tries to completely believe into __builtin__ objects as god given. Maybe I should modify this and look into tjhe exceptions module more closely, or is this solvable from the testing environment? ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Tue May 10 05:39:33 2005 From: tismer at stackless.com (Christian Tismer) Date: Tue, 10 May 2005 05:39:33 +0200 Subject: [pypy-dev] Just so you know... In-Reply-To: <20050509200041.GA28031@code1.codespeak.net> References: <20050509200041.GA28031@code1.codespeak.net> Message-ID: <42802CF5.80006@stackless.com> Armin Rigo wrote: > Hi all, > > Samuele and me have been working on a better approach for GenC, > something less hackish. It seems that we have a nice model by now, but > it's still a bit experimental. The description of this approach > requires more than I want to write down right now; we're starting a > motivation / documentation / TODO, in the following folder: > > http://codespeak.net/svn/user/pedronis/llrpy/ > > The prototype is in ll.py, the text in overview.txt. The idea is that > we'll send a more useful e-mail to pypy-dev once overview.txt gets > completed and expanded. In the meantime you can try to make sense out > of that if you feel like it :-) It makes very much sense, probably more than you expected. One thing that I'd like to mention again is rpystone.py . I added a few conditionals to wipe out the current problems with string and record objects. The result is rather amazing: From the former performance ratio of 1:3, we now get to a ratio of 1:20 !!! In the first place, this made me very happy. Then, I looked into the assembly and did some calculations, which made me less happy. We are not really doing good! The variable movement confuses the optimizer. ANd, at the end, most of it is not needed at all. At the moment, PyPy is roughly at a 1:2000 performance penalty. With proper structures, I think we will be able to keep the 1:20 performance improvement for every internal structure. But this still leaves a rest of a factor of 100 to us! It appearsto be a fact, that our current strategy of moving all state around across links confuses the MS compiler. I am going to write a c_rpython implementation to get more close results, but as a matter of fact, we need a better model. From experience, I have a rough estimate that we easily could achieve a factor of 50, if we were able to remove lots of variables which are carried around. But finally, we all know that even this wouldn't help us to break the big 1:2000 barrier in short time. Today I haven't written more than 3 lines of code, but worked quite a lot by brain storming, and I have come up with a new structure that will apply both to genc and geninterplevel, and I guess every other compiler back-end will be happy with it. I have invented ScopedGraphs, which are expressed as subclasses of FlowGraphs. The exact implementation is on my boilerplate. The very nice thing about ScopedGraph is: There is a 1:1 translation possible forth and back between FlowGraph and ScopedGraph representations of programs. The ScopedGraph is similar to FlowGraph, but it involves extra read-only variables which are introduced via local scope. That greatly reduces the size of the local state to be carried over the links, because all these read-only variables don't need to be shifted around. The basic idea is: A ScopedGraph is a nested structure of ScopedGraphs. It has similarities with the flowgraph of a function, since there is a single entry, involving the inputargs, and there is either a single exit or also an exceptional exit. The basic idea in advance to FlowGraph is to introduce a scope of read-only variables, that exists for all blocks inside the Scope object. The scope object itself is responsible for creation and destruction of the involved read-only variables. The transition from FlowGraph to ScopedGraph is made by a simple life-time analysis of variables, followed by a heuristic optimization pass. Related blocks are placed near to each other, sharing their local read-only variables in their common scope. This is not a unique mapping, but a reversible one. You can find many ScopedGraphs for a given FlowGraph, but optimization is available to choose one that minimizes the necessary state to be carried around. As a side effect, you get related data arranged into groups of blocks that are related to each other, and you get much smaller clean-up code for every block, because blocks are borrowing their container's state, and the container is responsiblefor the clean-ups. A nice side-effect of ScopedGraphs is that a ScopedGraph alone can easily be turned into a stand-alone function. A function is not much more, just the ability to be passed around as a first-class object is new. Without that, every function shows up as a ScopedGraph. This relationship doesn't go vice-versa, but it holds for over 90 % of what we have in PyPy. Side note: Sure, FlowGraph's blocks can be turned into functions as well. But this is a bad idea, because way too much state is carried around. In a sense, the FlowGraph idea is similar to the continuations with which I have coped with for so many years. After all, I have considered them to be a bad idea in places where they are not really needed. And this was the basic idea which led me into ScopedGraphs: FlowGraphs are very very capable, but they could support real continuations. That makes them over-sized for real problems. ScopedGraphs are bound to their local scope, and this matches the style in which we implemented PyPy most precisely. And this is finally how I think to cope with the other factor of 100 that we cannot handle just by generating efficient code: I think that by transforming functions into ScopedGraphs, we can do a lot of shuffling: - Expanding code by not calling certain functions but inlining their ScopedGraph, which *increases* code size, - Collapsing code by removing common sub-ScopedGraphs and turning them into stand-alone functions, which *decreases* code size. - in general, macrofying and unmacrofying code, in terms of ScopedGraphs I hope that part of this will help to solve the remaining 1:100 dilemma of our great Python abstraction, which turns out to be twice a number of magnitude slower as it should be. A ScopedGraph based implementation of geninterplevel.py will be available in less than two weeks. ciao -- chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From arigo at tunes.org Tue May 10 11:57:47 2005 From: arigo at tunes.org (Armin Rigo) Date: Tue, 10 May 2005 11:57:47 +0200 Subject: [pypy-dev] Magic problems In-Reply-To: <42801D33.7060108@stackless.com> References: <42801D33.7060108@stackless.com> Message-ID: <20050510095747.GA32603@code1.codespeak.net> Hi Christian, On Tue, May 10, 2005 at 04:32:19AM +0200, Christian Tismer wrote: > I discovered, that when testall.py ../*dunno*/test_geninterp.py > is run, the generated code does not expose any exception > branches!!! Not sure what you are referring to here. test_geninterp.py contains tests that use try:except:, and check that they are really called. Althought it was broken in the past few days (at least at one point it failed because it was out of sync with your new simpliciations), it works fine now. > This is probably due to the fact that some magic module > makes all exceptions into AssertionError subclasses. We do strange magic things with AssertionError, but definitely not that :-) > You can try this by writing a small function call, which usually > should expose at least one simple exception outlet. > It won't, but becomes a straight sequence of simple blocks. I think that what you are referring to is that a function like this: def f(x): g(x) h(x) will have a flow graph which is just a sequence of simple_call's, without any exceptional branch. It has always been so: the two calling operations, simple_call and call_args, are always allowed to raise any exception. If there is no explicit handler in the flow graph it just means that the exception will pass through the current function to its parent unhandled, unmodified. We are, in other words, keeping our graphs simpler by avoiding to stick a "catch-anything" branch pointing to a "re-raise-it-directly" exit after every simple_call and call_args. Anyway, if the target language supports exceptions natively (e.g. as in geninterp.py), you don't want this catch-and-reraise step everywhere. Of course, for the examples below you get the expected "catch-all" flow graph exception handler: def f(x): def f(x): try: try: g(x) g(x) except: finally: ... ... A bientot, Armin. From tismer at stackless.com Tue May 10 20:29:15 2005 From: tismer at stackless.com (Christian Tismer) Date: Tue, 10 May 2005 20:29:15 +0200 Subject: [pypy-dev] Magic problems In-Reply-To: <20050510095747.GA32603@code1.codespeak.net> References: <42801D33.7060108@stackless.com> <20050510095747.GA32603@code1.codespeak.net> Message-ID: <4280FD7B.3070107@stackless.com> Armin Rigo wrote: > Hi Christian, > > On Tue, May 10, 2005 at 04:32:19AM +0200, Christian Tismer wrote: > >>I discovered, that when testall.py ../*dunno*/test_geninterp.py >>is run, the generated code does not expose any exception >>branches!!! > > > Not sure what you are referring to here. test_geninterp.py contains > tests that use try:except:, and check that they are really called. > Althought it was broken in the past few days (at least at one point it > failed because it was out of sync with your new simpliciations), it > works fine now. Many thanks! Yes, I tricked myself by looking into the flowgraph of ovfcheck_lshift (debugging the "syntax error" case) and wondered why there areno exceptional branches left. good to hear ! :-) ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From hpk at trillke.net Thu May 12 22:15:14 2005 From: hpk at trillke.net (holger krekel) Date: Thu, 12 May 2005 22:15:14 +0200 Subject: [pypy-dev] europython sprints/upcoming activities Message-ID: <20050512201514.GA5609@solar.trillke.net> Hi, i have been asked by Erik to post some preliminary information on the upcoming EuroPython sprint(s). I guess we'll make some more complete announcement soon but meanwhile here is some brief information on upcoming activities in the next two months: - 15th-20th May IRC-coordinated session ending in a first PyPy release (targeting the goal of having an interpreter that passes 90% of CPython's core language tests but still runs on top of CPython). - 23rd-26th June: Pre-EuroPython sprint for people who already attended sprints previously (IOW this is not the best time for newcomers to join) - 27th-29th June: EuroPython! - 30th June: a day of break (really) - 1st-7th July: Post-EuroPython sprint open for anyone interested in PyPy development. It's likely there will be a strong focus on translation and low level backends. - 8th July: PyPy/EU-Project evaluation workshop The phyiscal meetings will all take place in Goetheborg (Sweden) where this year's EuroPython happens. Sprint locations, meeting points etc.pp. are to be announced soon. cheers, holger From Ben.Young at risk.sungard.com Fri May 13 11:19:42 2005 From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com) Date: Fri, 13 May 2005 10:19:42 +0100 Subject: [pypy-dev] (no subject) Message-ID: Hi there, I have been following the development of pypy and have a couple of questions about the recent development. I was wondering what the exact reasoning behind the new low level code. Is it just there to replace the code templates that are in genc? or is there a deeper reason? Already the code seems much more complicated than the code templates (and seems to contain its own templates in rlist.p for example). Will the new code be easily transferrable to other backends (e.g llvm or java) or is it tied to the c stuff? Sorry if these are silly questions, I am just trying to get a better understanding of why all this stuff is changing when you seemed so close to getting more goals finished. Keep up the good work! Cheers, Ben From arigo at tunes.org Fri May 13 12:55:03 2005 From: arigo at tunes.org (Armin Rigo) Date: Fri, 13 May 2005 12:55:03 +0200 Subject: [pypy-dev] (no subject) In-Reply-To: References: Message-ID: <20050513105503.GA32288@code1.codespeak.net> Hi Ben, On Fri, May 13, 2005 at 10:19:42AM +0100, Ben.Young at risk.sungard.com wrote: > I was wondering what the exact reasoning behind the new low level code. I am aware that more information about this would be useful... The primary reason is that the code templates in GenC have more or less been pushed to their limits. These templates are by far not the first attempt; GenC has known a number of iterations that all ended up in the trash because they were becoming too messy. The current GenC was more or less stalled in the middle of implementing lists and instances. The new approach feels for the first time like a clean solution. > Is it just there to replace the code templates that are in genc? or is there > a deeper reason? There is actually a deeper reason (more below)... > Already the code seems much more complicated than the > code templates (and seems to contain its own templates in rlist.p for > example). Will the new code be easily transferrable to other backends (e.g > llvm or java) or is it tied to the c stuff? Yes, the purpose is that all back-ends can implement only the low-level operations, and don't have to worry about the lists and dicts and instances of RPython. The code has become more complicated in the sense that it depends on other parts of PyPy again -- like the annotator. By itself, it is rather a kind of simplification. Note that rlist.py contains not templates but plain functions, which are annotated and translated like the rest of the input program. There are definitely obscure corners in rlist.py and rpython/typer.py, e.g. to implement variable-length operations like newlist. We've focused on these dark corners at the moment; writing a complete implementation of the list operations in rlist.py should definitely make it clearer. The "deeper purpose" is that templates of C code are only useful to generate C code -- so far it's obvious. But they are also limited to generating "normal" C code; you can't tweak them automatically to generate different kinds of C code. For example, if a C template contains a for loop, the generated code will contain the same for loop. But if we want, say, to generate C code that uses Stackless-like continuation-passing, then we can't write C for loops directly; we have to tweak them a bit. This kind of tweaking is easy if the input is a flow graph containing low-level operations only, but you can't tweak C code templates. (Unless, that is, you parse the C code somehow; but that's the kind of discussion hinted at in the overview.txt file.) > Sorry if these are silly questions, I am just trying to get a better > understanding of why all this stuff is changing when you seemed so close > to getting more goals finished. You're welcome! I hope this helped to clear up some points. I know that generating a C version of PyPy is the most concrete and most visible short-term goal of PyPy. It's also our most important goal at the moment. But when it is reasonable to do so, we're trying to keep the longer-term goals in mind; they are, hopefully, the reason for most of the indirections we appear to take... A bientot, Armin From eric at vanrietpaap.nl Fri May 13 19:15:47 2005 From: eric at vanrietpaap.nl (Eric van Riet Paap) Date: Fri, 13 May 2005 19:15:47 +0200 Subject: [pypy-dev] Sprint (talks) movies Message-ID: <4284E0C3.5060304@vanrietpaap.nl> Hello, Are any off the talks about PyPy recorded? I seem to remember they were, but I can't find any reference of that. I would very much like to view them. If there are no objections, I would also like to host them on my server and create a torrent for them. best regards Eric van Riet Paap From arigo at tunes.org Fri May 13 22:26:13 2005 From: arigo at tunes.org (Armin Rigo) Date: Fri, 13 May 2005 22:26:13 +0200 Subject: [pypy-dev] Just so you know... In-Reply-To: <20050509200041.GA28031@code1.codespeak.net> References: <20050509200041.GA28031@code1.codespeak.net> Message-ID: <20050513202613.GA6402@code1.codespeak.net> Hi, Here is some (unreviewed) documentation about the new LowLevelType approach: http://codespeak.net/pypy/index.cgi?doc/translation.html#the-rpython-typer Armin From cfbolz at gmx.de Sat May 14 01:13:23 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sat, 14 May 2005 01:13:23 +0200 Subject: [pypy-dev] Just so you know... In-Reply-To: <20050513202613.GA6402@code1.codespeak.net> References: <20050509200041.GA28031@code1.codespeak.net> <20050513202613.GA6402@code1.codespeak.net> Message-ID: <42853493.2060903@gmx.de> Hi Armin! > Here is some (unreviewed) documentation about the new LowLevelType > approach: > > http://codespeak.net/pypy/index.cgi?doc/translation.html#the-rpython-typer > Just some nitpicking/asking for clarification/remarks: From translation.html: When a block has been transformed in this way, all the links are considered; if the concrete types of the Variables that exit do not match the canonical low-level types expected by the target block, conversions are inserted. "Inserted" meaning that explicit cast operations are put append to the operations of the basic block? If that's the case you have to be careful in the presence of exception blocks because afterwards the cast is the last operation in the block and will be checked for exceptions. I had to do the same in genllvm and inserted "Cast Blocks" along the links, which contain only casting operations. And a more general note: I think that the whole low level/rpython typer approach makes a lot of sense. I implemented some of the things that are now redone by the new approach. So I was really doing too much work since some of the things I did would have to be done for every (remotely C-ish) backend which is obviously not a good solution. In addition all the templating hacks I did started to get brittle -- I had to add more and more code that did string manipulations on the template I used for the implementation of RPython's lists in LLVM. Regards, Carl Friedrich From tismer at stackless.com Sat May 14 03:18:48 2005 From: tismer at stackless.com (Christian Tismer) Date: Sat, 14 May 2005 03:18:48 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12253 - pypy/dist/pypy/tool In-Reply-To: <20050514001917.7C8ED27B8E@code1.codespeak.net> References: <20050514001917.7C8ED27B8E@code1.codespeak.net> Message-ID: <428551F8.80707@stackless.com> ale at codespeak.net wrote: > Author: ale > Date: Sat May 14 02:19:17 2005 > New Revision: 12253 > > Modified: > pypy/dist/pypy/tool/_enum_exceptions.py > Log: > added arguments checking. > > There are checks for correct number of arguments and correct types (for the case I understand an exact nuber of arguments > > I haven't run any test - it is too late for that. Will do in the morning Looks well done! After all, looking at this stuff again, I still like the approach, although it is limited in a way that you write a program that actually tries to catch stuff that you know already. Originally, I planned to catch all the __str__ formatting methods by probing them, too. But this seems to be very hard, especially in the only relevant case of unicode. I'd like to keep the code alive. Perhaps we can find an application for other cases. Especially the __init__ tracking seems to be working very well and could be extended. ciao -- chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From hpk at trillke.net Sat May 14 17:48:06 2005 From: hpk at trillke.net (holger krekel) Date: Sat, 14 May 2005 17:48:06 +0200 Subject: [pypy-dev] polishing / pypy-dev tracker Message-ID: <20050514154806.GL5609@solar.trillke.net> Hi pypy-dev, two notes: - Is everybody who is interested aware that we want to meet at 11am tomorrow (sunday, 15th May) for the PyPy release-polishing session? - i am finalizing & testing the new PyPy development tracker and am interested in comments and people trying it out a bit. It's currently located here: https://codespeak.net/issue/pypy-dev/ almost all people that ever commited to the PyPy code base should be able to use their normal codespeak login (svn/shell/pypy-eu/pypy-dev logins are all unified for PyPy). One of the new tracker features is that it's connected to svn checkins (based on Richard Jones's new integration code). If you commit to svn/pypy/dist with a a log message like: issue40 resolved This fixes a long standing issues blablabla then this message will show up in the tracker, get send to subscribers and will mark the issue as resolved. Another feature is that each user can edit a field 'notify on new issues' in his 'Details' and will then receive notification email on each newly created issue in that tracker (even if you are not on the nosy list). I am still randomly re-initializing the tracker database for testing purposes. I'll send a public note when the new tracker goes into production. Comments already welcome, though. cheers & see you, holger From tjreedy at udel.edu Sat May 14 21:33:14 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 14 May 2005 15:33:14 -0400 Subject: [pypy-dev] Re: Just so you know... References: <20050509200041.GA28031@code1.codespeak.net> <20050513202613.GA6402@code1.codespeak.net> Message-ID: "Armin Rigo" wrote in message news:20050513202613.GA6402 at code1.codespeak.net... > Hi, > > Here is some (unreviewed) documentation about the new LowLevelType > approach: > > http://codespeak.net/pypy/index.cgi?doc/translation.html#the-rpython-typer Just so you know, my U.S. IE browser has every section number followed by three careted (hatted) As: A^ as one character. Below is an example, but I am not sure how it will appear to others. 5.2.3? ? ? List representation TJR From hpk at trillke.net Sat May 14 21:45:38 2005 From: hpk at trillke.net (holger krekel) Date: Sat, 14 May 2005 21:45:38 +0200 Subject: [pypy-dev] Re: Just so you know... In-Reply-To: References: <20050509200041.GA28031@code1.codespeak.net> <20050513202613.GA6402@code1.codespeak.net> Message-ID: <20050514194538.GN5609@solar.trillke.net> Hi Terry, On Sat, May 14, 2005 at 15:33 -0400, Terry Reedy wrote: > "Armin Rigo" wrote in message > news:20050513202613.GA6402 at code1.codespeak.net... > > Hi, > > > > Here is some (unreviewed) documentation about the new LowLevelType > > approach: > > > > http://codespeak.net/pypy/index.cgi?doc/translation.html#the-rpython-typer > > Just so you know, my U.S. IE browser has every section number followed by > three careted (hatted) As: A^ as one character. Below is an example, but I > am not sure how it will appear to others. > 5.2.3? ? ? List representation the pages were using a non-standard encoding declaration. I think i just fixed it, does it work better for you now? holger From arigo at tunes.org Sat May 14 21:51:04 2005 From: arigo at tunes.org (Armin Rigo) Date: Sat, 14 May 2005 21:51:04 +0200 Subject: [pypy-dev] Just so you know... In-Reply-To: <42853493.2060903@gmx.de> References: <20050509200041.GA28031@code1.codespeak.net> <20050513202613.GA6402@code1.codespeak.net> <42853493.2060903@gmx.de> Message-ID: <20050514195104.GA27353@code1.codespeak.net> Hi Carl, On Sat, May 14, 2005 at 01:13:23AM +0200, Carl Friedrich Bolz wrote: > When a block has been transformed in this way, all the links are > considered; if the concrete types of the Variables that exit do not > match the canonical low-level types expected by the target block, > conversions are inserted. > > "Inserted" meaning that explicit cast operations are put append to the > operations of the basic block? No, there is a new block inserted along the link; the conversions are only interesting for a single exit of the block, the other exits may need different ones. (This idea was stolen from genllvm -- I remembered you described it to me before I wrote the translator/typer.py.) > And a more general note: I think that the whole low level/rpython typer > approach makes a lot of sense. I implemented some of the things that are > now redone by the new approach. So I was really doing too much work > since some of the things I did would have to be done for every (remotely > C-ish) backend which is obviously not a good solution. In addition all > the templating hacks I did started to get brittle -- I had to add more > and more code that did string manipulations on the template I used for > the implementation of RPython's lists in LLVM. Good to hear. Yes, I was quite afraid we were about to run into exactly the same troubles in genc. BTW genllvm is a nice source for inspiration which we're using freely to reorganize genc now :-) A bientot, Armin. From arigo at tunes.org Sat May 14 22:01:01 2005 From: arigo at tunes.org (Armin Rigo) Date: Sat, 14 May 2005 22:01:01 +0200 Subject: [pypy-dev] polishing / pypy-dev tracker In-Reply-To: <20050514154806.GL5609@solar.trillke.net> References: <20050514154806.GL5609@solar.trillke.net> Message-ID: <20050514200101.GB27353@code1.codespeak.net> Hi Holger, On Sat, May 14, 2005 at 05:48:06PM +0200, holger krekel wrote: > - Is everybody who is interested aware that we want to meet at > 11am tomorrow (sunday, 15th May) for the PyPy > release-polishing session? Yup! > - i am finalizing & testing the new PyPy development tracker > and am interested in comments and people trying it out > a bit. It's currently located here: > > https://codespeak.net/issue/pypy-dev/ I guess I'll need some real data to judge, but it looks bad in links and pretty unusable in Netscape 4.74. I guess you can't easily do something about it, and I should go forward and install a recent browser and be patient every time I want to start it... > One of the new tracker features is that it's connected to svn > checkins (based on Richard Jones's new integration code). > If you commit to svn/pypy/dist with a a log message like: > > issue40 resolved > > This fixes a long standing issues blablabla This is great! What are exactly the rules? A bientot, Armin. From hpk at trillke.net Sat May 14 22:32:48 2005 From: hpk at trillke.net (holger krekel) Date: Sat, 14 May 2005 22:32:48 +0200 Subject: [pypy-dev] polishing / pypy-dev tracker In-Reply-To: <20050514200101.GB27353@code1.codespeak.net> References: <20050514154806.GL5609@solar.trillke.net> <20050514200101.GB27353@code1.codespeak.net> Message-ID: <20050514203248.GO5609@solar.trillke.net> Hi Armin, On Sat, May 14, 2005 at 22:01 +0200, Armin Rigo wrote: > On Sat, May 14, 2005 at 05:48:06PM +0200, holger krekel wrote: > > - i am finalizing & testing the new PyPy development tracker > > and am interested in comments and people trying it out > > a bit. It's currently located here: > > > > https://codespeak.net/issue/pypy-dev/ > > I guess I'll need some real data to judge, but it looks bad in links and > pretty unusable in Netscape 4.74. What or who is Netscape? :-) > I guess you can't easily do something > about it, and I should go forward and install a recent browser and be > patient every time I want to start it... I'd like to try to make it reasonably work for links (and also for netscape if feasible). I guess it's the use of CSS which links doesn't recognize at all ASFAIK. But what is the main problem you are seeing? (I just tried links and it works as i would expect it given that it doesn't look at CSS. but i don't use it much, so i don't know what you would expect). > > One of the new tracker features is that it's connected to svn > > checkins (based on Richard Jones's new integration code). > > If you commit to svn/pypy/dist with a a log message like: > > > > issue40 resolved > > > > This fixes a long standing issues blablabla > > This is great! What are exactly the rules? - the content "issueNN STATUS" has to be on a single otherwise empty line - NN must be an existing issue number - STATUS is one of the possible status values: unread chatting need-eg (jargon for "need example") in-progress testing done-cbb (jargon for 'done, could-be-better') resolved I suggest we start working with the tracker tomorrow (for organizing the release polishing tasks) and see what we want to change about the tracker in the coming week. One open question is if we want to use 'keywords' aka 'topics'. One or more of them can be associated with each issue. We could use the directory names in dist/pypy as initial keywords/topic names so that it easy to correlate certain implementation parts to an issue. It also would allow us to define private or shared queries such as "show me all issues relating to 'interpreter'". cheers, holger From cfbolz at gmx.de Sat May 14 23:43:47 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sat, 14 May 2005 23:43:47 +0200 Subject: [pypy-dev] polishing / pypy-dev tracker In-Reply-To: <20050514200101.GB27353@code1.codespeak.net> References: <20050514154806.GL5609@solar.trillke.net> <20050514200101.GB27353@code1.codespeak.net> Message-ID: <42867113.9080509@gmx.de> Hi Armin, Armin Rigo wrote: [snip] > I guess I'll need some real data to judge, but it looks bad in links and > pretty unusable in Netscape 4.74. I guess you can't easily do something > about it, and I should go forward and install a recent browser and be > patient every time I want to start it... You could try dillo (www.dillo.org), should be reasonably fast, though I don't know how experimental their https support is. Regards, Carl Friedrich From tjreedy at udel.edu Sun May 15 02:39:44 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 14 May 2005 20:39:44 -0400 Subject: [pypy-dev] Re: Re: Just so you know... References: <20050509200041.GA28031@code1.codespeak.net><20050513202613.GA6402@code1.codespeak.net> <20050514194538.GN5609@solar.trillke.net> Message-ID: Fixed. From mwh at python.net Sun May 15 14:28:47 2005 From: mwh at python.net (Michael Hudson) Date: Sun, 15 May 2005 13:28:47 +0100 Subject: [pypy-dev] Re: polishing / pypy-dev tracker References: <20050514154806.GL5609@solar.trillke.net> <20050514200101.GB27353@code1.codespeak.net> Message-ID: <2macmw65c0.fsf@starship.python.net> Armin Rigo writes: > I guess I'll need some real data to judge, but it looks bad in links and > pretty unusable in Netscape 4.74. I guess you can't easily do something > about it, and I should go forward and install a recent browser and be > patient every time I want to start it... If you're used to Netscape 4, you won't be that disappointed with modern browsers' startup times... Cheers, mwh -- It's actually a corruption of "starling". They used to be carried. Since they weighed a full pound (hence the name), they had to be carried by two starlings in tandem, with a line between them. -- Alan J Rosenthal explains "Pounds Sterling" on asr From arigo at tunes.org Sun May 15 15:00:06 2005 From: arigo at tunes.org (Armin Rigo) Date: Sun, 15 May 2005 15:00:06 +0200 Subject: [pypy-dev] Re: polishing / pypy-dev tracker In-Reply-To: <2macmw65c0.fsf@starship.python.net> References: <20050514154806.GL5609@solar.trillke.net> <20050514200101.GB27353@code1.codespeak.net> <2macmw65c0.fsf@starship.python.net> Message-ID: <20050515130006.GA6652@code1.codespeak.net> Hi Michael, On Sun, May 15, 2005 at 01:28:47PM +0100, Michael Hudson wrote: > If you're used to Netscape 4, you won't be that disappointed with > modern browsers' startup times... Ah, thanks. Actually I'm not used to Netscape 4 but to links ( <1 sec), only using Netscape for whatever links is really not happy with. Armin From tismer at stackless.com Mon May 16 00:09:10 2005 From: tismer at stackless.com (Christian Tismer) Date: Mon, 16 May 2005 00:09:10 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12320 - pypy/dist/pypy/documentation In-Reply-To: <20050515175758.90AB727BC2@code1.codespeak.net> References: <20050515175758.90AB727BC2@code1.codespeak.net> Message-ID: <4287C886.1070606@stackless.com> hpk at codespeak.net wrote: > Author: hpk > Date: Sun May 15 19:57:58 2005 > New Revision: 12320 > > Modified: > pypy/dist/pypy/documentation/getting_started.txt > Log: > i am unsure what to do about things > such as t.view() which start a full gui. > > commenting t.view() out for now > to stop the tests opening pygame-windows Oops! > >>> t = Translator(test.is_perfect_number) > - >>> t.view() > + >>> #t.view() You mean for testing the embedded example? Well, but without the t.view(), the rest of the text makes no sense, too. Maybe we should put a flag somewhere else that certain things should be deactivated, at least in some batch testing we would not want any interactivity I guess. ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From lac at strakt.com Tue May 17 14:04:09 2005 From: lac at strakt.com (Laura Creighton) Date: Tue, 17 May 2005 14:04:09 +0200 Subject: [pypy-dev] PyPy issue tracker Message-ID: <200505171204.j4HC49mZ007007@ratthing-b246.strakt.com> When I went to https://codespeak.net/issue/pypy-dev/ and searched for issue 66 expecting the mail that holger just sent, I instead got a 404 not found. It's looking for Message: /pypy-dev/issue66 not https://codespeak.net/issue/pypy-eu/issue66 is this correct? Laura From arigo at tunes.org Tue May 17 14:09:21 2005 From: arigo at tunes.org (Armin Rigo) Date: Tue, 17 May 2005 14:09:21 +0200 Subject: [pypy-dev] PyPy issue tracker In-Reply-To: <200505171204.j4HC49mZ007007@ratthing-b246.strakt.com> References: <200505171204.j4HC49mZ007007@ratthing-b246.strakt.com> Message-ID: <20050517120921.GA16875@code1.codespeak.net> Hi Laura, On Tue, May 17, 2005 at 02:04:09PM +0200, Laura Creighton wrote: > When I went to https://codespeak.net/issue/pypy-dev/ > and searched for issue 66 expecting the mail that holger just sent, > I instead got a 404 not found. This issue 66 is in the EU tracker. The above URL in the development tracker. They are kept separate, so issue numbers are unrelated (and overlap). The confusion probably comes from the fact that Holger mentioned the dev-tracker in this EU-tracked issue :-) Armin From lac at strakt.com Tue May 17 14:14:15 2005 From: lac at strakt.com (Laura Creighton) Date: Tue, 17 May 2005 14:14:15 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12394 - pypy/dist/pypy/documentation In-Reply-To: Message from hpk@codespeak.net of "Tue, 17 May 2005 00:34:26 +0200." <20050516223426.F05F627BB9@code1.codespeak.net> References: <20050516223426.F05F627BB9@code1.codespeak.net> Message-ID: <200505171214.j4HCEFYS007066@ratthing-b246.strakt.com> In a message of Tue, 17 May 2005 00:34:26 +0200, hpk at codespeak.net writes: >Author: hpk >Date: Tue May 17 00:34:26 2005 >New Revision: 12394 > >Modified: > pypy/dist/pypy/documentation/getting_started.txt >Log: >issue37 in-progress > >introduced the 0.6 release and according explanations into the getting >started document. Basically i try to make downloading/unpacking the >upcoming 0.6 release an alternative to using an 'svn co' line. >Also i tried to avoid all the duplicate information we have everywhere. > >Somehow i don't like the very detailed subversion steps anymore. People >who don't have subversion are likely to just use the tar/zip files and >the others could be pointed to some official subversion documentation. >So I think we should further refactor the getting started document >accordingly and remove subversion details. This part of the documentation >stems from a time where almost nobody had subversion installed except us >:-) Instead of just removing it, can you make it into its own page and give a link to it. The problem is that our documentation on how to install subversion is _better_ than the one at the subversion site (unless that has changed recently). Every time I go someplace that doesn't have subversion, I go to their site, try their instructions, have it _fail_, remember, oh, yes, that's right and then go back to your instructions on how to get subversion which works. I can see the point of simplifying, but if other people have as much trouble as I do installing subversion from sourcen on a site that doesn't have a package manager, they may need our instructions. Laura From hpk at trillke.net Tue May 17 14:19:47 2005 From: hpk at trillke.net (holger krekel) Date: Tue, 17 May 2005 14:19:47 +0200 Subject: [pypy-dev] PyPy issue tracker In-Reply-To: <200505171204.j4HC49mZ007007@ratthing-b246.strakt.com> References: <200505171204.j4HC49mZ007007@ratthing-b246.strakt.com> Message-ID: <20050517121947.GV5609@solar.trillke.net> Hi Laura, On Tue, May 17, 2005 at 14:04 +0200, Laura Creighton wrote: > When I went to https://codespeak.net/issue/pypy-dev/ > and searched for issue 66 expecting the mail that holger just sent, > I instead got a 404 not found. It's looking for > Message: /pypy-dev/issue66 not https://codespeak.net/issue/pypy-eu/issue66 > > is this correct? there are two trackers: the development tracker and the EU tracker. You cannot mix the issue numbers across these two trackers, i.e. you need to look which tracker an issue belongs to. cheers, holger From cfbolz at gmx.de Tue May 17 15:23:54 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 17 May 2005 15:23:54 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12394 - pypy/dist/pypy/documentation In-Reply-To: <200505171214.j4HCEFYS007066@ratthing-b246.strakt.com> References: <20050516223426.F05F627BB9@code1.codespeak.net> <200505171214.j4HCEFYS007066@ratthing-b246.strakt.com> Message-ID: <4289F06A.4030903@gmx.de> > Instead of just removing it, can you make it into its own page and > give a link to it. I did exactly that. See http://codespeak.net/pypy/index.cgi?doc/svn-help.html Carl Friedrich From lac at strakt.com Tue May 17 17:03:53 2005 From: lac at strakt.com (Laura Creighton) Date: Tue, 17 May 2005 17:03:53 +0200 Subject: [pypy-dev] PyPy issue tracker In-Reply-To: Message from hpk@trillke.net (holger krekel) of "Tue, 17 May 2005 14:19:47 +0200." <20050517121947.GV5609@solar.trillke.net> References: <200505171204.j4HC49mZ007007@ratthing-b246.strakt.com> <20050517121947.GV5609@solar.trillke.net> Message-ID: <200505171503.j4HF3rPB009893@ratthing-b246.strakt.com> In a message of Tue, 17 May 2005 14:19:47 +0200, holger krekel writes: >Hi Laura, > >On Tue, May 17, 2005 at 14:04 +0200, Laura Creighton wrote: >> When I went to https://codespeak.net/issue/pypy-dev/ >> and searched for issue 66 expecting the mail that holger just sent, >> I instead got a 404 not found. It's looking for >> Message: /pypy-dev/issue66 not https://codespeak.net/issue/pypy-eu/issu >e66 >> >> is this correct? > >there are two trackers: the development tracker and the EU tracker. >You cannot mix the issue numbers across these two trackers, i.e. >you need to look which tracker an issue belongs to. > >cheers, > > holger Do we have a page where each tracker is one click away? Lazily yours, Laura From hpk at trillke.net Tue May 17 17:05:00 2005 From: hpk at trillke.net (holger krekel) Date: Tue, 17 May 2005 17:05:00 +0200 Subject: [pypy-dev] PyPy issue tracker In-Reply-To: <200505171503.j4HF3rPB009893@ratthing-b246.strakt.com> References: <200505171204.j4HC49mZ007007@ratthing-b246.strakt.com> <20050517121947.GV5609@solar.trillke.net> <200505171503.j4HF3rPB009893@ratthing-b246.strakt.com> Message-ID: <20050517150500.GA5609@solar.trillke.net> > >there are two trackers: the development tracker and the EU tracker. > >You cannot mix the issue numbers across these two trackers, i.e. > >you need to look which tracker an issue belongs to. > > > >cheers, > > > > holger > > Do we have a page where each tracker is one click away? the "contact" navigation item on the main web page or http://codespeak.net/pypy/index.cgi?contact cheers holger From lac at strakt.com Tue May 17 17:11:49 2005 From: lac at strakt.com (Laura Creighton) Date: Tue, 17 May 2005 17:11:49 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12394 - pypy/dist/pypy/documentation In-Reply-To: Message from Carl Friedrich Bolz of "Tue, 17 May 2005 15:23:54 +0200." <4289F06A.4030903@gmx.de> References: <20050516223426.F05F627BB9@code1.codespeak.net> <200505171214.j4HCEFYS007066@ratthing-b246.strakt.com> <4289F06A.4030903@gmx.de> Message-ID: <200505171511.j4HFBnAL009997@ratthing-b246.strakt.com> In a message of Tue, 17 May 2005 15:23:54 +0200, Carl Friedrich Bolz writes: >> Instead of just removing it, can you make it into its own page and >> give a link to it. > >I did exactly that. See >http://codespeak.net/pypy/index.cgi?doc/svn-help.html > >Carl Friedrich Thanks very much. Laura From Ben.Young at risk.sungard.com Wed May 18 09:40:10 2005 From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com) Date: Wed, 18 May 2005 08:40:10 +0100 Subject: [pypy-dev] Re: [pypy-svn] r12421 - in pypy/dist: . Pyrex pypy/module/builtin In-Reply-To: <20050517203238.DF00727C0A@code1.codespeak.net> Message-ID: Hi, You can remove me from the list. I only did a bit of fiddling a few months ago! In case this is needed: I hereby grant the pypy developers all copyright to any work I have done on the pypy source code. Cheers, Ben pypy-svn-bounces at codespeak.net wrote on 17/05/2005 21:32:38: > Author: hpk > Date: Tue May 17 22:32:38 2005 > New Revision: 12421 > > Added: > pypy/dist/LICENSE > pypy/dist/Pyrex/LICENSE > Modified: > pypy/dist/pypy/module/builtin/app_help.py > Log: > issue52 testing > > Added a first draft for a top-level LICENSE files, where the copyrights > got computed by examining all committers (via svn log) on the pypy and > py directories. The list is currently ordered by number of commits. > If desired i could extend the script and let it try to run "svn blame". > It probably would give a somewhat more appropriate picture. > > Additionally i put a pseudo-LICENSE into the Pyrex directory > (it was the only notice i could find from the Pyrex author). > > So the questions are (apart from the minor ordering issue): > > - do we want to reduce the list of PyPy copyright holders > somehow? > > - are there any no included sub-projects that miss copyright > statements or a LICENSE file? > > > > Added: pypy/dist/LICENSE > ============================================================================== > --- (empty file) > +++ pypy/dist/LICENSE Tue May 17 22:32:38 2005 > @@ -0,0 +1,96 @@ > +License for files in the pypy and py directory > +============================================== > + > +Except when otherwise stated (look for LICENSE files in directories or > +information at the beginning of each file) all software and > +documentation in the 'pypy' and 'py' directories is licensed as follows: > + > + The MIT License > + > + Permission is hereby granted, free of charge, to any person > + obtaining a copy of this software and associated documentation > + files (the "Software"), to deal in the Software without > + restriction, including without limitation the rights to use, > + copy, modify, merge, publish, distribute, sublicense, and/or > + sell copies of the Software, and to permit persons to whom the > + Software is furnished to do so, subject to the following conditions: > + > + The above copyright notice and this permission notice shall be included > + in all copies or substantial portions of the Software. > + > + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS > + OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF > MERCHANTABILITY, > + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL > + THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, > DAMAGES OR OTHER > + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING > + FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER > + DEALINGS IN THE SOFTWARE. > + > + > +PyPy Copyright holders 2003-2005 > +==================================================== > + > +Except when otherwise stated (look for LICENSE files or information at > +the beginning of each file) the files in the 'pypy' directory are each > +copyrighted by one or more of the following people and organizations: > + > + Armin Rigo > + Samuele Pedroni > + Holger Krekel > + Michael Hudson > + Christian Tismer > + Seo Sanghyeon > + Alex Martelli > + Stefan Schwarzer > + Patrick Maupin > + Carl Friedrich Bolz > + Bob Ippolito > + Anders Chrigstrom > + Jacob Hallen > + Marius Gedminas > + Laura Creighton > + Guido Van Rossum > + Richard Emslie > + Ludovic Aubry > + Adrien Di Mascio > + Stephan Diehl <> > + Dinu Gherman > + Guenter Jantzen > + Anders Lehmann > + Rocco Moretti > + Olivier Dormond > + Brian Dorsey > + Jonathan David Riehl > + Andreas Friedge > + Jens-Uwe Mager > + Alan McIntyre > + Lutz Paelike > + Jacek Generowicz > + Ben Young > + Michael Chermside > + > + Heinrich-Heine University, Germany > + AB Strakt, Sweden > + merlinux GmbH, Germany > + tismerysoft GmbH, Germany > + Logilab Paris, France > + DFKI GmbH, Germany > + > + > +py lib Copyright holders, 2003-2005 > +========================================== > + > +Except when otherwise stated (look for LICENSE files or information at > +the beginning of each file) the files in the 'py' directory are > +copyrighted by one or more of the following people and organizations: > + > + Holger Krekel > + merlinux GmbH, Germany > + Armin Rigo > + Jan Balster > + > +Contributors include:: > + > + Ian Bicking > + Grig Gheorghiu > + Bob Ippolito > > Added: pypy/dist/Pyrex/LICENSE > ============================================================================== > --- (empty file) > +++ pypy/dist/Pyrex/LICENSE Tue May 17 22:32:38 2005 > @@ -0,0 +1,13 @@ > +Copyright stuff: Pyrex is free of restrictions. You > +may use, redistribute, modify and distribute modified > +versions. > + > +The latest version of Pyrex can be found here: > + > +http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/ > + > +Greg Ewing, Computer Science Dept, +--------------------------------------+ > +University of Canterbury, | A citizen of NewZealandCorp, a | > +Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | > +greg at cosc.canterbury.ac.nz > + > > Modified: pypy/dist/pypy/module/builtin/app_help.py > ============================================================================== > --- pypy/dist/pypy/module/builtin/app_help.py (original) > +++ pypy/dist/pypy/module/builtin/app_help.py Tue May 17 22:32:38 2005 > @@ -17,7 +17,7 @@ > def license(): > print \ > """ > -Copyright (c) <2003-2004> > +Copyright (c) <2003-2005> > > Permission is hereby granted, free of charge, to any person > obtaining a copy of this software and associated documentation files > (the "Software"), to deal in the Software without restriction, > including without limitation the rights to use, copy, modify, merge, > publish, distribute, sublicense, and/or sell copies of the Software, > and to permit persons to whom the Software is furnished to do so, > subject to the following conditions: > > _______________________________________________ > pypy-svn mailing list > pypy-svn at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-svn > From hpk at trillke.net Wed May 18 09:39:48 2005 From: hpk at trillke.net (holger krekel) Date: Wed, 18 May 2005 09:39:48 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12421 - in pypy/dist: . Pyrex pypy/module/builtin In-Reply-To: References: <20050517203238.DF00727C0A@code1.codespeak.net> Message-ID: <20050518073948.GI5609@solar.trillke.net> Hi Ben, On Wed, May 18, 2005 at 08:40 +0100, Ben.Young at risk.sungard.com wrote: > You can remove me from the list. I only did a bit of fiddling a few months > ago! > > In case this is needed: I hereby grant the pypy developers all copyright > to any work I have done on the pypy source code. thanks, Ben! On a side note, I guess that we need to refine the "determining copyright holders" process some time soon. Generally the copyright stays with contributors (unless it's only a small fix/contribution) but even that gets more complicated because people are sometimes working "privately" and sometimes on company/university time. Due to the very liberal license (MIT-license) we are using i hope we can continue to avoid fights. This would probably be different if we used e.g. the GPL which gives copyright holders a lot more rights that users then don't have. cheers, holger From arigo at tunes.org Wed May 18 20:28:13 2005 From: arigo at tunes.org (Armin Rigo) Date: Wed, 18 May 2005 20:28:13 +0200 Subject: [pypy-dev] Remove your .pyc files :-) Message-ID: <20050518182813.GA7559@code1.codespeak.net> Hi, Warning, the current changes tend to break already-checked-out working copies because of .pyc files staying behind. If you get import errors, deleting all the .pyc files! (You might want to delete the directory pypy/_cache also, as it might grow over time and be only very occasionally purged automatically.) Armin From eric at vanrietpaap.nl Thu May 19 11:57:42 2005 From: eric at vanrietpaap.nl (Eric van Riet Paap) Date: Thu, 19 May 2005 11:57:42 +0200 Subject: [pypy-dev] llvm install documenation Message-ID: <428C6316.7030304@vanrietpaap.nl> Hi, In the getting_started docs it is mentioned where llvm can found. It does not mention what version of llvm to use. Version 1.5 came out yesterday. With the eminent PyPy 0.6 I think it would be good if everyone uses llvm 1.5 so we can better reproduce any errors. - Eric From guenter.jantzen at netcologne.de Thu May 19 13:31:00 2005 From: guenter.jantzen at netcologne.de (=?ISO-8859-1?Q?G=FCnter_Jantzen?=) Date: Thu, 19 May 2005 13:31:00 +0200 Subject: [pypy-dev] copyright In-Reply-To: References: Message-ID: <428C78F4.2050402@netcologne.de> Hi Pypy! Like Ben, I do not claim copyrights for marginal work. However I stay on pypy-dev and try to follow the big picture. Maybe some day I can help more :-) Cheers and good luck! Guenter Ben.Young at risk.sungard.com schrieb: > Hi, > > You can remove me from the list. I only did a bit of fiddling a few months > ago! > > In case this is needed: I hereby grant the pypy developers all copyright > to any work I have done on the pypy source code. > > Cheers, > Ben > > pypy-svn-bounces at codespeak.net wrote on 17/05/2005 21:32:38: > > >>Author: hpk >>Date: Tue May 17 22:32:38 2005 >>New Revision: 12421 >> >>Added: >> pypy/dist/LICENSE >> pypy/dist/Pyrex/LICENSE >>Modified: >> pypy/dist/pypy/module/builtin/app_help.py >>Log: >>issue52 testing >> >>Added a first draft for a top-level LICENSE files, where the copyrights >>got computed by examining all committers (via svn log) on the pypy and >>py directories. The list is currently ordered by number of commits. >>If desired i could extend the script and let it try to run "svn blame". >>It probably would give a somewhat more appropriate picture. >> >>Additionally i put a pseudo-LICENSE into the Pyrex directory >>(it was the only notice i could find from the Pyrex author). >> >>So the questions are (apart from the minor ordering issue): >> >>- do we want to reduce the list of PyPy copyright holders >> somehow? >> >>- are there any no included sub-projects that miss copyright >> statements or a LICENSE file? >> >> >> >>Added: pypy/dist/LICENSE >> > > ============================================================================== > >>--- (empty file) >>+++ pypy/dist/LICENSE Tue May 17 22:32:38 2005 >>@@ -0,0 +1,96 @@ >>+License for files in the pypy and py directory >>+============================================== >>+ >>+Except when otherwise stated (look for LICENSE files in directories or >>+information at the beginning of each file) all software and >>+documentation in the 'pypy' and 'py' directories is licensed as > > follows: > >>+ >>+ The MIT License >>+ >>+ Permission is hereby granted, free of charge, to any person >>+ obtaining a copy of this software and associated documentation >>+ files (the "Software"), to deal in the Software without >>+ restriction, including without limitation the rights to use, >>+ copy, modify, merge, publish, distribute, sublicense, and/or >>+ sell copies of the Software, and to permit persons to whom the >>+ Software is furnished to do so, subject to the following > > conditions: > >>+ >>+ The above copyright notice and this permission notice shall be > > included > >>+ in all copies or substantial portions of the Software. >>+ >>+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, > > EXPRESS > >>+ OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF >>MERCHANTABILITY, >>+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT > > SHALL > >>+ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, >>DAMAGES OR OTHER >>+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, > > ARISING > >>+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER > > >>+ DEALINGS IN THE SOFTWARE. >>+ >>+ >>+PyPy Copyright holders 2003-2005 >>+==================================================== >>+ >>+Except when otherwise stated (look for LICENSE files or information at >>+the beginning of each file) the files in the 'pypy' directory are each >>+copyrighted by one or more of the following people and organizations: >>+ >>+ Armin Rigo >>+ Samuele Pedroni >>+ Holger Krekel >>+ Michael Hudson >>+ Christian Tismer >>+ Seo Sanghyeon >>+ Alex Martelli >>+ Stefan Schwarzer >>+ Patrick Maupin >>+ Carl Friedrich Bolz >>+ Bob Ippolito >>+ Anders Chrigstrom >>+ Jacob Hallen >>+ Marius Gedminas >>+ Laura Creighton >>+ Guido Van Rossum >>+ Richard Emslie >>+ Ludovic Aubry >>+ Adrien Di Mascio >>+ Stephan Diehl <> >>+ Dinu Gherman >>+ Guenter Jantzen >>+ Anders Lehmann >>+ Rocco Moretti >>+ Olivier Dormond >>+ Brian Dorsey >>+ Jonathan David Riehl >>+ Andreas Friedge >>+ Jens-Uwe Mager >>+ Alan McIntyre >>+ Lutz Paelike >>+ Jacek Generowicz >>+ Ben Young >>+ Michael Chermside >>+ >>+ Heinrich-Heine University, Germany >>+ AB Strakt, Sweden >>+ merlinux GmbH, Germany >>+ tismerysoft GmbH, Germany >>+ Logilab Paris, France >>+ DFKI GmbH, Germany >>+ >>+ >>+py lib Copyright holders, 2003-2005 >>+========================================== >>+ >>+Except when otherwise stated (look for LICENSE files or information at >>+the beginning of each file) the files in the 'py' directory are >>+copyrighted by one or more of the following people and organizations: >>+ >>+ Holger Krekel >>+ merlinux GmbH, Germany >>+ Armin Rigo >>+ Jan Balster >>+ >>+Contributors include:: >>+ >>+ Ian Bicking >>+ Grig Gheorghiu >>+ Bob Ippolito >> >>Added: pypy/dist/Pyrex/LICENSE >> > > ============================================================================== > >>--- (empty file) >>+++ pypy/dist/Pyrex/LICENSE Tue May 17 22:32:38 2005 >>@@ -0,0 +1,13 @@ >>+Copyright stuff: Pyrex is free of restrictions. You >>+may use, redistribute, modify and distribute modified >>+versions. >>+ >>+The latest version of Pyrex can be found here: >>+ >>+http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/ >>+ >>+Greg Ewing, Computer Science Dept, > > +--------------------------------------+ > >>+University of Canterbury, | A citizen of NewZealandCorp, a | >>+Christchurch, New Zealand | wholly-owned subsidiary of USA > > Inc. | > >>+greg at cosc.canterbury.ac.nz >>+ >> >>Modified: pypy/dist/pypy/module/builtin/app_help.py >> > > ============================================================================== > >>--- pypy/dist/pypy/module/builtin/app_help.py (original) >>+++ pypy/dist/pypy/module/builtin/app_help.py Tue May 17 22:32:38 2005 >>@@ -17,7 +17,7 @@ >> def license(): >> print \ >> """ >>-Copyright (c) <2003-2004> >>+Copyright (c) <2003-2005> >> >> Permission is hereby granted, free of charge, to any person >>obtaining a copy of this software and associated documentation files >>(the "Software"), to deal in the Software without restriction, >>including without limitation the rights to use, copy, modify, merge, >>publish, distribute, sublicense, and/or sell copies of the Software, >>and to permit persons to whom the Software is furnished to do so, >>subject to the following conditions: >> >>_______________________________________________ >>pypy-svn mailing list >>pypy-svn at codespeak.net >>http://codespeak.net/mailman/listinfo/pypy-svn >> > > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From arigo at tunes.org Thu May 19 16:28:05 2005 From: arigo at tunes.org (Armin Rigo) Date: Thu, 19 May 2005 16:28:05 +0200 Subject: [pypy-dev] Sprint (talks) movies In-Reply-To: <4284E0C3.5060304@vanrietpaap.nl> References: <4284E0C3.5060304@vanrietpaap.nl> Message-ID: <20050519142805.GA23915@code1.codespeak.net> Hi Eric, On Fri, May 13, 2005 at 07:15:47PM +0200, Eric van Riet Paap wrote: > Are any off the talks about PyPy recorded? The PyCon one was recorded, but I cannot find it anywhere. There are links pointing to www.pycon.org/talks, but the whole site appears to be down -- showing a Zope error page :-( Armin From eric at vanrietpaap.nl Thu May 19 21:30:12 2005 From: eric at vanrietpaap.nl (Eric van Riet Paap) Date: Thu, 19 May 2005 21:30:12 +0200 Subject: [pypy-dev] coming to the Goteborg sprint Message-ID: <428CE944.709@vanrietpaap.nl> Hello all, I have booked my flight to Goteborg to attend my first sprint, I arrive on the 30th of june and depart on the 8th of july. So I would like to introduce myself in this email. My background is in the games industry. Instead of writing a nice story here, I will refrain myself to a list in chronological order. - BBS for C64 - Chess, checkers, Awari programs, mostly for dedicated (8 bit cpu) hardware. - Playstation one game called Dodgem Arena - Various webgames - Virtual reality software writting in C++ with OpenGL for graphics and Python for interaction in the 3d worlds - Custom-made CMS written in Python for Financial Times newspaper subscriptions - Fulltime dad for last 6 months best regards, Eric van Riet Paap Utrecht The Netherlands From hpk at trillke.net Thu May 19 21:59:49 2005 From: hpk at trillke.net (holger krekel) Date: Thu, 19 May 2005 21:59:49 +0200 Subject: [pypy-dev] coming to the Goteborg sprint In-Reply-To: <428CE944.709@vanrietpaap.nl> References: <428CE944.709@vanrietpaap.nl> Message-ID: <20050519195949.GT13541@solar.trillke.net> Hi Eric, On Thu, May 19, 2005 at 21:30 +0200, Eric van Riet Paap wrote: > I have booked my flight to Goteborg to attend my first sprint, I arrive on the 30th of june and depart on the 8th of july. Great! > So I would like to introduce myself in this email. > > My background is in the games industry. Instead of writing a nice story here, I will refrain myself to a list in chronological order. > - BBS for C64 > - Chess, checkers, Awari programs, mostly for dedicated (8 bit cpu) hardware. > - Playstation one game called Dodgem Arena > - Various webgames > - Virtual reality software writting in C++ with OpenGL for graphics and Python for interaction in the 3d worlds > - Custom-made CMS written in Python for Financial Times newspaper subscriptions > - Fulltime dad for last 6 months I guess the latter is the most challenging job :-) Actually you probably intended to send this to pypy-sprint but that is also rather public so don't worry. cheers, holger From arigo at tunes.org Fri May 20 23:43:40 2005 From: arigo at tunes.org (Armin Rigo) Date: Fri, 20 May 2005 23:43:40 +0200 Subject: [pypy-dev] PyPy released! Message-ID: <20050520214340.GA26290@code1.codespeak.net> The PyPy 0.6 release -------------------- *The PyPy Development Team is happy to announce the first public release of PyPy after two years of spare-time and half a year of EU funded development. The 0.6 release is eminently a preview release.* What it is and where to start ----------------------------- Getting started: http://codespeak.net/pypy/index.cgi?doc/getting_started.html PyPy Documentation: http://codespeak.net/pypy/index.cgi?doc PyPy Homepage: http://codespeak.net/pypy/ PyPy is a MIT-licensed reimplementation of Python written in Python itself. The long term goals are an implementation that is flexible and easy to experiment with and retarget to different platforms (also non-C ones) and such that high performance can be achieved through high-level implementations of dynamic optimisation techniques. The interpreter and object model implementations shipped with 0.6 can be run on top of CPython and implement the core language features of Python as of CPython 2.3. PyPy passes around 90% of the Python language regression tests that do not depend deeply on C-extensions. Some of that functionality is still made available by PyPy piggy-backing on the host CPython interpreter. Double interpretation and abstractions in the code-base make it so that PyPy running on CPython is quite slow (around 2000x slower than CPython ), this is expected. This release is intended for people that want to look and get a feel into what we are doing, playing with interpreter and perusing the codebase. Possibly to join in the fun and efforts. Interesting bits and highlights --------------------------------- The release is also a snap-shot of our ongoing efforts towards low-level translation and experimenting with unique features. * By default, PyPy is a Python version that works completely with new-style-classes semantics. However, support for old-style classes is still available. Implementations, mostly as user-level code, of their metaclass and instance object are included and can be re-made the default with the ``--oldstyle`` option. * In PyPy, bytecode interpretation and object manipulations are well separated between a bytecode interpreter and an *object space* which implements operations on objects. PyPy comes with experimental object spaces augmenting the standard one through delegation: * an experimental object space that does extensive tracing of bytecode and object operations; * the 'thunk' object space that implements lazy values and a 'become' operation that can exchange object identities. These spaces already give a glimpse in the flexibility potential of PyPy. See demo/fibonacci.py and demo/sharedref.py for examples about the 'thunk' object space. * The 0.6 release also contains a snapshot of our translation-efforts to lower level languages. For that we have developed an annotator which is capable of infering type information across our code base. The annotator right now is already capable of successfully type annotating basically *all* of PyPy code-base, and is included with 0.6. * From type annotated code, low-level code needs to be generated. Backends for various targets (C, LLVM,...) are included; they are all somehow incomplete and have been and are quite in flux. What is shipped with 0.6 is able to deal with more or less small/medium examples. Ongoing work and near term goals --------------------------------- Generating low-level code is the main area we are hammering on in the next months; our plan is to produce a PyPy version in August/September that does not need to be interpreted by CPython anymore and will thus run considerably faster than the 0.6 preview release. PyPy has been a community effort from the start and it would not have got that far without the coding and feedback support from numerous people. Please feel free to give feedback and raise questions. contact points: http://codespeak.net/pypy/index.cgi?contact contributor list: http://codespeak.net/pypy/index.cgi?doc/contributor.html have fun, Armin Rigo, Samuele Pedroni, Holger Krekel, Christian Tismer, Carl Friedrich Bolz PyPy development and activities happen as an open source project and with the support of a consortium funded by a two year EU IST research grant. Here is a list of partners of the EU project: Heinrich-Heine University (Germany), AB Strakt (Sweden) merlinux GmbH (Germany), tismerysoft GmbH(Germany) Logilab Paris (France), DFKI GmbH (Germany) ChangeMaker (Sweden) From briandorsey at gmail.com Sat May 21 08:37:29 2005 From: briandorsey at gmail.com (Brian Dorsey) Date: Fri, 20 May 2005 23:37:29 -0700 Subject: [pypy-dev] PyPy released! In-Reply-To: <20050520214340.GA26290@code1.codespeak.net> References: <20050520214340.GA26290@code1.codespeak.net> Message-ID: <66e877b70505202337701affb5@mail.gmail.com> On 20/05/05, Armin Rigo wrote: > The PyPy 0.6 release > -------------------- Congratulations everyone! I'd also like to add that I found the release document very clear and I especially like this paragraph: > PyPy is a MIT-licensed reimplementation of Python written in > Python itself. The long term goals are an implementation that > is flexible and easy to experiment with and retarget to > different platforms (also non-C ones) and such that high > performance can be achieved through high-level implementations > of dynamic optimisation techniques. Take care, -Brian From arigo at tunes.org Sat May 21 14:01:40 2005 From: arigo at tunes.org (Armin Rigo) Date: Sat, 21 May 2005 14:01:40 +0200 Subject: [pypy-dev] PyPy released! In-Reply-To: <20050520214340.GA26290@code1.codespeak.net> References: <20050520214340.GA26290@code1.codespeak.net> Message-ID: <20050521120140.GA7545@code1.codespeak.net> Hi ! On Fri, May 20, 2005 at 11:43:40PM +0200, Armin Rigo wrote: > The PyPy 0.6 release > -------------------- ...has already been superseded by the PyPy 0.6.1 bug-fix release. The bug is trivial enough: py.py crashes on string formatting the second time you run it. We hope to streamline the release process in the future to avoid such after-the-last-minute bug fixes. In the name of the team, please accept our apologizes, Armin Rigo From js.nospam at wolke7.net Sun May 22 11:45:18 2005 From: js.nospam at wolke7.net (js.nospam at wolke7.net) Date: Sun, 22 May 2005 11:45:18 +0200 (MEST) Subject: [pypy-dev] Python-LLVM bindings updated Message-ID: <26958.1116755118@www10.gmx.net> Hi, I've updated the Python LLVM bindings for LLVM 1.5 (attached). I also thought of setting up a CVS repository for this, but before I do that I thought I'd ask if you want to take the Python bindings to the main LLVM CVS? I noticed that the C language bindings would be maintained there. Changes in Python-LLVM: - Updated to match LLVM 1.5 API - Added Windows build support (created MSVC++ project file) - Added bindings for LLVM Transforms API Systems built on: - Linux 2.6.10 i686 (GCC 3.3.5) - Windows XP Home SP2 (MSVC++ 7.1) As before, you'll need Boost.Python and Python to build the bindings. If you're interested, a previous version of the Python bindings was also sent to llvm-dev: http://mail.cs.uiuc.edu/pipermail/llvmdev/2005-February/003463.html Cheers, Jarno -- 5 GB Mailbox, 50 FreeSMS http://www.gmx.net/de/go/promail +++ GMX - die erste Adresse f?r Mail, Message, More +++ From reid at x10sys.com Sun May 22 16:36:33 2005 From: reid at x10sys.com (Reid Spencer) Date: Sun, 22 May 2005 07:36:33 -0700 Subject: [pypy-dev] Re: [LLVMdev] Python-LLVM bindings updated In-Reply-To: <26958.1116755118@www10.gmx.net> References: <26958.1116755118@www10.gmx.net> Message-ID: <1116772593.5430.255.camel@bashful.x10sys.com> Hi Jarno, This is great! We would absolutely love to have python bindings for LLVM. From the look of your patch, I think we want this to become an LLVM project since it has configuration and compiler support that is new to the main LLVM repository. We don't want the main LLVM libraries to be dependent on Boost or Python. So, there's a couple things we need to have you do before this patch is accepted. 1. Please place it in a directory named something like "llvm-py", not in "lib" 2. Make this "llvm-py" directory look like an llvm project. You can find a sample project at llvm/projects/sample and a more full blown example in llvm/projects/Stacker. Please also make sure you read http://llvm.cs.uiuc.edu/docs/Projects.html which will help you set up a project. You might also find the Projects section of http://llvm.cs.uiuc.edu/docs/MakefileGuide.html useful. 3. Fix your Makefile to build a shared library properly. You have set TOOLNAME which is for building an executable tool not a shared library. You need to set LIBRARYNAME=..., not TOOLNAME=... 4. Fix the name of your project. It currently builds "llvm.so" which is not correct by our naming standards. It should result to something like "libLLVMpy.so" which will be obtained if you set LIBRARYNAME to "LLVMpy" 5. Consider waiting for or modifying this python interface latter to either directly use the (forthcoming) C interface or design the interface to have a similar structure and naming convention. 6. I looked (briefly) for a test suite but didn't find much. Could you please create a "test" directory that mimics the LLVM test directory so that "make check" will test out a few python programs that use the python interface to build working LLVM code. 7. Please submit gzipped, tar'd files with a ".tgz" suffix so that tar and gunzip can work on them without having to modify the file name (yes, this is a nit!) Once you've got the above accomplished, we can create a repository for the python interface and (with Oversight Group approval) give you cvs write access to it. That will allow you to make changes and maintain the python interface as LLVM grows without significant difficulty or burden. We look forward to adding Python to LLVM! Rei On Sun, 2005-05-22 at 11:45 +0200, js.nospam at wolke7.net wrote: > Hi, > > I've updated the Python LLVM bindings for LLVM 1.5 (attached). I also > thought of setting up a CVS repository for this, but before I do that > I thought I'd ask if you want to take the Python bindings to the main > LLVM CVS? I noticed that the C language bindings would be maintained > there. > > Changes in Python-LLVM: > - Updated to match LLVM 1.5 API > - Added Windows build support (created MSVC++ project file) > - Added bindings for LLVM Transforms API > > Systems built on: > - Linux 2.6.10 i686 (GCC 3.3.5) > - Windows XP Home SP2 (MSVC++ 7.1) > > As before, you'll need Boost.Python and Python to build the bindings. > If you're interested, a previous version of the Python bindings was > also sent to llvm-dev: > http://mail.cs.uiuc.edu/pipermail/llvmdev/2005-February/003463.html > > Cheers, Jarno > > -- > 5 GB Mailbox, 50 FreeSMS http://www.gmx.net/de/go/promail > +++ GMX - die erste Adresse fr Mail, Message, More +++ > _______________________________________________ LLVM Developers mailing list LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu http://mail.cs.uiuc.edu/mailman/listinfo/llvmdev From tismer at stackless.com Mon May 23 10:54:45 2005 From: tismer at stackless.com (Christian Tismer) Date: Mon, 23 May 2005 10:54:45 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12711 - pypy/dist/pypy/translator In-Reply-To: <20050521121932.19D4827B6D@code1.codespeak.net> References: <20050521121932.19D4827B6D@code1.codespeak.net> Message-ID: <42919A55.6030006@stackless.com> Hi Armin, I dodn't understand completely why this patch is needed: > Modified: pypy/dist/pypy/translator/geninterplevel.py > ============================================================================== > --- pypy/dist/pypy/translator/geninterplevel.py (original) > +++ pypy/dist/pypy/translator/geninterplevel.py Sat May 21 14:19:31 2005 > @@ -77,7 +77,7 @@ > import pypy # __path__ > import py.path > > -GI_VERSION = '1.1.0' # bump this for substantial changes > +GI_VERSION = '1.1.1' # bump this for substantial changes > # ____________________________________________________________ > > def eval_helper(self, typename, expr): > @@ -420,10 +420,12 @@ > self.initcode.append1('import sys') > self.initcode.append1('import os') > self.initcode.append1('libdir = os.path.join(pypy.__path__[0], "lib")\n' > - 'hold = sys.path[:]\n' > 'sys.path.insert(0, libdir)\n' > - 'import %s as _tmp\n' > - 'sys.path[:] = hold\n' % value.__name__) > + 'try:\n' > + ' import %s as _tmp\n' > + 'finally:\n' > + ' if libdir in sys.path:\n' > + ' sys.path.remove(libdir)\n' % value.__name__) What I'm doing at compiletime is to figure out whether lib is needed or not, and I know by compiletime whether the import works, etc. Then I generate init code that adheres to this. Why do I need to check there, too? Just as a safety belt if something breaks? ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From ac at strakt.com Mon May 23 10:57:31 2005 From: ac at strakt.com (=?ISO-8859-1?Q?Anders_Chrigstr=F6m?=) Date: Mon, 23 May 2005 10:57:31 +0200 Subject: [pypy-dev] New unicode implementation Message-ID: <42919AFB.5000108@strakt.com> I am about to merge my 'non-faked-unicode' branch. Given the amount of changes I won't be suprised it it upsets the annotaton and/or code generators. /Arre From tismer at stackless.com Mon May 23 11:32:06 2005 From: tismer at stackless.com (Christian Tismer) Date: Mon, 23 May 2005 11:32:06 +0200 Subject: [pypy-dev] Re: first release of PyPy In-Reply-To: References: <1116629407.638782.145950@g47g2000cwa.googlegroups.com> <1116644536.741250.249860@g44g2000cwa.googlegroups.com> <1116653460.808204.74460@g43g2000cwa.googlegroups.com> <87mzqpytep.fsf@wilson.rwth-aachen.de> <7xfywhxerd.fsf@ruckus.brouhaha.com> <87is1dyru8.fsf@wilson.rwth-aachen.de> Message-ID: <4291A316.9080505@stackless.com> Ville Vainio wrote: >>>>>>"Christian" == Christian Tismer writes: > > > >> PyPy is written in python, if it can be compiled then the programs > >> can > >> be as well. > > Christian> Well, this is not really true. PyPy is written in > Christian> RPython, a sub-language of Python that is implicitly > Christian> defined by "simple and static enough to be compilable". > > Could it be possible to tag some modules in application code as > RPython-compatible, making it possible to implement the speed critical > parts in RPython? Interesting idea. Especially since we have automatic translation from RPythonic application code to interpreter level. Maybe not for now, but I'm cc-ing pypy-dev. @rpythonic :-) -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From hpk at trillke.net Mon May 23 14:02:21 2005 From: hpk at trillke.net (holger krekel) Date: Mon, 23 May 2005 14:02:21 +0200 Subject: [pypy-dev] unicode merge / exception catching In-Reply-To: <20050523110559.E32FD27B8D@code1.codespeak.net> References: <20050523110559.E32FD27B8D@code1.codespeak.net> Message-ID: <20050523120221.GM13541@solar.trillke.net> Hi Arre, your merge looks pretty good so far. One issue relating to exceptions though. > Author: ac > Date: Mon May 23 13:05:59 2005 > New Revision: 12735 > ... > ... > --- pypy/dist/pypy/objspace/std/inttype.py (original) > +++ pypy/dist/pypy/objspace/std/inttype.py Mon May 23 13:05:59 2005 > @@ -28,6 +28,15 @@ > space.wrap(e.msg)) > except ParseStringOverflowError, e: > w_longval = retry_to_w_long(space, e.parser) > + elif space.is_true(space.isinstance(w_value, space.w_unicode)): > + try: > + from unicodeobject import unicode_to_decimal_w > + value = string_to_int(space, unicode_to_decimal_w(space, w_value)) > + except ParseStringError, e: > + raise OperationError(space.w_ValueError, > + space.wrap(e.msg)) > + except ParseStringOverflowError, e: > + w_longval = retry_to_w_long(space, e.parser) generally we should try to use "exact" exception catching both for readability and (i think, someone correct me) for easier translatability. This means that you should only try-except guard the exact call that can raise the exception. > +from pypy.objspace.std.fake import wrap_exception and the usage: > -def float__Unicode(space, w_uni): > - try: > - return space.wrap(float(unicode_to_decimal_w(space, w_uni))) > - except: > - wrap_exception(space) falls into the same "don't be so lazy and make the life possibly harder for translation or the reader" category :-) cheers, holger From arigo at tunes.org Mon May 23 14:22:45 2005 From: arigo at tunes.org (Armin Rigo) Date: Mon, 23 May 2005 14:22:45 +0200 Subject: [pypy-dev] IRC metting on translator Message-ID: <20050523122245.GA32045@code1.codespeak.net> Hi all, We have an IRC "appointement" to talk about the translator and get us all on the same page. I do apologize for a lack of communication about what Samuele and myself have been working on lately; this is an attempt at fixing that. Everybody is welcome, of course. Wednesday 15:00, Central Europe Time A bientot, Armin. From js.nospam at wolke7.net Mon May 23 21:18:01 2005 From: js.nospam at wolke7.net (=?iso-8859-1?Q?Jarno_Sepp=E4nen?=) Date: Mon, 23 May 2005 22:18:01 +0300 Subject: [pypy-dev] RE: [LLVMdev] Python-LLVM bindings updated In-Reply-To: <1116772593.5430.255.camel@bashful.x10sys.com> Message-ID: <000201c55fcc$23f40950$3601a8c0@JARNO> Hi, some discussion below. Reid wrote: > 3. Fix your Makefile to build a shared library properly. You have set > TOOLNAME which is for building an executable tool not a shared > library. You need to set LIBRARYNAME=..., not TOOLNAME=... I remember having failed to build a functioning Python extension module with LIBRARYNAME and SHARED_LIBRARY flags. Anyway, I cannot remember the details now so will try again. > 4. Fix the name of your project. It currently builds "llvm.so" which > is not correct by our naming standards. It should result to something > like "libLLVMpy.so" which will be obtained if you set LIBRARYNAME > to "LLVMpy" I'd like to challenge the usefulness of the naming standard here. I see its value in case of system-wide C/C++ libraries (going to /usr/lib/*), but the Python modules go into a Python-specific directory (/usr/lib/python2.4/site-packages/*), so the name must be unique within Python extensions only. > 5. Consider waiting for or modifying this python interface latter to > either directly use the (forthcoming) C interface or design the > interface to have a similar structure and naming convention. The Python interface represents the C++ interface, so I don't understand the motivation to use C in the middle? In my view the Python and C bindings are independent of each other. > Once you've got the above accomplished, we can create a repository for > the python interface and (with Oversight Group approval) give you cvs > write access to it. That will allow you to make changes and maintain the > python interface as LLVM grows without significant difficulty or burden. OK, so I'll get back when I have news about the directory and Makefile restructuring. Regards, Jarno From hpk at trillke.net Tue May 24 14:31:04 2005 From: hpk at trillke.net (holger krekel) Date: Tue, 24 May 2005 14:31:04 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12768 - pypy/branch/pycompiler/module/recparser/compiler In-Reply-To: <20050524122139.AC27227B56@code1.codespeak.net> References: <20050524122139.AC27227B56@code1.codespeak.net> Message-ID: <20050524123104.GV13541@solar.trillke.net> Hi Ludovic, On Tue, May 24, 2005 at 14:21 +0200, ludal at codespeak.net wrote: > Author: ludal > Date: Tue May 24 14:21:39 2005 > New Revision: 12768 > > Added: > pypy/branch/pycompiler/module/recparser/compiler/ > pypy/branch/pycompiler/module/recparser/compiler/__init__.py > pypy/branch/pycompiler/module/recparser/compiler/ast.py > pypy/branch/pycompiler/module/recparser/compiler/ast.txt > pypy/branch/pycompiler/module/recparser/compiler/astfactory.py > pypy/branch/pycompiler/module/recparser/compiler/astgen.py > pypy/branch/pycompiler/module/recparser/compiler/consts.py > pypy/branch/pycompiler/module/recparser/compiler/future.py > pypy/branch/pycompiler/module/recparser/compiler/misc.py > pypy/branch/pycompiler/module/recparser/compiler/pyassem.py > pypy/branch/pycompiler/module/recparser/compiler/pycodegen.py > pypy/branch/pycompiler/module/recparser/compiler/symbols.py > pypy/branch/pycompiler/module/recparser/compiler/syntax.py > pypy/branch/pycompiler/module/recparser/compiler/transformer.py > pypy/branch/pycompiler/module/recparser/compiler/visitor.py > Log: > * included modified version of the original compiler module if you plan on reusing the python compiler module, can you do it slightly differently? For modules/packages, that we are taking and modifying from CPython, we usually copy them from svn/pypy/dist/lib-python/2.3.4 to svn/pypy/dist/lib-python/modified-2.3.4 This allows to keep exact track of what got modified and also the base version we started working from. Note though that these modules are then imported at application-level only and if you want to explore if the compiler package can be modified to be translateable (good idea :-) and thus be written at RPython-level then the above place is not quite right. However, i'd still recommend to then do something like e.g.: svn cp http://codespeak.net/svn/pypy/dist/lib-python/2.3.4 \ http://codespeak.net/svn/branch/pycompiler/module/recparser/compiler so that when you perform 'svn log ...' you will see where that version came from. make sense to you? I also notice that you branched in a slightly unusual way, because so far we did the branching like this: svn cp http://codespeak.net/svn/pypy/dist http://codespeak.net/svn/pypy/branch/whatever this makes sure that you can checkout 'whatever' and have a full pypy-dist hierarchy underneath. With your approach you probably need to use an explicit checkout-target, e.g. 'svn co X pypy' (where the latter 'pypy' is the checkout target). cheers, holger From ludal at logilab.fr Tue May 24 15:23:02 2005 From: ludal at logilab.fr (Ludovic Aubry) Date: Tue, 24 May 2005 15:23:02 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12768 - pypy/branch/pycompiler/module/recparser/compiler In-Reply-To: <20050524123104.GV13541@solar.trillke.net> References: <20050524122139.AC27227B56@code1.codespeak.net> <20050524123104.GV13541@solar.trillke.net> Message-ID: <20050524132302.GA19868@crater.logilab.fr> On Tue, May 24, 2005 at 02:31:04PM +0200, holger krekel wrote: > Hi Ludovic, > > On Tue, May 24, 2005 at 14:21 +0200, ludal at codespeak.net wrote: > > Author: ludal > > Date: Tue May 24 14:21:39 2005 > > New Revision: 12768 > > > > Added: > > pypy/branch/pycompiler/module/recparser/compiler/ > > pypy/branch/pycompiler/module/recparser/compiler/__init__.py > > pypy/branch/pycompiler/module/recparser/compiler/ast.py > > pypy/branch/pycompiler/module/recparser/compiler/ast.txt > > pypy/branch/pycompiler/module/recparser/compiler/astfactory.py > > pypy/branch/pycompiler/module/recparser/compiler/astgen.py > > pypy/branch/pycompiler/module/recparser/compiler/consts.py > > pypy/branch/pycompiler/module/recparser/compiler/future.py > > pypy/branch/pycompiler/module/recparser/compiler/misc.py > > pypy/branch/pycompiler/module/recparser/compiler/pyassem.py > > pypy/branch/pycompiler/module/recparser/compiler/pycodegen.py > > pypy/branch/pycompiler/module/recparser/compiler/symbols.py > > pypy/branch/pycompiler/module/recparser/compiler/syntax.py > > pypy/branch/pycompiler/module/recparser/compiler/transformer.py > > pypy/branch/pycompiler/module/recparser/compiler/visitor.py > > Log: > > * included modified version of the original compiler module > > if you plan on reusing the python compiler module, can you > do it slightly differently? For modules/packages, that we are > taking and modifying from CPython, we usually copy them from > > svn/pypy/dist/lib-python/2.3.4 > > to > > svn/pypy/dist/lib-python/modified-2.3.4 > > This allows to keep exact track of what got modified and also > the base version we started working from. > > Note though that these modules are then imported at > application-level only and if you want to explore if the > compiler package can be modified to be translateable (good idea :-) > and thus be written at RPython-level then the above place is not quite Yes this is the intent > right. However, i'd still recommend to then do something > like e.g.: > > svn cp http://codespeak.net/svn/pypy/dist/lib-python/2.3.4 \ > http://codespeak.net/svn/branch/pycompiler/module/recparser/compiler > > so that when you perform 'svn log ...' you will see where that version > came from. yes I remembered we talked about that on #pypy but actually this version is from python-2.4 and also has a module from Tools/compiler I put a note to remember to rebase it to the correct version before merging > > make sense to you? > > I also notice that you branched in a slightly unusual way, because > so far we did the branching like this: > > svn cp http://codespeak.net/svn/pypy/dist http://codespeak.net/svn/pypy/branch/whatever > > this makes sure that you can checkout 'whatever' and have a full pypy-dist hierarchy > underneath. With your approach you probably need to use an explicit checkout-target, > e.g. 'svn co X pypy' (where the latter 'pypy' is the checkout target). yes the copy was done at dist/pypy level I just do svn switch at the root of the branch from a full checkout. I agree the path to the branch should reflect the path from dist so that people don't get confused. this way svn up allows to keep track of changes from trunk in py and lib-python/... > > cheers, > > holger > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev -- Ludovic Aubry LOGILAB, Paris (France). http://www.logilab.com http://www.logilab.fr http://www.logilab.org From hpk at trillke.net Tue May 24 17:58:12 2005 From: hpk at trillke.net (holger krekel) Date: Tue, 24 May 2005 17:58:12 +0200 Subject: [pypy-dev] who did that? Message-ID: <20050524155812.GC13541@solar.trillke.net> Hi folks, i am pretty suprised and actually quite shocked to have found the following unveiling picture from our pre-release work: http://spicy.negimaki.com/index.php?showimage=13 Can somebody please explain? cheers, holger From cfbolz at gmx.de Tue May 24 19:18:34 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 24 May 2005 19:18:34 +0200 Subject: [pypy-dev] who did that? In-Reply-To: <20050524155812.GC13541@solar.trillke.net> References: <20050524155812.GC13541@solar.trillke.net> Message-ID: <429361EA.1040307@gmx.de> holger krekel wrote: > i am pretty suprised and actually quite shocked to have found > the following unveiling picture from our pre-release work: > > http://spicy.negimaki.com/index.php?showimage=13 Googling for PyPy again, have we? Carl Friedrich From arigo at tunes.org Wed May 25 13:56:26 2005 From: arigo at tunes.org (Armin Rigo) Date: Wed, 25 May 2005 13:56:26 +0200 Subject: [pypy-dev] IRC metting on translator In-Reply-To: <20050523122245.GA32045@code1.codespeak.net> References: <20050523122245.GA32045@code1.codespeak.net> Message-ID: <20050525115626.GA11777@code1.codespeak.net> Hi, Please have a look at this "quick tour" of lltype.py: http://codespeak.net/pypy/index.cgi?doc/translation.html#low-level-types Armin From arigo at tunes.org Wed May 25 19:48:06 2005 From: arigo at tunes.org (Armin Rigo) Date: Wed, 25 May 2005 19:48:06 +0200 Subject: [pypy-dev] IRC metting on translator In-Reply-To: <20050525115626.GA11777@code1.codespeak.net> References: <20050523122245.GA32045@code1.codespeak.net> <20050525115626.GA11777@code1.codespeak.net> Message-ID: <20050525174806.GA24995@code1.codespeak.net> Hi, Attached, some logs of the screen session. Probably doesn't make sense without the IRC log as well, and even so. 1. playing in the interactive Python prompt with lltypes to build a list-like implementation 2. a "to-do" roadmap, mainly about how to implement various RPython types in low-level types 3. more playing about function pointers, and how they could be used to call external functions in libraries (thus going in the good direction to help with this general problem that we have never investigated in-depth). (Pasted at the end of 1. above). A bientot, Armin -------------- next part -------------- ****************************************************************************** Python 2.3.5 (#1, Feb 20 2005, 11:38:51) [GCC 3.4.3 20050110 (Gentoo Linux 3.4.3.20050110, ssp-3.4.3.20050110-0, pie-8.7 on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from pypy.rpython import rtyper >>> from pypy.rpython import rlist >>> from pypy.rpython.rlist import * >>> ll_newlist >>> from pypy.annotation.model import * >>> from pypy.annotation.listdef import * >>> s_list = SomeList(ListDef(None, s_item=SomeInteger())) >>> s_list SomeList(listdef=) >>> s_list.lowleveltype() >>> s_list.lowleveltype() == s_list.lowleveltype() True >>> LISTPTR = s_list.lowleveltype() >>> ll_newlist(LISTPTR, 5) <_ptrtype to struct list { items=... }> >>> ptr = _ >>> ptr.items <_ptrtype to array [ {item=0}, {item=0}, {item=0}, {item=0}, {item=0} ]> >>> ptr.items[0] <_ptrtype to struct { item=0 }> >>> LISTPTR >>> LISTPTR.TO >>> LISTPTR.TO.items >>> LISTPTR.TO.items.OF Traceback (most recent call last): File "", line 1, in ? AttributeError: '_PtrType' object has no attribute 'OF' >>> LISTPTR.TO.items.TO >>> LISTPTR.TO.items.TO.OF { item: Signed }> >>> ptr # attributes are the fields of the structure it points to <_ptrtype to struct list { items=... }> >>> ptr.items <_ptrtype to array [ {item=0}, {item=0}, {item=0}, {item=0}, {item=0} ]> >>> # that's a pointer to an array ... >>> ptr.items[0] <_ptrtype to struct { item=0 }> >>> ptr.items[0].item 0 >>> ptr.items[0].item = 42 >>> ptr.items <_ptrtype to array [ {item=42}, {item=0}, {item=0}, {item=0}, {item=0} ]> >>> # pointer to an array in the sense of the produces structure? KeyboardInterrupt >>> # for example, to resize a list, we would do something like: ... >>> LISTPTR >>> LISTPTR.TO >>> LISTPTR.TO.items >>> LISTPTR.TO.items.TO >>> LISTARRAY = _ >>> malloc(LISTARRAY, 6) <_ptrtype to array [ {item=0}, {item=0}, {item=0}, {item=0}, {item=0}, {item=0} ]> >>> newitems = _ >>> newitems[0].item = ptr.items[0].item >>> # etc in a for loop ... >>> newitems <_ptrtype to array [ {item=42}, {item=0}, {item=0}, {item=0}, {item=0}, {item=0} ]> >>> ptr.items = newitems # point to the new array, forget the old one >>> ptr <_ptrtype to struct list { items=... }> >>> ptr.items <_ptrtype to array [ {item=42}, {item=0}, {item=0}, {item=0}, {item=0}, {item=0} ]> >>> ****************************************************************************** >>> F = FuncType([Signed, Signed], Signed) >>> F Signed> >>> functionptr(F, 'externalname') # (TYPE, name) <_ptrtype to func externalname> >>> # a pointer to a function which has a name, and nothing more ... >>> # maybe with more explicit flags: >>> f = functionptr(F, 'sin', external='math') >>> f._obj.external 'math' >>> # so the code generators would know to try to find the given C function ... # from the given library (the details are very backend-specificm though). ... >>> # note that at the moment we're generating direct_call() operations ... # where the first argument is such a functionptr, but with ... # a graph=... attribute. -------------- next part -------------- * annotator/rtyper interface about specializations introduced by the annotator * low-level implementation of the RPython types: - int and bool - float - list and list-iterators * consider optimizing lists whose length never changes - dict * string-keyed dicts * immutable dicts with general keys - string - tuples - rpython object system (classes+instances) * instances are GcStruct with the instance fields found by annot. * subclassing: instance of subclass starts with an inlined GcStruct containing the parent part of the instance * instances have a pointer to a "vtable" * one global vtable per class, a Struct with the class fields found by annotation (methods are just class fields, of type 'function pointer') * subclassing: the vtable starts with an inlined Struct of the same type as the base class' vtable * probably special treatment and placement for constructor/__init__ * optimization: de-virtualize method calls when possible, i.e. when a call can only go to one possible method - conversions (- issues with PBCs, need to advance further to discover exactly what is entailed) * write a ll-graph interpreter for testing. * complete translator/c: - incref, decref, free, deallocators - exception handling, a la CPython for now * genllvm: - add support for new ll types - add support for garbage collection (probably reference counting for starters) - From tismer at stackless.com Thu May 26 03:10:26 2005 From: tismer at stackless.com (Christian Tismer) Date: Thu, 26 May 2005 03:10:26 +0200 Subject: [pypy-dev] dependencies created by the geninterp change Message-ID: <42952202.3000504@stackless.com> Hi Armin, I just read your conversation about unicode """ [22:32] aleale: in geninterplevel.py line 419-429 is the template for importing and only pypy/lib is added to the path [22:33] arigo: aleale: yes, but this is only if the import statement is at the global level in the app helper [22:33] arigo: in this case, it is "hidden" inside the unicode_encode__xxx function """ This is not true, I fear. The reason why the release crashed was that instead of enabling immediate import, again, you let it happen at global level. This is not completely equivalent, because it creates a useless reference to the module in the globals! So the same problem occours: the module must be accessible where _formatting is, at geninterp's result's runtime, or we crash. I can heal this probably by filtering all unused globals off. ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From cfbolz at gmx.de Thu May 26 15:26:28 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 26 May 2005 15:26:28 +0200 Subject: [pypy-dev] Diagram of translation process Message-ID: <4295CE84.3030108@gmx.de> Hi pypy-dev, I got a bit bored yesterday and threw together a diagram that shows the various stages of the PyPy translation process. Look there: http://codespeak.net/~cfbolz/translation.pdf Regards, Carl Friedrich From arigo at tunes.org Thu May 26 18:08:10 2005 From: arigo at tunes.org (Armin Rigo) Date: Thu, 26 May 2005 18:08:10 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12801 - pypy/dist/pypy/rpython In-Reply-To: <20050526095757.92C3B27B57@code1.codespeak.net> References: <20050526095757.92C3B27B57@code1.codespeak.net> Message-ID: <20050526160810.GA5641@code1.codespeak.net> Hi Eric, On Thu, May 26, 2005 at 11:57:57AM +0200, ericvrp at codespeak.net wrote: > pypy/dist/pypy/rpython/rint.py > Log: > added more rint methods. Someone please check if this is basicly right! Yes, great work (if boringly repetitive :-) The policy about signed versus unsigned operations is not very clearly defined for the moment. Some operations give a different result when interpreting their arguments as signed or unsigned, while others don't. A few details about it: > + def rtype_lshift((s_int1, s_int2)): > + if s_int1.unsigned or s_int2.unsigned: According to annotation/binaryop.py, lshift and rshift are signed/unsigned depending on the first argument only, not on the shift count. > + def rtype_pow((s_int1, s_int2)): > + #XXX RPythonTyper gives this error: TypeError: rtype_pow() takes exactly 1 argument (2 given) 'pow' is the equivalent of the built-in pow(), which can take a third argument. Our pow operation always takes three arguments. But at this point rtype_pow should probably only generate a two-arguments int_pow or uint_pow, optionally followed by a second operation, int_mod or uint_mod, depending on if the 3rd arg is a SomeInteger or a None: def rtype_pow((s_int1, s_int2), s_int3): ... v_result = direct_op(...) if isinstance(s_int3, SomeInteger): v_int3 = receive(...) v_result = direct_op(..., [v_result, v_int3], ...) elif s_int3.is_constant() and s_int3.const is None: pass else: raise TyperError("pow() 3rd argument must be int or None") return v_result > + def rtype_eq((s_int1, s_int2)): Should probably use the same logic as rtype_l & friends, to avoid considering a signed negative number as possibly equal to a large unsigned number. > + #Unary arithmetic operations Same signed/unsigned distinctions needed... For abs of an unsigned number, and for pos: > + def rtype_pos(s_int): > + #XXX I think this is a nop and should really be removed from the graph you can just return the received v_int variable directly, without any direct_op(). A bientot, Armin From hpk at trillke.net Thu May 26 18:23:00 2005 From: hpk at trillke.net (holger krekel) Date: Thu, 26 May 2005 18:23:00 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12801 - pypy/dist/pypy/rpython In-Reply-To: <20050526160810.GA5641@code1.codespeak.net> References: <20050526095757.92C3B27B57@code1.codespeak.net> <20050526160810.GA5641@code1.codespeak.net> Message-ID: <20050526162300.GC13541@solar.trillke.net> On Thu, May 26, 2005 at 18:08 +0200, Armin Rigo wrote: > On Thu, May 26, 2005 at 11:57:57AM +0200, ericvrp at codespeak.net wrote: > > pypy/dist/pypy/rpython/rint.py > > Log: > > added more rint methods. Someone please check if this is basicly right! > > Yes, great work (if boringly repetitive :-) Wouldn't it maybe make sense to generate all the repetitive code e.g. from a string template in this case? cheers, holger From hpk at trillke.net Thu May 26 23:37:22 2005 From: hpk at trillke.net (holger krekel) Date: Thu, 26 May 2005 23:37:22 +0200 Subject: [pypy-dev] Diagram of translation process In-Reply-To: <4295CE84.3030108@gmx.de> References: <4295CE84.3030108@gmx.de> Message-ID: <20050526213722.GE13541@solar.trillke.net> Hi Carl, On Thu, May 26, 2005 at 15:26 +0200, Carl Friedrich Bolz wrote: > I got a bit bored yesterday and threw together a diagram that shows the > various stages of the PyPy translation process. Look there: > > http://codespeak.net/~cfbolz/translation.pdf nice picture, i was thinking about something very similar today :-) It would be interesting to add translation aspects such as GC/threading. Hum, what do you think about adding a source-format file into pypy/documentation/image/ or something? (see https://codespeak.net/issue/pypy-dev/issue74 :-) cheers, holger From arigo at tunes.org Fri May 27 11:46:51 2005 From: arigo at tunes.org (Armin Rigo) Date: Fri, 27 May 2005 11:46:51 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12801 - pypy/dist/pypy/rpython In-Reply-To: <42961DAD.1010803@vanrietpaap.nl> References: <20050526095757.92C3B27B57@code1.codespeak.net> <20050526160810.GA5641@code1.codespeak.net> <42961DAD.1010803@vanrietpaap.nl> Message-ID: <20050527094651.GA16280@code1.codespeak.net> Hi Eric, (CC'ing to pypy-dev, hope you don't mind...) On Thu, May 26, 2005 at 09:04:13PM +0200, Eric van Riet Paap wrote: > About making itless verbose: I wasn't sure > how much dynamic coding was allowed in this file. Should this keep to > the RPython "coding standard"? No, this file doesn't have to follow the RPython rules. (Also note that even if it did, you are allowed to use 'exec' and any strange things when the module is imported and the class defined.) > Do we want a rbool.py for consistancy (and a rfloat.py). Or do we want > the whole thing in one file and assume its size will shrink when refactored? I guess rbool.py and rfloat.py make sense. Armin From cfbolz at gmx.de Fri May 27 12:09:55 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 27 May 2005 12:09:55 +0200 (MEST) Subject: [pypy-dev] [pypy-svn] r12828 - pypy/dist/pypy/documentation/image References: <20050527094651.GA16280@code1.codespeak.net> Message-ID: <4286.1117188595@www16.gmx.net> > Author: ericvrp > Date: Fri May 27 09:13:30 2005 > New Revision: 12828 > > Modified: > pypy/dist/pypy/documentation/image/translation.sxd > Log: > glow graph->flow graph glow graph? cool, we need one of those. > Still needs to be done to the pdf on the site also! fixed From arigo at tunes.org Sat May 28 02:16:08 2005 From: arigo at tunes.org (Armin Rigo) Date: Sat, 28 May 2005 02:16:08 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12855 - pypy/dist/pypy/rpython In-Reply-To: <20050527222507.4125C27B52@code1.codespeak.net> References: <20050527222507.4125C27B52@code1.codespeak.net> Message-ID: <20050528001608.GA25645@code1.codespeak.net> Hi Eric, On Sat, May 28, 2005 at 12:25:07AM +0200, ericvrp at codespeak.net wrote: > - v_int1 = receive(Unsigned, arg=0) > - v_int2 = receive(Unsigned, arg=1) > + v_int1 = _receive_may_cast(s_int1, Unsigned, 0) > + v_int2 = _receive_may_cast(s_int2, Unsigned, 1) As you just added casts between signed and unsigned in convert_from_to(), the _receive_may_cast() helper you define and use in this check-in should not be necessary -- indeed, the purpose of the original receive() is precisely to detect a mismatch between the actual and the requested type of the argument and generate a conversion if necessary, by calling convert_from_to(). Armin From arigo at tunes.org Sat May 28 13:00:38 2005 From: arigo at tunes.org (Armin Rigo) Date: Sat, 28 May 2005 13:00:38 +0200 Subject: [pypy-dev] _codecs.py test results Message-ID: <20050528110038.GA3092@code1.codespeak.net> Hi Anders, I ran the CPython tests with inprogress__codecs.py renamed to _codecs.py. As I should probably not check them in "the normal way", to avoid confusing the web reports, I checked them into http://codespeak.net/svn/user/arigo/hack/pypy-hack/testresult-codecs/ which you can check out and inspect (Holger, comments on a good way to dig into these 133 files locally?). There are 26 failures, which is 8 more than reported by http://codespeak.net/~hpk/pypy-testresult/ on the trunk -- not too bad, I suppose. Sorry, I didn't figure out so far which ones these 8 tests are... small Python scripts ahead :-) Also, we should move some unicode-specific tests from "non-core" to "core" now. If test_unicode takes ages to run we should try to reduce the longest-running loops. Armin From arigo at tunes.org Sat May 28 13:28:40 2005 From: arigo at tunes.org (Armin Rigo) Date: Sat, 28 May 2005 13:28:40 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12855 - pypy/dist/pypy/rpython In-Reply-To: <20050528001608.GA25645@code1.codespeak.net> References: <20050527222507.4125C27B52@code1.codespeak.net> <20050528001608.GA25645@code1.codespeak.net> Message-ID: <20050528112840.GA3514@code1.codespeak.net> On Sat, May 28, 2005 at 02:16:08AM +0200, Armin Rigo wrote: > Hi Eric, > > On Sat, May 28, 2005 at 12:25:07AM +0200, ericvrp at codespeak.net wrote: > > - v_int1 = receive(Unsigned, arg=0) > > - v_int2 = receive(Unsigned, arg=1) > > + v_int1 = _receive_may_cast(s_int1, Unsigned, 0) > > + v_int2 = _receive_may_cast(s_int2, Unsigned, 1) > > As you just added casts between signed and unsigned in > convert_from_to(), the _receive_may_cast() helper you define and use in > this check-in should not be necessary -- indeed, the purpose of the > original receive() is precisely to detect a mismatch between the actual > and the requested type of the argument and generate a conversion if > necessary, by calling convert_from_to(). > > > Armin > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From arigo at tunes.org Sat May 28 13:38:23 2005 From: arigo at tunes.org (Armin Rigo) Date: Sat, 28 May 2005 13:38:23 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12855 - pypy/dist/pypy/rpython In-Reply-To: <20050528112840.GA3514@code1.codespeak.net> References: <20050527222507.4125C27B52@code1.codespeak.net> <20050528001608.GA25645@code1.codespeak.net> <20050528112840.GA3514@code1.codespeak.net> Message-ID: <20050528113823.GA3659@code1.codespeak.net> Hi, (Oups! Wrong key. Sent an empty e-mail to pypy-dev, sorry...) On Sat, May 28, 2005 at 01:28:40PM +0200, Armin Rigo wrote: > > > - v_int1 = receive(Unsigned, arg=0) > > > - v_int2 = receive(Unsigned, arg=1) > > > + v_int1 = _receive_may_cast(s_int1, Unsigned, 0) > > > + v_int2 = _receive_may_cast(s_int2, Unsigned, 1) By the way, don't spend more time on this right now; I think it's a good time to decide on a slightly less surprizing interface. Apart from the whole global-fetching idea, another confusing aspect is that the rtype_xyz() methods receive s_abc arguments that are not the main thing they operate on -- they usually operate on variables (v_abc) instead. So I am thinking about introducing a "high-level operation" concept and changing the signature of the rtype_xyz() methods: class __extend__(pairtype(SomeInteger, SomeInteger)): def rtype_add(_, hlop, llops): # ignore the (s_int1, s_int2) pair if hlop.s_result.unsigned: v1, v2 = hlop.convertinputs(Unsigned, Unsigned) v3 = llops.generate('uint_add', [v1, v2], resulttype=Unsigned) return v3 else: v1, v2 = hlop.convertinputs(Signed, Signed) v3 = llops.generate('int_add', [v1, v2], resulttype=Signed) return v3 Armin From eric at vanrietpaap.nl Sat May 28 15:52:15 2005 From: eric at vanrietpaap.nl (Eric van Riet Paap) Date: Sat, 28 May 2005 15:52:15 +0200 Subject: [pypy-dev] possible flowgraph simplification? Message-ID: <4298778F.6070604@vanrietpaap.nl> Hi, When this function gets annotated... *def h(n): return not n* An additional block is generated. Is this something that should/can be avoided by the annotator? If this is unavoidable I would like to dive a little in the annotator simplifier to try to merge the blocks. cheers Eric From arigo at tunes.org Sat May 28 22:15:49 2005 From: arigo at tunes.org (Armin Rigo) Date: Sat, 28 May 2005 22:15:49 +0200 Subject: [pypy-dev] possible flowgraph simplification? In-Reply-To: <4298778F.6070604@vanrietpaap.nl> References: <4298778F.6070604@vanrietpaap.nl> Message-ID: <20050528201549.GA7001@code1.codespeak.net> Hi Eric, On Sat, May 28, 2005 at 03:52:15PM +0200, Eric van Riet Paap wrote: > *def h(n): > return not n* > > An additional block is generated. Is this something that should/can be > avoided by the annotator? This is due to the ultimate source of the flow graph, which is the pypy/interpreter/*.py code itself. We have something called space.not_() which is not really an operation per se, but a shortcut defined in baseobjspace.py: def not_(self, w_obj): return self.wrap(not self.is_true(w_obj)) ('self' is the space here.) The self.is_true() is responsible for creating the branch in the flow graph. Looking at the generated flow graphs like you do is, I guess, a good way to detect when is_true() is used a bit too much. In this case, it should be possible to rewrite not_() without using it, but using the 'space.nonzero()' operation instead. (The difference between nonzero() and is_true() is that nonzero() is a regular operation returning a wrapped boolean, which the flow graph translates as just one line in a block; is_true() returns a real True or False and the flow graph must generate a branch, where each branch records what occurs in each of the two cases.) E.g., this looks convoluted but does the trick: def not_(self, w_obj): return self.xor(self.nonzero(w_obj), self.w_True) Armin From arigo at tunes.org Sun May 29 00:05:59 2005 From: arigo at tunes.org (Armin Rigo) Date: Sun, 29 May 2005 00:05:59 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12855 - pypy/dist/pypy/rpython In-Reply-To: <20050528113823.GA3659@code1.codespeak.net> References: <20050527222507.4125C27B52@code1.codespeak.net> <20050528001608.GA25645@code1.codespeak.net> <20050528112840.GA3514@code1.codespeak.net> <20050528113823.GA3659@code1.codespeak.net> Message-ID: <20050528220559.GA8553@code1.codespeak.net> Hi, On Sat, May 28, 2005 at 01:38:23PM +0200, Armin Rigo wrote: > So I am thinking about introducing a "high-level operation" concept and > changing the signature of the rtype_xyz() methods: Done. It's slightly clearer and a lot less magical now... I guess I left a host of typos behind me, especially in rint.py and rfloat.py. Sorry about that. (Tests! tests!) A bientot, Armin From mansuk at gmail.com Sun May 29 20:34:18 2005 From: mansuk at gmail.com (Suman Karumuri) Date: Mon, 30 May 2005 00:04:18 +0530 Subject: [pypy-dev] Introduction Message-ID: Hi all, I am suman from india. I have completed my B.E(bachelors degree) in Computer science and inf. technology. I am currently working as a software programmer for ADP. I am highly interested in programming language research and can code in most of the imperative OO/procedural languages. I have used python to write soem scripts for the company i work for and i use it on a day to day basis to generate repetitive SQL code. My skillset includes, c/c++/java/python. In near future i want to implement a useful and successful visual programming language and so want to get a hands my hands dirty with a programming language implementation and hence chose pypy. I do not mind working on any part of the code, but as i can contribute my weekends i cannot work on peices which are needed for immediete development. -Suman From hpk at trillke.net Sun May 29 21:21:16 2005 From: hpk at trillke.net (holger krekel) Date: Sun, 29 May 2005 21:21:16 +0200 Subject: [pypy-dev] dev meeting 2nd June 3pm (CEST) Message-ID: <20050529192116.GG5694@solar.trillke.net> Hi pypy-dev, it makes sense to discuss a few issuses regarding future development work (apart from the very nice and successful translation meeting we had last week). I suggest to meet Thursday, 2nd June, 3pm (CEST == GMT+1) on #pypy at irc.freenode.net. Here are suggested topics: - sprint announcement/themes: although times and city are fixed already, the actual sprint topics for the post-europython sprint from 1st July - 7th July in Goetheborg (Sweden) are not yet decided and announced. However, almost all active developers will come and it's already safe to reserve the time and book your flights if you want to attend a newcomer-friendly sprint! - release planning: there is the idea to do a PyPy-0.6.2 around EuroPython or even as the finishing event of the public EuroPython sprint. Goals for this possible release need some discussion, though. Here is a suggestion: - move our lib-python base to 2.4.1 - make documentation more complete (e.g. interpreter/ docs are still missing) - integrate parser module (possibly making it RPython conformant) (- maybe offer a small tool to translate small RPython programs) - giving the translation efforts a stable base: One problem with ongoing development of PyPy is that the translation process is still somewhat fragile with respect to type inference (annotation) on our source tree. Therefore, it would be good if translation could for some time rely on a non-changing PyPy base. One simple idea is to fork off translation work in a branch, although i'd like to avoid that if possible. For starters, documentation changes in the branch would not be immediately seen on the website. Any other _simple_ ideas? - Advancing Issue Tracking: we probably want to decide how we proceed with issue tracking and especially classifications of issues. There is the simple idea of assigning an 'easy', 'medium' and 'hard' value to each issue to make it easier for newcomers and for ourselves. Moreover, if we go for a 0.6.2 release i think we should redesign the tracker schema to use 'release' instead of 'milestone'. - news/infos: there are news regarding a guy who is likely to enter the EU/PyPy project and who comes from the Squeak (Smalltalk mostly implemented in Smalltalk) world. Just so you know. - anything else i forgot (please post before instead of suggesting it at the meeting -> better planning) cheers & see you, holger From bert at impara.de Sun May 29 22:47:13 2005 From: bert at impara.de (Bert Freudenberg) Date: Sun, 29 May 2005 22:47:13 +0200 Subject: [pypy-dev] dev meeting 2nd June 3pm (CEST) In-Reply-To: <20050529192116.GG5694@solar.trillke.net> References: <20050529192116.GG5694@solar.trillke.net> Message-ID: <8CBF45B0-6D35-47D1-865B-1268E231C1A7@impara.de> Am 29.05.2005 um 21:21 schrieb holger krekel: > I suggest to meet > > Thursday, 2nd June, 3pm (CEST == GMT+1) > > on #pypy at irc.freenode.net. You mean GMT+2, so that would be 13:00 GMT? > - news/infos: > > there are news regarding a guy who is likely to enter > the EU/PyPy project and who comes from the Squeak (Smalltalk > mostly implemented in Smalltalk) world. Just so you know. That would be me, thanks for the introduction :-) I'll try to be at the chat, and in G?teborg, too. If someone wants to get an overview why Squeak might be interesting for PyPy, read this (old) paper: http://users.ipa.net/~dwighth/squeak/oopsla_squeak.html Cheers! - Bert - From hpk at trillke.net Sun May 29 23:02:34 2005 From: hpk at trillke.net (holger krekel) Date: Sun, 29 May 2005 23:02:34 +0200 Subject: [pypy-dev] dev meeting 2nd June 3pm (CEST) In-Reply-To: <8CBF45B0-6D35-47D1-865B-1268E231C1A7@impara.de> References: <20050529192116.GG5694@solar.trillke.net> <8CBF45B0-6D35-47D1-865B-1268E231C1A7@impara.de> Message-ID: <20050529210234.GH5694@solar.trillke.net> Hi Bert! On Sun, May 29, 2005 at 22:47 +0200, Bert Freudenberg wrote: > Am 29.05.2005 um 21:21 schrieb holger krekel: > > >I suggest to meet > > > > Thursday, 2nd June, 3pm (CEST == GMT+1) > > > >on #pypy at irc.freenode.net. > > You mean GMT+2, so that would be 13:00 GMT? hum, let's say it's 3pm german time :-) I always get confused about daylight saving. I think it is actually GMT+2 at the moment. > >- news/infos: > > > > there are news regarding a guy who is likely to enter > > the EU/PyPy project and who comes from the Squeak (Smalltalk > > mostly implemented in Smalltalk) world. Just so you know. > > That would be me, thanks for the introduction :-) I would have mentioned your name and more background but wasn't sure if you'd like publicity :-) > I'll try to be at > the chat, and in G?teborg, too. great! > If someone wants to get an overview why Squeak might be interesting > for PyPy, read this (old) paper: > > http://users.ipa.net/~dwighth/squeak/oopsla_squeak.html Interesting. Maybe we can make a comparison table about the difference in approaches. For example, squeak translation is source-to-source based while PyPy's approach reuses its own bytecode-interpreter to produce a control flow graph on which it then performs type inference. The main advantage is that the flow graph does not depend on particular bytecodes or language constructs. It merely records all operations on application objects and re-constructs the program flow. But you probably know all this already :-) cheers, holger From jacob at strakt.com Mon May 30 02:57:57 2005 From: jacob at strakt.com (Jacob Hallen) Date: Mon, 30 May 2005 02:57:57 +0200 Subject: [pypy-dev] dev meeting 2nd June 3pm (CEST) In-Reply-To: <20050529192116.GG5694@solar.trillke.net> References: <20050529192116.GG5694@solar.trillke.net> Message-ID: <200505300257.57322.jacob@strakt.com> s?ndag 29 maj 2005 21.21 skrev holger krekel: > Hi pypy-dev, > > it makes sense to discuss a few issuses regarding future > development work (apart from the very nice and successful > translation meeting we had last week). I suggest to meet > > Thursday, 2nd June, 3pm (CEST == GMT+1) I'll be in London and busy in meetings. However, I think Samuele will be quite competent to handle matters from the Strakt and G?teborg point of view. Jacob From hpk at trillke.net Mon May 30 08:44:12 2005 From: hpk at trillke.net (holger krekel) Date: Mon, 30 May 2005 08:44:12 +0200 Subject: [pypy-dev] dev meeting 2nd June 3pm (CEST) In-Reply-To: <200505300257.57322.jacob@strakt.com> References: <20050529192116.GG5694@solar.trillke.net> <200505300257.57322.jacob@strakt.com> Message-ID: <20050530064412.GL5694@solar.trillke.net> Hi Jacob, On Mon, May 30, 2005 at 02:57 +0200, Jacob Hallen wrote: > s?ndag 29 maj 2005 21.21 skrev holger krekel: > > Hi pypy-dev, > > > > it makes sense to discuss a few issuses regarding future > > development work (apart from the very nice and successful > > translation meeting we had last week). I suggest to meet > > > > Thursday, 2nd June, 3pm (CEST == GMT+1) > > I'll be in London and busy in meetings. However, I think Samuele will be quite > competent to handle matters from the Strakt and G?teborg point of view. Don't worry. I should have mentioned more prominently that this technical meeting is intended for PyPy developers irrespective of relations to the PyPy/EU project. Usually it should be fine if some of us keep the EU perspective in mind and we otherwise have open dev-meetings. cheers, holger From arigo at tunes.org Mon May 30 11:57:08 2005 From: arigo at tunes.org (Armin Rigo) Date: Mon, 30 May 2005 11:57:08 +0200 Subject: [pypy-dev] Re: [pypy-svn] r12876 - pypy/dist/lib-python/modified-2.3.4/encodings In-Reply-To: <20050529223823.64BD527BCE@code1.codespeak.net> References: <20050529223823.64BD527BCE@code1.codespeak.net> Message-ID: <20050530095708.GA29706@code1.codespeak.net> Hi Ale, On Mon, May 30, 2005 at 12:38:23AM +0200, ale at codespeak.net wrote: > - encode = codecs.escape_encode > - decode = codecs.escape_decode > + encode = staticmethod(codecs.escape_encode) > + decode = staticmethod(codecs.escape_decode) Sorry, I missed an older checkin (r11820) where you did something similar. There must be another way than modifying the encodings package (as you hint in the check-in message). Some time ago, for precisely this reason, we introduced a way to define functions that aren't bound in the normal Python way, but behave like CPython's builtin functions: def escape_encode( obj,errors='strict'): """None """ s = repr(obj) v = s[1:-1] return v,len(v) escape_encode = types.BuiltinFunctionType(escape_encode) I'm still unsure if it's a good solution to use it all over the place in _codecs.py because it's PyPy-specific. Another solution is to keep _codecs.py clean of this, but put the whole module in pypy/module/_codecs/app__codecs.py instead of pypy/lib/. This way, all functions exported by the _codecs module become "builtin functions" automatically. (It also allows you to hide parts you don't want to export, like the codec_search_path and codec_search_cache registry.) The advantage is that app__codecs.py by itself is still usable by non-PyPy projects. A bientot, Armin