From pedronis at strakt.com Tue Feb 1 18:47:00 2005 From: pedronis at strakt.com (Samuele Pedroni) Date: Tue, 01 Feb 2005 18:47:00 +0100 Subject: [pypy-dev] Summary of Leysin Sprint Message-ID: <41FFC094.7090006@strakt.com> Hello, This is a summary of what was accomplished and worked on during last sprint in Leysin. Leysin is a village in the Swiss Alps, home to Armin Rigo. The views of the surrounding mountains were spectacular. It seemed to inspire us to work harder. Thanks to Armin for organizing it: * All project participants met and worked together for the first time. We introduced newcomers to the project to the PyPy code base and the development process. We gave introductions to the basic functioning and the structure of the code base. py.test our unit testing framework was introduced to those who hadn't seen it before. * We also got to know our Process Manager, Bea D?ring from Changemaker. Bea will produce her own report. If you don't know the difference between 'project management', 'people management', and 'process management' then you probably would enjoy learning the differences from Bea. Bea interviewed the sprint participants to find out about their expectations, dissemination and more general ideas about the PyPy project and where it should go in the future. This Sprint went extremely well, but future ones should go even better, since we are learning how to improve all the time. Bea is making a report that discusses this more. * We made plans for new and improved infrastructure. Thank you svn, we can change our repository layout whenever we feel it would be a good idea. :-) But as for technical changes to pypy itself: * The flow object space now uses variable names derived from the original python ones and propagate them as possible. (Armin, Olivier Dormond) * The typedunwrap branch has been merged back to the trunk. (Armin, Samuele) Type specific unwraps (int_w, str_w, ...) should be used everywhere possible. The type generic unwrap should be used only in tests, at bootstrap time and or temporarily in cases where we are borrowing heavily from underlying CPython, because it is annotator unfriendly and is able to unwrap mixed types containers which are not reasonably RPythonic. * Updated documentation to describe new type specific unwrap operations. (Michael, Samuele) * We can now import and run the CPython tests on top of PyPy. This found lots of small problems, and a few not so small ones. So we made lots of small fixes, and implemented missing stuff, which was a good way for newcomers to get to know the codebase. We added or completed functionality to sys module: - _getframe() - set/getrecursionlevel() - exc_clear() ... (Adrien Di Mascio, Anders Chrigstr?m, Armin, Christian, Holger, Jacek Generowicz, Ludovic Aubry, Olivier, Samuele, Laura) * Work on the annotator: - SomeBuiltin.call - implemented annotation for some built-ins that were missing - SomeSlice support and annotating some ops involving them - iter built-in - dict methods support for SomeDict - preserve instance types when unifying with None - dict object dummy value is a W_Root subclass instance, so as not to confuse the annotator - taught the annotator how to deduce the type of e in try: ... except SomeException, e: ... <-- here - removal of most *args call sites used in the multi-method slicing mechanism, like what was done for gateways. The annotator no longer chokes on such call sites. :-) - wrote a small test case to show the remaining problems with the current setup of BuiltinCode and BuiltinFrame subclasses and the highly polymorphic BuiltinCode.func callable attribute. (Adrien, Michael, Samuele, some design discussion with Armin) * Added shortcuts is_w(.,.) (= space.is_true(space.is_(.,.))), eq_w(.,.) (= space.is_true(space.eq(.,.))) to base object space. (Adrien) * Allow arguments to be named in the stdobjspace's type's methods. As a first example, this enables list.sort(cmp=...). (Armin) * Generated app-level exceptions module and from there using geninterplevel an interp-level module (in particular this circumvent bootstrap issues). (Christian) This implementation approach for built-in modules should be polished and generalized. * Added python library as of 2.3.4 to our tree as dist/lib-python-2.3.4. After discussion, it was agreed that this should be left unchanged and modules needing modifications should be copied to pypy/lib (renamed from appspace) and modified there, also new pypy-specific app-level modules should live there. That means that as general rule we should copy and borrow from this specific Python version, in the not too distant future we will switch to Python 2.4. Pypy specific modified or new app-level tests should go to pypy/lib/test2. (Holger) Work is needed on how those should be run, we would like to be able to run them both on PyPy and on CPython. * py.test support for unittest and output-comparing tests has been added to dist/lib-python-2.3.4 (this is a special exception to the above rules), as conftest.py and pypy_unittest.py in dist/lib-python-2.3.4/test. Look at conftest.py for details, there are lists of passing tests there (Anders, Holger) * Work on our sio based file implementation. (Armin did a major rework) * Support for __del__ on user-defined classes. For now borrowing functionality from underlying CPython. (Armin) * Work has been started on an app-level implementation for old-style classes, we have a good chunk of the implementation but it is not that much tested nor integrated for now. (Christian, Samuele) * Marcus Denker gave a presentation on Squeak, its continuations based web framework Seaside and Croquet. We discussed whether PyPy is a sensible approach to give in the future comprehensive Python support as scripting language to Squeak/Croquet. * We set tasks for the next few weeks, and now begin to prepare for the next Sprint, before PyCon. regards. Samuele and Laura PS: I (Samuele) did the listing of tech changes, sorry if I missed someone's credits. From hpk at trillke.net Tue Feb 1 23:41:31 2005 From: hpk at trillke.net (holger krekel) Date: Tue, 1 Feb 2005 23:41:31 +0100 Subject: [pypy-dev] Re: PyPy pure python crypt module In-Reply-To: <41FFFEDC.4090901@dcube-resource.be> References: <41FFFEDC.4090901@dcube-resource.be> Message-ID: <20050201224131.GR9235@solar.trillke.net> Hello Jan, On Tue, Feb 01, 2005 at 23:12 +0100, Jan Dries wrote: > Hello, > > I noticed on http://www.python.org/moin/PyPySprint that you are looking > for a pure Python implementation of a number of modules, including the > crypt module. Note that for the crypt module there is such an > implementation available already: > > http://home.clear.net.nz/pages/c.evans/sw/ thanks for pointing that out! I have CCed pypy-dev with this information. The above crypt implementor notes that the code has some problem with int/longs which sounds familiar with PyPy :-) cheers, holger From florian.proff.schulze at gmx.net Wed Feb 2 13:03:41 2005 From: florian.proff.schulze at gmx.net (Florian Schulze) Date: Wed, 02 Feb 2005 13:03:41 +0100 Subject: [pypy-dev] Re: Summary of Leysin Sprint References: <41FFC094.7090006@strakt.com> Message-ID: On Tue, 01 Feb 2005 18:47:00 +0100, Samuele Pedroni wrote: > Hello, > > This is a summary of what was accomplished and worked on during last > sprint > in Leysin. Leysin is a village in the Swiss Alps, home to Armin Rigo. > The > views of the surrounding mountains were spectacular. It seemed to > inspire us > to work harder. Thanks to Armin for organizing it: [snip] > > regards. > Samuele and Laura > > > PS: I (Samuele) did the listing of tech changes, sorry if I missed > someone's credits. Thanks for this report. It's really nice to have an overview of the development of pypy. I think something like this is missing in some other (big) opensource projects. Keep up the great work, Florian Schulze From tinuviel at sparcs.kaist.ac.kr Wed Feb 2 16:27:47 2005 From: tinuviel at sparcs.kaist.ac.kr (Seo Sanghyeon) Date: Thu, 3 Feb 2005 00:27:47 +0900 Subject: [pypy-dev] xrange object start/stop/step attribute Message-ID: <20050202152747.GA19343@sparcs.kaist.ac.kr> I wanted to confirm that PyPy's applevel tests are indeed passing in CPython, and discovered this: pypy/module/test/test_builtin.py test_xrange_args tests start/stop/step attribute of xrange object. But these attributes got deprecated in 2.2, and is not present in 2.3 or 2.4... Well, that test is still useful though. By the way, is there already a tool to run applevel tests on CPython? Seo Sanghyeon From jacob at strakt.com Wed Feb 2 20:20:16 2005 From: jacob at strakt.com (Jacob =?iso-8859-1?q?Hall=E9n?=) Date: Wed, 2 Feb 2005 20:20:16 +0100 Subject: [pypy-dev] Pycon sprint - some logistics Message-ID: <200502022020.16420.jacob@strakt.com> Here are some loose notes for the Pycon sprint. Please send me comments on things you want to know and things I may have missed mentioning. The following people are known to come: Jacob Hall?n Anders Chrigstr?m Samuele Pedroni All 3 are booked on SAS, GOT-CPH-IAD 18 March leaving 07:20, arriving 15:20 BWI-KEF-CPH-GOT, leaving 25 March 20:15, arriving 26 March 15:05 IAD=Washington Dulles BWI=Baltimore International Airport KEF=Keflavik International Airport Holger Krekel Armin Rigo Christian Tismer Bea During These people will not attend: Adrien Di Mascio Ludovic Aubry Michael Hudson Attendance status of the following people is not yet known: Laura Creighton DFKI representative Anders Lehmann Alex Martelli Travel Special offers from KLM and Lufthansa have ended. SAS has a special offer that terminates 2005-02-28. Jacob, Samuele and Anders are booked on this one. Many dates are already blotted out, so book quickly if you want to use this offer. Transit IAD-Washington DC: Public bus: $3.00 (exact fare required) Runs once every hour. Leaves Dulles at XX.30 - XX.40, depending on hour of the day and weekday. Afternoon of 18 march, it will be XX.40. This bus has no special luggage racks, so many people with luggage is a problem. Take the bus to Roslyn, where you can walk to Key Bridge Hotel and Iwo Jima. You can also take the Metro to any place in DC. Washington Flyer Coach: $8.00 About twice per hour. XX.15 and XX.45. Goes to West Falls Church Metro Station, from where you can take the metro to Roslyn, or any other place in DC. Taxi: $45-50 plus tip Hotels: Currently our first choice is Best Western Key Bridge Hotel. This is situated close to Roslyn, which is a Metro station 2 stops from the conference venue. There is a late night store at the Metro and the hotel is clean, simple and relatively cheap. Second choice is Iwo Jima, which is a little further from Roslyn, but still a rather short walk away. We are waiting for people to rspond to Lauras inquiries about a youth hostel, but we should time out on this in a few days. Transit: Washington DC has a nicely working subway system. You buy a pre-paid ticket for some amount of cash. I suggest putting $10 on the card from the start. You insert your card in the gate when entering a station and you do the same when leaving. Depending on how far you have travelled and if it rush-hour or not, the machine will deduct what the fare is. Water: American tap water tastes like swimming-pool. Buy bottled water. The store just outside Roslyn (by the stop for the airport bus) carries water. Buy the gallon bottles of the cheapest brand. Food: There are many food options by the conference venue. On the ground floor and in the basement of the conference building, there is a selection of fast-food. It is cheap and it is garbage. However, it is probably not quite as bad as the conference lunches. If you leave the conference building and go up to Pensylvania Ave. Turn left and go half a block. There is a chinese fast-food place that actually does good food. There is a good sushi restaurant about 4 blocks away from the conference venue. Cross Pensylvania Ave. Turn directly right and go 3 blocks down the street. Just across the street from the conference venue is a mall with restaurants. There is a place that does sandwiches, which are rather good, but quite expensive. There is also Kinkeads in the mall, which is considered to be the #1 gourmet restaurant in DC. They actually have decent food, and compared to quality, this is not all that expensive. Finally, way out on the edge of town is a really nice Japanese restaurant. Very good sushi and very good Japanese dishes in general. It is located on McArthur Boulevard. Time off: If you have any time off in daytime, the Smithsonian Institutions are just a few Metro stops away. They are all for free, so you can simply slip in for an hour and look at their collections. Air&Space Museum is a must for everyone. The National Gallery is strongly recommended by me. Natural History and Ethnographics are probably interesting for some. The Whitehouse, The Congress, the Library of Congress and the Supreme court are also in the neighbourhood, as are the war memorials and a bunch of other stuff. Things to bring: - Beamer - they usually have some, but not enough to go around - Printer - they don't have any of those - Tea and cofee production equipment (110 Volts) - this is a problem - Plug converters - 240 Volts whatever you have -> Mickey Mouse voltage - Ear plugs - you may end up sleeping close to the ice machine, or the hotel elevator, or they may decide to hack up the concrete pavement next to your hotel. Network access usually works very well. This year they seem to be fully wireless. Other details: You must have a machine readable passport to be allowed into the US. They will take your picture and your finger prints on entry to the country. From hpk at trillke.net Wed Feb 2 22:36:20 2005 From: hpk at trillke.net (holger krekel) Date: Wed, 2 Feb 2005 22:36:20 +0100 Subject: [pypy-dev] Re: [pypy-svn] r8804 - in pypy/dist: lib-python-2.3.4 pypy/lib/test2 pypy/module In-Reply-To: <20050202130826.8635727BB1@code1.codespeak.net> References: <20050202130826.8635727BB1@code1.codespeak.net> Message-ID: <20050202213620.GU9235@solar.trillke.net> Hi Adim, On Wed, Feb 02, 2005 at 14:08 +0100, adim at codespeak.net wrote: > Author: adim > Date: Wed Feb 2 14:08:26 2005 > New Revision: 8804 > > Added: > pypy/dist/lib-python-2.3.4/itertools.py Added or modified modules should go to pypy/lib. Basically we want to keep the lib-python tree as clean as possible (only conftest.py and pypy_unittest.py are there as mentioned in Samuele's Leysin-Report mail). cheers, holger From hpk at trillke.net Wed Feb 2 22:37:49 2005 From: hpk at trillke.net (holger krekel) Date: Wed, 2 Feb 2005 22:37:49 +0100 Subject: [pypy-dev] Re: [pypy-svn] r8804 - in pypy/dist: lib-python-2.3.4 pypy/lib/test2 pypy/module In-Reply-To: <20050202213620.GU9235@solar.trillke.net> References: <20050202130826.8635727BB1@code1.codespeak.net> <20050202213620.GU9235@solar.trillke.net> Message-ID: <20050202213749.GV9235@solar.trillke.net> On Wed, Feb 02, 2005 at 22:36 +0100, holger krekel wrote: > On Wed, Feb 02, 2005 at 14:08 +0100, adim at codespeak.net wrote: > > Author: adim > > Date: Wed Feb 2 14:08:26 2005 > > New Revision: 8804 > > > > Added: > > pypy/dist/lib-python-2.3.4/itertools.py > > Added or modified modules should go to pypy/lib. > Basically we want to keep the lib-python tree > as clean as possible (only conftest.py and pypy_unittest.py > are there as mentioned in Samuele's Leysin-Report mail). Hi Adrien, Hehe, forget it. I read your next commit. sorry for the spam :-) holger From tismer at stackless.com Thu Feb 3 01:52:13 2005 From: tismer at stackless.com (Christian Tismer) Date: Thu, 03 Feb 2005 01:52:13 +0100 Subject: [pypy-dev] Flow problems Message-ID: <420175BD.4080308@stackless.com> Hi Armin, I tried to generate old-style classes interp-level. First problem: flow space has no w_dict. Well, I did a surrogate. As a follow-up, augmented flow-space barfed at unpackiterable without explicit length. I tried harder and undefined flow's unpackiterable. Furthermore, I had to add w_TypeError. This both was for sure bad, since now default's unpackiterable was in place, and in turn, flow tried to unwrap a non- -constant, causing File "D:\pypy\dist\pypy\objspace\flow\objspace.py", line 128, in unwrap raise UnwrapException So, in a way, the things I'm doing do not make sense. In a way, using flow for app2interp, poses extra demands on flow for which it was not designed. The question is, how hard is it to really add var-keywords for this case? It is necessary, and I would love to add it, but I don' understand what goes wrong, here. Thanks for advice -- chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Thu Feb 3 02:29:06 2005 From: tismer at stackless.com (Christian Tismer) Date: Thu, 03 Feb 2005 02:29:06 +0100 Subject: [pypy-dev] even more flow probs Message-ID: <42017E62.6050908@stackless.com> Hi Armin, I undid all my patching, but patched _classobj to have no **kwds any longer. After that, flow ran infinitely on this file. Virtual mem grew to over 500 MB. After 15 minutes, I interrupted the process. The traceback was: """ Traceback (most recent call last): File "D:\Python23\lib\site-packages\Pythonwin\pywin\framework\scriptutils.py", line 310, in RunScript exec codeObject in __main__.__dict__ File "D:\pypy\dist\pypy\translator\geninterplevel.py", line 1333, in ? gen.gen_source(fname, ftmpname) File "D:\pypy\dist\pypy\translator\geninterplevel.py", line 725, in gen_source self.gen_source_temp() File "D:\pypy\dist\pypy\translator\geninterplevel.py", line 788, in gen_source_temp self.gen_rpyfunction(func) File "D:\pypy\dist\pypy\translator\geninterplevel.py", line 842, in gen_rpyfunction body = list(self.rpyfunction_body(func, locals)) File "D:\pypy\dist\pypy\translator\geninterplevel.py", line 959, in rpyfunction_body SpaceClass=AugmentedFlowSpace) File "D:\pypy\dist\pypy\translator\translator.py", line 77, in getflowgraph graph = space.build_flow(func) File "D:\pypy\dist\pypy\objspace\flow\objspace.py", line 174, in build_flow ec.build_flow() File "D:\pypy\dist\pypy\objspace\flow\flowcontext.py", line 208, in build_flow block.patchframe(frame, self) File "D:\pypy\dist\pypy\objspace\flow\flowcontext.py", line 43, in patchframe replaylist) File "D:\pypy\dist\pypy\objspace\flow\flowcontext.py", line 49, in __init__ def __init__(self, listtoreplay, nextblock, booloutcome, nextreplaylist): KeyboardInterrupt >>> """ HEEEEEEELP, please! -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From cfbolz at gmx.de Thu Feb 3 03:59:27 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 3 Feb 2005 03:59:27 +0100 (MET) Subject: [pypy-dev] Hi! Message-ID: <13257.1107399567@www53.gmx.net> Hi pypy-list! I've [1] been following the pypy-dev mailing list for quite some time now and am really exited about this project. This weekend I checked out the code and started to play around with it a bit. Since there has been some talk about adding a LLVM backend and since this doesn't seem to have happened I decided to take a stab at it. I installed LLVM (which really is a pain), read the LLVM documentation and started to write a (very rudimentary) genllvm.py. It can already generate LLVM-assembler for simple functions (e.g. just ints, no function calls, no default arguments...). Then a Pyrex-wrapper for the functions is generated so that they can imported. For the function snippet.my_gcd the following LLVM-assembler code is generated: int %my_gcd(int %a_2, int %b_3) { block0: %r_7 = call int %mod(int %a_2, int %b_3) br label %block1 block3: %a_29 = phi int [%a_8, %block1] %b_30 = phi int [%b_9, %block1] %r_31 = phi int [%r_10, %block1] %v32 = phi bool [%v11, %block1] %r_21 = call int %mod(int %b_30, int %r_31) br label %block1 block2: %v4 = phi int [%b_9, %block1] ret int %v4 block1: %a_8 = phi int [%a_2, %block0], [%b_30, %block3] %b_9 = phi int [%b_3, %block0], [%r_31, %block3] %r_10 = phi int [%r_7, %block0], [%r_21, %block3] %v11 = call bool %is_true(int %r_10) br bool %v11, label %block3, label %block2 } Note that this exactly mirrors the flowgraph. I just use function calls for all SpaceOperations (though some probably have to be special-cased later). It is not neccessary to rename these functions since LLVM considers functions to be different if their signatures differ. The implementation of these functions is: int %mod(int %a, int %b) { %r = rem int %a, %b ret int %r } bool %is_true(int %a) { %b = cast int %a to bool ret bool %b } LLVM omptimizes the above code to: int %my_gcd(int %a_2, int %b_3) { block0: %r.i = rem int %a_2, %b_3 br label %block1 block3: %r.i1 = rem int %b_9, %r_10 br label %block1 block2: ret int %b_9 block1: %b_9 = phi int [ %b_3, %block0 ], [ %r_10, %block3 ] %r_10 = phi int [ %r.i, %block0 ], [ %r.i1, %block3 ] %b.i = seteq int %r_10, 0 br bool %b.i, label %block2, label %block3 } This is then compiled to native code. At the moment I'm not using the LLVM-API to generate this code since it was simpler to just do the string shuffling in python than to wrap and learn the LLVM-API. In my opinion this can be extended to nearly all of Python's data types as long as the annotation succeeds. I cannot yet judge wether other things like classes, exception handling, garbage collection etc. will be easy but we shall see. As for the code: It is quite convoluted and ad-hoc, I need to clean it up, write some more tests (I already wrote some) and extend it a bit before it is fit for someone else to see. Should I just post it or apply for checkin rights? What do you all think? Does my approach make sense or are there some obvious problems that I didn't see. Regards, Carl Friedrich [1] To introduce myself shorly: My name is Carl Friedrich Bolz, I'm 21 and studying maths and physics in Heidelberg, Germany, currently in my 3rd semester. I've been using Python since four years, mostly for my own projects. In addition I did/am doing a bit of C/C++ programming, mostly for 3D-graphics and high energy physics data analysis. From hpk at trillke.net Thu Feb 3 09:46:08 2005 From: hpk at trillke.net (holger krekel) Date: Thu, 3 Feb 2005 09:46:08 +0100 Subject: [pypy-dev] Hi! In-Reply-To: <13257.1107399567@www53.gmx.net> References: <13257.1107399567@www53.gmx.net> Message-ID: <20050203084608.GA9235@solar.trillke.net> Hi Carl! On Thu, Feb 03, 2005 at 03:59 +0100, Carl Friedrich Bolz wrote: > I've [1] been following the pypy-dev mailing list for quite some time now > and am really exited about this project. This weekend I checked out the > code and started to play around with it a bit. Since there has been some > talk about adding a LLVM backend and since this doesn't seem to have > happened I decided to take a stab at it. I installed LLVM (which really > is a pain), read the LLVM documentation and started to write a (very > rudimentary) genllvm.py. It can already generate LLVM-assembler for > simple functions (e.g. just ints, no function calls, no default > arguments...). Then a Pyrex-wrapper for the functions is generated so > that they can imported. sounds good. Before i comment further a disclaimer: i think that Armin, Samuele, Michael or Christian or others can possibly better provide more in-depth comments since i haven't much worked on the current translator/annotator codebase. One of the obstacles regarding LLVM is indeed its installation process, as is often the case with large C++ codebases not packaged by the distributions. If we want to use LLVM more then we should try to provide supplemental installation instructions i guess. > For the function snippet.my_gcd the following LLVM-assembler code is > generated: > > int %my_gcd(int %a_2, int %b_3) { > block0: > %r_7 = call int %mod(int %a_2, int %b_3) > br label %block1 > block3: > %a_29 = phi int [%a_8, %block1] > %b_30 = phi int [%b_9, %block1] > %r_31 = phi int [%r_10, %block1] > %v32 = phi bool [%v11, %block1] > %r_21 = call int %mod(int %b_30, int %r_31) > br label %block1 > block2: > %v4 = phi int [%b_9, %block1] > ret int %v4 > block1: > %a_8 = phi int [%a_2, %block0], [%b_30, %block3] > %b_9 = phi int [%b_3, %block0], [%r_31, %block3] > %r_10 = phi int [%r_7, %block0], [%r_21, %block3] > %v11 = call bool %is_true(int %r_10) > br bool %v11, label %block3, label %block2 > } > > Note that this exactly mirrors the flowgraph. I just use function calls > for all SpaceOperations (though some probably have to be special-cased > later). It is not neccessary to rename these functions since LLVM > considers functions to be different if their signatures differ. > (... nice example ...) > This is then compiled to native code. At the moment I'm not using the > LLVM-API to generate this code since it was simpler to just do the string > shuffling in python than to wrap and learn the LLVM-API. hehe. > In my opinion this can be extended to nearly all of Python's data types > as long as the annotation succeeds. I cannot yet judge wether other things > like classes, exception handling, garbage collection etc. will be easy but > we shall see. Oh don't worry, the other backends don't care too much for this, either :-) The Pyrex and GenC still cooperate with the CPython runtime and borrow its garbage collection among other things. Once we target a standalone (without the CPython runtime) version garbage collection needs to be done. (Usually at this point sometime drops in the two words "Boehm collector" :-) Exceptions get analyzed by the flow space in a way that makes generation of low-level code rather straightforward. > As for the code: It is quite convoluted and ad-hoc, I need to clean it up, > write some more tests (I already wrote some) and extend it a bit before it > is fit for someone else to see. Should I just post it or apply for checkin > rights? Apply for checkin rights, i'd say. I guess you are aware of at least our coding-style document http://codespeak.net/pypy/index.cgi?doc/coding-style and of the fact that we generally want MIT-licensed (BSD-licensed) code If so, how about you send me privately your desired account name? > What do you all think? Does my approach make sense or are there some > obvious problems that I didn't see. I think it makes sense. It would be great to have you at one of our next sprints and further explore the LLVM backend, i think. Btw, Armin and Christian intend to do cleanup work on the translator backends and it is sensible to already have LLVM in mind. What i would like to find out is if we could use a stripped down version of LLVM because i also guess that many supplemental (code generation) tasks are better done in Python than with wrapping and using some of the LLVM API. I guess i am going to download and try-installing the thing again :-) cheers, holger From cfbolz at gmx.de Thu Feb 3 15:00:36 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 3 Feb 2005 15:00:36 +0100 (MET) Subject: [pypy-dev] Hi! References: <20050203084608.GA9235@solar.trillke.net> Message-ID: <14612.1107439236@www42.gmx.net> Hi Holger! On Thu Feb 3 09:46:08 MET 2005, Holger Krekel wrote: > One of the obstacles regarding LLVM is indeed its installation > process, as is often the case with large C++ codebases not > packaged by the distributions. If we want to use LLVM more > then we should try to provide supplemental installation > instructions i guess. I can provide provide you with some pointers (Maybe I should write these installation instructions at one point): One of the biggest problems for me was that LLVM needs a farely recent GCC-Version to even compile. I think they recommend GCC 3.3.3/3.4.0 or later -- earlyer versions give "Internal compiler errors" or somethin like that. If you have a recent version of GCC installing the main LLVM package is still relatively easy. What's more difficult is the cfrontend package which is needed if you want to be able to compile c/c++ code to LLVM. This package consists of the ripped-of GCC-frontends for these languages. A precompiled version of this package exists for some platform. If these don't work (which they did for me eventually) you have to compile cfrontend: Quote from the docs: """This is currently a somewhat fragile, error-prone process, and you should _only_ try to do it if: 1. you really, really, really can't use the binaries we distribute 2. you are an elite GCC hacker """ One the other hand is the cfrontend not needed to user LLVM. You need it only if you want to compile c-code with it which is not neccessary for my llvm-backend for pypy (but might become so in the future). >> In my opinion this can be extended to nearly all of Python's data types >> as long as the annotation succeeds. I cannot yet judge wether other things >> like classes, exception handling, garbage collection etc. will be easy but >> we shall see. > Oh don't worry, the other backends don't care too much for > this, either :-) The Pyrex and GenC still cooperate with the > CPython runtime and borrow its garbage collection among other > things. Once we target a standalone (without the CPython > runtime) version garbage collection needs to be done. (Usually > at this point sometime drops in the two words "Boehm collector" :-) > Exceptions get analyzed by the flow space in a way that makes > generation of low-level code rather straightforward. I don't know enough about the memory management of LLVM to decide whether we can use the "Boehm collector". As far as I see it, it is a c/c++-library which could be difficult to use from within LLVM. LLVM has some gc-hooks though (and I think even a very simple gc to show how to use these), maybe we can work with that. > I think it makes sense. It would be great to have you at one of our > next sprints and further explore the LLVM backend, i think. Btw, > Armin and Christian intend to do cleanup work on the translator > backends and it is sensible to already have LLVM in mind. I won't be able to go to PyCon but I hope I can make it to one of the sprints after that at PyCon. Is there some sprint planning that goes a bit beyond just the next one? I'm even thinking about organizing a sprint here in Heidelberg this summer/fall though I don't know whether this will fly. > What i would like to find out is if we could use a stripped > down version of LLVM because i also guess that many > supplemental (code generation) tasks are better done in Python > than with wrapping and using some of the LLVM API. I guess i am > going to download and try-installing the thing again :-) We can probably try not to use the API altogether. Then we would only need the command line tools. The APIs would be useful for something Psyco-like though as it is possible to emit code and do JIT-compilation , and use it on-the-fly without writing anything to a file first. Regards, Carl Friedrich From arigo at tunes.org Thu Feb 3 16:10:57 2005 From: arigo at tunes.org (Armin Rigo) Date: Thu, 3 Feb 2005 15:10:57 +0000 Subject: [pypy-dev] xrange object start/stop/step attribute In-Reply-To: <20050202152747.GA19343@sparcs.kaist.ac.kr> References: <20050202152747.GA19343@sparcs.kaist.ac.kr> Message-ID: <20050203151057.GA26215@vicky.ecs.soton.ac.uk> Hi Seo, On Thu, Feb 03, 2005 at 12:27:47AM +0900, Seo Sanghyeon wrote: > pypy/module/test/test_builtin.py test_xrange_args tests start/stop/step > attribute of xrange object. But these attributes got deprecated in 2.2, > and is not present in 2.3 or 2.4... Ah. I guess we shouldn't test for them, then... > By the way, is there already a tool to run applevel tests on CPython? No... Armin From jacob at strakt.com Thu Feb 3 16:44:39 2005 From: jacob at strakt.com (Jacob =?iso-8859-1?q?Hall=E9n?=) Date: Thu, 3 Feb 2005 16:44:39 +0100 Subject: [pypy-dev] Hi! In-Reply-To: <14612.1107439236@www42.gmx.net> References: <20050203084608.GA9235@solar.trillke.net> <14612.1107439236@www42.gmx.net> Message-ID: <200502031644.39741.jacob@strakt.com> On torsdag 3 februari 2005 15.00, Carl Friedrich Bolz wrote: > I won't be able to go to PyCon but I hope I can make it to one of the > sprints after that at PyCon. Is there some sprint planning that goes a > bit beyond just the next one? I'm even thinking about organizing a > sprint here in Heidelberg this summer/fall though I don't know whether > this will fly. Yes, these are the sprint plans so far: 8. 19-22 March, before Pycon, Washington DC, USA 9. 24 April - 1 May, after ACCU UK, Oxford, UK 10. 30 June - 7 July, after Europython, G?teborg, Sweden 11. End August/Beginning of September 12. October 13. December (somewhere warm, focused on reports for EU project) 14. February 2006 15. March 2006, probably close to Pycon 16. May 2006 17. June 2006, probably close to Europython (CERN?) 18. August 2006 19. October 2006 20. November 2006 (Focused on reports for EU projects) Dates are still quite flexible for sprints beyond sprint 10. If you are willing to host a sprint in Heidelberg, we should make sure that we can get all the logistics to work and then ask the team if they want to do a sprint in Heidelberg. I'm the contact point for logistics. Best regards Jacob Hall?n From lac at strakt.com Thu Feb 3 20:01:19 2005 From: lac at strakt.com (Laura Creighton) Date: Thu, 03 Feb 2005 20:01:19 +0100 Subject: [pypy-dev] Hi! In-Reply-To: Message from "Carl Friedrich Bolz" of "Thu, 03 Feb 2005 15:00:36 +0100." <14612.1107439236@www42.gmx.net> References: <20050203084608.GA9235@solar.trillke.net> <14612.1107439236@www42.gmx.net> Message-ID: <200502031901.j13J1Jee003868@ratthing-b246.strakt.com> In a message of Thu, 03 Feb 2005 15:00:36 +0100, "Carl Friedrich Bolz" writes: >I won't be able to go to PyCon but I hope I can make it to one of the >sprints after that at PyCon. Is there some sprint planning that goes a >bit beyond just the next one? I'm even thinking about organizing a >sprint here in Heidelberg this summer/fall though I don't know whether >this will fly. Plan for a Sprint after ACCU/PythonUK in April in Oxford, and after EuroPython in G?teborg, Sweden. exact dates not set yet. I am working on setting these up. Laura x From sanxiyn at gmail.com Fri Feb 4 02:17:38 2005 From: sanxiyn at gmail.com (Sanghyeon Seo) Date: Fri, 4 Feb 2005 10:17:38 +0900 Subject: [pypy-dev] Hi! In-Reply-To: <20050203084608.GA9235@solar.trillke.net> References: <13257.1107399567@www53.gmx.net> <20050203084608.GA9235@solar.trillke.net> Message-ID: <5b02481705020317174fd97f3@mail.gmail.com> hpk wrote: > One of the obstacles regarding LLVM is indeed its installation > process, as is often the case with large C++ codebases not > packaged by the distributions. If we want to use LLVM more > then we should try to provide supplemental installation > instructions i guess. LLVM is currently being packaged for Debian. See http://bugs.debian.org/239415 Preliminary package does work. 1.4 release is currently packaged. Here's the installation instruction: 1. Add "deb http://toolchain.org/~ahs3 /" to /etc/apt/sources.list 2. apt-get install llvm 3. Happy hacking! Seo Sanghyeon From davidf at sjsoft.com Fri Feb 4 09:20:16 2005 From: davidf at sjsoft.com (David Fraser) Date: Fri, 04 Feb 2005 10:20:16 +0200 Subject: [pypy-dev] datetime module Message-ID: <42033040.5040806@sjsoft.com> Hi I have been testing out pypy for the first time... I wanted to get the datetime module working and found that there was a pure python implementation of it done before the C implementation, in the nondist/sandbox/datetime module on Python CVS. This implementation works fairly well on pypy: - in order to test it, the attached patch to test_datetime is needed to avoid problems with pickle and gc - other than pickling, the only test failures are below The datetime.py can be got directly here: http://cvs.sourceforge.net/viewcvs.py/*checkout*/python/python/nondist/sandbox/datetime/datetime.py?content-type=text%2Fplain&rev=1.160 ====================================================================== ERROR: test_combine (__main__.TestDateTime) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_datetime.py", line 1396, in test_combine dt = combine(time=t, date=d) TypeError: newfunc() got 2 unexpected keyword arguments ====================================================================== ERROR: test_more_astimezone (__main__.TestDateTimeTZ) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_datetime.py", line 2636, in test_more_astimezone dt = self.theclass.now(tz=f44m) TypeError: newfunc() got an unexpected keyword argument 'tz' ====================================================================== ERROR: test_tzinfo_fromtimestamp (__main__.TestDateTimeTZ) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_datetime.py", line 2423, in test_tzinfo_fromtimestamp again = meth(ts, tz=off42) TypeError: newfunc() got an unexpected keyword argument 'tz' ====================================================================== ERROR: test_tzinfo_now (__main__.TestDateTimeTZ) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_datetime.py", line 2385, in test_tzinfo_now again = meth(tz=off42) TypeError: newfunc() got an unexpected keyword argument 'tz' ====================================================================== FAIL: test_subclass_timedelta (__main__.TestTimeDelta) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_datetime.py", line 469, in test_subclass_timedelta self.assert_(type(t3) is timedelta) File "/share/sjsoft/code/Python/pypy/dist-pypy/lib-python-2.3.4/unittest.py", line 278, in failUnless if not expr: raise self.failureException, msg AssertionError David From davidf at sjsoft.com Fri Feb 4 09:46:51 2005 From: davidf at sjsoft.com (David Fraser) Date: Fri, 04 Feb 2005 10:46:51 +0200 Subject: [pypy-dev] datetime module and classmethods with kwargs In-Reply-To: <42033040.5040806@sjsoft.com> References: <42033040.5040806@sjsoft.com> Message-ID: <4203367B.4090407@sjsoft.com> David Fraser wrote: > Hi > > I have been testing out pypy for the first time... > I wanted to get the datetime module working and found that there was a > pure python implementation of it done before the C implementation, in > the nondist/sandbox/datetime module on Python CVS. > This implementation works fairly well on pypy: > - in order to test it, the attached patch to test_datetime is needed > to avoid problems with pickle and gc > - other than pickling, the only test failures are below > The datetime.py can be got directly here: > http://cvs.sourceforge.net/viewcvs.py/*checkout*/python/python/nondist/sandbox/datetime/datetime.py?content-type=text%2Fplain&rev=1.160 > To speed up my testing I created a cutdown version of the non-pickle tests that failed :-) One of them fails in standard python as well so is a problem with the datetime.py code The others are all due to classmethod not accepting keyword arguments. I've attached a patch to __builtin__module.py that enables it to accept keyword arguments, which makes all those tests pass. Seems simple, could it be applied? David From rev_anna_r at yahoo.com Fri Feb 4 09:44:00 2005 From: rev_anna_r at yahoo.com (Anna Ravenscroft) Date: Fri, 4 Feb 2005 09:44:00 +0100 Subject: [pypy-dev] Pycon sprint - some logistics In-Reply-To: <200502022020.16420.jacob@strakt.com> References: <200502022020.16420.jacob@strakt.com> Message-ID: On Feb 2, 2005, at 20:20, Jacob Hall?n wrote: > Here are some loose notes for the Pycon sprint. Please send me > comments on > things you want to know and things I may have missed mentioning. > > The following people are known to come: > > Jacob Hall?n > Anders Chrigstr?m > Samuele Pedroni > All 3 are booked on SAS, > GOT-CPH-IAD 18 March leaving 07:20, arriving 15:20 > BWI-KEF-CPH-GOT, leaving 25 March 20:15, arriving 26 March 15:05 > IAD=Washington Dulles > BWI=Baltimore International Airport > KEF=Keflavik International Airport > > Holger Krekel > Armin Rigo > Christian Tismer > > Bea During > > These people will not attend: > > Adrien Di Mascio > Ludovic Aubry > Michael Hudson > > Attendance status of the following people is not yet known: > > Laura Creighton > DFKI representative > > Anders Lehmann > Alex Martelli Alex has booked with KLM (better than the SAS prices we could find from Italy). Do we submit receipts to you for the reimbursement or how does that work? cordially, Anna Martelli Ravenscroft From davidf at sjsoft.com Fri Feb 4 09:55:08 2005 From: davidf at sjsoft.com (David Fraser) Date: Fri, 04 Feb 2005 10:55:08 +0200 Subject: [pypy-dev] datetime module and classmethods with kwargs In-Reply-To: <4203367B.4090407@sjsoft.com> References: <42033040.5040806@sjsoft.com> <4203367B.4090407@sjsoft.com> Message-ID: <4203386C.4030104@sjsoft.com> David Fraser wrote: > David Fraser wrote: > >> Hi >> >> I have been testing out pypy for the first time... >> I wanted to get the datetime module working and found that there was >> a pure python implementation of it done before the C implementation, >> in the nondist/sandbox/datetime module on Python CVS. >> This implementation works fairly well on pypy: >> - in order to test it, the attached patch to test_datetime is needed >> to avoid problems with pickle and gc >> - other than pickling, the only test failures are below >> The datetime.py can be got directly here: >> http://cvs.sourceforge.net/viewcvs.py/*checkout*/python/python/nondist/sandbox/datetime/datetime.py?content-type=text%2Fplain&rev=1.160 >> > > > To speed up my testing I created a cutdown version of the non-pickle > tests that failed :-) > One of them fails in standard python as well so is a problem with the > datetime.py code > The others are all due to classmethod not accepting keyword arguments. > I've attached a patch to __builtin__module.py that enables it to > accept keyword arguments, which makes all those tests pass. > > Seems simple, could it be applied? Of course attaching the patch as a patch would give too much away, so I have to send it as text/plain :-) David -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: __builtin__classmethod_kwargs.txt URL: From davidf at sjsoft.com Fri Feb 4 10:04:04 2005 From: davidf at sjsoft.com (David Fraser) Date: Fri, 04 Feb 2005 11:04:04 +0200 Subject: [pypy-dev] datetime module In-Reply-To: <42033040.5040806@sjsoft.com> References: <42033040.5040806@sjsoft.com> Message-ID: <42033A84.20208@sjsoft.com> David Fraser wrote: > Hi > > I have been testing out pypy for the first time... > I wanted to get the datetime module working and found that there was a > pure python implementation of it done before the C implementation, in > the nondist/sandbox/datetime module on Python CVS. > This implementation works fairly well on pypy: > - in order to test it, the attached patch to test_datetime is needed > to avoid problems with pickle and gc > - other than pickling, the only test failures are below > The datetime.py can be got directly here: > http://cvs.sourceforge.net/viewcvs.py/*checkout*/python/python/nondist/sandbox/datetime/datetime.py?content-type=text%2Fplain&rev=1.160 > > > ====================================================================== > FAIL: test_subclass_timedelta (__main__.TestTimeDelta) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "test_datetime.py", line 469, in test_subclass_timedelta > self.assert_(type(t3) is timedelta) > File > "/share/sjsoft/code/Python/pypy/dist-pypy/lib-python-2.3.4/unittest.py", > line 278, in failUnless > if not expr: raise self.failureException, msg > AssertionError This test fails in standard Python as well ... the basic problem is the datetime.py produced a derived type T when adding two T's together, the test expects it to produce a standard datetime.timedelta objects. I've attached a patch (as plain text) that makes the result of arithmetic operations between datetime.timedelta objects standard datetime.timedelta objects. Not sure why this is a good thing, or if it should be done to the rest of the datetime code (its not tested for) - there are a few other places where self.__class__ is used to generate a derived result With this patch and the classmodule patch all the non-pickle tests pass on my pypy. David PS this should check if text/x-patch mime types are now getting through to the list too. -------------- next part -------------- A non-text attachment was scrubbed... Name: datetime-timedelta-fixedclass.patch Type: text/x-patch Size: 1479 bytes Desc: not available URL: From rev_anna_r at yahoo.com Fri Feb 4 10:41:59 2005 From: rev_anna_r at yahoo.com (Anna Ravenscroft) Date: Fri, 4 Feb 2005 10:41:59 +0100 Subject: [pypy-dev] datetime module In-Reply-To: <42033040.5040806@sjsoft.com> References: <42033040.5040806@sjsoft.com> Message-ID: <3dd25d9d8dbd060dc633285dbe73fd18@yahoo.com> On Feb 4, 2005, at 9:20, David Fraser wrote: > Hi > > I have been testing out pypy for the first time... > I wanted to get the datetime module working and found that there was a > pure python implementation of it done before the C implementation, in > the nondist/sandbox/datetime module on Python CVS. > This implementation works fairly well on pypy: > - in order to test it, the attached patch to test_datetime is needed > to avoid problems with pickle and gc > - other than pickling, the only test failures are below > The datetime.py can be got directly here: > http://cvs.sourceforge.net/viewcvs.py/*checkout*/python/python/ > nondist/sandbox/datetime/datetime.py?content- > type=text%2Fplain&rev=1.160 > Awesome! I'll take a look imcst but, datetime is one of my favorite modules. Thanks for working on it! Anna From davidf at sjsoft.com Fri Feb 4 10:55:09 2005 From: davidf at sjsoft.com (David Fraser) Date: Fri, 04 Feb 2005 11:55:09 +0200 Subject: [pypy-dev] datetime module In-Reply-To: <3dd25d9d8dbd060dc633285dbe73fd18@yahoo.com> References: <42033040.5040806@sjsoft.com> <3dd25d9d8dbd060dc633285dbe73fd18@yahoo.com> Message-ID: <4203467D.1080600@sjsoft.com> Anna Ravenscroft wrote: > > On Feb 4, 2005, at 9:20, David Fraser wrote: > >> Hi >> >> I have been testing out pypy for the first time... >> I wanted to get the datetime module working and found that there was >> a pure python implementation of it done before the C implementation, >> in the nondist/sandbox/datetime module on Python CVS. >> This implementation works fairly well on pypy: >> - in order to test it, the attached patch to test_datetime is needed >> to avoid problems with pickle and gc >> - other than pickling, the only test failures are below >> The datetime.py can be got directly here: >> http://cvs.sourceforge.net/viewcvs.py/*checkout*/python/python/ >> nondist/sandbox/datetime/datetime.py?content- >> type=text%2Fplain&rev=1.160 >> > > Awesome! I'll take a look imcst but, datetime is one of my favorite > modules. Thanks for working on it! Sure, almost all the work had already been done for me :-) David From arigo at tunes.org Fri Feb 4 12:01:37 2005 From: arigo at tunes.org (Armin Rigo) Date: Fri, 4 Feb 2005 11:01:37 +0000 Subject: [pypy-dev] datetime module and classmethods with kwargs In-Reply-To: <4203386C.4030104@sjsoft.com> References: <42033040.5040806@sjsoft.com> <4203367B.4090407@sjsoft.com> <4203386C.4030104@sjsoft.com> Message-ID: <20050204110136.GA19156@vicky.ecs.soton.ac.uk> Hi David, On Fri, Feb 04, 2005 at 10:55:08AM +0200, David Fraser wrote: > >Seems simple, could it be applied? Sure, done. Thanks for the fix, and don't hesitate to ask for check-in rights if you're doing more of them :-) Armin From davidf at sjsoft.com Fri Feb 4 12:54:06 2005 From: davidf at sjsoft.com (David Fraser) Date: Fri, 04 Feb 2005 13:54:06 +0200 Subject: [pypy-dev] datetime module and classmethods with kwargs In-Reply-To: <20050204110136.GA19156@vicky.ecs.soton.ac.uk> References: <42033040.5040806@sjsoft.com> <4203367B.4090407@sjsoft.com> <4203386C.4030104@sjsoft.com> <20050204110136.GA19156@vicky.ecs.soton.ac.uk> Message-ID: <4203625E.1060204@sjsoft.com> Armin Rigo wrote: >Hi David, > >On Fri, Feb 04, 2005 at 10:55:08AM +0200, David Fraser wrote: > > >>>Seems simple, could it be applied? >>> >>> >Sure, done. Thanks for the fix, and don't hesitate to ask for check-in rights >if you're doing more of them :-) > > Wow, cool. Let me see what happens... David From tinuviel at sparcs.kaist.ac.kr Fri Feb 4 18:07:34 2005 From: tinuviel at sparcs.kaist.ac.kr (Seo Sanghyeon) Date: Sat, 5 Feb 2005 02:07:34 +0900 Subject: [pypy-dev] Some more tests passing Message-ID: <20050204170734.GA6068@localhost.localdomain> As of r8877, some more CPython regression tests are passing, unmodified. test_textwrap is passing in 393 seconds. test_difflib is passing in 205 seconds. test_pprint is passing in 318 seconds. Probably more. For comparison, my machine benchmarks at 21276 pystones. Why don't you reply to this message, if you got some more tests passing? Seo Sanghyeon From sanxiyn at gmail.com Fri Feb 4 18:58:23 2005 From: sanxiyn at gmail.com (Sanghyeon Seo) Date: Sat, 5 Feb 2005 02:58:23 +0900 Subject: [pypy-dev] Some more tests passing In-Reply-To: <20050204170734.GA6068@localhost.localdomain> References: <20050204170734.GA6068@localhost.localdomain> Message-ID: <5b02481705020409584c53485d@mail.gmail.com> I wrote: > Why don't you reply to this message, if you got some more tests passing? And now with r8788: test_sgmllib is passing in 133 seconds. test_htmllib is passing in 6 seconds. test_htmlparser is passing in 796 seconds. Time is from what unittest reports. And it doesn't include PyPy startup and "faking" times -- that explains "6 seconds" of test_htmllib. More to come... From sanxiyn at gmail.com Fri Feb 4 18:59:26 2005 From: sanxiyn at gmail.com (Sanghyeon Seo) Date: Sat, 5 Feb 2005 02:59:26 +0900 Subject: [pypy-dev] Some more tests passing In-Reply-To: <5b02481705020409584c53485d@mail.gmail.com> References: <20050204170734.GA6068@localhost.localdomain> <5b02481705020409584c53485d@mail.gmail.com> Message-ID: <5b0248170502040959a200506@mail.gmail.com> I wrote, by mistake: > And now with r8788: I meant r8878... From arigo at tunes.org Sat Feb 5 12:35:38 2005 From: arigo at tunes.org (Armin Rigo) Date: Sat, 5 Feb 2005 11:35:38 +0000 Subject: [pypy-dev] Some more tests passing In-Reply-To: <20050204170734.GA6068@localhost.localdomain> References: <20050204170734.GA6068@localhost.localdomain> Message-ID: <20050205113538.GA16728@vicky.ecs.soton.ac.uk> Hi Seo, On Sat, Feb 05, 2005 at 02:07:34AM +0900, Seo Sanghyeon wrote: > As of r8877, some more CPython regression tests are passing, unmodified. Cool ! > Why don't you reply to this message, if you got some more tests passing? There is a place in lib-python-2.3.4/test/conftest.py where we can list the tests that pass. The idea is that someone with a lot of time before him could type 'py.test' in this directory and check that all the tests that used to pass still pass. A bient?t, Armin From lac at strakt.com Sat Feb 5 14:12:06 2005 From: lac at strakt.com (Laura Creighton) Date: Sat, 05 Feb 2005 14:12:06 +0100 Subject: [pypy-dev] Some more tests passing In-Reply-To: Message from Armin Rigo of "Sat, 05 Feb 2005 11:35:38 GMT." <20050205113538.GA16728@vicky.ecs.soton.ac.uk> References: <20050204170734.GA6068@localhost.localdomain> <20050205113538.GA16728@vicky.ecs.soton.ac.uk> Message-ID: <200502051312.j15DC6MJ019478@theraft.strakt.com> In a message of Sat, 05 Feb 2005 11:35:38 GMT, Armin Rigo writes: >Hi Seo, > >On Sat, Feb 05, 2005 at 02:07:34AM +0900, Seo Sanghyeon wrote: >> As of r8877, some more CPython regression tests are passing, unmodified >. > >Cool ! > >> Why don't you reply to this message, if you got some more tests passing >? > >There is a place in lib-python-2.3.4/test/conftest.py where we can list t >he >tests that pass. The idea is that someone with a lot of time before him >could >type 'py.test' in this directory and check that all the tests that used t >o >pass still pass. > > >A bient?t, > >Armin At some point I want to build a 'how to run a CPython test, fix any bugs' for the 60% goal for PyCon. (Assuming Seo doesn't fix them all :-) ) Is this what you want people officially to do? Laura From bhartsho at yahoo.com Sun Feb 6 09:46:29 2005 From: bhartsho at yahoo.com (Hart's Antler) Date: Sun, 6 Feb 2005 00:46:29 -0800 (PST) Subject: [pypy-dev] Using PyPy as a compiler Message-ID: <20050206084629.42318.qmail@web30810.mail.mud.yahoo.com> Hi, i am new to PyPy, i'm hoping someone can help me get started. I want to use translator to turn some python code into a pyx files so that i can compile them and later load them as modules in my program. I did a simple test and i was able to get some pyrex code from the translator, but when i saved that to a pyx file and called 'pyrexc mymodule.pyx' on it there were errors. I noticed there is a Pyrex folder (a Pyrex fork?) in the PyPy project, but i'm not what to use in there to convert mymodule.pyx to a .c file so gcc can compile it. btw, is todays current version in svn working? I get alot of errors when calling translator.view(), i'm doing the test.is_perfect_number tutorial. thanks for PyPy, -brett >>> t.view() Traceback (most recent call last): File "", line 1, in ? File "translator.py", line 112, in view FlowGraphPage(self).display() File "/root/imp/dist-pypy/pypy/translator/tool/graphpage.py", line 36, in display get_layout(self).display() File "/root/imp/dist-pypy/pypy/translator/tool/pygame/graphclient.py", line 60, in get_layout return InProcessConnexion().download(homepage) File "/root/imp/dist-pypy/pypy/translator/tool/pygame/graphclient.py", line 45, in download return ClientGraphLayout(self, key, page.source, page.links) File "/root/imp/dist-pypy/pypy/translator/tool/pygame/graphclient.py", line 22, in __init__ cmdexec('dot -Tplain %s>%s' % (str(DOT_FILE),str(PLAIN_FILE))) File "/root/imp/dist-pypy/py/process/cmdexec.py", line 66, in posix_exec_cmd ''.join(out), ''.join(err)) py.__impl__.process.cmdexec.ExecutionFailed: ExecutionFailed: 127 dot -Tplain /tmp/usession-4/graph.dot>/tmp/usession-4/graph.plain /bin/sh: line 1: dot: command not found __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From hpk at trillke.net Sun Feb 6 09:57:22 2005 From: hpk at trillke.net (holger krekel) Date: Sun, 6 Feb 2005 09:57:22 +0100 Subject: [pypy-dev] Using PyPy as a compiler In-Reply-To: <20050206084629.42318.qmail@web30810.mail.mud.yahoo.com> References: <20050206084629.42318.qmail@web30810.mail.mud.yahoo.com> Message-ID: <20050206085722.GC9235@solar.trillke.net> Hi Brett, On Sun, Feb 06, 2005 at 00:46 -0800, Hart's Antler wrote: > Hi, i am new to PyPy, i'm hoping someone can help me get started. > > I want to use translator to turn some python code into a pyx files so that i can compile them and > later load them as modules in my program. I did a simple test and i was able to get some pyrex > code from the translator, but when i saved that to a pyx file and called 'pyrexc mymodule.pyx' on > it there were errors. I noticed there is a Pyrex folder (a Pyrex fork?) in the PyPy project, but > i'm not what to use in there to convert mymodule.pyx to a .c file so gcc can compile it. The control flow at the lower level really likes jumping around and thus we are using a slightly modified version of Pyrex which supports GOTOs or rather generic "CINLINE" statement IIRC :-) You might try to use pypy/tool/pyrexc mymodule.pyx which uses our modified versions of Pyrex. > btw, is todays current version in svn working? I get alot of errors when calling > translator.view(), i'm doing the test.is_perfect_number tutorial. ... > py.__impl__.process.cmdexec.ExecutionFailed: ExecutionFailed: 127 dot -Tplain > /tmp/usession-4/graph.dot>/tmp/usession-4/graph.plain > /bin/sh: line 1: dot: command not found You need to install graphviz (available on most distributions) which comes with the 'dot' commandline tool. Then this should nicely work. have fun, holger From arigo at tunes.org Sun Feb 6 11:55:29 2005 From: arigo at tunes.org (Armin Rigo) Date: Sun, 6 Feb 2005 10:55:29 +0000 Subject: [pypy-dev] Some more tests passing In-Reply-To: <200502051312.j15DC6MJ019478@theraft.strakt.com> References: <20050204170734.GA6068@localhost.localdomain> <20050205113538.GA16728@vicky.ecs.soton.ac.uk> <200502051312.j15DC6MJ019478@theraft.strakt.com> Message-ID: <20050206105529.GA13382@vicky.ecs.soton.ac.uk> Hi Laura, On Sat, Feb 05, 2005 at 02:12:06PM +0100, Laura Creighton wrote: > At some point I want to build a 'how to run a CPython test, fix any bugs' for > the 60% goal for PyCon. (Assuming Seo doesn't fix them all :-) ) > Is this what you want people officially to do? Yes :-) Here are some hints: cd lib-python-2.3.4/test/ py.test test_somestuff.py This runs the test using PyPy's testing machinery. If you suspect that py.test might not do the correct thing, you can also try: py.py test_somestuff.py which is the immediate equivalent of "python test_somestuff.py", which just runs the test without all of py.test's integration (and without speed optimizations either). Tests that pass should be listed in lib-python-2.3.4/test/conftest.py. Some tests really check for some dark internal details of CPython and have no chance to pass unmodified. In this case it is possible to copy and modify them for PyPy: svn cp lib-python-2.3.4/test/test_darkcorner.py pypy/lib/test2/ Then you can fix the test in pypy/lib/test2/. Note that a number of tests need such small tweaks because old-style classes are not integrated in PyPy by default, so the classes defined by the test are new-style ones, although the test expects them to behave like old-style ones. I guess that such tests can be temporarily fixed if it's easy, otherwise just ignored. Armin From bhartsho at yahoo.com Sun Feb 6 14:31:37 2005 From: bhartsho at yahoo.com (Hart's Antler) Date: Sun, 6 Feb 2005 05:31:37 -0800 (PST) Subject: [pypy-dev] Using PyPy as a compiler In-Reply-To: <20050206085722.GC9235@solar.trillke.net> Message-ID: <20050206133137.86957.qmail@web30804.mail.mud.yahoo.com> Hi Holger, I tried pyrexc in pypy/tool but i get the same error as with the normal pyrexc: bash-2.05b# dist-pypy/pypy/tool/pyrexc test.pyx /root/imp/test.pyx:5:17: Declarator should be empty /root/imp/test.pyx:5:17: Expected '>' The pyrex generated code looks like this: def hi(): return hi__2ad67a3c() cdef object hi__2ad67a3c(): # untyped variables: v0 v13 v21 v3 v7 v3 = () v7 = ('hello world', v3) v13 = () v21 = (v13) v0 = None return v0 which was generated from a function that looks like this: def hi(): return 'hello world' Cheers, -brett --- holger krekel wrote: > Hi Brett, > > On Sun, Feb 06, 2005 at 00:46 -0800, Hart's Antler wrote: > > Hi, i am new to PyPy, i'm hoping someone can help me get started. > > > > I want to use translator to turn some python code into a pyx files so that i can compile them > and > > later load them as modules in my program. I did a simple test and i was able to get some > pyrex > > code from the translator, but when i saved that to a pyx file and called 'pyrexc mymodule.pyx' > on > > it there were errors. I noticed there is a Pyrex folder (a Pyrex fork?) in the PyPy project, > but > > i'm not what to use in there to convert mymodule.pyx to a .c file so gcc can compile it. > > The control flow at the lower level really likes jumping around > and thus we are using a slightly modified version of Pyrex which > supports GOTOs or rather generic "CINLINE" statement IIRC :-) > > You might try to use pypy/tool/pyrexc mymodule.pyx which uses > our modified versions of Pyrex. > > > btw, is todays current version in svn working? I get alot of errors when calling > > translator.view(), i'm doing the test.is_perfect_number tutorial. > > ... > > > py.__impl__.process.cmdexec.ExecutionFailed: ExecutionFailed: 127 dot -Tplain > > /tmp/usession-4/graph.dot>/tmp/usession-4/graph.plain > > /bin/sh: line 1: dot: command not found > > You need to install graphviz (available on most distributions) which > comes with the 'dot' commandline tool. Then this should nicely work. > > have fun, > > holger > __________________________________ Do you Yahoo!? Yahoo! Mail - Helps protect you from nasty viruses. http://promotions.yahoo.com/new_mail From hpk at trillke.net Sun Feb 6 14:34:43 2005 From: hpk at trillke.net (holger krekel) Date: Sun, 6 Feb 2005 14:34:43 +0100 Subject: [pypy-dev] Using PyPy as a compiler In-Reply-To: <20050206133137.86957.qmail@web30804.mail.mud.yahoo.com> References: <20050206085722.GC9235@solar.trillke.net> <20050206133137.86957.qmail@web30804.mail.mud.yahoo.com> Message-ID: <20050206133443.GH9235@solar.trillke.net> On Sun, Feb 06, 2005 at 05:31 -0800, Hart's Antler wrote: > Hi Holger, > > I tried pyrexc in pypy/tool but i get the same error as with the normal pyrexc: > bash-2.05b# dist-pypy/pypy/tool/pyrexc test.pyx > /root/imp/test.pyx:5:17: Declarator should be empty > /root/imp/test.pyx:5:17: Expected '>' > > The pyrex generated code looks like this: > def hi(): > return hi__2ad67a3c() > cdef object hi__2ad67a3c(): > # untyped variables: v0 v13 v21 v3 v7 > v3 = () > v7 = ('hello world', v3) > v13 = () > v21 = (v13) > v0 = None > return v0 > > > which was generated from a function that looks like this: > def hi(): return 'hello world' You don't happen to translate a print statement somewhere, do you? holger From adimasci at gmail.com Sun Feb 6 15:10:45 2005 From: adimasci at gmail.com (Adrien Di Mascio) Date: Sun, 6 Feb 2005 15:10:45 +0100 Subject: [pypy-dev] Using PyPy as a compiler In-Reply-To: <20050206133137.86957.qmail@web30804.mail.mud.yahoo.com> References: <20050206085722.GC9235@solar.trillke.net> <20050206133137.86957.qmail@web30804.mail.mud.yahoo.com> Message-ID: <437de85005020606107bb574cd@mail.gmail.com> Hi Brett, On Sun, 6 Feb 2005 05:31:37 -0800 (PST), Hart's Antler wrote: > The pyrex generated code looks like this: > def hi(): > return hi__2ad67a3c() > cdef object hi__2ad67a3c(): > # untyped variables: v0 v13 v21 v3 v7 > v3 = () > v7 = ('hello world', v3) [...] > which was generated from a function that looks like this: > def hi(): return 'hello world' Your original function probably has a print statement and print statements, AFAIK, are not properly translated for now. This is a "known" issue, however I'm not sure someone is actively looking at how to fix it. Cheers, Adrien. From cfbolz at gmx.de Sun Feb 6 17:53:11 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sun, 6 Feb 2005 17:53:11 +0100 (MET) Subject: [pypy-dev] LLVM backend Message-ID: <20224.1107708791@www55.gmx.net> Hi pypy-list! I just checked my LLVM-backend in (I hope I did nothing wrong). It resides in pypy/translator/llvm. It is still a bit rough but should work for most functions that use just ints and bools (I started addint some list and string support but at the moment only lists of length one work and strings are not tested at all). To try it out use the function 'llvmcompile' in the module pypy.translator.llvm.genllvm on an annotated flowgraph: %~/pypy/translator/llvm> python -i genllvm.py >>> t = Translator(test.my_gcd) >>> a = t.annotate([int, int]) >>> f = llvmcompile(t) >>> f(10, 2) 2 >>> You need to have the LLVM executables (llvm-as, llc, llvmc) in your path, as well as 'as' and gcc. The LLVM c/c++ frontend is not needed for genllvm to work (which eases installation a bit). llvmcompile tries to produce LLVM-assembly out the entry function of the translator and of all the functions the entry function calls. The assembly is then optimized by LLVM and native code (a .o file) is generated. Additionally the LLVM backend produces a pyrex wrapper for the generated function to be able to use the LLVM produced version from python. Since all the dynamic features of Python are not supported yet (and since the LLVM optimizers seem to be very good) it is not surprising that the produced functions are pretty fast (faster than C? ;-). How genllvm works: For every function, constant and variable of the flowgraph an object is generated that knows how representate the object in LLVM. These LLVMRepr-objects can have dependencies on other LLVMRepr objects, for example a variable depends on the representation of its type, a function depends on all its variables and constants... (this seems to be a bit similar to the nameof* functions of genc and genjava but I'm not shure of that) To generate LLVM-code this dependenciy-tree is walked depth-first and every object is asked for its global declarations and then for the functions it needs. For example in the global declaration phase the used types would return their own declaration. In the function phase the used types return the space-ops that are defined for them and the functions return their own llvm-code. The code generation for the functions to be translated takes place in the FunctionRepr class. This is quite straightforward since the structure of flowgraphs corresponds very well to the control flow mechanisms of LLVM - for example direct support for phi nodes. So far this works quite well but a lot of issues remain open: - Does my approach makes sense at all? I never did anything even remotely similar so I might be doing lots of stupid things. - I think there should be some more intelligent way to produce the necessary LLLVM-implementations for the space operations of more complex types than just writing them in LLVM-assembler, which can be quite tedious (it's no fun writing programs in SSA form). - List and Strings should be relatively easy to implement with arrays. I'm not quite shure wether I manage to do it, I'll just ask questions if I run into problems. - Are tuples really only used for returning multiple values from a function? If yes they could be avoided altogether using additional pointer arguments that point to where the return value should be stored. - I don't know how exactly 'interned strings' work in CPython so I don't know what to do with dicts yet. - Classes, GC, exceptions and all that... Regards, Carl Friedrich From hpk at trillke.net Sun Feb 6 19:01:17 2005 From: hpk at trillke.net (holger krekel) Date: Sun, 6 Feb 2005 19:01:17 +0100 Subject: [pypy-dev] LLVM backend In-Reply-To: <20224.1107708791@www55.gmx.net> References: <20224.1107708791@www55.gmx.net> Message-ID: <20050206180116.GN9235@solar.trillke.net> Hi Carl, On Sun, Feb 06, 2005 at 17:53 +0100, Carl Friedrich Bolz wrote: > I just checked my LLVM-backend in (I hope I did nothing wrong). It resides > in pypy/translator/llvm. It is still a bit rough but should work for most > functions that use just ints and bools (I started addint some list and > string support but at the moment only lists of length one work and strings > are not tested at all). cool! So I just installed llvm-1.4 (on gcc-3.3.4, gentoo i386) without frontends and tried to run the tests. But there is apparently some odd bytecode "corruption" going on ... > To try it out use the function 'llvmcompile' in the module > pypy.translator.llvm.genllvm on an annotated flowgraph: > %~/pypy/translator/llvm> python -i genllvm.py > >>> t = Translator(test.my_gcd) > >>> a = t.annotate([int, int]) > >>> f = llvmcompile(t) e.g. here i get: <... traceback ...> ExecutionFailed: 1 llc _my_gcd_wrap.o.bc -f -o _my_gcd_wrap.s llc: bytecode didn't read correctly any idea what could be the problem here? cheers, holger From cfbolz at gmx.de Sun Feb 6 21:44:13 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sun, 6 Feb 2005 21:44:13 +0100 (MET) Subject: [pypy-dev] LLVM backend References: <20050206180116.GN9235@solar.trillke.net> Message-ID: <23505.1107722653@www69.gmx.net> Hi Holger, > So I just installed llvm-1.4 (on gcc-3.3.4, gentoo i386) without > frontends and tried to run the tests. But there is apparently some odd > bytecode "corruption" going on ... [...] > <... traceback ...> > ExecutionFailed: 1 llc _my_gcd_wrap.o.bc -f -o _my_gcd_wrap.s > llc: bytecode didn't read correctly That sure seems strange because it means that LLVM liked the assembler created by my code, but not its own optimized bytecode. Would you send me the content of your last /tmp/usession-? folder? The tests probably don't work either, right? Regards, Carl Friedrich From hpk at trillke.net Sun Feb 6 21:56:35 2005 From: hpk at trillke.net (holger krekel) Date: Sun, 6 Feb 2005 21:56:35 +0100 Subject: [pypy-dev] LLVM backend In-Reply-To: <23505.1107722653@www69.gmx.net> References: <20050206180116.GN9235@solar.trillke.net> <23505.1107722653@www69.gmx.net> Message-ID: <20050206205635.GP9235@solar.trillke.net> On Sun, Feb 06, 2005 at 21:44 +0100, Carl Friedrich Bolz wrote: > Hi Holger, > > > So I just installed llvm-1.4 (on gcc-3.3.4, gentoo i386) without > > frontends and tried to run the tests. But there is apparently some odd > > bytecode "corruption" going on ... > [...] > > <... traceback ...> > > ExecutionFailed: 1 llc _my_gcd_wrap.o.bc -f -o _my_gcd_wrap.s > > llc: bytecode didn't read correctly > > That sure seems strange because it means that LLVM liked the assembler > created by my code, but not its own optimized bytecode. Would you send me > the content of your last /tmp/usession-? folder? I have copied it to codespeak.net:/home/hpk/usession-126 . cheers, holger From arigo at tunes.org Sun Feb 6 21:53:07 2005 From: arigo at tunes.org (Armin Rigo) Date: Sun, 6 Feb 2005 20:53:07 +0000 Subject: [pypy-dev] LLVM backend In-Reply-To: <20224.1107708791@www55.gmx.net> References: <20224.1107708791@www55.gmx.net> Message-ID: <20050206205307.GA16820@vicky.ecs.soton.ac.uk> Hi Carl, On Sun, Feb 06, 2005 at 05:53:11PM +0100, Carl Friedrich Bolz wrote: > - Does my approach makes sense at all? I never did anything even remotely > similar so I might be doing lots of stupid things. I believe it makes a lot of sense, yes. For writing the implementation of individual operations, instead of assembler, there might be a way to write them in some other language and compile them with an LLVM compiler. I don't know if we would have to do it in C and install the C front-end, or if there is some simpler C-like language "natively" supported by LLVM. But in both cases such an approach might be helpful. A bientot, Armin From pypy at lypanov.net Sun Feb 6 22:28:02 2005 From: pypy at lypanov.net (Alexander Kellett) Date: Sun, 6 Feb 2005 22:28:02 +0100 Subject: [pypy-dev] LLVM backend In-Reply-To: <20050206205307.GA16820@vicky.ecs.soton.ac.uk> References: <20224.1107708791@www55.gmx.net> <20050206205307.GA16820@vicky.ecs.soton.ac.uk> Message-ID: <090f8a6d5162fcaa3044568fa7b6dd1c@lypanov.net> hi carl / arigo, could i ask of a quick favour? i'm interested in the performance of : d=0 1000000000.times { d = d + 1 } puts d as in, just a loop that goes from d=0 to d=1000000000 and prints d out at the end. for some stupid reason a few people have decided to benchmark this with mono and ruby and i wanted to show them the llvm and psyco results so they could weep ;) thanks! Alex On Feb 6, 2005, at 9:53 PM, Armin Rigo wrote: > Hi Carl, > > On Sun, Feb 06, 2005 at 05:53:11PM +0100, Carl Friedrich Bolz wrote: >> - Does my approach makes sense at all? I never did anything even >> remotely >> similar so I might be doing lots of stupid things. > > I believe it makes a lot of sense, yes. For writing the > implementation of > individual operations, instead of assembler, there might be a way to > write > them in some other language and compile them with an LLVM compiler. I > don't > know if we would have to do it in C and install the C front-end, or if > there > is some simpler C-like language "natively" supported by LLVM. But in > both > cases such an approach might be helpful. > > > A bientot, > > Armin > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From cfbolz at gmx.de Sun Feb 6 23:34:05 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sun, 6 Feb 2005 23:34:05 +0100 (MET) Subject: [pypy-dev] LLVM backend References: <090f8a6d5162fcaa3044568fa7b6dd1c@lypanov.net> Message-ID: <25050.1107729245@www49.gmx.net> Hi Alex, On 6 Feb 2005 22:28:02 +0100, Alexander Kellett wrote: > i'm interested in the performance of : > d=0 > 1000000000.times { > d = d + 1 > } > puts d > as in, just a loop that goes from d=0 to d=1000000000 and prints d out > at the end. this wouldn't prove anything since LLVM will just omptimize everything away. It would (if it were already possible to use a print statement) lead to something which is equivalent to puts 1000000000 and would not provide any performance measurement. Regards, Carl Friedrich From cfbolz at gmx.de Mon Feb 7 01:38:07 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Mon, 7 Feb 2005 01:38:07 +0100 (MET) Subject: [pypy-dev] LLVM backend References: <20050206205635.GP9235@solar.trillke.net> Message-ID: <32476.1107736687@www47.gmx.net> Hi Holger, I might have found the error though I'm not sure. I checken in a fix and added some debug output, could you try again and post it if it fails? Regards, Carl Friedrich From bhartsho at yahoo.com Mon Feb 7 03:25:39 2005 From: bhartsho at yahoo.com (Hart's Antler) Date: Sun, 6 Feb 2005 18:25:39 -0800 (PST) Subject: [pypy-dev] Using PyPy as a compiler In-Reply-To: <437de85005020606107bb574cd@mail.gmail.com> Message-ID: <20050207022539.73540.qmail@web30806.mail.mud.yahoo.com> Hi, your right my function did have a print statement in there. I fixed that and it worked! Is there a doc that lists the limitations translator has? Now i am trying to compile some other functions but having another problem. pyrexc gives this error: mymodule.pyx:11:12: Syntax error in simple statement list that line from mymodule is: cinline "Label2:" I'm trying to compile a bunch of other functions to, and they all error on lines that look like that one, and always line 11:12. Here is the python code: def moveNode(self, url): parent = self.xpath(url)[0] self.parentNode.removeChild(self) parent.appendChild(self) return 'Thanks, finished moving node.' move node, and the other functions having the problems all need to use this function as well, i think the problem is somewhere in here: def _resolveAttribute(self, name): if name is 'parentNode': if type(self._parentNode) is types.DictType: self.materialize(self._parentNode) return self._parentNode elif name is 'childNodes': for child in self._childNodes: if type(child) is types.DictType: self.materialize(child) return self._childNodes elif name is 'firstChild': if type(self._childNodes[0]) is types.DictType: self.materialize(self._childNodes[0]) return self._childNodes[0] elif name is 'lastChild': if type(self._childNodes[-1]) is types.DictType: self.materialize(self._childNodes[-1]) return self._childNodes[-1] else: raise AttributeError Pyrex translation: def moveNode(self_18577, url_18578): return moveNode__2da99144(self_18577, url_18578) cdef object moveNode__2da99144(object self_18577, object url_18578): # untyped variables: etype_18580 evalue_18581 parent_18592 parent_18692 parent_18697 parent_18703 parent_18710 parent_18718 parent_18727 parent_18737 parent_18748 parent_18758 parent_18767 parent_18775 self_18681 self_18684 self_18688 self_18693 self_18699 self_18706 self_18714 self_18723 self_18733 self_18744 self_18754 self_18763 self_18771 self_18778 self_18784 self_18790 self_18795 url_18682 url_18685 url_18689 url_18694 url_18700 url_18707 url_18715 url_18724 url_18734 url_18745 url_18755 url_18764 url_18772 url_18779 url_18785 url_18791 url_18796 v18579 v18582 v18586 v18600 v18613 v18625 v18639 v18655 v18683 v18686 v18687 v18690 v18691 v18695 v18696 v18698 v18701 v18702 v18704 v18705 v18708 v18709 v18711 v18712 v18713 v18716 v18717 v18719 v18720 v18721 v18722 v18725 v18726 v18728 v18729 v18730 v18731 v18732 v18735 v18736 v18738 v18739 v18740 v18741 v18742 v18743 v18746 v18747 v18749 v18750 v18751 v18752 v18753 v18756 v18757 v18759 v18760 v18761 v18762 v18765 v18766 v18768 v18769 v18770 v18773 v18774 v18776 v18777 v18780 v18781 v18782 v18783 v18786 v18787 v18788 v18789 v18792 v18793 v18794 v18797 v18798 try: v18582 = self_18577.xpath except AttributeError, last_exc_value: last_exception = last_exc_value.__class__ self_18795, url_18796, v18797, v18798 = self_18577, url_18578, last_exception, last_exc_value etype_18580, evalue_18581 = AttributeError, v18798 cinline "Label2:" raise etype_18580, evalue_18581 else: self_18681, url_18682, v18683 = self_18577, url_18578, v18582 try: v18586 = v18683(url_18682) except Exception, last_exc_value: last_exception = last_exc_value.__class__ self_18790, url_18791, v18792, v18793, v18794 = self_18681, url_18682, v18683, last_exception, last_exc_value etype_18580, evalue_18581 = v18793, v18794 cinline "goto Label2;" else: self_18684, url_18685, v18686, v18687 = self_18681, url_18682, v18683, v18586 try: parent_18592 = v18687[0] except IndexError, last_exc_value: last_exception = last_exc_value.__class__ self_18778, url_18779, v18780, v18781, v18782, v18783 = self_18684, url_18685, v18686, v18687, last_exception, last_exc_value etype_18580, evalue_18581 = IndexError, v18783 cinline "goto Label2;" except KeyError, last_exc_value: last_exception = last_exc_value.__class__ self_18784, url_18785, v18786, v18787, v18788, v18789 = self_18684, url_18685, v18686, v18687, last_exception, last_exc_value etype_18580, evalue_18581 = KeyError, v18789 cinline "goto Label2;" else: self_18688, url_18689, v18690, v18691, parent_18692 = self_18684, url_18685, v18686, v18687, parent_18592 try: v18600 = self_18688.parentNode except AttributeError, last_exc_value: last_exception = last_exc_value.__class__ self_18771, url_18772, v18773, v18774, parent_18775, v18776, v18777 = self_18688, url_18689, v18690, v18691, parent_18692, last_exception, last_exc_value etype_18580, evalue_18581 = AttributeError, v18777 cinline "goto Label2;" else: self_18693, url_18694, v18695, v18696, parent_18697, v18698 = self_18688, url_18689, v18690, v18691, parent_18692, v18600 try: v18613 = v18698.removeChild except AttributeError, last_exc_value: last_exception = last_exc_value.__class__ self_18763, url_18764, v18765, v18766, parent_18767, v18768, v18769, v18770 = self_18693, url_18694, v18695, v18696, parent_18697, v18698, last_exception, last_exc_value etype_18580, evalue_18581 = AttributeError, v18770 cinline "goto Label2;" else: self_18699, url_18700, v18701, v18702, parent_18703, v18704, v18705 = self_18693, url_18694, v18695, v18696, parent_18697, v18698, v18613 try: v18625 = v18705(self_18699) except Exception, last_exc_value: last_exception = last_exc_value.__class__ self_18754, url_18755, v18756, v18757, parent_18758, v18759, v18760, v18761, v18762 = self_18699, url_18700, v18701, v18702, parent_18703, v18704, v18705, last_exception, last_exc_value etype_18580, evalue_18581 = v18761, v18762 cinline "goto Label2;" else: self_18706, url_18707, v18708, v18709, parent_18710, v18711, v18712, v18713 = self_18699, url_18700, v18701, v18702, parent_18703, v18704, v18705, v18625 try: v18639 = parent_18710.appendChild except AttributeError, last_exc_value: last_exception = last_exc_value.__class__ self_18744, url_18745, v18746, v18747, parent_18748, v18749, v18750, v18751, v18752, v18753 = self_18706, url_18707, v18708, v18709, parent_18710, v18711, v18712, v18713, last_exception, last_exc_value etype_18580, evalue_18581 = AttributeError, v18753 cinline "goto Label2;" else: self_18714, url_18715, v18716, v18717, parent_18718, v18719, v18720, v18721, v18722 = self_18706, url_18707, v18708, v18709, parent_18710, v18711, v18712, v18713, v18639 try: v18655 = v18722(self_18714) except Exception, last_exc_value: last_exception = last_exc_value.__class__ self_18733, url_18734, v18735, v18736, parent_18737, v18738, v18739, v18740, v18741, v18742, v18743 = self_18714, url_18715, v18716, v18717, parent_18718, v18719, v18720, v18721, v18722, last_exception, last_exc_value etype_18580, evalue_18581 = v18742, v18743 cinline "goto Label2;" else: self_18723, url_18724, v18725, v18726, parent_18727, v18728, v18729, v18730, v18731, v18732 = self_18714, url_18715, v18716, v18717, parent_18718, v18719, v18720, v18721, v18722, v18655 v18579 = 'Thanks, finished moving node.' return v18579 --- Adrien Di Mascio wrote: > Hi Brett, > > On Sun, 6 Feb 2005 05:31:37 -0800 (PST), Hart's Antler > wrote: > > The pyrex generated code looks like this: > > def hi(): > > return hi__2ad67a3c() > > cdef object hi__2ad67a3c(): > > # untyped variables: v0 v13 v21 v3 v7 > > v3 = () > > v7 = ('hello world', v3) > [...] > > which was generated from a function that looks like this: > > def hi(): return 'hello world' > > Your original function probably has a print statement and print > statements, AFAIK, are not properly translated for now. This is a > "known" issue, however I'm not sure someone is actively looking at how > to fix it. > > Cheers, > Adrien. > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From hpk at trillke.net Mon Feb 7 08:34:48 2005 From: hpk at trillke.net (holger krekel) Date: Mon, 7 Feb 2005 08:34:48 +0100 Subject: [pypy-dev] Using PyPy as a compiler In-Reply-To: <20050207022539.73540.qmail@web30806.mail.mud.yahoo.com> References: <437de85005020606107bb574cd@mail.gmail.com> <20050207022539.73540.qmail@web30806.mail.mud.yahoo.com> Message-ID: <20050207073448.GR9235@solar.trillke.net> Hi Brett, On Sun, Feb 06, 2005 at 18:25 -0800, Hart's Antler wrote: > Hi, your right my function did have a print statement in there. I fixed that and it worked! Is > there a doc that lists the limitations translator has? > > Now i am trying to compile some other functions but having another problem. pyrexc gives this > error: > mymodule.pyx:11:12: Syntax error in simple statement list > that line from mymodule is: > cinline "Label2:" this is the "cinline" extension to pyrex, earlier mentioned in this thread. Did you really use dist-pypy/pypy/tool/pyrexc to compile your code? holger From hpk at trillke.net Mon Feb 7 08:46:19 2005 From: hpk at trillke.net (holger krekel) Date: Mon, 7 Feb 2005 08:46:19 +0100 Subject: [pypy-dev] LLVM backend In-Reply-To: <32476.1107736687@www47.gmx.net> References: <20050206205635.GP9235@solar.trillke.net> <32476.1107736687@www47.gmx.net> Message-ID: <20050207074619.GT9235@solar.trillke.net> Hi Carl, On Mon, Feb 07, 2005 at 01:38 +0100, Carl Friedrich Bolz wrote: > Hi Holger, > > I might have found the error though I'm not sure. I checken in a fix and > added some debug output, could you try again and post it if it fails? it still fails ... "py.test -x" in the translator/llvm directory gives: inserting into sys.path: /home/hpk/projects/dist-pypy ============================= test process starts ============================= testing-mode: inprocess executable : /usr/bin/python (2.3.4-final-0) using py lib: /home/hpk/projects/dist-pypy/py initial testconfig 0: /home/hpk/projects/dist-pypy/pypy/conftest.py/. initial testconfig 1: /home/hpk/projects/dist-pypy/py/test/defaultconfig.py/. =============================================================================== llvm__________test_genllvm.py[18] ...F _______________________________________________________________________________ def test_simple1(self): > f = compile_function(llvmsnippet.simple1, []) [/home/hpk/projects/dist-pypy/pypy/translator/llvm/test/test_genllvm.py:86] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ def compile_function(function, annotate): t = Translator(function) a = t.annotate(annotate) gen = LLVMGenerator(t) > return gen.compile() [/home/hpk/projects/dist-pypy/pypy/translator/llvm/test/test_genllvm.py:20] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ def compile(self, optimize=True): from pypy.tool.udir import udir name = self.l_entrypoint.llvmname()[1:] llvmfile = udir.join('%s.ll' % name) f = llvmfile.open('w') self.write(f) f.close() pyxfile = udir.join('%s_wrap.pyx' % name) f = pyxfile.open('w') f.write(self.l_entrypoint.get_pyrex_source()) f.close() mod = build_llvm_module.make_module_from_llvm(llvmfile, pyxfile, > optimize) [/home/hpk/projects/dist-pypy/pypy/translator/llvm/genllvm.py:68] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ def make_module_from_llvm(llvmfile, pyxfile, optimize=True): include_dir = autopath.this_dir dirpath = llvmfile.dirpath() lastdir = path.local() os.chdir(str(dirpath)) modname = pyxfile.purebasename ops1 = ["llvm-as %s -f -o %s.bc" % (llvmfile, llvmfile.purebasename), "llvmc -f -O3 %s.bc -o %s_optimized.o" % (llvmfile.purebasename, llvmfile.purebasename), "llc %s_optimized.o.bc -f -o %s.s" % (llvmfile.purebasename, llvmfile.purebasename), "as %s.s -o %s.o" % (llvmfile.purebasename, llvmfile.purebasename)] if not optimize: ops1 = ["llvm-as %s -f" % llvmfile, "llc %s.bc -f -o %s.s" % (llvmfile.purebasename, llvmfile.purebasename), "as %s.s -o %s.o" % (llvmfile.purebasename, llvmfile.purebasename)] ops2 = ["gcc -c -fPIC -I/usr/include/python %s.c" % pyxfile.purebasename, "gcc -shared %s.o %s.o -o %s.so" % (llvmfile.purebasename, modname, modname)] try: if debug: print "modname", modname c = stdoutcapture.Capture(mixed_out_err = True) if debug: print "working in", path.local() try: try: for op in ops1: print op E cmdexec(op) ~ ExecutionFailed: cmdexec('llc simple1_optimized.o.bc -f -o simple1.s') << ExecutionFailed: 1 llc simple1_optimized.o.bc -f -o simple1.s llc: bytecode didn't read correctly. [/home/hpk/projects/dist-pypy/pypy/translator/llvm/build_llvm_module.py:58] [testcode : /home/hpk/projects/dist-pypy/pypy/translator/llvm/test/test_genllvm.py:85] [modulepath: TestGenLLVM.test_simple1] - - - - - - - - - - - - - - - recorded stdout - - - - - - - - - - - - - - - modname simple1_wrap working in /tmp/usession-129 llvm-as /tmp/usession-129/simple1.ll -f -o simple1.bc llvmc -f -O3 simple1.bc -o simple1_optimized.o llc simple1_optimized.o.bc -f -o simple1.s _______________________________________________________________________________ ============= tests finished: 3 passed, 1 failed in 9.79 seconds ============= contents of usession-129 are on codespeak.net:/home/hpk/usession-129 i am beginning to suspect my installation. Is there an easy way to verify that my installation works? cheers, holger From hpk at trillke.net Mon Feb 7 08:52:30 2005 From: hpk at trillke.net (holger krekel) Date: Mon, 7 Feb 2005 08:52:30 +0100 Subject: [pypy-dev] LLVM backend In-Reply-To: <20050207074619.GT9235@solar.trillke.net> References: <20050206205635.GP9235@solar.trillke.net> <32476.1107736687@www47.gmx.net> <20050207074619.GT9235@solar.trillke.net> Message-ID: <20050207075230.GU9235@solar.trillke.net> On Mon, Feb 07, 2005 at 08:46 +0100, holger krekel wrote: > - - - - - - - - - - - - - - - recorded stdout - - - - - - - - - - - - - - - > modname simple1_wrap > working in /tmp/usession-129 > llvm-as /tmp/usession-129/simple1.ll -f -o simple1.bc > llvmc -f -O3 simple1.bc -o simple1_optimized.o > llc simple1_optimized.o.bc -f -o simple1.s > _______________________________________________________________________________ > ============= tests finished: 3 passed, 1 failed in 9.79 seconds ============= > > contents of usession-129 are on codespeak.net:/home/hpk/usession-129 > i am beginning to suspect my installation. Is there an easy way to > verify that my installation works? i manually redid the last steps above and at llvmc -f -O3 simple1.bc -o simple1_optimized.o llvmc: Unexpected unknown exception occurred. happens but doesn't return exitcode != 0 and so the cmd execution doesn't fail properly. So the original "Bytecode corruption" really means it can't find the file i think :-) holger From bhartsho at yahoo.com Mon Feb 7 09:23:47 2005 From: bhartsho at yahoo.com (Hart's Antler) Date: Mon, 7 Feb 2005 00:23:47 -0800 (PST) Subject: [pypy-dev] Using PyPy as a compiler In-Reply-To: <20050207073448.GR9235@solar.trillke.net> Message-ID: <20050207082347.34012.qmail@web30808.mail.mud.yahoo.com> --- holger krekel wrote: > Hi Brett, > > On Sun, Feb 06, 2005 at 18:25 -0800, Hart's Antler wrote: > > Hi, your right my function did have a print statement in there. I fixed that and it worked! > Is > > there a doc that lists the limitations translator has? > > > > Now i am trying to compile some other functions but having another problem. pyrexc gives this > > error: > > mymodule.pyx:11:12: Syntax error in simple statement list > > that line from mymodule is: > > cinline "Label2:" > > this is the "cinline" extension to pyrex, earlier mentioned in this thread. > Did you really use dist-pypy/pypy/tool/pyrexc to compile your code? > > holger > Hi Holger, I am sure i was calling dist-pypy/pypy/tool/pyrexc. For a sanity check i did an md5sum on both dist-pypy/pypy/tool/pyrexc and /usr/bin/pyrexc, they are different. I also tried the test again with both versions of pyrexc and they both give the same error. -brett __________________________________ Do you Yahoo!? Yahoo! Mail - Helps protect you from nasty viruses. http://promotions.yahoo.com/new_mail From arigo at tunes.org Mon Feb 7 12:27:18 2005 From: arigo at tunes.org (Armin Rigo) Date: Mon, 7 Feb 2005 11:27:18 +0000 Subject: [pypy-dev] Using PyPy as a compiler In-Reply-To: <20050207082347.34012.qmail@web30808.mail.mud.yahoo.com> References: <20050207073448.GR9235@solar.trillke.net> <20050207082347.34012.qmail@web30808.mail.mud.yahoo.com> Message-ID: <20050207112718.GA25771@vicky.ecs.soton.ac.uk> Hi Brett, On Mon, Feb 07, 2005 at 12:23:47AM -0800, Hart's Antler wrote: > > > mymodule.pyx:11:12: Syntax error in simple statement list > > > that line from mymodule is: > > > cinline "Label2:" This is really due to pyrexc not using our modified version. If you look inside pypy/tool/pyrexc, you will see that it is only a thin wrapper importing Pyrex. I guess that this finds the version of Pyrex that you have on your system instead of our own, even though the autopath hackery is supposed to set up the path so that it doesn't occur... Well I guess you need to add some print statements here and there to be sure of what version of the Pyrex package was found. Armin From cfbolz at gmx.de Mon Feb 7 13:47:28 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Mon, 7 Feb 2005 13:47:28 +0100 (MET) Subject: [pypy-dev] LLVM backend References: <20050207075230.GU9235@solar.trillke.net> Message-ID: <13807.1107780448@www51.gmx.net> Hi Holger, On Mon, Feb 07, 2005 at 08:46 +0100, holger krekel wrote: > > - - - - - - - - - - - - - - - recorded stdout - - - - - - - - - - - - - > - - > > modname simple1_wrap > > working in /tmp/usession-129 > > llvm-as /tmp/usession-129/simple1.ll -f -o simple1.bc > > llvmc -f -O3 simple1.bc -o simple1_optimized.o > > llc simple1_optimized.o.bc -f -o simple1.s > > > _______________________________________________________________________________ > > ============= tests finished: 3 passed, 1 failed in 9.79 seconds > ============= > > > > contents of usession-129 are on codespeak.net:/home/hpk/usession-129 > > i am beginning to suspect my installation. Is there an easy way to > > verify that my installation works? Since simple1 is a very, well, simple function so it should be a good test whether your installation works. This is all very strange. I checked your simple1.bc file and it is identical to the file my LLVM produces so your installation can't be a total loss. Maybe llvmc behaves strangely if the frontend is missing, it's just some sort of wrapper for all the other llvm tools. I'll change build_llvm_module to not use llvmc tonight. > > i manually redid the last steps above and at > > llvmc -f -O3 simple1.bc -o simple1_optimized.o > llvmc: Unexpected unknown exception occurred. > > happens but doesn't return exitcode != 0 and so the cmd execution > doesn't fail properly. So the original "Bytecode corruption" > really means it can't find the file i think :-) > > holger > You could also try to do the translation again using %~/pypy/translator/llvm> python -i genllvm.py >>> t = Translator(test.my_gcd) >>> a = t.annotate([int, int]) >>> f = llvmcompile(t, false) which turns optimization off (llvmc is then not used at all). If that works the important llvm tools work (namely the assembler llvm-as and the code-generator llc). Thanks for trying all this! Regards, Carl Friedrich From cfbolz at gmx.de Mon Feb 7 13:56:35 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Mon, 7 Feb 2005 13:56:35 +0100 (MET) Subject: [pypy-dev] LLVM backend References: <13807.1107780448@www51.gmx.net> Message-ID: <19186.1107780995@www51.gmx.net> On Mon, 7 Feb 2005 13:47:28 +0100 (MET), Carl Friedrich Bolz wrote: > You could also try to do the translation again using > > %~/pypy/translator/llvm> python -i genllvm.py > >>> t = Translator(test.my_gcd) > >>> a = t.annotate([int, int]) > >>> f = llvmcompile(t, false) Oops, should be "f = llvmcompile(t, False)" of course (I have been writing to much LLVM-code lately :-). Carl Friedrich From pedronis at strakt.com Mon Feb 7 18:12:42 2005 From: pedronis at strakt.com (Samuele Pedroni) Date: Mon, 07 Feb 2005 18:12:42 +0100 Subject: [pypy-dev] dist-simpler-multimethods branch Message-ID: <4207A18A.6070900@strakt.com> The branch Armin and me worked on over the weekend is ready to be merged. It is a reimplementation of multimethods for standard objspace using chained single-dispatch and user-defined precedences for types and their conversions. This means that it is clear what matches will be tried, for example for bools bool int long float ANY Although right now precedences and conversions are given in a very exhaustive way in std/model.py, more declarative approaches can be devised for this. Using well-defined precedence lists means that it should be possible, if necessary, to switch to other internal implementations strategies for dispatching. Chained single dispatching is realised by attaching methods to the partecipating classes. (*) Although we are not using it right now, apart for slicing, the approach means that is also easy to have fine control per argument position and per multimethod of what types conversions and precedences are allowed. The branch also reuses gateways instead of its own frame and code special classes. Samuele (*) to give an idea these are the methods attached with indicated where they are put plus the generated free-standing entry-point function for the add example in test_multimethod, you can follow what happens with calls like __add_perform_call(space, W_IntObject(), W_IntObject()) __add_perform_call(space, W_BoolObject(), W_IntObject()) ... In: W_BoolObject def __add__W_IntObject(arg1, space, arg0): return add__Int_Int(space, arg0, delegate_b2i(arg1)) ************************************************************ In: W_IntObject def __add__W_IntObject(arg1, space, arg0): return add__Int_Int(space, arg0, arg1) ************************************************************ In: W_Root def __add__W_IntObject(arg1, space, arg0): return raiseFailedToImplement() ************************************************************ In: W_BoolObject def __add(arg0, space, arg1): return arg1.__add__W_IntObject(space, delegate_b2i(arg0)) ************************************************************ In: W_Root def __add__W_IntObject(arg1, space, arg0): return raiseFailedToImplement() ************************************************************ In: W_IntObject def __add(arg0, space, arg1): return arg1.__add__W_IntObject(space, arg0) ************************************************************ In: W_Root def __add(arg0, space, arg1): return raiseFailedToImplement() ************************************************************ Free-standing entry-point: def __add_perform_call(space, arg0, arg1): return arg0.__add(space, arg1) From arigo at tunes.org Mon Feb 7 19:32:30 2005 From: arigo at tunes.org (Armin Rigo) Date: Mon, 7 Feb 2005 18:32:30 +0000 Subject: [pypy-dev] Re: [pypy-svn] r8960 - pypy/dist/pypy/objspace/std In-Reply-To: <20050207170719.1ED4027C51@code1.codespeak.net> References: <20050207170719.1ED4027C51@code1.codespeak.net> Message-ID: <20050207183230.GA12828@vicky.ecs.soton.ac.uk> Hi Anders, On Mon, Feb 07, 2005 at 06:07:19PM +0100, ac at codespeak.net wrote: > Log: > Make it possible to run (but with failing tests) cpythons test_builtins Hum, I still cannot run the test in my own machine, but I can run it on codespeak. Moreover I can still run it on codespeak without your patch... So my guess is that something unrelated is going on here. (Your patch itself is fine, though, as a good performance improvement.) Armin From bhartsho at yahoo.com Tue Feb 8 02:57:38 2005 From: bhartsho at yahoo.com (Hart's Antler) Date: Mon, 7 Feb 2005 17:57:38 -0800 (PST) Subject: [pypy-dev] Using PyPy as a compiler In-Reply-To: <20050207112718.GA25771@vicky.ecs.soton.ac.uk> Message-ID: <20050208015738.65901.qmail@web30810.mail.mud.yahoo.com> Hello, I've got the pypy compiler working! see the function at the bottom. But now i'm having some problems when importing the module and testing some of its functions. here's the commands i typed to make the module: pypy/tool/pyrexc myModule.pyx gcc -c -fPIC -I/usr/include/python2.3/ myModule.c gcc -shared myModule.o -o myModule.so when i import the module and try isFloatingPoint(None, '0.001') it returns the error below, looks like it can not find 'contains' where does contain come from, should i have linked to another library when making the .so file? >>> mod.isFloatingPoint(None, '0.001') Traceback (most recent call last): File "", line 1, in ? File "b11a3ad330bf84c9dabf6b4e40ba4baa.pyx", line 806, in b11a3ad330bf84c9dabf6b4e40ba4baa.isFloatingPoint return isFloatingPoint__2dbc1614(self_3077, s_3078) File "b11a3ad330bf84c9dabf6b4e40ba4baa.pyx", line 833, in b11a3ad330bf84c9dabf6b4e40ba4baa.isFloatingPoint__2dbc1614 v3119 = contains('0123456789.', char_3145) NameError: contains def isFloatingPoint(self, s): r = True chars = '0123456789.' for char in s: if char not in chars: return False return r def isFloatingPoint(self_3077, s_3078): return isFloatingPoint__2dbc1614(self_3077, s_3078) cdef object isFloatingPoint__2dbc1614(object self_3077, object s_3078): # untyped variables: char_3109 char_3113 char_3115 char_3124 char_3127 char_3143 char_3145 char_3148 char_3150 char_3155 char_3157 char_3162 etype_3080 evalue_3081 s_3087 s_3108 s_3112 s_3123 s_3126 s_3139 s_3142 s_3147 s_3154 s_3161 s_3167 self_3086 self_3107 self_3111 self_3122 self_3125 self_3138 self_3141 self_3146 self_3153 self_3160 self_3166 v3079 v3082 v3088 v3110 v3114 v3119 v3120 v3140 v3144 v3149 v3151 v3152 v3156 v3158 v3159 v3163 v3164 v3165 v3168 v3169 try: v3082 = iter(s_3078) except TypeError, last_exc_value: last_exception = last_exc_value.__class__ self_3166, s_3167, v3168, v3169 = self_3077, s_3078, last_exception, last_exc_value etype_3080, evalue_3081 = TypeError, v3169 raise etype_3080, evalue_3081 else: self_3138, s_3139, v3140 = self_3077, s_3078, v3082 self_3086, s_3087, v3088 = self_3138, s_3139, v3140 self_3111, s_3112, v3114 = self_3086, s_3087, v3088 cinline "Label5:" try: char_3115 = v3114.next() except StopIteration, last_exc_value: last_exception = last_exc_value.__class__ self_3160, s_3161, char_3162, v3163, v3164, v3165 = self_3111, s_3112, char_3113, v3114, last_exception, last_exc_value self_3122, s_3123, char_3124 = self_3160, s_3161, char_3162 self_3125, s_3126, char_3127 = self_3122, s_3123, char_3124 v3079 = 1 cinline "Label9:" return v3079 else: self_3141, s_3142, char_3143, v3144, char_3145 = self_3111, s_3112, char_3113, v3114, char_3115 v3119 = contains('0123456789.', char_3145) v3120 = not not v3119 if v3120 == 1: self_3153, s_3154, char_3155, v3156, char_3157, v3158, v3159 = self_3141, s_3142, char_3143, v3144, char_3145, v3119, v3120 self_3107, s_3108, char_3109, v3110 = self_3153, s_3154, char_3157, v3156 self_3111, s_3112, char_3113, v3114 = self_3107, s_3108, char_3109, v3110 cinline "goto Label5;" else: # v3120 == 0 self_3146, s_3147, char_3148, v3149, char_3150, v3151, v3152 = self_3141, s_3142, char_3143, v3144, char_3145, v3119, v3120 v3079 = 0 cinline "goto Label9;" def pypyCompile(txt): import md5 from pypy.translator.translator import Translator txt = txt.replace('print ', '#print ') hashed = md5.new( txt ).hexdigest() # before compiling, check if there is a cached .so file if os.path.exists('%s.so'%hashed): cmd = 'import %s as module' %hashed exec cmd return module mod = {} code = compile(txt, 'pypy translator', 'exec') exec code in mod pyx = "" for a in mod: func = mod[a] #if func and func != '__builtins__': if type(func) == types.FunctionType: print 'converting python to pyrex for function', func t = Translator(func) d = t.pyrex() if 0: #for debugging name = 'tmp/%s.pyx' %a f = open(name, 'wb') f.write(d) f.close() os.system('pypy/tool/pyrexc %s' %name) if os.path.exists(name): os.system('gcc -c -fPIC -I/usr/include/python2.3/ %s.c' %name) else: raise 'stop' pyx += d + '\n' url = hashed + '.pyx' f = open(url, 'wb') f.write(pyx) f.close() print 'saved .pyx file', url os.system('pypy/tool/pyrexc %s' %url) if os.path.exists(url): print 'pyrex translated to c' os.system('gcc -c -fPIC -I/usr/include/python2.3/ %s.c' %hashed) print 'compiled c file to o' os.system('gcc -shared %s.o -o %s.so' %(hashed,hashed)) print 'compiled o file to so file' else: raise 'stop' cmd = 'import %s as module' %hashed exec cmd return module __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From sanxiyn at gmail.com Tue Feb 8 05:41:57 2005 From: sanxiyn at gmail.com (Sanghyeon Seo) Date: Tue, 8 Feb 2005 13:41:57 +0900 Subject: [pypy-dev] Using PyPy as a compiler In-Reply-To: <20050208015738.65901.qmail@web30810.mail.mud.yahoo.com> References: <20050207112718.GA25771@vicky.ecs.soton.ac.uk> <20050208015738.65901.qmail@web30810.mail.mud.yahoo.com> Message-ID: <5b0248170502072041669fe950@mail.gmail.com> On Mon, 7 Feb 2005 17:57:38 -0800 (PST), Hart's Antler wrote: > I've got the pypy compiler working! see the function at the bottom. > But now i'm having some problems when importing the module and testing > some of its functions. Great! > when i import the module and try isFloatingPoint(None, '0.001') it returns the error > below, looks like it can not find 'contains' where does contain come from, should i > have linked to another library when making the .so file? No, you don't need to link another library. I think I have fixed this one now. Update and try again? > >>> mod.isFloatingPoint(None, '0.001') > Traceback (most recent call last): > File "", line 1, in ? > File "b11a3ad330bf84c9dabf6b4e40ba4baa.pyx", line 806, in > b11a3ad330bf84c9dabf6b4e40ba4baa.isFloatingPoint > return isFloatingPoint__2dbc1614(self_3077, s_3078) > File "b11a3ad330bf84c9dabf6b4e40ba4baa.pyx", line 833, in > b11a3ad330bf84c9dabf6b4e40ba4baa.isFloatingPoint__2dbc1614 > v3119 = contains('0123456789.', char_3145) > NameError: contains > > def isFloatingPoint(self, s): > r = True > chars = '0123456789.' > for char in s: > if char not in chars: > return False > return r Seo Sanghyeon From bhartsho at yahoo.com Tue Feb 8 09:59:21 2005 From: bhartsho at yahoo.com (Hart's Antler) Date: Tue, 8 Feb 2005 00:59:21 -0800 (PST) Subject: [pypy-dev] Using PyPy as a compiler In-Reply-To: <5b0248170502072041669fe950@mail.gmail.com> Message-ID: <20050208085921.3247.qmail@web30807.mail.mud.yahoo.com> Hi Seo, Thanks, your updates fixed the problem! I was able to compile most of an XML DOM and XPath implementation to C, it generated alot of C code though, almost 35,000 lines, i'll do some bench marks later comparing it to normal python and psyco. But I wasn't able to use the annotator to simplify the types, is that still pretty experimental? If anybody wants to try the compiled XML DOM, shoot me an email and i'll send you the source. Cheers, -brett --- Sanghyeon Seo wrote: > On Mon, 7 Feb 2005 17:57:38 -0800 (PST), Hart's Antler > wrote: > > I've got the pypy compiler working! see the function at the bottom. > > But now i'm having some problems when importing the module and testing > > some of its functions. > > Great! > > > when i import the module and try isFloatingPoint(None, '0.001') it returns the error > > below, looks like it can not find 'contains' where does contain come from, should i > > have linked to another library when making the .so file? > > No, you don't need to link another library. I think I have fixed this > one now. Update > and try again? > > > >>> mod.isFloatingPoint(None, '0.001') > > Traceback (most recent call last): > > File "", line 1, in ? > > File "b11a3ad330bf84c9dabf6b4e40ba4baa.pyx", line 806, in > > b11a3ad330bf84c9dabf6b4e40ba4baa.isFloatingPoint > > return isFloatingPoint__2dbc1614(self_3077, s_3078) > > File "b11a3ad330bf84c9dabf6b4e40ba4baa.pyx", line 833, in > > b11a3ad330bf84c9dabf6b4e40ba4baa.isFloatingPoint__2dbc1614 > > v3119 = contains('0123456789.', char_3145) > > NameError: contains > > > > def isFloatingPoint(self, s): > > r = True > > chars = '0123456789.' > > for char in s: > > if char not in chars: > > return False > > return r > > Seo Sanghyeon > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From mwh at python.net Tue Feb 8 11:07:57 2005 From: mwh at python.net (Michael Hudson) Date: Tue, 08 Feb 2005 10:07:57 +0000 Subject: [pypy-dev] Re: Using PyPy as a compiler In-Reply-To: <20050208085921.3247.qmail@web30807.mail.mud.yahoo.com> (Hart's Antler's message of "Tue, 8 Feb 2005 00:59:21 -0800 (PST)") References: <5b0248170502072041669fe950@mail.gmail.com> <20050208085921.3247.qmail@web30807.mail.mud.yahoo.com> Message-ID: <2mmzuf5pqa.fsf@starship.python.net> Hart's Antler writes: > Hi Seo, > > Thanks, your updates fixed the problem! I was able to compile most > of an XML DOM and XPath implementation to C, That's cool! > it generated alot of C code though, almost 35,000 lines, That's not surprising. > i'll do some bench marks later comparing it to normal python and > psyco. Don't expect wonders at this point, but please do try -- it'd be interesting to see what does happen. > But I wasn't able to use the annotator to simplify the types, is > that still pretty experimental? Yes, certainly. Did the way it blew up make even vague sense? :) > If anybody wants to try the compiled XML DOM, shoot me an email and > i'll send you the source. It might be more interesting to see how far we are from being able to annotate your code, but the answer might be "a pretty long way"... Cheers, mwh -- I believe C++ instills fear in programmers, fear that the interaction of some details causes unpredictable results. -- Erik Naggum, comp.lang.lisp From arigo at tunes.org Tue Feb 8 11:45:32 2005 From: arigo at tunes.org (Armin Rigo) Date: Tue, 8 Feb 2005 10:45:32 +0000 Subject: [pypy-dev] Re: [pypy-svn] r8967 - in pypy/dist/pypy: lib tool In-Reply-To: <20050207185804.6021B27C63@code1.codespeak.net> References: <20050207185804.6021B27C63@code1.codespeak.net> Message-ID: <20050208104532.GA29295@vicky.ecs.soton.ac.uk> Hi, On Mon, Feb 07, 2005 at 07:58:04PM +0100, tismer at codespeak.net wrote: > Log: > added class NiceTranslate for compilation of dynamically > generated methods, see _classobj.py . > The generated code objects have correct source info > instead of just "" Funny how often we reinvent this particular wheel! That's at least the third way to work around this limitation of exec/compile that we have in PyPy. The first one should be removed, the second one is dyncode from the py lib, the third one is yours. My guess is that we need a very simple one that we use systematically for all of our dynamic compilation needs in PyPy, which interfaces nicely with dyncode but does't require the py lib to be imported outside of debugging code. Armin From cfbolz at gmx.de Tue Feb 8 12:27:20 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 8 Feb 2005 12:27:20 +0100 (MET) Subject: [pypy-dev] LLVM backend References: <20050207075230.GU9235@solar.trillke.net> Message-ID: <15237.1107862040@www9.gmx.net> Hi Holger, On Mon, 7 Feb 2005 08:52:30 +0100, holger krekel wrote: > i manually redid the last steps above and at > > llvmc -f -O3 simple1.bc -o simple1_optimized.o > llvmc: Unexpected unknown exception occurred. > > happens but doesn't return exitcode != 0 and so the cmd execution > doesn't fail properly. So the original "Bytecode corruption" > really means it can't find the file i think :-) I just removed the usage of llvmc so if that was really the problem it could work now. The only llvm tools that genllvm now uses are llvm-as (which works in your case) and llc (produces native code). On the other hand I really doubt that the only reason for the failure of llvmc is the fact that the frontend is missing: I compiled llvm without the frontend yesterday and llvmc worked. Has anyone else tried this and experienced the same behaviour? Regards, Carl Friedrich From bhartsho at yahoo.com Wed Feb 9 02:31:29 2005 From: bhartsho at yahoo.com (Hart's Antler) Date: Tue, 8 Feb 2005 17:31:29 -0800 (PST) Subject: [pypy-dev] Re: Using PyPy as a compiler In-Reply-To: <20050208100804.F09AF27C31@code1.codespeak.net> Message-ID: <20050209013129.36732.qmail@web30808.mail.mud.yahoo.com> Hi, unannotated, i found that the pypy to c compiled code runs slower than the same function in python. GCC fails to compile any of the annotated DOM code. Here's some errors it produced for int annotated. module_1d49eab472565cbc4619f8ff8cc35f49.c:214: error: duplicate member `__iter__' module_1d49eab472565cbc4619f8ff8cc35f49.c:215: error: duplicate member `mro' module_1d49eab472565cbc4619f8ff8cc35f49.c:216: error: duplicate member `values' module_1d49eab472565cbc4619f8ff8cc35f49.c:217: error: duplicate member `__repr__' module_1d49eab472565cbc4619f8ff8cc35f49.c:218: error: duplicate member `__hash__' module_1d49eab472565cbc4619f8ff8cc35f49.c:219: error: duplicate member `__ge__' module_1d49eab472565cbc4619f8ff8cc35f49.c:220: error: duplicate member `iteritems' module_1d49eab472565cbc4619f8ff8cc35f49.c:221: error: duplicate member `__getattribute__' module_1d49eab472565cbc4619f8ff8cc35f49.c:222: error: duplicate member `pop' module_1d49eab472565cbc4619f8ff8cc35f49.c:223: error: duplicate member `has_key' module_1d49eab472565cbc4619f8ff8cc35f49.c:224: error: duplicate member `__lt__' module_1d49eab472565cbc4619f8ff8cc35f49.c:225: error: duplicate member `__init__' module_1d49eab472565cbc4619f8ff8cc35f49.c:226: error: duplicate member `__new__' module_1d49eab472565cbc4619f8ff8cc35f49.c:227: error: duplicate member `__contains__' module_1d49eab472565cbc4619f8ff8cc35f49.c:228: error: duplicate member `__cmp__' module_1d49eab472565cbc4619f8ff8cc35f49.c:229: error: duplicate member `__le__' module_1d49eab472565cbc4619f8ff8cc35f49.c:230: error: duplicate member `itervalues' module_1d49eab472565cbc4619f8ff8cc35f49.c:231: error: duplicate member `__doc__' module_1d49eab472565cbc4619f8ff8cc35f49.c:232: error: duplicate member `__len__' module_1d49eab472565cbc4619f8ff8cc35f49.c:233: error: duplicate member `__ne__' module_1d49eab472565cbc4619f8ff8cc35f49.c:234: error: duplicate member `__getitem__' module_1d49eab472565cbc4619f8ff8cc35f49.c: In function `__pyx_f_39module_1d49eab472565cbc4619f8ff8cc35f49_appendChild__2dc4c10c': module_1d49eab472565cbc4619f8ff8cc35f49.c:801: error: request for member `_childNodes' in something not a structure or union module_1d49eab472565cbc4619f8ff8cc35f49.c:801: warning: assignment makes pointer from integer without a cast module_1d49eab472565cbc4619f8ff8cc35f49.c:893: error: request for member `_childNodes' in something not a structure or union module_1d49eab472565cbc4619f8ff8cc35f49.c:893: warning: assignment makes pointer from integer without a cast module_1d49eab472565cbc4619f8ff8cc35f49.c:979: error: request for member `_childNodes' in something not a structure or union module_1d49eab472565cbc4619f8ff8cc35f49.c:979: warning: assignment makes pointer from integer without a cast module_1d49eab472565cbc4619f8ff8cc35f49.c: In function `__pyx_f_39module_1d49eab472565cbc4619f8ff8cc35f49__xpath_attr__2dc4c5dc': module_1d49eab472565cbc4619f8ff8cc35f49.c:2383: error: request for member `ELEMENT_NODE' in something not a structure or union module_1d49eab472565cbc4619f8ff8cc35f49.c:2383: warning: assignment makes pointer from integer without a cast module_1d49eab472565cbc4619f8ff8cc35f49.c: In function `__pyx_f_39module_1d49eab472565cbc4619f8ff8cc35f49__xpath_text__2dc4c5a4': module_1d49eab472565cbc4619f8ff8cc35f49.c:4814: error: request for member `TEXT_NODE' in something not a structure or union -brett __________________________________ Do you Yahoo!? All your favorites on one personal page ? Try My Yahoo! http://my.yahoo.com From lac at strakt.com Wed Feb 9 12:02:48 2005 From: lac at strakt.com (Laura Creighton) Date: Wed, 9 Feb 2005 12:02:48 +0100 Subject: [pypy-dev] 'My Python is too slow' Message-ID: <200502091102.j19B2mZn015813@theraft.strakt.com> http://www.linuxdevcenter.com/pub/a/linux/2005/01/27/freevo.html Speeding these guys up would make a nice demo. Laura From lac at strakt.com Wed Feb 9 12:03:49 2005 From: lac at strakt.com (Laura Creighton) Date: Wed, 9 Feb 2005 12:03:49 +0100 Subject: [pypy-dev] OSCON proposals closing Feb 13. Message-ID: <200502091103.j19B3nrD015897@theraft.strakt.com> Are we sending anything? Laura From hpk at trillke.net Wed Feb 9 12:13:05 2005 From: hpk at trillke.net (holger krekel) Date: Wed, 9 Feb 2005 12:13:05 +0100 Subject: [pypy-dev] OSCON proposals closing Feb 13. In-Reply-To: <200502091103.j19B3nrD015897@theraft.strakt.com> References: <200502091103.j19B3nrD015897@theraft.strakt.com> Message-ID: <20050209111305.GM16046@solar.trillke.net> On Wed, Feb 09, 2005 at 12:03 +0100, Laura Creighton wrote: > Are we sending anything? i have already opened an issue here: https://codespeak.net/issue/pypy-eu/issue67 where you should have got mail about (if not please tell me privately). Summary: i am interested to (co-)write the proposal and go there ... cheers, holger From lac at strakt.com Wed Feb 9 13:18:32 2005 From: lac at strakt.com (Laura Creighton) Date: Wed, 9 Feb 2005 13:18:32 +0100 Subject: [pypy-dev] can you renumber roundup issues Message-ID: <200502091218.j19CIWMl026131@theraft.strakt.com> If you sort by id you get the order: 1 11 12 2 3 4 55 58 59 6 which isn't what we want. I also don't understand why there are gaps -- this is what you get when you sort grouped by 'milestone' I would have expected 1 11 12 13 14 15 16 ... where are those guys? You get them when you group by activity. I don't think that I understand the 'group by' feature at all. can I make my own groups and my own search criteria? I haven't found a url that explains this yet, can you point me at it? I figure adding 10,000 to every issue should do the trick. I have too many issues, but don't expect that we will have more than 99,999 of them. Or perhaps I am being optimistic? Laura From jacob at strakt.com Wed Feb 9 17:25:16 2005 From: jacob at strakt.com (Jacob =?iso-8859-1?q?Hall=E9n?=) Date: Wed, 9 Feb 2005 17:25:16 +0100 Subject: [pypy-dev] Pycon sprint - some logistics In-Reply-To: References: <200502022020.16420.jacob@strakt.com> Message-ID: <200502091725.16748.jacob@strakt.com> On fredag 4 februari 2005 09.44, Anna Ravenscroft wrote: > Alex has booked with KLM (better than the SAS prices we could find from > Italy). Do we submit receipts to you for the reimbursement or how does > that work? Anna, we are still working on finding out exactly what the requirements are. To the best of our knowledge, the steps will be 1. Submit a request to join the Pypy project as an individual, with the purpose of attending a specific sprint. Submit an expected budget. 2. Get accepted by the Technical Committee 3. Receive forms from the project Co-ordinator (DFKI) to acceed to the EU contract and recieve copies of the Association Agreement between the individual and the Project Consortium. 4. Review, sign and return items from 3. 5. The project Co-ordinator submits the contract Accession form to the EU. 6. Wait. 7. Wait. 8. The EU probably says ok. 9. If they do, attend the sprint. If not, you are welcome at your own cost. 10. Submit receipts for hotel and accomodation. Fill in form for extra food costs. 11. Send to Co-ordinator, together with payment information. 12. Get paid. Neither hard nor simple, I'd say. Best regards Jacob From jlg at dds.nl Thu Feb 10 11:52:22 2005 From: jlg at dds.nl (Johannes Gijsbers) Date: Thu, 10 Feb 2005 11:52:22 +0100 Subject: [pypy-dev] Re: can you renumber roundup issues References: <200502091218.j19CIWMl026131@theraft.strakt.com> Message-ID: Really short introduction: I work on Roundup occasionally. I'm an interested onlooker with the pypy project. I might join a sprint if another one is held in the Netherlands. On Wed, 09 Feb 2005 13:18:32 +0100, Laura Creighton wrote: > If you sort by id you get the order: 1 11 12 2 3 4 55 58 59 6 Yes, you're using a backend that doesn't work with numeric ids yet. This means the ids are stored as strings, so you get an alpha sort. Maybe one of the codespeak admins can switch the tracker to a backend that does support numeric ids (e.g.: PostgreSQL)? Instructions are here: http://roundup.sourceforge.net/doc-0.7/admin_guide.html#migrating-backends. > which isn't what we want. I also don't understand why there are gaps -- > this is what you get when you sort grouped by 'milestone' I would have > expected 1 11 12 13 14 15 16 ... where are those guys? You get them when > you group by activity. I don't think that I understand the 'group by' > feature at all. can I make my own groups and my own search criteria? I > haven't found a url that explains this yet, can you point me at it? Nope, you can't make your own groups. Try taking a look at the 'Show by Workpackage' query. All the issues that have workpackage 'wp00_general' are sorted under one heading, all the issues with the workpackage 'wp01_management' are sorted under one heading, etc. The issues are then sorted within the subheading. Cheers, Johannes From hpk at trillke.net Thu Feb 10 12:06:56 2005 From: hpk at trillke.net (holger krekel) Date: Thu, 10 Feb 2005 12:06:56 +0100 Subject: [pypy-dev] Re: can you renumber roundup issues In-Reply-To: References: <200502091218.j19CIWMl026131@theraft.strakt.com> Message-ID: <20050210110656.GL28668@solar.trillke.net> Hi Johannes, On Thu, Feb 10, 2005 at 11:52 +0100, Johannes Gijsbers wrote: > Really short introduction: I work on Roundup occasionally. I'm an > interested onlooker with the pypy project. I might join a sprint if > another one is held in the Netherlands. great, i am sure we'll revisit the Netherlands at some point! > On Wed, 09 Feb 2005 13:18:32 +0100, Laura Creighton wrote: > > > If you sort by id you get the order: 1 11 12 2 3 4 55 58 59 6 > > Yes, you're using a backend that doesn't work with numeric ids yet. This > means the ids are stored as strings, so you get an alpha sort. Maybe one > of the codespeak admins can switch the tracker to a backend that does > support numeric ids (e.g.: PostgreSQL)? Instructions are here: > http://roundup.sourceforge.net/doc-0.7/admin_guide.html#migrating-backends. thanks for pointing it out. I plan to do a revision in some weeks taking into account all suggestions. I guess going with PostgresSQL is a good idea, then. cheers & thanks, holger From ac at strakt.com Thu Feb 10 12:16:41 2005 From: ac at strakt.com (=?ISO-8859-1?Q?Anders_Chrigstr=F6m?=) Date: Thu, 10 Feb 2005 12:16:41 +0100 Subject: [pypy-dev] Re: [pypy-svn] r8960 - pypy/dist/pypy/objspace/std In-Reply-To: <20050207183230.GA12828@vicky.ecs.soton.ac.uk> References: <20050207170719.1ED4027C51@code1.codespeak.net> <20050207183230.GA12828@vicky.ecs.soton.ac.uk> Message-ID: <420B4299.4020006@strakt.com> Armin Rigo wrote: > Hi Anders, > > On Mon, Feb 07, 2005 at 06:07:19PM +0100, ac at codespeak.net wrote: > >>Log: >>Make it possible to run (but with failing tests) cpythons test_builtins > > > Hum, I still cannot run the test in my own machine, but I can run it on > codespeak. Moreover I can still run it on codespeak without your patch... So > my guess is that something unrelated is going on here. (Your patch itself is > fine, though, as a good performance improvement.) The test to be made runnable is self.assertRaises(MemoryError, list, xrange(sys.maxint // 2)) Without the performance improvement it simply takes longer to run than I am willing to wait. This test is only performed on 32 bit systems hence no problems on codespeak. As for not working on your machine i have no idea why. If you still have problems I'll be intrested in what tracebacks you get. /Arre From arigo at tunes.org Thu Feb 10 16:16:07 2005 From: arigo at tunes.org (Armin Rigo) Date: Thu, 10 Feb 2005 15:16:07 +0000 Subject: [pypy-dev] Re: [pypy-svn] r8960 - pypy/dist/pypy/objspace/std In-Reply-To: <420B4299.4020006@strakt.com> References: <20050207170719.1ED4027C51@code1.codespeak.net> <20050207183230.GA12828@vicky.ecs.soton.ac.uk> <420B4299.4020006@strakt.com> Message-ID: <20050210151607.GA1919@vicky.ecs.soton.ac.uk> Hi Anders, On Thu, Feb 10, 2005 at 12:16:41PM +0100, Anders Chrigstr?m wrote: > The test to be made runnable is > self.assertRaises(MemoryError, list, xrange(sys.maxint // 2)) > Without the performance improvement it simply takes longer to run than I > am willing to wait. This test is only performed on 32 bit systems hence > no problems on codespeak. As for not working on your machine i have no > idea why. Ah, thanks for the comments. I discovered that it's related to the CPython version. I get an OperationError wrapping an assertion error when I try "py.test test_builtin.py" in Python 2.4, but not in 2.3. Armin From mwh at python.net Fri Feb 11 13:49:56 2005 From: mwh at python.net (Michael Hudson) Date: Fri, 11 Feb 2005 12:49:56 +0000 Subject: [pypy-dev] Re: [pypy-svn] r9100 - in pypy/dist/pypy/objspace/std: . test In-Reply-To: <20050211094047.E405E27BB1@code1.codespeak.net> (ac@codespeak.net's message of "Fri, 11 Feb 2005 10:40:47 +0100 (MET)") References: <20050211094047.E405E27BB1@code1.codespeak.net> Message-ID: <2mekfn45xn.fsf@starship.python.net> ac at codespeak.net writes: > Author: ac > Date: Fri Feb 11 10:40:47 2005 > New Revision: 9100 > > Added: > pypy/dist/pypy/objspace/std/test/test_fake.py (contents, props changed) > Modified: > pypy/dist/pypy/objspace/std/fake.py > Log: > Make faked types a little more inheritable. Nice one. It might be worth checking if this lets Python 2.3's random module work now. Cheers, mwh -- 34. The string is a stark data structure and everywhere it is passed there is much duplication of process. It is a perfect vehicle for hiding information. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From sanxiyn at gmail.com Fri Feb 11 14:19:35 2005 From: sanxiyn at gmail.com (Sanghyeon Seo) Date: Fri, 11 Feb 2005 22:19:35 +0900 Subject: [pypy-dev] Re: [pypy-svn] r9100 - in pypy/dist/pypy/objspace/std: . test In-Reply-To: <2mekfn45xn.fsf@starship.python.net> References: <20050211094047.E405E27BB1@code1.codespeak.net> <2mekfn45xn.fsf@starship.python.net> Message-ID: <5b0248170502110519196fcc0a@mail.gmail.com> Michael Hudson wrote: > ac at codespeak.net writes: > > Log: > > Make faked types a little more inheritable. > Nice one. It might be worth checking if this lets Python 2.3's random > module work now. No it doesn't work. From mwh at python.net Fri Feb 11 14:27:13 2005 From: mwh at python.net (Michael Hudson) Date: Fri, 11 Feb 2005 13:27:13 +0000 Subject: [pypy-dev] Re: [pypy-svn] r9100 - in pypy/dist/pypy/objspace/std: . test In-Reply-To: <5b0248170502110519196fcc0a@mail.gmail.com> (Sanghyeon Seo's message of "Fri, 11 Feb 2005 22:19:35 +0900") References: <20050211094047.E405E27BB1@code1.codespeak.net> <2mekfn45xn.fsf@starship.python.net> <5b0248170502110519196fcc0a@mail.gmail.com> Message-ID: <2m1xbn447i.fsf@starship.python.net> Sanghyeon Seo writes: > Michael Hudson wrote: >> ac at codespeak.net writes: >> > Log: >> > Make faked types a little more inheritable. >> Nice one. It might be worth checking if this lets Python 2.3's random >> module work now. > > No it doesn't work. Darn. Cheers, mwh -- : exploding like a turd Never had that happen to me, I have to admit. They do that often in your world? -- Eric The Read & Dave Brown, asr From lac at strakt.com Fri Feb 11 18:32:25 2005 From: lac at strakt.com (Laura Creighton) Date: Fri, 11 Feb 2005 18:32:25 +0100 Subject: [pypy-dev] Re: [pypy-svn] r9112 - pypy/dist/pypy/tool In-Reply-To: Message from arigo@codespeak.net of "Fri, 11 Feb 2005 14:57:14 +0100." <20050211135714.9047327BE8@code1.codespeak.net> References: <20050211135714.9047327BE8@code1.codespeak.net> Message-ID: <200502111732.j1BHWPa2003387@theraft.strakt.com> That was me, mindlessly following a ruleset in my head to convert UnitTestLanguage into code. Some days I am a stupid as a machine. laura In a message of Fri, 11 Feb 2005 14:57:14 +0100, arigo at codespeak.net writes: >Author: arigo >Date: Fri Feb 11 14:57:14 2005 >New Revision: 9112 > >Modified: > pypy/dist/pypy/tool/utestconvert.py >Log: >Not sure why failUnlessEqual(x,y) is converted into assert not x!=y. > > >Modified: pypy/dist/pypy/tool/utestconvert.py >========================================================================= >===== >--- pypy/dist/pypy/tool/utestconvert.py (original) >+++ pypy/dist/pypy/tool/utestconvert.py Fri Feb 11 14:57:14 2005 >@@ -18,11 +18,11 @@ > d['assertEqual'] = ('assert', ' ==', [2,3]) > d['failIfEqual'] = ('assert not', ' ==', [2,3]) > d['assertNotEqual'] = ('assert', ' !=', [2,3]) >-d['failUnlessEqual'] = ('assert not', ' !=', [2,3]) >+d['failUnlessEqual'] = ('assert', ' ==', [2,3]) > d['assertAlmostEqual'] = ('assert round', ' ==', [2,3,4]) > d['failIfAlmostEqual'] = ('assert not round', ' ==', [2,3,4]) > d['assertNotAlmostEqual'] = ('assert round', ' !=', [2,3,4]) >-d['failUnlessAlmostEquals'] = ('assert not round', ' !=', [2,3,4]) >+d['failUnlessAlmostEquals'] = ('assert round', ' ==', [2,3,4]) > # PyPy specific > d['assertRaises_w'] = ('self.space.raises_w', '', ['Any'] >) > d['assertEqual_w'] = ('assert self.space.eq_w','',['Any']) >_______________________________________________ >pypy-svn mailing list >pypy-svn at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-svn From mwh at python.net Fri Feb 11 14:26:00 2005 From: mwh at python.net (Michael Hudson) Date: Fri, 11 Feb 2005 13:26:00 +0000 Subject: [pypy-dev] Re: [pypy-svn] r9100 - in pypy/dist/pypy/objspace/std: . test References: <20050211094047.E405E27BB1@code1.codespeak.net> <2mekfn45xn.fsf@starship.python.net> <5b0248170502110519196fcc0a@mail.gmail.com> Message-ID: <2m650z449j.fsf@starship.python.net> Sanghyeon Seo writes: > Michael Hudson wrote: >> ac at codespeak.net writes: >> > Log: >> > Make faked types a little more inheritable. >> Nice one. It might be worth checking if this lets Python 2.3's random >> module work now. > > No it doesn't work. Darn. Cheers, mwh -- That one is easily explained away as massively intricate conspiracy, though. -- Chris Klein, alt.sysadmin.recovery From hpk at trillke.net Sat Feb 12 11:35:54 2005 From: hpk at trillke.net (holger krekel) Date: Sat, 12 Feb 2005 11:35:54 +0100 Subject: [pypy-dev] Re: [pypy-svn] r9152 - pypy/dist/pypy/lib In-Reply-To: <20050212082720.36CC227BCB@code1.codespeak.net> References: <20050212082720.36CC227BCB@code1.codespeak.net> Message-ID: <20050212103554.GF28668@solar.trillke.net> Hi Seo, On Sat, Feb 12, 2005 at 09:27 +0100, sanxiyn at codespeak.net wrote: > Author: sanxiyn > Date: Sat Feb 12 09:27:20 2005 > New Revision: 9152 > > Added: > pypy/dist/pypy/lib/decimal.py > Log: > decimal module from 2.4. All doctests pass. we wanted to put only modified modules relating to cpython 2.3.4 in our "lib-python-2.3.4" overriding "pypy/lib" directory. Maybe it makes sense to rename pypy/lib to pypy/lib-override-2.3.4 to make this more obvious :-) cheers, holger From hpk at trillke.net Mon Feb 14 15:48:55 2005 From: hpk at trillke.net (holger krekel) Date: Mon, 14 Feb 2005 15:48:55 +0100 Subject: [pypy-dev] Re: [pypy-svn] r9196 - pypy/dist/lib-python-2.3.4/test In-Reply-To: <20050214092101.A990627B6D@code1.codespeak.net> References: <20050214092101.A990627B6D@code1.codespeak.net> Message-ID: <20050214144855.GX28668@solar.trillke.net> Hi Anders, On Mon, Feb 14, 2005 at 10:21 +0100, ac at codespeak.net wrote: > Author: ac > Date: Mon Feb 14 10:21:01 2005 > New Revision: 9196 > > Modified: > pypy/dist/lib-python-2.3.4/test/pypy_unittest.py > Log: > Work around problem with py.test's raises(). What would that problem be? I tried some cases (wrong or missing exception) which all seemed to work ... cheers, holger From tismer at stackless.com Tue Feb 15 02:38:17 2005 From: tismer at stackless.com (Christian Tismer) Date: Tue, 15 Feb 2005 02:38:17 +0100 Subject: [pypy-dev] Ann: PyPy Sprint before PYCON 2005 in Washington Message-ID: <42115289.8070505@stackless.com> PyPy Sprint before PYCON 2005 in Washington ------------------------------------------- In the four days from 19th March till 22th March (inclusive) the PyPy team will host a sprint on their new Python-in-Python implementation. The PyPy project was granted funding by the European Union as part of its Sixth Framework Program, and is now on track to produce a stackless Python-in-Python Just-in-Time Compiler by December 2006. Our Python implementation, released under the MIT/BSD license, already provides new levels of flexibility and extensibility at the core interpreter and object implementation level. Armin Rigo and Holger Krekel will also give talks about PyPy and the separate py.test tool (used to perform various kinds of testing in PyPy) during the conference. Naturally, we are eager to see how the other re-implementation of Python, namely IronPython, is doing and to explore collaboration possibilities. Of course, that will depend on the degree of openness that Microsoft wants to employ. The Pycon2005 sprint is going to focus on reaching compatibility with CPython (currently we target version 2.3.4) for our PyPy version running on top of CPython. One goal of the sprint is to pass 60% or more of the unmodified regression tests of mainline CPython. It will thus be a great way to get to know CPython and PyPy better at the same time! Other possible work areas include: - translation to C to get a first working lower-level representation of the interpreter "specified in Python" - integrating and implementing a full parser/compiler chain written in Python maybe already targetting the new AST-branch of mainline CPython - fixing various remaining issues that will come up while trying to reach the compatibility goal - integrate or code pure python implementations of some Python modules currently written in C. - whatever issues you come up with! (please tell us before hand so we can better plan introductions etc.pp.) Besides core developers, Bea D?ring will be present to help improving and document our sprint and agile development process. We are going to give tutorials about PyPy's basic concepts and provide help to newcomers usually by pairing them with experienced pypythonistas. However, we kindly ask newcomers to be present on the first day's morning (19th of March) of the sprint to be able to get everyone a smooth start into the sprint. So far most newcomers had few problems in getting a good start into our codebase. However, it is good to have the following preparational points in mind: - some experience with programming in the Python language and interest to dive deeper - subscription to pypy-dev and pypy-sprint at http://codespeak.net/pypy/index.cgi?lists - have a subversion-client, Pygame and graphviz installed on the machine you bring to the sprint. - have a look at our current documentation, especially the architecture and getting-started documents under http://codespeak.net/pypy/index.cgi?doc The pypy-dev and pypy-sprint lists are also the contact points for raising questions and suggesting and discussing sprint topics beforehand. We are on #pypy on irc.freenode.net most of the time. Please don't hesitate to contact us or introduce yourself and your interests! Logistics --------- Organizational details will be posted to pypy-sprint and are or will be available in the Pycon2005-Sprint wiki here: http://www.python.org/moin/PyConDC2005/Sprints Registration ------------ send mail to pypy-sprint at codespeak.net, stating the days you can be present and any specific interests if applicable. Registered Participants ----------------------- all days: Jacob Hall?n Armin Rigo Holger Krekel Samuele Pedroni Anders Chrigstr?m Bea D?ring Christian Tismer Richard Emslie -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From sabre at nondot.org Thu Feb 17 06:29:01 2005 From: sabre at nondot.org (Chris Lattner) Date: Wed, 16 Feb 2005 23:29:01 -0600 (CST) Subject: [pypy-dev] RE: LLVM backend Message-ID: Carl Friedrich Bolz wrote: > I just checked my LLVM-backend in (I hope I did nothing wrong). It > resides in pypy/translator/llvm. Wow cool, I had no idea that you guys were this far along! Here are some answers to LLVM related questions: > I just removed the usage of llvmc so if that was really the problem > it could work now. The only llvm tools that genllvm now uses are > llvm-as (which works in your case) and llc (produces native code). Yup, unfortunately, 'llvmc' was not quite stable when the 1.4 release was put out. It's a bit better in LLVM CVS now, but is still a work in progress. Just not using it is a good way to go for now. > - I think there should be some more intelligent way to produce the > necessary LLLVM-implementations for the space operations of more > complex types than just writing them in LLVM-assembler, which can be > quite tedious (it's no fun writing programs in SSA form). Oh yeah, generating SSA is quite a pain. The traditional way that LLVM front-ends deal with this is to use 'alloca's for local variables and use explicit loads/stores to them. This generates really gross looking code, but the LLVM optimizers (specifically mem2reg) rip them up. For example, for something simple like: X = 1; ... = X; ... X = 2; You can generate code that looks like this: %X = alloca int ;; in the entry block for the fn ... store int 1, int* %X ... = load int* %X ... store int 2, int* %X ... If you run this sort of code through the LLVM "-mem2reg" optimization, it will promote all of these to SSA values, so you don't have to do it yourself. If the "..."'s contain control flow, this is a non-trivial task. :) > - List and Strings should be relatively easy to implement with arrays. > I'm not quite shure wether I manage to do it, I'll just ask questions > if I run into problems. If you have any LLVM specific question, please feel free to contact the llvmdev list (http://mail.cs.uiuc.edu/mailman/listinfo/llvmdev), the people on the list are quite helpful and would love to help you in any way they can. > - Are tuples really only used for returning multiple values from a > function? If yes they could be avoided altogether using additional > pointer arguments that point to where the return value should be > stored. On the LLVM side of things, the only way for a function to return multiple values is (as you mention) to pass a pointer that indicates where to store the result. In the future (say 3-4 months out), LLVM will be extended to allow functions to return multiple values in registers. Another thought: I see that you're currently using llc to build your programs, have you considered using the LLVM JIT? Anyway, if you have any questions or run into problems, again, we'd love to help, just let us know. :) -Chris -- http://nondot.org/sabre/ http://llvm.cs.uiuc.edu/ From hpk at trillke.net Thu Feb 17 10:10:08 2005 From: hpk at trillke.net (holger krekel) Date: Thu, 17 Feb 2005 10:10:08 +0100 Subject: [pypy-dev] RE: LLVM backend In-Reply-To: References: Message-ID: <20050217091008.GH28668@solar.trillke.net> Hey Chris! On Wed, Feb 16, 2005 at 23:29 -0600, Chris Lattner wrote: > it is great that you joined the pypy-dev list! Welcome! We have been following LLVM with interest for quite some time and are at a stage where we are seriously considering how we (re-)organize our translation backends and LLVM is a primary contender, (if we can make sure that the installation of the very basic tools is feasible :-) Special thanks, btw, to Carl Friedrich Bolz which went ahead with LLVM integration where the rest of us will have to catch up, soon. We are currently buried in refactorings to make type inference and some internal mechanisms on the pypy side easier ... When PyPy follows the LLVM route i personally hope that we can contribute something like a "(restricted) python frontend" for LLVM back (which would likely require an install of Python but that should be easier than a multitude of C++ libraries :-) cheers, holger From arathorn at fastwebnet.it Thu Feb 17 10:27:33 2005 From: arathorn at fastwebnet.it (Paolo Invernizzi) Date: Thu, 17 Feb 2005 10:27:33 +0100 Subject: [pypy-dev] Re: LLVM backend In-Reply-To: References: Message-ID: Chris Lattner wrote: > Another thought: I see that you're currently using llc to build your > programs, have you considered using the LLVM JIT? It would be interesting to have e Boost Python wrap of LLVM for playing with... One of the most valuable feature of LLVM is the generation of new code 'on-the-fly' with the JIT... My regret is that I have no time to play with that beautiful piece of code (pypy and, now also llvm) *sob* --- Paolo Invernizzi From cfbolz at gmx.de Thu Feb 17 11:08:53 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 17 Feb 2005 11:08:53 +0100 (MET) Subject: [pypy-dev] RE: LLVM backend References: Message-ID: <23305.1108634933@www35.gmx.net> Hi Chris, On Wed, 16 Feb 2005 23:29:01 -0600 (CST), Chris Lattner wrote: > > > > Carl Friedrich Bolz wrote: > > I just checked my LLVM-backend in (I hope I did nothing wrong). It > > resides in pypy/translator/llvm. > > Wow cool, I had no idea that you guys were this far along! Depends on what you call 'this' :-). At the moment I can only compile a small subset of RPython. RPython itself is an informally defined restricted subset of Python that makes it easy to be compiled (Disclaimer: I'm not really qualified to explain all this since I started with pypy rather recently). For example a variable should only hold values of the same type. The RPython source code is then converted to a flow graph in SSA form. The types of the variables in the flow graph can then be inferred, if the types of the entry functions are given. This is all done by some parts of pypy. The LLVM-backend has now a very easy job: A flow graph in SSA form with type-annotations maps directly to LLVM code. [snip] > > > - I think there should be some more intelligent way to produce the > > necessary LLLVM-implementations for the space operations of more > > complex types than just writing them in LLVM-assembler, which can be > > quite tedious (it's no fun writing programs in SSA form). > > Oh yeah, generating SSA is quite a pain. The traditional way that LLVM > front-ends deal with this is to use 'alloca's for local variables and use > explicit loads/stores to them. This generates really gross looking code, > but the LLVM optimizers (specifically mem2reg) rip them up. For example, > for something simple like: > > X = 1; > ... > = X; > ... > X = 2; > > You can generate code that looks like this: > > %X = alloca int ;; in the entry block for the fn > ... > store int 1, int* %X > ... > = load int* %X > ... > store int 2, int* %X > ... > > If you run this sort of code through the LLVM "-mem2reg" optimization, it > will promote all of these to SSA values, so you don't have to do it > yourself. If the "..."'s contain control flow, this is a non-trivial > task. :) The problem is not to transform RPython to SSA since the pypy-tools do all the difficult work. I was talking about the implementation of Python's more interesting types like lists and dictionaries. All the methods of these types have to be implemented somehow, which I did from hand in LLVM assembler at first. This is what I meant when I talked about 'pain' ;-). I still have no good solution for this. At the moment I do the following: The methods of the list objects are implemented in C as arrays of pointers to "object" and turned to LLVM code (by compiling and disassembling it). The result is used as a kind of template: All the occurences of the pointers to "object" are replaced by the type of the values the list is supposed to hold. This sounds rather brittle but works quite well at the moment. I will probably run into limitations whith this later. For example if I implement exceptions (which should not be too complicated using invoke and unwind) I can't raise them from within the C code that produces the list implementation. [snip] > Another thought: I see that you're currently using llc to build your > programs, have you considered using the LLVM JIT? At the moment I don't produce standalone programs but rather shared libraries that can be loaded into Python as modules to get access to the LLVM-compiled functions. So I really need to use llc. > > Anyway, if you have any questions or run into problems, again, we'd love > to help, just let us know. :) > > -Chris I'll do that. Thanks a lot. Regards, Carl Friedrich From sabre at nondot.org Thu Feb 17 17:09:41 2005 From: sabre at nondot.org (Chris Lattner) Date: Thu, 17 Feb 2005 10:09:41 -0600 (CST) Subject: [pypy-dev] RE: LLVM backend In-Reply-To: <20050217091008.GH28668@solar.trillke.net> References: <20050217091008.GH28668@solar.trillke.net> Message-ID: On Thu, 17 Feb 2005, holger krekel wrote: > Hey Chris! > > On Wed, Feb 16, 2005 at 23:29 -0600, Chris Lattner wrote: >> > > it is great that you joined the pypy-dev list! Welcome! Thanks! :) > We have been following LLVM with interest for quite some time > and are at a stage where we are seriously considering how we > (re-)organize our translation backends and LLVM is a primary > contender, (if we can make sure that the installation of the > very basic tools is feasible :-) > Special thanks, btw, to Carl Friedrich Bolz which went ahead > with LLVM integration where the rest of us will have to catch > up, soon. We are currently buried in refactorings to make type > inference and some internal mechanisms on the pypy side easier ... Ok. :) > When PyPy follows the LLVM route i personally hope that we can > contribute something like a "(restricted) python frontend" for > LLVM back (which would likely require an install of Python but > that should be easier than a multitude of C++ libraries :-) That would be great. C/C++ are pretty ugly, I'm sure a (even restricted) python front-end would be much nicer. :) -Chris -- http://nondot.org/sabre/ http://llvm.cs.uiuc.edu/ From sabre at nondot.org Thu Feb 17 17:50:12 2005 From: sabre at nondot.org (Chris Lattner) Date: Thu, 17 Feb 2005 10:50:12 -0600 (CST) Subject: [pypy-dev] RE: LLVM backend In-Reply-To: <23305.1108634933@www35.gmx.net> References: <23305.1108634933@www35.gmx.net> Message-ID: On Thu, 17 Feb 2005, Carl Friedrich Bolz wrote: >> Wow cool, I had no idea that you guys were this far along! > > Depends on what you call 'this' :-). At the moment I can only compile a > small subset of RPython. RPython itself is an informally defined restricted > subset of Python that makes it easy to be compiled (Disclaimer: I'm not > really qualified to explain all this since I started with pypy rather > recently). For example a variable should only hold values of the same type. > The RPython source code is then converted to a flow graph in SSA form. The > types of the variables in the flow graph can then be inferred, if the types > of the entry functions are given. This is all done by some parts of pypy. > The LLVM-backend has now a very easy job: A flow graph in SSA form with > type-annotations maps directly to LLVM code. Okay, that makes sense, a lot of sense in fact :) > The problem is not to transform RPython to SSA since the pypy-tools do all > the difficult work. Ok, sorry :( > I was talking about the implementation of Python's more interesting > types like lists and dictionaries. All the methods of these types have > to be implemented somehow, which I did from hand in LLVM assembler at > first. This is what I meant when I talked about 'pain' ;-). Ok. > I still have no good solution for this. At the moment I do the following: > The methods of the list objects are implemented in C as arrays of pointers > to "object" and turned to LLVM code (by compiling and disassembling it). The > result is used as a kind of template: All the occurences of the pointers to > "object" are replaced by the type of the values the list is supposed to > hold. This sounds rather brittle but works quite well at the moment. So let me see if I understand correctly: you're writing the code you want in C, compiling it with the LLVM C compiler, and using void*'s. This means you get things like "[100 x sbyte*]" instead of "[100 x dictionary*]" as you would like. To get back the correct type, you're having the compiler "copy and paste" this code, inserting the correct types as appropriate. To me, I don't think it's worth doing this. Instead, why not just compile these function to LLVM bytecode and link them into the program as you need them, inserting cases to/from sbyte* as appropriate? Sure, some amount of type-safety will be "lost" at the LLVM level, but I don't think that will significantly impeed the optimizers. The other nice thing about this approach is that you can modify the runtime library more easily, and performance won't be bad from the result: the LLVM inliner can inline these methods where it makes sense. Am I missing something? > I will probably run into limitations whith this later. For example if I > implement exceptions (which should not be too complicated using invoke and > unwind) I can't raise them from within the C code that produces the list > implementation. Yeah, that is annoying. One trick that works well for doing the 'unwind' is to just write a simple llvm function that unwinds, and call it from C. Invoke is a bit more tricky unfortunately. > [snip] >> Another thought: I see that you're currently using llc to build your >> programs, have you considered using the LLVM JIT? > > At the moment I don't produce standalone programs but rather shared > libraries that can be loaded into Python as modules to get access to the > LLVM-compiled functions. So I really need to use llc. Ah ok. For some reason, I thought you were doing things at runtime, sorry about that. :) >> Anyway, if you have any questions or run into problems, again, we'd love >> to help, just let us know. :) > > I'll do that. Thanks a lot. Sounds good, -Chris -- http://nondot.org/sabre/ http://llvm.cs.uiuc.edu/ From jriehl at cs.uchicago.edu Thu Feb 17 20:43:07 2005 From: jriehl at cs.uchicago.edu (Jonathan David Riehl) Date: Thu, 17 Feb 2005 13:43:07 -0600 (CST) Subject: [pypy-dev] RE: LLVM backend In-Reply-To: <23305.1108634933@www35.gmx.net> References: <23305.1108634933@www35.gmx.net> Message-ID: On Thu, 17 Feb 2005, Carl Friedrich Bolz wrote: > ... > The problem is not to transform RPython to SSA since the pypy-tools do all > the difficult work. I was talking about the implementation of Python's more > interesting types like lists and dictionaries. All the methods of these > types have to be implemented somehow, which I did from hand in LLVM > assembler at first. This is what I meant when I talked about 'pain' ;-). > > I still have no good solution for this. At the moment I do the following: > The methods of the list objects are implemented in C as arrays of pointers > to "object" and turned to LLVM code (by compiling and disassembling it). The > result is used as a kind of template: All the occurences of the pointers to > "object" are replaced by the type of the values the list is supposed to > hold. This sounds rather brittle but works quite well at the moment. > > I will probably run into limitations whith this later. For example if I > implement exceptions (which should not be too complicated using invoke and > unwind) I can't raise them from within the C code that produces the list > implementation. > ... Previous attempts to generate C code simply have inlined the calls the Python VM would have made. I wonder if you aren't making things too difficult by chosing not to leverage more of the Python C API. In PyFront, I wanted to use type inference to move more of the data flow into native C types. This model of just generating C code that does what the Python VM would have done has other benefits. Namely, the exception handling was simple, even if it was done in a brute force fashion. Each Python C API call was checked for an exception and goto's were used to goto exception handling code when NULL was returned. Anyway, I look forward to looking at what's new here, and hopefully have another RPython straw man (and implementation) to pitch at the PyCon sprint. -Jon From arigo at tunes.org Fri Feb 18 19:18:48 2005 From: arigo at tunes.org (Armin Rigo) Date: Fri, 18 Feb 2005 18:18:48 +0000 Subject: [pypy-dev] RE: LLVM backend In-Reply-To: References: <23305.1108634933@www35.gmx.net> Message-ID: <20050218181848.GA9446@vicky.ecs.soton.ac.uk> Hi Chris, On Thu, Feb 17, 2005 at 10:50:12AM -0600, Chris Lattner wrote: > So let me see if I understand correctly: you're writing the code you want > in C, compiling it with the LLVM C compiler, and using void*'s. This > means you get things like "[100 x sbyte*]" instead of "[100 x > dictionary*]" as you would like. To get back the correct type, you're > having the compiler "copy and paste" this code, inserting the correct > types as appropriate. > > To me, I don't think it's worth doing this. Instead, why not just compile > these function to LLVM bytecode and link them into the program as you need > them, inserting cases to/from sbyte* as appropriate? At some point, I can imagine that we will need a wider choice than just "a pointer". For example, an array of integers would hold "int" instead of a pointer. I guess you can cast between ints and pointers, but that's only the start of the trouble. An array of tuples (int, int) would ideally be stored as an array of struct { int a,b; }, and you can't fit such a struct into a single void*. Conversely, an array of void* is wasteful to implement strings as an array of characters... Armin From dialtone at divmod.com Sat Feb 19 00:10:30 2005 From: dialtone at divmod.com (Valentino Volonghi aka Dialtone) Date: Sat, 19 Feb 2005 00:10:30 +0100 Subject: [pypy-dev] Type inference documentation Message-ID: <421675E6.2080808@divmod.com> For poor people like me, pypy wannabe contributor, I think it would be good to have some updated documentation about type-inference/annotation as done in pypy. The only doc I can find is heavily outdated, as stated in the first lines of the file. Is this documentation planned? Sorry for the wrong post in py-dev ML :( cheers -- Valentino Volonghi aka Dialtone Now Running MacOSX 10.3.8 Blog: http://vvolonghi.blogspot.com http://weever.berlios.de From mwh at python.net Sat Feb 19 21:51:06 2005 From: mwh at python.net (Michael Hudson) Date: Sat, 19 Feb 2005 20:51:06 +0000 Subject: [pypy-dev] Re: Type inference documentation In-Reply-To: <421675E6.2080808@divmod.com> (Valentino Volonghi aka Dialtone's message of "Sat, 19 Feb 2005 00:10:30 +0100") References: <421675E6.2080808@divmod.com> Message-ID: <2mis4oxohx.fsf@starship.python.net> Valentino Volonghi aka Dialtone writes: > For poor people like me, pypy wannabe contributor, I think it would > be good to have some updated documentation about > type-inference/annotation as done in pypy. Would be nice, wouldn't it! > The only doc I can find is heavily outdated, as stated in the first > lines of the file. This is the danger of spending time writing documentation in the PyPy world. Mind you, I don't think we're going to throw everything out of the window again any time soon... > Is this documentation planned? Well, certainly. I don't know how soon, though. It's also one of the most mind bending parts of the project currently. Maybe you should start somewhere else... Cheers, mwh -- glyph: I don't know anything about reality. -- from Twisted.Quotes From mwh at python.net Sat Feb 19 22:19:59 2005 From: mwh at python.net (Michael Hudson) Date: Sat, 19 Feb 2005 21:19:59 +0000 Subject: [pypy-dev] Re: Type inference documentation In-Reply-To: <2mis4oxohx.fsf@starship.python.net> (Michael Hudson's message of "Sat, 19 Feb 2005 20:51:06 +0000") References: <421675E6.2080808@divmod.com> <2mis4oxohx.fsf@starship.python.net> Message-ID: <2macq0xn5s.fsf@starship.python.net> Michael Hudson writes: > Well, certainly. I don't know how soon, though. It's also one of > the most mind bending parts of the project currently. To make my point the way I originally intended too (sigh): If I was to work on lowering the bar to entry of the annotator, currently I would work on making it less incomprehensible before working on documentation. Cheers, mwh -- In that case I suggest that to get the correct image you look at the screen from inside the monitor whilst standing on your head. -- James Bonfield, http://www.ioccc.org/2000/rince.hint From hpk at trillke.net Sat Feb 19 22:39:40 2005 From: hpk at trillke.net (holger krekel) Date: Sat, 19 Feb 2005 22:39:40 +0100 Subject: [pypy-dev] Re: Type inference documentation In-Reply-To: <2macq0xn5s.fsf@starship.python.net> References: <421675E6.2080808@divmod.com> <2mis4oxohx.fsf@starship.python.net> <2macq0xn5s.fsf@starship.python.net> Message-ID: <20050219213940.GK28668@solar.trillke.net> On Sat, Feb 19, 2005 at 21:19 +0000, Michael Hudson wrote: > Michael Hudson writes: > > > Well, certainly. I don't know how soon, though. It's also one of > > the most mind bending parts of the project currently. > > To make my point the way I originally intended too (sigh): If I was to > work on lowering the bar to entry of the annotator, currently I would > work on making it less incomprehensible before working on > documentation. Some people even think that before making it less incrompehensible and writing documentation we should actually make it work :-) Nevertheless i would hope that some things can be said about annotation (our jargon for type inference) that would not be heavily outdated so soon. In fact, since the Vilnius sprint in November annotation seems to be relatively stable and we are shifting efforts to make the interpreter/StdObjectSpace behave in a way that allows the current annotation to work. Btw, in the just re-merged "interpapp" branch we wrestled quite a bit with the interactions between interpreter level code and application level code that are kind of a mine field for type inference, especially exposing interp-level functions at application level (like all the descriptors). Here it also seems to make sense to document the new cleaned up way of doing app/interp interactions, including the new "LazyModule" mechanism for the sys and builtin module which are now realized as packages whose name bindings get resolved lazily. (have a look into pypy/module/builtin and sys2). cheers, holger From mwh at python.net Sat Feb 19 23:58:44 2005 From: mwh at python.net (Michael Hudson) Date: Sat, 19 Feb 2005 22:58:44 +0000 Subject: [pypy-dev] Re: Type inference documentation In-Reply-To: <20050219213940.GK28668@solar.trillke.net> (holger krekel's message of "Sat, 19 Feb 2005 22:39:40 +0100") References: <421675E6.2080808@divmod.com> <2mis4oxohx.fsf@starship.python.net> <2macq0xn5s.fsf@starship.python.net> <20050219213940.GK28668@solar.trillke.net> Message-ID: <2mzmy0w40r.fsf@starship.python.net> hpk at trillke.net (holger krekel) writes: > On Sat, Feb 19, 2005 at 21:19 +0000, Michael Hudson wrote: > >> To make my point the way I originally intended too (sigh): If I was to >> work on lowering the bar to entry of the annotator, currently I would >> work on making it less incomprehensible before working on >> documentation. > > Some people even think that before making it less incrompehensible > and writing documentation we should actually make it work :-) Different premise :) Cheers, mwh -- ARTHUR: But which is probably incapable of drinking the coffee. -- The Hitch-Hikers Guide to the Galaxy, Episode 6 From cfbolz at gmx.de Sun Feb 20 23:01:22 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sun, 20 Feb 2005 23:01:22 +0100 (MET) Subject: [pypy-dev] RE: LLVM backend References: Message-ID: <11836.1108936882@www42.gmx.net> Hi Chris, On Thu, 17 Feb 2005 10:50:12 -0600 (CST), Chris Lattner wrote: [snip] > So let me see if I understand correctly: you're writing the code you want > in C, compiling it with the LLVM C compiler, and using void*'s. This > means you get things like "[100 x sbyte*]" instead of "[100 x > dictionary*]" as you would like. To get back the correct type, you're > having the compiler "copy and paste" this code, inserting the correct > types as appropriate. > > To me, I don't think it's worth doing this. Instead, why not just > compile these function to LLVM bytecode and link them into the program as > you need them, inserting cases to/from sbyte* as appropriate? Sure, some > amount of type-safety will be "lost" at the LLVM level, but I don't think > that will significantly impeed the optimizers. The other nice thing > about this approach is that you can modify the runtime library more > easily, and performance won't be bad from the result: the LLVM inliner > can inline these methods where it makes sense. Am I missing something? Well, it was extremely easy to implement which justifies the approach a bit ;-). Since it works quite well I think I'll stick with it a bit and change it if problems come up (for example if there are lists of so many different types that the code bloat gets unbearable). >> I will probably run into limitations whith this later. For example if I >> implement exceptions (which should not be too complicated using invoke >> and unwind) I can't raise them from within the C code that produces the >> list implementation. > > Yeah, that is annoying. One trick that works well for doing the 'unwind' > is to just write a simple llvm function that unwinds, and call it from C. > > Invoke is a bit more tricky unfortunately. Ok, maybe it is enough to unwind from within those functions. I can't think of any string, list, dict... method that actually catches an exception. So it seems best to set some global variables that indicate the error and then call a function written in llvm that unwinds. Does this sound ok? [snip] Thanks for your help! Regards, Carl Friedrich From cfbolz at gmx.de Sun Feb 20 23:02:55 2005 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sun, 20 Feb 2005 23:02:55 +0100 (MET) Subject: [pypy-dev] RE: LLVM backend References: Message-ID: <14847.1108936975@www42.gmx.net> Hi Jon, On Thu, 17 Feb 2005 13:43:07 -0600 (CST) Jonathan David Riehl wrote: > Previous attempts to generate C code simply have inlined the calls the > Python VM would have made. I wonder if you aren't making things too > difficult by chosing not to leverage more of the Python C API. In > PyFront, I wanted to use type inference to move more of the data flow > into native C types. I think that's basically what genc (the C code generator) does: It replaces the RPython code by equivalent calls to the Python C API (to the best of my knowledge: I don't really know genc very well). With genllvm I explicitely try not to use the Python C API. I don't think that that's making things too dificult: RPython is designed to make translation to a low level language easy. > This model of just generating C code that does what the Python VM would > have done has other benefits. Namely, the exception handling was simple, > even if it was done in a brute force fashion. Each Python C API call was > checked for an exception and goto's were used to goto exception handling > code when NULL was returned. Oh, I think Exception handling should be relatively easy in LLVM too since it has excellent exception support. There is an 'unwind' instruction that unwinds the stack and and 'invoke' instruction: invoke calls a function and specifies two basic blocks. If the called function returns normally, execution is continued in the first basic block. If callee (or some subsequent function) calls unwind, execution is continued in the second basic block. > Anyway, I look forward to looking at what's new here, and hopefully have > another RPython straw man (and implementation) to pitch at the PyCon > sprint. I won't come to PyCon (but maybe I can hang around on IRC, if someone wants to take a look a genllvm and has questions). Regards, Carl Friedrich From sabre at nondot.org Sun Feb 20 23:15:52 2005 From: sabre at nondot.org (Chris Lattner) Date: Sun, 20 Feb 2005 16:15:52 -0600 (CST) Subject: [pypy-dev] RE: LLVM backend In-Reply-To: <11836.1108936882@www42.gmx.net> References: <11836.1108936882@www42.gmx.net> Message-ID: On Sun, 20 Feb 2005, Carl Friedrich Bolz wrote: >> easily, and performance won't be bad from the result: the LLVM inliner >> can inline these methods where it makes sense. Am I missing something? > > Well, it was extremely easy to implement which justifies the approach a bit > ;-). Since it works quite well I think I'll stick with it a bit and change > it if problems come up (for example if there are lists of so many different > types that the code bloat gets unbearable). ok :) >> Invoke is a bit more tricky unfortunately. > > Ok, maybe it is enough to unwind from within those functions. I can't think > of any string, list, dict... method that actually catches an exception. So > it seems best to set some global variables that indicate the error and then > call a function written in llvm that unwinds. Does this sound ok? Yup, makes sense, -Chris -- http://nondot.org/sabre/ http://llvm.cs.uiuc.edu/ From sanxiyn at gmail.com Mon Feb 21 01:59:58 2005 From: sanxiyn at gmail.com (Sanghyeon Seo) Date: Mon, 21 Feb 2005 09:59:58 +0900 Subject: [pypy-dev] Re: [pypy-svn] r9372 - pypy/dist/pypy/lib In-Reply-To: <20050221004637.AEBCB27BBA@code1.codespeak.net> References: <20050221004637.AEBCB27BBA@code1.codespeak.net> Message-ID: <5b02481705022016594170f44@mail.gmail.com> I wrote: > Log: > _classobj's geninterplevel_name: Don't reference w_builtin > > --- pypy/dist/pypy/lib/_classobj.py (original) > +++ pypy/dist/pypy/lib/_classobj.py Mon Feb 21 01:46:37 2005 > @@ -101,7 +101,7 @@ > return > i += 1 > # let geninterplevel retrieve the PyPy builtin instead > -_seqiter.geninterplevel_name = lambda gen: "(space.getattr(space.w_builtin, %s))" % gen.nameof('_seqiter') > +_seqiter.geninterplevel_name = lambda gen: "space.builtin.get('_seqiter')" > > OLD_STYLE_CLASSES_IMPL = object() However, this doesn't work... How to get this right? Seo Sanghyeon