From regmi.amit at gmail.com Thu Nov 1 05:03:19 2007 From: regmi.amit at gmail.com (amit) Date: Thu, 01 Nov 2007 00:03:19 -0400 Subject: [pypy-dev] Hi all. I am Looking for a project idea. Message-ID: <47295007.6060305@gmail.com> Hi, I went across this page http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html My project requires me to modify an existing compiler or interpreter, or write a program from scratch. I could not figure out how much time it would take to say write an interpreter for Aldor. If someone has some suggestion on any idea that I could put an effort into for 4 weeks (that's when my project work is due) I would be very happy to learn more of pypy. I have fair amount of python coding experience. Regards, Regmee Univ. of Western Ontario, CA From lac at openend.se Thu Nov 1 06:07:12 2007 From: lac at openend.se (Laura Creighton) Date: Thu, 01 Nov 2007 06:07:12 +0100 Subject: [pypy-dev] Hi all. I am Looking for a project idea. In-Reply-To: Message from amit of "Thu, 01 Nov 2007 00:03:19 -0400." <47295007.6060305@gmail.com> References: <47295007.6060305@gmail.com> Message-ID: <200711010507.lA157CBg003464@theraft.openend.se> In a message of Thu, 01 Nov 2007 00:03:19 -0400, amit writes: >Hi, > >I went across this page >http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html > >My project requires me to modify an existing compiler or interpreter, or >write a program from scratch. I could not figure out how much time it >would take to say write an interpreter for Aldor. If someone has some >suggestion on any idea that I could put an effort into for 4 weeks >(that's when my project work is due) I would be very happy to learn more >of pypy. > >I have fair amount of python coding experience. > > >Regards, >Regmee >Univ. of Western Ontario, CA The pypy team hangs out on irc.freenode.net in the channel #pypy. Most of us live in Europe, so we'll already be there when you get this mail. All I know about Aldor is that it is a functional language that some people use to do math. I went here: http://www.aldor.org/wiki/index.php/Main_Page and didn't learn much because most of the links are stale. Whether modifying pypy makes sense depends a lot on what sort of language you would like to generate. PyPy was designed for dynamic langauges -- so a functional one, while not impossible, may not be the best fit. Do you have to produce an Aldor compiler? The other thing that matters is exactly what your teacher wants you to learn. PyPy's architecture is unique in the world, and we will be happy to talk your ear off about why this is a) necessary and b) a great idea. But it may not be what your teacher is expecting, which can be either fantastic or extremely unfortunate, depending on the teacher. come by and talk to us, Laura Creighton ps - 4 of us are right now putting the polish on a presentation we are giving to IBM Research in NY on Monday. So we have less time to chat than usual. But the rest of the usual suspects will be around :-) From cfbolz at gmx.de Thu Nov 1 10:38:10 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 01 Nov 2007 10:38:10 +0100 Subject: [pypy-dev] Hi all. I am Looking for a project idea. In-Reply-To: <47295007.6060305@gmail.com> References: <47295007.6060305@gmail.com> Message-ID: <47299E82.40805@gmx.de> Hi Amit, amit wrote: > I went across this page > http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html > > My project requires me to modify an existing compiler or interpreter, or > write a program from scratch. I could not figure out how much time it > would take to say write an interpreter for Aldor. If someone has some > suggestion on any idea that I could put an effort into for 4 weeks > (that's when my project work is due) I would be very happy to learn more > of pypy. > > I have fair amount of python coding experience. The answer, as usual, is "depends". Starting an interpreter for this language from scratch is probably very hard if you are not re-using code from existing implementations. Is there some sort of intermediate code format that you can reuse? Can you reuse the parser and type inferencing engine somehow (since you probably don't want to rewrite that)? Another reason why Aldor might not be suited well (if I understand things correctly) is that it is an extension language to a CAS system, which means you would need to call back into the CAS at certain points. This is probably quite some additional effort which is not directly relevant to the real problem (writing an interpreter). If you are not dead-set on Aldor you could think about implementing another language or about working on one of our existing interpreters (or on the translation toolchain, which really is a compiler). An example of a project that would be useful for us and involves a bit of compiler and a bit of interpreter hacking is the following: Allow PyPy-JVM (the PyPy Python interpreter translated to the JVM) to use Java classes from within Python code, similarly to what Jython provides. This is implemented for PyPy-CLI already, so you could roughly follow this implementation. It would need some hacking on the translation toolchain, especially on the Java backend to support interaction with Java classes. It needs a bit of knowledge of the JVM, especially of reflection to dynamically use java classes. And finally it needs some hacking on the interpreter to expose these features to the user. It's probably a bit much for a four week project, but both Anto Cuni (the author of the CLI backend and general PyPy contributor) and me could mentor you a bit, if you feel like it. Anyway, that's just one idea, we can also think of others, if you tell us a bit more about your interests. Cheers, Carl Friedrich From cfbolz at gmx.de Thu Nov 1 10:47:05 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 01 Nov 2007 10:47:05 +0100 Subject: [pypy-dev] Hi all. I am Looking for a project idea. In-Reply-To: <200711010507.lA157CBg003464@theraft.openend.se> References: <47295007.6060305@gmail.com> <200711010507.lA157CBg003464@theraft.openend.se> Message-ID: <4729A099.5050003@gmx.de> Hi Laura! Laura Creighton wrote: [snip] > Whether modifying pypy makes sense depends a lot on what sort of > language you would like to generate. PyPy was designed for dynamic > langauges -- so a functional one, while not impossible, may not be the > best fit. Just about this particular point: I am not sure that functional languages per se are a bad fit for PyPy. On the one hand there are functional languages such as Erlang that are dynamically typed ("dynamic" and "functional" are really orthogonal properties). On the other hand the implementation of some functional languages uses the staticness of the type system mostly "on the outside" to report to the user and not "on the inside" to actually implement the language. I guess this is especially true for an interpreter of a functional language. I might be totally misinformed there, though. [snip] Cheers, Carl Friedrich From lac at openend.se Thu Nov 1 12:38:51 2007 From: lac at openend.se (Laura Creighton) Date: Thu, 01 Nov 2007 12:38:51 +0100 Subject: [pypy-dev] Hi all. I am Looking for a project idea. In-Reply-To: Message from Carl Friedrich Bolz of "Thu, 01 Nov 2007 10:47:05 +0100." <4729A099.5050003@gmx.de> References: <47295007.6060305@gmail.com> <200711010507.lA157CBg003464@theraft.openend.se> <4729A099.5050003@gmx.de> Message-ID: <200711011138.lA1BcpDG004812@theraft.openend.se> In a message of Thu, 01 Nov 2007 10:47:05 +0100, Carl Friedrich Bolz writes: >Hi Laura! > >Laura Creighton wrote: >[snip] >> Whether modifying pypy makes sense depends a lot on what sort of >> language you would like to generate. PyPy was designed for dynamic >> langauges -- so a functional one, while not impossible, may not be the >> best fit. > >Just about this particular point: I am not sure that functional >languages per se are a bad fit for PyPy. On the one hand there are >functional languages such as Erlang that are dynamically typed >("dynamic" and "functional" are really orthogonal properties). On the >other hand the implementation of some functional languages uses the >staticness of the type system mostly "on the outside" to report to the >user and not "on the inside" to actually implement the language. I guess >this is especially true for an interpreter of a functional language. I >might be totally misinformed there, though. > >[snip] > >Cheers, > >Carl Friedrich I think that all that what I wrote indicates is my belief that writing a Haskell front end for PyPy would be hard. Thanks for taking my foot out of my mouth. Laura From cfbolz at gmx.de Thu Nov 1 12:48:00 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 01 Nov 2007 12:48:00 +0100 Subject: [pypy-dev] Hi all. I am Looking for a project idea. In-Reply-To: <200711011138.lA1BcpDG004812@theraft.openend.se> References: <47295007.6060305@gmail.com> <200711010507.lA157CBg003464@theraft.openend.se> <4729A099.5050003@gmx.de> <200711011138.lA1BcpDG004812@theraft.openend.se> Message-ID: <4729BCF0.9080305@gmx.de> Hi Laura, Laura Creighton wrote: > I think that all that what I wrote indicates is my belief that > writing a Haskell front end for PyPy would be hard. Thanks for > taking my foot out of my mouth. I am not even completely sure about that, if you reuse the parser, typechecker etc from Haskell (it's probably even an interesting research question whether our JIT would help there), but let's see until someone is crazy enough to try :-). Cheers, Carl Friedrich From ondrej at certik.cz Thu Nov 1 12:56:36 2007 From: ondrej at certik.cz (Ondrej Certik) Date: Thu, 1 Nov 2007 12:56:36 +0100 Subject: [pypy-dev] Hi all. I am Looking for a project idea. In-Reply-To: <4729BCF0.9080305@gmx.de> References: <47295007.6060305@gmail.com> <200711010507.lA157CBg003464@theraft.openend.se> <4729A099.5050003@gmx.de> <200711011138.lA1BcpDG004812@theraft.openend.se> <4729BCF0.9080305@gmx.de> Message-ID: <85b5c3130711010456k2c9ff94cufaf41e94fb397177@mail.gmail.com> > I am not even completely sure about that, if you reuse the parser, > typechecker etc from Haskell (it's probably even an interesting research > question whether our JIT would help there), but let's see until someone > is crazy enough to try :-). How about writing an interpreter for the Mathematica language [1]? There are so many scientific programs written in Mathematica and if they could be run on an opensource platform, it'd be cool. We were thinking about it in SymPy [2], but I don't have much time nor knowledge to do it. But I can help with the symbolic facilities. Also as a backend to the Aldor interpreter, if needed. But at the beginning, would it be hard to write at least some simple parser to the most common subset of Mathematica language, together with SymPy as a backend to do the symbolic calculations? I'd like to do it to see which things are still not working correctly in SymPy. And also so that it's not hard for people who know Mathematica, to switch. Ondrej [1] http://reference.wolfram.com/mathematica/guide/LanguageOverview.html [2] http://code.google.com/p/sympy/issues/detail?id=161 From regmi.amit at gmail.com Thu Nov 1 15:14:33 2007 From: regmi.amit at gmail.com (amit) Date: Thu, 01 Nov 2007 10:14:33 -0400 Subject: [pypy-dev] Hi all. I am Looking for a project idea. In-Reply-To: <47299E82.40805@gmx.de> References: <47295007.6060305@gmail.com> <47299E82.40805@gmx.de> Message-ID: <4729DF49.3070302@gmail.com> Hi Carl, Anto, This is what I am going to play with for the next few weeks. Idea: "Allow PyPy-JVM to use Java classes from within Python code" OR "pypy-jvm integration for ... " Some hacking to do :-) a) on the translation toolchain, especially on the Java backend to support interaction with Java classes. ( not sure what it really means ) b) on the PyPy interpreter to expose these features to the user. Steps I should follow: a) look at the implemented for PyPy-CLI (look at /pypy/module/clr and pypy/translator/cli/dotnet.py) b) gain some knowledge of reflection in JVM and the bytecode, to use java classes. Pinch me if I am wrong. And give me more pointers which would make my life easier. - Regmee Carl Friedrich Bolz wrote: > Hi Amit, > > amit wrote: >> I went across this page >> http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html >> >> My project requires me to modify an existing compiler or interpreter, >> or write a program from scratch. I could not figure out how much time >> it would take to say write an interpreter for Aldor. If someone has >> some suggestion on any idea that I could put an effort into for 4 >> weeks (that's when my project work is due) I would be very happy to >> learn more of pypy. >> >> I have fair amount of python coding experience. > > The answer, as usual, is "depends". Starting an interpreter for this > language from scratch is probably very hard if you are not re-using > code from existing implementations. Is there some sort of intermediate > code format that you can reuse? Can you reuse the parser and type > inferencing engine somehow (since you probably don't want to rewrite > that)? > > Another reason why Aldor might not be suited well (if I understand > things correctly) is that it is an extension language to a CAS system, > which means you would need to call back into the CAS at certain > points. This is probably quite some additional effort which is not > directly relevant to the real problem (writing an interpreter). > > If you are not dead-set on Aldor you could think about implementing > another language or about working on one of our existing interpreters > (or on the translation toolchain, which really is a compiler). > > An example of a project that would be useful for us and involves a bit > of compiler and a bit of interpreter hacking is the following: Allow > PyPy-JVM (the PyPy Python interpreter translated to the JVM) to use > Java classes from within Python code, similarly to what Jython > provides. This is implemented for PyPy-CLI already, so you could > roughly follow this implementation. It would need some hacking on the > translation toolchain, especially on the Java backend to support > interaction with Java classes. It needs a bit of knowledge of the JVM, > especially of reflection to dynamically use java classes. And finally > it needs some hacking on the interpreter to expose these features to > the user. > > It's probably a bit much for a four week project, but both Anto Cuni > (the author of the CLI backend and general PyPy contributor) and me > could mentor you a bit, if you feel like it. > > Anyway, that's just one idea, we can also think of others, if you tell > us a bit more about your interests. > > Cheers, > > Carl Friedrich > From tismer at stackless.com Fri Nov 2 20:29:40 2007 From: tismer at stackless.com (Christian Tismer) Date: Fri, 02 Nov 2007 20:29:40 +0100 Subject: [pypy-dev] windows path issue solved Message-ID: <472B7AA4.9020905@stackless.com> Hi, it took a while to find a way to get os initialization right on windows, especially because the problem pops up only when pypy is compiled. sys.executable is not in all cases set correctly, yet, but it suffices to find pypy's libraries reliably. see revision 48247 cheers - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From fijal at genesilico.pl Sun Nov 4 12:20:23 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 04 Nov 2007 12:20:23 +0100 Subject: [pypy-dev] Hi all. I am Looking for a project idea. In-Reply-To: <47299E82.40805@gmx.de> References: <47295007.6060305@gmail.com> <47299E82.40805@gmx.de> Message-ID: <472DAAF7.8060607@genesilico.pl> > It's probably a bit much for a four week project, but both Anto Cuni > (the author of the CLI backend and general PyPy contributor) and me > could mentor you a bit, if you feel like it. > > Anyway, that's just one idea, we can also think of others, if you tell > us a bit more about your interests. > > Cheers, > > Carl Friedrich > Hi amit. I would be happy to help as well (just came back from short trip). Cheers, fijal :. From pedronis at openend.se Sun Nov 4 16:01:48 2007 From: pedronis at openend.se (Samuele Pedroni) Date: Sun, 04 Nov 2007 16:01:48 +0100 Subject: [pypy-dev] [pypy-svn] r48286 - pypy/dist/pypy/rpython In-Reply-To: <20071104141633.DE4BF81DC@code0.codespeak.net> References: <20071104141633.DE4BF81DC@code0.codespeak.net> Message-ID: <472DDEDC.4070003@openend.se> are you sure this tweak is still correct, I cannot remember the intention of this code but I'm sure it was not as it was for random reasons... fijal at codespeak.net wrote: > Author: fijal > Date: Sun Nov 4 15:16:32 2007 > New Revision: 48286 > > Modified: > pypy/dist/pypy/rpython/rpbc.py > Log: > This effectively mutes some rpbc rtyper warnings > > > Modified: pypy/dist/pypy/rpython/rpbc.py > ============================================================================== > --- pypy/dist/pypy/rpython/rpbc.py (original) > +++ pypy/dist/pypy/rpython/rpbc.py Sun Nov 4 15:16:32 2007 > @@ -481,8 +481,8 @@ > if r_value.lowleveltype is Void: > continue > try: > - thisattrvalue = frozendesc.attrcache[attr] > - except KeyError: > + thisattrvalue = frozendesc.read_attribute(attr) > + except AttributeError: > warning("Desc %r has no attribute %r" % (frozendesc, attr)) > continue > llvalue = r_value.convert_const(thisattrvalue) > _______________________________________________ > pypy-svn mailing list > pypy-svn at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-svn > From fijal at genesilico.pl Sun Nov 4 16:17:48 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 04 Nov 2007 16:17:48 +0100 Subject: [pypy-dev] [pypy-svn] r48286 - pypy/dist/pypy/rpython In-Reply-To: <472DDEDC.4070003@openend.se> References: <20071104141633.DE4BF81DC@code0.codespeak.net> <472DDEDC.4070003@openend.se> Message-ID: <472DE29C.1000508@genesilico.pl> Samuele Pedroni wrote: > are you sure this tweak is still correct, I cannot remember the > intention of this code but I'm sure > it was not as it was for random reasons.. I've checked in different version, which does not issue warning only, but does not change semantics of that. :. From arigo at tunes.org Sun Nov 4 16:28:13 2007 From: arigo at tunes.org (Armin Rigo) Date: Sun, 4 Nov 2007 16:28:13 +0100 Subject: [pypy-dev] [pypy-svn] r48286 - pypy/dist/pypy/rpython In-Reply-To: <472DDEDC.4070003@openend.se> References: <20071104141633.DE4BF81DC@code0.codespeak.net> <472DDEDC.4070003@openend.se> Message-ID: <20071104152813.GA22792@code0.codespeak.net> Hi Fijal, On Sun, Nov 04, 2007 at 04:01:48PM +0100, Samuele Pedroni wrote: > > - thisattrvalue = frozendesc.attrcache[attr] > > - except KeyError: > > + thisattrvalue = frozendesc.read_attribute(attr) > > + except AttributeError: Argh! That's basically too dangerous. Please revert this checkin. There is a reason why we wrote it the other way around. I don't want to go into details now but basically what you did will ask for a new annotation, which should absolutely not be done during RTyping but only during the annotation phase. Armin From fijal at genesilico.pl Sun Nov 4 16:36:49 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 04 Nov 2007 16:36:49 +0100 Subject: [pypy-dev] [pypy-svn] r48286 - pypy/dist/pypy/rpython In-Reply-To: <20071104152813.GA22792@code0.codespeak.net> References: <20071104141633.DE4BF81DC@code0.codespeak.net> <472DDEDC.4070003@openend.se> <20071104152813.GA22792@code0.codespeak.net> Message-ID: <472DE711.1040105@genesilico.pl> Armin Rigo wrote: > Hi Fijal, > > On Sun, Nov 04, 2007 at 04:01:48PM +0100, Samuele Pedroni wrote: > >>> - thisattrvalue = frozendesc.attrcache[attr] >>> - except KeyError: >>> + thisattrvalue = frozendesc.read_attribute(attr) >>> + except AttributeError: >>> > > Argh! That's basically too dangerous. Please revert this checkin. > There is a reason why we wrote it the other way around. I don't want to > go into details now but basically what you did will ask for a new > annotation, which should absolutely not be done during RTyping but only > during the annotation phase. > > > Armin > > Ok, ok :) I did this already. :. From exarkun at divmod.com Sun Nov 4 17:16:43 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Sun, 4 Nov 2007 11:16:43 -0500 Subject: [pypy-dev] [pypy-svn] r48286 - pypy/dist/pypy/rpython In-Reply-To: <20071104152813.GA22792@code0.codespeak.net> Message-ID: <20071104161643.8162.1312480229.divmod.quotient.31532@ohm> On Sun, 4 Nov 2007 16:28:13 +0100, Armin Rigo wrote: >Hi Fijal, > >On Sun, Nov 04, 2007 at 04:01:48PM +0100, Samuele Pedroni wrote: >> > - thisattrvalue = frozendesc.attrcache[attr] >> > - except KeyError: >> > + thisattrvalue = frozendesc.read_attribute(attr) >> > + except AttributeError: > >Argh! That's basically too dangerous. Please revert this checkin. >There is a reason why we wrote it the other way around. I don't want to >go into details now but basically what you did will ask for a new >annotation, which should absolutely not be done during RTyping but only >during the annotation phase. > Did a unit test break? :) > >Armin >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev > From lac at openend.se Tue Nov 6 13:00:28 2007 From: lac at openend.se (Laura Creighton) Date: Tue, 6 Nov 2007 13:00:28 +0100 Subject: [pypy-dev] gbg sprint Message-ID: <200711061200.lA6C0S0K023498@theraft.openend.se> If you go to http://www.gso.se/ change the language to english, if you prefer, and then look at the calendar you can see what is playing at the concerthouse. jacob and I already have tickets for the piano concert which should be fantastic on the 20th and then have tickets for the 22nd for the Ravel concert. Anybody want me to get some more tickets for them? Laura From cfbolz at gmx.de Tue Nov 6 13:07:29 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 06 Nov 2007 13:07:29 +0100 Subject: [pypy-dev] RPython unicode support design question Message-ID: <47305901.3080001@gmx.de> Hi all, Maciek is working on adding unicode support for RPython and him and me were discussing some of the design issues. One of the questions we had is how mixing of strings and unicode should work in RPython. My proposal is the following: forbid _all_ mixing of strings and unicode in RPython. Mixing of strings and unicode would result in a SomeObject (e.g. "hello" + u"world" wouldn't be allowed). The only way to turn a byte string into a unicode object would be to use the encode and decode methods and I guess in the beginning the only allowed encoding would be "ascii" (we can think about supporting other encodings later, or leave it to something in pypy/rlib). This has many advantages, it forces the programmer to cleanly separate between unicode and byte strings and it makes some part of the implementation less messy. Does that make sense to everybody? Are there other proposals? Cheers, Carl Friedrich From amauryfa at gmail.com Tue Nov 6 13:49:34 2007 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 6 Nov 2007 13:49:34 +0100 Subject: [pypy-dev] RPython unicode support design question In-Reply-To: <47305901.3080001@gmx.de> References: <47305901.3080001@gmx.de> Message-ID: Carl Friedrich Bolz wrote: > Hi all, > > Maciek is working on adding unicode support for RPython and him and me > were discussing some of the design issues. One of the questions we had > is how mixing of strings and unicode should work in RPython. > > My proposal is the following: forbid _all_ mixing of strings and unicode > in RPython. Mixing of strings and unicode would result in a SomeObject > (e.g. "hello" + u"world" wouldn't be allowed). The only way to turn a > byte string into a unicode object would be to use the encode and decode > methods and I guess in the beginning the only allowed encoding would > be "ascii" (we can think about supporting other encodings later, or > leave it to something in pypy/rlib). > > This has many advantages, it forces the programmer to cleanly separate > between unicode and byte strings and it makes some part of the > implementation less messy. +1. Python3.0 takes the same direction, and RPython is supposed to be a subset of CPython. -- Amaury Forgeot d'Arc From arigo at tunes.org Tue Nov 6 14:23:31 2007 From: arigo at tunes.org (Armin Rigo) Date: Tue, 6 Nov 2007 14:23:31 +0100 Subject: [pypy-dev] RPython unicode support design question In-Reply-To: <47305901.3080001@gmx.de> References: <47305901.3080001@gmx.de> Message-ID: <20071106132331.GA20805@code0.codespeak.net> Hi Carl, On Tue, Nov 06, 2007 at 01:07:29PM +0100, Carl Friedrich Bolz wrote: > My proposal is the following: forbid _all_ mixing of strings and unicode > in RPython. Right now we allow mixing plain and unicode characters, at least in the annotator. There is at least one use case for this: you can write in a uniform way some code that gets two specialized versions, one for string and one for unicode. I use this e.g. in objspace.std.formatting, where two versions of the formatter build a list called 'result' in both cases, but which is either a list of chars or a list of unichars. It's full of code like result += '0' * padding which works in both cases because the character is turned into a unichar if necessary - that is, if 'result' is already known to be a list of unichars. So while I see the motivation for forbidding implicit conversion, I also think that there are good use cases where it makes sense to have it - at least for single characters. I guess that, as usual, the "RPython way" is to forbid implicit conversions from full strings to full unicode strings, until the day where we have a good use case for it :-) Armin From arigo at tunes.org Tue Nov 6 14:29:01 2007 From: arigo at tunes.org (Armin Rigo) Date: Tue, 6 Nov 2007 14:29:01 +0100 Subject: [pypy-dev] gbg sprint In-Reply-To: <200711061200.lA6C0S0K023498@theraft.openend.se> References: <200711061200.lA6C0S0K023498@theraft.openend.se> Message-ID: <20071106132901.GA21832@code0.codespeak.net> Hi Laura, On Tue, Nov 06, 2007 at 01:00:28PM +0100, Laura Creighton wrote: > jacob and I already have tickets for the piano concert which > should be fantastic on the 20th and then have tickets for the 22nd > for the Ravel concert. Anybody want me to get some more tickets for > them? While this sounds fantastic, I'm afraid I'll be far too tired for anything at all, on these evenings so close to our return from this crazy trip of ours. Armin From lac at openend.se Tue Nov 6 14:31:35 2007 From: lac at openend.se (Laura Creighton) Date: Tue, 06 Nov 2007 14:31:35 +0100 Subject: [pypy-dev] gbg sprint In-Reply-To: Message from Armin Rigo of "Tue, 06 Nov 2007 14:29:01 +0100." <20071106132901.GA21832@code0.codespeak.net> References: <200711061200.lA6C0S0K023498@theraft.openend.se> <20071106132901.GA21832@code0.codespeak.net> Message-ID: <200711061331.lA6DVZ57006999@theraft.openend.se> In a message of Tue, 06 Nov 2007 14:29:01 +0100, Armin Rigo writes: >Hi Laura, > >On Tue, Nov 06, 2007 at 01:00:28PM +0100, Laura Creighton wrote: >> jacob and I already have tickets for the piano concert which >> should be fantastic on the 20th and then have tickets for the 22nd >> for the Ravel concert. Anybody want me to get some more tickets for >> them? > >While this sounds fantastic, I'm afraid I'll be far too tired for >anything at all, on these evenings so close to our return from this >crazy trip of ours. > > >Armin I may be in the same state. Hope not. But I suspect I can find a home for my season's ticket if that is the case. Laura From cfbolz at gmx.de Tue Nov 6 14:36:22 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 06 Nov 2007 14:36:22 +0100 Subject: [pypy-dev] gbg sprint In-Reply-To: <200711061331.lA6DVZ57006999@theraft.openend.se> References: <200711061200.lA6C0S0K023498@theraft.openend.se> <20071106132901.GA21832@code0.codespeak.net> <200711061331.lA6DVZ57006999@theraft.openend.se> Message-ID: <47306DD6.1030709@gmx.de> Laura Creighton wrote: > In a message of Tue, 06 Nov 2007 14:29:01 +0100, Armin Rigo writes: >> Hi Laura, >> >> On Tue, Nov 06, 2007 at 01:00:28PM +0100, Laura Creighton wrote: >>> jacob and I already have tickets for the piano concert which >>> should be fantastic on the 20th and then have tickets for the 22nd >>> for the Ravel concert. Anybody want me to get some more tickets for >>> them? >> While this sounds fantastic, I'm afraid I'll be far too tired for >> anything at all, on these evenings so close to our return from this >> crazy trip of ours. >> >> >> Armin > > I may be in the same state. Hope not. But I suspect I can find a > home for my season's ticket if that is the case. > Whew, is it that bad _already_? Cheers, Carl Friedrich From cfbolz at gmx.de Wed Nov 7 10:02:44 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 07 Nov 2007 10:02:44 +0100 Subject: [pypy-dev] [pypy-svn] r48350 - pypy/dist/pypy/objspace/flow In-Reply-To: <20071107041858.EE8DC81AB@code0.codespeak.net> References: <20071107041858.EE8DC81AB@code0.codespeak.net> Message-ID: <47317F34.8090000@gmx.de> tismer at codespeak.net wrote: > Author: tismer > Date: Wed Nov 7 05:18:57 2007 > New Revision: 48350 > > Modified: > pypy/dist/pypy/objspace/flow/specialcase.py > Log: > some over-due relaxing to __import__ annotation > > Modified: pypy/dist/pypy/objspace/flow/specialcase.py > ============================================================================== > --- pypy/dist/pypy/objspace/flow/specialcase.py (original) > +++ pypy/dist/pypy/objspace/flow/specialcase.py Wed Nov 7 05:18:57 2007 > @@ -6,7 +6,18 @@ > from pypy.tool.cache import Cache > > def sc_import(space, fn, args): > - w_name, w_glob, w_loc, w_frm = args.fixedunpack(4) > + args_w, kwds_w = args.unpack() > + assert kwds_w == {}, "should not call %r with keyword arguments" % (fn,) > + assert len(args_w) > 0 and len(args_w) <= 4, 'import needs 1 to 4 arguments' > + w_name = args_w[0] > + w_None = space.wrap(None) > + w_glob, w_loc, w_frm = w_None, w_None, w_None > + if len(args_w) > 1: > + w_glob = args_w[1] > + if len(args_w) > 2: > + w_loc = args_w[2] > + if len(args_w) > 3: > + w_frm = args_w[3] > if not isinstance(w_loc, Constant): > # import * in a function gives us the locals as Variable > # we always forbid it as a SyntaxError Hi Christian, could you please write a couple of tests for this? It's very hard to see what this code is doing and whether it is broken without tests for its functionality. Cheers, Carl Friedrich From tismer at stackless.com Wed Nov 7 14:13:07 2007 From: tismer at stackless.com (Christian Tismer) Date: Wed, 07 Nov 2007 14:13:07 +0100 Subject: [pypy-dev] gbg sprint In-Reply-To: <200711061200.lA6C0S0K023498@theraft.openend.se> References: <200711061200.lA6C0S0K023498@theraft.openend.se> Message-ID: <4731B9E3.7070608@stackless.com> Laura Creighton wrote: > If you go to http://www.gso.se/ > change the language to english, if you prefer, > and then look at the calendar you can see what is playing at the > concerthouse. > > jacob and I already have tickets for the piano concert which > should be fantastic on the 20th and then have tickets for the 22nd > for the Ravel concert. Anybody want me to get some more tickets for > them? Yes, please, both! cio - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Wed Nov 7 14:20:41 2007 From: tismer at stackless.com (Christian Tismer) Date: Wed, 07 Nov 2007 14:20:41 +0100 Subject: [pypy-dev] [pypy-svn] r48350 - pypy/dist/pypy/objspace/flow In-Reply-To: <47317F34.8090000@gmx.de> References: <20071107041858.EE8DC81AB@code0.codespeak.net> <47317F34.8090000@gmx.de> Message-ID: <4731BBA9.40708@stackless.com> > could you please write a couple of tests for this? It's very hard to see > what this code is doing and whether it is broken without tests for its > functionality. It is just allowing for less than 4 arguments in __import__. Maybe there is a more elegant/bvious way? The existing code always uses the 4 argument version, and it works, no import would work if that goes wrong. I have one use case (not yet checked in) that works with this change. So I see no need to add more direct tests. Nothing would work if this is wrong. Just a corner case where the annotator is too restrictive. (of course I could have used 4 arguments, but it's not nice) cheers - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From fijal at genesilico.pl Thu Nov 8 16:04:46 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Thu, 08 Nov 2007 16:04:46 +0100 Subject: [pypy-dev] Traceback surgery Message-ID: <4733258E.4080602@genesilico.pl> I've got an issue about distribution lib. I would like to do a bit more surgery of presented traceback, which needs cutting and sticking it back together. On top of cpython it's impossible, because this stuff is read only (you cannot modify tb.tb_next for example). Do we have a good reason why readonlyness remain in pypy? Of course I can cheat and use transparent proxy for that, but as long as this code would never work on top of cpython, can we lift those restrictions? Cheers, fijal :. From arigo at tunes.org Thu Nov 8 17:15:11 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 8 Nov 2007 17:15:11 +0100 Subject: [pypy-dev] gbg sprint In-Reply-To: <47306DD6.1030709@gmx.de> References: <200711061200.lA6C0S0K023498@theraft.openend.se> <20071106132901.GA21832@code0.codespeak.net> <200711061331.lA6DVZ57006999@theraft.openend.se> <47306DD6.1030709@gmx.de> Message-ID: <20071108161511.GA16849@code0.codespeak.net> Hi Carl, On Tue, Nov 06, 2007 at 02:36:22PM +0100, Carl Friedrich Bolz wrote: > Whew, is it that bad _already_? I bet your knowledge of jet-lag is theoretical :-) In practice it means that you're completely tired at the beginning of the trip, not at the end. We're all getting adjusted, slowly, so in a few days we'll be fine. And then we'll fly back home and troubles start again :-/ A bientot, Armin From tismer at stackless.com Thu Nov 8 17:26:52 2007 From: tismer at stackless.com (Christian Tismer) Date: Thu, 08 Nov 2007 17:26:52 +0100 Subject: [pypy-dev] gbg sprint In-Reply-To: <20071108161511.GA16849@code0.codespeak.net> References: <200711061200.lA6C0S0K023498@theraft.openend.se> <20071106132901.GA21832@code0.codespeak.net> <200711061331.lA6DVZ57006999@theraft.openend.se> <47306DD6.1030709@gmx.de> <20071108161511.GA16849@code0.codespeak.net> Message-ID: <473338CC.4090103@stackless.com> Armin Rigo wrote: > Hi Carl, > > On Tue, Nov 06, 2007 at 02:36:22PM +0100, Carl Friedrich Bolz wrote: >> Whew, is it that bad _already_? > > I bet your knowledge of jet-lag is theoretical :-) In practice it means > that you're completely tired at the beginning of the trip, not at the > end. We're all getting adjusted, slowly, so in a few days we'll be > fine. And then we'll fly back home and troubles start again :-/ Yes, and usually, at least for me much worse in the other direction, so you're toasted during the first sprinting days. good luck and fun, hopefully -- chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Thu Nov 8 17:33:01 2007 From: tismer at stackless.com (Christian Tismer) Date: Thu, 08 Nov 2007 17:33:01 +0100 Subject: [pypy-dev] Tests Are Good For You (was: r48350 - pypy/dist/pypy/objspace/flow) In-Reply-To: <4731BBA9.40708@stackless.com> References: <20071107041858.EE8DC81AB@code0.codespeak.net> <47317F34.8090000@gmx.de> <4731BBA9.40708@stackless.com> Message-ID: <47333A3D.6030307@stackless.com> Christian Tismer wrote: >> could you please write a couple of tests for this? It's very hard to see >> what this code is doing and whether it is broken without tests for its >> functionality. > > It is just allowing for less than 4 arguments in __import__. > Maybe there is a more elegant/bvious way? > > The existing code always uses the 4 argument version, and it > works, no import would work if that goes wrong. > > I have one use case (not yet checked in) that works with this > change. So I see no need to add more direct tests. Nothing > would work if this is wrong. > Just a corner case where the annotator is too restrictive. > (of course I could have used 4 arguments, but it's not nice) Who wrote that? Me? Of course there is a test now. Especially since my use cases touches Windows only, I would easily shoot myself in the foot. So yes, in any case, write tests. tests-make-your-windows-support-survive - ly y'rs -- chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From arigo at tunes.org Thu Nov 8 17:36:15 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 8 Nov 2007 17:36:15 +0100 Subject: [pypy-dev] [pypy-svn] r48350 - pypy/dist/pypy/objspace/flow In-Reply-To: <4731BBA9.40708@stackless.com> References: <20071107041858.EE8DC81AB@code0.codespeak.net> <47317F34.8090000@gmx.de> <4731BBA9.40708@stackless.com> Message-ID: <20071108163615.GA17125@code0.codespeak.net> Hi Christian, On Wed, Nov 07, 2007 at 02:20:41PM +0100, Christian Tismer wrote: > > could you please write a couple of tests for this? It's very hard to see > > what this code is doing and whether it is broken without tests for its > > functionality. > > It is just allowing for less than 4 arguments in __import__. > Maybe there is a more elegant/bvious way? The point is that if it's untested, the next time someone needs to refactor something around this file, your un-checked-in experiments are going to break, and everybody is going to feel unhappy about it. So, can you invest a little time now to avoid long discussions later? Thanks! A bientot, Armin. From tismer at stackless.com Thu Nov 8 20:11:40 2007 From: tismer at stackless.com (Christian Tismer) Date: Thu, 08 Nov 2007 20:11:40 +0100 Subject: [pypy-dev] [pypy-svn] r48350 - pypy/dist/pypy/objspace/flow In-Reply-To: <20071108163615.GA17125@code0.codespeak.net> References: <20071107041858.EE8DC81AB@code0.codespeak.net> <47317F34.8090000@gmx.de> <4731BBA9.40708@stackless.com> <20071108163615.GA17125@code0.codespeak.net> Message-ID: <47335F6C.7050902@stackless.com> Armin Rigo wrote: > Hi Christian, > > On Wed, Nov 07, 2007 at 02:20:41PM +0100, Christian Tismer wrote: >>> could you please write a couple of tests for this? It's very hard to see >>> what this code is doing and whether it is broken without tests for its >>> functionality. >> It is just allowing for less than 4 arguments in __import__. >> Maybe there is a more elegant/bvious way? > > The point is that if it's untested, the next time someone needs to > refactor something around this file, your un-checked-in experiments are > going to break, and everybody is going to feel unhappy about it. So, > can you invest a little time now to avoid long discussions later? > Thanks! understood and done yesterday, alreasy ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From anto.cuni at gmail.com Thu Nov 8 20:25:25 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Thu, 08 Nov 2007 20:25:25 +0100 Subject: [pypy-dev] [pypy-svn] r48350 - pypy/dist/pypy/objspace/flow In-Reply-To: <47335F6C.7050902@stackless.com> References: <20071107041858.EE8DC81AB@code0.codespeak.net> <47317F34.8090000@gmx.de> <4731BBA9.40708@stackless.com> <20071108163615.GA17125@code0.codespeak.net> <47335F6C.7050902@stackless.com> Message-ID: <473362A5.6010302@gmail.com> Christian Tismer wrote: > understood and done yesterday, alreasy ^ nice typo: I bet I hasn't been hard :-) From faassen at startifact.com Sun Nov 11 09:14:54 2007 From: faassen at startifact.com (Martijn Faassen) Date: Sun, 11 Nov 2007 09:14:54 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog Message-ID: Hi there, http://www.tbray.org/ongoing/When/200x/2007/11/08/PyPy It might be worth it for someone to comment on it, as the one comment there is rather negative about PyPy (it'd really help to have PyPy interpreters in production somewhere, as before you do, the criticism in the comment cannot be properly countered) I'm not sure how to interpret Tim Bray's post. Frankly, I was a bit disappointed by what made it in there. It's interesting to analyze as it gives some clue about how your pitch was received and understood. It's too bad the potential of a compliant, maintainable Python implementation on the JVM doesn't seem to make it to his blog. I assume you also pitched actual work on *Python* on, say, the JVM, to him as a possible candidate for funding. As opposed to any arbitrary language, which basically lets Tim change the topic to a Ruby discussion. It's also too bad that the whole idea of instrumenting the translation process to do neat stuff doesn't really make it to his blog post. The tainting stuff is mentioned, but that's it. What about garbage collectors and JITs? Regards, Martijn From jacob at openend.se Sun Nov 11 18:16:01 2007 From: jacob at openend.se (Jacob =?iso-8859-1?q?Hall=E9n?=) Date: Sun, 11 Nov 2007 18:16:01 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: References: Message-ID: <200711111816.01317.jacob@openend.se> Tim Bray has a very strong Ruby focus. Our presentation gave a broad outline of the capabilities of PyPy, including pluggable GCs and had a fairly strong focus on JIT aspects. Tim had a good understanding of the JIT aspects. Why they didn't make it to his blog - who knows. Jacob Den Sunday 11 November 2007 09.14.54 skrev Martijn Faassen: > Hi there, > > http://www.tbray.org/ongoing/When/200x/2007/11/08/PyPy > > It might be worth it for someone to comment on it, as the one comment > there is rather negative about PyPy (it'd really help to have PyPy > interpreters in production somewhere, as before you do, the criticism in > the comment cannot be properly countered) > > I'm not sure how to interpret Tim Bray's post. Frankly, I was a bit > disappointed by what made it in there. It's interesting to analyze as it > gives some clue about how your pitch was received and understood. > > It's too bad the potential of a compliant, maintainable Python > implementation on the JVM doesn't seem to make it to his blog. I assume > you also pitched actual work on *Python* on, say, the JVM, to him as a > possible candidate for funding. As opposed to any arbitrary language, > which basically lets Tim change the topic to a Ruby discussion. > > It's also too bad that the whole idea of instrumenting the translation > process to do neat stuff doesn't really make it to his blog post. The > tainting stuff is mentioned, but that's it. What about garbage > collectors and JITs? > > Regards, > > Martijn > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From lac at openend.se Sun Nov 11 15:39:18 2007 From: lac at openend.se (Laura Creighton) Date: Sun, 11 Nov 2007 15:39:18 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: Message from Martijn Faassen of "Sun, 11 Nov 2007 09:14:54 +0100." References: Message-ID: <200711111439.lABEdIZ5011340@theraft.openend.se> In a message of Sun, 11 Nov 2007 09:14:54 +0100, Martijn Faassen writes: >Hi there, > >http://www.tbray.org/ongoing/When/200x/2007/11/08/PyPy > >It might be worth it for someone to comment on it, as the one comment >there is rather negative about PyPy (it'd really help to have PyPy >interpreters in production somewhere, as before you do, the criticism in >the comment cannot be properly countered) Somebody not in the project can comment on it if they like. I'm not in the habit of responding to troll-bait. If that person sincerely thinks that we haven't produced much in 2 years then he either is completely unaware of what we have done, or he cannot understand it. I'm fine with combatting ignorance, but not the with people in the 'my-sports-teams-is-better-than-your-sports-team' mode. > >I'm not sure how to interpret Tim Bray's post. Frankly, I was a bit >disappointed by what made it in there. It's interesting to analyze as it >gives some clue about how your pitch was received and understood. Tim can understand all the implications of new technology and we really impressed him. His problem was in believing that we had done what we had said, and now that we can do what we promised. And he is not techie -- in the sense of being a language designer who has read all the neat scientific papers of language design -- he lives close to the programming-not-deeply-informed-by-computer science world. > >It's too bad the potential of a compliant, maintainable Python >implementation on the JVM doesn't seem to make it to his blog. I assume >you also pitched actual work on *Python* on, say, the JVM, to him as a >possible candidate for funding. As opposed to any arbitrary language, >which basically lets Tim change the topic to a Ruby discussion. Tim is very Ruby focused. He's not interested in python, he said, because 'the ruby community is more vibrant'. Vibrant is measured in body-count. He's funding all the reimplimentations of Ruby, and know s the people, and likes the people. So he has a certain amount of loyalty to them, too. He really wants to make Rails faster. He says Django is "as good as rails" or something to that effect, but that doesn't change the fact that what he wants to do is make rails faster. >It's also too bad that the whole idea of instrumenting the translation >process to do neat stuff doesn't really make it to his blog post. The >tainting stuff is mentioned, but that's it. What about garbage >collectors and JITs? He saw all of these. When I asked about funding for tools to make it easier to, say, find out if Erlang-type concurrency was a good idea, he said he would think about it. His wide-finder experiment only showed that the Erlang string-processing libraries aren't as good as Python's, not if Erlang's concurrency model would be a good things to stick into languages. He liked the idea of experimentation, very much, but we had just blown him away with the architecture that he didn't want to think about that then. Plus he got an emergency trip to Japan put in his plate at short notice. Think of this as very positive. We go spend a whole day at Sun on Monday, focused on the JVM. John Rose, who is organising this, and who knows Samuele from a currently running Java expert committee, has invited some people, including some JRuby people to attend. John Rose is already _really excited about this_. What's the worst that can happen? We get to support our Python habit by building an interpreter for Ruby. Somebody was going to come along and do it sooner or later, i.e. as soon as PyPy is making everything go at speed, not just integer math. i.e. as soon as we stop spending 50% of our time in the garbage collector! There is nothing but good news here. Be happy for us. >Regards, > >Martijn > >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev From faassen at startifact.com Mon Nov 12 19:53:33 2007 From: faassen at startifact.com (Martijn Faassen) Date: Mon, 12 Nov 2007 19:53:33 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <200711111439.lABEdIZ5011340@theraft.openend.se> References: <200711111439.lABEdIZ5011340@theraft.openend.se> Message-ID: Hey, Laura Creighton wrote: > In a message of Sun, 11 Nov 2007 09:14:54 +0100, Martijn Faassen writes: >> Hi there, >> >> http://www.tbray.org/ongoing/When/200x/2007/11/08/PyPy >> >> It might be worth it for someone to comment on it, as the one comment >> there is rather negative about PyPy (it'd really help to have PyPy >> interpreters in production somewhere, as before you do, the criticism in >> the comment cannot be properly countered) > > Somebody not in the project can comment on it if they like. I'm > not in the habit of responding to troll-bait. If that person > sincerely thinks that we haven't produced much in 2 years then > he either is completely unaware of what we have done, or he > cannot understand it. > I'm fine with combatting ignorance, but not the with people in the > 'my-sports-teams-is-better-than-your-sports-team' mode. It's a negative message in a widely read blog. Even if it's offensive and wrong, it's also bad marketing for the project. By now any (minor) damage it did is done though, I guess, as it'll disappear off people's radar again. I of course don't agree that you didn't accomplish anything, but of course on the other hand I continue on my mission to poke you guys gently about seeing PyPy into production (my own work). Once PyPy is used in production there's an easy way to blow any such criticism out of the water. If it never happens, you've accomplished research goals, which are worthwhile by itself, but I'd prefer to see something more pragmatic myself in addition. :) >> I'm not sure how to interpret Tim Bray's post. Frankly, I was a bit >> disappointed by what made it in there. It's interesting to analyze as it >> gives some clue about how your pitch was received and understood. > > Tim can understand all the implications of new technology and we > really impressed him. His problem was in believing that we had done > what we had said, and now that we can do what we promised. And he > is not techie -- in the sense of being a language designer who > has read all the neat scientific papers of language design -- he > lives close to the programming-not-deeply-informed-by-computer science > world. I also live close to that world. It's an audience you want to reach, I think, so it was interesting to read what came out of your meeting from that side. I guess he'll believe you once you have a compliant JITed PyPy interpreter running on the JVM in production, then. :) >> It's too bad the potential of a compliant, maintainable Python >> implementation on the JVM doesn't seem to make it to his blog. I assume >> you also pitched actual work on *Python* on, say, the JVM, to him as a >> possible candidate for funding. As opposed to any arbitrary language, >> which basically lets Tim change the topic to a Ruby discussion. > > Tim is very Ruby focused. He's not interested in python, he said, > because 'the ruby community is more vibrant'. Vibrant is measured > in body-count. body-count? As in how many people are part of it (I imagine Python would do pretty well compared to Python) or how many people die in horrible flame wars? :) > He's funding all the reimplimentations of Ruby, > and know s the people, and likes the people. So he has a certain > amount of loyalty to them, too. That makes sense. > He really wants to make Rails faster. He says Django is "as good as > rails" or something to that effect, but that doesn't change the fact > that what he wants to do is make rails faster. Heh, as someone from the minor Grok camp I have my own opinions about what is as good as what. :) [snip] > Think of this as very positive. It's glad to hear from your side, thanks! I didn't mean to criticize you at all, I was just curious to explain the blog entry which presented PyPy in a somewhat unusual light. > We go spend a whole day at Sun on Monday, focused on the JVM. > John Rose, who is organising this, and who knows Samuele from > a currently running Java expert committee, has invited some > people, including some JRuby people to attend. John Rose is > already _really excited about this_. > > What's the worst that can happen? We get to support our Python habit > by building an interpreter for Ruby. The worst thing that can happen is that you build no interpreters in any language that can be used in production. :) If you start building a Ruby interpreter, the worst thing that can happen is that the Python interpreter never gets completed. > Somebody was going to come along > and do it sooner or later, i.e. as soon as PyPy is making everything > go at speed, not just integer math. i.e. as soon as we stop spending > 50% of our time in the garbage collector! I'm not saying you shouldn't build a Ruby interpreter. I'd debate against building one without funding, unless this doesn't distract a lot. I'd also debate against shifting the major focus on building Ruby in PyPy before Python's done. I'm not seriously thinking you are proposing this, but you started talking about worst-case scenarios. :) Anyway, I think you're also saying this, as a Python interpreter is your main technology demonstrator and test-bed. > There is nothing but good news here. Be happy for us. All right, I'm happy for you. :) Regards, Martijn From anto.cuni at gmail.com Tue Nov 13 00:02:51 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 13 Nov 2007 00:02:51 +0100 Subject: [pypy-dev] __file__ attribute of mixed modules Message-ID: <4738DB9B.6070101@gmail.com> Hi all, I'm having troubles running pypy-cli on windows. One of the problem is due to the __file__ attribute of mixed modules; currently it's something like path/to/mixed/module/*.py. When site.py runs, it tries to compute the abspath of every module loaded; after a bit of indirections, abspath call the posix__getfullpath helper written in C#. The most obvious (and probably most correct) way to implement posix__getfullpath is to delegate to System.IO.Path.GetFullPath; here is where problems come, since the CLR implementation of GetFullPath complains if we pass it a name with an asterisk (Mono doesn't). This prevents pypy-cli to start. One possible solution would be to place a check inside the C# helper and not call GetFullPath in case there is an asterisk in the name (or maybe remove the asterisk, call GetFullPath and re-insert the asterisk). I think this solution is ugly and hackish. Another solution is to change the way we assign __file__ to mixed modules; e.g., we could use the name of the directory itself, or __init__.py instead of *.py. Or maybe some weird name like 'i_dont_really_exist.py', etc. What do you think? ciao Anto From faassen at startifact.com Tue Nov 13 02:43:55 2007 From: faassen at startifact.com (Martijn Faassen) Date: Tue, 13 Nov 2007 02:43:55 +0100 Subject: [pypy-dev] unicode error when building the JVM version of PyPy Message-ID: Hi there, I just tried to build the JVM version of the trunk with the following command (which may be altogether wrong): python2.4 translate.py --text --batch --backend=jvm targetpypystandalone.py A while into the translation, I got the following error, which I'm dumping here in case it might be useful to someone: [jvm:graphs] Rendered 7600/7648 (approx. 99.37%) [Timer] Timings: [Timer] annotate - 384.4 s [Timer] rtype_ootype - 112.8 s [Timer] backendopt_ootype - 85.2 s [Timer] source_jvm - 111.1 s [Timer] ======================================== [Timer] Total: - 693.5 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 273, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/driver.py", line 772, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/tool/taskengine.py", line 112, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/driver.py", line 281, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/driver.py", line 739, in task_source_jvm [translation:ERROR] self.jvmsource = self.gen.generate_source() [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/jvm/genjvm.py", line 260, in generate_source [translation:ERROR] GenOO.generate_source(self) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/oosupport/genoo.py", line 68, in generate_source [translation:ERROR] self.db.gen_constants(self.ilasm) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/oosupport/database.py", line 35, in gen_constants [translation:ERROR] self.constant_generator.gen_constants(ilasm) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/jvm/constant.py", line 108, in gen_constants [translation:ERROR] super(JVMConstantGenerator, self).gen_constants(ilasm) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/oosupport/constant.py", line 245, in gen_constants [translation:ERROR] self._initialize_data(gen, all_constants) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/oosupport/constant.py", line 268, in _initialize_data [translation:ERROR] if not const.initialize_data(self, gen): [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/oosupport/constant.py", line 542, in initialize_data [translation:ERROR] push_constant(self.db, FIELD_TYPE, value, gen) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/oosupport/constant.py", line 55, in push_constant [translation:ERROR] return constgen.push_primitive_constant(gen, TYPE, value) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/oosupport/constant.py", line 122, in push_primitive_constant [translation:ERROR] gen.push_primitive_constant(TYPE, value) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/jvm/generator.py", line 1103, in push_primitive_constant [translation:ERROR] self.load_string(str(value._str)) [translation:ERROR] UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in position 0: ordinal not in range(128) Might have something to do with the recent unicode work? Regards, Martijn From tismer at stackless.com Tue Nov 13 09:19:13 2007 From: tismer at stackless.com (Christian Tismer) Date: Tue, 13 Nov 2007 09:19:13 +0100 Subject: [pypy-dev] __file__ attribute of mixed modules In-Reply-To: <4738DB9B.6070101@gmail.com> References: <4738DB9B.6070101@gmail.com> Message-ID: <47395E01.7000004@stackless.com> Antonio Cuni wrote: > Hi all, > I'm having troubles running pypy-cli on windows. > > One of the problem is due to the __file__ attribute of mixed modules; > currently it's something like path/to/mixed/module/*.py. > > When site.py runs, it tries to compute the abspath of every module > loaded; after a bit of indirections, abspath call the posix__getfullpath > helper written in C#. > > The most obvious (and probably most correct) way to implement > posix__getfullpath is to delegate to System.IO.Path.GetFullPath; here is > where problems come, since the CLR implementation of GetFullPath > complains if we pass it a name with an asterisk (Mono doesn't). This > prevents pypy-cli to start. > > One possible solution would be to place a check inside the C# helper and > not call GetFullPath in case there is an asterisk in the name (or maybe > remove the asterisk, call GetFullPath and re-insert the asterisk). I > think this solution is ugly and hackish. > > Another solution is to change the way we assign __file__ to mixed > modules; e.g., we could use the name of the directory itself, or > __init__.py instead of *.py. Or maybe some weird name like > 'i_dont_really_exist.py', etc. I think we have to choose some different notation than "*". It is probably meant as a wild card, but I'm not astonished that this doesn't work well with all path functions. We should stick with correct filename syntax for all platforms. In any case, I would recommend to stick with GetFullPath and not implement them alone. The rules are that weird and half-defined that it is fine to let MS worry about that. The __init__.py naming is not that bad. There are other ways possible to distinguish mixed modules from others. ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From anto.cuni at gmail.com Tue Nov 13 11:23:53 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 13 Nov 2007 11:23:53 +0100 Subject: [pypy-dev] unicode error when building the JVM version of PyPy In-Reply-To: References: Message-ID: <47397B39.80304@gmail.com> Martijn Faassen wrote: > Hi there, > > I just tried to build the JVM version of the trunk with the following > command (which may be altogether wrong): > > python2.4 translate.py --text --batch --backend=jvm > targetpypystandalone.py > > A while into the translation, I got the following error, which I'm > dumping here in case it might be useful to someone: [cut] > Might have something to do with the recent unicode work? Hi Martijn, thank you for the signaling. Indeed, the bug was caused by genjvm not handling properly unicode constants. It has been fixed in revision 48638. ciao Anto From lac at openend.se Tue Nov 13 16:23:50 2007 From: lac at openend.se (Laura Creighton) Date: Tue, 13 Nov 2007 16:23:50 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: Message from Martijn Faassen of "Mon, 12 Nov 2007 19:53:33 +0100." References: <200711111439.lABEdIZ5011340@theraft.openend.se> Message-ID: <200711131523.lADFNoWO025693@theraft.openend.se> In a message of Mon, 12 Nov 2007 19:53:33 +0100, Martijn Faassen writes: >Hey, > >Laura Creighton wrote: >> In a message of Sun, 11 Nov 2007 09:14:54 +0100, Martijn Faassen writes >: >>> Hi there, >>> >>> http://www.tbray.org/ongoing/When/200x/2007/11/08/PyPy >>> >>> It might be worth it for someone to comment on it, as the one comment >>> there is rather negative about PyPy (it'd really help to have PyPy >>> interpreters in production somewhere, as before you do, the criticism >in >>> the comment cannot be properly countered) >> >> Somebody not in the project can comment on it if they like. I'm >> not in the habit of responding to troll-bait. If that person >> sincerely thinks that we haven't produced much in 2 years then >> he either is completely unaware of what we have done, or he >> cannot understand it. > > > I'm fine with combatting ignorance, but not the with people in the > > 'my-sports-teams-is-better-than-your-sports-team' mode. > >It's a negative message in a widely read blog. Even if it's offensive >and wrong, it's also bad marketing for the project. By now any (minor) >damage it did is done though, I guess, as it'll disappear off people's >radar again. > >I of course don't agree that you didn't accomplish anything, but of >course on the other hand I continue on my mission to poke you guys >gently about seeing PyPy into production (my own work). > >Once PyPy is used in production there's an easy way to blow any such >criticism out of the water. If it never happens, you've accomplished >research goals, which are worthwhile by itself, but I'd prefer to see >something more pragmatic myself in addition. :) Us, too. But Samuele, Armin, Jacob and myself are now on a 2 week US road show speaking with major players in US companies every day of the week. We don't have time for baby-trolls even if we had the inclination. >I also live close to that world. It's an audience you want to reach, I >think, so it was interesting to read what came out of your meeting from >that side. > >I guess he'll believe you once you have a compliant JITed PyPy >interpreter running on the JVM in production, then. :) Nothing that dramatic. Just a Ruby that runs at the speed of CPython. Though, goodness knows, perhaps John Rose and Charles Nutter will tell him that we are for real, and that will be enough. It's hard to know the difference between a genuine revolution in the industry, and a lot of hot air. But the odds are on anything radical being the second. After all, if it is so hot, why hasn't somebody else already done it? There is no answer to that. The hard things that change the world were often done by people who weren't expected to amount to anything, and just kept on truckin' anyway. Sooner, rather than later, Tim Bray will have his answer, and nobody will blame him for being skeptical about our ability to deliver on what we promise, because we promise a lot, and something that has never been done before. Sometimes the caterpillar wants to become a larger, healthier caterpillar. Sometimes it's time to become a butterfly. >>> It's too bad the potential of a compliant, maintainable Python >>> implementation on the JVM doesn't seem to make it to his blog. I assum >e >>> you also pitched actual work on *Python* on, say, the JVM, to him as a > >>> possible candidate for funding. As opposed to any arbitrary language, >>> which basically lets Tim change the topic to a Ruby discussion. >> >> Tim is very Ruby focused. He's not interested in python, he said, >> because 'the ruby community is more vibrant'. Vibrant is measured >> in body-count. > >body-count? As in how many people are part of it (I imagine Python would >do pretty well compared to Python) or how many people die in horrible >flame wars? :) I think 'how many people attend Ruby developers conferences' and 'how many people use rails' but I am not 100% convinced of that. >It's glad to hear from your side, thanks! I didn't mean to criticize you >at all, I was just curious to explain the blog entry which presented >PyPy in a somewhat unusual light. > >> We go spend a whole day at Sun on Monday, focused on the JVM. >> John Rose, who is organising this, and who knows Samuele from >> a currently running Java expert committee, has invited some >> people, including some JRuby people to attend. John Rose is >> already _really excited about this_. And we left with him very impressed. Wew had a blast. I think he would liek to attend a Sprint. And he would like it if we made our next pypy focus -- getting the jit to work with the JVM. Wouldn't that be fun. >> There is nothing but good news here. Be happy for us. > >All right, I'm happy for you. :) Thank you, and we will keep your concerns under consideration. But please remember that the PyPy dev team needs to eat, pay rent, and the like. While developing a Ruby front end may not be the most fun thing we could do, it may be plenty more fun than the other things we would end up doing instead just to make the financial ends meet. Laura, happier than ever today. > >Regards, > >Martijn > >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev From faassen at startifact.com Tue Nov 13 19:42:03 2007 From: faassen at startifact.com (Martijn Faassen) Date: Tue, 13 Nov 2007 19:42:03 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <200711131523.lADFNoWO025693@theraft.openend.se> References: <200711111439.lABEdIZ5011340@theraft.openend.se> <200711131523.lADFNoWO025693@theraft.openend.se> Message-ID: Laura Creighton wrote: > In a message of Mon, 12 Nov 2007 19:53:33 +0100, Martijn Faassen writes: [snip] >>> Tim is very Ruby focused. He's not interested in python, he said, >>> because 'the ruby community is more vibrant'. Vibrant is measured >>> in body-count. >> body-count? As in how many people are part of it (I imagine Python would >> do pretty well compared to Python) or how many people die in horrible >> flame wars? :) > > I think 'how many people attend Ruby developers conferences' and > 'how many people use rails' but I am not 100% convinced of that. I'm convinced the body count of Python users is probably still quite a bit larger than Ruby users. I also have a strong suspicion far more people are using Python web frameworks than there are people using Rails, hype aside, but I may be wrong about that. [snip John Rose] > And we left with him very impressed. Wew had a blast. I think he > would liek to attend a Sprint. And he would like it if we made > our next pypy focus -- getting the jit to work with the JVM. > > Wouldn't that be fun. Yes, that'd be of great practical importance, too. :) I have this vague dream of making Zope 3 (and Grok) run on the JVM using PyPy. Being able to use the Java libraries would be nice. >>> There is nothing but good news here. Be happy for us. >> All right, I'm happy for you. :) > > Thank you, and we will keep your concerns under consideration. > But please remember that the PyPy dev team needs to eat, pay > rent, and the like. While developing a Ruby front end may > not be the most fun thing we could do, it may be plenty more > fun than the other things we would end up doing instead just > to make the financial ends meet. As someone who needs to eat and pay rent and the like I think I know what you're talking about. I think making it practical and useful to run a PyPy-based Python interpreter in production projects is probably the best shot this project has at becoming more self-sustaining. You're close. So I'll definitely complain if you spend a lot of time on Ruby (or Smalltalk for that matter) before Python's all the way there. I think that'd be a bad idea for a whole range of reasons - get the last 10% done for Python first before you take even more on your plates. We all know that last 10% tends to be the hardest part. Focus is important. If you can't make that work for Python, you'd have a hard time making it work for any other language too (or convincing people that you can). Of course I realize I have no real voice in this project, but that's my input. Regards, Martijn From jacob at openend.se Wed Nov 14 03:53:18 2007 From: jacob at openend.se (Jacob =?iso-8859-1?q?Hall=E9n?=) Date: Wed, 14 Nov 2007 03:53:18 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: References: <200711131523.lADFNoWO025693@theraft.openend.se> Message-ID: <200711140353.18810.jacob@openend.se> Den Tuesday 13 November 2007 19.42.03 skrev Martijn Faassen: > I'm convinced the body count of Python users is probably still quite a > bit larger than Ruby users. I also have a strong suspicion far more > people are using Python web frameworks than there are people using > Rails, hype aside, but I may be wrong about that. It doesn't really matter how many users you have, when someone measures vibrant by how many excited people he meets at a conference. There were 600 at a recent Rails conference, while Pycon gathers 400. This is the way Tim Bray sees things and it is not our mission to tell him that he is wrong. It is our mission to get him to pay for PyPy development. He is the customer, and the customer is always right, ;-) > I think making it practical and useful to run a PyPy-based Python > interpreter in production projects is probably the best shot this > project has at becoming more self-sustaining. You're close. > > So I'll definitely complain if you spend a lot of time on Ruby (or > Smalltalk for that matter) before Python's all the way there. I think > that'd be a bad idea for a whole range of reasons - get the last 10% > done for Python first before you take even more on your plates. We all > know that last 10% tends to be the hardest part. Focus is important. If > you can't make that work for Python, you'd have a hard time making it > work for any other language too (or convincing people that you can). Of > course I realize I have no real voice in this project, but that's my input. The reason we are not there yet for Python is financial, not due to a lack of focus. (And what yo may have seen as a lack of focus during the funded period was the totally opposite. We focused on delivering what we had promised the EU, not on what would be the fastest path to be a viable alternative to CPython.) If we get funded to do Ruby, this will benefit Python, but the focus will be on delivering what the one who pays wants out of the project, not on making a better Python. Jacob From tismer at stackless.com Wed Nov 14 04:47:39 2007 From: tismer at stackless.com (Christian Tismer) Date: Wed, 14 Nov 2007 04:47:39 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: References: <200711111439.lABEdIZ5011340@theraft.openend.se> <200711131523.lADFNoWO025693@theraft.openend.se> Message-ID: <473A6FDB.1060301@stackless.com> Martijn Faassen wrote: > Laura Creighton wrote: >> In a message of Mon, 12 Nov 2007 19:53:33 +0100, Martijn Faassen writes: > [snip] >>>> Tim is very Ruby focused. He's not interested in python, he said, >>>> because 'the ruby community is more vibrant'. Vibrant is measured >>>> in body-count. >>> body-count? As in how many people are part of it (I imagine Python would >>> do pretty well compared to Python) or how many people die in horrible >>> flame wars? :) >> I think 'how many people attend Ruby developers conferences' and >> 'how many people use rails' but I am not 100% convinced of that. > > I'm convinced the body count of Python users is probably still quite a > bit larger than Ruby users. I also have a strong suspicion far more > people are using Python web frameworks than there are people using > Rails, hype aside, but I may be wrong about that. Just to put my 2 cent in here: I am back to an old idea from the pre-pypy area. A small team was trying to create something like a scripting kernel. Ruby, Tcl, Python and even Perl, based upon a common set of structures and algorithms. This was Gordon Mc. Millan, Jean-Claude Wippler, Matt Newman. A way too small group with way too large intentions. But in the end, having a group like PyPy, extending across a set of languages, might be the realization of an old wish. I think this could work for pypy: Not trying to be the one-and-only language right now and comparing with other languages. I would rather like to avoid this discussion at all and to get PyPy out of such discussions. If we are going for some ruby support, this will enhance the ability and generality of PyPy. This holds for other languages as well. We have choosen to implement everything using RPython. I do think this was a good decision. Besides that, PyPy could avoid to be an advocate for any language. There is enough advocacy for this and other languages. PyPy is more than one language, and it is not a decision. It is a way to improve lots of languages. PyPy, changing its target as not (yet) trying to be the one and only better Python, but to be the first really capable collection of building blocks for interpreted languages could be much easier to argue for, because that part of PyPy does not have any competitors. Reducing that to one language is not a benefit at all. Making PyPy more general by becoming less Python focused might change the focus of many, realizing that there is a new concept around, of building interpreters with a Jit, able to tailor its outcome to many special applications. Removing the language constraint seems to be a natural atep in this evolution, broadening PyPy's scope, and getting rid of silly language discussions. In the end, we can reduce the problem by extending to "any interesting language" which can widen PyPy's scope substantially. My 2 cent, maybe becoming more -- chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From paul.degrandis at gmail.com Wed Nov 14 05:49:06 2007 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Tue, 13 Nov 2007 23:49:06 -0500 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <473A6FDB.1060301@stackless.com> References: <200711111439.lABEdIZ5011340@theraft.openend.se> <200711131523.lADFNoWO025693@theraft.openend.se> <473A6FDB.1060301@stackless.com> Message-ID: <9c0bb8a00711132049m36f31fddma0bb4aa7e247490@mail.gmail.com> I have stayed quiet thus far but I wanted to make sure everyone knew I was still around and thought I'd contribute to the discussion now. I have been asked to give the same PyPy talk three times in the Philadelphia area, and everytime I give a talk based on some of mwh's slides, much to the same tune that Chris was singing in his email. The fact alone that PyPy team members are ABLE to go to different companies with different interests and pitch the same product is AMAZING and shows the strength of of the project and product that everyone has helped form, shape, and create. Let people say what they want. Personally, I don't think Tim's blog was harsh, and anyone who has spent ANY time on the Internet can clearly see through the comment (and recognize it for what it is). I wish I knew about the trip sooner, I would have volunteered my girlfriend's apartment in NYC and helped set you up with some other appointments. Please let me know if anyone has a desire to pitch PyPy to anyone else in industry. I have some quality contacts. Paul On 11/13/07, Christian Tismer wrote: > Martijn Faassen wrote: > > Laura Creighton wrote: > >> In a message of Mon, 12 Nov 2007 19:53:33 +0100, Martijn Faassen writes: > > [snip] > >>>> Tim is very Ruby focused. He's not interested in python, he said, > >>>> because 'the ruby community is more vibrant'. Vibrant is measured > >>>> in body-count. > >>> body-count? As in how many people are part of it (I imagine Python would > >>> do pretty well compared to Python) or how many people die in horrible > >>> flame wars? :) > >> I think 'how many people attend Ruby developers conferences' and > >> 'how many people use rails' but I am not 100% convinced of that. > > > > I'm convinced the body count of Python users is probably still quite a > > bit larger than Ruby users. I also have a strong suspicion far more > > people are using Python web frameworks than there are people using > > Rails, hype aside, but I may be wrong about that. > > Just to put my 2 cent in here: > > I am back to an old idea from the pre-pypy area. > A small team was trying to create something like a scripting kernel. > Ruby, Tcl, Python and even Perl, based upon a common set of > structures and algorithms. > This was Gordon Mc. Millan, Jean-Claude Wippler, Matt Newman. > A way too small group with way too large intentions. > But in the end, having a group like PyPy, extending across > a set of languages, might be the realization of an old wish. > > I think this could work for pypy: > Not trying to be the one-and-only language right now and comparing > with other languages. I would rather like to avoid this discussion at > all and to get PyPy out of such discussions. > > If we are going for some ruby support, this will enhance the ability > and generality of PyPy. This holds for other languages as well. > > We have choosen to implement everything using RPython. I do think > this was a good decision. Besides that, PyPy could avoid to be > an advocate for any language. There is enough advocacy for this > and other languages. PyPy is more than one language, and it is > not a decision. It is a way to improve lots of languages. > > PyPy, changing its target as not (yet) trying to be the one and only > better Python, but to be the first really capable collection of > building blocks for interpreted languages could be much easier to > argue for, because that part of PyPy does not have any competitors. > Reducing that to one language is not a benefit at all. > > Making PyPy more general by becoming less Python focused might > change the focus of many, realizing that there is a new concept > around, of building interpreters with a Jit, able to tailor its > outcome to many special applications. > > Removing the language constraint seems to be a natural atep > in this evolution, broadening PyPy's scope, and getting rid of > silly language discussions. In the end, we can reduce the problem > by extending to "any interesting language" which can widen PyPy's > scope substantially. > > My 2 cent, maybe becoming more -- chris > -- > Christian Tismer :^) > tismerysoft GmbH : Have a break! Take a ride on Python's > Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ > 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ > work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 > PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 > whom do you want to sponsor today? http://www.stackless.com/ > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From lac at openend.se Wed Nov 14 13:24:41 2007 From: lac at openend.se (Laura Creighton) Date: Wed, 14 Nov 2007 13:24:41 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: Message from Jacob =?iso-8859-1?q?Hall=E9n?= of "Wed, 14 Nov 2007 03:53:18 +0100." <200711140353.18810.jacob@openend.se> References: <200711131523.lADFNoWO025693@theraft.openend.se> <200711140353.18810.jacob@openend.se> Message-ID: <200711141224.lAECOfar016106@theraft.openend.se> In a message of Wed, 14 Nov 2007 03:53:18 +0100, Jacob Hall?n writes: >> So I'll definitely complain if you spend a lot of time on Ruby (or >> Smalltalk for that matter) before Python's all the way there. I think >> that'd be a bad idea for a whole range of reasons - get the last 10% >> done for Python first before you take even more on your plates. We all >> know that last 10% tends to be the hardest part. Focus is important. If >> you can't make that work for Python, you'd have a hard time making it >> work for any other language too (or convincing people that you can). Of >> course I realize I have no real voice in this project, but that's my in >put. I don't think you understand the state of our project. There is practically nothing left that is 'python-centric' left to work on, unless you have a burning desire to have the 2.5 features we don't have yet, which is scheduled to get finished next week anyway. This is hardly any work. Where we need work is to make the thing run a lot faster. So, better gc, more work on the JIT -- and play better with c-extensions, and enable us to use the stackless transform on CLI and JVM, and the generated JIT on that as well. All this stuff is translation level stuff. Thus it is relevant to all the app-level input languages we support, not any particular one. Speed up one, you speed up them all. It took a 10 day sprint for some of the pypyers and the squeak experts at the SCG to get Smalltalk up and running. http://pypysqueak.blogspot.com/ Assuming we could find a small group of similarly competant ruby language experts, we ought to be able to do the same thing in the same sort of time range. If we cannot find them, and have to do it all ourselves, it will take longer, since we aren't experts on the arcane and obscure details about the ruby language that only ruby wizards know. And then we can take Tim Bray's money and give him the fast ruby he wants by working on precisely the same things that will give you the fast python you want. So I don't see why you aren't up in the front row cheering these developments as loudly as you can. Right now the most promising path to get you what you want, which I see as a fast python that is production-quality and suitable for deploying Grok web apps, runs hand in hand with Tim Bray's desire for a fast ruby that is production-quality and suitable to deploy rails apps. You guys want the same thing! You don't even have to squint! Why aren't you cheering? Laura From exarkun at divmod.com Wed Nov 14 13:43:59 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 14 Nov 2007 07:43:59 -0500 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <200711141224.lAECOfar016106@theraft.openend.se> Message-ID: <20071114124359.8162.153752536.divmod.quotient.35721@ohm> On Wed, 14 Nov 2007 13:24:41 +0100, Laura Creighton wrote: >In a message of Wed, 14 Nov 2007 03:53:18 +0100, Jacob Hall?n writes: > >>> So I'll definitely complain if you spend a lot of time on Ruby (or >>> Smalltalk for that matter) before Python's all the way there. I think >>> that'd be a bad idea for a whole range of reasons - get the last 10% >>> done for Python first before you take even more on your plates. We all >>> know that last 10% tends to be the hardest part. Focus is important. If >>> you can't make that work for Python, you'd have a hard time making it >>> work for any other language too (or convincing people that you can). Of >>> course I realize I have no real voice in this project, but that's my in >>put. > >I don't think you understand the state of our project. There is >practically nothing left that is 'python-centric' left to work on, >unless you have a burning desire to have the 2.5 features we don't >have yet, which is scheduled to get finished next week anyway. This >is hardly any work. Laura, I think I understand the point you're trying to make, but if you consider library support on PyPy as something which is "python-centric", then there really is a lot of work left to do: very few existing real Python programs can run on PyPy yet. This is not to say stackless, garbage collectors, and the JIT are not important (maybe even more important, I don't know), but they aren't the only remaining things necessary for PyPy to be a good Python runtime. Jean-Paul From faassen at startifact.com Wed Nov 14 14:23:35 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 14 Nov 2007 14:23:35 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <200711140353.18810.jacob@openend.se> References: <200711131523.lADFNoWO025693@theraft.openend.se> <200711140353.18810.jacob@openend.se> Message-ID: Hey, Jacob Hall?n wrote: > Den Tuesday 13 November 2007 19.42.03 skrev Martijn Faassen: >> I'm convinced the body count of Python users is probably still quite a >> bit larger than Ruby users. I also have a strong suspicion far more >> people are using Python web frameworks than there are people using >> Rails, hype aside, but I may be wrong about that. > > It doesn't really matter how many users you have, when someone measures > vibrant by how many excited people he meets at a conference. There > were 600 at a recent Rails conference, while Pycon gathers 400. > > This is the way Tim Bray sees things and it is not our mission to tell > him that he is wrong. It is our mission to get him to pay for PyPy > development. He is the customer, and the customer is always right, ;-) He's not *my* customer, so I can say whatever I like. :) Since I'm not your customer either, obviously I'm *not* always right, which means we can actually have a conversation. >> I think making it practical and useful to run a PyPy-based Python >> interpreter in production projects is probably the best shot this >> project has at becoming more self-sustaining. You're close. >> >> So I'll definitely complain if you spend a lot of time on Ruby (or >> Smalltalk for that matter) before Python's all the way there. I think >> that'd be a bad idea for a whole range of reasons - get the last 10% >> done for Python first before you take even more on your plates. We all >> know that last 10% tends to be the hardest part. Focus is important. If >> you can't make that work for Python, you'd have a hard time making it >> work for any other language too (or convincing people that you can). Of >> course I realize I have no real voice in this project, but that's my input. > > The reason we are not there yet for Python is financial, not due to a lack > of focus. (And what yo may have seen as a lack of focus during the funded > period was the totally opposite. We focused on delivering what we had > promised the EU, not on what would be the fastest path to be a viable > alternative to CPython.) > If we get funded to do Ruby, this will benefit Python, but the focus will be > on delivering what the one who pays wants out of the project, not on > making a better Python. I'm not the customer, but you'd have an awfully hard time selling this story to me. If I were looking for a technology to make Ruby work faster, and I got some people saying PyPy, but um, we don't have an actual completed language on it, not even Python, I would be skeptical. Moreover, if you seriously start developing a Ruby interpreter before Python is done, I'll call lack of focus. Similarly if you seriously start developer a Smalltalk interpreter. The PyPy project is researchy. Its people likes to play around with research topics, exploring possibilities, and of course that's more fun than to do the grind of making stuff work in production. (I'm not implying you haven't gone through a lot of grind; I'm just explaining the psychology that increases the risk) I don't want to talk about the history here: whether there may or may not have been a lack of focus in the past, during, say, the funding period. I understand the requirements of this period. But this project is, in my opinion, at present at *risk* of suffering of a lack of focus. I think this focus should be a production-ready Python interpreter. That's what you set out to do. It's not done yet. If you seriously start building other interpreters (beyond, perhaps, some exploration), you'll get sidetracked from that indefinitely. So, if you can get a Ruby interpreter funded at this stage, fine. I'm happy for you. But I will give up hope I'll be able to use any PyPy-based interpreter for the foreseeable future. Python, Ruby, or otherwise. (and you'd be free to prove me wrong, of course) Regards, Martijn From faassen at startifact.com Wed Nov 14 14:40:32 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 14 Nov 2007 14:40:32 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <200711141224.lAECOfar016106@theraft.openend.se> References: <200711131523.lADFNoWO025693@theraft.openend.se> <200711140353.18810.jacob@openend.se> <200711141224.lAECOfar016106@theraft.openend.se> Message-ID: Laura Creighton wrote: > In a message of Wed, 14 Nov 2007 03:53:18 +0100, Jacob Hall?n writes: > >>> So I'll definitely complain if you spend a lot of time on Ruby (or >>> Smalltalk for that matter) before Python's all the way there. I think >>> that'd be a bad idea for a whole range of reasons - get the last 10% >>> done for Python first before you take even more on your plates. We all >>> know that last 10% tends to be the hardest part. Focus is important. If >>> you can't make that work for Python, you'd have a hard time making it >>> work for any other language too (or convincing people that you can). Of >>> course I realize I have no real voice in this project, but that's my in >> put. > > I don't think you understand the state of our project. There is > practically nothing left that is 'python-centric' left to work on, > unless you have a burning desire to have the 2.5 features we don't > have yet, which is scheduled to get finished next week anyway. This > is hardly any work. Your Python-centric work is not done until I have a serious shot at downloading or building a PyPy-based interpreter, run it on top of *some* backend, and can start porting, say, Zope 3 on top of it. Or Twisted. Or lxml. Or Pygame. > Where we need work is to make the thing run a lot faster. So, better > gc, more work on the JIT -- and play better with c-extensions, and > enable us to use the stackless transform on CLI and JVM, and the > generated JIT on that as well. I'm well aware that lots of the work is there. But there are multiple ways of going about making this work. You can take on projects of a large magnitude such as the implementation of new interpreters, or you can focus on making it work right with one-and-a-half interpreters first, i.e. a Python one + a few proof of concept ones. Both will involve sacrifices to the project. If you take on another interpreter project, you'll sacrifice the ability to get anything in production in the foreseeable future. If you take on just Python (and a half), you'll sacrifice some of the generic nature of PyPy's work, at least for now, in the face of expediency. > All this stuff is translation level stuff. Thus it is relevant to all > the app-level input languages we support, not any particular one. > Speed up one, you speed up them all. I've been programming for a while too, you know. I don't buy this. You can't continue to *always* solve the generic problems without any consideration for an actual production goal. If you want to write a generic system for X, the way to go about it is not to take on different projects A, B, C, D and E equally. The way to go about it is to think about all of these projects, start off generic and keep it in the back of your mind, and then focus on getting project A to the end of the road. After this works reasonably well and people are using it and giving you feedback, you expand the breath again to B, then C, then D. If you start B before you finish A, or give all equal weight, you'll flounder around indefinitely. So far I think you've been doing this right. You've been focusing on making a Python interpreter work primarily. Other interpreters in RPython work too, but you haven't put a major push behind any one of them, just got them far enough to experiment with smaller cases, and prove some basic concepts there. The main focus has been Python. But it isn't there yet. > It took a 10 day sprint for some > of the pypyers and the squeak experts at the SCG to get Smalltalk up > and running. http://pypysqueak.blogspot.com/ Yes, I read that. It's cool work. And there's not a single real-world Smalltalk program in the world you can run on top of that today, right? How much more time and effort do you think it will take to get that far? How much more time and effort before it would it actually make *sense* for a Smalltalk developer to do so? I'd like to find out the same answer for Python, too. When do you start recommending a PyPy based interpreter as equally good for production-use (in some circumstances) as CPython, or Jython or IronPython? > Assuming we could find a > small group of similarly competant ruby language experts, we ought to > be able to do the same thing in the same sort of time range. If we > cannot find them, and have to do it all ourselves, it will take > longer, since we aren't experts on the arcane and obscure details > about the ruby language that only ruby wizards know. You'll be able to do a proof of concept in a short amount of time. That's all. > And then we can take Tim Bray's money and give him the fast ruby he > wants by working on precisely the same things that will give you the > fast python you want. So I don't see why you aren't up in the front > row cheering these developments as loudly as you can. Because I think I know how to finish software development projects, and because I see danger signals. > Right now the > most promising path to get you what you want, which I see as a fast python > that is production-quality and suitable for deploying Grok web apps, > runs hand in hand with Tim Bray's desire for a fast ruby that is > production-quality and suitable to deploy rails apps. You guys want > the same thing! You don't even have to squint! Why aren't you > cheering? Because if you take on anything bigger than a proof of concept for Ruby now (and even that is a lack of focus I don't think you can afford), I'm pretty sure I can wait a few years more before I can do that. Again, I'd be happy if you proved me wrong. I'm just giving you my perspective. I'm not speaking as a customer, and I know there's money involved and that's part of the real-world concerns of the projects too, but I am speaking as a friend. Regards, Martijn From sanxiyn at gmail.com Wed Nov 14 15:04:29 2007 From: sanxiyn at gmail.com (Sanghyeon Seo) Date: Wed, 14 Nov 2007 23:04:29 +0900 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: References: <200711131523.lADFNoWO025693@theraft.openend.se> <200711140353.18810.jacob@openend.se> <200711141224.lAECOfar016106@theraft.openend.se> Message-ID: <5b0248170711140604p24c75e6ftf03618dd1c8943ae@mail.gmail.com> 2007/11/14, Martijn Faassen : > Your Python-centric work is not done until I have a serious shot at > downloading or building a PyPy-based interpreter, run it on top of > *some* backend, and can start porting, say, Zope 3 on top of it. Or > Twisted. Or lxml. Or Pygame. Well, no Python implementation other than CPython runs Zope, Twisted, lxml, or pygame. (Correct me if I'm wrong.) Although I believe Jython ran some part of old version of Twisted. And effbot included IronPython support in ElementTree 1.2.7. But then, Jython and IronPython don't *need* to run Zope. Their survival isn't dependent on that. -- Seo Sanghyeon From faassen at startifact.com Wed Nov 14 15:06:34 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 14 Nov 2007 15:06:34 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <9c0bb8a00711132049m36f31fddma0bb4aa7e247490@mail.gmail.com> References: <200711111439.lABEdIZ5011340@theraft.openend.se> <200711131523.lADFNoWO025693@theraft.openend.se> <473A6FDB.1060301@stackless.com> <9c0bb8a00711132049m36f31fddma0bb4aa7e247490@mail.gmail.com> Message-ID: Paul deGrandis wrote: > I have stayed quiet thus far but I wanted to make sure everyone knew I > was still around and thought I'd contribute to the discussion now. > > I have been asked to give the same PyPy talk three times in the > Philadelphia area, and everytime I give a talk based on some of mwh's > slides, much to the same tune that Chris was singing in his email. > > The fact alone that PyPy team members are ABLE to go to different > companies with different interests and pitch the same product is > AMAZING and shows the strength of of the project and product that > everyone has helped form, shape, and create. Let me make it clear: I think PyPy should work for multiple languages. I also think PyPy has accomplished an amazing amount of amazing things. What I am debating is the way to go about getting where you evidently want to go now: to be a generic system that can support multiple languages. I believe the best way to go there is to make it work, and by that I mean really work, for one language, which I believe should be Python. What does it *mean* when I say: PyPy should really work for Python? I believe PyPy will really work for Python as soon as it makes sense for some users to actually use it in some circumstances. More sense than using CPython, IronPython or Jython. Why do I think this should be the focus? There are two reasons, one marketing related and the other one software-engineering related. The marketing reason is that you'll have a much much much more easy sell of generic PyPy technology once someone is using it in production. The other marketing reason is developer marketing: once you're using the technology in production, you'll get more developers motivated enough to improve the underlying technology. More mindshare. The software-engineering related reason is that you will find out, and solve, the last details only when you actually try to use the software for serious purposes, i.e, in production. If you expand the scope of your project too far before it works well in one particular case for actual people, you run the risk of never getting to those details. Those details are very important. You need to restrict your scope to get that far, before you widen your scope again to take on new challenges. If you don't restrict your scope you're in serious risk of losing your way. (I know this probably sounds like vague handwaving to you. I guess the original wiki is the best resource I know of distilled software engineering wisdom.) > Let people say what they want. Personally, I don't think Tim's blog > was harsh, and anyone who has spent ANY time on the Internet can > clearly see through the comment (and recognize it for what it is). I don't think Tim Bray's blog was harsh either. I was just disappointed some of the technology wasn't highlighted and was curious about how that had happened. The (harsh) comment to that blog I believe cannot be countered by spending time on the Internet: you won't find any people who are using PyPy interpreters in production. Before that it's hard to determine from the outside whether the project succeeded or not, if you at least take the PyPy project as a project to deliver production-ready interpreters at all and not just a research project. Again, I'm not trying to criticize what the project has already done. I'm trying to be a voice of pragmatism and the voice of a potential open source end-user. Listen to me if you like; it's not an enemy voice. There are two PyPy technologies that I can see that are close to pragmatic fruition now: * writing CPython modules in RPython (I completely lost track of what the state of this is since there have been a lot of changes) * running a Python interpreter that is in some ways better than an existing one. I was under the impression that this was supposed to be the more central focus of the project, not RPython. You know, "The PyPy project aims at producing a flexible and fast Python implementation.", what it says on your web page. Being a generic platform for interpreter developers or creating a pragmatic Smalltalk or Ruby interpreter are not close to fruition right now. The debate is now whether taking on those goals *now* will also further the goals above. Laura thinks it will, and I think it won't. (I think it'd be great to take on those goals later.) Besides accomplishing your goals there is of course also the matter of financial survival for the project. If you can sell your technology at all that's a plus for survival. I'm leaving finances out of the picture and you can argue I shouldn't. If we bring them into the picture, you can sell to me the idea that taking on more will help the survival of the project, and otherwise *nothing* will ever get done. You can't sell to me that this is actually good from a pragmatic software engineering perspective (where you want to bring projects into production), or from an open source project perspective (where you want to attract contributors who find your platform of use). Regards, Martijn From bborcic at gmail.com Wed Nov 14 15:25:35 2007 From: bborcic at gmail.com (Boris Borcic) Date: Wed, 14 Nov 2007 15:25:35 +0100 Subject: [pypy-dev] thunk object space : become() quirks Message-ID: <77f4ed2b0711140625s3b11e45ej61394be678abc368@mail.gmail.com> Hello, Looking for a way to use pypy in any manner, I came up with an idea for a deterministic sudoku solver that would use the 'become()' feature of the thunk object space. However, this doesn't work as I expected. The following shell trace illustrates my problem. Is this a bug or otherwise ? Can anybody explain ? BTW, I am using the pypy 1.0.0 build 41438 windows binary distribution. [subquestion : is pypy-dev appropriate for such a question ?] TIA, Boris Borcic >>>> s = lambda *x : set(x) >>>> s1 = s(1) >>>> s2 = s(2) >>>> s3 = s(3) >>>> s4 = s(4) >>>> s1,s2,s3,s4 (set([1]), set([2]), set([3]), set([4])) >>>> s1.update(s2) >>>> become(s2,s1) >>>> s3.update(s4) >>>> become(s4,s3) >>>> s1,s2,s3,s4 (set([1, 2]), set([1, 2]), set([3, 4]), set([3, 4])) >>>> s4.update(s2) >>>> s4 set([3, 4]) ------------------> expected set([1,2,3,4]) >>>> s2 set([1, 2]) >>>> s4 is s3 True >>>> s3.update(s2) >>>> s4 set([3, 4, 2]) ------------------> expected set([1,2,3,4]) >>>> s3 set([3, 4, 2]) >>>> s1 set([1, 2]) >>>> s1 is s2 True >>>> s3.update(s1) >>>> s4 set([3, 4, 2, 1]) >>>> From stephan.diehl at gmx.net Wed Nov 14 15:28:24 2007 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Wed, 14 Nov 2007 15:28:24 +0100 Subject: [pypy-dev] another pypy blog entry from sun Message-ID: <200711141528.24474.stephan.diehl@gmx.net> just in case you don't read programming.reddit.com: there is another blog enty from somebody at Sun (John Rose) http://blogs.sun.com/jrose/entry/a_day_with_pypy Have fun Stephan From cfbolz at gmx.de Wed Nov 14 15:41:20 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 14 Nov 2007 15:41:20 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: References: <200711111439.lABEdIZ5011340@theraft.openend.se> <200711131523.lADFNoWO025693@theraft.openend.se> <473A6FDB.1060301@stackless.com> <9c0bb8a00711132049m36f31fddma0bb4aa7e247490@mail.gmail.com> Message-ID: <473B0910.2010203@gmx.de> Hi Martijn, let me first note that I agree with a lot of your points, when not taking the financial side of things into account. I will ignore the financial side of things in this mail (since I don't think anybody proposes to focus on something completely different than our Python interpreter _without_ major financial incentives). Martijn Faassen wrote: [snip] > Again, I'm not trying to criticize what the project has already done. > I'm trying to be a voice of pragmatism and the voice of a potential open > source end-user. Listen to me if you like; it's not an enemy voice. > There are two PyPy technologies that I can see that are close to > pragmatic fruition now: > > * writing CPython modules in RPython (I completely lost track of what > the state of this is since there have been a lot of changes) It's still working but mostly accidentally. It needs some rethinking and a rewrite. We have now better machinery and also ideas how to make it really fast, but it's a manpower problem. There isn't anyone interested enough in supporting this in the current PyPy team. If somebody outside the current PyPy-team really wanted this and were willing to work on it we (or at least me personally) would give him all the support we can. > * running a Python interpreter that is in some ways better than an > existing one. I was under the impression that this was supposed to be > the more central focus of the project, not RPython. You know, "The PyPy > project aims at producing a flexible and fast Python implementation.", > what it says on your web page. Well, I guess what we really need to get adoption in this area is a reason for people to switch to PyPy's Python interpreter. Right now doing this means you get a slower Python, with less available extensions. Of course you get nice features like lazy objects, Stackless, transparent proxies, etc. But, it's not clear to me that these features are actually worth switching for anyone, after all there is no massive adoption of Stackless Python either (which has some of the benefits and lacks some of the disadvantages of PyPy). I guess one obvious candidate for something PyPy can provide but CPython will never be able to provide is the sandboxing work that was done recently. > Being a generic platform for interpreter developers or creating a > pragmatic Smalltalk or Ruby interpreter are not close to fruition right > now. The debate is now whether taking on those goals *now* will also > further the goals above. Laura thinks it will, and I think it won't. (I > think it'd be great to take on those goals later.) Completely sidenoteish and beside the point, but: I disagree about that it's hard to get a pragmatic Smalltalk interpreter, since you only need to implement the VM (which we already did) and some primitives to get a good Smalltalk VM, since all the hard bits are done by the image anyway, and we don't plan to re-create an image but just reuse Squeak's. End sidenote. Cheers, Carl Friedrich From fijall at gmail.com Wed Nov 14 15:49:11 2007 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 14 Nov 2007 15:49:11 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <473B0910.2010203@gmx.de> References: <200711111439.lABEdIZ5011340@theraft.openend.se> <200711131523.lADFNoWO025693@theraft.openend.se> <473A6FDB.1060301@stackless.com> <9c0bb8a00711132049m36f31fddma0bb4aa7e247490@mail.gmail.com> <473B0910.2010203@gmx.de> Message-ID: <693bc9ab0711140649y1eca8ffbi68e04c1eea964d59@mail.gmail.com> > > Martijn Faassen wrote: > [snip] > > Again, I'm not trying to criticize what the project has already done. > > I'm trying to be a voice of pragmatism and the voice of a potential > open > > source end-user. Listen to me if you like; it's not an enemy voice. > > There are two PyPy technologies that I can see that are close to > > pragmatic fruition now: > > > > * writing CPython modules in RPython (I completely lost track of what > > the state of this is since there have been a lot of changes) > > It's still working but mostly accidentally. It needs some rethinking and > a rewrite. We have now better machinery and also ideas how to make it > really fast, but it's a manpower problem. There isn't anyone interested > enough in supporting this in the current PyPy team. If somebody outside > the current PyPy-team really wanted this and were willing to work on it > we (or at least me personally) would give him all the support we can. Me as well. The thing is that extcompiler is keeping alive a lot of code which we don't need otherwise. I would be more than happy to help someone reimplement it (shouldn't be a hard task anyway, our machinery is better suited by now). I can even imagine doing that for money. But I'm not willing to spend my free time on that (besides helping someone, which always goes into an account). The main reason is that we really don't want to fork our efforts into too many directions. Also note that while this might be a good idea, you still end up with RPython, which is nice, cool, better than pyrex or C, but for example error reporting is somewhat not-as-good-as-possible. We're trying to improve it anyway, but making extcompiler nice cool tool would need some external will. Cheers, fijal -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at ar.media.kyoto-u.ac.jp Wed Nov 14 15:38:49 2007 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 14 Nov 2007 23:38:49 +0900 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <473B0910.2010203@gmx.de> References: <200711111439.lABEdIZ5011340@theraft.openend.se> <200711131523.lADFNoWO025693@theraft.openend.se> <473A6FDB.1060301@stackless.com> <9c0bb8a00711132049m36f31fddma0bb4aa7e247490@mail.gmail.com> <473B0910.2010203@gmx.de> Message-ID: <473B0879.4090604@ar.media.kyoto-u.ac.jp> Carl Friedrich Bolz wrote: > Hi Martijn, > > let me first note that I agree with a lot of your points, when not > taking the financial side of things into account. I will ignore the > financial side of things in this mail (since I don't think anybody > proposes to focus on something completely different than our Python > interpreter _without_ major financial incentives). > > Martijn Faassen wrote: > [snip] > > Again, I'm not trying to criticize what the project has already done. > > I'm trying to be a voice of pragmatism and the voice of a potential open > > source end-user. Listen to me if you like; it's not an enemy voice. > > There are two PyPy technologies that I can see that are close to > > pragmatic fruition now: > > > > * writing CPython modules in RPython (I completely lost track of what > > the state of this is since there have been a lot of changes) > > It's still working but mostly accidentally. It needs some rethinking and > a rewrite. We have now better machinery and also ideas how to make it > really fast, but it's a manpower problem. There isn't anyone interested > enough in supporting this in the current PyPy team. If somebody outside > the current PyPy-team really wanted this and were willing to work on it > we (or at least me personally) would give him all the support we can. Sorry for jumping in, and I hope this is not too OT, but what is missing for a 'real' CPython alternative ? - some bits of the standard library: how much ? Can this be worked on by people 'outside pypy' (by outside, I mean people who know python, who are willing to learn rpython, but who cannot work on the whole translation/gc/JIT/whatever because they are not smart enough, someone like me :) ) - some language constructs (my understanding is that the current language supported is python2.4 ?) ? - implementation problems (some things too slow, not backward compatible) ? cheers, David From cfbolz at gmx.de Wed Nov 14 15:49:52 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 14 Nov 2007 15:49:52 +0100 Subject: [pypy-dev] thunk object space : become() quirks In-Reply-To: <77f4ed2b0711140625s3b11e45ej61394be678abc368@mail.gmail.com> References: <77f4ed2b0711140625s3b11e45ej61394be678abc368@mail.gmail.com> Message-ID: <473B0B10.4050409@gmx.de> Hi Boris, Boris Borcic wrote: > Looking for a way to use pypy in any manner, I came up with an idea for a > deterministic sudoku solver that would use the 'become()' feature of the thunk > object space. However, this doesn't work as I expected. The following shell > trace illustrates my problem. Is this a bug or otherwise ? Can anybody explain ? > > BTW, I am using the pypy 1.0.0 build 41438 windows binary distribution. > > [subquestion : is pypy-dev appropriate for such a question ?] It's a bug, I have a clue where it is and will try to fix it. Thanks for reporting! Yes, the list is completely appropriate. Cheers, Carl Friedrich From faassen at startifact.com Wed Nov 14 15:56:48 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 14 Nov 2007 15:56:48 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <473B0910.2010203@gmx.de> References: <200711111439.lABEdIZ5011340@theraft.openend.se> <200711131523.lADFNoWO025693@theraft.openend.se> <473A6FDB.1060301@stackless.com> <9c0bb8a00711132049m36f31fddma0bb4aa7e247490@mail.gmail.com> <473B0910.2010203@gmx.de> Message-ID: <8928d4e90711140656y31dbcc5cteba5077b5bcd4ac1@mail.gmail.com> Hey, On Nov 14, 2007 3:41 PM, Carl Friedrich Bolz wrote: > Martijn Faassen wrote: [snip] > > * writing CPython modules in RPython (I completely lost track of what > > the state of this is since there have been a lot of changes) > > It's still working but mostly accidentally. It needs some rethinking and > a rewrite. We have now better machinery and also ideas how to make it > really fast, but it's a manpower problem. There isn't anyone interested > enough in supporting this in the current PyPy team. If somebody outside > the current PyPy-team really wanted this and were willing to work on it > we (or at least me personally) would give him all the support we can. Thanks for the update! I'd be interested in joining a future sprint (next year, say) to work on this (and generally get more familiar with PyPy's codebase). > > * running a Python interpreter that is in some ways better than an > > existing one. I was under the impression that this was supposed to be > > the more central focus of the project, not RPython. You know, "The PyPy > > project aims at producing a flexible and fast Python implementation.", > > what it says on your web page. > > Well, I guess what we really need to get adoption in this area is a > reason for people to switch to PyPy's Python interpreter. Yes, the main reason. In addition of course you want to make it as easy as possible, but you'll get at least some people even if it starts out relatively hard, as long as the gains outweigh the pain. > Right now > doing this means you get a slower Python, with less available > extensions. Of course you get nice features like lazy objects, > Stackless, transparent proxies, etc. But, it's not clear to me that > these features are actually worth switching for anyone, after all there > is no massive adoption of Stackless Python either (which has some of the > benefits and lacks some of the disadvantages of PyPy). > > I guess one obvious candidate for something PyPy can provide but CPython > will never be able to provide is the sandboxing work that was done > recently. One thing that would definitely help adoption is a JIT, of course. I also like Holger's idea of making it work real well with Java on the JVM platform. Even though some of the familiar Python extensions would be missing, there are a ton of Java libraries. It might be worthwhile for people to port their existing programs or libraries to the PyPy/JVM interpreter. Of course Jython offers these features too in a more stable form, but PyPy has the promise of better portability plus hopefully speed gains in the future. > > Being a generic platform for interpreter developers or creating a > > pragmatic Smalltalk or Ruby interpreter are not close to fruition right > > now. The debate is now whether taking on those goals *now* will also > > further the goals above. Laura thinks it will, and I think it won't. (I > > think it'd be great to take on those goals later.) > > Completely sidenoteish and beside the point, but: I disagree about that > it's hard to get a pragmatic Smalltalk interpreter, since you only need > to implement the VM (which we already did) and some primitives to get a > good Smalltalk VM, since all the hard bits are done by the image anyway, > and we don't plan to re-create an image but just reuse Squeak's. End > sidenote. That's interesting to hear. I imagine Smalltalk has a bit of a special nature in that it is designed to be easy to bootstrap. How much more work would it be to make the existing image work though? And when would people be attracted over to this platform? Regards, Martijn From fijall at gmail.com Wed Nov 14 15:57:13 2007 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 14 Nov 2007 15:57:13 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <473B0879.4090604@ar.media.kyoto-u.ac.jp> References: <200711111439.lABEdIZ5011340@theraft.openend.se> <200711131523.lADFNoWO025693@theraft.openend.se> <473A6FDB.1060301@stackless.com> <9c0bb8a00711132049m36f31fddma0bb4aa7e247490@mail.gmail.com> <473B0910.2010203@gmx.de> <473B0879.4090604@ar.media.kyoto-u.ac.jp> Message-ID: <693bc9ab0711140657n6806e551vd479794a45c3d295@mail.gmail.com> On Nov 14, 2007 3:38 PM, David Cournapeau wrote: > Carl Friedrich Bolz wrote: > > Hi Martijn, > > > > let me first note that I agree with a lot of your points, when not > > taking the financial side of things into account. I will ignore the > > financial side of things in this mail (since I don't think anybody > > proposes to focus on something completely different than our Python > > interpreter _without_ major financial incentives). > > > > Martijn Faassen wrote: > > [snip] > > > Again, I'm not trying to criticize what the project has already done. > > > I'm trying to be a voice of pragmatism and the voice of a potential > open > > > source end-user. Listen to me if you like; it's not an enemy voice. > > > There are two PyPy technologies that I can see that are close to > > > pragmatic fruition now: > > > > > > * writing CPython modules in RPython (I completely lost track of what > > > the state of this is since there have been a lot of changes) > > > > It's still working but mostly accidentally. It needs some rethinking and > > a rewrite. We have now better machinery and also ideas how to make it > > really fast, but it's a manpower problem. There isn't anyone interested > > enough in supporting this in the current PyPy team. If somebody outside > > the current PyPy-team really wanted this and were willing to work on it > > we (or at least me personally) would give him all the support we can. > Sorry for jumping in, and I hope this is not too OT, but what is missing > for a 'real' CPython alternative ? > - some bits of the standard library: how much ? Can this be worked > on by people 'outside pypy' (by outside, I mean people who know python, > who are willing to learn rpython, but who cannot work on the whole > translation/gc/JIT/whatever because they are not smart enough, someone > like me :) ) Yes and yest. This is our major problem and it could be worked by people from outside, but I would wait for two things a) completion of app-level ctypes (it's not far, needs only pure-python code) b) separate compilation (hard, but very high on our todo list) > > - some language constructs (my understanding is that the current > language supported is python2.4 ?) ? No, this is perfectly fine. > > - implementation problems (some things too slow, not backward > compatible) ? > Slow, yes. Some things. But this is incremental work all over the place. Recently there has been a lot of improvements in string/unicode area for example. If you feel like profiling certain parts (like regular expressions), would be cool. Cheers, fijal -------------- next part -------------- An HTML attachment was scrubbed... URL: From faassen at startifact.com Wed Nov 14 16:09:03 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 14 Nov 2007 16:09:03 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <5b0248170711140604p24c75e6ftf03618dd1c8943ae@mail.gmail.com> References: <200711131523.lADFNoWO025693@theraft.openend.se> <200711140353.18810.jacob@openend.se> <200711141224.lAECOfar016106@theraft.openend.se> <5b0248170711140604p24c75e6ftf03618dd1c8943ae@mail.gmail.com> Message-ID: Sanghyeon Seo wrote: > 2007/11/14, Martijn Faassen : >> Your Python-centric work is not done until I have a serious shot at >> downloading or building a PyPy-based interpreter, run it on top of >> *some* backend, and can start porting, say, Zope 3 on top of it. Or >> Twisted. Or lxml. Or Pygame. > > Well, no Python implementation other than CPython runs Zope, Twisted, > lxml, or pygame. (Correct me if I'm wrong.) > > Although I believe Jython ran some part of old version of Twisted. And > effbot included IronPython support in ElementTree 1.2.7. > > But then, Jython and IronPython don't *need* to run Zope. Their > survival isn't dependent on that. Sure, you're right, that's true. See another mail where I give a better description of what I mean with "done": something that motivates people to use it in production. That said, I am of course also seriously interested in running existing code on the PyPy platform. Regards, Martijn From faassen at startifact.com Wed Nov 14 16:14:11 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 14 Nov 2007 16:14:11 +0100 Subject: [pypy-dev] another pypy blog entry from sun In-Reply-To: <200711141528.24474.stephan.diehl@gmx.net> References: <200711141528.24474.stephan.diehl@gmx.net> Message-ID: Hey, Stephan Diehl wrote: > just in case you don't read programming.reddit.com: > there is another blog enty from somebody at Sun (John Rose) > http://blogs.sun.com/jrose/entry/a_day_with_pypy Yeah, I saw it too. Nice and positive, and in many parts over my head. :) Regards, Martijn From faassen at startifact.com Wed Nov 14 16:13:00 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 14 Nov 2007 16:13:00 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <473B0879.4090604@ar.media.kyoto-u.ac.jp> References: <200711111439.lABEdIZ5011340@theraft.openend.se> <200711131523.lADFNoWO025693@theraft.openend.se> <473A6FDB.1060301@stackless.com> <9c0bb8a00711132049m36f31fddma0bb4aa7e247490@mail.gmail.com> <473B0910.2010203@gmx.de> <473B0879.4090604@ar.media.kyoto-u.ac.jp> Message-ID: David Cournapeau wrote: [snip] > Sorry for jumping in, and I hope this is not too OT, but what is missing > for a 'real' CPython alternative ? I'm happy you are jumping in, I think stuff like this should be discussed. Perhaps we should start another thread, though. > - some bits of the standard library: how much ? Can this be worked > on by people 'outside pypy' (by outside, I mean people who know python, > who are willing to learn rpython, but who cannot work on the whole > translation/gc/JIT/whatever because they are not smart enough, someone > like me :) ) > - some language constructs (my understanding is that the current > language supported is python2.4 ?) ? > - implementation problems (some things too slow, not backward > compatible) ? I'd say that is correct, in addition of course there needs to be: * something to motivate people to start using it. I.e. some feature or library access or performance benefit. And (not essential at first, but important nonetheless): * any easy starting point. I.e. Joe Random Hacker can install the PyPy interpreter and get started without a lot of hassle. Regards, Martijn From paul.degrandis at gmail.com Wed Nov 14 16:19:07 2007 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Wed, 14 Nov 2007 10:19:07 -0500 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: References: <200711111439.lABEdIZ5011340@theraft.openend.se> <200711131523.lADFNoWO025693@theraft.openend.se> <473A6FDB.1060301@stackless.com> <9c0bb8a00711132049m36f31fddma0bb4aa7e247490@mail.gmail.com> Message-ID: <9c0bb8a00711140719i23f0a03ds79dd5073eeddef9f@mail.gmail.com> Martijn, Don't get me wrong, like Carl, without taking finances into account, I agree with you totally. As a software engineering major, I can also see most of your arguments from a purely engineering and practical perspective. I mean, c'mon, who doesn't want to see PyPy offer you the ultimate Python experience. I don't know about anyone else, but that's what brought me to PyPy in the first place. I love common lisp and lisp macros, I like operating at different levels in a language to achieve elegant solutions, and I love python. I felt like PyPy could give this too me and more. Has PyPy been that for me? Yes and no. Like you I wish there was complete python support, but the architecture and abilities of PyPy as an infrastructure blow me away. It has potential to grow, stretch, and extend, in an elegant and sound manner, with little effort. It's hard to find such a system that is capable of that with sucha caring and vibrant community surrounding it. As a grad student working in academia and doing side projects, I do whatever I have to do to learn, better myself, and pay the bills. The perfect case is Summer Of Code. While working on the jvm backend, Drexel University secured me funding, resources, and a lab to do my autonomic computing research, and I'd be a fool not to follow it, even it meant shelving some things on PyPy for a bit. Financial concerns have become a part of PyPy, it's just the way it is. I appreciate your response and admire your passion for the project. By no means do I see contributions to this discussion as a threat or an attack. Regards, Paul On 11/14/07, Martijn Faassen wrote: > Paul deGrandis wrote: > > I have stayed quiet thus far but I wanted to make sure everyone knew I > > was still around and thought I'd contribute to the discussion now. > > > > I have been asked to give the same PyPy talk three times in the > > Philadelphia area, and everytime I give a talk based on some of mwh's > > slides, much to the same tune that Chris was singing in his email. > > > > The fact alone that PyPy team members are ABLE to go to different > > companies with different interests and pitch the same product is > > AMAZING and shows the strength of of the project and product that > > everyone has helped form, shape, and create. > > Let me make it clear: I think PyPy should work for multiple languages. I > also think PyPy has accomplished an amazing amount of amazing things. > > What I am debating is the way to go about getting where you evidently > want to go now: to be a generic system that can support multiple > languages. I believe the best way to go there is to make it work, and by > that I mean really work, for one language, which I believe should be Python. > > What does it *mean* when I say: PyPy should really work for Python? I > believe PyPy will really work for Python as soon as it makes sense for > some users to actually use it in some circumstances. More sense than > using CPython, IronPython or Jython. > > Why do I think this should be the focus? There are two reasons, one > marketing related and the other one software-engineering related. > > The marketing reason is that you'll have a much much much more easy sell > of generic PyPy technology once someone is using it in production. The > other marketing reason is developer marketing: once you're using the > technology in production, you'll get more developers motivated enough to > improve the underlying technology. More mindshare. > > The software-engineering related reason is that you will find out, and > solve, the last details only when you actually try to use the software > for serious purposes, i.e, in production. If you expand the scope of > your project too far before it works well in one particular case for > actual people, you run the risk of never getting to those details. Those > details are very important. You need to restrict your scope to get that > far, before you widen your scope again to take on new challenges. If you > don't restrict your scope you're in serious risk of losing your way. (I > know this probably sounds like vague handwaving to you. I guess the > original wiki is the best resource I know of distilled software > engineering wisdom.) > > > Let people say what they want. Personally, I don't think Tim's blog > > was harsh, and anyone who has spent ANY time on the Internet can > > clearly see through the comment (and recognize it for what it is). > > I don't think Tim Bray's blog was harsh either. I was just disappointed > some of the technology wasn't highlighted and was curious about how that > had happened. The (harsh) comment to that blog I believe cannot be > countered by spending time on the Internet: you won't find any people > who are using PyPy interpreters in production. Before that it's hard to > determine from the outside whether the project succeeded or not, if you > at least take the PyPy project as a project to deliver production-ready > interpreters at all and not just a research project. > > Again, I'm not trying to criticize what the project has already done. > I'm trying to be a voice of pragmatism and the voice of a potential open > source end-user. Listen to me if you like; it's not an enemy voice. > There are two PyPy technologies that I can see that are close to > pragmatic fruition now: > > * writing CPython modules in RPython (I completely lost track of what > the state of this is since there have been a lot of changes) > > * running a Python interpreter that is in some ways better than an > existing one. I was under the impression that this was supposed to be > the more central focus of the project, not RPython. You know, "The PyPy > project aims at producing a flexible and fast Python implementation.", > what it says on your web page. > > Being a generic platform for interpreter developers or creating a > pragmatic Smalltalk or Ruby interpreter are not close to fruition right > now. The debate is now whether taking on those goals *now* will also > further the goals above. Laura thinks it will, and I think it won't. (I > think it'd be great to take on those goals later.) > > Besides accomplishing your goals there is of course also the matter of > financial survival for the project. If you can sell your technology at > all that's a plus for survival. I'm leaving finances out of the picture > and you can argue I shouldn't. If we bring them into the picture, you > can sell to me the idea that taking on more will help the survival of > the project, and otherwise *nothing* will ever get done. You can't sell > to me that this is actually good from a pragmatic software engineering > perspective (where you want to bring projects into production), or from > an open source project perspective (where you want to attract > contributors who find your platform of use). > > Regards, > > Martijn > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From faassen at startifact.com Wed Nov 14 16:24:43 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 14 Nov 2007 16:24:43 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <9c0bb8a00711140719i23f0a03ds79dd5073eeddef9f@mail.gmail.com> References: <200711111439.lABEdIZ5011340@theraft.openend.se> <200711131523.lADFNoWO025693@theraft.openend.se> <473A6FDB.1060301@stackless.com> <9c0bb8a00711132049m36f31fddma0bb4aa7e247490@mail.gmail.com> <9c0bb8a00711140719i23f0a03ds79dd5073eeddef9f@mail.gmail.com> Message-ID: <8928d4e90711140724m18a11a1en4e9561a3f870c690@mail.gmail.com> Hey Paul, On Nov 14, 2007 4:19 PM, Paul deGrandis wrote: [snip] > Financial concerns have become a part of PyPy, it's just the way it is. I do understand that. I've been writing open source software for a living and paying the rent since 2001 or thereabouts. I want to make sure that the potential drawbacks of various strategies are also highlighted. Financial concerns are after all just one concern of the project. > I appreciate your response and admire your passion for the project. > By no means do I see contributions to this discussion as a threat or > an attack. Thank you, that's good to hear! Thanks for the kind words. I'm glad we're all still getting along. :) Regards, Martijn From grobinson at goombah.com Wed Nov 14 16:46:11 2007 From: grobinson at goombah.com (Gary Robinson) Date: Wed, 14 Nov 2007 10:46:11 -0500 Subject: [pypy-dev] pypy on Tim Bray's blog Message-ID: <20071114104611046496.82f4f35b@goombah.com> > Well, I guess what we really need to get adoption in this area is a > reason for people to switch to PyPy's Python interpreter. Right now > doing this means you get a slower Python, with less available > extensions. Of course you get nice features like lazy objects, > Stackless, transparent proxies, etc. But, it's not clear to me that > these features are actually worth switching for anyone, after all there > is no massive adoption of Stackless Python either (which has some of the > benefits and lacks some of the disadvantages of PyPy). The thing that I've been waiting for is for pypy to live up to its promise of being significantly faster than cPython. If it was, and barring other major problems, I'd be using it in production right now. -- Gary Robinson CTO Emergent Music, LLC grobinson at goombah.com 207-942-3463 Company: http://www.goombah.com Blog: http://www.garyrobinson.net From cfbolz at gmx.de Wed Nov 14 17:16:48 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 14 Nov 2007 17:16:48 +0100 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) Message-ID: <473B1F70.3070108@gmx.de> Hi all, I thought I would start a new thread for discussing what PyPy needs to become "production ready" (whatever that is) and succeed as a Python implementation. This is something were outside opinions are very welcome (of course there is no guarantee we can implement all of your wishes :-) ). Obvious problems are: - some missing extension modules of the stdlib. no clue which of them are really important, depends on the application you want to run, I guess. This can be worked on by outsiders. - lack of 3rd-party extension-module support. This is hard to change. Once we implement ctypes the problem will be mitigated, but not solved: it's no fun at all to wrap a large C++ library with ctypes, and there are lot's of existing non-ctypes based modules out there. - Speed. The JIT is still not in a state where it really speeds up arbitrary Python code. I expect this to change sooner or later. However, it's not an area were a lot of people can help. - for PyPy-JVM: bindings to allow the interaction with arbitrary Java libraries, threading support ... Comments? Cheers, Carl Friedrich From anto.cuni at gmail.com Wed Nov 14 18:21:12 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Wed, 14 Nov 2007 18:21:12 +0100 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: <473B1F70.3070108@gmx.de> References: <473B1F70.3070108@gmx.de> Message-ID: <473B2E88.8020402@gmail.com> Carl Friedrich Bolz wrote: > - for PyPy-JVM: bindings to allow the interaction with arbitrary Java > libraries, threading support Moreover, I would add to this list that the possibility to compile python to jvm bytecode instead of python bytecode; maybe a pypy-jvm would be usable even without it, but e.g. developing applets requires it. Hopefully, I'll be able to work on this (an its counterpart for .NET) in the next months. ciao Anto From d.mcneil at qmul.ac.uk Wed Nov 14 18:40:54 2007 From: d.mcneil at qmul.ac.uk (Douglas McNeil) Date: Wed, 14 Nov 2007 17:40:54 +0000 (GMT) Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: <473B1F70.3070108@gmx.de> References: <473B1F70.3070108@gmx.de> Message-ID: cfb wrote: > I thought I would start a new thread for discussing what PyPy needs to > become "production ready" (whatever that is) and succeed as a Python > implementation. > [...] > - Speed. The JIT is still not in a state where it really speeds up > arbitrary Python code. I expect this to change sooner or later. > However, it's not an area were a lot of people can help. Arbitrary code is less interesting to me than JIT-powered fast numerical code. Moreover, we numerics types have much lower standards of "production ready" than the general public, and are willing to turn on options with names like --make-dangerous-assumptions-about-code-for-speed -do-not-use-this-flag-really-do-not-use-it-i-warned-you. Currently there is no One Obvious Way to do heavy numerical programming in python. To actually get things done requires a mix of numpy, boost, psyco, pyrex, pyinline, SWIG, some of the existing pypy tools -- even wrapped shedskin if you're feeling brave. The toolset is unwieldy. Yes, it's true that these often suffice -- I've run hundreds of semianalytic models over the past week myself using numpy/pygsl -- but I can't write my main production codes in python. And it's frustrating when you write a nice piece of code and then bump up against speed limits you can't escape without ugly inline hacks I can't expect the people I encourage to use python for science to learn. This is probably the most low-hanging fruit there could be for a (fully float-aware) JIT. The functions tends to be embarrassingly simple, and seldom leave the int/float/list domain. Most numerical code is borderline RPython as-is. >From previous discussions, I suspect I'm not the only lurker-fan who would be willing to commit time to working on numericentric graph optimizations when that becomes a worthwhile investment. There's no reason that the mostly-fortran bits of python code shouldn't run almost as fast as fortran after amortizing the JIT costs. Doug -- Queen Mary College, University of London "Still creating worlds.. Mathematical Sciences, Astronomy Unit .. but now with an accent!" From anto.cuni at gmail.com Wed Nov 14 19:20:54 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Wed, 14 Nov 2007 19:20:54 +0100 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: References: <473B1F70.3070108@gmx.de> Message-ID: <473B3C86.2090609@gmail.com> Douglas McNeil wrote: > This is probably the most low-hanging fruit there could be for a (fully > float-aware) JIT. The functions tends to be embarrassingly simple, and > seldom leave the int/float/list domain. Most numerical code is borderline > RPython as-is. if everything you need is to execute some function that plays with numbers even at the costs of losing some python semantics (e.g., automatic int to long promotion), maybe RPython can do the job. Create the following file as pypy/translator/goal/targetfibo.py def target(*args): return fibo, [int] def fibo(N): "calculates the Nth fibonacci number" a, b = 1, 1 for i in range(N-1): a, b = b, a+b return a Then, translate it: $ cd /path/to/pypy/translator/goal: $ ./translate.py --batch targetfibo.py $ /tmp/usession-${USER}/testing_1 Here you can find the file testing_1.so that can be imported in CPython as a module; the module contains the function fibo you defined above: >>> import testing_1 >>> map(testing_1.fibo, range(1, 10)) [1, 1, 2, 3, 5, 8, 13, 21, 34] ===================== == BIG FAT WARNING == ===================== I don't think we want to officially support this feature, and might suddenly stop working at any time. Moreover, there are a lot of drawbacks: - it can crash your python interpreter: e.g., if I try to divide by 0 inside the rpython function without a try/except block, my python prints "Floating point exception", then crashes. Segfaults could happily happens, too; - only few input/output types are supported; it should work with int, float and str; list and dictionaries are not supported (you can use them inside your function, but they can't cross the borders); Anyway, if all you need is to work with numbers and no much more, this might fit your needs. Just to make it clear, I'm not talking about the extcompiler: this is another piece of code which in theory does not suffer from the problems above and offers much more, but its status is not clear at the moment and will probably be rewritten soon or later. ciao Anto From fijall at gmail.com Wed Nov 14 20:05:38 2007 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 14 Nov 2007 20:05:38 +0100 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: <473B3C86.2090609@gmail.com> References: <473B1F70.3070108@gmx.de> <473B3C86.2090609@gmail.com> Message-ID: <693bc9ab0711141105s29c07897o65eef87450cdedbc@mail.gmail.com> Just a small note. We've got nowadays pieces of code that can help you handle safety-for-speed even in compiled rpython code, --sandbox uses those. (Like check for NULL pointers, enable assertions, etc.) But besides, I fully agree to the BIG FAT WARNING. This code is mostly there for tests. Even strings should not cross this border Cheers, fijal -------------- next part -------------- An HTML attachment was scrubbed... URL: From faassen at startifact.com Wed Nov 14 20:24:37 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 14 Nov 2007 20:24:37 +0100 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: <473B1F70.3070108@gmx.de> References: <473B1F70.3070108@gmx.de> Message-ID: Hey Carl, Carl Friedrich Bolz wrote: > I thought I would start a new thread for discussing what PyPy needs to > become "production ready" (whatever that is) and succeed as a Python > implementation. This is something were outside opinions are very welcome > (of course there is no guarantee we can implement all of your wishes :-) > ). Thanks for starting this discussion. Happy to see it! > Obvious problems are: > - some missing extension modules of the stdlib. no clue which of them > are really important, depends on the application you want to run, I > guess. This can be worked on by outsiders. > > - lack of 3rd-party extension-module support. This is hard to change. > Once we implement ctypes the problem will be mitigated, but not > solved: it's no fun at all to wrap a large C++ library with ctypes, > and there are lot's of existing non-ctypes based modules out there. Yes, this one would be a major challenge. If you can crack it and the PyPy interpreter offers another benefit (an obvious one is speed), you will be in awesome position. If not, the other benefits will have to weigh more strongly. This is definitely one to evolve slowly over time if it's possible at all. > - Speed. The JIT is still not in a state where it really speeds up > arbitrary Python code. I expect this to change sooner or later. > However, it's not an area were a lot of people can help. > > - for PyPy-JVM: bindings to allow the interaction with arbitrary Java > libraries, threading support Does this already exist for PyPy-CLR? - An easy starting point. I'd say one other important point is to have an easy starting point. I.e. Joe Random Hacker can install the PyPy interpreter and get started without a lot of hassle. Of course if the benefits outweigh the installation hassle people will still install it, but you'll limit your audience. It's not that hard now of course: creating new interpreters is just a matter of waiting a long time. I also heard you were working on an online PyPy build system, but I'm not sure what the status of this is. Still, things like Windows installers and inclusion in Linux distributions (or at least making packages available for them) will eventually be of great value. Generally, for any version of PyPy, there are questions like stability and memory usage. - Stability. Stability is obvious: we'll find out a lot more on how stable things are once production-use ramps up. I suspect PyPy is a in a good position due to the way interpreters are generated. - Memory usage. Memory usage is related to performance. If a PyPy interpreter is faster than CPython but uses up vasts amounts of memory (especially if it seems to leak), that'll be a problem. I know there's a lot of work being done on garbage collection. I don't really know what the expected outcome is here. :) - Release management. Don't underestimate the matter of release management. I like my foundational software to have a pattern of regular releases, and I think many other developers share this with me. I can't do production development on some SVN trunk. I need the foundation to be stable and unchanging, unless I explicitly change it. If there are major bugs, I'd expect a quick bugfix release. Doing a proper release can be quite a bit of effort, and needs people that feel responsible to do this rather frustrating and boring work. The CPython project does this well. Regards, Martijn From charles.nutter at sun.com Wed Nov 14 20:28:37 2007 From: charles.nutter at sun.com (Charles Oliver Nutter) Date: Wed, 14 Nov 2007 13:28:37 -0600 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: <473B2E88.8020402@gmail.com> References: <473B1F70.3070108@gmx.de> <473B2E88.8020402@gmail.com> Message-ID: <473B4C65.5090109@sun.com> Antonio Cuni wrote: > Carl Friedrich Bolz wrote: > >> - for PyPy-JVM: bindings to allow the interaction with arbitrary Java >> libraries, threading support > > Moreover, I would add to this list that the possibility to compile > python to jvm bytecode instead of python bytecode; maybe a pypy-jvm > would be usable even without it, but e.g. developing applets requires it. > > Hopefully, I'll be able to work on this (an its counterpart for .NET) in > the next months. Even just a simple backend python compiler for JVM would be a huge start, since it would cover 90% of cases people are interested in any sort of python-on-JVM. On the JRuby project we've gotten the most interest so far from folks simply wanting access to what the JVM and Java platform have to offer, and only recently from people interested in improved performance. The same could likely apply to Python if PyPy could produce a compiler backend for JVM. - Charlie From anto.cuni at gmail.com Wed Nov 14 20:45:10 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Wed, 14 Nov 2007 20:45:10 +0100 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: References: <473B1F70.3070108@gmx.de> Message-ID: <473B5046.6040705@gmail.com> Martijn Faassen wrote: >> - for PyPy-JVM: bindings to allow the interaction with arbitrary Java >> libraries, threading support > > Does this already exist for PyPy-CLR? yes, but it's more or less only a proof of concept. You can use .NET classes from Python but you can't, e.g., inherit from a .NET class and override a method in Python. Also, I suspect it to be a bit slow since it's mostly implemented at application level, though maybe this is not an issue. My personal plan is to port this feature to pypy-jvm soon or later; it will be even possible to share most of the code between the two backends, I think. ciao Anto From anto.cuni at gmail.com Wed Nov 14 20:56:38 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Wed, 14 Nov 2007 20:56:38 +0100 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: <473B4C65.5090109@sun.com> References: <473B1F70.3070108@gmx.de> <473B2E88.8020402@gmail.com> <473B4C65.5090109@sun.com> Message-ID: <473B52F6.2040009@gmail.com> Charles Oliver Nutter wrote: >> Moreover, I would add to this list that the possibility to compile >> python to jvm bytecode instead of python bytecode; maybe a pypy-jvm >> would be usable even without it, but e.g. developing applets requires >> it. >> >> Hopefully, I'll be able to work on this (an its counterpart for .NET) >> in the next months. > Even just a simple backend python compiler for JVM would be a huge > start, since it would cover 90% of cases people are interested in any > sort of python-on-JVM. On the JRuby project we've gotten the most > interest so far from folks simply wanting access to what the JVM and > Java platform have to offer, and only recently from people interested in > improved performance. The same could likely apply to Python if PyPy > could produce a compiler backend for JVM. Uhm, I'm not sure what you mean with "backend python compiler for JVM". In PyPy's terminology, backends are the last piece of the translation toolchain and they are needed to translate RPython programs. If you meant such a backend, it's already there and works very well (it can compile the whole pypy interpreter for example). If you meant a "Python to JVM bytecode compiler", that's what I was talking about too :-). I'm not sure that having it will cover 90% of people interested in pypy-jvm; to fully exploit python on jvm we need to allow access to Java classes from Python (and vice-versa), but this is unrelated to having a python-to-jvm compiler (for example, pypy-cli allows to access .NET classes without compiling python code to CLI). Also, the current plan for having a Python-to-jvm compiler is not to write it by hand, but to automatically generating it by reusing the same techniques we use for JIT, i.e. by partially evaluating the main interpreter loop assuming the bytecode as a constant. This approach will have the advantages of a) being correct "by design" b) working for both pypy-jvm and pypy-cli and c) working not only for python, but also for other languages implemented in rpython. ciao Anto From ondrej at certik.cz Wed Nov 14 21:39:20 2007 From: ondrej at certik.cz (Ondrej Certik) Date: Wed, 14 Nov 2007 20:39:20 +0000 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: References: <473B1F70.3070108@gmx.de> Message-ID: <85b5c3130711141239n88f6a0drb16d842eb34765f2@mail.gmail.com> > >From previous discussions, I suspect I'm not the only lurker-fan who would > be willing to commit time to working on numericentric graph optimizations > when that becomes a worthwhile investment. There's no reason that the > mostly-fortran bits of python code shouldn't run almost as fast as fortran > after amortizing the JIT costs. Exactly. There are many people who would be willing to help with that, but they won't, until it's clear, that the main authors of pypy are going to make sure pypy is stable enough and support this. For now, I use Cython: http://cython.org/ because it gives me the speed of C and it's a subset of Python, in a similar spirit to RPython. The pypy's way is very clean and better, but unfortunately, pypy is not ready for a production use and I need something that I can use now. And the authors of Cython make sure that it works. Now. Ondrej From tjreedy at udel.edu Thu Nov 15 00:50:46 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 14 Nov 2007 18:50:46 -0500 Subject: [pypy-dev] pypy on Tim Bray's blog References: <200711131523.lADFNoWO025693@theraft.openend.se> <200711140353.18810.jacob@openend.se> <200711141224.lAECOfar016106@theraft.openend.se> Message-ID: "Laura Creighton" wrote in message news:200711141224.lAECOfar016106 at theraft.openend.se... My reading of Bray's post is that it was cautiously positive -- about as much as you could expect in public comments from someone *considering* funding some work. The first comment about 'vibrant community' is already responded to. You should focus on private negotiations with Bray or any other prospect. |Why aren't you cheering? How about Go go PyPy team. Make real our Python dream. ? More seriously: as a Python advocate, I would prefer direct work on Python. But I really hope you can get funding rather than disband. So let us hope you get a choice of funding offers. tjr From faassen at startifact.com Thu Nov 15 02:02:05 2007 From: faassen at startifact.com (Martijn Faassen) Date: Thu, 15 Nov 2007 02:02:05 +0100 Subject: [pypy-dev] another pypy blog entry from sun In-Reply-To: <200711141528.24474.stephan.diehl@gmx.net> References: <200711141528.24474.stephan.diehl@gmx.net> Message-ID: Stephan Diehl wrote: > just in case you don't read programming.reddit.com: > there is another blog enty from somebody at Sun (John Rose) > http://blogs.sun.com/jrose/entry/a_day_with_pypy It's also interesting to read the comments on Reddit: http://programming.reddit.com/info/60jlu/comments/ It's just a couple of comments, and the first one is quite positive, but it also shows some of the perceptions this project has around the net as not being very practical: "Good to see PyPy getting some air time. For language implementation enthusiasts, it's truly worth looking into; their on-the-fly partial evaluation is a significant step, both conceptually and practically, beyond the techniques pioneered by SELF. I'm losing hope that it'll ever be much more than a research project, though, in spite of how far it's gotten. Hopefully I'll be proven wrong!" And the response: "Indeed I would like to know if we can expect anything that we can use outside of pypy any time soon." More random comments of course, but I'll use them gladly to support the point I'm trying to make. :) Regards, Martijn From fijal at genesilico.pl Thu Nov 15 02:16:20 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Thu, 15 Nov 2007 02:16:20 +0100 Subject: [pypy-dev] Pypy and app-level ctypes Message-ID: <473B9DE4.5000204@genesilico.pl> As a discussion regarding pypy being production-ready heats a bit, I've go a practical question: How useful would be to have app-level ctypes for pypy? Another question how less useful it would be if it won't provide python-api part (which would be as easy to provide as C-level bindings for CPython I think)? Also how much helps situation the fact that we have ctypes_platform thingie, which would help you to take away platform dependencies (ie will ask gcc for correct sizes of certain types, structures, #defs etc.) Cheers, fijal :. From faassen at startifact.com Thu Nov 15 02:39:07 2007 From: faassen at startifact.com (Martijn Faassen) Date: Thu, 15 Nov 2007 02:39:07 +0100 Subject: [pypy-dev] Pypy and app-level ctypes In-Reply-To: <473B9DE4.5000204@genesilico.pl> References: <473B9DE4.5000204@genesilico.pl> Message-ID: Maciek Fijalkowski wrote: > As a discussion regarding pypy being production-ready heats a bit, I've > go a practical question: > > How useful would be to have app-level ctypes for pypy? Another question > how less useful it would be if it won't provide python-api part (which > would be as easy to provide as C-level bindings for CPython I think)? > Also how much helps situation the fact that we have ctypes_platform > thingie, which would help you to take away platform dependencies (ie > will ask gcc for correct sizes of certain types, structures, #defs etc.) If you're asking people like me, I'm afraid don't understand the details enough to answer that one. Could you elaborate a bit more (if indeed you are addressing people like me)? I think I understand you're asking whether it'd be useful if you could write ctypes-based software in software being run by the PyPy interpreter, but I don't know what you mean by not providing the python-api part. Regards, Martijn From fijal at genesilico.pl Thu Nov 15 02:51:54 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Thu, 15 Nov 2007 02:51:54 +0100 Subject: [pypy-dev] Pypy and app-level ctypes In-Reply-To: References: <473B9DE4.5000204@genesilico.pl> Message-ID: <473BA63A.1000308@genesilico.pl> Martijn Faassen wrote: > Maciek Fijalkowski wrote: > >> As a discussion regarding pypy being production-ready heats a bit, I've >> go a practical question: >> >> How useful would be to have app-level ctypes for pypy? Another question >> how less useful it would be if it won't provide python-api part (which >> would be as easy to provide as C-level bindings for CPython I think)? >> Also how much helps situation the fact that we have ctypes_platform >> thingie, which would help you to take away platform dependencies (ie >> will ask gcc for correct sizes of certain types, structures, #defs etc.) >> > > If you're asking people like me, I'm afraid don't understand the details > enough to answer that one. Could you elaborate a bit more (if indeed you > are addressing people like me)? > > I think I understand you're asking whether it'd be useful if you could > write ctypes-based software in software being run by the PyPy > interpreter, but I don't know what you mean by not providing the > python-api part. > > Regards, > > Martijn > Sorry for not elaborating on this. Yes, the question is about whole usefulness of ctypes being in PyPy (there are mixed opinions I think). Ctypes provide python api, which is just ctypes mapping of a CPython C structures presented in a convinient way. People seem to use it, and we won't provide this part for sure. My question is whether it still present some value (I assume yes, because there is ctypes-based pygame, which I use on my day-to-day activities for a while now and didn't notice any problems). Cheers, fijal :. From david at ar.media.kyoto-u.ac.jp Thu Nov 15 06:09:26 2007 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Thu, 15 Nov 2007 14:09:26 +0900 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: References: <473B1F70.3070108@gmx.de> Message-ID: <473BD486.6030806@ar.media.kyoto-u.ac.jp> Martijn Faassen wrote: > > Yes, this one would be a major challenge. If you can crack it and the > PyPy interpreter offers another benefit (an obvious one is speed), you > will be in awesome position. If not, the other benefits will have to > weigh more strongly. This is definitely one to evolve slowly over time > if it's possible at all. I don't understand the C++ part. How many useful python modules use C++ ? For a start, my understanding is that python itself (interprter + stdlib) do not use C++ at all. Wrapping C++ in any language other than itself is a pain anyway :) I am personally interested in two things: being able to use any pure python extension, and numerical code (I am a heavy user and sometimes contributor to numpy/scipy). I don't think I am alone in this niche. For example, someone on the numpy-discussion list asked about being able to use numpy from iron python: http://groups.google.com/group/c-extensions-for-ironpython/browse_thread/thread/88102263f8586fd0 what can be done about that ? Since I don't understand how pypy can use/intend to use c-based extensions (I thought ctypes was the way, but it looks like I misunderstood things), I am not sure what the possibilities are *now*. cheers, David From amauryfa at gmail.com Thu Nov 15 10:21:22 2007 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 15 Nov 2007 10:21:22 +0100 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: <473BD486.6030806@ar.media.kyoto-u.ac.jp> References: <473B1F70.3070108@gmx.de> <473BD486.6030806@ar.media.kyoto-u.ac.jp> Message-ID: David Cournapeau wrote: > Martijn Faassen wrote: > > > > Yes, this one would be a major challenge. If you can crack it and the > > PyPy interpreter offers another benefit (an obvious one is speed), you > > will be in awesome position. If not, the other benefits will have to > > weigh more strongly. This is definitely one to evolve slowly over time > > if it's possible at all. > I don't understand the C++ part. How many useful python modules use C++ > ? For a start, my understanding is that python itself (interprter + > stdlib) do not use C++ at all. Well, each time you have an interesting C++ library, you have an interesting python module written in C++. Two kinds of modules come to my mind: - wxPython is a good wrapper around the big C++ wxWidgets library. A cross-platform GUI, 500 classes, 6000 methods are exposed. Of course the underlying GTK and Win32 functions are C calls, but do you want to port wxWidgets to rpython? - I like to test and automate my own C++ applications with python... > Wrapping C++ in any language other than > itself is a pain anyway :) Indeed. But there are tools to generate most of the painful code: Swig, boost::python, and others. With Swig, you can even subclass C++ classes and override virtual methods in python. I wish pypy could do the same. If I understand correctly, pypy now uses its own "rffi" mechanism (for "RPython foreign function call"?) to access external C functions. rffi itself does not use ctypes: when translating a function call, it is enough to write the C function call, and link with the correct library. ctypes is used only when executing external functions on top of CPython: in tests, or in the pypy.py interpreter. We could do it differently, for example by quickly compiling an extension module for each function we need. It could be useful in some cases when ctypes cannot be used. For example when the C "function" is actually a macro, or a C++ function. Should we extend rffi to handle C++? -- Amaury Forgeot d'Arc From faassen at startifact.com Thu Nov 15 13:23:07 2007 From: faassen at startifact.com (Martijn Faassen) Date: Thu, 15 Nov 2007 13:23:07 +0100 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: <473BD486.6030806@ar.media.kyoto-u.ac.jp> References: <473B1F70.3070108@gmx.de> <473BD486.6030806@ar.media.kyoto-u.ac.jp> Message-ID: David Cournapeau wrote: > Martijn Faassen wrote: >> Yes, this one would be a major challenge. If you can crack it and the >> PyPy interpreter offers another benefit (an obvious one is speed), you >> will be in awesome position. If not, the other benefits will have to >> weigh more strongly. This is definitely one to evolve slowly over time >> if it's possible at all. > I don't understand the C++ part. How many useful python modules use C++ > ? For a start, my understanding is that python itself (interprter + > stdlib) do not use C++ at all. Wrapping C++ in any language other than > itself is a pain anyway :) I took "wrapping a major C++ library using ctypes" as an example of the complexities involved. If PyPy *doesn't* offer a way to reuse existing CPython extensions, this means it'll have to be done again. As someone who started with lxml, I can tell you that wrapping a major C library with Pyrex properly is also a lot of hard work - making garbage collection work as a Python programmer expects is a huge challenge. Doing that again for PyPy would feel like a waste of effort. > I am personally interested in two things: being able to use any pure > python extension, and numerical code (I am a heavy user and sometimes > contributor to numpy/scipy). I don't think I am alone in this niche. For > example, someone on the numpy-discussion list asked about being able to > use numpy from iron python: > > http://groups.google.com/group/c-extensions-for-ironpython/browse_thread/thread/88102263f8586fd0 > > what can be done about that ? Since I don't understand how pypy can > use/intend to use c-based extensions (I thought ctypes was the way, but > it looks like I misunderstood things), I am not sure what the > possibilities are *now*. I think the minimum PyPy will do is make it possible to wrap C-based extensions using something like ctypes. As far as I understand that work hasn't been completed yet. The original post was also talking about the possible maximum, which is to support arbitrary third-party extensions that bind to non-python libraries, such as NumPy, pygame and lxml. Regards, Martijn From ndbecker2 at gmail.com Thu Nov 15 14:06:33 2007 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 15 Nov 2007 08:06:33 -0500 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) References: <473B1F70.3070108@gmx.de> <473BD486.6030806@ar.media.kyoto-u.ac.jp> Message-ID: David Cournapeau wrote: > Martijn Faassen wrote: >> >> Yes, this one would be a major challenge. If you can crack it and the >> PyPy interpreter offers another benefit (an obvious one is speed), you >> will be in awesome position. If not, the other benefits will have to >> weigh more strongly. This is definitely one to evolve slowly over time >> if it's possible at all. > I don't understand the C++ part. How many useful python modules use C++ > ? For a start, my understanding is that python itself (interprter + > stdlib) do not use C++ at all. Wrapping C++ in any language other than > itself is a pain anyway :) > I use this extensively. I have a large library of signal processing that is written in c++ using boost-python. All the c++ code is written in generic style, with concrete instantiations exposed to python. From lac at openend.se Thu Nov 15 14:49:25 2007 From: lac at openend.se (Laura Creighton) Date: Thu, 15 Nov 2007 14:49:25 +0100 Subject: [pypy-dev] Do we want to do this? Message-ID: <200711151349.lAFDnPSY016038@theraft.openend.se> http://www.fosdem.org/2008/call_for_devrooms "FOSDEM will take place on 23 and 24 February 2008, and we provide so-called Developer Rooms ("devrooms") as well as Stands (booths) to Free and Opensource projects. For every devroom we invite a project to organize talks, open discussions and/or hacking sessions. We offer an environment for developer collaboration, but we have only a limited number of rooms at our disposal. " Something for FOSS projects? All Stand or Developer room request must be sent in *before Monday 2007-11-26, 23:59 CET*. ----------------- Looks like it could be fun Laura From lac at openend.se Thu Nov 15 19:02:53 2007 From: lac at openend.se (Laura Creighton) Date: Thu, 15 Nov 2007 19:02:53 +0100 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: Message from Douglas McNeil of "Wed, 14 Nov 2007 17:40:54 GMT." References: <473B1F70.3070108@gmx.de> Message-ID: <200711151802.lAFI2rG3029422@theraft.openend.se> In a message of Wed, 14 Nov 2007 17:40:54 GMT, Douglas McNeil writes: > > >Arbitrary code is less interesting to me than JIT-powered fast numerical >code. Moreover, we numerics types have much lower standards of >"production ready" than the general public, and are willing to turn on >options with names like --make-dangerous-assumptions-about-code-for-speed >-do-not-use-this-flag-really-do-not-use-it-i-warned-you. > I have this uncontrollable urge to make these options for you. :-) thanks for brightening an already pretty bright day. Laura From bea at changemaker.nu Thu Nov 15 22:12:21 2007 From: bea at changemaker.nu (=?ISO-8859-1?Q?Beatrice_D=FCring?=) Date: Thu, 15 Nov 2007 22:12:21 +0100 Subject: [pypy-dev] another pypy blog entry from sun In-Reply-To: References: <200711141528.24474.stephan.diehl@gmx.net> Message-ID: <473CB635.3010707@changemaker.nu> Hi there Martijn Faassen skrev: > Stephan Diehl wrote: > >> just in case you don't read programming.reddit.com: >> there is another blog enty from somebody at Sun (John Rose) >> http://blogs.sun.com/jrose/entry/a_day_with_pypy >> > > It's also interesting to read the comments on Reddit: > > http://programming.reddit.com/info/60jlu/comments/ > > > More random comments of course, but I'll use them gladly to support the > point I'm trying to make. :) > > Regards, > > Martijn > I said it at Europython in Vilnius and I will say it again (as some of the others in the dev-team have done in prior emails): - I _really_ appreciate the time and effort you spend in communicating with (us) pypy developers regarding the focus and future of the project. I personally feel that your perspective is a crucial one that we seriously need to take into account, we are standing at the crossroads now so to say. Again - thanks for sharing your thoughts, in some sense I think you are one of our biggest PyPy fans ;-) Cheers Bea D?ring -- Beatrice D?ring, PMP Change Maker J?rntorget 3 413 04 Gothenburg www.changemaker.nu email: bea at changemaker.nu Phone: +46 31 7750940 Cellphone: +46 734 22 89 06 PyPy: www.pypy.org From faassen at startifact.com Fri Nov 16 00:39:21 2007 From: faassen at startifact.com (Martijn Faassen) Date: Fri, 16 Nov 2007 00:39:21 +0100 Subject: [pypy-dev] more problems building PyPy-JVM Message-ID: Hi there, To start off, please note I haven't worked with Java in the past. I'm the proper kind of newbie to try out PyPy-JVM. :) I now get build process process to complete, but then I run into a new problem. Apparently it needs something called 'jasmin'. It was in my Ubuntu Feisty, so I installed it. Now when I run it again, I get: translation:info] Wrote JVM code [translation:info] Compiling JVM source... Invoking jasmin on ['/tmp/usession-1/pypy/pypy/InterlinkImplementation_6.j', '/tmp/usession-1/pypy/pypy/throwUnicodeDecodeError_1.j', '/tmp/usession-1/pypy/pypy/exceptions/ValueError_10.j', '/tmp/usession-1/pypy/pypy/Object_14.j', '/tmp/usession-1/pypy/pypy/exceptions/UnicodeDecodeError_meta_18.j', '/tmp/usession-1/pypy/pypy/exceptions/UnicodeError_meta_19.j', '/tmp/usession-1/pypy/pypy/exceptions/UnicodeDecodeError_8.j', '/tmp/usession-1/pypy/pypy/exceptions/Exception_12.j', '/tmp/usession-1/pypy/pypy/Object_meta_16.j', '/tmp/usession-1/pypy/pypy/ll_raise_OSError__Signed_5.j', '/tmp/usession-1/pypy/pypy/exceptions/EnvironmentError_meta_30.j', '/tmp/usession-1/pypy/pypy/throwZeroDivisionError_4.j', '/tmp/usession-1/pypy/pypy/exceptions/EnvironmentError_27.j', '/tmp/usession-1/pypy/pypy/throwValueError_0.j', '/tmp/usession-1/pypy/pypy/exceptions/ValueError_meta_20.j', '/tmp/usession-1/pypy/pypy/Object_15.j', '/tmp/usession-1/pypy/pypy/throwOverflowError_3.j', '/tmp/usession-1/pypy/pypy/throwIndexError_2.j', '/tmp/usession-1/pypy/pypy/exceptions/OSError_26.j', '/tmp/usession-1/pypy/pypy/Object_13.j', '/tmp/usession-1/pypy/pypy/exceptions/OSError_meta_29.j', '/tmp/usession-1/pypy/pypy/exceptions/OverflowError_meta_45.j', '/tmp/usession-1/pypy/pypy/exceptions/Exception_meta_22.j', '/tmp/usession-1/pypy/pypy/exceptions/StandardError_meta_21.j', '/tmp/usession-1/pypy/pypy/exceptions/UnicodeError_9.j', '/tmp/usession-1/pypy/pypy/Main.j', '/tmp/usession-1/pypy/pypy/exceptions/StandardError_11.j', '/tmp/usession-1/pypy/pypy/exceptions/LookupError_meta_52.j'] So far so good, I think... Then however: [Timer] Timings: [Timer] annotate - 329.6 s [Timer] rtype_ootype - 104.8 s [Timer] backendopt_ootype - 77.2 s [Timer] source_jvm - 96.8 s [Timer] compile_jvm - 2.0 s [Timer] ======================================== [Timer] Total: - 610.4 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 273, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/driver.py", line 810, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/tool/taskengine.py", line 112, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/driver.py", line 281, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/driver.py", line 746, in task_compile_jvm [translation:ERROR] self.jvmsource.compile() [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/jvm/genjvm.py", line 165, in compile [translation:ERROR] self._invoke(jascmd + files, False) [translation:ERROR] File "/home/faassen/install/pypy-dist/pypy/translator/jvm/genjvm.py", line 107, in _invoke [translation:ERROR] raise JvmSubprogramError(res, args, stdout, stderr) [translation:ERROR] JvmSubprogramError: Error code 0 running ['jasmin', '-d', '/tmp/usession-1/pypy', '/tmp/usession-1/pypy/pypy/InterlinkImplementation_6.j', '/tmp/usession-1/pypy/pypy/throwUnicodeDecodeError_1.j', '/tmp/usession-1/pypy/pypy/exceptions/ValueError_10.j', '/tmp/usession-1/pypy/pypy/Object_14.j', '/tmp/usession-1/pypy/pypy/exceptions/UnicodeDecodeError_meta_18.j', '/tmp/usession-1/pypy/pypy/exceptions/UnicodeError_meta_19.j', '/tmp/usession-1/pypy/pypy/exceptions/UnicodeDecodeError_8.j', '/tmp/usession-1/pypy/pypy/exceptions/Exception_12.j', '/tmp/usession-1/pypy/pypy/Object_meta_16.j', '/tmp/usession-1/pypy/pypy/ll_raise_OSError__Signed_5.j', '/tmp/usession-1/pypy/pypy/exceptions/EnvironmentError_meta_30.j', '/tmp/usession-1/pypy/pypy/throwZeroDivisionError_4.j', '/tmp/usession-1/pypy/pypy/exceptions/EnvironmentError_27.j', '/tmp/usession-1/pypy/pypy/throwValueError_0.j', '/tmp/usession-1/pypy/pypy/exceptions/ValueError_meta_20.j', '/tmp/usession-1/pypy/pypy/Object_15.j', '/tmp/usession-1/pypy/pypy/throwOverflowError_3.j', '/tmp/usession-1/pypy/pypy/throwIndexError_2.j', '/tmp/usession-1/pypy/pypy/exceptions/OSError_26.j', '/tmp/usession-1/pypy/pypy/Object_13.j', '/tmp/usession-1/pypy/pypy/exceptions/OSError_meta_29.j', '/tmp/usession-1/pypy/pypy/exceptions/OverflowError_meta_45.j', '/tmp/usession-1/pypy/pypy/exceptions/Exception_meta_22.j', '/tmp/usession-1/pypy/pypy/exceptions/StandardError_meta_21.j', '/tmp/usession-1/pypy/pypy/exceptions/UnicodeError_9.j', '/tmp/usession-1/pypy/pypy/Main.j', '/tmp/usession-1/pypy/pypy/exceptions/StandardError_11.j', '/tmp/usession-1/pypy/pypy/exceptions/LookupError_meta_52.j'] Do I need a different version of jasmin? 'jasmin -version' reports: Jasmin version: v sable-1.2 Also for my next step, assuming I get it to work after this, what'd I need do to start it up to run a script or see a Python prompt? The pypy docs on the website still have the JVM project being in an earlier stage than it really is, so it doesn't really say yet. Regards, Martijn From faassen at startifact.com Fri Nov 16 01:28:06 2007 From: faassen at startifact.com (Martijn Faassen) Date: Fri, 16 Nov 2007 01:28:06 +0100 Subject: [pypy-dev] another pypy blog entry from sun In-Reply-To: <473CB635.3010707@changemaker.nu> References: <200711141528.24474.stephan.diehl@gmx.net> <473CB635.3010707@changemaker.nu> Message-ID: Hey, Beatrice D?ring wrote: [snip] > - I _really_ appreciate the time and effort you spend in communicating > with (us) pypy developers regarding the focus and future of the project. I personally > feel that your perspective is a crucial one that we seriously need to > take into account, we are standing at the crossroads now so to say. Thanks Bea, and also the others. I will continue speaking out whenever I feel I need to then. :) PyPy also seems to be getting at a stage where I can afford to start to play with it more (plus I have more time to make for it), so I am slowly starting to putting some actions in where my words are. The goal: to make PyPy safe for people like me. I don't promise to contribute much there yet, but it's a start. > Again - thanks for sharing your thoughts, in some sense I think you > are one of our biggest PyPy fans ;-) Sure! Though if a fan is someone who can't constructively criticize, then I'd prefer the word 'friend'. Regards, Martijn From david at ar.media.kyoto-u.ac.jp Fri Nov 16 05:08:27 2007 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 16 Nov 2007 13:08:27 +0900 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: References: <473B1F70.3070108@gmx.de> <473BD486.6030806@ar.media.kyoto-u.ac.jp> Message-ID: <473D17BB.1070704@ar.media.kyoto-u.ac.jp> Amaury Forgeot d'Arc wrote: > David Cournapeau wrote: >> Martijn Faassen wrote: >>> Yes, this one would be a major challenge. If you can crack it and the >>> PyPy interpreter offers another benefit (an obvious one is speed), you >>> will be in awesome position. If not, the other benefits will have to >>> weigh more strongly. This is definitely one to evolve slowly over time >>> if it's possible at all. >> I don't understand the C++ part. How many useful python modules use C++ >> ? For a start, my understanding is that python itself (interprter + >> stdlib) do not use C++ at all. > > Well, each time you have an interesting C++ library, you have an > interesting python module written in C++. > Two kinds of modules come to my mind: > - wxPython is a good wrapper around the big C++ wxWidgets library. A > cross-platform GUI, > 500 classes, 6000 methods are exposed. Of course the underlying GTK > and Win32 functions are C calls, but do you want to port wxWidgets to > rpython? > - I like to test and automate my own C++ applications with python... I certainly did not mean that there was no interesting C++ library to wrap. But in the context of "making pypy usable for production use", I am not sure this is a priority. To make it short: I can imagine a python implementation with the stdlib to be of some use, and this does not involve C++. > > Indeed. But there are tools to generate most of the painful code: > Swig, boost::python, and others. > With Swig, you can even subclass C++ classes and override virtual > methods in python. > I wish pypy could do the same. Both methods involve parsing C++, which is not fun. For example, concerning wxwidget, I for once think it would be much easier to wrap gtk than wxwidget (if we think: would be good to have a GUI lib for pypy); incidently, that's how pygtk is generated (some python + scheme code generate the pygtk wrapper around gtk). ctypes can use codegen to automatically wrap C libraries: I have not used it extensively myself, but it is used by some people to wrap non trivial C libraries. > > If I understand correctly, pypy now uses its own "rffi" mechanism (for > "RPython foreign function call"?) to access external C functions. > rffi itself does not use ctypes: when translating a function call, it > is enough to write the C function call, and link with the correct > library. ctypes is used only when executing external functions on top > of CPython: in tests, or in the pypy.py interpreter. > We could do it differently, for example by quickly compiling an > extension module for each function we need. > It could be useful in some cases when ctypes cannot be used. For > example when the C "function" is actually a macro, or a C++ function. > > Should we extend rffi to handle C++? ctypes does not handle C++ for strong (e.g. non trivial) reasons. I mean everybody knows that C++ and C are totally different in this respect. C is easy to use from other languages, C++ is next to impossible. Extending rffi to handle C++ is an enormous task, no ? Again, in the context of making pypy 'production-usable', is it really a priority ? Can't we first do things which do not require C++ support ? David From anto.cuni at gmail.com Fri Nov 16 10:11:47 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 16 Nov 2007 10:11:47 +0100 Subject: [pypy-dev] more problems building PyPy-JVM In-Reply-To: References: Message-ID: <473D5ED3.3070303@gmail.com> Martijn Faassen wrote: > Hi there, Hi Martijn, > To start off, please note I haven't worked with Java in the past. I'm > the proper kind of newbie to try out PyPy-JVM. :) [cut] > Do I need a different version of jasmin? 'jasmin -version' reports: > > Jasmin version: v sable-1.2 Yes, it looks like you need another version of jasmin; the mine is this: $ jasmin -version Jasmin version: v2.2 I think you can download it from here: http://jasmin.sourceforge.net/ I honestly don't know what "sable" means in you version string. > Also for my next step, assuming I get it to work after this, what'd I > need do to start it up to run a script or see a Python prompt? The pypy > docs on the website still have the JVM project being in an earlier stage > than it really is, so it doesn't really say yet. Ah, good point, thanks; I suppose I could update the docs to explain how to install everything needed and how to translate pypy-jvm. Anyway, after the compilation you can find your pypy-jvm.jar inside the translator/goal directory; to run it simply type: $ java -jar pypy-jvm.jar Together with the jar file we create also a script called pypy-jvm that invokes java -jar, so if you are on unix you can also type: $ ./pypy-jvm ciao Anto From pfraser at spatialmedia.com Fri Nov 16 10:53:39 2007 From: pfraser at spatialmedia.com (Peter Fraser) Date: Fri, 16 Nov 2007 22:53:39 +1300 Subject: [pypy-dev] naive VM question Message-ID: <473D68A3.9080908@spatialmedia.com> I have to assume the following is either not useful, not practical, or both -but I have to ask... Version I. It seems to me that PyPy might be useful as a kind of "Co-Python" -viz. the PyPy VM running as a CPython extension, as a way to utilise a second core from a single CPython process. I'm realistic about the level of integration that could be achieved, -but even something that amounted to (fast) RPC inside a process might be useful given the second vm was arguably "free for the using" on a multicore system. If this was in fact useful, it might be an on-ramp into the Python mainstream for PyPy. PyPy needs on-ramps imho. Version II. Is "multiple isolated (but bridged) VM's in a single process" a possible general technique? If multiple cooperating processes is a reasonable response to the "single threaded VM meets multi-core architecture" problem -then is "multiple VM's in a single process" a better one? I'm talking about an ugly, brute force alternative to a multi threaded VM. Viz. simply duplicating / munging / mangling / tagging as much of the interpreter code as was necessary to allow say two (but not n) threads to safely run at once. 2 heaps, 2 GILS and any given thread only ever running against one of these "sub vm's". Then perhaps add explicitly managed opportunities for memory sharing and other integration strategies betwen the two. It strikes me as something that you wouldn't want to attempt in C, but might be possible in the PyPy translation. "Dual-core Python" .."Hydra" -or is this just crazy talk? Pete F From ndbecker2 at gmail.com Fri Nov 16 12:49:43 2007 From: ndbecker2 at gmail.com (Neal Becker) Date: Fri, 16 Nov 2007 06:49:43 -0500 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) References: <473B1F70.3070108@gmx.de> <473BD486.6030806@ar.media.kyoto-u.ac.jp> <473D17BB.1070704@ar.media.kyoto-u.ac.jp> Message-ID: [...] >> Indeed. But there are tools to generate most of the painful code: >> Swig, boost::python, and others. >> With Swig, you can even subclass C++ classes and override virtual >> methods in python. >> I wish pypy could do the same. > Both methods involve parsing C++, which is not fun. For example, > concerning wxwidget, I for once think it would be much easier to wrap > gtk than wxwidget (if we think: would be good to have a GUI lib for > pypy); incidently, that's how pygtk is generated (some python + scheme > code generate the pygtk wrapper around gtk). > boost::python does not involve parsing c++. From cfbolz at gmx.de Fri Nov 16 12:52:17 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 16 Nov 2007 12:52:17 +0100 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: References: <473B1F70.3070108@gmx.de> <473BD486.6030806@ar.media.kyoto-u.ac.jp> <473D17BB.1070704@ar.media.kyoto-u.ac.jp> Message-ID: <473D8471.3090503@gmx.de> Neal Becker wrote: > [...] >>> Indeed. But there are tools to generate most of the painful code: >>> Swig, boost::python, and others. >>> With Swig, you can even subclass C++ classes and override virtual >>> methods in python. >>> I wish pypy could do the same. >> Both methods involve parsing C++, which is not fun. For example, >> concerning wxwidget, I for once think it would be much easier to wrap >> gtk than wxwidget (if we think: would be good to have a GUI lib for >> pypy); incidently, that's how pygtk is generated (some python + scheme >> code generate the pygtk wrapper around gtk). >> > > boost::python does not involve parsing c++. yip, it does. The compiler parses C++, after all :-). nitpickingly-yours, Carl Friedrich From david at ar.media.kyoto-u.ac.jp Fri Nov 16 12:50:07 2007 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Fri, 16 Nov 2007 20:50:07 +0900 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: <473D8471.3090503@gmx.de> References: <473B1F70.3070108@gmx.de> <473BD486.6030806@ar.media.kyoto-u.ac.jp> <473D17BB.1070704@ar.media.kyoto-u.ac.jp> <473D8471.3090503@gmx.de> Message-ID: <473D83EF.1040703@ar.media.kyoto-u.ac.jp> Carl Friedrich Bolz wrote: > Neal Becker wrote: >> [...] >>>> Indeed. But there are tools to generate most of the painful code: >>>> Swig, boost::python, and others. >>>> With Swig, you can even subclass C++ classes and override virtual >>>> methods in python. >>>> I wish pypy could do the same. >>> Both methods involve parsing C++, which is not fun. For example, >>> concerning wxwidget, I for once think it would be much easier to wrap >>> gtk than wxwidget (if we think: would be good to have a GUI lib for >>> pypy); incidently, that's how pygtk is generated (some python + scheme >>> code generate the pygtk wrapper around gtk). >>> >> boost::python does not involve parsing c++. > > yip, it does. The compiler parses C++, after all :-). > > nitpickingly-yours, You are not nitpicking at all: boost::python uses Py++, which uses gccxml to parse the C++. There is no way around it: since it is extremely difficult to interoperate with any C++ ABI, you need to interoperate at the source level, parsing the monstruosity that C++ is compared to C. David From 2007b at usenet.alexanderweb.de Fri Nov 16 14:29:32 2007 From: 2007b at usenet.alexanderweb.de (Alexander Schremmer) Date: Fri, 16 Nov 2007 14:29:32 +0100 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) References: <473B1F70.3070108@gmx.de> <473B3C86.2090609@gmail.com> <693bc9ab0711141105s29c07897o65eef87450cdedbc@mail.gmail.com> Message-ID: Hi, Maciej Fijalkowski wrote: > But besides, I fully agree to the BIG FAT WARNING. This code is mostly > there for tests. Even strings should not cross this border Hmm, it seems quite stable, though (besides a few bug in the exception handling). Note that passing and receiving strings works fine (there . It can be used to speed up string based algorithms quite heavily by moving it into such an RPython function. Kind regards, Alexander From bborcic at gmail.com Fri Nov 16 15:28:39 2007 From: bborcic at gmail.com (Boris Borcic) Date: Fri, 16 Nov 2007 15:28:39 +0100 Subject: [pypy-dev] naive VM question In-Reply-To: <473D68A3.9080908@spatialmedia.com> References: <473D68A3.9080908@spatialmedia.com> Message-ID: Peter Fraser wrote: > -or is this just crazy talk? Very confused at many levels. Putting aside the relevance to pypy, you seem to confuse VM designs and VM instances, in the sense that for some mysterious reason you expect VM instances to play together fantastically better if they are of different designs. But AFAIK, identical twins tend to sync rather better than arbitrary brothers or sisters... From ndbecker2 at gmail.com Fri Nov 16 15:41:12 2007 From: ndbecker2 at gmail.com (Neal Becker) Date: Fri, 16 Nov 2007 09:41:12 -0500 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) References: <473B1F70.3070108@gmx.de> <473BD486.6030806@ar.media.kyoto-u.ac.jp> <473D17BB.1070704@ar.media.kyoto-u.ac.jp> <473D8471.3090503@gmx.de> <473D83EF.1040703@ar.media.kyoto-u.ac.jp> Message-ID: David Cournapeau wrote: > Carl Friedrich Bolz wrote: >> Neal Becker wrote: >>> [...] >>>>> Indeed. But there are tools to generate most of the painful code: >>>>> Swig, boost::python, and others. >>>>> With Swig, you can even subclass C++ classes and override virtual >>>>> methods in python. >>>>> I wish pypy could do the same. >>>> Both methods involve parsing C++, which is not fun. For example, >>>> concerning wxwidget, I for once think it would be much easier to wrap >>>> gtk than wxwidget (if we think: would be good to have a GUI lib for >>>> pypy); incidently, that's how pygtk is generated (some python + scheme >>>> code generate the pygtk wrapper around gtk). >>>> >>> boost::python does not involve parsing c++. >> >> yip, it does. The compiler parses C++, after all :-). >> >> nitpickingly-yours, > You are not nitpicking at all: boost::python uses Py++, which uses > gccxml to parse the C++. There is no way around it: since it is > extremely difficult to interoperate with any C++ ABI, you need to > interoperate at the source level, parsing the monstruosity that C++ is > compared to C. > No. You can use py++ if you like to generate boost::python. This (py++) is fairly new, and I believe relatively few boost::python users follow this route. Most of us write boost::python wrappers by hand (it's usually pretty easy). From mithrandi at mithrandi.za.net Fri Nov 16 16:48:30 2007 From: mithrandi at mithrandi.za.net (Tristan Seligmann) Date: Fri, 16 Nov 2007 17:48:30 +0200 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: References: <473B1F70.3070108@gmx.de> <473BD486.6030806@ar.media.kyoto-u.ac.jp> <473D17BB.1070704@ar.media.kyoto-u.ac.jp> <473D8471.3090503@gmx.de> <473D83EF.1040703@ar.media.kyoto-u.ac.jp> Message-ID: <20071116154830.GF2775@mithrandi.za.net> * Neal Becker [2007-11-16 09:41:12 -0500]: > No. You can use py++ if you like to generate boost::python. This (py++) is > fairly new, and I believe relatively few boost::python users follow this > route. Most of us write boost::python wrappers by hand (it's usually > pretty easy). It may be easy, but it's certainly extremely tedious work. In any event, there's not that much difference between running a C++ compiler over the code, and running a C++ parser over the code. Both of these actions are somewhat impractical to do on the fly, which is unfortunate. -- mithrandi, i Ainil en-Balandor, a faer Ambar -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: Digital signature URL: From mithrandi at mithrandi.za.net Fri Nov 16 16:50:28 2007 From: mithrandi at mithrandi.za.net (Tristan Seligmann) Date: Fri, 16 Nov 2007 17:50:28 +0200 Subject: [pypy-dev] naive VM question In-Reply-To: <473D68A3.9080908@spatialmedia.com> References: <473D68A3.9080908@spatialmedia.com> Message-ID: <20071116155028.GG2775@mithrandi.za.net> * Peter Fraser [2007-11-16 22:53:39 +1300]: > It seems to me that PyPy might be useful as a kind of "Co-Python" -viz. > the PyPy VM running as a CPython extension, as a way to utilise a second > core from a single CPython process. > > I'm realistic about the level of integration that could be achieved, > -but even something that amounted to (fast) RPC inside a process might > be useful given the second vm was arguably "free for the using" on a > multicore system. Why not just run two CPython VMs, instead of one CPython VM and one PyPy VM? > If multiple cooperating processes is a reasonable response to the > "single threaded VM meets multi-core architecture" problem -then is > "multiple VM's in a single process" a better one? If you're running multiple VMs, and you want to take advantage of multi-core architecture, running them in multiple processes seems like a much better idea than running them in separate threads in the same process. -- mithrandi, i Ainil en-Balandor, a faer Ambar -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: Digital signature URL: From exarkun at divmod.com Fri Nov 16 18:13:04 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Fri, 16 Nov 2007 12:13:04 -0500 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: <473B1F70.3070108@gmx.de> Message-ID: <20071116171304.8162.375321666.divmod.quotient.36612@ohm> On Wed, 14 Nov 2007 17:16:48 +0100, Carl Friedrich Bolz wrote: >Hi all, > >I thought I would start a new thread for discussing what PyPy needs to >become "production ready" (whatever that is) and succeed as a Python >implementation. This is something were outside opinions are very welcome >(of course there is no guarantee we can implement all of your wishes :-) >). > >Obvious problems are: > - some missing extension modules of the stdlib. no clue which of them > are really important, depends on the application you want to run, I > guess. This can be worked on by outsiders. > I tried running a piece of real-world software the other day, an IRC bot which I use regularly in a number of channels, using pypy-c with boehm gc. I had to disable imports of the pwd and syslog module and add an early return to a function which would have otherwise called setuid and setgid from the os module, since none of these things are available on pypy. I filed tickets in the issue tracker for these three things. After I did that, the bot was able to start up and connect to a network, join channels and provide basic functionality. > - lack of 3rd-party extension-module support. This is hard to change. > Once we implement ctypes the problem will be mitigated, but not > solved: it's no fun at all to wrap a large C++ library with ctypes, > and there are lot's of existing non-ctypes based modules out there. Some of the functionality of the bot is unavailable on PyPy, since it relies on the third-party PyCrypto extension module. > > - Speed. The JIT is still not in a state where it really speeds up > arbitrary Python code. I expect this to change sooner or later. > However, it's not an area were a lot of people can help. The bot was slightly sluggish on PyPy, as compared to on CPython. This was mostly noticable at startup and shutdown. Network latency probably greatly overshadowed any performance difference responding to events from the IRC server. Overall, for this application, PyPy is fast enough. > > - for PyPy-JVM: bindings to allow the interaction with arbitrary Java > libraries, threading support > Humm. Nothing to say to this. :) Jean-Paul From charles.nutter at sun.com Fri Nov 16 20:14:27 2007 From: charles.nutter at sun.com (Charles Oliver Nutter) Date: Fri, 16 Nov 2007 13:14:27 -0600 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: <473B52F6.2040009@gmail.com> References: <473B1F70.3070108@gmx.de> <473B2E88.8020402@gmail.com> <473B4C65.5090109@sun.com> <473B52F6.2040009@gmail.com> Message-ID: <473DEC13.3000609@sun.com> Antonio Cuni wrote: > If you meant a "Python to JVM bytecode compiler", that's what I was > talking about too :-). > I'm not sure that having it will cover 90% of people interested in > pypy-jvm; to fully exploit python on jvm we need to allow access to Java > classes from Python (and vice-versa), but this is unrelated to having a > python-to-jvm compiler (for example, pypy-cli allows to access .NET > classes without compiling python code to CLI). Wiring in support for calling Java libraries won't be particularly difficult, given the many reflective capabilities already present for Java. But I agree it needs to be there for most people to find a lot of use. However, if PyPy on JVM can be brought to a point where it can actually run real Python apps (especially apps that would be useful in the domains where Java is useful) the integration question can be delayed a bit. Jython, for example, can't run anything. If PyPy could beat that in the short term and get something non-trivial running well, it would be a huge bonus. > Also, the current plan for having a Python-to-jvm compiler is not to > write it by hand, but to automatically generating it by reusing the same > techniques we use for JIT, i.e. by partially evaluating the main > interpreter loop assuming the bytecode as a constant. This approach will > have the advantages of a) being correct "by design" b) working for both > pypy-jvm and pypy-cli and c) working not only for python, but also for > other languages implemented in rpython. That's the work I meant, thanks for clarifying. Is this actively being worked right now? - Charlie From simon at arrowtheory.com Fri Nov 16 22:20:40 2007 From: simon at arrowtheory.com (Simon Burton) Date: Fri, 16 Nov 2007 13:20:40 -0800 Subject: [pypy-dev] pycon 2008 Message-ID: <20071116132040.e41ebc03.simon@arrowtheory.com> Is anyone planning on going ? doing a talk ? Simon. From tismer at stackless.com Sat Nov 17 02:46:19 2007 From: tismer at stackless.com (Christian Tismer) Date: Sat, 17 Nov 2007 02:46:19 +0100 Subject: [pypy-dev] pycon 2008 In-Reply-To: <20071116132040.e41ebc03.simon@arrowtheory.com> References: <20071116132040.e41ebc03.simon@arrowtheory.com> Message-ID: <473E47EB.4070004@stackless.com> Simon Burton wrote: > Is anyone planning on going ? doing a talk ? I will, probably. Talking more & better about Stackless PyPy. cheers -- chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From anto.cuni at gmail.com Sat Nov 17 10:46:23 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Sat, 17 Nov 2007 10:46:23 +0100 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: <473DEC13.3000609@sun.com> References: <473B1F70.3070108@gmx.de> <473B2E88.8020402@gmail.com> <473B4C65.5090109@sun.com> <473B52F6.2040009@gmail.com> <473DEC13.3000609@sun.com> Message-ID: <473EB86F.7020907@gmail.com> Charles Oliver Nutter wrote: > Wiring in support for calling Java libraries won't be particularly > difficult, given the many reflective capabilities already present for > Java. But I agree it needs to be there for most people to find a lot of > use. Indeed, I think this is the easiest part to do. > However, if PyPy on JVM can be brought to a point where it can actually > run real Python apps (especially apps that would be useful in the > domains where Java is useful) the integration question can be delayed a > bit. Jython, for example, can't run anything. If PyPy could beat that in > the short term and get something non-trivial running well, it would be a > huge bonus. I think that since we are highly compatible with CPython, existing Python apps are likely to work in pypy-jvm, modulo the fact that we still miss many libraries. For example, as Jean Paul said he was able to run a twisted IRC bot using pypy-c; I think/hope it to work also on pypy-jvm once we have implemented modules like socket & co. > That's the work I meant, thanks for clarifying. Is this actively being > worked right now? The JVM backend has been mostly written by me and Niko Matsakis. At the moment both of us work on it in our spare time, but if my phd proposal gets accepted I should be able to work on it much more continuously. ciao Anto From florian.schulze at gmx.net Sat Nov 17 14:15:47 2007 From: florian.schulze at gmx.net (Florian Schulze) Date: Sat, 17 Nov 2007 14:15:47 +0100 Subject: [pypy-dev] Modern GC Message-ID: Interesting stuff about a modern garbage collector: http://www.cs.technion.ac.il/~erez/projects.html Regards, Florian Schulze From lac at openend.se Sat Nov 17 15:24:47 2007 From: lac at openend.se (Laura Creighton) Date: Sat, 17 Nov 2007 15:24:47 +0100 Subject: [pypy-dev] pycon 2008 In-Reply-To: Message from Christian Tismer of "Sat, 17 Nov 2007 02:46:19 +0100." <473E47EB.4070004@stackless.com> References: <20071116132040.e41ebc03.simon@arrowtheory.com> <473E47EB.4070004@stackless.com> Message-ID: <200711171424.lAHEOlK1018889@theraft.openend.se> In a message of Sat, 17 Nov 2007 02:46:19 +0100, Christian Tismer writes: >Simon Burton wrote: >> Is anyone planning on going ? doing a talk ? > Some people from Open End will be coming. How many is not set yet. Laura From charles.nutter at sun.com Sun Nov 18 02:14:47 2007 From: charles.nutter at sun.com (Charles Oliver Nutter) Date: Sat, 17 Nov 2007 19:14:47 -0600 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: <473EB86F.7020907@gmail.com> References: <473B1F70.3070108@gmx.de> <473B2E88.8020402@gmail.com> <473B4C65.5090109@sun.com> <473B52F6.2040009@gmail.com> <473DEC13.3000609@sun.com> <473EB86F.7020907@gmail.com> Message-ID: <473F9207.2050105@sun.com> Antonio Cuni wrote: > The JVM backend has been mostly written by me and Niko Matsakis. At the > moment both of us work on it in our spare time, but if my phd proposal > gets accepted I should be able to work on it much more continuously. That's excellent. Having just completed a dynlang compiler on JVM, I'm available to help talk strategy on anything. Please let me know how I can help (given that I don't have a lot of coding time to spare right now). - Charlie From pfraser at spatialmedia.com Sun Nov 18 16:29:00 2007 From: pfraser at spatialmedia.com (Peter Fraser) Date: Mon, 19 Nov 2007 04:29:00 +1300 Subject: [pypy-dev] Very confused? -can someone at least tell me..? (was naive vm question) Message-ID: <47405A3C.4030203@spatialmedia.com> Ok -let me try again. If someone could answer 1. below, then I would be very grateful. Better yet, tell me where I leave the road thereafter. If I still sound hopelessly confused then I won't bother the list further, but I will be looking for someone to pay money to, for some skype-consulting to explain where I am going wrong. 1. Suppose you ran the PyPy-C interpreter as a CPython extension. Could the extension release the CPython GIL? 2. My naive bet is that it could -because the PyPy interpreter is not using CPython "in any essential way". 3. Leaving aside how useful this would be -can we agree that if indeed you could release the CPython GIL, then you could have two threads interpreting Python code, in one process, simultaneously. 4. This is possible because you have two of everything. Two code bases, two memory managers, two GILS etc etc etc. 5. The separation is so extreme that opportunities for cooperation between the two interpreters is severly curtailed. The type implementations aren't interchangeable for example. 6. Would this arrangement even be useful? This point is tangential. Personally I'm interested in any opportunity to utilize a second core, from one CPython process, simply using an extension -even with the severe constraints of messaging between two totally separate, incompatible VMs. But that is tangential. If this proved popular then that would be great for PyPy i think. But it is tangential. 7. The question I'm really getting at is, are there opportunities for a middle ground? Ok, so two threads running in one interpreter with no GIL is too intimate. Two distinct interpreters (a CPython and a PyPyC) in one process are too isolated (as are two threads in two processes). What is in the middle (other than locking)? 8. Yes, I am confusing designs and instances Boris. I'm trying to understand exactly what are the design differences, that create the implementation diferences, that conspire to make the CPython with PyPy-C extension mix, two separate instances. I'm speculating about whether you could contrive to create *just enough difference* to sustain two "instances" of essentialy the same interpreter, in one process. 9. I'm just talking about sufficient partitioning to achieve 2 instances in one process -where "instance" means that a thread running in either will never touch the data of the other. 10. A further wild hunch is that this partitioning might in part be achieved by wholesale code duplication, and the munging of names -as opposed to smarter code (viz structurally more sophisticated code to support the partition). I wouldn't be surprised to be totally wrong on this one and the concept doesn't hinge on it. It does scream "translation aspect" to me however. 11. I assume that if its possible, this partitioned "dual core" (as it were) interpreter would be far more easily achieved as a PyPy translation aspect, than by hand. This, the PyPyC extension GIL question, and the aim of PyPy to support different features is why I'm asking the question here. Also, much as I don't like fandom as a phenomena -I allow myself to be a PyPy fan. 12. But how useful would this be? Why do I think this would be better than simply running two processes with some kind of inter-process messaging? Simply speed. Suffice to say that I have a "cross process bad, in process good" mindset. 13. I'm betting on the identical types to give you far better cooperative efficiency than the CPython + PyPy-C approach, even if we don't try to actually share objects (but hey, that might be possible, a la posh, but in one process and on windows). 14. Finally -the key idea that niggles at me is that while the expensive, dense locking required for free threading presumably gives you an unbounded(ish) number of threads -the number of cores on modern machines is very bounded. Hence my grasping for fast, in process, messaging based concurrency that takes advantage of only having to support a bounded number of threads (2 for now) with their own instances -ie "dual core Python" (a good metaphor that) as a PyPy translation aspect. >>Putting aside the relevance to pypy<< Well I thought it was, but if someone could just answer 1. above, I concede that I could go away. If there is a more appropriate PyPy list for this kind of question -please tell me. Twins? I'm after viable Siamese twins. Co-joined but not too co-joined. PyPy really is aboue genetic engineering isn't it? Tweak the DNA and grow a new interpreter. regards Pete F From david at ar.media.kyoto-u.ac.jp Mon Nov 19 04:02:49 2007 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Mon, 19 Nov 2007 12:02:49 +0900 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: References: <473B1F70.3070108@gmx.de> <473BD486.6030806@ar.media.kyoto-u.ac.jp> <473D17BB.1070704@ar.media.kyoto-u.ac.jp> <473D8471.3090503@gmx.de> <473D83EF.1040703@ar.media.kyoto-u.ac.jp> Message-ID: <4740FCD9.20002@ar.media.kyoto-u.ac.jp> Neal Becker wrote: > David Cournapeau wrote: > >> Carl Friedrich Bolz wrote: >>> Neal Becker wrote: >>>> [...] >>>>>> Indeed. But there are tools to generate most of the painful code: >>>>>> Swig, boost::python, and others. >>>>>> With Swig, you can even subclass C++ classes and override virtual >>>>>> methods in python. >>>>>> I wish pypy could do the same. >>>>> Both methods involve parsing C++, which is not fun. For example, >>>>> concerning wxwidget, I for once think it would be much easier to wrap >>>>> gtk than wxwidget (if we think: would be good to have a GUI lib for >>>>> pypy); incidently, that's how pygtk is generated (some python + scheme >>>>> code generate the pygtk wrapper around gtk). >>>>> >>>> boost::python does not involve parsing c++. >>> yip, it does. The compiler parses C++, after all :-). >>> >>> nitpickingly-yours, >> You are not nitpicking at all: boost::python uses Py++, which uses >> gccxml to parse the C++. There is no way around it: since it is >> extremely difficult to interoperate with any C++ ABI, you need to >> interoperate at the source level, parsing the monstruosity that C++ is >> compared to C. >> > No. You can use py++ if you like to generate boost::python. This (py++) is > fairly new, and I believe relatively few boost::python users follow this > route. Most of us write boost::python wrappers by hand (it's usually > pretty easy). I was talking in the context of automating wrapping. Of course, you can wrap by hand, and it is as easy as in C for a few functions (I myself much prefer ctypes, though, but I am obviously biased against C++ :) ). But for 500 classes / 3000 functions, I doubt this is practical (the only big C++ wrapper I am somewhat familiar with in python is pyqt, which uses its own thing for the wrapping...). Once you need something at least partially automatic, you need parsing the code you are wrapping. For pypy, this is what is needed, no ? cheers, David From amauryfa at gmail.com Mon Nov 19 07:36:46 2007 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Mon, 19 Nov 2007 07:36:46 +0100 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: <4740FCD9.20002@ar.media.kyoto-u.ac.jp> References: <473B1F70.3070108@gmx.de> <473BD486.6030806@ar.media.kyoto-u.ac.jp> <473D17BB.1070704@ar.media.kyoto-u.ac.jp> <473D8471.3090503@gmx.de> <473D83EF.1040703@ar.media.kyoto-u.ac.jp> <4740FCD9.20002@ar.media.kyoto-u.ac.jp> Message-ID: David Cournapeau wrote: > > No. You can use py++ if you like to generate boost::python. This (py++) is > > fairly new, and I believe relatively few boost::python users follow this > > route. Most of us write boost::python wrappers by hand (it's usually > > pretty easy). > I was talking in the context of automating wrapping. Of course, you can > wrap by hand, and it is as easy as in C for a few functions (I myself > much prefer ctypes, though, but I am obviously biased against C++ :) ). > But for 500 classes / 3000 functions, I doubt this is practical (the > only big C++ wrapper I am somewhat familiar with in python is pyqt, > which uses its own thing for the wrapping...). wxPython has the same number of functions, and uses Swig to do the wrapping. > Once you need something at least partially automatic, you need parsing > the code you are wrapping. For pypy, this is what is needed, no ? This is not necessary, and sometimes not suitable. With Swig for example, the syntax for classes and function declarations is indeed based on C++, but you often need a lot of tweaks to make it generate code that will work properly. These are called "typemap", "extend", "director" and other "features", and these additional directives are not C++. Swig uses an internal representation of the parsed file, which can be dumped to XML for example. It is possible to imagine another way to create the same representation and have the same wrapper generated. Of course I have something like rffi in mind... I don't think that Swig itself can be used by pypy, I just wanted to show that automatic wrapping of C++ is possible and can be used by pypy, without the help of ctypes. Cheers, -- Amaury Forgeot d'Arc From david at ar.media.kyoto-u.ac.jp Mon Nov 19 08:01:34 2007 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Mon, 19 Nov 2007 16:01:34 +0900 Subject: [pypy-dev] missing things for making PyPy "production" ready (for some value of production) In-Reply-To: References: <473B1F70.3070108@gmx.de> <473BD486.6030806@ar.media.kyoto-u.ac.jp> <473D17BB.1070704@ar.media.kyoto-u.ac.jp> <473D8471.3090503@gmx.de> <473D83EF.1040703@ar.media.kyoto-u.ac.jp> <4740FCD9.20002@ar.media.kyoto-u.ac.jp> Message-ID: <474134CE.4040202@ar.media.kyoto-u.ac.jp> Amaury Forgeot d'Arc wrote: > > wxPython has the same number of functions, and uses Swig to do the wrapping. swig parses C++ headers, and basically a significant part of C++. There is really no way to do any kind of wrapping of C++ without parsing it, or at least a significant portion of it. > > This is not necessary, and sometimes not suitable. > With Swig for example, the syntax for classes and function > declarations is indeed based on C++, but you often need a lot of > tweaks to make it generate code that will work properly. These are > called "typemap", "extend", "director" and other "features", and these > additional directives are not C++. > > Swig uses an internal representation of the parsed file, which can be > dumped to XML for example. It is possible to imagine another way to > create the same representation and have the same wrapper generated. Of > course I have something like rffi in mind... Swig has been in development for more than 10 years now. And the C++ part is extremely difficult. As said in swig doc themselves: """ Because of its complexity and the fact that C++ can be difficult to integrate with itself let alone other languages, SWIG only provides support for a subset of C++ features. Fortunately, this is now a rather large subset. In part, the problem with C++ wrapping is that there is no semantically obvious (or automatic ) way to map many of its advanced features into other languages. As a simple example, consider the problem of wrapping C++ multiple inheritance to a target language with no such support. Similarly, the use of overloaded operators and overloaded functions can be problematic when no such capability exists in a target language. """ > I don't think that Swig itself can be used by pypy, I just wanted to > show that automatic wrapping of C++ is possible and can be used by > pypy, without the help of ctypes. My point is not that it is impossible to wrap C++ (doing with ctypes is impossible anyway, by the way: ctypes does not support C++), but: - C++ is significantly harder than C to wrap - A ffi for C++ is incredibly harder than C - Supporting C could already go towards a production ready pypy. C++ is certainly a nice add-on, but less important than C. I also actually think the swig approach to be quite useful, and maybe the path of least effort: C++->C wrapper, and then wrap C using ctypes: if the swig internal representation is reusable (although I have some experience with C++ and swig, I never tried the dump option you are talking about), then most of the work is done. David From faassen at startifact.com Mon Nov 19 18:10:09 2007 From: faassen at startifact.com (Martijn Faassen) Date: Mon, 19 Nov 2007 18:10:09 +0100 Subject: [pypy-dev] making PyPy production ready: followup Message-ID: Hi there, I'll continue in my role as the unofficial nag of this project. So we've had a useful discussion about making PyPy production ready. Excellent! Now we need to follow up on this. We need to distill a list of concrete tasks from it. We need to figure out who does what. Who in the PyPy project is going to make this list? Who will make sure that it gets done (or decides bits *won't* get done right now)? This brings me to commitment again, and leadership. I know I've asked about commitment before, in connection to the RPython for CPython modules facility. I got the answer "we'll need to discuss this at EuroPython", and there was some discussion there. I got the general impression that finishing the interpreter was considered to be of more importance. The current state of this support is rather in limbo: it's still there, but should be reworked to use new stuff that's there now, and it's unclear who will do it or when. Anyway, that's fine, as the main goal, I take it, is the interpreter. Good. And then I heard nothing much more until this current discussion... Perhaps I missed something? So what about the interpreter? What is the plan to bring it into production? Will the first goal be the PyPy-JVM idea that was thrown around at EuroPython? The C-backend? Something else? Someone needs to show leadership and make a plan. There needs to be a certain level of commitment to this plan from the project and project members. Even explicitly saying you are not going to do X and Y (or not right now) is valuable, as it helps people to know what to expect and where to help. Does the PyPy project commit to working on a Python interpreter in any shape and form that people can use in production at all? One that is released? If so, what's the timeline? When is the alpha going to be, when the beta? Do you only commit to this if you get volunteer contributors? Can you consider lowering ambition and do it without further volunteers? If you need further volunteers, will you give them the responsibility and authority to actually make hard decisions to make a release possible? Note that you'll get more volunteers that contribute if they have something to expect in the foreseeable future. I already mentioned release management. If you don't have some form of leadership on this, you risk scenarios like: * "we can't release right now as we're still working on X" * X is done. in the mean time, someone started Y " "We can't release right now as we're still working on Y" * Y is done. in the mean time, someone started Z * "We can't release right now as we're still working on Z" * and so on. You need to have someone or some group of people who are aware of these issues, who have the aim to make a release, and who can clearly say: yeah, I *know* we really need Y, but we are in stabilization mode right now, so please wait or do it on a branch. Making a release is *hard*. It's lots of frustrating work. You know, as you made the releases for the research part of the project. After this, the releases stopped. If you want a successful open source project, you need to make regular releases that people can use. You need to make new release announcements, as people read them. You also need to show some commitment towards production use to the public, and releases in this direction is the best way to do it. I believe doing this will be of tremendous open source marketing value to the project. You've gone all the way up to 1.0, but you've stated explicitly it's not intended for production use. Now is the time to let the world know you're progressing beyond this. Let the internet know that the PyPy project is a serious contender in the Python interpreter world, instead of an experiment. So, what's the plan? Regards, Martijn P.S. I'm leaving the funding discussion out of this explicitly. I assume you want to make PyPy work as an open source project. I'm also assuming that there is a way to get a production-ready interpreter by volunteer effort without external funding. If the project decides that there is *no* such way, then please let everybody know that too. From ondrej at certik.cz Mon Nov 19 18:16:56 2007 From: ondrej at certik.cz (Ondrej Certik) Date: Mon, 19 Nov 2007 18:16:56 +0100 Subject: [pypy-dev] making PyPy production ready: followup In-Reply-To: References: Message-ID: <85b5c3130711190916kb36d2e8v93b83d9586c9460a@mail.gmail.com> > Someone needs to show leadership and make a plan. There needs to be a > certain level of commitment to this plan from the project and project Yes, this is really important. Ondrej From brian at dorseys.org Tue Nov 20 06:06:40 2007 From: brian at dorseys.org (Brian Dorsey) Date: Mon, 19 Nov 2007 21:06:40 -0800 Subject: [pypy-dev] pycon 2008 In-Reply-To: <20071116132040.e41ebc03.simon@arrowtheory.com> References: <20071116132040.e41ebc03.simon@arrowtheory.com> Message-ID: <66e877b70711192106q73bd09baj2e54693ea5c83c71@mail.gmail.com> On Nov 16, 2007 1:20 PM, Simon Burton wrote: > Is anyone planning on going ? doing a talk ? I'm planning to go and just submitted a talk proposal for a py.test talk based on Holger & Maciej's talk at EuroPython. If it is accepted, I will likely co-present with either or both of them if they can come to PyCon. Take care, -Brian From cfbolz at gmx.de Tue Nov 20 13:40:37 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 20 Nov 2007 13:40:37 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects Message-ID: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> Hi all, Richard and I are about to finally remove support for the extension compiler, RCTypes and the ability to support CPython Objects in the translation. We will do this work on the new-genc-tests-wrapper branch (which now has a slightly strange name). If anybody has good arguments for keeping these things we would like to hear about it very soon. The RCTypes functionality is implemented by rffi these days. The extension compiler can be reimplemented with rffi if someone is interested in doing that. The current incarnation is not giving fast modules anyway (and is based on RCTypes and various strange hacks). Cheers, Carl Friedrich & Richard From faassen at startifact.com Tue Nov 20 15:11:26 2007 From: faassen at startifact.com (Martijn Faassen) Date: Tue, 20 Nov 2007 15:11:26 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> Message-ID: Hey, Carl Friedrich Bolz wrote: > Richard and I are about to finally remove support for the extension > compiler, RCTypes and the ability to support CPython Objects in the > translation. We will do this work on the new-genc-tests-wrapper branch > (which now has a slightly strange name). > > If anybody has good arguments for keeping these things we would like to > hear about it very soon. The RCTypes functionality is implemented by > rffi these days. The extension compiler can be reimplemented with rffi > if someone is interested in doing that. The current incarnation is not > giving fast modules anyway (and is based on RCTypes and various strange > hacks). I'm not objecting. This was discussed in the past and it was deemed a good idea. I'll just note what this might mean to you in terms of project perception and perception of commitment: You're replacing one of the few areas of PyPy that at least *seemed* to be useful in the near term for production use (even though it evidently wasn't really) with "the extension compiler can be reimplemented with rffi if someone is interested in doing that". I.e. you're removing your implicit commitment (in the form of code that more or less worked) and replacing it with an explicit lack of commitment or interest in making this work. Meanwhile the project has not given many signals about its current level of commitment to produce a Python interpreter that people can use in production. Initially I got the impression that people wanted to synchronize at EuroPython after the closure of the EU project, but what was the conclusion? Perhaps this sprint is a good opportunity. The perception could easily arise that there is no interest in giving an impression of commitment, and thus not to expect a production-quality Python interpreter in the foreseeable future. Is this perception correct? Are people even talking about giving a level of commitment, or is this whole discussion uninteresting to the project altogether? I'd like to know so I can adjust my own perception accordingly. Regards, Martijn From cfbolz at gmx.de Tue Nov 20 16:38:19 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 20 Nov 2007 16:38:19 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> Message-ID: <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> Hi Martijn, Martijn Faassen wrote: > I'll just note what this might mean to you in terms of project > perception and perception of commitment: > > You're replacing one of the few areas of PyPy that at least *seemed* to > be useful in the near term for production use (even though it evidently > wasn't really) with "the extension compiler can be reimplemented with > rffi if someone is interested in doing that". > > I.e. you're removing your implicit commitment (in the form of code that > more or less worked) and replacing it with an explicit lack of > commitment or interest in making this work. I really thought we were through with that. So, again: First let me note that I think the extcompiler is a promising idea. I agree that the idea could lead to something generally useful. However: The current implementation is not useful at all. It wasn't used seriously by anybody. Richard tried to use it and he found that it wasn't creating anything useful, since the resulting CPython extensions weren't really any faster than just writing the stuff in pure Python to begin with. For these reasons removing this piece of code is not showing lack of commitment or lack of leadership or sending a wrong signal in my opinion. It is just getting rid of a failed implementation idea. > Meanwhile the project has not given many signals about its current level > of commitment to produce a Python interpreter that people can use in > production. Initially I got the impression that people wanted to > synchronize at EuroPython after the closure of the EU project, but what > was the conclusion? Perhaps this sprint is a good opportunity. > > The perception could easily arise that there is no interest in giving an > impression of commitment, and thus not to expect a production-quality > Python interpreter in the foreseeable future. Is this perception > correct? Are people even talking about giving a level of commitment, or > is this whole discussion uninteresting to the project altogether? I'd > like to know so I can adjust my own perception accordingly. Because of what I wrote above, I don't think your conclusions are fair. Cheers, Carl Friedrich From cfbolz at gmx.de Tue Nov 20 18:07:26 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 20 Nov 2007 18:07:26 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> Message-ID: <348899050711200907l6b81c4fr9ae4e7c54e598e87@mail.gmail.com> this message of richard didn't make it to the list, I am forwarding it. ---------- Forwarded message ---------- From: Richard Emslie Date: 20.11.2007 17:49 Subject: Re: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects To: Carl Friedrich Bolz Cc: Martijn Faassen , pypy-dev at codespeak.net On Tue, 20 Nov 2007, Carl Friedrich Bolz wrote: > Hi Martijn, > > Martijn Faassen wrote: >> I'll just note what this might mean to you in terms of project >> perception and perception of commitment: >> >> You're replacing one of the few areas of PyPy that at least *seemed* to >> be useful in the near term for production use (even though it evidently >> wasn't really) with "the extension compiler can be reimplemented with >> rffi if someone is interested in doing that". >> >> I.e. you're removing your implicit commitment (in the form of code that >> more or less worked) and replacing it with an explicit lack of >> commitment or interest in making this work. > > I really thought we were through with that. So, again: > > First let me note that I think the extcompiler is a promising idea. I > agree that the idea could lead to something generally useful. However: > The current implementation is not useful at all. It wasn't used > seriously by anybody. Richard tried to use it and he found that it > wasn't creating anything useful, since the resulting CPython > extensions weren't really any faster than just writing the stuff in > pure Python to begin with. > To be more accurate - what was found was that the overhead of calling in and out of the of the generated extension module outweighed the gain in speed generated by the rpython code. Also the generated code was not 100% stable and even though I spent a few weeks hacking the generated code, it was still segfaulting (whereas the non extension module was fine.) All in all I agree that removing the current implementation is the best course of action for moving forward. Cheers - Richard From janzert at janzert.com Tue Nov 20 18:12:40 2007 From: janzert at janzert.com (Janzert) Date: Tue, 20 Nov 2007 12:12:40 -0500 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> Message-ID: Carl Friedrich Bolz wrote: [Snip] > First let me note that I think the extcompiler is a promising idea. I > agree that the idea could lead to something generally useful. However: > The current implementation is not useful at all. It wasn't used > seriously by anybody. Richard tried to use it and he found that it > wasn't creating anything useful, since the resulting CPython > extensions weren't really any faster than just writing the stuff in > pure Python to begin with. > [Snip] I don't have the experience, time or motivation to maintain the extcompiler so I completely understand if you need to remove it. But I do want to say that there are situations where it is useful and a speed up. My use of the extcompiler was making a mixed module that is a board representation and move generator for a board game (Arimaa). The resulting extcompiler module was approximately 10 times faster than the corresponding regular python module. So I think the ability to see speed increases with the current extcompiler is probably heavily dependent on how frequently the cpython to rpython interface has to be crossed. Janzert From faassen at startifact.com Tue Nov 20 18:17:07 2007 From: faassen at startifact.com (Martijn Faassen) Date: Tue, 20 Nov 2007 18:17:07 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> Message-ID: Carl Friedrich Bolz wrote: > Martijn Faassen wrote: >> I'll just note what this might mean to you in terms of project >> perception and perception of commitment: >> >> You're replacing one of the few areas of PyPy that at least *seemed* to >> be useful in the near term for production use (even though it evidently >> wasn't really) with "the extension compiler can be reimplemented with >> rffi if someone is interested in doing that". >> >> I.e. you're removing your implicit commitment (in the form of code that >> more or less worked) and replacing it with an explicit lack of >> commitment or interest in making this work. > > I really thought we were through with that. I said I wasn't objecting. I'm not objecting. I'm just pointing out what it means to me. I'm also asking the next question: CPython extensions are definitely off the table. so what's up with the Python interpreter? I thought that a way to make cool Python bindings with RPython might be a way to get PyPy to practical use. So I told people that they should seriously consider putting some effort behind this feature (whatever form it might take), as this seemed the closest to real-world use. I asked a set of questions in june. Back then, I got two answers to my questions. One from Maciek and one from Bea. Both said, we'll discuss this at EuroPython. At EuroPython I got the impression there was little interest in making this work and that people wanted to work on making PyPy mature first. It wasn't very clear, but that's my impression. Okay, the interpreter first, that's my understanding. As an aside, it's now after EuroPython, so I'll try answering these questions myself now: * what are the reasons this change is going to be made? [some answers were given on another thread back then. Basically, the current way is broken.] * would this benefit just core PyPy or would it benefit extension module writers too? [Only core PyPy. If some work is done by volunteers, by extension module writers too as the rewrite is cleaner] * when do you expect the new way to be finalized? [For the core PyPy stuff: in November 2007. The core PyPy thing is more or less done now, I understand, being cleaned up now. For the extension module compiler: this depends on other people, we are not going to do it] * will you make a real commitment to make the "Test in CPython first" feature work again? Or is did you just sketched out possible ways to implement this in the new system and you hope some contributor is going to do the work? [I'm not sure about this one. Can you write a CTypes module and test it in CPython before running it with PyPy, or not anymore?] * will you make a commitment to make this new way useful and supported for extension module writers, or should they invest their time in some other technology? I.e. what are the changes a future shift will make me change things again? [We won't make any such commitment, and they should make an investment in some other technology like CTypes or Pyrex or do it themselves with PyPy with the knowledge the core team isn't very interested themselves] * How well documented will this be for CPython extension developers? Will you market this feature to developers? Things like this indicate your commitment in another way. :) [This question is irrelevant, as there is no commitment] Are my answers correct? I understand that there's no commitment to make it work, and that you'll just have this feature go away. It was broken in the first place. That's fine - I have no objections, as I said. I'm just telling you what it means for me. > For these reasons removing this piece of code is not showing lack of > commitment or lack of leadership or sending a wrong signal in my > opinion. It is just getting rid of a failed implementation idea. It's fine that you don't commit to this feature. I'd preferred better communication, but that's not a major complaint. It was clear enough to me already that this feature was in limbo. What worries me is that I don't see any commitment to make *any* feature work for production-use. Not the extension compiler, obviously. So that brings me to the rest of the email: it leaves the other potential practical feature that was discussed at EuroPython: the production interpreter. People preferred focusing on that as it wasn't considered to be that far off, and it was more central to the goal of the project than an extension compiler for CPython. So, please let me know whether you have a commitment to make a production ready interpreter work. Here are some questions that people might ask about the production Python interpreter: * Does the project have any commitment towards producing a production-usable Python interpreter, for whatever value of 'production'? I.e. is the project interested in these questions at all, or do they consider these questions beside the point? * If the project does make a commitment, does the project commit to release management to churn out new versions of the production interpreter at a regular basis? * If the project does make a commitment, when do you expect the production PyPy interpreter to be ready for early adopters? When do you think it will be solid enough to reach lots of people? Is there some form of timeline? * What platform will you focus on first? (JVM, .net, C, something else?) * Will the production interpreter contain JIT technology? If not, when is this expected? If this is to be a research project with no concrete plans for a production Python interpreter that's fine. I'll certainly stop bugging you about these topics. If the plan is to start serious work on Smalltalk or Ruby interpreters first before finishing the Python interpreter, or if the plan is to work on all three at the same time, forgive me when I adjust my expectations downwards for a Python interpreter. I will wish you luck. I hope that all this was already discussed and clear within the project, and just not communicated widely yet. If so, please let us know the answers. If it wasn't discussed yet, then please discuss this kind of stuff at the sprint. It seems like a good opportunity. If you do aim to finish the Python interpreter, I'd expect to see a list of concrete goals and date estimates, and then please share it with us if you'd like? I feel a bit like I'm asking crazy unrealistic impossible questions, but I don't think I am? Wasn't the project supposed to come up with a plan for these things in the post-EU phase? All these questions are from an interested bystander who may want to commit some time on this project (more than reading and writing emails) in the next year if he does see some idea that the project is indeed sharing some of his goals. I think there are more people like me you could attract if you give us a bit of hope. Regards, Martijn From faassen at startifact.com Tue Nov 20 18:20:02 2007 From: faassen at startifact.com (Martijn Faassen) Date: Tue, 20 Nov 2007 18:20:02 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> Message-ID: <8928d4e90711200920h10c46218l35e13af20d9d3b7d@mail.gmail.com> Hey, On Nov 20, 2007 5:49 PM, Richard Emslie wrote: [snip] > To be more accurate - what was found was that the overhead of calling in > and out of the of the generated extension module outweighed the gain in > speed generated by the rpython code. Also the generated code was not 100% > stable and even though I spent a few weeks hacking the generated code, it > was still segfaulting (whereas the non extension module was fine.) All in > all I agree that removing the current implementation is the best course of > action for moving forward. Great! I also understood from the previous discussions the old way was broken. But you talk about moving forward - is the plan to get back to establish the original feature again with new technology? That's not the impression I got from the email, but it'd be great if my impression was wrong. Regards, Martijn From ondrej at certik.cz Tue Nov 20 19:30:19 2007 From: ondrej at certik.cz (Ondrej Certik) Date: Tue, 20 Nov 2007 19:30:19 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> Message-ID: <85b5c3130711201030x4daf4e7cr136a112b71f784f4@mail.gmail.com> I couldn't say it better, completely agree with Martijn. It's fine, if pypy is going to be a research project, but I was thinking for many years, that the aim is to get a production interpreter. In fact, there will be dozens people willing to help out, but pypy first needs to solve 90% of things for them first. Otherwise they will not join, that's how it works. So I myself will also invest my time to other projects, that can solve 90% of things for me now, Cython for example. Ondrej On 11/20/07, Martijn Faassen wrote: > Carl Friedrich Bolz wrote: > > Martijn Faassen wrote: > >> I'll just note what this might mean to you in terms of project > >> perception and perception of commitment: > >> > >> You're replacing one of the few areas of PyPy that at least *seemed* to > >> be useful in the near term for production use (even though it evidently > >> wasn't really) with "the extension compiler can be reimplemented with > >> rffi if someone is interested in doing that". > >> > >> I.e. you're removing your implicit commitment (in the form of code that > >> more or less worked) and replacing it with an explicit lack of > >> commitment or interest in making this work. > > > > I really thought we were through with that. > > I said I wasn't objecting. I'm not objecting. I'm just pointing out what > it means to me. I'm also asking the next question: CPython extensions > are definitely off the table. so what's up with the Python interpreter? > > I thought that a way to make cool Python bindings with RPython might be > a way to get PyPy to practical use. So I told people that they should > seriously consider putting some effort behind this feature (whatever > form it might take), as this seemed the closest to real-world use. > > I asked a set of questions in june. Back then, I got two answers to my > questions. One from Maciek and one from Bea. Both said, we'll discuss > this at EuroPython. > > At EuroPython I got the impression there was little interest in making > this work and that people wanted to work on making PyPy mature first. It > wasn't very clear, but that's my impression. Okay, the interpreter > first, that's my understanding. > > As an aside, it's now after EuroPython, so I'll try answering these > questions myself now: > > * what are the reasons this change is going to be made? > > [some answers were given on another thread back then. Basically, the > current way is broken.] > > * would this benefit just core PyPy or would it benefit extension module > writers too? > > [Only core PyPy. If some work is done by volunteers, by extension module > writers too as the rewrite is cleaner] > > * when do you expect the new way to be finalized? > > [For the core PyPy stuff: in November 2007. The core PyPy thing is more > or less done now, I understand, being cleaned up now. For the extension > module compiler: this depends on other people, we are not going to do it] > > * will you make a real commitment to make the "Test in CPython first" > feature work again? Or is did you just sketched out possible ways to > implement this in the new system and you hope some contributor is going > to do the work? > > [I'm not sure about this one. Can you write a CTypes module and test it > in CPython before running it with PyPy, or not anymore?] > > * will you make a commitment to make this new way useful and supported > for extension module writers, or should they invest their time in some > other technology? I.e. what are the changes a future shift will make me > change things again? > > [We won't make any such commitment, and they should make an investment > in some other technology like CTypes or Pyrex or do it themselves with > PyPy with the knowledge the core team isn't very interested themselves] > > * How well documented will this be for CPython extension developers? > Will you market this feature to developers? Things like this indicate > your commitment in another way. :) > > [This question is irrelevant, as there is no commitment] > > Are my answers correct? I understand that there's no commitment to make > it work, and that you'll just have this feature go away. It was broken > in the first place. That's fine - I have no objections, as I said. I'm > just telling you what it means for me. > > > For these reasons removing this piece of code is not showing lack of > > commitment or lack of leadership or sending a wrong signal in my > > opinion. It is just getting rid of a failed implementation idea. > > It's fine that you don't commit to this feature. I'd preferred better > communication, but that's not a major complaint. It was clear enough to > me already that this feature was in limbo. > > What worries me is that I don't see any commitment to make *any* feature > work for production-use. Not the extension compiler, obviously. So that > brings me to the rest of the email: it leaves the other potential > practical feature that was discussed at EuroPython: the production > interpreter. People preferred focusing on that as it wasn't considered > to be that far off, and it was more central to the goal of the project > than an extension compiler for CPython. > > So, please let me know whether you have a commitment to make a > production ready interpreter work. > > Here are some questions that people might ask about the production > Python interpreter: > > * Does the project have any commitment towards producing a > production-usable Python interpreter, for whatever value of > 'production'? I.e. is the project interested in these questions at all, > or do they consider these questions beside the point? > > * If the project does make a commitment, does the project commit to > release management to churn out new versions of the production > interpreter at a regular basis? > > * If the project does make a commitment, when do you expect the > production PyPy interpreter to be ready for early adopters? When do you > think it will be solid enough to reach lots of people? Is there some > form of timeline? > > * What platform will you focus on first? (JVM, .net, C, something else?) > > * Will the production interpreter contain JIT technology? If not, when > is this expected? > > If this is to be a research project with no concrete plans for a > production Python interpreter that's fine. I'll certainly stop bugging > you about these topics. If the plan is to start serious work on > Smalltalk or Ruby interpreters first before finishing the Python > interpreter, or if the plan is to work on all three at the same time, > forgive me when I adjust my expectations downwards for a Python > interpreter. I will wish you luck. > > I hope that all this was already discussed and clear within the project, > and just not communicated widely yet. If so, please let us know the > answers. If it wasn't discussed yet, then please discuss this kind of > stuff at the sprint. It seems like a good opportunity. If you do aim to > finish the Python interpreter, I'd expect to see a list of concrete > goals and date estimates, and then please share it with us if you'd like? > > I feel a bit like I'm asking crazy unrealistic impossible questions, but > I don't think I am? Wasn't the project supposed to come up with a plan > for these things in the post-EU phase? > > All these questions are from an interested bystander who may want to > commit some time on this project (more than reading and writing emails) > in the next year if he does see some idea that the project is indeed > sharing some of his goals. I think there are more people like me you > could attract if you give us a bit of hope. > > Regards, > > Martijn > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From tismer at stackless.com Tue Nov 20 19:57:08 2007 From: tismer at stackless.com (Christian Tismer) Date: Tue, 20 Nov 2007 19:57:08 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> Message-ID: <47432E04.8020908@stackless.com> Martijn Faassen wrote: > Carl Friedrich Bolz wrote: >> Martijn Faassen wrote: >>> I'll just note what this might mean to you in terms of project >>> perception and perception of commitment: >>> >>> You're replacing one of the few areas of PyPy that at least *seemed* to >>> be useful in the near term for production use (even though it evidently >>> wasn't really) with "the extension compiler can be reimplemented with >>> rffi if someone is interested in doing that". >>> >>> I.e. you're removing your implicit commitment (in the form of code that >>> more or less worked) and replacing it with an explicit lack of >>> commitment or interest in making this work. >> I really thought we were through with that. > > I said I wasn't objecting. I'm not objecting. I'm just pointing out what > it means to me. I'm also asking the next question: CPython extensions > are definitely off the table. so what's up with the Python interpreter? I do not see where you are pulling that statement off. It does not make sense after what has been said, before. There is also little resources to continue this thread that intensively, since we actually very busy doing a clean-up sprint. It is most likely that somebody will want to support building extensions using rffi. cheers - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From lypanov at mac.com Tue Nov 20 21:08:56 2007 From: lypanov at mac.com (Alexander Kellett) Date: Tue, 20 Nov 2007 21:08:56 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> Message-ID: <2982FB05-7E91-487D-A7D8-CA8EFF080BB0@mac.com> On Nov 20, 2007, at 3:11 PM, Martijn Faassen wrote: > The perception could easily arise that there is no interest in > giving an > impression of commitment, and thus not to expect a production-quality > Python interpreter in the foreseeable future. Is this perception > correct? Are people even talking about giving a level of commitment, > or > is this whole discussion uninteresting to the project altogether? I'd > like to know so I can adjust my own perception accordingly. This thread is getting a bit repetitive. Until PyPy is in some non-marginal manner much better than cpython no one will be interested anyway. Its the niches that make a non canonical implementation successful. Getting any random project to run on top of pypy amounts not to 20% work but to 80% as most of the work will be writing very boring extension modules. Doing so is just not a worthwhile goal. I truly hope that the JIT and GC get the work they need. An implementation that performs 10x faster on numeric code will already spark a lot of interest, as has been seen in the case of psycho. Given that adding Ruby as an alternative frontend may very well be the only way to get this funded, I don't really understand why this is even being talked about. If you really want pypy to become "production quality", then Q&A is needed, so start a compatibility testing project or something... Alex --- Alexander Kellett PGP - 0x6BFA8EF3, FPR: DA65 D6DE 56A9 D715 EFB6 A948 B2EF 6622 6BFA 8EF3 From tismer at stackless.com Tue Nov 20 21:50:37 2007 From: tismer at stackless.com (Christian Tismer) Date: Tue, 20 Nov 2007 21:50:37 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: <2982FB05-7E91-487D-A7D8-CA8EFF080BB0@mac.com> References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <2982FB05-7E91-487D-A7D8-CA8EFF080BB0@mac.com> Message-ID: <4743489D.4040205@stackless.com> Alexander Kellett wrote: > On Nov 20, 2007, at 3:11 PM, Martijn Faassen wrote: >> The perception could easily arise that there is no interest in >> giving an >> impression of commitment, and thus not to expect a production-quality >> Python interpreter in the foreseeable future. Is this perception >> correct? Are people even talking about giving a level of commitment, >> or >> is this whole discussion uninteresting to the project altogether? I'd >> like to know so I can adjust my own perception accordingly. > > This thread is getting a bit repetitive. Indeed. > Until PyPy is in some non-marginal manner much better than cpython no > one will be interested anyway. This is true, but in a different way than you might think. Many people have their own perception of "good". But good is not simply only reading like "speed". There are lots of other criteria, where PyPy is already very good at, with no single competitor. Of course, the broad audience wants everything at once, including out-performing CPython and implementing everything at the same time. I think that is quite much to expect. And very few people valuate all the new possibilities with PyPy's configurable features at all. Anyway, "no one will be interested" is just a thoughtless statement. There are lots of directions where PyPy might successfully move to. With or without Jit or Gc, with or without supporting tons of extension modules or even by finding ways to just wrap them. There are already many people interested in what PyPy can do today, and I want to call this being successful right now. cheers - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From lypanov at mac.com Tue Nov 20 22:04:49 2007 From: lypanov at mac.com (Alexander Kellett) Date: Tue, 20 Nov 2007 22:04:49 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: <4743489D.4040205@stackless.com> References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <2982FB05-7E91-487D-A7D8-CA8EFF080BB0@mac.com> <4743489D.4040205@stackless.com> Message-ID: On Nov 20, 2007, at 9:50 PM, Christian Tismer wrote: >> Until PyPy is in some non-marginal manner much better than cpython >> no one will be interested anyway. > > There are lots of other criteria, where PyPy is already > very good at, with no single competitor. > Anyway, "no one will be interested" is just a thoughtless > statement. Agreed, my apologies, it wasn't the best way to put it. I attempted to list all the various areas of interest but gave up after getting to the 7th knowing full well that there are many more than that... what I really wanted to get across was that putting *primary* emphasis on "production quality" would be terrible for the project in my opinion. Alex --- Alexander Kellett PGP - 0x6BFA8EF3, FPR: DA65 D6DE 56A9 D715 EFB6 A948 B2EF 6622 6BFA 8EF3 From tismer at stackless.com Wed Nov 21 00:00:09 2007 From: tismer at stackless.com (Christian Tismer) Date: Wed, 21 Nov 2007 00:00:09 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <2982FB05-7E91-487D-A7D8-CA8EFF080BB0@mac.com> <4743489D.4040205@stackless.com> Message-ID: <474366F9.7020801@stackless.com> Alexander Kellett wrote: ... > Agreed, my apologies, it wasn't the best way to put it. I attempted to > list all the various areas of interest but gave up after getting to the > 7th knowing full well that there are many more than that... what I > really wanted to get across was that putting *primary* emphasis on > "production quality" would be terrible for the project in my opinion. No problem, I'm quite often falling into this trap as well. I was after speed in the first place, but the way the project was started and continued, it opened a huge lot of new opportunities, and we now really need to finish some directions, which is actually being worked on. Having such an amount of working tests already, I'm not really worried about production-readiness. It is more about completeness of needed modules, to be able to run arbitrary Python code. But even that has gone pretty far, already. So for me, a usable PyPy, in the sense of "runs very many Python programs" is more like a matter of weeks, not months. Being completely superior than CPython in speed, correctness, flexibility and everything is of course something that requires many more months. But the proof of doability is there. This should be worth enough to find sponsors, I guess. cheers - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From faassen at startifact.com Wed Nov 21 13:00:49 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 21 Nov 2007 13:00:49 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: <47432E04.8020908@stackless.com> References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> <47432E04.8020908@stackless.com> Message-ID: <8928d4e90711210400i4b6661c1k474411f2614bd788@mail.gmail.com> Hi there, On Nov 20, 2007 7:57 PM, Christian Tismer wrote: > Martijn Faassen wrote: [snip] > > I said I wasn't objecting. I'm not objecting. I'm just pointing out what > > it means to me. I'm also asking the next question: CPython extensions > > are definitely off the table. so what's up with the Python interpreter? > > I do not see where you are pulling that statement off. > It does not make sense after what has been said, before. > > There is also little resources to continue this thread > that intensively, since we actually very busy doing a > clean-up sprint. > > It is most likely that somebody will want to support > building extensions using rffi. I am pulling statements out of my hat because the project isn't really handing the out, is it? The CPython extension mechanism was discussed in june. I asked some questions. I never got an answer, except my strong impression that people wanted to focus on the interpreter itself at EuroPython. I now, half a year later, hear that the mechanism is taken about and if people want to bring it back the new way, they can. All this together gives me the strong impression the project doesn't care much about this feature at the present and is not going to do much about this, as a project. Volunteers are welcome to try to add it back at some point Regards, Martijn From faassen at startifact.com Wed Nov 21 13:04:40 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 21 Nov 2007 13:04:40 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: <2982FB05-7E91-487D-A7D8-CA8EFF080BB0@mac.com> References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <2982FB05-7E91-487D-A7D8-CA8EFF080BB0@mac.com> Message-ID: <8928d4e90711210404o482a7110y7905423408035ee7@mail.gmail.com> Hey, On Nov 20, 2007 9:08 PM, Alexander Kellett wrote: > On Nov 20, 2007, at 3:11 PM, Martijn Faassen wrote: > > The perception could easily arise that there is no interest in > > giving an > > impression of commitment, and thus not to expect a production-quality > > Python interpreter in the foreseeable future. Is this perception > > correct? Are people even talking about giving a level of commitment, > > or > > is this whole discussion uninteresting to the project altogether? I'd > > like to know so I can adjust my own perception accordingly. > > This thread is getting a bit repetitive. Yes. For about half a year now. > If you really want pypy to become "production quality", then Q&A is > needed, so start a compatibility testing project or something... With this level of commitment to support this from the PyPy team I think people will be seriously discouraged from putting any of their time in it. When I put time in something, I want to know I can get something back from it within a reasonable time frame. I want to have the impression that the team behind the project cares about this kind of use. I'm trying to find out. I've been trying to find out for half a year. So far I'm pretty discouraged by the answers, or non-answers. Regards, Martijn From faassen at startifact.com Wed Nov 21 13:12:17 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 21 Nov 2007 13:12:17 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: <4743489D.4040205@stackless.com> References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <2982FB05-7E91-487D-A7D8-CA8EFF080BB0@mac.com> <4743489D.4040205@stackless.com> Message-ID: <8928d4e90711210412n408e3afcw9d212d455351d069@mail.gmail.com> Hi, On Nov 20, 2007 9:50 PM, Christian Tismer wrote: [snip] I agree that there are many ways in which PyPy could be interesting for production use, not just a high-performance full-fledged Python interpreter. I've been asking about the interpreter as I got the strong impression that this was the main aim of the project when I asked about this kind of stuff at EuroPython. As an example, I think running this interpreter on, say, the, JVM, could be quite interesting, given its compliance with the language, and if it exposes Java libraries. > There are already many people interested in what PyPy can do > today, and I want to call this being successful right now. Some people were quite interested in the CPython extension language system. That has been in limbo for half a year, and the project just told us that people can add it back with the new system if they should have such a desire. That's what prompted me to speak up about the perceptions this project may have. If you want to get interested people to contribute, you'll need to offer features that in some way compete successfully with other systems. On the extension mechanism front, you won't compete with Pyrex/Cython or CTypes right now. Theoretically PyPy could come up with something nicer than Pyrex, but in practice one would need to invest a lot of time to make it work first. There's no interest from the project itself to invest significant time on it. Building interpreters is more important. On the interpreter front however, I've gotten a worryingly familiar lack of concreteness and unwillingness to promise anything. I'd suggest the project get more concrete and sets some targets. Once you do that, you might get more people willing to help. Regards, Martijn From faassen at startifact.com Wed Nov 21 13:32:26 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 21 Nov 2007 13:32:26 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <2982FB05-7E91-487D-A7D8-CA8EFF080BB0@mac.com> <4743489D.4040205@stackless.com> Message-ID: Hey, Alexander Kellett wrote: [snip] > what I > really wanted to get across was that putting *primary* emphasis on > "production quality" would be terrible for the project in my opinion. Why do you think that? Don't you think it's getting time after 4 years of work? What exactly would be "terrible for the project"? I've been trying to point out that if the project doesn't put some emphasis on getting *anything* (interpreter, extension language compiler or space chickens) in production, the project is not going to attract more volunteers and it's losing credibility on getting anything in production use at all. The web page says that the project aims to provide two things: a translation and support framework for dynamic languages, and "a compliant, flexible and fast implementation of the Python Language using the above framework to enable new advanced features without having to encode low level details into it." I sort of thought this implied the goal of having people actually use it. (Perhaps it's not. In that case, I'll stop paying so much attention. Tell me!) When I ask questions about the plan to get there, I don't get answers. I don't get answers when they're not sprinting, because they need to talk at the sprint first (EuroPython). I don't get answers when they *are* sprinting as they're too busy sprinting to talk about this stuff. Forgive me for getting frustrated. Regards, Martijn From faassen at startifact.com Wed Nov 21 13:36:35 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 21 Nov 2007 13:36:35 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> Message-ID: Hey, Martijn Faassen wrote: [snip] > I asked a set of questions in june. Back then, I got two answers to my > questions. One from Maciek and one from Bea. Both said, we'll discuss > this at EuroPython. Oh, and I forgot this reply from Armin: > That's a very good list of questions. Indeed, we don't have clear > answers for them yet, but just vague thoughts. I think it's fair to > say that we'll discuss this at EuroPython and make some commitments > then; for now we're only tossing ideas around. Regards, Martijn From fijall at gmail.com Wed Nov 21 13:44:58 2007 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 21 Nov 2007 13:44:58 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> Message-ID: <693bc9ab0711210444t231ef038ne2a12a0337b53ec3@mail.gmail.com> Martijn Faassen wrote: Hey, Martijn, you're effectively suspending sprint for moments, because people tend to read your mails. Please wait a week before actually sending anything (even reply to that email), as would be cooler to have sprint running rather than replying your emails. Cheers, fijal -------------- next part -------------- An HTML attachment was scrubbed... URL: From cfbolz at gmx.de Wed Nov 21 14:04:46 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 21 Nov 2007 14:04:46 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: <693bc9ab0711210444t231ef038ne2a12a0337b53ec3@mail.gmail.com> References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> <693bc9ab0711210444t231ef038ne2a12a0337b53ec3@mail.gmail.com> Message-ID: <348899050711210504r48205933laa1c8e36d27e99a9@mail.gmail.com> Maciej Fijalkowski wrote: > Martijn Faassen wrote: >> Hey, >> > Martijn, you're effectively suspending sprint for moments, because > people tend to read your mails. Please wait a week before actually > sending anything (even reply to that email), as would be cooler to have > sprint running rather than replying your emails. FWIW, while I tend to find Martijn's mails upsetting, I find this over the top. We are not really out for forbidding people to post on our mailing list, which is supposed to be an open discussion forum. Cheers, Carl Friedrich From faassen at startifact.com Wed Nov 21 14:05:34 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 21 Nov 2007 14:05:34 +0100 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: <693bc9ab0711210444t231ef038ne2a12a0337b53ec3@mail.gmail.com> References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> <693bc9ab0711210444t231ef038ne2a12a0337b53ec3@mail.gmail.com> Message-ID: <8928d4e90711210505w6d18d48y36d0f1b7ad39824d@mail.gmail.com> Hey, On Nov 21, 2007 1:44 PM, Maciej Fijalkowski wrote: > Martijn Faassen wrote: > Hey, > > > Martijn, you're effectively suspending sprint for moments, because people > tend to read your mails. Please wait a week before actually sending anything > (even reply to that email), as would be cooler to have sprint running rather > than replying your emails. You know what, I'll stop bothering you all together. Regards, Martijn [unsubscribed] From simon at arrowtheory.com Wed Nov 21 16:07:22 2007 From: simon at arrowtheory.com (Simon Burton) Date: Wed, 21 Nov 2007 07:07:22 -0800 Subject: [pypy-dev] Removal of RCTypes, the extension compiler, support for PyObjects In-Reply-To: <8928d4e90711210505w6d18d48y36d0f1b7ad39824d@mail.gmail.com> References: <348899050711200440g6a3e9630p13b2ebf55acfcf6d@mail.gmail.com> <348899050711200738g4d8dcfbfl8ff45b1b59f510b4@mail.gmail.com> <693bc9ab0711210444t231ef038ne2a12a0337b53ec3@mail.gmail.com> <8928d4e90711210505w6d18d48y36d0f1b7ad39824d@mail.gmail.com> Message-ID: <20071121070722.33c8ad2a.simon@arrowtheory.com> On Wed, 21 Nov 2007 14:05:34 +0100 "Martijn Faassen" wrote: ... > > You know what, I'll stop bothering you all together. > > Regards, > > Martijn > > [unsubscribed] Martijn, FWIW I have really appreciated your emails.. it's quite a lot of energy you have put into writing them. Please come back ! (I often rely on my friends to remind me of what I have said in the past...) Simon. From arigo at tunes.org Thu Nov 22 05:14:38 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 22 Nov 2007 05:14:38 +0100 Subject: [pypy-dev] naive VM question In-Reply-To: <473D68A3.9080908@spatialmedia.com> References: <473D68A3.9080908@spatialmedia.com> Message-ID: <20071122041438.GA14772@code0.codespeak.net> Hi Peter, On Fri, Nov 16, 2007 at 10:53:39PM +1300, Peter Fraser wrote: > Is "multiple isolated (but bridged) VM's in a single process" a > possible general technique? > > If multiple cooperating processes is a reasonable response to the > "single threaded VM meets multi-core architecture" problem -then is > "multiple VM's in a single process" a better one? This is probably yet another place where PyPy could go much more easily than CPython: multiple VMs in the same process which would by default not have any share state, but expose primitives to the Python programmer to explicitly share some objects. This is not trivial: there is some research and experimentation to be done here. Right now, the main issue with that is that it's definitely "yet another cool thing that we could do with PyPy". As was discussed at length, in the current situation the core team's feeling is that it needs to focus on providing a useful "baseline" Python interpreter. Someone else is welcome to jump in and play, though. A bientot, Armin. From arigo at tunes.org Thu Nov 22 05:31:58 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 22 Nov 2007 05:31:58 +0100 Subject: [pypy-dev] Very confused? -can someone at least tell me..? (was naive vm question) In-Reply-To: <47405A3C.4030203@spatialmedia.com> References: <47405A3C.4030203@spatialmedia.com> Message-ID: <20071122043158.GB14772@code0.codespeak.net> Hi Peter, On Mon, Nov 19, 2007 at 04:29:00AM +1300, Peter Fraser wrote: > 1. Suppose you ran the PyPy-C interpreter as a CPython extension. Could > the extension release the CPython GIL? Yes. The part of your argument which seems confusing, though, is that you don't really need to have one CPython and one PyPy in order to run Python code on two CPUs in parallel. You don't gain much (except confusion) by running one CPython and one PyPy for this purpose. (There are other reasons why you might want to do that, like using some special PyPy feature in a subpart of your CPython application.) In fact, the only restriction is that putting two CPythons in the same process is not really possible. Aside from that, you could do any of the following: * run several CPythons in two different processes and communicate using pipes, or shared memory, etc. * run several PyPys in either the same or different processes An advantage of PyPy over CPython in this domain is that because multiple PyPys can run in the same process you could experiment with communication models based on in-process shared data and locks. A bientot, Armin From cumber at netspace.net.au Thu Nov 22 06:48:49 2007 From: cumber at netspace.net.au (cumber at netspace.net.au) Date: Thu, 22 Nov 2007 16:48:49 +1100 Subject: [pypy-dev] naive VM question In-Reply-To: <20071122041438.GA14772@code0.codespeak.net> References: <473D68A3.9080908@spatialmedia.com> <20071122041438.GA14772@code0.codespeak.net> Message-ID: <1195710529.474518411d28e@webmail.netspace.net.au> First I should probably introduce myself. I've been lurking on the pypy-dev list for about 6 months now, ever since I discovered the project. I think PyPy looks like it has truly awe-inspiring potential, and I'm just about getting to the point where I am able to play around with it and maybe think about contributing a bit. Sadly sprints are a bit hard to get to from Australia though. So yes. Name's Ben. *tips hat* Quoting Armin Rigo : > Hi Peter, > > On Fri, Nov 16, 2007 at 10:53:39PM +1300, Peter Fraser wrote: > > Is "multiple isolated (but bridged) VM's in a single process" a > > possible general technique? > > > > If multiple cooperating processes is a reasonable response to the > > "single threaded VM meets multi-core architecture" problem -then is > > "multiple VM's in a single process" a better one? > > This is probably yet another place where PyPy could go much more easily > than CPython: multiple VMs in the same process which would by default > not have any share state, but expose primitives to the Python programmer > to explicitly share some objects. This is not trivial: there is some > research and experimentation to be done here. > > Right now, the main issue with that is that it's definitely "yet another > cool thing that we could do with PyPy". As was discussed at length, in > the current situation the core team's feeling is that it needs to focus > on providing a useful "baseline" Python interpreter. Someone else is > welcome to jump in and play, though. This is an area I would love to work on, but I think it's a bit beyond my capabilities just yet. I have a much more modest project I'd like to do with pypy more immediately. I was planning on posting here for advice after I've poked around a bit (haven't yet had time to really do anything since I did my first svn checkout the other day), in fact. I want to see if I can use pypy to make a python shell, both to become more familliar with pypy's internals and because I want one. :) But anyway, one of the things I would like to work on in the future if I get a bit more expertise is making pypy "multi-core friendly", so I guess if there are any ideas already floating around of lines of experimentation to try, I certainly would like to hear them. -- Ben Mellor ------------------------------------------------------------ This email was sent from Netspace Webmail: http://www.netspace.net.au From cfbolz at gmx.de Fri Nov 23 00:07:45 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 23 Nov 2007 00:07:45 +0100 Subject: [pypy-dev] [pypy-svn] r48962 - pypy/dist/pypy/translator/microbench In-Reply-To: <20071122230533.428D0823C@code0.codespeak.net> References: <20071122230533.428D0823C@code0.codespeak.net> Message-ID: <348899050711221507x57e05da9hcddd4390e44df432@mail.gmail.com> antocuni at codespeak.net wrote: > Author: antocuni > Date: Fri Nov 23 00:05:30 2007 > New Revision: 48962 > > Modified: > pypy/dist/pypy/translator/microbench/microbench.py > Log: > print the unexpected output in case of error Puh, thank you. This has been annoying me since forever. Cheers, Carl Friedrich From jcea at argo.es Fri Nov 23 01:42:11 2007 From: jcea at argo.es (Jesus Cea) Date: Fri, 23 Nov 2007 01:42:11 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: References: <200711131523.lADFNoWO025693@theraft.openend.se> <200711140353.18810.jacob@openend.se> <200711141224.lAECOfar016106@theraft.openend.se> <5b0248170711140604p24c75e6ftf03618dd1c8943ae@mail.gmail.com> Message-ID: <474621E3.2020906@argo.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martijn Faassen wrote: > Sure, you're right, that's true. See another mail where I give a better > description of what I mean with "done": something that motivates people > to use it in production. > > That said, I am of course also seriously interested in running existing > code on the PyPy platform. A lot of years ago I asked Armin Rigo about reactivating the virtual machine code in pysco. New versions of psyco only works in x86, but first ones were generating bytecodes for a low level virtual machine able to run in my Sparc's, for example. I was very impressed about th especialization thing. Five year ago, perhaps?. The answer was: "psyco is deprecated. Pypy is the future". Pypy project is impressive, but I can't yet benefice for psyco performance in my Sparcs. In fact I can't use pypy anything "real" yet. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR0Yh45lgi5GaxT1NAQKeswP+Lat7wMN+Dd1W7JLGeuc4UHNA8YQ0VB2l Ks7Bek6mYNGYeQx7kbEGxLMPQieQWmXUHTfSpbiXDR54sj9jwz/MWkZbCdmaycZS byXXutF6cZ+NrjrTTbpd9ftig/c9dasTJBKyiOda4O2knesK1n5/61VvVNbwy7D6 +PT6i7ZJCBw= =Pvb9 -----END PGP SIGNATURE----- From jacob at openend.se Fri Nov 23 02:43:57 2007 From: jacob at openend.se (Jacob =?iso-8859-1?q?Hall=E9n?=) Date: Fri, 23 Nov 2007 02:43:57 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <474621E3.2020906@argo.es> References: <474621E3.2020906@argo.es> Message-ID: <200711230243.57493.jacob@openend.se> fredagen den 23 november 2007 skrev Jesus Cea: > Martijn Faassen wrote: > > Sure, you're right, that's true. See another mail where I give a better > > description of what I mean with "done": something that motivates people > > to use it in production. > > > > That said, I am of course also seriously interested in running existing > > code on the PyPy platform. > > A lot of years ago I asked Armin Rigo about reactivating the virtual > machine code in pysco. New versions of psyco only works in x86, but > first ones were generating bytecodes for a low level virtual machine > able to run in my Sparc's, for example. I was very impressed about th > especialization thing. > > Five year ago, perhaps?. > > The answer was: "psyco is deprecated. Pypy is the future". > > Pypy project is impressive, but I can't yet benefice for psyco > performance in my Sparcs. In fact I can't use pypy anything "real" yet. I don't think people who look at the PyPy project from the outside understand what a tremendous undertaking it is. Psyco was an effort to make Python fast. Stackless was an effort to add extra flexibility. Jython was made to support a different platform. PyPy is working on doing all this and add much greater maintainability in the face of language changes on top of that. This is a much harder problem than the sum of the 4 aspects and it takes a huge effort to get it right. We currently have an estimate that it will take another 82 man months before we have a version that is fully mature and a viable replecement for CPython. A reasonable guess is that we would see the equivalent of approximately one full time worker on the project if we can't pay people to do the work. This would mean that we would be done in 7 years. Not a happy prospect. With funding, there are enough people with relevant understanding of the project who could tackle the remaining tasks and be done in about one year. This is why we are currently focusing on cleaning up the code base, so that we are presentable to prospective paying customers, rather than forging ahead with more JIT work, or targetting yet another processor family. If you want JIT optimizations for the Sparc, your best option is to look at the work already done for the x86 and the PPC and do the same code generation for the Sparc processor. Your second best option is to find someone with in-depth knowledge about code generation for the Sparc to do it. Your third best option is to find someone who will pay members of the PyPy project to do the work. Otherwise I'm afraid it will have a priority which is lower than the tasks included in the 82 man months. Best regards Jacob Hall?n From fijall at gmail.com Fri Nov 23 14:57:59 2007 From: fijall at gmail.com (Maciej Fijalkowski) Date: Fri, 23 Nov 2007 14:57:59 +0100 Subject: [pypy-dev] pypy on Tim Bray's blog In-Reply-To: <200711230243.57493.jacob@openend.se> References: <474621E3.2020906@argo.es> <200711230243.57493.jacob@openend.se> Message-ID: <693bc9ab0711230557q52a35fc9sf7fd4a93bce8c3f6@mail.gmail.com> > > > We currently have an estimate that it will take another 82 man months > before > we have a version that is fully mature and a viable replecement for > CPython. > > A reasonable guess is that we would see the equivalent of approximately > one > full time worker on the project if we can't pay people to do the work. > This > would mean that we would be done in 7 years. Not a happy prospect. I would rather count stuff from slightly more unrealistic position and would say that we're rather likely to produce reasonable results in roughly 2 years, but that's just me :) Cheers, fijal -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.de Fri Nov 23 19:12:08 2007 From: holger at merlinux.de (holger krekel) Date: Fri, 23 Nov 2007 19:12:08 +0100 Subject: [pypy-dev] 82 person months [was: Re: pypy on Tim Bray's blog] In-Reply-To: <200711230243.57493.jacob@openend.se> References: <474621E3.2020906@argo.es> <200711230243.57493.jacob@openend.se> Message-ID: <20071123181208.GN7368@solar.trillke> Hi Jacob, all! On Fri, Nov 23, 2007 at 02:43 +0100, Jacob Hall?n wrote: > We currently have an estimate that it will take another 82 man months before > we have a version that is fully mature and a viable replecement for CPython. Could you detail how it gets to this number 82? Which tasks are considered? And which persons performing the months? best, holger From simon at arrowtheory.com Sun Nov 25 19:33:19 2007 From: simon at arrowtheory.com (Simon Burton) Date: Sun, 25 Nov 2007 10:33:19 -0800 Subject: [pypy-dev] rctypes RIP Message-ID: <20071125103319.bc091265.simon@arrowtheory.com> Just for the record can someone say what was the revision number where rctypes was removed ? I still have code that uses this. thanks, Simon. From cfbolz at gmx.de Mon Nov 26 19:04:42 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Mon, 26 Nov 2007 19:04:42 +0100 Subject: [pypy-dev] rctypes RIP In-Reply-To: <20071125103319.bc091265.simon@arrowtheory.com> References: <20071125103319.bc091265.simon@arrowtheory.com> Message-ID: <474B0ABA.4050303@gmx.de> Hi Simon, RIP = rest in pieces? :-) Simon Burton wrote: > Just for the record can someone say what was > the revision number where rctypes was removed ? > I still have code that uses this. 49048, I think. Cheers, Carl Friedrich From niko at alum.mit.edu Thu Nov 29 19:46:21 2007 From: niko at alum.mit.edu (Niko Matsakis) Date: Thu, 29 Nov 2007 19:46:21 +0100 Subject: [pypy-dev] is anyone actively hacking on the JVM backend? Message-ID: I have been making small improvements to the JVM backend: I am aiming towards eliminating any remaining conformance problems, and also towards fixing a few warts in the implementation. Right now I am wringing out the last few bugs from a check-in that would allow multiple PyPy-generated interpreters to be used simultaneously. Currently, the code uses a few static fields that would conflict if, for example, someone tried to load both a Python and a Smalltalk interpreter into the same JVM. These static fields were used to allow the helper code (src/PyPy.java and friends) to create instances of exception classes and other RPython-generated code. The new code creates a single instance of the PyPy helper class per generated interpreter and use non-static-methods instead, which should be better. Anyway, I just want to be sure I am not stepping on anyone's toes if I check this in, since it makes small changes all over the place to ensure that the proper PyPy object is pushed onto the stack, etc. Niko From fijall at gmail.com Thu Nov 29 20:15:36 2007 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 29 Nov 2007 20:15:36 +0100 Subject: [pypy-dev] is anyone actively hacking on the JVM backend? In-Reply-To: References: Message-ID: <693bc9ab0711291115r4de2ee54oca397c04e385720f@mail.gmail.com> As noone answers... I think amitReg is doing some CLI related stuff, which means you can stomp on each other on oosupport, but he's doing rather library integration, so sounds very unlikely. Besides, I'm not aware of anyone. Cheers, fija On Nov 29, 2007 7:46 PM, Niko Matsakis wrote: > I have been making small improvements to the JVM backend: I am aiming > towards eliminating any remaining conformance problems, and also > towards fixing a few warts in the implementation. > > Right now I am wringing out the last few bugs from a check-in that > would allow multiple PyPy-generated interpreters to be used > simultaneously. Currently, the code uses a few static fields that > would conflict if, for example, someone tried to load both a Python > and a Smalltalk interpreter into the same JVM. These static fields > were used to allow the helper code (src/PyPy.java and friends) to > create instances of exception classes and other RPython-generated > code. The new code creates a single instance of the PyPy helper class > per generated interpreter and use non-static-methods instead, which > should be better. > > Anyway, I just want to be sure I am not stepping on anyone's toes if I > check this in, since it makes small changes all over the place to > ensure that the proper PyPy object is pushed onto the stack, etc. > > > Niko > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From regmi.amit at gmail.com Thu Nov 29 20:24:33 2007 From: regmi.amit at gmail.com (amit) Date: Thu, 29 Nov 2007 14:24:33 -0500 Subject: [pypy-dev] is anyone actively hacking on the JVM backend? In-Reply-To: <693bc9ab0711291115r4de2ee54oca397c04e385720f@mail.gmail.com> References: <693bc9ab0711291115r4de2ee54oca397c04e385720f@mail.gmail.com> Message-ID: <474F11F1.3060503@gmail.com> Hi Niko, I am working on the "CLR" module --- ..pypy/module/clr/ Just trying to play around with "interp_clr.py" and "app_clr.py" in there. Hope your commits will make it more better. regards, AmitReg Maciej Fijalkowski wrote: > As noone answers... I think amitReg is doing some CLI related stuff, > which means you can stomp on each other on oosupport, but he's doing > rather library integration, so sounds very unlikely. Besides, I'm not > aware of anyone. > > Cheers, > fija > > On Nov 29, 2007 7:46 PM, Niko Matsakis > wrote: > > I have been making small improvements to the JVM backend: I am aiming > towards eliminating any remaining conformance problems, and also > towards fixing a few warts in the implementation. > > Right now I am wringing out the last few bugs from a check-in that > would allow multiple PyPy-generated interpreters to be used > simultaneously. Currently, the code uses a few static fields that > would conflict if, for example, someone tried to load both a Python > and a Smalltalk interpreter into the same JVM. These static fields > were used to allow the helper code (src/PyPy.java and friends) to > create instances of exception classes and other RPython-generated > code. The new code creates a single instance of the PyPy helper class > per generated interpreter and use non-static-methods instead, which > should be better. > > Anyway, I just want to be sure I am not stepping on anyone's toes if I > check this in, since it makes small changes all over the place to > ensure that the proper PyPy object is pushed onto the stack, etc. > > > Niko > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > > > > ------------------------------------------------------------------------ > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From paul.degrandis at gmail.com Thu Nov 29 21:04:31 2007 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Thu, 29 Nov 2007 15:04:31 -0500 Subject: [pypy-dev] is anyone actively hacking on the JVM backend? In-Reply-To: <474F11F1.3060503@gmail.com> References: <693bc9ab0711291115r4de2ee54oca397c04e385720f@mail.gmail.com> <474F11F1.3060503@gmail.com> Message-ID: <9c0bb8a00711291204v55ae528bj4fb08b056c3b2d2d@mail.gmail.com> I'm working on some stuff but it's been stalled for awhile because I didn't have Java6 on the mac. That changed yesterday, but I'm dealing with school.... so have at it! Let me know if I can help with anything! Paul On 11/29/07, amit wrote: > Hi Niko, > > I am working on the "CLR" module --- ..pypy/module/clr/ > Just trying to play around with "interp_clr.py" and "app_clr.py" in > there. > Hope your commits will make it more better. > > regards, > AmitReg > > > > Maciej Fijalkowski wrote: > > As noone answers... I think amitReg is doing some CLI related stuff, > > which means you can stomp on each other on oosupport, but he's doing > > rather library integration, so sounds very unlikely. Besides, I'm not > > aware of anyone. > > > > Cheers, > > fija > > > > On Nov 29, 2007 7:46 PM, Niko Matsakis > > wrote: > > > > I have been making small improvements to the JVM backend: I am aiming > > towards eliminating any remaining conformance problems, and also > > towards fixing a few warts in the implementation. > > > > Right now I am wringing out the last few bugs from a check-in that > > would allow multiple PyPy-generated interpreters to be used > > simultaneously. Currently, the code uses a few static fields that > > would conflict if, for example, someone tried to load both a Python > > and a Smalltalk interpreter into the same JVM. These static fields > > were used to allow the helper code (src/PyPy.java and friends) to > > create instances of exception classes and other RPython-generated > > code. The new code creates a single instance of the PyPy helper class > > per generated interpreter and use non-static-methods instead, which > > should be better. > > > > Anyway, I just want to be sure I am not stepping on anyone's toes if I > > check this in, since it makes small changes all over the place to > > ensure that the proper PyPy object is pushed onto the stack, etc. > > > > > > Niko > > _______________________________________________ > > pypy-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/pypy-dev > > > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > pypy-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/pypy-dev > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From rhamph at gmail.com Thu Nov 29 21:20:49 2007 From: rhamph at gmail.com (Adam Olsen) Date: Thu, 29 Nov 2007 13:20:49 -0700 Subject: [pypy-dev] naive VM question Message-ID: > From: Armin Rigo > > Hi Peter, > > On Fri, Nov 16, 2007 at 10:53:39PM +1300, Peter Fraser wrote: > > Is "multiple isolated (but bridged) VM's in a single process" a > > possible general technique? > > > > If multiple cooperating processes is a reasonable response to the > > "single threaded VM meets multi-core architecture" problem -then is > > "multiple VM's in a single process" a better one? > > This is probably yet another place where PyPy could go much more easily > than CPython: multiple VMs in the same process which would by default > not have any share state, but expose primitives to the Python programmer > to explicitly share some objects. This is not trivial: there is some > research and experimentation to be done here. This sounds similar to what I'm doing with CPython: normal objects are contained within a single Monitor, whereas shareable objects (explicitly given a thread-safe API) can be passed between. http://code.google.com/p/python-safethread/ -- Adam Olsen, aka Rhamphoryncus From niko at alum.mit.edu Thu Nov 29 21:52:26 2007 From: niko at alum.mit.edu (Niko Matsakis) Date: Thu, 29 Nov 2007 21:52:26 +0100 Subject: [pypy-dev] is anyone actively hacking on the JVM backend? In-Reply-To: <474F11F1.3060503@gmail.com> References: <693bc9ab0711291115r4de2ee54oca397c04e385720f@mail.gmail.com> <474F11F1.3060503@gmail.com> Message-ID: <97635AE3-75F4-4452-9805-740F8A992C47@alum.mit.edu> Hopefully, these changes will not particularly affect anyone not actively touching the jvm directory. The only changes in oosupport are the addition of various "prepare_" methods to the Generator class. These methods give the jvm backend an opportunity to push a this pointer here and there, but for those methods I added a default "do-nothing" implementation. For example, there is a "prepare_call_primitive()" which is called before the arguments for a primitive are pushed onto the stack. I think that the oosupport code could probably be refactored, though I'm not precisely sure how right now. Right now it is basically a thin layer that extracts some useful information from the 'op' objects and passes it to the generators; furthermore, it uses this stack-based API which doesn't give much information to the backend about why something is being pushed (i.e., is it a function argument? if so, to what function? etc). This is ok as far as it goes, but maybe there is something more elegant? As I said, I don't have any precise suggestions right now, but it's something to think about. Niko On Nov 29, 2007, at 8:24 PM, amit wrote: > Hi Niko, > > I am working on the "CLR" module --- ..pypy/module/clr/ > Just trying to play around with "interp_clr.py" and > "app_clr.py" in there. > Hope your commits will make it more better. > > regards, > AmitReg > > > > Maciej Fijalkowski wrote: >> As noone answers... I think amitReg is doing some CLI related >> stuff, which means you can stomp on each other on oosupport, but >> he's doing rather library integration, so sounds very unlikely. >> Besides, I'm not aware of anyone. >> >> Cheers, >> fija >> >> On Nov 29, 2007 7:46 PM, Niko Matsakis > >> wrote: >> >> I have been making small improvements to the JVM backend: I am >> aiming >> towards eliminating any remaining conformance problems, and also >> towards fixing a few warts in the implementation. >> >> Right now I am wringing out the last few bugs from a check-in that >> would allow multiple PyPy-generated interpreters to be used >> simultaneously. Currently, the code uses a few static fields that >> would conflict if, for example, someone tried to load both a >> Python >> and a Smalltalk interpreter into the same JVM. These static >> fields >> were used to allow the helper code (src/PyPy.java and friends) to >> create instances of exception classes and other RPython-generated >> code. The new code creates a single instance of the PyPy helper >> class >> per generated interpreter and use non-static-methods instead, >> which >> should be better. >> >> Anyway, I just want to be sure I am not stepping on anyone's >> toes if I >> check this in, since it makes small changes all over the place to >> ensure that the proper PyPy object is pushed onto the stack, etc. >> >> >> Niko >> _______________________________________________ >> pypy-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/pypy-dev >> >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> pypy-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/pypy-dev > From santagada at gmail.com Thu Nov 29 22:06:26 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Thu, 29 Nov 2007 19:06:26 -0200 Subject: [pypy-dev] is anyone actively hacking on the JVM backend? In-Reply-To: <97635AE3-75F4-4452-9805-740F8A992C47@alum.mit.edu> References: <693bc9ab0711291115r4de2ee54oca397c04e385720f@mail.gmail.com> <474F11F1.3060503@gmail.com> <97635AE3-75F4-4452-9805-740F8A992C47@alum.mit.edu> Message-ID: <97FD0105-ED1B-4FC3-BCF6-D59318636BAE@gmail.com> Em 29/11/2007, ?s 18:52, Niko Matsakis escreveu: > Hopefully, these changes will not particularly affect anyone not > actively touching the jvm directory. Why didn't you did all this changes on a branch, then at least people would be knowing what you were doing... not that me or anyone was messing with jvm, but we could also follow the development on pypy-svn. -- Leonardo Santagada From fijal at genesilico.pl Thu Nov 29 22:33:10 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Thu, 29 Nov 2007 22:33:10 +0100 Subject: [pypy-dev] is anyone actively hacking on the JVM backend? In-Reply-To: <97FD0105-ED1B-4FC3-BCF6-D59318636BAE@gmail.com> References: <693bc9ab0711291115r4de2ee54oca397c04e385720f@mail.gmail.com> <474F11F1.3060503@gmail.com> <97635AE3-75F4-4452-9805-740F8A992C47@alum.mit.edu> <97FD0105-ED1B-4FC3-BCF6-D59318636BAE@gmail.com> Message-ID: <474F3016.5050508@genesilico.pl> Leonardo Santagada wrote: > Em 29/11/2007, ?s 18:52, Niko Matsakis escreveu: > > >> Hopefully, these changes will not particularly affect anyone not >> actively touching the jvm directory. >> > > Why didn't you did all this changes on a branch, then at least people > would be knowing what you were doing... not that me or anyone was > messing with jvm, but we could also follow the development on pypy-svn. > > -- > Leonardo Santagada > I suppose for exactly this reason that noone is doing anything on the jvm directory anyway. Cheers, fijal :. From niko at alum.mit.edu Thu Nov 29 22:49:20 2007 From: niko at alum.mit.edu (Niko Matsakis) Date: Thu, 29 Nov 2007 22:49:20 +0100 Subject: [pypy-dev] is anyone actively hacking on the JVM backend? In-Reply-To: <474F3016.5050508@genesilico.pl> References: <693bc9ab0711291115r4de2ee54oca397c04e385720f@mail.gmail.com> <474F11F1.3060503@gmail.com> <97635AE3-75F4-4452-9805-740F8A992C47@alum.mit.edu> <97FD0105-ED1B-4FC3-BCF6-D59318636BAE@gmail.com> <474F3016.5050508@genesilico.pl> Message-ID: <13AB3BDF-C2C8-4E91-9682-26A353AE63D3@alum.mit.edu> Just realized I only sent my reply to Leonardo. I may have made out these changes as more than they are: it's just the results of the last few hours work... anyway, I am mostly finished testing now and nothing seems to be broken. :) Niko On Nov 29, 2007, at 10:33 PM, Maciek Fijalkowski wrote: > Leonardo Santagada wrote: >> Em 29/11/2007, ?s 18:52, Niko Matsakis escreveu: >> >> >>> Hopefully, these changes will not particularly affect anyone not >>> actively touching the jvm directory. >>> >> >> Why didn't you did all this changes on a branch, then at least >> people would be knowing what you were doing... not that me or >> anyone was messing with jvm, but we could also follow the >> development on pypy-svn. >> >> -- >> Leonardo Santagada >> > I suppose for exactly this reason that noone is doing anything on > the jvm directory anyway. > > Cheers, > fijal > > > > :. > From anto.cuni at gmail.com Fri Nov 30 01:10:00 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 30 Nov 2007 01:10:00 +0100 Subject: [pypy-dev] is anyone actively hacking on the JVM backend? In-Reply-To: References: Message-ID: <474F54D8.1020806@gmail.com> Niko Matsakis wrote: > Right now I am wringing out the last few bugs from a check-in that > would allow multiple PyPy-generated interpreters to be used > simultaneously. Currently, the code uses a few static fields that > would conflict if, for example, someone tried to load both a Python > and a Smalltalk interpreter into the same JVM. These static fields > were used to allow the helper code (src/PyPy.java and friends) to > create instances of exception classes and other RPython-generated > code. The new code creates a single instance of the PyPy helper class > per generated interpreter and use non-static-methods instead, which > should be better. I think this new feature is very cool. Is the logic for all the InterLink stuff in jvm or in oosupport? I think it would be nice to reuse the same ideas also for the cli backend. ciao Anto From niko at alum.mit.edu Fri Nov 30 09:30:03 2007 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 30 Nov 2007 09:30:03 +0100 Subject: [pypy-dev] is anyone actively hacking on the JVM backend? In-Reply-To: <474F54D8.1020806@gmail.com> References: <474F54D8.1020806@gmail.com> Message-ID: >> Right now I am wringing out the last few bugs from a check-in that >> would allow multiple PyPy-generated interpreters to be used >> simultaneously. Currently, the code uses a few static fields that >> would conflict if, for example, someone tried to load both a >> Python and a Smalltalk interpreter into the same JVM. These >> static fields were used to allow the helper code (src/PyPy.java >> and friends) to create instances of exception classes and other >> RPython-generated code. The new code creates a single instance of >> the PyPy helper class per generated interpreter and use non-static- >> methods instead, which should be better. > > I think this new feature is very cool. Is the logic for all the > InterLink stuff in jvm or in oosupport? I think it would be nice to > reuse the same ideas also for the cli backend. Right now it's mostly in the JVM backend, though I had to insert a few hooks into oosupport. I wasn't sure whether the CLI had the same "interlinking" problem. I can look into whether more of the code could be pulled into oosupport, but if I recall the two main places I had to add hooks were: 1. When rewrite the opcode table, I check now if the action for an opcode is a virtual method: if so, instead of rewriting it to "PushAllArgs, Invoke Method, Store Result," I rewrite it to: "Push PyPy object, PushAllArgs, Invoke Method, Store Result." 2. I added prepare_ hooks for oostring, oounicode, and primitives, which basically just push the receiver object. We could probably pull more into oosupport, particularly if it gained the ability to talk about static fields and things. Niko From anto.cuni at gmail.com Fri Nov 30 10:38:00 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 30 Nov 2007 10:38:00 +0100 Subject: [pypy-dev] is anyone actively hacking on the JVM backend? In-Reply-To: References: <474F54D8.1020806@gmail.com> Message-ID: <474FD9F8.4050905@gmail.com> Niko Matsakis wrote: > Right now it's mostly in the JVM backend, though I had to insert a few > hooks into oosupport. I wasn't sure whether the CLI had the same > "interlinking" problem. yes, the problems are nearly the same for the cli backend; basically, we want: 1) to raise RPython exception from C#/Java code 2) to create records with a specific set of fields Right now the solution for (1) is very hackish: I have a stub file called main.exe which contains dummy functions to raise exceptions; pypylib.dll is compiled against main.exe, but then we compile the real executable so at runtime pypylib.dll is linked to a new main.exe which contains functions with the same name&signature as the dummy ones. Basically, we have a circular dependency between pypylib.dll and main.exe; believe it or not, in .NET it works :-). I know, this solution is very ugly and moreover it forces the executable to be named main.exe. Interlink sound a much better approach :-). For (2), the current solution is to write the records we want directly in C#/Java, and let the backend to special case those when rendering the name. Unfortunately, I can't see a general solution here; the interlink approach works only for records which you only need to create and not to manipulate, but e.g. it can't work for StatResult because you still need to set individual fields after its creation. I guess this is the cause why StatResult is still implemented "the old way" in genjvm. > I can look into whether more of the code could > be pulled into oosupport, but if I recall the two main places I had to > add hooks were: > > 1. When rewrite the opcode table, I check now if the action for an > opcode is a virtual method: if so, instead of rewriting it to > "PushAllArgs, Invoke Method, Store Result," I rewrite it to: "Push PyPy > object, PushAllArgs, Invoke Method, Store Result." Uhm... sorry, I can't see why this is necessary; could you explain a bit more in detail please? > 2. I added prepare_ hooks for oostring, oounicode, and primitives, which > basically just push the receiver object. > > We could probably pull more into oosupport, particularly if it gained > the ability to talk about static fields and things. That would be nice; it would be a nice sprint topic, if only there were a sprint planned soon :-). ciao Anto From cfbolz at gmx.de Fri Nov 30 16:18:47 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 30 Nov 2007 16:18:47 +0100 Subject: [pypy-dev] svn statistics Message-ID: <348899050711300718s5b74fa9wd03d557eb37912b0@mail.gmail.com> Hi all! I found a reasonably nice analysis tool for SVN repositories and set it loose on pypy-dist. The result can be found here: http://wyvern.cs.uni-duesseldorf.de/~cfbolz/pypystats/ It's not yet perfect, I should tell it to exclude unicodedata. However, it seems to be able to deal with our strange merging patterns and is much better than my hacked up scripts, so I tend to keep using this after some tweaking. Any suggestions on improvements? The program used is this: http://www.statsvn.org/ Cheers, Carl Friedrich From niko at alum.mit.edu Fri Nov 30 19:13:32 2007 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 30 Nov 2007 19:13:32 +0100 Subject: [pypy-dev] svn statistics In-Reply-To: <348899050711300718s5b74fa9wd03d557eb37912b0@mail.gmail.com> References: <348899050711300718s5b74fa9wd03d557eb37912b0@mail.gmail.com> Message-ID: Man, I got to write more code! I weigh in at only 0.8% of the codebase! ;) I particularly like this "churn" chart: I guess you can see some major refactorings? Or more likely imports of large chunks of code... Niko On Nov 30, 2007, at 4:18 PM, Carl Friedrich Bolz wrote: > Hi all! > > I found a reasonably nice analysis tool for SVN repositories and set > it loose on pypy-dist. The result can be found here: > > http://wyvern.cs.uni-duesseldorf.de/~cfbolz/pypystats/ > > It's not yet perfect, I should tell it to exclude unicodedata. > However, it seems to be able to deal with our strange merging patterns > and is much better than my hacked up scripts, so I tend to keep using > this after some tweaking. Any suggestions on improvements? > > The program used is this: http://www.statsvn.org/ > > Cheers, > > Carl Friedrich > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From van.lindberg at gmail.com Fri Nov 30 23:05:55 2007 From: van.lindberg at gmail.com (VanL) Date: Fri, 30 Nov 2007 16:05:55 -0600 Subject: [pypy-dev] PyPy and GHOP Message-ID: <47508943.3030803@gmail.com> For those who are not aware, Google just launched the Google Highly Open Participation (GHOP) project. GHOP introduces high-school-age kids to Open Source by giving them small but meaningful tasks to do. I have noticed at least one PyPy-related task proposal for GHOP, but I think that there is more opportunity here. Specifically, some of the "production-ready" threads noted that it is difficult to get a PyPy environment set up. Could that be a task? How about an example of translating a function or module to rpython? How about actual translations of simple parts of the standard library - maybe a function at a time? PyPy is well suited to providing GHOP tasks, because there are lots of interesting bits of code that could be translated into rpython. That is a little bit more substantial than simple documentation tasks, but still easy enough to do (if the translation is of single functions or a small group of related functions). What do you all think? From fijal at genesilico.pl Fri Nov 30 23:58:57 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Fri, 30 Nov 2007 23:58:57 +0100 Subject: [pypy-dev] PyPy and GHOP In-Reply-To: <47508943.3030803@gmail.com> References: <47508943.3030803@gmail.com> Message-ID: <475095B1.6000605@genesilico.pl> VanL wrote: > For those who are not aware, Google just launched the Google Highly Open > Participation (GHOP) project. GHOP introduces high-school-age kids to > Open Source by giving them small but meaningful tasks to do. > > I have noticed at least one PyPy-related task proposal for GHOP, but I > think that there is more opportunity here. > > Specifically, some of the "production-ready" threads noted that it is > difficult to get a PyPy environment set up. Could that be a task? > > How about an example of translating a function or module to rpython? How > about actual translations of simple parts of the standard library - > maybe a function at a time? > > PyPy is well suited to providing GHOP tasks, because there are lots of > interesting bits of code that could be translated into rpython. That is > a little bit more substantial than simple documentation tasks, but still > easy enough to do (if the translation is of single functions or a small > group of related functions). > > What do you all think? > > That's me who submitted it, sorry for not notifying pypy-dev earlier. I think so as well :-) Cheers, fijal :.