From arigo at tunes.org Tue Nov 2 17:19:05 2004 From: arigo at tunes.org (Armin Rigo) Date: Tue, 2 Nov 2004 16:19:05 +0000 Subject: [pypy-dev] More on optimization In-Reply-To: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> Message-ID: <20041102161905.GA30013@vicky.ecs.soton.ac.uk> Hi Samuele, So LLVM comes again under the focus; we should really give it a serious try at some point. There are clearly two point of views that we can take on RPython. The first point of view is that RPython is a kind of nice syntax over a C- or Java-like language (in this view integers are primitive, and type inference is used to fill in the missing type declarations). The alternative is to see RPython as semantically close to Python, and all the variables contain objects, but we can obtain better performance via optimization. The current annotation-and-Pyrex-or-Lisp approach follows the 1st approach, but I feel like the 2nd one can give better results. Well, I guess you guessed :-) Let me try to explore a few consequences of such a choice. A general critique I have regarding a lot of research (including the paper you cited) is that the techniques are flow-insensitive. I'm not sure if it is a problem for the 1st approach but it is for the 2nd one: the low-level flow graph of, say, "a+b" is: if type(a) is int and type(b) is int: x = make_int_object(add_long(a.ob_ival, b.ob_ival)) else: ... It is essential to follow the exact control path in this function, which tells us that a and b only have a field 'ob_ival' to read when they are actually of type int. There are still good ideas to consider from the paper you cited (which references did you think would be most relevant?). One is that this algorithm is nicely context-sensitive (i.e. it is almost like all the functions were systematically inlined, which is great for optimization -- but which is impossible in practice, so this algorithm gives an efficient way to compute what *would* occur if all functions were inlined). This looks like an alternative to the traditional way of making functions polymorphic (where we try to guess, when a function gets called with various kinds of arguments, if we should analyse the function several times with each specialized set of arguments or if we should generalize and analyse the function only once). I tend now to think that the "structure-inlining" problem is best attacked at the more general level, independently of RPython. It's quite possible that it could be done with an adaptation of the techniques that we already have in the annotation subdirectory. It would be simpler because we have fewer, more elementary types. It would be harder because we'd have to focus on the basic problem of tracking references more precisely. Trying to sketch the basics by putting together some equivalents of the SomeXxx classes and factories, it looks like it works... (but some more thinking needed) Re function pointers: in Java we'd probably have to group functions by families (two functions are in the same family as soon as there is a single variable that could point to either one), and then replace function pointers by integers and use switches... Lots of one-method classes with a common interface look like a waste... But the integer tricks seems independent of the optimization techniques. Did you have better tricks in mind that would require more care? Armin From pedronis at bluewin.ch Tue Nov 2 18:09:57 2004 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Tue, 02 Nov 2004 18:09:57 +0100 Subject: [pypy-dev] More on optimization In-Reply-To: <20041102161905.GA30013@vicky.ecs.soton.ac.uk> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> Message-ID: <4187BF65.4090906@bluewin.ch> Armin Rigo wrote: > Hi Samuele, > > So LLVM comes again under the focus; we should really give it a serious try at > some point. > > There are clearly two point of views that we can take on RPython. The first > point of view is that RPython is a kind of nice syntax over a C- or Java-like > language (in this view integers are primitive, and type inference is used to > fill in the missing type declarations). The alternative is to see RPython as > semantically close to Python, and all the variables contain objects, but we > can obtain better performance via optimization. The current > annotation-and-Pyrex-or-Lisp approach follows the 1st approach, but I feel > like the 2nd one can give better results. Well, I guess you guessed :-) > I see, I just think that maybe we should try to get as far as possible with what we constructed so far, for example even ignoring structure inlining for the first approx. In Java, lisp etc is not that much a relevant problem because for example Java arrays carry a length anyway and are heap allocated. One thing with approach 2 is that we need to rewrite a fair chunk of Python semantics themself in the compiler that way. I'm thinking that as long as we depend/use on CPython ref counting, ownership issues will crop up, they will likely not mix easely with trying to be clever with structures. In general I think that likely both 1 and 2 are worth pursuing, because of differences for target languages, characheteristic of produced code. OTOH I'm sure whether trying to go with full force with 2 is the best thing to get the first protototype interpreter running as an extension in CPython, especially thinking about ref counting. But you worked more on that, so you can best judge what is necessary. > I tend now to think that the "structure-inlining" problem is best attacked at > the more general level, independently of RPython. It's quite possible that it > could be done with an adaptation of the techniques that we already have in the > annotation subdirectory. It would be simpler because we have fewer, more > elementary types. It would be harder because we'd have to focus on the basic > problem of tracking references more precisely. Trying to sketch the basics by > putting together some equivalents of the SomeXxx classes and factories, it > looks like it works... (but some more thinking needed) yes, that is what I was thinking about, you need to track "types" for thing like object/structure from a specific creation point or coming from reading a specific field and track their propagation. > Re function pointers: in Java we'd probably have to group functions by > families (two functions are in the same family as soon as there is a single > variable that could point to either one), and then replace function pointers > by integers and use switches... Lots of one-method classes with a common > interface look like a waste... But the integer tricks seems independent of > the optimization techniques. Did you have better tricks in mind that would > require more care? > Jython right now use the switch thing in some cases, but I suspect it is not JIT inlining friendly so a lot of inner classes are preferable. From pedronis at bluewin.ch Tue Nov 2 18:16:14 2004 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Tue, 02 Nov 2004 18:16:14 +0100 Subject: [pypy-dev] More on optimization In-Reply-To: <4187BF65.4090906@bluewin.ch> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> <4187BF65.4090906@bluewin.ch> Message-ID: <4187C0DE.9080008@bluewin.ch> Samuele Pedroni wrote: > > OTOH I'm sure whether trying to go with full force with 2 is the best oops, I'm not sure > thing to get the first protototype interpreter running as an extension > in CPython, especially thinking about ref counting. > From hpk at trillke.net Tue Nov 2 19:36:53 2004 From: hpk at trillke.net (holger krekel) Date: Tue, 2 Nov 2004 19:36:53 +0100 Subject: [pypy-dev] More on optimization In-Reply-To: <20041102161905.GA30013@vicky.ecs.soton.ac.uk> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> Message-ID: <20041102183653.GP15456@solar.trillke.net> Hi Armin, hi Samuele, [Armin Rigo Tue, Nov 02, 2004 at 04:19:05PM +0000] > So LLVM comes again under the focus; we should really give it a serious try at > some point. indeed. We may also give joeq (http://joeq.sourceforge.net) a look although development seems to be stalled. > There are clearly two point of views that we can take on RPython. The first > point of view is that RPython is a kind of nice syntax over a C- or Java-like > language (in this view integers are primitive, and type inference is used to > fill in the missing type declarations). The alternative is to see RPython as > semantically close to Python, and all the variables contain objects, but we > can obtain better performance via optimization. The current > annotation-and-Pyrex-or-Lisp approach follows the 1st approach, but I feel > like the 2nd one can give better results. Well, I guess you guessed :-) To me the pyrex approach was just a short-cut to produce what your new C-backend now produces directly. > Let me try to explore a few consequences of such a choice. A general critique > I have regarding a lot of research (including the paper you cited) is that the > techniques are flow-insensitive. the choice of the backend should not influence the flow/annotation analysis, should it? I guess i don't see the connection you seem to imply between these two parts of the translation process. > I'm not sure if it is a problem for the 1st > approach but it is for the 2nd one: the low-level flow graph of, say, "a+b" > is: > > if type(a) is int and type(b) is int: > x = make_int_object(add_long(a.ob_ival, b.ob_ival)) > else: > ... > > It is essential to follow the exact control path in this function, which tells > us that a and b only have a field 'ob_ival' to read when they are actually of > type int. Does this relate to a previous idea of generating two flow branches for the above if/else that would not get merged at the next point after the if/else? IIRC, we currently merge the "framestates" right after the if/else. cheers, holger From arigo at tunes.org Wed Nov 3 15:10:21 2004 From: arigo at tunes.org (Armin Rigo) Date: Wed, 3 Nov 2004 14:10:21 +0000 Subject: [pypy-dev] More on optimization In-Reply-To: <20041102183653.GP15456@solar.trillke.net> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> <20041102183653.GP15456@solar.trillke.net> Message-ID: <20041103141021.GA30865@vicky.ecs.soton.ac.uk> Hi Holger, On Tue, Nov 02, 2004 at 07:36:53PM +0100, holger krekel wrote: > To me the pyrex approach was just a short-cut to produce what your new > C-backend now produces directly. Sure, but the issue under discussion is the next step: how type information should be used. The C-backend doesn't use them now. > the choice of the backend should not influence the flow/annotation analysis, > should it? I guess i don't see the connection you seem to imply > between these two parts of the translation process. There are two options. What I called "option 1" is to use the existing annotation analysis, and then make use of this information in the backends. This is a good approach for generating Java code, but maybe not so much for C, where "option 2" would give better results. This "option 2" would not use the existing annotation analysis at all. Instead, it would replace the (unannotated) high-level operations with low-level operations and then do a slightly different kind of analysis on these low-level operations, and finally the back-end would turn that into C. As far as I can tell, only "option 2" can give the best results to translate, say, our W_ListObject class, with its 'ob_item' field containing a list that is "owned" by the W_ListObject instance and never visible to other parts of the code. > > if type(a) is int and type(b) is int: > > x = make_int_object(add_long(a.ob_ival, b.ob_ival)) > > else: > > ... > > Does this relate to a previous idea of generating two flow branches for > the above if/else that would not get merged at the next point > after the if/else? IIRC, we currently merge the "framestates" right > after the if/else. No, it's not about merging or not the two flow branches (or the results of the analysis) after the if/else: in both cases, flow-sensitive or flow-insensitive, they are merged. The difference between flow-sensitive and flow-insensitive is that in the former case, if we discover that we know the types of a and b statically, then we only explore the first branch (so there is no merging to be done, because the 2nd branch is never analysed). This is what we do currently in our annotations. By contrast, flow-insensitive algorithms like the one in the paper Samuele pointed out (or the one done by Starkiller) don't work like that: they just look at all instructions in the function and accumulate their possible effects, without worrying about the control flow. A bient?t, Armin From hpk at trillke.net Wed Nov 3 15:28:42 2004 From: hpk at trillke.net (holger krekel) Date: Wed, 3 Nov 2004 15:28:42 +0100 Subject: [pypy-dev] More on optimization In-Reply-To: <20041103141021.GA30865@vicky.ecs.soton.ac.uk> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> <20041102183653.GP15456@solar.trillke.net> <20041103141021.GA30865@vicky.ecs.soton.ac.uk> Message-ID: <20041103142842.GY15456@solar.trillke.net> Hi Armin, [Armin Rigo Wed, Nov 03, 2004 at 02:10:21PM +0000] > On Tue, Nov 02, 2004 at 07:36:53PM +0100, holger krekel wrote: > > > if type(a) is int and type(b) is int: > > > x = make_int_object(add_long(a.ob_ival, b.ob_ival)) > > > else: > > > ... > > > > Does this relate to a previous idea of generating two flow branches for > > the above if/else that would not get merged at the next point > > after the if/else? IIRC, we currently merge the "framestates" right > > after the if/else. > > No, it's not about merging or not the two flow branches (or the results of the > analysis) after the if/else: in both cases, flow-sensitive or > flow-insensitive, they are merged. The difference between flow-sensitive and > flow-insensitive is that in the former case, if we discover that we know the > types of a and b statically, then we only explore the first branch (so there > is no merging to be done, because the 2nd branch is never analysed). This is > what we do currently in our annotations. So we do already keep possibly different function versions for differently typed sets of input parameters to this function? I guess i am mixing up flow analysis and annotation somewhat here. Or even have a deeper misunderstanding :-) > By contrast, flow-insensitive algorithms like the one in the > paper Samuele pointed out (or the one done by Starkiller) > don't work like that: they just look at all instructions in > the function and accumulate their possible effects, without > worrying about the control flow. right. They have to do that because they look at the static source code and not at a representation obtained from abstract execution like we do in objspace/flow. cheers, holger From arigo at tunes.org Wed Nov 3 15:37:53 2004 From: arigo at tunes.org (Armin Rigo) Date: Wed, 3 Nov 2004 14:37:53 +0000 Subject: [pypy-dev] More on optimization In-Reply-To: <4187BF65.4090906@bluewin.ch> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> <4187BF65.4090906@bluewin.ch> Message-ID: <20041103143753.GB30865@vicky.ecs.soton.ac.uk> Hi Samuele, On Tue, Nov 02, 2004 at 06:09:57PM +0100, Samuele Pedroni wrote: > In general I think that likely both 1 and 2 are worth pursuing, because > of differences for target languages, characheteristic of produced code. Makes sense. We must set priorities, and trying to go as far as possible with the existing approach certainly looks like a good idea. This means that we shouldn't put too many optimization efforts on the current C backend. I suggest that the next goal would be to get something that works, even slowly, and try to keep the Lisp backend up-to-date. I think that the type annotations are not used much in the Lisp backend so far, but it should be easy to generate type declarations. It might be a good way to get a "fast enough" translation. > I see, I just think that maybe we should try to get as far as possible > with what we constructed so far, for example even ignoring structure > inlining for the first approx. In Java, lisp etc is not that much a > relevant problem because for example Java arrays carry a length anyway > and are heap allocated. Indeed, it looks like structure inlining ("option 2") is not really useful for all target languages. > One thing with approach 2 is that we need to rewrite a fair chunk of > Python semantics themself in the compiler that way. Yes, though these semantics have to be somewhere anyway; currently they are distributed among the type inference, the backend itself, and in manually written support code for the target language (i.e. the C macro file genc.h, or the snippets of Lisp code included in gencl.py). Approach 2 would put them in a central place, making the meta-programming Psyco-ish approach more explicit. > I'm thinking that as long as we depend/use on CPython ref counting, > ownership issues will crop up, they will likely not mix easely with > trying to be clever with structures. I don't think that reference counting is a major problem. The way I see approach 2 so far is that we'd have basically two kinds of annotations for the low-level graphs, SomeInlinedStructure and SomeHeapStructure. We'd use the former as long as possible, i.e. as long as we can track all references to the structure; and when we no longer can, we mark the structure's creation point as "too complex" and reflow from there using SomeHeapStructure instead. Only the latter would have reference counters and a CPython-compatible layout in memory. > OTOH I'm sure whether trying to go with full force with 2 is the best > thing to get the first protototype interpreter running as an extension > in CPython, especially thinking about ref counting. I agree with you now for the reasons above, even if I think that the refcounts are not a major issue (but maybe I'm overlooking something). > Jython right now use the switch thing in some cases, but I suspect it is > not JIT inlining friendly so a lot of inner classes are preferable. Good to know. Well, I guess we'll have to try both and compare, anyway. A bient?t, Armin From arigo at tunes.org Wed Nov 3 15:50:46 2004 From: arigo at tunes.org (Armin Rigo) Date: Wed, 3 Nov 2004 14:50:46 +0000 Subject: [pypy-dev] More on optimization In-Reply-To: <20041103142842.GY15456@solar.trillke.net> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> <20041102183653.GP15456@solar.trillke.net> <20041103141021.GA30865@vicky.ecs.soton.ac.uk> <20041103142842.GY15456@solar.trillke.net> Message-ID: <20041103145046.GA18341@vicky.ecs.soton.ac.uk> Hi Holger, On Wed, Nov 03, 2004 at 03:28:42PM +0100, holger krekel wrote: > So we do already keep possibly different function versions for > differently typed sets of input parameters to this function? Er, no... > I guess i am mixing up flow analysis and annotation somewhat > here. Or even have a deeper misunderstanding :-) Hum. We have the flow analysis which, using merge point detection, produces a flow graph that follows the control flow of the input functions (one graph per function). This works fine and we don't plan to change anything there. Then we have the annotation phase which follows by abstract interpretation the existing flow graphs, and which merges the dicovered information exactly when two branches meet in flow graph. This is a "flow-sensitive" kind of analysis. It is good because in this kind of code: if x: y = 1 else: y = 2 when the annotation follows the flow graph of this piece of code, if 'x' is not a constant, then both paths are followed and when the two paths merge, the two incompatible results ("y is the constant 1" and "y is the constant 2") are merged into less precise information ("y is an integer"). But if 'x' is a constant, then only one path will be followed, and we keep the precise knowledge of the value of 'y' even after the if/else statement. Flow-insensitive algorithms just look at the whole function and see two statements: "y=1" and "y=2". From that they guess that 'y' must be an integer no matter what. They can't tell you the value of 'y' even if they knew the value of 'x'. This difference is in the algorithm used; it's not related to: > right. They have to do that because they look at the static > source code and not at a representation obtained from abstract > execution like we do in objspace/flow. No, that's not related. It's just that there are big advantages if you don't care about the precise flow for some kind of algorithms. We could also collect all the operations that appear in a flow graph and deduce things in a flow-insensitive manner if we wanted to; and conversely Starkiller could also look at the static source and still infer things in a more flow-sensitive way. A bient?t, Armin. From hpk at trillke.net Wed Nov 3 16:40:01 2004 From: hpk at trillke.net (holger krekel) Date: Wed, 3 Nov 2004 16:40:01 +0100 Subject: [pypy-dev] More on optimization In-Reply-To: <20041103145046.GA18341@vicky.ecs.soton.ac.uk> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> <20041102183653.GP15456@solar.trillke.net> <20041103141021.GA30865@vicky.ecs.soton.ac.uk> <20041103142842.GY15456@solar.trillke.net> <20041103145046.GA18341@vicky.ecs.soton.ac.uk> Message-ID: <20041103154000.GA27665@solar.trillke.net> Hi again, [Armin Rigo Wed, Nov 03, 2004 at 02:50:46PM +0000] > Hi Holger, > > On Wed, Nov 03, 2004 at 03:28:42PM +0100, holger krekel wrote: > > So we do already keep possibly different function versions for > > differently typed sets of input parameters to this function? > > Er, no... > > > I guess i am mixing up flow analysis and annotation somewhat > > here. Or even have a deeper misunderstanding :-) > > Hum. We have the flow analysis which, using merge point detection, produces a > flow graph that follows the control flow of the input functions (one graph per > function). This works fine and we don't plan to change anything there. > > Then we have the annotation phase which follows by abstract interpretation the > existing flow graphs, ... me hum, too :-) this means that http://codespeak.net/pypy/index.cgi?doc/architecture.html is not right when it says: The translation process is done in three steps: - producing a flow graph representation of the standard interpreter. A combination of a plain interpreter and a flow object space performs "abstract interpretation" to record the flow of objects and execution throughout a python program into such a flow graph. - ... because your current prescribed notion of abstract interpretation starts with already existing flow graphs. Hum. I am ready to accept your current use of the terms but it is confusing that we keep changing notions and terms quite a bit ... I guess that when i get confused about this that other people could get confused about this as well. > and which merges the dicovered information exactly when > two branches meet in flow graph. This is a "flow-sensitive" kind of > analysis. It is good because in this kind of code: > > if x: > y = 1 > else: > y = 2 > > when the annotation follows the flow graph of this piece of code, if 'x' is > not a constant, then both paths are followed and when the two paths merge, the > two incompatible results ("y is the constant 1" and "y is the constant 2") are > merged into less precise information ("y is an integer"). But if 'x' is a > constant, then only one path will be followed, and we keep the precise > knowledge of the value of 'y' even after the if/else statement. But if one part of the program uses the one branch and another part uses the other branch then you will generalize the function to annotate 'y' as being an integer. This was what i was refering to when i was suprisedly asking: So we do already keep possibly different function versions for differently typed sets of input parameters to this function? to which the answer apparently is "no". Only when just one branch is used will the flow-sensitive annotation analysis regard 'y' as being a constant. > > right. They have to do that because they look at the static > > source code and not at a representation obtained from abstract > > execution like we do in objspace/flow. > > No, that's not related. It's just that there are big advantages if you don't > care about the precise flow for some kind of algorithms. We could also > collect all the operations that appear in a flow graph and deduce things in a > flow-insensitive manner if we wanted to; and conversely Starkiller could also > look at the static source and still infer things in a more flow-sensitive way. But in order to do that starkiller would probably need to utilize a representation that is reflecting the actual possible flows of execution. I didn't intend to claim that looking at source code couldn't be done in a flow-sensitive manner. Such a claim wouldn't make much sense as source code and its execution flow are mostly equivalent. cheers, holger From arigo at tunes.org Wed Nov 3 17:58:31 2004 From: arigo at tunes.org (Armin Rigo) Date: Wed, 3 Nov 2004 16:58:31 +0000 Subject: [pypy-dev] More on optimization In-Reply-To: <20041103154000.GA27665@solar.trillke.net> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> <20041102183653.GP15456@solar.trillke.net> <20041103141021.GA30865@vicky.ecs.soton.ac.uk> <20041103142842.GY15456@solar.trillke.net> <20041103145046.GA18341@vicky.ecs.soton.ac.uk> <20041103154000.GA27665@solar.trillke.net> Message-ID: <20041103165831.GA26801@vicky.ecs.soton.ac.uk> Hi Holger, On Wed, Nov 03, 2004 at 04:40:01PM +0100, holger krekel wrote: > me hum, too :-) this means that > > http://codespeak.net/pypy/index.cgi?doc/architecture.html > > is not right when it says: > > The translation process is done in three steps: > > - producing a flow graph representation of the standard > interpreter. A combination of a plain interpreter and a > flow object space performs "abstract interpretation" to record > the flow of objects and execution throughout a python program > into such a flow graph. > > - ... > > because your current prescribed notion of abstract > interpretation starts with already existing flow graphs. Hum. Abstract interpretation is the name of a very general technique: we use it twice. We use it to produce the flow graph (i.e. the architecture document is correct), and then we also use it to produce annotations on the graph (the architecture document doesn't say how we do that, so it's not wrong not to say we do it with another kind of abstract interpretation too). Now I see where the confusion comes from. The e-mails in this thread have nothing to do with the flowobjspace or the flow graph construction itself. What you said about it is correct but off topic: the topic is annotations and backends. All the "abstract interpretation" I talked about is the one we do to infer the annotations. > But if one part of the program uses the one branch and another > part uses the other branch then you will generalize the function > to annotate 'y' as being an integer. Right. Now I understand your question: > So we do already keep possibly different function versions > for differently typed sets of input parameters to this function? The original code example with "if type(a) is int and ..." was supposed to be inlined at a lot of places, i.e. to replace verbatim all the high-level "add" operations. You're right, I was quite vague. If that code example was put in a separate function, then probably all the branches would have to be analysed anyway, unless we kept different versions of the function for differently typed sets of inputs. Instead, to avoid the problem, it's better to create an inlined copy of the code everywhere and then solve the code bloat by optimizing away all branches apart one. A bient?t, Armin From hpk at trillke.net Wed Nov 3 20:16:34 2004 From: hpk at trillke.net (holger krekel) Date: Wed, 3 Nov 2004 20:16:34 +0100 Subject: [pypy-dev] More on optimization In-Reply-To: <20041103165831.GA26801@vicky.ecs.soton.ac.uk> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> <20041102183653.GP15456@solar.trillke.net> <20041103141021.GA30865@vicky.ecs.soton.ac.uk> <20041103142842.GY15456@solar.trillke.net> <20041103145046.GA18341@vicky.ecs.soton.ac.uk> <20041103154000.GA27665@solar.trillke.net> <20041103165831.GA26801@vicky.ecs.soton.ac.uk> Message-ID: <20041103191634.GE27665@solar.trillke.net> Hi Armin, [Armin Rigo Wed, Nov 03, 2004 at 04:58:31PM +0000] > Abstract interpretation is the name of a very general technique: we use it > twice. We use it to produce the flow graph (i.e. the architecture document is > correct), and then we also use it to produce annotations on the graph (the > architecture document doesn't say how we do that, so it's not wrong not to say > we do it with another kind of abstract interpretation too). Thanks for the clarification. > Now I see where the confusion comes from. The e-mails in this thread have > nothing to do with the flowobjspace or the flow graph construction itself. > What you said about it is correct but off topic: the topic is annotations and > backends. All the "abstract interpretation" I talked about is the one we do > to infer the annotations. Right. However, in the past we have sometimes been mixing the flow analysis and annotation part, both in talking and coding. I do recall a discussion regarding the determination of "merge points" in the flow analysis that we _could_ connect to type/annotation analysis (during a screen-session where we hacked together on the flow/* code) but let's postpone this as i agree it doesn't neccessarily relate to the current topic. Btw, it will be interesting to come up with a formal definition of abstract interpretation at some point along with somewhat more formal definitions of the other terms we often use. (not neccessarily completly mathematically definitions, mind you). Kind of a a brief "glossary" thingie to allow us and others to understand what we are talking about. Maybe you or Samuele could even start a brief but current "glossary.txt" before the Vilnius sprint ... ? cheers, holger From arigo at tunes.org Thu Nov 4 14:56:20 2004 From: arigo at tunes.org (Armin Rigo) Date: Thu, 4 Nov 2004 13:56:20 +0000 Subject: [pypy-dev] More on optimization In-Reply-To: <4187BF65.4090906@bluewin.ch> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> <4187BF65.4090906@bluewin.ch> Message-ID: <20041104135620.GA18699@vicky.ecs.soton.ac.uk> On Tue, Nov 02, 2004 at 06:09:57PM +0100, Samuele Pedroni wrote: > (...) In Java, lisp etc is not that much a > relevant problem because for example Java arrays carry a length anyway > and are heap allocated. Ops, by the way: we can't use a Java array to implement a RPython list because you can't resize Java arrays. So there is an extra indirection even in Java and there is a need to remove it with enough analysis and optimizations... Or am I missing something? In Lisp you can declare resizable arrays, though it may mean an extra indirection too. Armin From pedronis at bluewin.ch Thu Nov 4 15:15:49 2004 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Thu, 04 Nov 2004 15:15:49 +0100 Subject: [pypy-dev] More on optimization In-Reply-To: <20041104135620.GA18699@vicky.ecs.soton.ac.uk> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> <4187BF65.4090906@bluewin.ch> <20041104135620.GA18699@vicky.ecs.soton.ac.uk> Message-ID: <418A3995.2030107@bluewin.ch> Armin Rigo wrote: > On Tue, Nov 02, 2004 at 06:09:57PM +0100, Samuele Pedroni wrote: > >>(...) In Java, lisp etc is not that much a >>relevant problem because for example Java arrays carry a length anyway >>and are heap allocated. > > > Ops, by the way: we can't use a Java array to implement a RPython list because > you can't resize Java arrays. So there is an extra indirection even in Java > and there is a need to remove it with enough analysis and optimizations... > Or am I missing something? > it depends on how you implement things, realloc needs to be expressed as alloc and copy but I don't see what force you to incapsulate the array in an extra layer object. > In Lisp you can declare resizable arrays, though it may mean an extra > indirection too. > > > Armin > From pedronis at bluewin.ch Thu Nov 4 15:51:50 2004 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Thu, 04 Nov 2004 15:51:50 +0100 Subject: [pypy-dev] More on optimization In-Reply-To: <418A3995.2030107@bluewin.ch> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> <4187BF65.4090906@bluewin.ch> <20041104135620.GA18699@vicky.ecs.soton.ac.uk> <418A3995.2030107@bluewin.ch> Message-ID: <418A4206.1060507@bluewin.ch> Samuele Pedroni wrote: > Armin Rigo wrote: > >> On Tue, Nov 02, 2004 at 06:09:57PM +0100, Samuele Pedroni wrote: >> >>> (...) In Java, lisp etc is not that much a relevant problem because >>> for example Java arrays carry a length anyway and are heap allocated. >> >> >> >> Ops, by the way: we can't use a Java array to implement a RPython list >> because >> you can't resize Java arrays. So there is an extra indirection even >> in Java >> and there is a need to remove it with enough analysis and >> optimizations... >> Or am I missing something? >> > > it depends on how you implement things, realloc needs to be expressed as > alloc and copy but I don't see what force you to incapsulate the array > in an extra layer object. > oops, sorry now I see, you want RPython lists to be lists, that's probably different that what we wanted originally. listobject is still written asssuming that ob_item is a chunk of memory (i), but in other parts we are assuming RPython list can be extended and appended to (ii). (btw question is btw whether resizing policy should be in listobject on in the RPython list sematics.) With approach 2 we would go from all the fields necesessary from a general list to just len+items for the first case because some thing are constants. It's still true that both an ad-hoc approach given knowledge about the specific type that exist in RPython, or a general one is possible. In Java as first approx you would use an array for (i) and ArrayList for (ii) I still think that a baseline intepreter with straightforward optimizations is a good first goal, also because we can use it as reference to track our progress and see what gives the biggest payoffs. From arigo at tunes.org Thu Nov 4 23:45:15 2004 From: arigo at tunes.org (Armin Rigo) Date: Thu, 4 Nov 2004 22:45:15 +0000 Subject: [pypy-dev] More on optimization In-Reply-To: <418A4206.1060507@bluewin.ch> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> <4187BF65.4090906@bluewin.ch> <20041104135620.GA18699@vicky.ecs.soton.ac.uk> <418A3995.2030107@bluewin.ch> <418A4206.1060507@bluewin.ch> Message-ID: <20041104224515.GA23396@vicky.ecs.soton.ac.uk> Hi Samuele, On Thu, Nov 04, 2004 at 03:51:50PM +0100, Samuele Pedroni wrote: > >it depends on how you implement things, realloc needs to be expressed as > > alloc and copy but I don't see what force you to incapsulate the array > >in an extra layer object. > > > > oops, sorry now I see, you want RPython lists to be lists, that's > probably different that what we wanted originally. No, I'm thinking about the following problem: suppose that an RPython list is meant to represent an expensively resizeable array. The problem with implementing that using a bare Java array is that when you realloc and copy, all the references to the array should be updated to point to the new array. This is impossible unless you either track all references using complex analysis, or you encapsulate the array in another Java object. It's like, say, in Python where you cannot use bare strings to implement mutable strings: although you can use slices and concatenation to build a modified string, you cannot update all the places that pointed to the old string to now point to the new string. So you have to encapsulate the string in an instance. Armin From hpk at trillke.net Sat Nov 6 11:33:16 2004 From: hpk at trillke.net (holger krekel) Date: Sat, 6 Nov 2004 11:33:16 +0100 Subject: [pypy-dev] explicit documentation (re)organization Message-ID: <20041106103316.GP15506@solar.trillke.net> hi folks, i may not be alone in thinking that the current documentation organization in PyPy leaves an lot to be desired. IMO the main problem: we don't have a clear and maintained entry point for documentation for developers (tm) and we don't cleanly link all the _relevant_ and somewhat up-to-date documentation. Also we link to a lot of half-way/not decent/wrong documentation. So i plan to move towards i model that is already used with the upcoming py lib (http://codespeak.net/py ): - there would be a directory which holds an "index.txt" which will be ResT-translated in order to serve as the main developer documentation entry point - this index.txt will link to the various important other documentation chapters _explicitely_ - linking to the chapters happens with relative links so that you can rest-convert and check links before checking in. With the py lib there even is a test that checks that the documentation does not produce warnings. Currently, in PyPy we are using an implicit scheme which basically rest-converts everything that was dropped into some directories. IMO the result is confusing, unordered and didn't really work out. I believe that getting the documentation in a better shape is crucial to involve new developers (and even the current ones :-) Comments? I'd like to do the reorganization over the weekend and would afterwards appreciate any help in bringing the documentation up to date, especially deleting or moving all the obsolete stuff. This is real work but it really makes a difference so please help ... cheers, holger From lac at strakt.com Sat Nov 6 12:19:04 2004 From: lac at strakt.com (Laura Creighton) Date: Sat, 06 Nov 2004 12:19:04 +0100 Subject: [pypy-dev] explicit documentation (re)organization In-Reply-To: Message from hpk@trillke.net (holger krekel) of "Sat, 06 Nov 2004 11:33:16 +0100." <20041106103316.GP15506@solar.trillke.net> References: <20041106103316.GP15506@solar.trillke.net> Message-ID: <200411061119.iA6BJ4PM013315@ratthing-b246.strakt.com> I would like all the documents produced to cotinue to be under svn source code control. And I really do not want obsolete things deleted, only moved. At some point, one of us is likely to want to do a paper on the development process, and it will be very useful to have things that document what we used to think, and when we thought it for that purpose. Also, the fact that we have decided against a certain approach does not make it therefore uninteresting. I'd like to be able to go back and think about roads not taken at some point. This will be useful for new developers as well. If somebody suggests we do something which we once thought was a good idea but do no longer, it will be a lot easier to point them at the discussion that made us reject the idea rather than try to reconstruct it. When you reconstruct the idea, what you most often do is write 'why I would reject the idea now' believing that the reason you have now is the reason you had then, or a development of the one that you had then. This is often true. But sometimes the reason you have for adopting a new decision isn't actually a valid one, but the evidence for that shows up slowly. By the time you are all in agreement that the original reason you had for adopting this plan of action is not valid, you will have generated other reasons for having this plan of action -- including the old standby 'but we have a heck of a lot of code written with this plan in mind'. It often takes fresh eyes to see that the decision was the wrong one, and possibly it is time to yank out the code and start that bit over. We've already done this a bit -- have done plan a, and then decided to do it over as plan b, and then on the third think decide that plan a was better after all. This means we should be careful about saving old documentation, because what we toss out may come back later. Or something similar in a different shape -- they way we still have multi-methods now, but they don't do what they did a year and a half ago ... While organising the documentation is likely to help, I think that the reason the py lib documentation is better than the pypy documentation is because Holger sat down one weekend and wrote a lot of it. I don't see the same sort of commitment to writing documentation for the pypy project because more of the pypy code is 'experimental -- subject to change'. The people who know the most about the code -- those who are writing it -- don't want to document something they might want to rewrite next week, and so it goes, because it is actually hard to find a good time to document the code that was done which is one reason why the WP approach may be good for us. We will have to document in bits for the EU in any case. But the ongoing problem remains. Would you rather document the stuff that we already have, that might change, or write the changes? I think we are going to always feel a push towards writing the code. Perhaps a way for us to turn discussion we had on the mailing lists into lists of things for newcomers to read would be more effective? This is only an idea. I am not sure of that. I am going to be away travelling, first to Brussels and then to the USA until Vilnius, and so I cannot help with documentation much until Vilnius. Then I could help a lot, provided that there is enough I understand that needs documenting. When I run out of things I understand, you will have to teach me stuff -- which might not be what you want to do, and you would prefer hacking. Laura From hpk at trillke.net Sat Nov 6 12:34:31 2004 From: hpk at trillke.net (holger krekel) Date: Sat, 6 Nov 2004 12:34:31 +0100 Subject: [pypy-dev] explicit documentation (re)organization In-Reply-To: <200411061119.iA6BJ4PM013315@ratthing-b246.strakt.com> References: <20041106103316.GP15506@solar.trillke.net> <200411061119.iA6BJ4PM013315@ratthing-b246.strakt.com> Message-ID: <20041106113431.GR15506@solar.trillke.net> Hi Laura, [Laura Creighton Sat, Nov 06, 2004 at 12:19:04PM +0100] > I would like all the documents produced to cotinue to be under svn source code > control. sure, using subversion for holding the basic documentation is certainly not going to go away :-) > And I really do not want obsolete things deleted, only moved. At > some point, one of us is likely to want to do a paper on the development > process, and it will be very useful to have things that document what > we used to think, and when we thought it for that purpose. Also, the fact > that we have decided against a certain approach does not make it therefore > uninteresting. I'd like to be able to go back and think about roads not > taken at some point. Yes, I see the point. However, when we delete things in a certain revision they don't go away from the repository. It is always possible to look at former revisions and see what happened. I completly agree that this is something we want to have. Also Jum and me are working on the new codespeak setup which will feature a "trac" environment which allows to search for older commits and files ... > ... > While organising the documentation is likely to help, I think that the reason > the py lib documentation is better than the pypy documentation is because > Holger sat down one weekend and wrote a lot of it. Actually even some more than just a weekend. > I don't see the same sort of commitment to writing > documentation for the pypy project because more of the pypy > code is 'experimental -- subject to change'. The people who > know the most about the code -- those who are writing it -- > don't want to document something they might want to rewrite > next week, and so it goes, because it is actually hard to > find a good time to document the code that was done which is > one reason why the WP approach may be good for us. Well, the py lib is not documented completly either. I think it makes most sense to document the most stable parts (which we partly already did but it's not organized well enough IMO). For futuristic stuff it may also be good to write down the ideas and mark them as "futuristic, subject to change". > We will have to document in bits for the EU in any case. Yes, although i see the "documentation for developers" somewhat independent of reporting to the EU especially since the EU hasn't even confirmed yet. > But the ongoing problem remains. Would you rather document the stuff that we > already have, that might change, or write the changes? I think we are going to > always feel a push towards writing the code. > > Perhaps a way for us to turn discussion we had on the mailing lists into > lists of things for newcomers to read would be more effective? This is only > an idea. I am not sure of that. I do think that instead of writing long mails spending the time to write a "future" chapter is often a good idea. > I am going to be away travelling, first to Brussels and then to the USA until > Vilnius, and so I cannot help with documentation much until Vilnius. Then > I could help a lot, provided that there is enough I understand that needs > documenting. When I run out of things I understand, you will have to teach > me stuff -- which might not be what you want to do, and you would prefer hacking. Btw, Laura&Jacob, please list your arrival/departure dates and accomodation status at http://codespeak.net/moin/pypy/moin.cgi/VilniusSprintAttendants cheers, holger From lac at strakt.com Sat Nov 6 14:01:53 2004 From: lac at strakt.com (Laura Creighton) Date: Sat, 06 Nov 2004 14:01:53 +0100 Subject: [pypy-dev] explicit documentation (re)organization In-Reply-To: Message from holger krekel of "Sat, 06 Nov 2004 12:34:31 +0100." <20041106113431.GR15506@solar.trillke.net> References: <20041106103316.GP15506@solar.trillke.net> <200411061119.iA6BJ4PM013315@ratthing-b246.strakt.com> <20041106113431.GR15506@solar.trillke.net> Message-ID: <200411061301.iA6D1roR013543@ratthing-b246.strakt.com> In a message of Sat, 06 Nov 2004 12:34:31 +0100, holger krekel writes: >Hi Laura, > >[Laura Creighton Sat, Nov 06, 2004 at 12:19:04PM +0100] >> I would like all the documents produced to cotinue to be under svn sour >ce code >> control. > >sure, using subversion for holding the basic documentation is certainly >not going to go away :-) > >> And I really do not want obsolete things deleted, only moved. At >> some point, one of us is likely to want to do a paper on the developmen >t >> process, and it will be very useful to have things that document what >> we used to think, and when we thought it for that purpose. Also, the f >act >> that we have decided against a certain approach does not make it theref >ore >> uninteresting. I'd like to be able to go back and think about roads no >t >> taken at some point. > >Yes, I see the point. However, when we delete things in a certain revisi >on >they don't go away from the repository. It is always possible to look a >t >former revisions and see what happened. I completly agree that this >is something we want to have. Also Jum and me are working on the >new codespeak setup which will feature a "trac" environment which >allows to search for older commits and files ... The trac sounds interesting. Requiring that I look things up on the repository might be ok for documents that are current, but have changes in them, but I think is inappropriate for entire documents that are obsolete. We've run into this problem before. I've written something, and it becomes obsolete. Somewhere along the line, in an effort to not confuse people with old stuff, you take the what seems reasonable step of removing this. What you don't know is that that particular document was the one and only time I had ever managed to make ReST do . Or I thought up a neat way to express something, or I had to look up the spelling of a word. All in all, this means that the document continues to have present and lasting value to me beyond its value to you -- or anybody else. But I don't notice that it is gone until the next time I need something from it, and then I expect to be able to find it by clicking on something. So what I would prefer is a quasi-mirror of the pypy site, with a header of 'obsolete'. If it isn't in the current, then I can find old docs in the same location as they would be if they were still current. I need to be able to do 'click-click-click -- aha, there is the table I was looking for --' and then get on with what I want. Laura ps - updated the page, and bonus hero points for working over one weekend. From pedronis at bluewin.ch Sat Nov 6 19:47:58 2004 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Sat, 06 Nov 2004 19:47:58 +0100 Subject: [pypy-dev] More on optimization In-Reply-To: <20041104224515.GA23396@vicky.ecs.soton.ac.uk> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> <4187BF65.4090906@bluewin.ch> <20041104135620.GA18699@vicky.ecs.soton.ac.uk> <418A3995.2030107@bluewin.ch> <418A4206.1060507@bluewin.ch> <20041104224515.GA23396@vicky.ecs.soton.ac.uk> Message-ID: <418D1C5E.4060803@bluewin.ch> Armin Rigo wrote: > Hi Samuele, > > On Thu, Nov 04, 2004 at 03:51:50PM +0100, Samuele Pedroni wrote: > >>>it depends on how you implement things, realloc needs to be expressed as >>>alloc and copy but I don't see what force you to incapsulate the array >>>in an extra layer object. >>> >> >>oops, sorry now I see, you want RPython lists to be lists, that's >>probably different that what we wanted originally. > > > No, I'm thinking about the following problem: suppose that an RPython list is > meant to represent an expensively resizeable array. that what I was implicitly thinking with "be lists". My point anyway was simply that for Java the first relevant thing is to dinstiguish constant-size lists (the code in listobject uses such things) vs. variable size lists, and both then have natural reprs. In a more subtle approach also lists that have always just _one_ reference to them in the whole heap (and are not passed as parameters perhaps) can likely also be implemented with a direct array reference. It is true that the generalization and understanding of this kind of criteria is what we need for general structure inlining. From anna at aleax.it Sun Nov 7 10:16:59 2004 From: anna at aleax.it (Anna Martelli Ravenscroft) Date: Sun, 07 Nov 2004 10:16:59 +0100 Subject: [pypy-dev] explicit documentation (re)organization In-Reply-To: <200411061301.iA6D1roR013543@ratthing-b246.strakt.com> References: <20041106103316.GP15506@solar.trillke.net> <200411061119.iA6BJ4PM013315@ratthing-b246.strakt.com> <20041106113431.GR15506@solar.trillke.net> <200411061301.iA6D1roR013543@ratthing-b246.strakt.com> Message-ID: <418DE80B.9060609@aleax.it> Laura Creighton wrote: > In a message of Sat, 06 Nov 2004 12:34:31 +0100, holger krekel writes: > What you don't know is that that particular document was the one and only > time I had ever managed to make ReST do . Or I thought up a > neat way to express something, or I had to look up the spelling of a word. > All in all, this means that the document continues to have present and lasting > value to me beyond its value to you -- or anybody else. In other words, your document has meta-value over and above whatever it happens to be documenting. This is a good point. > But I don't notice that it is gone until the next time I need something > from it, and then I expect to be able to find it by clicking on something. > So what I would prefer is a quasi-mirror of the pypy site, with a header > of 'obsolete'. If it isn't in the current, then I can find old docs in the > same location as they would be if they were still current. I need to be > able to do 'click-click-click -- aha, there is the table I was looking for --' > and then get on with what I want. Laura and Holger - is there a way I can help with this? I miss working with you guys, and this seems like somewhere that I could be useful, whether or not I fully comprehend the ins and outs of where the pypy project is at the moment. Please let me know - offlist if you like. Anna From lac at strakt.com Sun Nov 7 11:06:08 2004 From: lac at strakt.com (Laura Creighton) Date: Sun, 07 Nov 2004 11:06:08 +0100 Subject: [pypy-dev] explicit documentation (re)organization In-Reply-To: Message from Anna Martelli Ravenscroft of "Sun, 07 Nov 2004 10:16:59 +0100." <418DE80B.9060609@aleax.it> References: <20041106103316.GP15506@solar.trillke.net> <200411061119.iA6BJ4PM013315@ratthing-b246.strakt.com> <20041106113431.GR15506@solar.trillke.net> <200411061301.iA6D1roR013543@ratthing-b246.strakt.com> <418DE80B.9060609@aleax.it> Message-ID: <200411071006.iA7A68jH016451@ratthing-b246.strakt.com> In a message of Sun, 07 Nov 2004 10:16:59 +0100, Anna Martelli Ravenscroft writes: >Laura and Holger - >is there a way I can help with this? I miss working with you guys, and >this seems like somewhere that I could be useful, whether or not I fully >comprehend the ins and outs of where the pypy project is at the moment. > >Please let me know - offlist if you like. > >Anna I miss working with you too. But I leave tomorrow to host a anti-sw-pat conference at EU Parl in Brussels. Then I fly to Santa Cruz, USA, and then I fly directly to the Vilnius Sprint. So it will be a while before I can continue this discussion, by which time I expect something to already happen. Laura From hpk at trillke.net Sun Nov 7 12:03:19 2004 From: hpk at trillke.net (holger krekel) Date: Sun, 7 Nov 2004 12:03:19 +0100 Subject: [pypy-dev] explicit documentation (re)organization In-Reply-To: <200411061301.iA6D1roR013543@ratthing-b246.strakt.com> References: <20041106103316.GP15506@solar.trillke.net> <200411061119.iA6BJ4PM013315@ratthing-b246.strakt.com> <20041106113431.GR15506@solar.trillke.net> <200411061301.iA6D1roR013543@ratthing-b246.strakt.com> Message-ID: <20041107110319.GW15506@solar.trillke.net> Hi Laura, [Laura Creighton Sat, Nov 06, 2004 at 02:01:53PM +0100] > What you don't know is that that particular document was the one and only > time I had ever managed to make ReST do . > Or I thought up a neat way to express something, or I had to > look up the spelling of a word. > ... > All in all, this means that the document continues to have present and lasting > value to me beyond its value to you -- or anybody else. I don't think that the "pypy documentation for and from developers" should primarily strive to have this side-function of preserving bits and pieces of formatting or other meta-info that might be helpful to particular users. IMHO the pypy documentation serves two purposes: - letting the developers easily organize up-to-date documentation, by integrating hacking at ReST files into the coding process as seemless as possible - letting others easily see this current view of the developers regarding the implementation and architecture of PyPy That being said i definitely see the point of wanting to look at an earlier revision of a file, especially if it has been deleted from the repo. To date there is no nice tool i know off that provides you a complete search functionality over ancestor revisions (not even trac at the moment although it goes some way). So we might move documents into some 'attic' instead of deleting them. But note that most often files will be modified and not deleted. And we certainly don't want to keep all older revisions of modified files in some 'attic' folder as this would lead to a plethora of files and would undermine the whole point of a version control system. Oh, and btw, if you say "i know that at EuroPython 2004 there was this nice ReST-thingie i did" you can issue svn up -r "06-10-2004" in the doc directory which gets you right back to the complete view at that time. Taking this single-step indirection seems reasonable enough to me for most cases. cheers, holger From hpk at trillke.net Sun Nov 7 12:06:53 2004 From: hpk at trillke.net (holger krekel) Date: Sun, 7 Nov 2004 12:06:53 +0100 Subject: [pypy-dev] explicit documentation (re)organization In-Reply-To: <418DE80B.9060609@aleax.it> References: <20041106103316.GP15506@solar.trillke.net> <200411061119.iA6BJ4PM013315@ratthing-b246.strakt.com> <20041106113431.GR15506@solar.trillke.net> <200411061301.iA6D1roR013543@ratthing-b246.strakt.com> <418DE80B.9060609@aleax.it> Message-ID: <20041107110653.GX15506@solar.trillke.net> Hi Anna, [Anna Martelli Ravenscroft Sun, Nov 07, 2004 at 10:16:59AM +0100] > is there a way I can help with this? I miss working with you guys, and > this seems like somewhere that I could be useful, whether or not I fully > comprehend the ins and outs of where the pypy project is at the moment. My current intention is to first sort out the documentation that is resonably up to date with the code and that can obviously only be judged by people being involved with the code. After we are there, i am sure you can help with improving and making it more easily accessible to "first-time PyPyers" where it actually helps not knowing too much :-) cheers, holger From anna at aleax.it Sun Nov 7 12:11:28 2004 From: anna at aleax.it (Anna Martelli Ravenscroft) Date: Sun, 07 Nov 2004 12:11:28 +0100 Subject: [pypy-dev] explicit documentation (re)organization In-Reply-To: <20041107110653.GX15506@solar.trillke.net> References: <20041106103316.GP15506@solar.trillke.net> <200411061119.iA6BJ4PM013315@ratthing-b246.strakt.com> <20041106113431.GR15506@solar.trillke.net> <200411061301.iA6D1roR013543@ratthing-b246.strakt.com> <418DE80B.9060609@aleax.it> <20041107110653.GX15506@solar.trillke.net> Message-ID: <418E02E0.10402@aleax.it> holger krekel wrote: > Hi Anna, > > [Anna Martelli Ravenscroft Sun, Nov 07, 2004 at 10:16:59AM +0100] > >>is there a way I can help with this? I miss working with you guys, and >>this seems like somewhere that I could be useful, whether or not I fully >>comprehend the ins and outs of where the pypy project is at the moment. > > > My current intention is to first sort out the documentation > that is resonably up to date with the code and that can > obviously only be judged by people being involved with the > code. After we are there, i am sure you can help with > improving and making it more easily accessible to > "first-time PyPyers" where it actually helps not knowing > too much :-) Kewl! Sounds like just my kind of niche. ;-) I could hardly be accused of "knowing too much" when it comes to pypy. I've been following the list (in my "copious spare time") so feel free to post here once you've reorganized so I can start looking at it, or email me directly to let me know if there are specific areas where you want me to start. Anna From arigo at tunes.org Sun Nov 7 13:01:00 2004 From: arigo at tunes.org (Armin Rigo) Date: Sun, 7 Nov 2004 12:01:00 +0000 Subject: [pypy-dev] More on optimization In-Reply-To: <418D1C5E.4060803@bluewin.ch> References: <20041031110745.GA25406@vicky.ecs.soton.ac.uk> <20041102161905.GA30013@vicky.ecs.soton.ac.uk> <4187BF65.4090906@bluewin.ch> <20041104135620.GA18699@vicky.ecs.soton.ac.uk> <418A3995.2030107@bluewin.ch> <418A4206.1060507@bluewin.ch> <20041104224515.GA23396@vicky.ecs.soton.ac.uk> <418D1C5E.4060803@bluewin.ch> Message-ID: <20041107120100.GA4089@vicky.ecs.soton.ac.uk> Hi Samuele, On Sat, Nov 06, 2004 at 07:47:58PM +0100, Samuele Pedroni wrote: > that what I was implicitly thinking with "be lists". My point anyway was > simply that for Java the first relevant thing is to dinstiguish > constant-size lists (the code in listobject uses such things) vs. > variable size lists, and both then have natural reprs. Ah: I didn't realize that listobject's "ob_item" was never resized. I though that _list_resize() would extend the existing list object, but instead it creates a new one and assigns it to "w_self.ob_item". I was insisting on this point based on this misunderstanding, sorry. Indeed, it looks much easier to just detect that "ob_item" never changes its size, and use a plain Java array because of that. Armin From lac at strakt.com Sun Nov 7 13:30:25 2004 From: lac at strakt.com (Laura Creighton) Date: Sun, 07 Nov 2004 13:30:25 +0100 Subject: [pypy-dev] explicit documentation (re)organization In-Reply-To: Message from holger krekel of "Sun, 07 Nov 2004 12:03:19 +0100." <20041107110319.GW15506@solar.trillke.net> References: <20041106103316.GP15506@solar.trillke.net> <200411061119.iA6BJ4PM013315@ratthing-b246.strakt.com> <20041106113431.GR15506@solar.trillke.net> <200411061301.iA6D1roR013543@ratthing-b246.strakt.com> <20041107110319.GW15506@solar.trillke.net> Message-ID: <200411071230.iA7CUP81016769@ratthing-b246.strakt.com> In a message of Sun, 07 Nov 2004 12:03:19 +0100, holger krekel writes: >Hi Laura, >I don't think that the "pypy documentation for and from >developers" should primarily strive to have this side-function >of preserving bits and pieces of formatting or other meta-info >that might be helpful to particular users. > >IMHO the pypy documentation serves two purposes: > >- letting the developers easily organize up-to-date > documentation, by integrating hacking at ReST files > into the coding process as seemless as possible We share this goal. > >- letting others easily see this current view of the developers > regarding the implementation and architecture of PyPy But we don't share this one. I want us to write documentation primarily for us. I think that there can be a place for 'documentation for others, and introductory material' but that is not, primarily, what I am interested in doing when I am writing documentation. I write documentation first for me, and then for the people I work with, and last and by far the least for other interested parties. This changes, of course, when you are writing an 'Introductory Manual for New Users' or some such, where the explicit goal of the work is to communicate with new users. I am not opposed to the writing of such things, but I don't see it as having much to do with the ongoing documentation of a project, except coincidentally. The ongoing documentation that we write for ourselves is constantly becoming obsolete as what we are doing differs from what we thought we would do when we wrote something. There is nothing wrong with this. Efforts to keep what we wrote in sync with what we actually are doing, in my experience have always resulted in people documenting less and less and less. I don't want the responsibility of keeping what I wrote current with what we are doing. If I am stuck with this, I will document to the minimum extent possible. >That being said i definitely see the point of wanting to >look at an earlier revision of a file, especially if it >has been deleted from the repo. To date there is no nice >tool i know off that provides you a complete search functionality >over ancestor revisions (not even trac at the moment although it >goes some way). Now you know why I wanted to write a CAPS system at Strakt. :-) Some day real soon now, it will even do this properly. >So we might move documents into some 'attic' instead of deleting them. >But note that most often files will be modified and not deleted. >And we certainly don't want to keep all older revisions of modified >files in some 'attic' folder as this would lead to a plethora of files >and would undermine the whole point of a version control system. Yes. Outlining the cases when you want to undermine the whole point of a version control system is precisely what I am trying to do. When documents change a lot over their lifetime, there is often a case to be made for having different versions of the document (mar 2000, oct 2000, dec 2001) and the like available, as reference documents. Deciding which documents deserve such treatment is an art, more than a science. Without such a history, you end up presenting your current ideas without the context of how they evolved to what they are at present. In some cases, this 'dis-in-history'-ing is the actual goal, and what you are trying to do. But in other cases, it is the historical development of the roadmap, that is of interest, not wherever the arrow 'We are Here' currently points. They are contradictory goals, and extremely hard to realise at the same time. What actually happens, wehn documenting, in my experience, is that somebody writes a document, and it gets modidied quite a bit while it is the focus of some work that is being hacked on right now, or will be hacked on in the immediate future. Then we finish hacking that, and move on to other goals and make other documents. Time passes. The code that the original doc referred to gets refactored and modified in the light of new ideas we have in the context of our new goals. The original doc does not get touched, because it is no longer interesting in the context of where development is happening now, or next week, or next month. Eventually somebody decides that the document is so obsolete that they delete it. > >Oh, and btw, if you say "i know that at EuroPython 2004 there was >this nice ReST-thingie i did" you can issue > > svn up -r "06-10-2004" > >in the doc directory which gets you right back to the complete >view at that time. Taking this single-step indirection seems >reasonable enough to me for most cases. Not for me. I am not looking for something to read, but something to scan and to search. Which means I want it as a webpage I can glance at, and that Google can index. Also, I don't know about you, but when you remember a phrase from a book or a webpage, don't you remember stuff like 'about 2/3rds of the way through the book, on a right-facing page, second to last sentence before a new paragraph' ? People who remember information independently of their presentation medium have different searching and retreival needs than those who remember their information with lots of embedded context. It's a neat hard problem in cognative science and human-computer interfaces. > >cheers, > > holger take care, Laura >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev From hpk at trillke.net Sun Nov 7 14:03:15 2004 From: hpk at trillke.net (holger krekel) Date: Sun, 7 Nov 2004 14:03:15 +0100 Subject: [pypy-dev] explicit documentation (re)organization In-Reply-To: <200411071230.iA7CUP81016769@ratthing-b246.strakt.com> References: <20041106103316.GP15506@solar.trillke.net> <200411061119.iA6BJ4PM013315@ratthing-b246.strakt.com> <20041106113431.GR15506@solar.trillke.net> <200411061301.iA6D1roR013543@ratthing-b246.strakt.com> <20041107110319.GW15506@solar.trillke.net> <200411071230.iA7CUP81016769@ratthing-b246.strakt.com> Message-ID: <20041107130315.GZ15506@solar.trillke.net> [Laura Creighton Sun, Nov 07, 2004 at 01:30:25PM +0100] > In a message of Sun, 07 Nov 2004 12:03:19 +0100, holger krekel writes: > >- letting others easily see this current view of the developers > > regarding the implementation and architecture of PyPy > > But we don't share this one. I want us to write documentation primarily > for us. I think that there can be a place for 'documentation for > others, and introductory material' but that is not, primarily, what I > am interested in doing when I am writing documentation. I write documentation > first for me, and then for the people I work with, and last and by far > the least for other interested parties. In an open source context this distinction is a fluid thing. If we want more people to understand PyPy and contribute i don't see much value in trying to come up with strict distinctions. > The ongoing documentation that we write for ourselves is constantly becoming > obsolete as what we are doing differs from what we thought we would do when > we wrote something. I hope we can evolve into a model where we document things that are pretty stable. For example, the interpreter <-> object space interaction hasn't changed much since 2002 (!). But i would argue that with today's organization you wouldn't easily get to all the documentation describing it. At least this can be improved by putting aways with all the links that are just distracting. > ... > What actually happens, wehn documenting, in my experience, is that > somebody writes a document, and it gets modidied quite a bit while it > is the focus of some work that is being hacked on right now, or will > be hacked on in the immediate future. Honestly, i don't see this happening with most of the projects i am involved with. A positive example is the core CPython documentation development, probably. It certainly doesn't happen much with PyPy so far. > >Oh, and btw, if you say "i know that at EuroPython 2004 there was > >this nice ReST-thingie i did" you can issue > > > > svn up -r "06-10-2004" > > > >in the doc directory which gets you right back to the complete > >view at that time. Taking this single-step indirection seems > >reasonable enough to me for most cases. > > Not for me. I am not looking for something to read, but something to > scan and to search. Which means I want it as a webpage I can glance > at, and that Google can index. Maybe, but this is getting quickly out of scope of my current effort. Also, if you make all versions accessible to google how do you think google will order what it finds? Especially mentioning google makes IMHO a good case for offering only up-to-date documentation on the web because otherwise people will get at all the outdated information too easily. Hum, i guess i should just postpone changing anything about the documentation right now. I have enough to do before the Vilnius Sprint, anyway. cheers, holger From rodsenra at gpr.com.br Mon Nov 8 13:55:34 2004 From: rodsenra at gpr.com.br (Rodrigo Dias Arruda Senra) Date: Mon, 8 Nov 2004 10:55:34 -0200 Subject: [pypy-dev] explicit documentation (re)organization In-Reply-To: <200411071230.iA7CUP81016769@ratthing-b246.strakt.com> References: <20041106103316.GP15506@solar.trillke.net> <200411061119.iA6BJ4PM013315@ratthing-b246.strakt.com> <20041106113431.GR15506@solar.trillke.net> <200411061301.iA6D1roR013543@ratthing-b246.strakt.com> <20041107110319.GW15506@solar.trillke.net> <200411071230.iA7CUP81016769@ratthing-b246.strakt.com> Message-ID: <20041108105534.0000156c@Fenix> [ Holger ]: ------------ >>- letting the developers easily organize up-to-date >> documentation, by integrating hacking at ReST files >> into the coding process as seemless as possible >>- letting others easily see this current view of the developers >> regarding the implementation and architecture of PyPy [ Laura Creighton ]: -------------------------------------------------------- > I am not looking for something to read, but something to > scan and to search. Which means I want it as a webpage I can glance > at, and that Google can index. Hi, I apologize for intruding into this discussion. But for versioning issues, distributed editing and view of ReST docs + scan/search facilities in the Python world is usually dealt with Zope/Plone. Have you considered using these tools ? Perhaps a slightly modified workflow (doing copy&tag-on-editing), could substitute *true* versioning alternatives. Moreover, nothing prevents docs from being periodically imported into svn. Just a lurker's uninvited humble comment ;o) best regards, Rod senra From hpk at trillke.net Mon Nov 8 17:02:13 2004 From: hpk at trillke.net (holger krekel) Date: Mon, 8 Nov 2004 17:02:13 +0100 Subject: [pypy-dev] explicit documentation (re)organization In-Reply-To: <20041108105534.0000156c@Fenix> References: <20041106103316.GP15506@solar.trillke.net> <200411061119.iA6BJ4PM013315@ratthing-b246.strakt.com> <20041106113431.GR15506@solar.trillke.net> <200411061301.iA6D1roR013543@ratthing-b246.strakt.com> <20041107110319.GW15506@solar.trillke.net> <200411071230.iA7CUP81016769@ratthing-b246.strakt.com> <20041108105534.0000156c@Fenix> Message-ID: <20041108160213.GU15506@solar.trillke.net> Hi Rodrigo, [Rodrigo Dias Arruda Senra Mon, Nov 08, 2004 at 10:55:34AM -0200] > I apologize for intruding into this discussion. > But for versioning issues, distributed editing and view > of ReST docs + scan/search facilities in the Python world > is usually dealt with Zope/Plone. Have you considered using > these tools ? Well, i think it would be overkill. It is nice that the text files documenetation controled by svn really is the reference and you can look at history and everything with cmdline tools. It also opens the possibility to put it very close to the source code, also work-flow wise. I could possibly imagine, though, to have a zope/plone based _view_ of the documentation but it should then still access the repository for the content, for example with some of the work that Kapil Thangavelu has been doing. cheers, holger From tismer at stackless.com Thu Nov 18 10:22:53 2004 From: tismer at stackless.com (Christian Tismer) Date: Thu, 18 Nov 2004 11:22:53 +0200 Subject: [pypy-dev] autopath obsolete? Message-ID: <419C69ED.80703@stackless.com> Hi colleagues, I was wondering why we still need stuff like autopath? I simply use a pypy.pth file in my standard python folder which contains the path to pypy, and things work fine. Why do we really need autopath? -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From bob at redivi.com Thu Nov 18 10:36:00 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu, 18 Nov 2004 11:36:00 +0200 Subject: [pypy-dev] autopath obsolete? In-Reply-To: <419C69ED.80703@stackless.com> References: <419C69ED.80703@stackless.com> Message-ID: <4213F217-3945-11D9-AAFD-000A9567635C@redivi.com> On Nov 18, 2004, at 11:22 AM, Christian Tismer wrote: > I was wondering why we still need stuff like autopath? > I simply use a pypy.pth file in my standard python > folder which contains the path to pypy, and things > work fine. > Why do we really need autopath? Good question, it certainly looks worthless to me when you have a properly configured Python environment. It would probably make more sense to have ONE autopath-like module where you say something like: from pypy.tool import pypypath pypypath.dir_of(__name__) to determine the current path, pypypath.dir_of('pypy') to determine the root dir of pypy. dir_of should probably accept varargs and do os.path.join(...) as a shortcut, because that seems to be the most common case. -bob From mwh at python.net Thu Nov 18 11:19:36 2004 From: mwh at python.net (Michael Hudson) Date: Thu, 18 Nov 2004 10:19:36 +0000 Subject: [pypy-dev] Re: autopath obsolete? References: <419C69ED.80703@stackless.com> Message-ID: <2mllczzcg7.fsf@starship.python.net> Christian Tismer writes: > Hi colleagues, > > I was wondering why we still need stuff like autopath? > I simply use a pypy.pth file in my standard python > folder which contains the path to pypy, and things > work fine. > Why do we really need autopath? Well, if you run stuff from inside the pypy directory it's a good idea to get the current directory *out* of sys.path. autopath is a bit of a hack, I agree, but it's not costing much, is it? Cheers, mwh -- CLiki pages can be edited by anybody at any time. Imagine the most fearsomely comprehensive legal disclaimer you have ever seen, and double it -- http://ww.telent.net/cliki/index From bob at redivi.com Thu Nov 18 11:33:09 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu, 18 Nov 2004 12:33:09 +0200 Subject: [pypy-dev] Re: autopath obsolete? In-Reply-To: <2mllczzcg7.fsf@starship.python.net> References: <419C69ED.80703@stackless.com> <2mllczzcg7.fsf@starship.python.net> Message-ID: <3DE7FAE3-394D-11D9-AAFD-000A9567635C@redivi.com> On Nov 18, 2004, at 12:19 PM, Michael Hudson wrote: > Christian Tismer writes: > >> Hi colleagues, >> >> I was wondering why we still need stuff like autopath? >> I simply use a pypy.pth file in my standard python >> folder which contains the path to pypy, and things >> work fine. >> Why do we really need autopath? > > Well, if you run stuff from inside the pypy directory it's a good idea > to get the current directory *out* of sys.path. Well, autopath doesn't do that, it just shuffles the pypy base dir to sys.path[0]. > autopath is a bit of a hack, I agree, but it's not costing much, is it? The fact that it scatters itself all over the repository is kind of a maintenance hassle I think. It doesn't seem to do anything particularly useful... -bob From hpk at trillke.net Thu Nov 18 12:00:36 2004 From: hpk at trillke.net (holger krekel) Date: Thu, 18 Nov 2004 12:00:36 +0100 Subject: [pypy-dev] autopath obsolete? In-Reply-To: <419C69ED.80703@stackless.com> References: <419C69ED.80703@stackless.com> Message-ID: <20041118110036.GS12400@solar.trillke.net> [Christian Tismer Thu, Nov 18, 2004 at 11:22:53AM +0200] > Hi colleagues, > > I was wondering why we still need stuff like autopath? > I simply use a pypy.pth file in my standard python > folder which contains the path to pypy, and things > work fine. > Why do we really need autopath? The basic idea of autopath is that after you do svn co http://codespeak.net/svn/pypy/trunk/src and then python src/pypy/test_all.py or any other test file or e.g. python src/pypy/interpreter/py.py it just works without _any_ setup/installation steps. For the same reason we come with the neccessary dependencies to run py.py. cheers, holger From mwh at python.net Thu Nov 18 12:08:01 2004 From: mwh at python.net (Michael Hudson) Date: Thu, 18 Nov 2004 11:08:01 +0000 Subject: [pypy-dev] Re: autopath obsolete? References: <419C69ED.80703@stackless.com> <2mllczzcg7.fsf@starship.python.net> <3DE7FAE3-394D-11D9-AAFD-000A9567635C@redivi.com> Message-ID: <2mhdnnza7i.fsf@starship.python.net> Bob Ippolito writes: > On Nov 18, 2004, at 12:19 PM, Michael Hudson wrote: > >> Well, if you run stuff from inside the pypy directory it's a good idea >> to get the current directory *out* of sys.path. > > Well, autopath doesn't do that, it just shuffles the pypy base dir to > sys.path[0]. Bah, I did, then it didn't and now it does again. Cheers, mwh -- Guido (like us!) is a bit schizophrenic here: he wants to be a benevolent dictator, but also wants to treat people like grownups. This probably worked better before Python got a large American audience <0.9 wink>. -- Tim Peters, 10 Feb 2000 From hpk at trillke.net Thu Nov 18 12:22:53 2004 From: hpk at trillke.net (holger krekel) Date: Thu, 18 Nov 2004 12:22:53 +0100 Subject: [pypy-dev] Re: [pypy-svn] r7364 - in pypy/trunk/src/pypy: appspace appspace/test module/test In-Reply-To: <20041118110701.A61195B0AB@thoth.codespeak.net> References: <20041118110701.A61195B0AB@thoth.codespeak.net> Message-ID: <20041118112253.GT12400@solar.trillke.net> [arigo at codespeak.net Thu, Nov 18, 2004 at 12:07:01PM +0100] > Author: arigo > Date: Thu Nov 18 12:07:01 2004 > New Revision: 7364 > > [... wrestling commits with appspace-modules ... ] > > Directly testing the class 'complex' buried in __builtin__module.py is funny, > beacuse it's difficult to import this __builtin__module.py directly into > CPython. Well, it's not impossible, given sufficiently obscure hacks. Yadda > yadda boum. Committing applevel_in_cpython.py for that purpose, i.e.: testing > directly in CPython some of the app-level code written in the > module/*module.py files. > > That's obviously a wrong solution. We need something more like splitting the > module/*module.py into several files (e.g. class complex:...) and most of them > could just be imported in CPython too. Such modules should probably be allowed to be a directory which can have multiple files, all of which will be put into module's namespace. This should also make it easier to test them from CPython ... holger From bob at redivi.com Thu Nov 18 12:54:34 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu, 18 Nov 2004 13:54:34 +0200 Subject: [pypy-dev] autopath obsolete? In-Reply-To: <20041118110036.GS12400@solar.trillke.net> References: <419C69ED.80703@stackless.com> <20041118110036.GS12400@solar.trillke.net> Message-ID: <9D608CDA-3958-11D9-AAFD-000A9567635C@redivi.com> On Nov 18, 2004, at 1:00 PM, holger krekel wrote: > [Christian Tismer Thu, Nov 18, 2004 at 11:22:53AM +0200] >> Hi colleagues, >> >> I was wondering why we still need stuff like autopath? >> I simply use a pypy.pth file in my standard python >> folder which contains the path to pypy, and things >> work fine. >> Why do we really need autopath? > > The basic idea of autopath is that after you do > > svn co http://codespeak.net/svn/pypy/trunk/src > > and then > > python src/pypy/test_all.py > > or any other test file or e.g. > > python src/pypy/interpreter/py.py > > it just works without _any_ setup/installation steps. > For the same reason we come with the neccessary > dependencies to run py.py. It's not unreasonable to ask people to add src to their PYTHONPATH by some means (pth file or otherwise). Requiring this simple step, common to just about every other sane Python package, eliminates the need for a distributed autopath entirely. As Michael has mentioned, it makes sense to have a module that cleans sys.path so that you don't have anything crazy in there, but autopath as implemented seems like quite an unnecessary hack to me. I've just committed a smarter implementation that deals with symlinks and such more gracefully. -bob From arigo at tunes.org Fri Nov 19 09:24:23 2004 From: arigo at tunes.org (Armin Rigo) Date: Fri, 19 Nov 2004 08:24:23 +0000 Subject: [pypy-dev] autopath obsolete? In-Reply-To: <9D608CDA-3958-11D9-AAFD-000A9567635C@redivi.com> References: <419C69ED.80703@stackless.com> <20041118110036.GS12400@solar.trillke.net> <9D608CDA-3958-11D9-AAFD-000A9567635C@redivi.com> Message-ID: <20041119082423.GA20083@vicky.ecs.soton.ac.uk> Hi Bob, On Thu, Nov 18, 2004 at 01:54:34PM +0200, Bob Ippolito wrote: > It's not unreasonable to ask people to add src to their PYTHONPATH by > some means (pth file or otherwise). For reference (as Holger told us here), the real reason for autopath is to make all this sys.path hacking local to the project. We can have several copies or branches of PyPy on the same machine and don't have to worry that starting py.py from one of these copies will actually import the pypy package of another. Armin From tismer at stackless.com Fri Nov 19 15:23:20 2004 From: tismer at stackless.com (Christian Tismer) Date: Fri, 19 Nov 2004 16:23:20 +0200 Subject: [pypy-dev] SHA 256, 384, 512: Index of ftp://ftp.philosophysw.com/pub/software/ Message-ID: <419E01D8.7020501@stackless.com> Hi Jacob, Laura, while you are at it... I'd really love to have these as well. Just in case you want to do more hashing stuff :-) ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From bob at redivi.com Fri Nov 19 23:06:10 2004 From: bob at redivi.com (Bob Ippolito) Date: Sat, 20 Nov 2004 00:06:10 +0200 Subject: [pypy-dev] Fwd: [Python-Dev] print "%X" % id(object()) not so nice Message-ID: <384F932E-3A77-11D9-925A-000A9567635C@redivi.com> Looks like I'm not the only one who has noticed this :) Begin forwarded message: > From: James Y Knight > Date: November 19, 2004 10:48:19 PM EET > To: Python Dev > Subject: [Python-Dev] print "%X" % id(object()) not so nice > > I think id() should never be returning a negative number. Both these > behaviors are poor: > > In 2.3: > >>> print "%X" % id(o) > __main__:1: FutureWarning: %u/%o/%x/%X of negative int will return a > signed string in Python 2.4 and up > A5F48198 > > In 2.4: > >>> print "%X" %id(o) > -5FC84D08 > > Pointers are conventionally never treated or printed as signed. In 2.3 > and before, it usually ended up okay, besides the warning, because > "%X" had broken behavior. In 2.4, now it's ending up doing the wrong > thing and printing a confusing value. I propose that id() always > return a positive value. This means that it will sometimes have to > return a long instead of an int, but, it already does that under some > circumstances on some architectures. > > Comments? > > James > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/bob%40redivi.com From tismer at stackless.com Sat Nov 20 17:11:37 2004 From: tismer at stackless.com (Christian Tismer) Date: Sat, 20 Nov 2004 18:11:37 +0200 Subject: [pypy-dev] display problem Message-ID: <419F6CB9.6030904@stackless.com> Hi friends, for somebody who knows about PyGame: When I have the window fullscreen on Windows, switch to other applications and come back later, I get the following exception: """ File "D:\pypy\src\pypy\translator\tool\pygame\graphdisplay.py", line 591, in run self.process_event(events.pop(0)) File "D:\pypy\src\pypy\translator\tool\pygame\graphdisplay.py", line 513, in process_event method(event) File "D:\pypy\src\pypy\translator\tool\pygame\graphdisplay.py", line 562, in process_VideoResize self.resize(event.size) File "D:\pypy\src\pypy\translator\tool\pygame\graphdisplay.py", line 54, in resize self.screen = pygame.display.set_mode((w, h), HWSURFACE|RESIZABLE, 32) pygame.error: DirectDraw2::CreateSurface(PRIMARY): Unsupported mode """ My guess is that some of the mode flags are not suitable if the screen is in full size mode. But I don't know. ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From mwh at python.net Wed Nov 24 01:25:51 2004 From: mwh at python.net (Michael Hudson) Date: Wed, 24 Nov 2004 00:25:51 +0000 Subject: [pypy-dev] late night warning reading Message-ID: <2mhdngxfcg.fsf@starship.python.net> I just thought I share this one: *** WARNING: setattr not wanted on SomePBC(const=, knowntype=, prebuiltinstances={: True}) Oops :-) Cheers, mwh -- Or if you happen to be resigned to the size of your trouser snake and would rather not be reminded of it, training a shared classifier to reject penis-enlargement spam stops Barry from getting the help he so desperately needs. -- Tim Peters, c.l.python From mwh at python.net Wed Nov 24 02:54:19 2004 From: mwh at python.net (Michael Hudson) Date: Wed, 24 Nov 2004 01:54:19 +0000 Subject: [pypy-dev] extreme slowness Message-ID: <2m1xekxb90.fsf@starship.python.net> A few small hacks allowed me to pass the issue currently blocking (attached, why not). Then I thought it got stuck in an infinite loop, but no, it was merely being very, very slow. It takes about 15 minutes to crash on tismerysoft.de, a 3 Ghz P4, so I'd guess at least an hour on this machine, maybe quite a bit more than that. I think what is happening is that *each and every* instance of Function is having it's run() method analyzed, or something like that. More than once, because they get reflowed at least once. This is probably not that good an idea :) Cheers, mwh -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: a-few-small-hacks.diff URL: -------------- next part -------------- -- ARTHUR: But which is probably incapable of drinking the coffee. -- The Hitch-Hikers Guide to the Galaxy, Episode 6 From mwh at python.net Wed Nov 24 10:53:18 2004 From: mwh at python.net (Michael Hudson) Date: Wed, 24 Nov 2004 09:53:18 +0000 Subject: [pypy-dev] Re: extreme slowness References: <2m1xekxb90.fsf@starship.python.net> Message-ID: <2mwtwbwp2p.fsf@starship.python.net> Michael Hudson writes: > A few small hacks allowed me to pass the issue currently blocking > (attached, why not). Then I thought it got stuck in an infinite loop, > but no, it was merely being very, very slow. It takes about 15 > minutes to crash on tismerysoft.de, a 3 Ghz P4, so I'd guess at least > an hour on this machine, maybe quite a bit more than that. Or alternatively it will run my machine out of swap. Oh well! I guess all the debugging history stuff we keep doesn't exactly help from this point of view. Cheers, mwh -- Worryingly, DEFUN appears to be a function that removes all the fun from something: after using it all your code is converted to C++. -- Tim Bradshaw, comp.lang.lisp From hpk at trillke.net Wed Nov 24 11:04:43 2004 From: hpk at trillke.net (holger krekel) Date: Wed, 24 Nov 2004 11:04:43 +0100 Subject: [pypy-dev] Re: extreme slowness In-Reply-To: <2mwtwbwp2p.fsf@starship.python.net> References: <2m1xekxb90.fsf@starship.python.net> <2mwtwbwp2p.fsf@starship.python.net> Message-ID: <20041124100443.GG4550@solar.trillke.net> [Michael Hudson Wed, Nov 24, 2004 at 09:53:18AM +0000] > Michael Hudson writes: > > > A few small hacks allowed me to pass the issue currently blocking > > (attached, why not). Then I thought it got stuck in an infinite loop, > > but no, it was merely being very, very slow. It takes about 15 > > minutes to crash on tismerysoft.de, a 3 Ghz P4, so I'd guess at least > > an hour on this machine, maybe quite a bit more than that. > > Or alternatively it will run my machine out of swap. Oh well! > > I guess all the debugging history stuff we keep doesn't exactly help > from this point of view. yes, i guess "-text" should turn debugging history off because without the flowviewer you stand no chance of browsing this information anyway. Also, interspersed "if debughistory: ..." would make it more obvious what is just there for debugging purposes and what is really needed for annotation/translation to work properly. cheers, holger From mwh at python.net Wed Nov 24 11:30:00 2004 From: mwh at python.net (Michael Hudson) Date: Wed, 24 Nov 2004 10:30:00 +0000 Subject: [pypy-dev] Re: extreme slowness References: <2m1xekxb90.fsf@starship.python.net> <2mwtwbwp2p.fsf@starship.python.net> <20041124100443.GG4550@solar.trillke.net> Message-ID: <2mpt23wndj.fsf@starship.python.net> hpk at trillke.net (holger krekel) writes: > [Michael Hudson Wed, Nov 24, 2004 at 09:53:18AM +0000] >> Michael Hudson writes: >> >> > A few small hacks allowed me to pass the issue currently blocking >> > (attached, why not). Then I thought it got stuck in an infinite loop, >> > but no, it was merely being very, very slow. It takes about 15 >> > minutes to crash on tismerysoft.de, a 3 Ghz P4, so I'd guess at least >> > an hour on this machine, maybe quite a bit more than that. >> >> Or alternatively it will run my machine out of swap. Oh well! >> >> I guess all the debugging history stuff we keep doesn't exactly help >> from this point of view. > > yes, i guess "-text" should turn debugging history off because without > the flowviewer you stand no chance of browsing this information anyway. Well. The reason I was running it on my machine and not tismerysoft.de was that I wanted to look at the flowviewer :) > Also, interspersed "if debughistory: ..." would make it more > obvious what is just there for debugging purposes and what is > really needed for annotation/translation to work properly. I think after someone (you? Christian?) mentioned the confusingness of this, most of the uses of debug only stuff is commented now. Still, indentation would probably help. Cheers, mwh -- (Unfortunately, while you get Tom Baker saying "then we were attacked by monsters", he doesn't flash and make "neeeeooww-sploot" noises.) -- Gareth Marlow, ucam.chat, from Owen Dunn's review of the year From lac at strakt.com Wed Nov 24 12:20:39 2004 From: lac at strakt.com (Laura Creighton) Date: Wed, 24 Nov 2004 12:20:39 +0100 Subject: [pypy-dev] svn error -- is this one on codespeak or mine? Message-ID: <200411241120.iAOBKdkW026616@ratthing-b246.strakt.com> I'm now happily at home. But when I did an svn up on my home local pypy tree I got this: ... long list of expected good updates .... U branch/src-new-utest/std/path/local/api.py U branch/src-new-utest/std/path/local/local_test.py A branch/src-new-utest/std/path/local/local.py U branch/src-new-utest/std/path/local/_posix.py D branch/src-new-utest/std/path/local/xxlocal.py D branch/src-new-utest/std/_package_test.py D branch/src-new-utest/std/_package_.py Updated external to revision 7654. Fetching external item into 'branch/typeinference/vpath' svn: REPORT request failed on '/svn/!svn/vcc/default' svn: Invalid editor anchoring; at least one of the input paths is not a directory and there was no source entry --------------- Is this a codespeak error? or do I have something configured wrong here? Laura From hpk at trillke.net Wed Nov 24 15:06:32 2004 From: hpk at trillke.net (holger krekel) Date: Wed, 24 Nov 2004 15:06:32 +0100 Subject: [pypy-dev] svn error -- is this one on codespeak or mine? In-Reply-To: <200411241120.iAOBKdkW026616@ratthing-b246.strakt.com> References: <200411241120.iAOBKdkW026616@ratthing-b246.strakt.com> Message-ID: <20041124140632.GK4550@solar.trillke.net> Hi Laura, [Laura Creighton Wed, Nov 24, 2004 at 12:20:39PM +0100] > > I'm now happily at home. But when I did an svn up on my home local pypy tree I > got this: > > ... long list of expected good updates .... > U branch/src-new-utest/std/path/local/api.py > U branch/src-new-utest/std/path/local/local_test.py > A branch/src-new-utest/std/path/local/local.py > U branch/src-new-utest/std/path/local/_posix.py > D branch/src-new-utest/std/path/local/xxlocal.py > D branch/src-new-utest/std/_package_test.py > D branch/src-new-utest/std/_package_.py > Updated external to revision 7654. This shows two things: - the svn server works - you are probably using a very old branch are you trying to checkout pypy? If so then you might want to try to move your current pypy src-directory away and cleanly checkout with something like svn co http://codespeak.net/svn/pypy/trunk/src srcpypy (and if you are trying to checkout the py lib then this comes with pypy's src-directory anyway) cheers, holger From tismer at stackless.com Wed Nov 24 15:44:21 2004 From: tismer at stackless.com (Christian Tismer) Date: Wed, 24 Nov 2004 16:44:21 +0200 Subject: [pypy-dev] bug in flow graph? Message-ID: <41A49E45.6010004@stackless.com> Howdy, I added testwise the md5 implementation to my app code translator test cases. It looks quite good, but there is a problem with the flow space: md5 unfortuantely uses copy.deepcopy, which makes the flow space dig into that, and we crash with the message AssertionError: Not generating the same operation sequence: v4162 = simple_call((type type), v4157) ---> | while repeating we see here | v4163 = simple_call((builtin_function_or_method get), v4162) v4163 = simple_call((builtin_function_or_method get), v4162) v4164 = is_true(v4163) >>> I guess there is so much stuff in the copy module that the flow space gets confused somehow. But well, it may be a real bug. For now, I'll replace the copy.deepcopy part of md5 with something else, to protect the analysis. Might this be related to the fact, that copy.deepcopy also does not work with the MD5 object, when I try it in the pypy shell? ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Wed Nov 24 16:04:19 2004 From: tismer at stackless.com (Christian Tismer) Date: Wed, 24 Nov 2004 17:04:19 +0200 Subject: [pypy-dev] bug in flow graph? Message-ID: <41A4A2F3.50209@stackless.com> Howdy, I added testwise the md5 implementation to my app code translator test cases. It looks quite good, but there is a problem with the flow space: md5 unfortuantely uses copy.deepcopy, which makes the flow space dig into that, and we crash with the message AssertionError: Not generating the same operation sequence: v4162 = simple_call((type type), v4157) ---> | while repeating we see here | v4163 = simple_call((builtin_function_or_method get), v4162) v4163 = simple_call((builtin_function_or_method get), v4162) v4164 = is_true(v4163) >>> I guess there is so much stuff in the copy module that the flow space gets confused somehow. But well, it may be a real bug. For now, I'll replace the copy.deepcopy part of md5 with something else, to protect the analysis. Might this be related to the fact, that copy.deepcopy also does not work with the MD5 object, when I try it in the pypy shell? ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From arigo at tunes.org Wed Nov 24 20:39:16 2004 From: arigo at tunes.org (Armin Rigo) Date: Wed, 24 Nov 2004 19:39:16 +0000 Subject: [pypy-dev] svn error -- is this one on codespeak or mine? In-Reply-To: <200411241120.iAOBKdkW026616@ratthing-b246.strakt.com> References: <200411241120.iAOBKdkW026616@ratthing-b246.strakt.com> Message-ID: <20041124193916.GB25037@vicky.ecs.soton.ac.uk> Hello Laura, On Wed, Nov 24, 2004 at 12:20:39PM +0100, Laura Creighton wrote: > ... > U branch/src-new-utest/std/path/local/api.py > ... > Fetching external item into 'branch/typeinference/vpath' To me it looks like you are trying to check out (or rather update) the whole codespeak.net/svn/pypy tree, including all branches and tags! This is probably not a good idea. You should only ever check out codespeak.net/svn/pypy/trunk. Armin Safely-back-home-as-well From lac at strakt.com Wed Nov 24 21:27:20 2004 From: lac at strakt.com (Laura Creighton) Date: Wed, 24 Nov 2004 21:27:20 +0100 Subject: [pypy-dev] svn error -- is this one on codespeak or mine? In-Reply-To: Message from Armin Rigo of "Wed, 24 Nov 2004 19:39:16 GMT." <20041124193916.GB25037@vicky.ecs.soton.ac.uk> References: <200411241120.iAOBKdkW026616@ratthing-b246.strakt.com> <20041124193916.GB25037@vicky.ecs.soton.ac.uk> Message-ID: <200411242027.iAOKRKdP027947@ratthing-b246.strakt.com> In a message of Wed, 24 Nov 2004 19:39:16 GMT, Armin Rigo writes: >Hello Laura, > >On Wed, Nov 24, 2004 at 12:20:39PM +0100, Laura Creighton wrote: >> ... >> U branch/src-new-utest/std/path/local/api.py >> ... >> Fetching external item into 'branch/typeinference/vpath' > >To me it looks like you are trying to check out (or rather update) the wh >ole >codespeak.net/svn/pypy tree, including all branches and tags! This is >probably not a good idea. You should only ever check out >codespeak.net/svn/pypy/trunk. > > >Armin Safely-back-home-as-well You are correct (as usual). I was one level higher in the directory than I thought I was. Laura From rxe at ukshells.co.uk Thu Nov 25 00:33:55 2004 From: rxe at ukshells.co.uk (Richard Emslie) Date: Wed, 24 Nov 2004 23:33:55 +0000 (GMT) Subject: [pypy-dev] Sprint wonderings Message-ID: Hi, I hope the past sprint was fun and I was wondering if any kind person from it would like to give an overview of activities, developments etc.... please? That would be really great!! Thanks, Richard From guenter.jantzen at netcologne.de Thu Nov 25 00:38:16 2004 From: guenter.jantzen at netcologne.de (=?Windows-1252?Q?G=FCnter_Jantzen?=) Date: Thu, 25 Nov 2004 00:38:16 +0100 Subject: [pypy-dev] 7564 - seems to be great login. Congratulations! Message-ID: <00cb01c4d27e$ac92a360$2e5ca8d5@Lieschen2> > Date: Mon, 22 Nov 2004 14:56:26 +0100 (MET) > From: arigo at codespeak.net > Subject: [pypy-svn] r7564 - pypy/trunk/src/goal > To: pypy-svn at codespeak.net > Message-ID: <20041122135626.BBE495AF42 at thoth.codespeak.net> > Content-Type: text/plain; charset=UTF-8 > > Author: arigo > Date: Mon Nov 22 14:56:26 2004 > New Revision: 7564 > > Modified: > pypy/trunk/src/goal/translate_pypy.py > Log: > Yoohoo! translate_pypy.py -no-a works! From sanxiyn at gmail.com Thu Nov 25 04:00:22 2004 From: sanxiyn at gmail.com (Sanghyeon Seo) Date: Thu, 25 Nov 2004 12:00:22 +0900 Subject: [pypy-dev] Sprint wonderings In-Reply-To: References: Message-ID: <5b024817041124190011bd2227@mail.gmail.com> Richard Emslie wrote: > I hope the past sprint was fun and I was wondering if any kind person from > it would like to give an overview of activities, developments etc... > please? That would be really great! I don't know much, but from pypy-svn checkin messages it seems the good place to start is to run: pypy/trunk/src/goal/translate_pypy.py You can see what options are available by passing -h (for help) to it. It will try to annotate and translate PyPy, and will also launch Pygame viewer to navigate some enourmous datastructures it produces. Lots of changes in annotation directory. builtin.py annotates some builtin functions' effect. Pygame viewer is at translator/tools/pygame. In translator directory, main changes were in genc.h (Macro and utility function definitions) and genc.py. appspace directory got file object written in Python, using low-level os module. Also MD5 and SHA module in Python. No doubt others will do much better job on this than me. :-) From mwh at python.net Thu Nov 25 13:42:35 2004 From: mwh at python.net (Michael Hudson) Date: Thu, 25 Nov 2004 12:42:35 +0000 Subject: [pypy-dev] Re: Sprint wonderings References: Message-ID: <2mllcqw150.fsf@starship.python.net> Richard Emslie writes: > Hi, > > I hope the past sprint was fun and I was wondering if any kind person > from it would like to give an overview of activities, developments > etc.... please? That would be really great!! Hmm, I was there and I could do with the same! :-) Reading pypy-svn is probably the best way to get an idea of what happened. Most of the work was on the annontator/translator, although there was some appspace stuff -- md5, sha-1 hashes and files, although files are not plumbed into the rest of pypy yet -- and the usual polishing of the standard objects space. On the annotator side, there seems to be a general feeling that we have roughly the right approach now, although there are still an unknown number of hopefully small details to work out. The translator basically works, but there are some nasty hacks knocking about and we haven't really tackled the gnarly issue of how to make use of type annotations (currently the annotator is only used to work out *which* functions are called, and the knowledge of *how* they are called is ignored). Evolution, not revolution, I'd say. Cheers, mwh -- MGM will not get your whites whiter or your colors brighter. It will, however, sit there and look spiffy while sucking down a major honking wad of RAM. -- http://www.xiph.org/mgm/ From mwh at python.net Thu Nov 25 13:57:24 2004 From: mwh at python.net (Michael Hudson) Date: Thu, 25 Nov 2004 12:57:24 +0000 Subject: [pypy-dev] Re: [pypy-svn] r7670 - pypy/trunk/src/pypy/annotation In-Reply-To: <20041125104547.3F1085B151@thoth.codespeak.net> (arigo@codespeak.net's message of "Thu, 25 Nov 2004 11:45:47 +0100 (MET)") References: <20041125104547.3F1085B151@thoth.codespeak.net> Message-ID: <2md5y2w0gb.fsf@starship.python.net> arigo at codespeak.net writes: > Author: arigo > Date: Thu Nov 25 11:45:46 2004 > New Revision: 7670 > > Modified: > pypy/trunk/src/pypy/annotation/bookkeeper.py > Log: > Fixed the handling of the implicit call to __init__ upon instance creation. Thank you! I didn't actually mean to check that code in, but it didn't obviously break anything. I should probably still have backed it out. Cheers, mwh PS: pypy-svn should probably have a Reply-To: of pypy-dev. The usual reasons why this is Evil and Wrong don't really apply to this list (esp. as I don't think @codespeak.net addresses work :). Cheers, mwh -- The ability to quote is a serviceable substitute for wit. -- W. Somerset Maugham From tismer at stackless.com Thu Nov 25 15:48:10 2004 From: tismer at stackless.com (Christian Tismer) Date: Thu, 25 Nov 2004 16:48:10 +0200 Subject: [pypy-dev] Re: Sprint wonderings In-Reply-To: <2mllcqw150.fsf@starship.python.net> References: <2mllcqw150.fsf@starship.python.net> Message-ID: <41A5F0AA.9040201@stackless.com> Michael Hudson wrote: > Richard Emslie writes: > > >>Hi, >> >>I hope the past sprint was fun and I was wondering if any kind person >>from it would like to give an overview of activities, developments >>etc.... please? That would be really great!! > > > Hmm, I was there and I could do with the same! :-) Well, let me add the little bits from my side. It took me long to get an understanding of everything since I was practically out of the project for half a year. Then I worked on a different topic which we had not touched yet, but it made some sense. I'm trying to generate interpreter level code from appspace code. That is, I run the flow machine over some python code which is *not* supposed to be restricted Python, but application code. The latter is used to implement lots of functions which would look nasty when implemented in RPython (see for instance _formatting.py, md5.py etc.). The idea is to do a flow analysis of these implementations without annotation and create RPython source code for that what the interpreter would have executed. The hope is that we *then* can apply the translator to this generated Python code, so it gets annotated, simplified and turned into C code. I'm still working on it, and there are a lot of things wrong or missing, but it gets better and will probably be ready in a couple of days. I guess it will be built in as a preprocessor, which compiles app code into interpreter code in some cache directory. Note that this transformation does not treat the app code as full Python, but again some restrictions apply: - globals are constant - methods never change - imports are done at compile time - exceptions are not generated if there is no try clause - imported stuff must obey the same restrictions > Evolution, not revolution, I'd say. Yeah, but the results made me very positive about PyPy's future. ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From hpk at trillke.net Thu Nov 25 19:05:54 2004 From: hpk at trillke.net (holger krekel) Date: Thu, 25 Nov 2004 19:05:54 +0100 Subject: [pypy-dev] Re: [pypy-svn] r7671 - pypy/trunk/src/pypy/annotation In-Reply-To: <20041125113725.BAD515B051@thoth.codespeak.net> References: <20041125113725.BAD515B051@thoth.codespeak.net> Message-ID: <20041125180554.GT4550@solar.trillke.net> [arigo at codespeak.net Thu, Nov 25, 2004 at 12:37:25PM +0100] > Author: arigo > Date: Thu Nov 25 12:37:25 2004 > New Revision: 7671 > > Modified: > pypy/trunk/src/pypy/annotation/binaryop.py > pypy/trunk/src/pypy/annotation/unaryop.py > Log: > The assert contains() is still failing for wrong reasons related to revision > numbers. Gave up and implemented a contains() that ignores revision numbers. Heh! :-) holger (just arrived back home) From bauflo3 at web.de Thu Nov 25 16:53:33 2004 From: bauflo3 at web.de (Florian Bauer) Date: Thu, 25 Nov 2004 16:53:33 +0100 Subject: [pypy-dev] binascii.py in pure python Message-ID: <1424705715@web.de> Hi there, I checked the list of missing C modules on the wiki pages. I could contribute some code for binascii.py. Some time ago i started porting the C module to python. It's halfway done, but some functions are pretty usable and have some test coverage. Right now, I don't have time to download pypy and play with it, so I don't know if there are any issues with integrating my code. It would be best if I could develop and test the module on CPython. Cheers Florian ________________________________________________________________ Verschicken Sie romantische, coole und witzige Bilder per SMS! Jetzt neu bei WEB.DE FreeMail: http://freemail.web.de/?mc=021193 From hpk at trillke.net Thu Nov 25 19:09:15 2004 From: hpk at trillke.net (holger krekel) Date: Thu, 25 Nov 2004 19:09:15 +0100 Subject: [pypy-dev] binascii.py in pure python In-Reply-To: <1424705715@web.de> References: <1424705715@web.de> Message-ID: <20041125180915.GU4550@solar.trillke.net> Hi Florian, [Florian Bauer Thu, Nov 25, 2004 at 04:53:33PM +0100] > Hi there, > > I checked the list of missing C modules on the wiki pages. > I could contribute some code for binascii.py. nice. > Some time ago i started porting the C module to python. > It's halfway done, but some functions are pretty usable and have some test coverage. > Right now, I don't have time to download pypy and play with it, so I don't know if there are any issues with integrating my code. It would be best if I could develop and test the module on CPython. that's just fine. Actually it's faster to develop against CPython and not let the current PyPy interpret your application level code. cheers, holger From hpk at trillke.net Thu Nov 25 19:18:17 2004 From: hpk at trillke.net (holger krekel) Date: Thu, 25 Nov 2004 19:18:17 +0100 Subject: [pypy-dev] Re: [pypy-svn] r7670 - pypy/trunk/src/pypy/annotation In-Reply-To: <2md5y2w0gb.fsf@starship.python.net> References: <20041125104547.3F1085B151@thoth.codespeak.net> <2md5y2w0gb.fsf@starship.python.net> Message-ID: <20041125181817.GV4550@solar.trillke.net> [Michael Hudson Thu, Nov 25, 2004 at 12:57:24PM +0000] > PS: pypy-svn should probably have a Reply-To: of pypy-dev. The usual > reasons why this is Evil and Wrong don't really apply to this list > (esp. as I don't think @codespeak.net addresses work :). They work for some of us. You can easily make it work for you if you put a ".forward" file in your home directory on codespeak. But i agree that replying to a pypy-svn notification should just get you pypy-dev. I configured it this way now. If we discover that people are regularly misposting private details of their love live in reply to a svn notification we can revert it, i guess. cheers, holger From tismer at stackless.com Thu Nov 25 19:40:32 2004 From: tismer at stackless.com (Christian Tismer) Date: Thu, 25 Nov 2004 20:40:32 +0200 Subject: [pypy-dev] binascii.py in pure python In-Reply-To: <20041125180915.GU4550@solar.trillke.net> References: <1424705715@web.de> <20041125180915.GU4550@solar.trillke.net> Message-ID: <41A62720.2090702@stackless.com> holger krekel wrote: > Hi Florian, > > [Florian Bauer Thu, Nov 25, 2004 at 04:53:33PM +0100] > >>Hi there, >> >>I checked the list of missing C modules on the wiki pages. >>I could contribute some code for binascii.py. > > > nice. > > >>Some time ago i started porting the C module to python. >>It's halfway done, but some functions are pretty usable and have some test coverage. >>Right now, I don't have time to download pypy and play with it, so I don't know if there are any issues with integrating my code. It would be best if I could develop and test the module on CPython. > > > that's just fine. Actually it's faster to develop against CPython and > not let the current PyPy interpret your application level code. As an addition: I'm just busy figuring out how to make C modules which are implemented in Python easier to compile back to C. It turned out that there is only little to change if your implementation does not use fancy features like generators. If you can assume that - all your globals are constant after initialization - you don't use ints or longs larger than machine words - methods are constants and not shadowed by instance vars - exceptions are raised only if you provide a try statement - no generators - no imports of modules which don't obey these rules - use __all__ to report the exports then this module is almost ready to become a builtin module. I just have to convert the exported objects in __all__ an give them an application-python interface, again. ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Thu Nov 25 20:04:18 2004 From: tismer at stackless.com (Christian Tismer) Date: Thu, 25 Nov 2004 21:04:18 +0200 Subject: [pypy-dev] appspace considerations and genrpy Message-ID: <41A62CB2.2050504@stackless.com> Hi Armin, all, finally I realized that I'm doing way too much in my translation efforts. It happened to me when I turned md5.py into RPython using my prototypic genrpy transformer, that this is in many cases not needed at all. If you look at md5.py, you'll see that it *is* already almost restricted Python. I can just move almost all functions over to the interpreter level. In a sense, md5 and sha do not really belong to the appspace modules. Instead, they are almost ready internal implementations. Just the interface to the exported objects and methods needs to be created, to provide the importable builtin module. How do we handle such things? I guess such almost-interpreter level things should have a different folder. The published module would simply be filled with app2interp gateway calls to the internal stuff, or, well, directly writing down the wrappers would be fine as well, I guess. So my idea for cases like such is: Instead of transforming everything from appspace into interpreter level, we can use stuff almost as-is. Only non-trivial datatypes or unsupported features need to be translated from app to interp. Please correct me if I'm wrong. I was just shocked when I read the transformed MD5 source, like "hell, what am I doing here, this is no app code at all!" :-) Any ideas how to express the necessary actions? Do we move those files elsewhere? Do we use the __all__ attribute to name the interface stuff, or do we simply add the interface things by hand? thx & ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From arigo at tunes.org Fri Nov 26 11:56:31 2004 From: arigo at tunes.org (Armin Rigo) Date: Fri, 26 Nov 2004 10:56:31 +0000 Subject: [pypy-dev] Sprint wonderings In-Reply-To: References: Message-ID: <20041126105631.GA27524@vicky.ecs.soton.ac.uk> Hi, For a number of us, the sprint was a full week of running translate_pypy.py and wait for the next crash, then debugging and thinking and fixing and workarounding the issue. Then start again at step #1. In the last days it generally took some time (and lots of memory) before it ran into a problem and crashed. It also reported increasingly large amounts of warnings... well, more precisely, we turned increasingly many errors into warnings :-) These ones report problems that should eventually be fixed. This mostly occupied Marius, Holger, Michael and myself. (Cheers to Marius, btw, for having caught up so quickly with PyPy!) Some time in the middle of it all, the basic annotation-less 'translate_pypy.py -no-a' was able to compile and run (-6)*(-7) and successfully gave us the ultimate answer. I guess the next step here is to try to change the 'entry_point' so that 'translate_pypy' can actually compile and run the interactive command-line interpreter. Bob worked on more specific pieces of code: the user interface of the Pygame graph viewer (Marius did bits of that too), and the genc.py C code generation, which now generates code with tons of debugging features. The best one is probably that if the generated C code fails with a Python exception, you can use pdb.pm() to debug it! You see the C frames with the content of their variables, with line numbers set correctly. pdb will display extracts of the C source code in the tracebacks :-) Christian worked on using the flow graph analysis to understand application-level code and transform them to interpreter-level code. For example, each + in the source becomes an add operation in the flow graph (using the existing almost-unmodified flow object space), which then (this is Christian's work) gets translated as a space.add() by genrpy.py. This is useful for application-level helper code, for example string formatting, which is currently written in _formatting.py as regular, app-level Python code. The goal is to turn this nice piece of code into its obfuscated interp-level equivalent automatically. This is essentially an optimization only, which will allow the code to run a bit faster because PyPy's own interpreter doesn't have to interpret the app-level _formatting.py every time it has to do a string formatting operation. It also allows future more advanced optimizations. Jacob and Laura, who didn't feel like digging into the PyPy internals too deepdy, focused on reimplementing some of the C functionality in plain Python. We now have MD5 and SHA-1 and a 'class file' replacement based on low-level primitives (generally the 'os' module). You can play with the 'file' class by importing 'appspace/_file.py' in CPython. The general guidelines to integrate this kind of pure Python module with PyPy is: (1) try the module in py.py when it works in CPython, and (2) plug it -- for the 'file' class, it's done by inserting 'from _file import file' in 'module/__builtin__module.py'. Warning, this will probably be quite slow :-) Finally, we got news from the EU. In all likelyhood the project will really start on the 1st of December. Yippee! End of the phase 1 of the administrative overload! Now we have to plan for more closely-packed sprints. The next one might take place somewhere around the end of January in Switzerland. Last but not least, thanks to the Programmers of Vilnius for the excellent sprint organization! See you soon, Armin. From xavier at zope-europe.org Fri Nov 26 13:59:14 2004 From: xavier at zope-europe.org (Xavier Heymans) Date: Fri, 26 Nov 2004 13:59:14 +0100 Subject: [pypy-dev] Congratulations to the Pypy team Message-ID: Last week Paul and I participated to the Calibre conference in Den Haag, Netherlands. Paul was invited to make the keynote speech. Some high-ranking EU commissioners were presents. They were very interested in open source and knew all the details about the Pypy project. Congratulations to the Pypy team for your project, your excellent lobby and for having open the EU minds to pen source. Xavier Xavier Heymans ----- xavier at zope-europe.org T?l : 00 32 - (0)10 45 99 02 Mobile : 00 32 - (0)478 516 777 Zope Europe Association ASBL Avenue Maeterlinck 18, 1348 Louvain-La-Neuve, Belgium Registered in Belgium n? 865 879 012 - VAT/TVA : BE 0865 879 012 From hpk at trillke.net Fri Nov 26 16:09:24 2004 From: hpk at trillke.net (holger krekel) Date: Fri, 26 Nov 2004 16:09:24 +0100 Subject: [pypy-dev] Congratulations to the Pypy team In-Reply-To: References: Message-ID: <20041126150924.GE4550@solar.trillke.net> Hi Xavier, [Xavier Heymans Fri, Nov 26, 2004 at 01:59:14PM +0100] > Last week Paul and I participated to the Calibre conference in Den Haag, > Netherlands. Paul was invited to make the keynote speech. Some high-ranking > EU commissioners were presents. They were very interested in open source and > knew all the details about the Pypy project. > > Congratulations to the Pypy team for your project, your excellent lobby and > for having open the EU minds to pen source. Great to hear! This is another positive sign that eventually everything will work out nicely. Just this monday the signing process was initiated by the EU which - nevertheless - still does not imply a formalized commitment. But we are all getting ready to fully start the EU project on 1st of December. Btw, everyone please spread the word that we plan to do about 14 coding sprints in the next two years in various places, ranging from Pycon in Washington DC to South Korea but most of them taking place in Europe. European hackers will very likely have the possibility to "join" the EU project and receive travel + accomodation costs for these sprints refunded. We still need to sort out ways to make this as unbuerocratic as possible, though, and there are some questions and details that we must first resolve. And if anyone likes to help with organizing one of the sprints that would be very helpful as these events really form the foundation of the PyPy project and they are much more fun and easier to do if there are locals to interact with ... cheers and greetings, holger From bauflo3 at web.de Fri Nov 26 16:40:18 2004 From: bauflo3 at web.de (Florian Bauer) Date: Fri, 26 Nov 2004 16:40:18 +0100 Subject: [pypy-dev] binascii.py in pure python In-Reply-To: <41A62720.2090702@stackless.com> References: <1424705715@web.de> <20041125180915.GU4550@solar.trillke.net> <41A62720.2090702@stackless.com> Message-ID: <41A74E62.30404@web.de> Christian Tismer wrote: > holger krekel wrote: > >> Hi Florian, >> [Florian Bauer Thu, Nov 25, 2004 at 04:53:33PM +0100] >> >>> Hi there, >>> >>> I checked the list of missing C modules on the wiki pages. >>> I could contribute some code for binascii.py. >> >> >> >> nice. >> >>> Some time ago i started porting the C module to python. >>> It's halfway done, but some functions are pretty usable and have >>> some test coverage. >>> Right now, I don't have time to download pypy and play with it, so I >>> don't know if there are any issues with integrating my code. It >>> would be best if I could develop and test the module on CPython. >> >> >> >> that's just fine. Actually it's faster to develop against >> CPython and >> not let the current PyPy interpret your application level code. > > > As an addition: > I'm just busy figuring out how to make C modules which are implemented > in Python easier to compile back to C. > It turned out that there is only little to change if your > implementation does not use fancy features like generators. > If you can assume that > - all your globals are constant after initialization > - you don't use ints or longs larger than machine words > - methods are constants and not shadowed by instance vars > - exceptions are raised only if you provide a try statement > - no generators > - no imports of modules which don't obey these rules > - use __all__ to report the exports > > then this module is almost ready to become a builtin module. > I just have to convert the exported objects in __all__ > an give them an application-python interface, again. > > ciao - chris This is pretty trivial in the case of binascii.py. The interface assumes strings, not iterables, so there's no need for fancy stuff. What I'm thinking about is whether I should use regular expressions or not. I haven't played around with it yet, but I guess that at least for running under CPython re.sub woud be faster than a state machine with the loop coded in python. But in pypy, maybe not. Any thoughts on this matter? Btw. I will need some time to clean up the module. In the next 10 days I'm pretty busy with matching the deadline for my thesis . But afterwards, I'll clean up my code and submit it. Florian From hpk at trillke.net Fri Nov 26 16:51:36 2004 From: hpk at trillke.net (holger krekel) Date: Fri, 26 Nov 2004 16:51:36 +0100 Subject: [pypy-dev] binascii.py in pure python In-Reply-To: <41A74E62.30404@web.de> References: <1424705715@web.de> <20041125180915.GU4550@solar.trillke.net> <41A62720.2090702@stackless.com> <41A74E62.30404@web.de> Message-ID: <20041126155136.GG4550@solar.trillke.net> Hi Florian, [Florian Bauer Fri, Nov 26, 2004 at 04:40:18PM +0100] > What I'm thinking about is whether I should use regular expressions or > not. I haven't played around with it yet, but I guess that at least for > running under CPython re.sub woud be faster than a state machine with > the loop coded in python. But in pypy, maybe not. Any thoughts on this > matter? We don't have a regexp implementation in PyPy yet. We just reuse cpython's c-coded one. IIRC, Jonathan David Riehl had some kind of a python-coded regexpish state machine for his tokenizer+parser suite completly written in Python. The according code is here: http://codespeak.net/svn/basil/trunk/basil/lang/python/ > Btw. I will need some time to clean up the module. In the next 10 days > I'm pretty busy with matching the deadline for my thesis . > But afterwards, I'll clean up my code and submit it. fine! and good luck ... holger From tismer at stackless.com Fri Nov 26 16:57:15 2004 From: tismer at stackless.com (Christian Tismer) Date: Fri, 26 Nov 2004 17:57:15 +0200 Subject: [pypy-dev] binascii.py in pure python In-Reply-To: <41A74E62.30404@web.de> References: <1424705715@web.de> <20041125180915.GU4550@solar.trillke.net> <41A62720.2090702@stackless.com> <41A74E62.30404@web.de> Message-ID: <41A7525B.3080103@stackless.com> Florian Bauer wrote: > Christian Tismer wrote: > >> holger krekel wrote: >> >>> Hi Florian, >>> [Florian Bauer Thu, Nov 25, 2004 at 04:53:33PM +0100] >>> >>>> Hi there, >>>> >>>> I checked the list of missing C modules on the wiki pages. >>>> I could contribute some code for binascii.py. >>> >>> >>> >>> >>> nice. >>> >>>> Some time ago i started porting the C module to python. >>>> It's halfway done, but some functions are pretty usable and have >>>> some test coverage. >>>> Right now, I don't have time to download pypy and play with it, so I >>>> don't know if there are any issues with integrating my code. It >>>> would be best if I could develop and test the module on CPython. >>> >>> >>> >>> >>> that's just fine. Actually it's faster to develop against >>> CPython and >>> not let the current PyPy interpret your application level code. >> >> >> >> As an addition: >> I'm just busy figuring out how to make C modules which are implemented >> in Python easier to compile back to C. >> It turned out that there is only little to change if your >> implementation does not use fancy features like generators. >> If you can assume that >> - all your globals are constant after initialization >> - you don't use ints or longs larger than machine words >> - methods are constants and not shadowed by instance vars >> - exceptions are raised only if you provide a try statement >> - no generators >> - no imports of modules which don't obey these rules >> - use __all__ to report the exports >> >> then this module is almost ready to become a builtin module. >> I just have to convert the exported objects in __all__ >> an give them an application-python interface, again. >> >> ciao - chris > > > This is pretty trivial in the case of binascii.py. The interface assumes > strings, not iterables, so there's no need for fancy stuff. Good. > What I'm thinking about is whether I should use regular expressions or > not. I looked over binascii.c and found no real reason to use regexen. If I would do it, I would probably take the C source and tweak it until it is Python. Or did you plan to do a re implementation? :-)) > I haven't played around with it yet, but I guess that at least for > running under CPython re.sub woud be faster than a state machine with > the loop coded in python. But in pypy, maybe not. Any thoughts on this > matter? Well,re.sub uses state machines as well, and re will finally be implemented in Python as well. So there will not be that difference. I you use state machines, write simple code and just think you are coding in C. Your code will later be translated into C, and it will be simplified to use machine words as much as possible. Good luck with your thesis - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From bauflo3 at web.de Fri Nov 26 17:35:06 2004 From: bauflo3 at web.de (Florian Bauer) Date: Fri, 26 Nov 2004 17:35:06 +0100 Subject: [pypy-dev] binascii.py in pure python In-Reply-To: <41A7525B.3080103@stackless.com> References: <1424705715@web.de> <20041125180915.GU4550@solar.trillke.net> <41A62720.2090702@stackless.com> <41A74E62.30404@web.de> <41A7525B.3080103@stackless.com> Message-ID: <41A75B3A.8060902@web.de> Christian Tismer wrote: > Florian Bauer wrote: > >> Christian Tismer wrote: >> >>> holger krekel wrote: >>> >>>> Hi Florian, >>>> [Florian Bauer Thu, Nov 25, 2004 at 04:53:33PM +0100] >>>> >>>>> Hi there, >>>>> >>>>> I checked the list of missing C modules on the wiki pages. >>>>> I could contribute some code for binascii.py. >>>> >>>> >>>> >>>> >>>> >>>> nice. >>>> >>>>> Some time ago i started porting the C module to python. >>>>> It's halfway done, but some functions are pretty usable and have >>>>> some test coverage. >>>>> Right now, I don't have time to download pypy and play with it, so >>>>> I don't know if there are any issues with integrating my code. It >>>>> would be best if I could develop and test the module on CPython. >>>> >>>> >>>> >>>> >>>> >>>> that's just fine. Actually it's faster to develop against >>>> CPython and >>>> not let the current PyPy interpret your application level code. >>> >>> >>> >>> >>> As an addition: >>> I'm just busy figuring out how to make C modules which are implemented >>> in Python easier to compile back to C. >>> It turned out that there is only little to change if your >>> implementation does not use fancy features like generators. >>> If you can assume that >>> - all your globals are constant after initialization >>> - you don't use ints or longs larger than machine words >>> - methods are constants and not shadowed by instance vars >>> - exceptions are raised only if you provide a try statement >>> - no generators >>> - no imports of modules which don't obey these rules >>> - use __all__ to report the exports >>> >>> then this module is almost ready to become a builtin module. >>> I just have to convert the exported objects in __all__ >>> an give them an application-python interface, again. >>> >>> ciao - chris >> >> >> >> This is pretty trivial in the case of binascii.py. The interface >> assumes strings, not iterables, so there's no need for fancy stuff. > > > Good. > >> What I'm thinking about is whether I should use regular expressions >> or not. > > > I looked over binascii.c and found no real reason to use regexen. > If I would do it, I would probably take the C source and tweak it > until it is Python. > That's what I'm doing :-) > Or did you plan to do a re implementation? :-)) Rather not. >> I haven't played around with it yet, but I guess that at least for >> running under CPython re.sub woud be faster than a state machine with >> the loop coded in python. But in pypy, maybe not. Any thoughts on >> this matter? > > > Well,re.sub uses state machines as well, and re will finally be > implemented in Python as well. So there will not be that difference. > > I you use state machines, write simple code and just think you > are coding in C. Your code will later be translated into C, > and it will be simplified to use machine words as much as possible. Ok. Thats what my code looks like at the moment. > Good luck with your thesis - chris > Thanks (also to Holger)! Florian From xavier at zope-europe.org Fri Nov 26 18:01:08 2004 From: xavier at zope-europe.org (Xavier Heymans) Date: Fri, 26 Nov 2004 18:01:08 +0100 Subject: [pypy-dev] Congratulations to the Pypy team In-Reply-To: <20041126150924.GE4550@solar.trillke.net> Message-ID: Hi Holger, At the conference the Pypy project was presented as approved. It was listed within the EU projects the day before. I think you are among the first open source projects approved. So I guess you have a 99% change to get it trough. The Eu commissioner explained that it had been difficult to support it because most people do not understand how open source is organized. I would be interested to receive information about your project because the same work has to be done with Zope, Plone, ... Who should I contact? Crossing fingers until December 1 and good luck, Xavier > De?: hpk at trillke.net (holger krekel) > Date?: Fri, 26 Nov 2004 16:09:24 +0100 > ??: Xavier Heymans > Cc?: pypy-dev at codespeak.net > Objet?: Re: [pypy-dev] Congratulations to the Pypy team > > Hi Xavier, > > [Xavier Heymans Fri, Nov 26, 2004 at 01:59:14PM +0100] >> Last week Paul and I participated to the Calibre conference in Den Haag, >> Netherlands. Paul was invited to make the keynote speech. Some high-ranking >> EU commissioners were presents. They were very interested in open source and >> knew all the details about the Pypy project. >> >> Congratulations to the Pypy team for your project, your excellent lobby and >> for having open the EU minds to pen source. > > Great to hear! This is another positive sign that eventually everything > will work out nicely. Just this monday the signing process was initiated > by the EU which - nevertheless - still does not imply a formalized commitment. > But we are all getting ready to fully start the EU project on 1st of December. > > Btw, everyone please spread the word that we plan to do about 14 coding > sprints > in the next two years in various places, ranging from Pycon in Washington DC > to South Korea but most of them taking place in Europe. European hackers > will very likely have the possibility to "join" the EU project and receive > travel + accomodation costs for these sprints refunded. We still need to > sort out ways to make this as unbuerocratic as possible, though, and there > are some questions and details that we must first resolve. > > And if anyone likes to help with organizing one of the sprints that would > be very helpful as these events really form the foundation of the PyPy > project and they are much more fun and easier to do if there are locals > to interact with ... > > cheers and greetings, > > holger > From hpk at trillke.net Fri Nov 26 18:16:21 2004 From: hpk at trillke.net (Holger Krekel) Date: Fri, 26 Nov 2004 18:16:21 +0100 Subject: [pypy-dev] Congratulations to the Pypy team In-Reply-To: References: <20041126150924.GE4550@solar.trillke.net> Message-ID: <20041126171621.GJ4550@solar.trillke.net> Hi Xavier, [Xavier Heymans Fri, Nov 26, 2004 at 06:01:08PM +0100] > Hi Holger, > > At the conference the Pypy project was presented as approved. It was listed > within the EU projects the day before. I think you are among the first open > source projects approved. So I guess you have a 99% change to get it trough. > The Eu commissioner explained that it had been difficult to support it > because most people do not understand how open source is organized. Indeed, both sides have to learn i presume. > I would be interested to receive information about your project because the > same work has to be done with Zope, Plone, ... Who should I contact? As we two are scheduled to meet sometime in December we can have a beer over it :-) Also I am going to give a talk about EU/PyPy/open source funding issues at the chaos communication conference (CCC) in Berlin end of this year. Other than that I suppose it all depends a bit on the nature of your questions who you should best talk to. cheers, holger From lac at strakt.com Fri Nov 26 22:01:30 2004 From: lac at strakt.com (Laura Creighton) Date: Fri, 26 Nov 2004 22:01:30 +0100 Subject: [pypy-dev] Congratulations to the Pypy team In-Reply-To: Message from Xavier Heymans of "Fri, 26 Nov 2004 18:01:08 +0100." References: Message-ID: <200411262101.iAQL1Ut1001254@ratthing-b246.strakt.com> In a message of Fri, 26 Nov 2004 18:01:08 +0100, Xavier Heymans writes: >Hi Holger, > >At the conference the Pypy project was presented as approved. It was list >ed >within the EU projects the day before. I think you are among the first op >en >source projects approved. So I guess you have a 99% change to get it trou >gh. >The Eu commissioner explained that it had been difficult to support it >because most people do not understand how open source is organized. > >I would be interested to receive information about your project because t >he >same work has to be done with Zope, Plone, ... Who should I contact? > >Crossing fingers until December 1 and good luck, > >Xavier At some point, I think it would be good to write a document for the EU - 'Making it easier to fund open source projects' -- why the rules that you use makes things harder for us. From what you say, it looks as if there are some EU Commissioners who would welcome reading such a thing. Laura From mwh at python.net Sun Nov 28 16:47:49 2004 From: mwh at python.net (Michael Hudson) Date: Sun, 28 Nov 2004 15:47:49 +0000 Subject: [pypy-dev] Re: [pypy-svn] r7670 - pypy/trunk/src/pypy/annotation References: <20041125104547.3F1085B151@thoth.codespeak.net> <2md5y2w0gb.fsf@starship.python.net> <20041125181817.GV4550@solar.trillke.net> Message-ID: <2mhdnaug9m.fsf@starship.python.net> hpk at trillke.net (holger krekel) writes: > [Michael Hudson Thu, Nov 25, 2004 at 12:57:24PM +0000] >> PS: pypy-svn should probably have a Reply-To: of pypy-dev. The usual >> reasons why this is Evil and Wrong don't really apply to this list >> (esp. as I don't think @codespeak.net addresses work :). > > They work for some of us. You can easily make it work for you > if you put a ".forward" file in your home directory on codespeak. I just typed 'mutt' on codespeak and found that mwh at codespeak.net does in fact work, but that it's mostly spammers who've noticed this so far... I have enough email addresses already, I think. > But i agree that replying to a pypy-svn notification should > just get you pypy-dev. I configured it this way now. Thank you all the same :) > If we discover that people are regularly misposting private > details of their love live in reply to a svn notification we can > revert it, i guess. :) Cheers, mwh -- glyph: I don't know anything about reality. -- from Twisted.Quotes From tismer at stackless.com Mon Nov 29 13:32:11 2004 From: tismer at stackless.com (Christian Tismer) Date: Mon, 29 Nov 2004 13:32:11 +0100 Subject: [pypy-dev] right-dragging the graph Message-ID: <41AB16CB.1030503@stackless.com> Hi Armin, I thought it was an oversight that the image is not fixed under the cursor when right-dragging it, and I thought to repair this. But I saw self.viewer.shiftoffset(-2*dx, -2*dy) in process_MouseMotion (graphdisplay.py), so this is probably intended behavior? (I'd personally prefer to keep the mouse at the exact picture position) ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From mwh at python.net Mon Nov 29 19:15:02 2004 From: mwh at python.net (Michael Hudson) Date: Mon, 29 Nov 2004 18:15:02 +0000 Subject: [pypy-dev] dictionaries in RPython Message-ID: <2my8gkses9.fsf@starship.python.net> What is the current attitude to dictionaries in RPython? At one point I thought the attitude was "constant dictionaries, constantly known string-only keys" but in particular pypy.interpreter.argument isn't even vaguely close to this measure. This is what crashed translate_pypy after about 110 minutes of CPU time :) -- there's an assert that the item in aDict[some_item] is constant. I think to cope effectively with argument.py the SomeDict annotation needs quite a lot of work. Cheers, mwh -- i am trying to get Asterisk to work it is stabbing me in the face yes ... i seem to recall that feature in the documentation -- from Twisted.Quotes From lac at strakt.com Tue Nov 30 15:37:31 2004 From: lac at strakt.com (Laura Creighton) Date: Tue, 30 Nov 2004 15:37:31 +0100 Subject: [pypy-dev] Re: [PyCON-Organizers] Jim Hugunin as keynote speaker In-Reply-To: Message from "A.M. Kuchling" of "Tue, 30 Nov 2004 09:12:41 EST." <20041130141241.GD31655@rogue.amk.ca> References: <20041130141241.GD31655@rogue.amk.ca> Message-ID: <200411301437.iAUEbVFV014370@ratthing-b246.strakt.com> In a message of Tue, 30 Nov 2004 09:12:41 EST, "A.M. Kuchling" writes: >Jim Hugunin has agreed to do a keynote speech at PyCon 2005. In >private e-mail, Steve and I agreed that Feb. 1st was a reasonable >deadline for getting a title and abstract from Jim. It'll most likely >be about IronPython, but that's up to Jim. We should update the Wiki >and web pages to mention this. Jim will get free registration. > >Questions: > >Guido traditionally gives a keynote, so we now have two >speakers. Do we need a third keynote for day 3? > >Does this suggest a theme of alternative Python implementations for >PyCon? We could ask the Jython, PyPy, and Python/Parrot people if >there's anything they want to present -- possibly Christian Tismer for >Stackless, too. (I doubt we can help Samuele Pedroni attend the >conference, though, which means it's unlikely Jython can be >represented.) A panel discussion between implementors might be >useful/educational. Teams could also organize sprints of their own. >I doubt there's any cross-implementation code to be written, but >that's a possibility. Suggested things to do: > > * Ping the Jython, PyPy, Parrot, Stackless groups: anything they w >ant to > present? Any sprints they want to run? > * If enough presenters show up, schedule a panel discussion. > (Requires a moderator, and at least a ghost of an agenda.) > >We could set a deadline for some sort of contest, like the pie-thon >competition at OSCON; is that a good idea? (Pro: it's a good way to >encourage progress; it's very concrete; and it gets press. Con: Time >is short for that; not clear what the goal is -- running Python >benchmarks has been done, anything moree may be too difficult. >Ideas?) > >IMHO two keynotes are enough, but the Parrot/Python angle may suggest >a third keynote, or at least an invited talk: Sam Ruby >(http://www.intertwingly.net/blog/) is the current person working on >it, and he's an XML guy who's given keynotes before (e.g. notes on one >keynote are at http://www.jepstone.net/radio/2002/10/10.html#a209.) > >--amk PyPy is funded as of Dec 1. They are signing the papers _now_ I believe. Samuele Pedroni is coming to work for Strakt, to do PyPy. He can come to Pycon. We're _all_ thinking about coming and having a Sprint. I forwarded this to pypy-dev -- I am sure that the rest of the gang will have things to say. Laura -- for PyPy dev.