From Moshe Zadka Tue Feb 1 02:20:41 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 1 Feb 2000 04:20:41 +0200 (IST) Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: Message-ID: On Mon, 31 Jan 2000 gvwilson@nevex.com wrote: > > It (?:) would allow you to incorporate logic into expressions. > > There are contexts where only expressions are allowed, such as: > > - lambdas > > - DTML expr attributes > > in which I'd very much like to incorporate tests. > > Don't know much about DTML, but believe that being able to put > conditionals in lambdas would make the latter much more useful. Giving Guido one less reason to dislike ?: Seriously, I disagree with the basic premise of you and Jim: you can *already* have control structures inside experessions with and/or, so even for Jim's twisted meanings for "add new functionality to the language" it's not true. > Is anything easier with ?: --- yes. For the reader, for the writer or for the cp4e-newbie learning Python? cluttering-another-mailing-list-ly y'rs, Z. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From esr@thyrsus.com Tue Feb 1 02:33:18 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 31 Jan 2000 21:33:18 -0500 Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: ; from Moshe Zadka on Tue, Feb 01, 2000 at 04:20:41AM +0200 References: Message-ID: <20000131213317.A25109@thyrsus.com> Moshe Zadka : > > Don't know much about DTML, but believe that being able to put > > conditionals in lambdas would make the latter much more useful. > > Giving Guido one less reason to dislike ?: You laugh, but this was in fact in my mind. -- Eric S. Raymond Nearly all men can stand adversity, but if you want to test a man's character, give him power. -- Abraham Lincoln From tim_one@email.msn.com Tue Feb 1 03:03:59 2000 From: tim_one@email.msn.com (Tim Peters) Date: Mon, 31 Jan 2000 22:03:59 -0500 Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: <200001311436.JAA15213@eric.cnri.reston.va.us> Message-ID: <000801bf6c60$fc41e3a0$d709143f@tim> [Guido] > ... > The solution can be the same as what Algol used: 'if' outside > parentheses is a statement, and inside parentheses is an expression. > It's a bit of a grammar rearrangement, but totally unambiguous. If I didn't know better , I'd say there's an actual consensus here: it seems we would all agree to "(if cond then true else false)" spelling. Not to be overlooked is Christian's enhancement, allowing "elif" too (beats all heck out of guessing how ?: nests!). Eric, are you also in agreement? > However, the added keyword means it won't be in 1.6. I had already channeled that for the group's benefit . For 1.6, without absurd overloading of some other keyword (like "if cond def true ..."), that seems to leave (if cond: true else false) How much is that hated? I hate it myself, not least because I'd have to change IDLE's parser to guarantee it couldn't get confused by the colon here: big = ( if x >= y: x else y) Given the genesis of these things, Barry would probably have to commit similar hackery on pymode. That shouldn't drive it, but it's worth a thought. The best argument against any enhancement of this nature was Jim Fulton's argument for it: it makes lambda more attractive . BTW, I'm not entirely convinced "then" would have to be a keyword to make "if x then y else z" work: couldn't the grammar use WORD and verify it's specifically "then" later? I'm not impressed by "but that would allow 'then = (if then then else then)'!" arguments (yes, it would; no, nobody would *do* that except in an hysterical c.l.py posting <0.1 wink>). From gvwilson@nevex.com Tue Feb 1 03:13:07 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Mon, 31 Jan 2000 22:13:07 -0500 (EST) Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: Message-ID: > Moshe Zadka wrote: > Seriously, I disagree with the basic premise of you and Jim: you can > *already* have control structures inside experessions with and/or, so > even for Jim's twisted meanings for "add new functionality to the > language" it's not true. Have you ever tried to teach this to relatively experienced programmers (never mind novices) whose background is Fortran, Modula-2, Ada, or anything other than C/C++/Java? It's *really* hard to get the idea across. I tried it in the first Python course I taught at LANL, and wound up spending 15 minutes trying to unconfuse people (and failed). Afterward, one guy said that it made even less sense than using integer offsets to fake linked lists in Fortran-77... Greg From Moshe Zadka Tue Feb 1 03:17:53 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 1 Feb 2000 05:17:53 +0200 (IST) Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: Message-ID: On Mon, 31 Jan 2000 gvwilson@nevex.com wrote: > > Moshe Zadka wrote: > > Seriously, I disagree with the basic premise of you and Jim: you can > > *already* have control structures inside experessions with and/or, so > > even for Jim's twisted meanings for "add new functionality to the > > language" it's not true. > You're most certainly right. This, however, refers to usability, not functionality. It makes existing functionality usable by non-timbots. A worthy goal, but it's not the same as adding new functionality. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From esr@thyrsus.com Tue Feb 1 03:25:01 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 31 Jan 2000 22:25:01 -0500 Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: <000801bf6c60$fc41e3a0$d709143f@tim>; from Tim Peters on Mon, Jan 31, 2000 at 10:03:59PM -0500 References: <200001311436.JAA15213@eric.cnri.reston.va.us> <000801bf6c60$fc41e3a0$d709143f@tim> Message-ID: <20000131222501.B25299@thyrsus.com> Tim Peters : > > ... > > The solution can be the same as what Algol used: 'if' outside > > parentheses is a statement, and inside parentheses is an expression. > > It's a bit of a grammar rearrangement, but totally unambiguous. > > If I didn't know better , I'd say there's an actual consensus here: > it seems we would all agree to "(if cond then true else false)" spelling. > Not to be overlooked is Christian's enhancement, allowing "elif" too (beats > all heck out of guessing how ?: nests!). > > Eric, are you also in agreement? I'm not sure. I'm concerned that this syntax will actually be more viually confusing that ?:. That having been said, I am more interested in having ternary-select be in the language than I am in arguing about its exact syntax. -- Eric S. Raymond Men trained in arms from their infancy, and animated by the love of liberty, will afford neither a cheap or easy conquest. -- From the Declaration of the Continental Congress, July 1775. From ping@lfw.org Tue Feb 1 03:56:28 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Mon, 31 Jan 2000 21:56:28 -0600 (EST) Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: <000801bf6c60$fc41e3a0$d709143f@tim> Message-ID: On Mon, 31 Jan 2000, Tim Peters wrote: > If I didn't know better , I'd say there's an actual consensus here: > it seems we would all agree to "(if cond then true else false)" spelling. Actually, i'm afraid i don't. I initially chose the "then/else" spelling specifically because "if" flags the eye to the beginning of a statement. My line of thinking was, "'then' for expressions, 'if' for statements." > I had already channeled that for the group's benefit . For 1.6, > without absurd overloading of some other keyword (like "if cond def true > ..."), that seems to leave > > (if cond: true else false) Sorry, i like that even less. It seems to abuse the colon, for the colon (to me) signals both (a) This Is A Statement and (b) This Begins A Block, neither of which are true here. > How much is that hated? I hate it myself, not least because I'd have to > change IDLE's parser to guarantee it couldn't get confused by the colon > here: > > big = ( > if x >= y: > x else y) I consider this a symptom of the colon-abuse i just described... > BTW, I'm not entirely convinced "then" would have to be a keyword to make > "if x then y else z" work: couldn't the grammar use WORD and verify it's > specifically "then" later? Quite possibly, yes -- though i figured it wouldn't be *too* large an issue to make "then" a keyword, since i can't imagine anyone naming a symbol "then" except under the most freakish of circumstances. A quick check shows no symbol by that name in any of the Python scripts or modules we have here. -- ?!ng From ping@lfw.org Tue Feb 1 04:05:45 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Mon, 31 Jan 2000 22:05:45 -0600 (EST) Subject: [Python-Dev] timelocal Message-ID: I noticed that there doesn't currently seem to be a function that takes a tuple of date information (like that returned by localtime()) and turns it into a Unix time -- effectively, the reverse operation to localtime() or gmtime(). It's a very useful operation to have when working with dates and times. Perl has it in the timelocal.pl script and i was thinking of stealing the logic in it to add a similar function to Python. So, before i go reinvent the wheel -- have i missed the fact that we already have this function? Would anyone care if it was added? Thanks for your thoughts... -- ?!ng From fdrake@acm.org Tue Feb 1 03:49:47 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 31 Jan 2000 22:49:47 -0500 (EST) Subject: [Python-Dev] timelocal In-Reply-To: References: Message-ID: <14486.22491.791218.69042@weyr.cnri.reston.va.us> Ka-Ping Yee writes: > So, before i go reinvent the wheel -- have i missed the > fact that we already have this function? Would anyone > care if it was added? I'd love to see this added! -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From tim_one@email.msn.com Tue Feb 1 03:49:57 2000 From: tim_one@email.msn.com (Tim Peters) Date: Mon, 31 Jan 2000 22:49:57 -0500 Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: Message-ID: <000a01bf6c67$680a4cc0$d709143f@tim> [Tim] >> If I didn't know better , I'd say there's an actual >> consensus here: it seems we would all agree to "(if cond then >> true else false)" spelling. [Ka-Ping Yee] > Actually, i'm afraid i don't. I initially chose the "then/else" > spelling specifically because "if" flags the eye to the beginning > of a statement. My line of thinking was, "'then' for expressions, > 'if' for statements." OK, I'm baffled. I probably don't recall your suggestion -- the implication is that it didn't use the word "if"? If so, I probably read it and assumed you left out the "if" my mistake . Seriously, "excessively novel" isn't called for here: *tons* of languages have used if/then/else for this purpose without difficulty. ... >> ... couldn't the grammar use WORD and verify it's >> specifically "then" later? > Quite possibly, yes -- though i figured it wouldn't be *too* > large an issue to make "then" a keyword, since i can't imagine > anyone naming a symbol "then" except under the most freakish > of circumstances. A quick check shows no symbol by that name > in any of the Python scripts or modules we have here. No keyword has been added to Python since "lambda", and you can be certain Guido will never add another (at least not to Python1) -- this is an absolute non-starter. Ping, *you* used to know this better than anyone . From gvwilson@nevex.com Tue Feb 1 04:00:56 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Mon, 31 Jan 2000 23:00:56 -0500 (EST) Subject: [Python-Dev] Re: ternary operators (fwd) Message-ID: So I asked Simon Peyton-Jones and Phil Wadler how languages other than C and its derivatives did conditional expressions; one of the replies was: On Mon, 31 Jan 2000, Philip Wadler wrote: > Tony Hoare had a nice ternary `if'. He writes > > if c then d else e > > as > > d e > > This satisfies an associative law: > > d (e f) = d f = (d e) f > > The paper appeared, I think, in CACM circa 1982. Perhaps this would be a good time to ask around and see what else people have in their back pockets? A-week-of-coding-can-sometimes-save-an-hour-in-the-library'ly yours, Greg From ping@lfw.org Tue Feb 1 04:27:02 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Mon, 31 Jan 2000 22:27:02 -0600 (EST) Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: <000a01bf6c67$680a4cc0$d709143f@tim> Message-ID: On Mon, 31 Jan 2000, Tim Peters wrote: > [Tim] > >> If I didn't know better , I'd say there's an actual > >> consensus here: it seems we would all agree to "(if cond then > >> true else false)" spelling. > > [Ka-Ping Yee] > > Actually, i'm afraid i don't. I initially chose the "then/else" > > spelling specifically because "if" flags the eye to the beginning > > of a statement. My line of thinking was, "'then' for expressions, > > 'if' for statements." > > OK, I'm baffled. I probably don't recall your suggestion -- the implication > is that it didn't use the word "if"? If so, I probably read it and assumed > you left out the "if" my mistake . Yeah, my suggestion was, e.g. def abs(x): return x > 0 then x else -x Might as well summarize the other suggestions so far: return x > 0 ? x else -x return x > 0 ? x : -x return if x > 0: x else -x Have i missed any? Oh, yes, and here is the control group. return x > 0 and x or -x return (x > 0 and [x] or [-x])[0] if x > 0: return x else: return -x > Seriously, "excessively novel" isn't called for here: > *tons* of languages have used if/then/else for this > purpose without difficulty. Yes, you're right about that. > No keyword has been added to Python since "lambda", and you can be certain > Guido will never add another (at least not to Python1) -- this is an > absolute non-starter. Ping, *you* used to know this better than anyone > . Okay, okay. You probably have a better memory about this than i do. :) Assuming that "then" will never be made a keyword, i would probably go with "x > 0 ? x else -x". "if" seems to shout "statement" too loudly at me, and colons seem too loaded. Another issue with the last suggestion: how do you explain putting a colon after the condition but not after the "else"? -- ?!ng From ping@lfw.org Tue Feb 1 04:29:40 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Mon, 31 Jan 2000 22:29:40 -0600 (EST) Subject: [Python-Dev] Re: ternary operators (fwd) In-Reply-To: Message-ID: On Mon, 31 Jan 2000 gvwilson@nevex.com wrote: > So I asked Simon Peyton-Jones and Phil Wadler how languages other than C > and its derivatives did conditional expressions; one of the replies was: > > On Mon, 31 Jan 2000, Philip Wadler wrote: > > > Tony Hoare had a nice ternary `if'. He writes > > > > if c then d else e > > > > as > > > > d e Wait -- how the heck is this supposed to parse if 'c' is an arbitrary expression? I think we've run out of bracket-like thingies. (Yes, that would be the technical term). It also looks pretty opaque -- even if you do intuit that it's ternary-select (which i wouldn't if you hadn't told me) it still isn't really obvious which side is which. -- ?!ng From gvwilson@nevex.com Tue Feb 1 04:15:58 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Mon, 31 Jan 2000 23:15:58 -0500 (EST) Subject: [Python-Dev] Re: ternary operators (fwd) In-Reply-To: Message-ID: > > On Mon, 31 Jan 2000, Philip Wadler wrote: > > > Tony Hoare had a nice ternary `if'. He writes > > > if c then d else e > > > as > > > d e > Ka-Ping Yee writes: > Wait -- how the heck is this supposed to parse if 'c' is an > arbitrary expression? Replace '<' and '>' with the symbols of your choice --- embedding the conditional in the middle is still an interesting idea... > I think we've run out of bracket-like thingies. (Yes, that > would be the technical term). > -- ?!ng Well, if people are going to spell there names like *that*, it's no wonder we've run out of bracket-like thingies... :-) save-the-punctionally yours Greg From esr@thyrsus.com Tue Feb 1 04:24:13 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 31 Jan 2000 23:24:13 -0500 Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: ; from Ka-Ping Yee on Mon, Jan 31, 2000 at 10:27:02PM -0600 References: <000a01bf6c67$680a4cc0$d709143f@tim> Message-ID: <20000131232413.A26454@thyrsus.com> Ka-Ping Yee : > Might as well summarize the other suggestions so far: > > return if x > 0: x else -x > return x > 0 ? x else -x Mixing ? or : with a keyword is just *ugly*. Yes, I know I said I wasn't that interested in arguing syntax, but these make my gorge rise. > return x > 0 ? x : -x I think the other suggestions are making this one look better. At this point I have to like either this or the algol68 style Guido mentioned. -- Eric S. Raymond The conclusion is thus inescapable that the history, concept, and wording of the second amendment to the Constitution of the United States, as well as its interpretation by every major commentator and court in the first half-century after its ratification, indicates that what is protected is an individual right of a private citizen to own and carry firearms in a peaceful manner. -- Report of the Subcommittee On The Constitution of the Committee On The Judiciary, United States Senate, 97th Congress, second session (February, 1982), SuDoc# Y4.J 89/2: Ar 5/5 From da@ski.org Tue Feb 1 05:33:41 2000 From: da@ski.org (David Ascher) Date: Mon, 31 Jan 2000 21:33:41 -0800 Subject: [Python-Dev] Re: ternary operators (fwd) References: Message-ID: <003d01bf6c75$e6700e20$0100000a@ski.org> From: > > > > d e > Replace '<' and '>' with the symbols of your choice --- embedding the > conditional in the middle is still an interesting idea... Which brings us back to if and else: a = condition ? truecase : falsecase would be a = truecase if condition else falsecase What's unintuitive at this point is the shortcutting, assuming that it would still apply. 'condition' would get evaluated before truecase, and truecase might not get evaluated at all. Still, it 'reads' good to me. Not sure it's all that Pythonic though. --david From Moshe Zadka Tue Feb 1 05:51:08 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 1 Feb 2000 07:51:08 +0200 (IST) Subject: [Python-Dev] Re: ternary operators (fwd) In-Reply-To: Message-ID: On Mon, 31 Jan 2000 gvwilson@nevex.com wrote: > So I asked Simon Peyton-Jones and Phil Wadler how languages other than C > and its derivatives did conditional expressions; one of the replies was: > Perhaps this would be a good time to ask around and see what else people > have in their back pockets? Adding nothing to the discussion, let me just mention how Scheme (my second favourite language) does it: (if bool true-result else-result) Well, of course this isn't relevant to Python because statements are not expressions, but there might be something in it: if could be also used as a function taking three arguments: print if(long_answer, "Very good!", "correct") I have no idea how this will play with the parser. Anyone? -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From gstein@lyra.org Tue Feb 1 07:15:04 2000 From: gstein@lyra.org (Greg Stein) Date: Mon, 31 Jan 2000 23:15:04 -0800 (PST) Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: Message-ID: On Mon, 31 Jan 2000, Ka-Ping Yee wrote: > On Mon, 31 Jan 2000, Tim Peters wrote: >... > > No keyword has been added to Python since "lambda", and you can be certain > > Guido will never add another (at least not to Python1) -- this is an > > absolute non-starter. Ping, *you* used to know this better than anyone > > . > > Okay, okay. You probably have a better memory about this than i do. :) Actually, Tim's memory is failing. "assert" was added in 1.5. :-) Not that this adds to the debate at all :-), but I figured that I am obligated to knock a chair leg out from under Tim every now and then. Cheers, -g p.s. I'm against a ternary operator. use an if/else statement. use def instead of lambda (lambda is the only rational basis given so far to add the operator, but it is bogus to start with) -- Greg Stein, http://www.lyra.org/ From tim_one@email.msn.com Tue Feb 1 07:45:42 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 1 Feb 2000 02:45:42 -0500 Subject: [Python-Dev] Re: ternary operators (fwd) In-Reply-To: Message-ID: <000301bf6c88$579e2fc0$242d153f@tim> [Moshe Zadka] > ... > Adding nothing to the discussion, let me just mention how Scheme (my > second favourite language) does it: > > (if bool true-result else-result) > > Well, of course this isn't relevant to Python because statements are not > expressions, but there might be something in it: if could be also used > as a function taking three arguments: "if" is a special form in Scheme; all Python functions are strict; short-circuiting is essential here. > print if(long_answer, "Very good!", "correct") > > I have no idea how this will play with the parser. Anyone? if(a, b, c) can't be distiguished from if (a, b, c): before the colon is seen, so it requires more lookahead than Python's parser is comfortable doing. From tim_one@email.msn.com Tue Feb 1 07:50:06 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 1 Feb 2000 02:50:06 -0500 Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: Message-ID: <000401bf6c88$f4ed2ec0$242d153f@tim> [Tim] >> No keyword has been added to Python since "lambda" ... [Greg Stein] > Actually, Tim's memory is failing. "assert" was added in 1.5. :-) > > Not that this adds to the debate at all :-), but I figured that I am > obligated to knock a chair leg out from under Tim every now and then. Actually, in 1.5 Guido *removed* lambda; then he added assert; then he had second thoughts about lambda and added it again. That's why it's the last one added. You're right: the older I get the less effective my bullshit gets . Done well, young one! > p.s. I'm against a ternary operator. use an if/else statement. use > def instead of lambda (lambda is the only rational basis given so far to > add the operator, but it is bogus to start with) Ah, there's nothing like the smell of consensus in the morning! Except maybe napalm. From tim_one@email.msn.com Tue Feb 1 08:15:08 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 1 Feb 2000 03:15:08 -0500 Subject: [Python-Dev] Re: ternary operators (fwd) In-Reply-To: Message-ID: <000601bf6c8c$73eb87a0$242d153f@tim> [Greg Wilson, quoting Philip Wadler] > Tony Hoare had a nice ternary `if'. He writes > > if c then d else e > > as > > d e Noting that he first wrote "if c then d else e" so it would be *clear* what the heck he was talking about. So that's exactly the point at which Python should stop. After that, it's no longer clear, just formally elegant. I love Haskell too (and thank Philip for that), but it ain't Python. > This satisfies an associative law: > > d (e f) = d f = (d e) f Nobody writes nested ?: using the same condition twice; it's more interesting as an endcase absorption law. Does x (y z) = (x y) z ? Nope (e.g., suppose c1 is true and c2 is false). From tim_one@email.msn.com Tue Feb 1 08:28:42 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 1 Feb 2000 03:28:42 -0500 Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: Message-ID: <000701bf6c8e$592adb80$242d153f@tim> [Ping] > Yeah, my suggestion was, e.g. > > def abs(x): > return x > 0 then x else -x Yup, I *did* mentally insert an "if" for you. You're welcome . > Might as well summarize the other suggestions so far: > > return x > 0 ? x else -x Toss it -- *nobody* wants it (if that's one I suggested, I get to retract it). > return x > 0 ? x : -x Has anyone other than Eric come out explicitly in favor of this spelling (I know others have come out in favor of a ternary operator, but don't recall much love for this syntax)? > return if x > 0: x else -x I get to retract that too (heck, I said I hated that one in the same sentence I tossed it out ...). > Have i missed any? Yes, the utterly conventional and perfectly clear one: return (if x > 0 then x else -x) If we don't buy into Guido's keyword argument, it's feasible. There was also some Perlish atrocity (not that I'm judging ...) like return x if x > 0 else -x > Assuming that "then" will never be made a keyword, i would probably > go with "x > 0 ? x else -x". "if" seems to shout "statement" too > loudly at me, and colons seem too loaded. Too late: it was mine & I retracted it . "if" doesn't really *shout* stmt unless it's at the start of a line. Wrap it in parens too and it's barely a whisper. > Another issue with the last suggestion: how do you explain putting a > colon after the condition but not after the "else"? The truth: "Because Guido is afraid of new keywords" . From gvwilson@nevex.com Tue Feb 1 13:07:26 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Tue, 1 Feb 2000 08:07:26 -0500 (EST) Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: <000701bf6c8e$592adb80$242d153f@tim> Message-ID: Maybe this would be a good time to write a dozen or so examples using alternative syntaxes (syntaxii?), and stick 'em in front of a bunch of people who haven't been part of this discussion, and ask 'em "what does this do"? (Footnote: check out David Scanlan's article in the Sept. 1989 issue of "IEEE Software". Turns out that flowcharts are actually more readable than pseudocode --- the research that claimed to show otherwise was biased by (among other things) comparing structured pseudocode with spaghetti flowcharts... One li'l bit of sloppy science, and the world turns its back on something useful...) Ellipsistically yours, Greg From guido@CNRI.Reston.VA.US Tue Feb 1 13:31:59 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Tue, 01 Feb 2000 08:31:59 -0500 Subject: [Python-Dev] timelocal In-Reply-To: Your message of "Mon, 31 Jan 2000 22:05:45 CST." References: Message-ID: <200002011331.IAA18515@eric.cnri.reston.va.us> > I noticed that there doesn't currently seem to be a > function that takes a tuple of date information (like > that returned by localtime()) and turns it into a > Unix time -- effectively, the reverse operation to > localtime() or gmtime(). To go from a local time tuple to Unix time, there's time.mktime(). To go from a gm time tuple to Unix time, there's calendar.timegm(). (The latter only exists in the CVS version.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@CNRI.Reston.VA.US Tue Feb 1 13:37:46 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Tue, 01 Feb 2000 08:37:46 -0500 Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: Your message of "Mon, 31 Jan 2000 22:27:02 CST." References: Message-ID: <200002011337.IAA18548@eric.cnri.reston.va.us> > Yeah, my suggestion was, e.g. > > def abs(x): > return x > 0 then x else -x > > Might as well summarize the other suggestions so far: > > return x > 0 ? x else -x > > return x > 0 ? x : -x > > return if x > 0: x else -x > > Have i missed any? return if x > 0 then x else -x In some contexts, parentheses must be used, e.g. (if c then a else b)[0] = 1 > > No keyword has been added to Python since "lambda", and you can be certain > > Guido will never add another (at least not to Python1) -- this is an > > absolute non-starter. Ping, *you* used to know this better than anyone > > . > > Okay, okay. You probably have a better memory about this than i do. :) Hm, I was just thinking that 'then' wouldn't be the hardest keyword to add... But I should probably stick with Tim's suggestion. > Assuming that "then" will never be made a keyword, i would probably > go with "x > 0 ? x else -x". "if" seems to shout "statement" too > loudly at me, and colons seem too loaded. But "if" is in good company. > Another issue with the last suggestion: how do you explain putting a > colon after the condition but not after the "else"? Whoever proposed that was terribly confused. --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas.heller@ion-tof.com Tue Feb 1 15:35:17 2000 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Tue, 1 Feb 2000 16:35:17 +0100 Subject: [Python-Dev] Proposal: Registry access for Python on Windows Message-ID: <02a501bf6cc9$f1681620$4500a8c0@thomasnotebook> I propose to include functions to access the windows registry into the python 1.6 core. I have thrown together some code which I will post hopefully tomorrow, but I would like to hear some comments before. ---------------------------------------------------------------------- winreg - registry access module Constants: HKEY_CLASSES_ROOT, HKEY_LOCAL_MACHINE, HKEY_CURRENT_USER, HKEY_USERS - bases of registry KEY_READ, KEY_WRITE, KEY_ALL_ACCESS - access rights REG_SZ, REG_DWORD, REG_BINARY - typecodes for values Exception: error - raised when a function fails. Will usually contain a windows error code and a textual description. Functions: OpenKey (base, subkey, [, rights=KEY_READ]) -> integer Opens an existing key with the specified access rights and returns an integer handle. The handle must be closed by a call to CloseKey(). CreateKey (base, subkey [, sam=KEY_ALL_ACCESS]) -> integer Creates a new subkey or opens an exsiting one and returns an integer handle. base must be one of the HKEY_ constants or an integer handle. The handle must be closed by a call to CloseKey(). CloseKey (handle) Closes a key handle. EnumValues (handle, subkey) -> sequence Returns a sequence containing name, value, typecode triples for each existing value. EnumKeys (handle, subkey) -> sequence Returns a sequence containing the names of all subkeys. QueryValue (handle, subkey) -> tuple Returns a tuple containing name, value and typecode. SetValue (handle, name, typecode, value) Sets the value of a name to value. value must be a string for REG_SZ or REG_BINARY, an integer for REG_DWORD. DeleteValue (handle, valuename) Deletes the value. DeleteKey (handle, name [,recursive=0) Deletes the named key if no subkeys exist. If recursive is given as a true value, subkeys are recursively deleted. This is done with Reg* calls, NOT with SHDeleteKey and SHDeleteEmptyKey functions, so should work under any flavor of NT and Win9x. Note: To use the SetValue and Delete* functions, the key must have been opened with KEY_WRITE_ACCESS or KEY_ALL_ACCESS. ---------------------------------------------------------------------- Open Questions: Is the recursive-flag for DeleteKey() too dangerous? Should I switch from an integer handle to a full blown python-object, which would call CloseKey() automatically in the destructor? A higher level interface would be nice, but this should probably implemented on top of this module in python. Comments appreciated! Thomas Heller ION-TOF GmbH From guido@CNRI.Reston.VA.US Tue Feb 1 16:11:03 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Tue, 01 Feb 2000 11:11:03 -0500 Subject: [Python-Dev] Proposal: Registry access for Python on Windows In-Reply-To: Your message of "Tue, 01 Feb 2000 16:35:17 +0100." <02a501bf6cc9$f1681620$4500a8c0@thomasnotebook> References: <02a501bf6cc9$f1681620$4500a8c0@thomasnotebook> Message-ID: <200002011611.LAA18829@eric.cnri.reston.va.us> > I propose to include functions to access the windows registry > into the python 1.6 core. > > I have thrown together some code which I will post hopefully > tomorrow, but I would like to hear some comments before. > > ---------------------------------------------------------------------- > winreg - registry access module > > Constants: > HKEY_CLASSES_ROOT, HKEY_LOCAL_MACHINE, HKEY_CURRENT_USER, > HKEY_USERS - bases of registry > KEY_READ, KEY_WRITE, KEY_ALL_ACCESS - access rights > REG_SZ, REG_DWORD, REG_BINARY - typecodes for values > > Exception: > error - raised when a function fails. Will usually contain > a windows error code and a textual description. > > Functions: > OpenKey (base, subkey, [, rights=KEY_READ]) -> integer > Opens an existing key with the specified access rights > and returns an integer handle. The handle must be closed > by a call to CloseKey(). > > CreateKey (base, subkey [, sam=KEY_ALL_ACCESS]) -> integer > Creates a new subkey or opens an exsiting one > and returns an integer handle. > base must be one of the HKEY_ constants or an integer handle. > The handle must be closed by a call to CloseKey(). > > CloseKey (handle) > Closes a key handle. > > EnumValues (handle, subkey) -> sequence > Returns a sequence containing name, value, typecode triples > for each existing value. > > EnumKeys (handle, subkey) -> sequence > Returns a sequence containing the names of all subkeys. > > QueryValue (handle, subkey) -> tuple > Returns a tuple containing name, value and typecode. > > SetValue (handle, name, typecode, value) > Sets the value of a name to value. > value must be a string for REG_SZ or REG_BINARY, > an integer for REG_DWORD. > > DeleteValue (handle, valuename) > Deletes the value. > > DeleteKey (handle, name [,recursive=0) > Deletes the named key if no subkeys exist. If recursive is > given as a true value, subkeys are recursively deleted. > This is done with Reg* calls, NOT with SHDeleteKey and > SHDeleteEmptyKey functions, so should work under any > flavor of NT and Win9x. > > Note: To use the SetValue and Delete* functions, the key must have > been opened with KEY_WRITE_ACCESS or KEY_ALL_ACCESS. > ---------------------------------------------------------------------- > Open Questions: > Is the recursive-flag for DeleteKey() too dangerous? > Should I switch from an integer handle to a full blown > python-object, which would call CloseKey() automatically in the > destructor? > A higher level interface would be nice, but this should probably > implemented > on top of this module in python. > > Comments appreciated! As long as we're redesigning the API and not just copying the registry access functions fromn win32api, shouldn't this be made a little bit more OO? Things that return a handle should return an object, and things taking a handle should be methods of that object. Also, the structured return values should probably be turned into objects with attributes. It might be acceptable to do this as a Python wrapper; in that case perhaps the C module should be called _winreg. --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Tue Feb 1 16:42:44 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 1 Feb 2000 11:42:44 -0500 (EST) Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: References: Message-ID: <14487.3332.466772.754653@weyr.cnri.reston.va.us> Greg Stein writes: > p.s. I'm against a ternary operator. use an if/else statement. use > def instead of lambda (lambda is the only rational basis given so far to > add the operator, but it is bogus to start with) Actually, the places I'd use it most would be probably be in constructing parameters to string formatting operations. Grepping back in my memory, that's usually where I've wanted it. I very rarely use lambda, and usually change it on the next iteration on those cases when I do. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From gmcm@hypernet.com Tue Feb 1 16:46:36 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Tue, 1 Feb 2000 11:46:36 -0500 Subject: [Python-Dev] Proposal: Registry access for Python on Windows In-Reply-To: <02a501bf6cc9$f1681620$4500a8c0@thomasnotebook> Message-ID: <1262698908-4003053@hypernet.com> Thomas Heller wrote: > I propose to include functions to access the windows registry > into the python 1.6 core. > > I have thrown together some code which I will post hopefully > tomorrow, but I would like to hear some comments before. [snip API - which appears to be a fairly low level wrap] > Open Questions: > Is the recursive-flag for DeleteKey() too dangerous? As long as it defaults to non-recursive, it's fine by me. > Should I switch from an integer handle to a full blown > python-object, which would call CloseKey() automatically in the > destructor? Absodefiposilutely Yes. Not so sure you need typecode. There are 10 types defined, but most tools only do 3 (string, dword and binary) and most usage is of 2 (dword and string - which can hold binary data). So I'd think you could infer pretty safely. Actually, I guess this boils down to intent - "Python as a complete registry tool" would require all the types, but "Python can use the registry" could automatically use dword or string, as appropriate. - Gordon From gmcm@hypernet.com Tue Feb 1 16:53:30 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Tue, 1 Feb 2000 11:53:30 -0500 Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: <14487.3332.466772.754653@weyr.cnri.reston.va.us> References: Message-ID: <1262698493-4028008@hypernet.com> Fred L. Drake, Jr. writes: > Greg Stein writes: > > p.s. I'm against a ternary operator. use an if/else statement. use > > def instead of lambda (lambda is the only rational basis given so far to > > add the operator, but it is bogus to start with) > > Actually, the places I'd use it most would be probably be in > constructing parameters to string formatting operations. Grepping > back in my memory, that's usually where I've wanted it. Boy does that ring a big bell. Was ambivalent, now I'm all for it (either C syntax or "then" syntax, don't care). > I very rarely use lambda, and usually change it on the next > iteration on those cases when I do. Use them mostly in GUIs where no smarts are required. - Gordon From tismer@tismer.com Tue Feb 1 17:22:15 2000 From: tismer@tismer.com (Christian Tismer) Date: Tue, 01 Feb 2000 18:22:15 +0100 Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? References: <1262698493-4028008@hypernet.com> Message-ID: <38971647.D03FC543@tismer.com> Gordon McMillan wrote: > > Fred L. Drake, Jr. writes: > > Greg Stein writes: > > > p.s. I'm against a ternary operator. use an if/else statement. use > > > def instead of lambda (lambda is the only rational basis given so far to > > > add the operator, but it is bogus to start with) > > > > Actually, the places I'd use it most would be probably be in > > constructing parameters to string formatting operations. Grepping > > back in my memory, that's usually where I've wanted it. > > Boy does that ring a big bell. Was ambivalent, now I'm all for it > (either C syntax or "then" syntax, don't care). So am I! Conditional expressions make very much sense for constructing objects on-the-fly. It is quite common to denote tuples, lists and dicts directly in Python code, like so: mapping_table = { 1: "one", 2: "two", } and so on. To parameterise them, we can either use different tables embedded in an if, or use an expression inside the denotation: language = 1 mapping_table = { 1: ("one", "eins")[language], 2: ("two", "zwei")[language], } but with expressions, we get to mapping_table = { 1: language == 0 ? "one" : "eins", 2: language == 0 ? "two" : "zwei", } Well, maybe I had better chosen a different example, since the language example looks better with indexing here. How would we spell an elif? Would we at all? # languages: 0 - English 1 - German 2 - Finnish mapping_table = { 1: language == 0 ? "one" : language == 1 ? "eins", "yksi", 2: language == 0 ? "two" : language == 2 ? "zwei", "kaksi", 3: language == 0 ? "three" : language == 2 ? "drei", "kolme", } (yes the zero slot is missing - forgot what that is in Finnish:) Would conditionals also be valid on the LHS of assignments? target ? a : b = language ? "one" : "eins" grmbl - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Düppelstr. 31 : *Starship* http://starship.python.net 12163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From Moshe Zadka Tue Feb 1 19:59:27 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 1 Feb 2000 21:59:27 +0200 (IST) Subject: [Python-Dev] __contains__ hook Message-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. Send mail to mime@docserver.cac.washington.edu for more info. ---559023410-851401618-949435167=:17780 Content-Type: TEXT/PLAIN; charset=US-ASCII Here's a very preliminary, very hackish version of a hook for the "in" operator. Basically, use like: class spam: def __contains__(self, o): return 1 6 in spam() (answers 1) I must say I was horrified by the current way the operator was handled: very non-OO-ish. I'd much rather there'd be a slot in the sequence interface for this method. This is why there's still no way to use the hook with regular C extension types. Have fun! (BTW: I've tested it only minimally, so it might break your Python. Use with caution). PS. Eric, you can implement sets the *right* way this time. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. ---559023410-851401618-949435167=:17780 Content-Type: TEXT/PLAIN; charset=US-ASCII; name=1 Content-ID: Content-Description: Content-Transfer-Encoding: BASE64 KioqIC4uLy4uLy4uL3B5dGhvbi9kaXN0L3NyYy9PYmplY3RzL2Fic3RyYWN0 LmMJRnJpIE9jdCAxNSAxNDowOTowMiAxOTk5DQotLS0gT2JqZWN0cy9hYnN0 cmFjdC5jCVR1ZSBGZWIgIDEgMTA6MzQ6MzQgMjAwMA0KKioqKioqKioqKioq KioqDQoqKiogMTExMCwxMTE1ICoqKioNCi0tLSAxMTEwLDExNDAgLS0tLQ0K ICAJCX0NCiAgCQlyZXR1cm4gMDsNCiAgCX0NCisgCS8qIHNwZWNpYWwgY2Fz ZSBmb3IgaW5zdGFuY2VzLiBCYXNpY2FsbHkgZW11bGF0aW5nIFB5dGhvbiBj b2RlLA0KKyAJICAgYnV0IG9wdGltaXphdGlvbnMgd2lsbCBjb21lIGxhdGVy ICovDQorIAlpZiAoUHlJbnN0YW5jZV9DaGVjayh3KSkgew0KKyAJCVB5T2Jq ZWN0ICpweV9fY29udGFpbnNfXywgKnB5X3JldCwgKnB5X2FyZ3M7DQorIAkJ aW50IHJldDsNCisgDQorIAkJcHlfX2NvbnRhaW5zX18gPSBQeU9iamVjdF9H ZXRBdHRyU3RyaW5nKHcsICJfX2NvbnRhaW5zX18iKTsNCisgCQlpZihweV9f Y29udGFpbnNfXyA9PSBOVUxMKSANCisgCQkJcmV0dXJuIC0xOw0KKyAJCXB5 X2FyZ3MgPSBQeVR1cGxlX05ldygxKTsNCisgCQlpZihweV9hcmdzID09IE5V TEwpIHsNCisgCQkJUHlfREVDUkVGKHB5X19jb250YWluc19fKTsNCisgCQkJ cmV0dXJuIC0xOw0KKyAJCX0NCisgCQlQeV9JTkNSRUYodik7DQorIAkJUHlU dXBsZV9TRVRfSVRFTShweV9hcmdzLCAwLCB2KTsNCisgCQlweV9yZXQgPSBQ eU9iamVjdF9DYWxsT2JqZWN0KHB5X19jb250YWluc19fLCBweV9hcmdzKTsN CisgCQlQeV9ERUNSRUYocHlfX2NvbnRhaW5zX18pOw0KKyAJCVB5X0RFQ1JF RihweV9hcmdzKTsNCisgCQlpZihweV9yZXQgPT0gTlVMTCkgDQorIAkJCXJl dHVybiAtMTsNCisgCQlyZXQgPSBQeU9iamVjdF9Jc1RydWUocHlfcmV0KTsN CisgCQlQeV9ERUNSRUYocHlfYXJncyk7DQorIAkJcmV0dXJuIHJldDsNCisg CX0NCiAgDQogIAlzcSA9IHctPm9iX3R5cGUtPnRwX2FzX3NlcXVlbmNlOw0K ICAJaWYgKHNxID09IE5VTEwgfHwgc3EtPnNxX2l0ZW0gPT0gTlVMTCkgew0K ---559023410-851401618-949435167=:17780-- From tim_one@email.msn.com Wed Feb 2 07:35:02 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 2 Feb 2000 02:35:02 -0500 Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: Message-ID: <000601bf6d50$0424e000$28a0143f@tim> [Greg Wilson] > (Footnote: check out David Scanlan's article in the Sept. 1989 issue of > "IEEE Software". Turns out that flowcharts are actually more readable > than pseudocode --- the research that claimed to show otherwise was > biased by (among other things) comparing structured pseudocode with > spaghetti flowcharts... One li'l bit of sloppy science, and the world > turns its back on something useful...) Oh, I don't know -- "visual programming" systems keep getting reinvented, so I doubt they'll be lost to us forever: executable flowcharts are at least as sensible as executable pseudocode (which latter Python partly aimed for). I'm old enough that I actually suffered many textbooks that used flowcharts. As I recall, they were absolutely worthless -- and in reviewing a few of them just now, I see that this assessment was far too generous . Lines crossing willy-nilly, dozens of single- and two(!)-letter "off page connectors", ... yikes! If the study used spaghetti flowcharts, I expect they used what was simply common practice at the time. I have seen a few good flowcharts, though, and they were cool. How about a "folding" graphical editor, so we could find & expand the logic levels of particular interest without losing the big picture ... oops-just-realized-this-has-nothing-to-do-with-1.6-ly y'rs - tim From tim_one@email.msn.com Wed Feb 2 07:35:04 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 2 Feb 2000 02:35:04 -0500 Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: <200002011337.IAA18548@eric.cnri.reston.va.us> Message-ID: <000701bf6d50$05624200$28a0143f@tim> [Tim, makes a huge production over Guido's supposed intractability on the issue of new keywords] [Guido] > Hm, I was just thinking that 'then' wouldn't be the hardest keyword to > add... Man, you *are* Unfathomable <0.9 wink>! Even if true, you should never have admitted to it <0.1 wink>. > But I should probably stick with Tim's suggestion. Tim would. [Ping] >> Another issue with the last suggestion: how do you explain putting a >> colon after the condition but not after the "else"? > Whoever proposed that was terribly confused. That was me. It was just brainstorming, so you wouldn't have to . i-disowned-it-in-the-same-paragraph-i-introduced-it-but- all-ideas-take-on-a-hideous-life-of-their-own-ly y'rs - tim From ping@lfw.org Wed Feb 2 11:19:11 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Wed, 2 Feb 2000 06:19:11 -0500 (EST) Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: <200002011337.IAA18548@eric.cnri.reston.va.us> Message-ID: On Tue, 1 Feb 2000, Guido van Rossum wrote: > > Hm, I was just thinking that 'then' wouldn't be the hardest keyword to > add... But I should probably stick with Tim's suggestion. Wow, maybe i shouldn't have given in so fast. Oh, well. :) -- ?!ng From Vladimir.Marangozov@inrialpes.fr Wed Feb 2 11:30:51 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Wed, 2 Feb 2000 12:30:51 +0100 (CET) Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: <200001301332.IAA12252@eric.cnri.reston.va.us> from "Guido van Rossum" at Jan 30, 2000 08:32:18 AM Message-ID: <200002021130.MAA07517@python.inrialpes.fr> Guido van Rossum wrote: > > Dear developers, > > Eric Raymond has sent me the following patch, which adds conditional > expressions to Python. I'd like to hear opinions on whether this is a > good thing to add to Python, and whether this is the right syntax. > Although Eric's patch is cute, I think this language shortcut is unnecessary for Python. I'd support a do/while shortcut though ;-) It's more popular and would fit "naturally" into the language for most programmers. Overall, it seems to me that the next major version of Python needs to be cleaned, instead of being enriched. Both the language and the internals. We have enough experience already for moving in this direction. (I'm not speaking about new packages/libs or additional core facilities, like Unicode, that need to integrated). -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From gvwilson@nevex.com Wed Feb 2 13:26:31 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Wed, 2 Feb 2000 08:26:31 -0500 (EST) Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: <000701bf6d50$05624200$28a0143f@tim> Message-ID: > [Tim] > Man, you *are* Unfathomable <0.9 wink>! Even if true, you should > never have admitted to it <0.1 wink>. I think that's supposed to be , isn't it? Greg From fredrik@pythonware.com Wed Feb 2 13:56:40 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 2 Feb 2000 14:56:40 +0100 Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? References: Message-ID: <00d901bf6d85$5577dd60$f29b12c2@secret.pythonware.com> Greg Wilson wrote: > > [Tim] > > Man, you *are* Unfathomable <0.9 wink>! Even if true, you should > > never have admitted to it <0.1 wink>. >=20 > I think that's supposed to be , isn't it? not in this case. quoting a leading bot implementor: "We did requirements and task analysis, iterative design, and user testing. You'd almost think emails were an interface between people and bots." and I can assure you that proving that human beings don't like weird but formally well designed syntaxes was the easy part of that project... (but don't tell the schemers ;-) "Larry Wall should be shot. Along with Bill Joy and Eric Allman." -- Daniel Finster, on comp.lang.lisp "Why, just because you guys frittered away a 20-year headstart?" -- Larry Wall From Vladimir.Marangozov@inrialpes.fr Wed Feb 2 14:19:43 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Wed, 2 Feb 2000 15:19:43 +0100 (CET) Subject: [Python-Dev] Python 1.6 timing In-Reply-To: <200001210129.UAA28292@eric.cnri.reston.va.us> from "Guido van Rossum" at Jan 20, 2000 08:29:30 PM Message-ID: <200002021419.PAA07712@python.inrialpes.fr> Guido van Rossum wrote: > > As an experiment, I've collected about 40 messages with suggested > patches in them that I found in my inbox; the oldest are nearly two > years old. > > You can access these from this address: > > http://www.python.org/~guido/patch/ > > I would love any help I could get in responding with these, and taking > action in the form of patches. Is this just for the sake of the experiment or this patch selection is the one you really haven't had the time to deal with? I see a couple of patches proposing GC implementations (a very delicate issue) of which only half a single patch would be acceptable for now (the one renaming malloc to PyMem_Malloc). There's one message (No 8) suggesting modifications to README and to the python.org Web page. How can we help with that? I have problems understanding two patches submitted by Gerrit Holl. They mention "Hallo Guido". :-) > I propose that if you decide that a particular patch is worth checking > in, you ask the author for the bugrelease or wetsign disclaimer and let > me know that I can check it in; if changes to the patch are needed, > I propose that you negotiate these with the author first. Understood. Sharing your collection of suggested patches definitely increases your chances to get some help and answer the authors in time! -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From guido@CNRI.Reston.VA.US Wed Feb 2 14:23:03 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 02 Feb 2000 09:23:03 -0500 Subject: [Python-Dev] Python 1.6 timing In-Reply-To: Your message of "Wed, 02 Feb 2000 15:19:43 +0100." <200002021419.PAA07712@python.inrialpes.fr> References: <200002021419.PAA07712@python.inrialpes.fr> Message-ID: <200002021423.JAA21511@eric.cnri.reston.va.us> > > http://www.python.org/~guido/patch/ > > > > I would love any help I could get in responding with these, and taking > > action in the form of patches. > > Is this just for the sake of the experiment or this patch selection > is the one you really haven't had the time to deal with? These are ones that I really haven't had the time to deal with (witness the dates on some). > I see a couple of patches proposing GC implementations (a very delicate > issue) of which only half a single patch would be acceptable for now > (the one renaming malloc to PyMem_Malloc). Maybe you can help me formulate a reply? I guess the patch collection isn't just about patches -- it's about the general level of effort that some patches require. I just don't feel comfortable with saying "no" without a good reason, and "no time" isn't a good enough reason. So I welcome any form of comments on these patch proposals. > There's one message (No 8) suggesting modifications to README and to the > python.org Web page. How can we help with that? The README is checked into the CVS. For the web page I will gladly accept patches to the HTML. > I have problems understanding two patches submitted by Gerrit Holl. > They mention "Hallo Guido". :-) Oops, sorry. The first one is proposing a validate() function for pathnames. In the second one, the patch code speaks for itself -- it makes an exception class conform to the rule that exceptions must inherit from Exception. > > I propose that if you decide that a particular patch is worth checking > > in, you ask the author for the bugrelease or wetsign disclaimer and let > > me know that I can check it in; if changes to the patch are needed, > > I propose that you negotiate these with the author first. > > Understood. > > Sharing your collection of suggested patches definitely increases > your chances to get some help and answer the authors in time! Thanks. Apart from Tim Peters who immediately tackled a typo in the FAQ, no-one else has offered any help with these. I'm glad that you've at least taken the time to look at them. I'm still being inundated with patches... But as long as this experiment hasn't shown more success I will keep them in my inbox. Or is there a better idea? Should I forward all patches I get to python-dev? To a separate list? --Guido van Rossum (home page: http://www.python.org/~guido/) From Vladimir.Marangozov@inrialpes.fr Wed Feb 2 15:54:13 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Wed, 2 Feb 2000 16:54:13 +0100 (CET) Subject: [Python-Dev] Python 1.6 timing In-Reply-To: <200002021423.JAA21511@eric.cnri.reston.va.us> from "Guido van Rossum" at Feb 02, 2000 09:23:03 AM Message-ID: <200002021554.QAA07902@python.inrialpes.fr> Guido van Rossum wrote: > > I guess the patch collection isn't just about patches -- it's about > the general level of effort that some patches require. I just don't > feel comfortable with saying "no" without a good reason, and "no time" > isn't a good enough reason. So I welcome any form of comments on > these patch proposals. Understood. > > Or is there a better idea? Should I forward all patches I get to > python-dev? Certainly not. This isn't the purpose of the list, except maybe if a patch is too fundamental to be just a patch. :-) > To a separate list? This would be fine. A public list discussing all submitted patches would be helpful. There are some patches that require routine testing that few people here would find the time to perform. Others require "critical mass" of opinions to be adopted, rejected or deferred. Maybe we could set the gross operation mode of such a list as follows: You publish systematically all patches (except those that get checked in directly) The discussion focuses patches submitted during the past month. Some of them are accepted, some are rejected, many are "deferred" (for various reasons). Deferred patches are those that have an undecided future and get archived. In this case, a mail notification is sent to the author explaining the situation. If the author thinks the patch is worthy and "decidable", s/he has to resubmit the patch for reviving the discussion on the list and for trying to gain more proponents/favorable opinions. (because the list is discussing 1 month old patches only). If the author pushes real hard, either a decision would be taken, either s/he will end up convinced that the patch is "undecidable" for the time being :-). Besides, all serious patchers would become members of this list, which isn't so bad (given that presently, you're the only contact person for patch related material/submissions and you're mainly discussing a subset of the submissions, one-by-one, with the authors in person). This operation mode would ensure that the "light" patches, mostly the various bug fixes, will end up with a decision. The "hard" ones, those that introduce new functionality or behavior, will be seriously discussed and will eventually end up deferred (and archived) for future consideration. Thus, the archive would constitute yet another "memory" of the development process, accessible to all interested parties. This forum, like python-dev, would have a consultative mission, preserving your decision rights, so it's something you'd probably want to try (provided that you fix the rules of the game at its creation). -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From guido@CNRI.Reston.VA.US Wed Feb 2 16:35:03 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 02 Feb 2000 11:35:03 -0500 Subject: [Python-Dev] __contains__ hook In-Reply-To: Your message of "Tue, 01 Feb 2000 21:59:27 +0200." References: Message-ID: <200002021635.LAA22671@eric.cnri.reston.va.us> Moshe seems eager to get comments on this post :-) > Here's a very preliminary, very hackish version of a hook for the "in" > operator. > > Basically, use like: > > class spam: > def __contains__(self, o): > return 1 > 6 in spam() (answers 1) > > I must say I was horrified by the current way the operator was handled: > very non-OO-ish. I'd much rather there'd be a slot in the sequence > interface for this method. This is why there's still no way to use the > hook with regular C extension types. > > Have fun! > > (BTW: I've tested it only minimally, so it might break your Python. Use > with caution). > > PS. > Eric, you can implement sets the *right* way this time. For those who, like me, are too lazy to unpack attachments, here's the text of Moshe's patch: > *** ../../../python/dist/src/Objects/abstract.c Fri Oct 15 14:09:02 1999 > --- Objects/abstract.c Tue Feb 1 10:34:34 2000 > *************** > *** 1110,1115 **** > --- 1110,1140 ---- > } > return 0; > } > + /* special case for instances. Basically emulating Python code, > + but optimizations will come later */ > + if (PyInstance_Check(w)) { > + PyObject *py__contains__, *py_ret, *py_args; > + int ret; > + > + py__contains__ = PyObject_GetAttrString(w, "__contains__"); > + if(py__contains__ == NULL) > + return -1; > + py_args = PyTuple_New(1); > + if(py_args == NULL) { > + Py_DECREF(py__contains__); > + return -1; > + } > + Py_INCREF(v); > + PyTuple_SET_ITEM(py_args, 0, v); > + py_ret = PyObject_CallObject(py__contains__, py_args); > + Py_DECREF(py__contains__); > + Py_DECREF(py_args); > + if(py_ret == NULL) > + return -1; > + ret = PyObject_IsTrue(py_ret); > + Py_DECREF(py_args); > + return ret; > + } > > sq = w->ob_type->tp_as_sequence; > if (sq == NULL || sq->sq_item == NULL) { I like the idea of overloading 'in' (and 'not in') with __contains__. There are several issues with this patch though (apart from the fact that he left out the disclaimer from http://www.python.org/1.5/bugrelease.html :-). First of all, it actually breaks 'in' for descendants of UserList, and other classes that define __getitem__ but not __contains__. That's easily fixed by clearing the error and jumping forward instead of returning an error when the GetAttrString() call fails. Second, it's customary to define a static object variable initialized to NULL, which is set to the interned string object; this speeds up the lookup a bit using PyObject_GetAttr(). Micro-nit: I want a space between 'if' and '('. It just looks better. But the real issue is what Moshe himself already brings up: contains should have a slot in the type struct, so extension types can also define this. Moshe, do you feel like doing this right? --Guido van Rossum (home page: http://www.python.org/~guido/) From gward@cnri.reston.va.us Wed Feb 2 16:44:48 2000 From: gward@cnri.reston.va.us (Greg Ward) Date: Wed, 2 Feb 2000 11:44:48 -0500 Subject: [Python-Dev] __contains__ hook In-Reply-To: <200002021635.LAA22671@eric.cnri.reston.va.us>; from guido@cnri.reston.va.us on Wed, Feb 02, 2000 at 11:35:03AM -0500 References: <200002021635.LAA22671@eric.cnri.reston.va.us> Message-ID: <20000202114447.A18515@cnri.reston.va.us> On 02 February 2000, Guido van Rossum said: > I like the idea of overloading 'in' (and 'not in') with __contains__. > There are several issues with this patch though (apart from the fact > that he left out the disclaimer from > http://www.python.org/1.5/bugrelease.html :-). I agree at a language level; the current way to "overload" 'in' is... ummm... weird. It seems like there's a a simple and natural "magic method" corresponding to almost every operator, so any operators that *don't* get that treatment are second-class citizens. As for the implementation of __contains__, I'm just not familiar enough with Python internals to comment. I'll let the rest of you argue over that. Greg From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Wed Feb 2 16:56:43 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Wed, 2 Feb 2000 11:56:43 -0500 (EST) Subject: [Python-Dev] Python 1.6 timing References: <200002021419.PAA07712@python.inrialpes.fr> <200002021423.JAA21511@eric.cnri.reston.va.us> Message-ID: <14488.25035.985792.589165@anthem.cnri.reston.va.us> >>>>> "Guido" == Guido van Rossum writes: Guido> The README is checked into the CVS. For the web page I Guido> will gladly accept patches to the HTML. I'm willing to put the entire python.org web site under CVS. This would at least make it easier for others to send us patches to the .ht files against the latest revisions. Is there any interest in this? It wouldn't take me long. Guido> Or is there a better idea? Should I forward all patches I Guido> get to python-dev? To a separate list? Probably a separate list. xemacs.org runs a xemacs-patches mailing list with a replybot on it that scans for patches. It sends back a different response based on whether it finds a patch or not. Then there's a group of lieutenants that keep an eye on the patches and work out their applicability. We could set something like that up fairly easily. -Barry From guido@CNRI.Reston.VA.US Wed Feb 2 17:00:47 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 02 Feb 2000 12:00:47 -0500 Subject: [Python-Dev] Python 1.6 timing In-Reply-To: Your message of "Wed, 02 Feb 2000 11:56:43 EST." <14488.25035.985792.589165@anthem.cnri.reston.va.us> References: <200002021419.PAA07712@python.inrialpes.fr> <200002021423.JAA21511@eric.cnri.reston.va.us> <14488.25035.985792.589165@anthem.cnri.reston.va.us> Message-ID: <200002021700.MAA22997@eric.cnri.reston.va.us> > Guido> Or is there a better idea? Should I forward all patches I > Guido> get to python-dev? To a separate list? > > Probably a separate list. xemacs.org runs a xemacs-patches mailing > list with a replybot on it that scans for patches. It sends back a > different response based on whether it finds a patch or not. Then > there's a group of lieutenants that keep an eye on the patches and > work out their applicability. We could set something like that up > fairly easily. This sounds like a useful idea. It should also check for the disclaimer text and send an appropriate apply if a patch is found without a disclaimer. Of course, the replybot would need to be smart enough to find things like patches hiding in BASE64-encoded attachments... On the other hand, perhaps it would be better to deal with patches the same way as with bug reports -- the Jitterbug database isn't perfect, but it makes it possible to check regularly whether something has been dealt with or not, much better than a simple mailing list. (There are already lieutenants scanning the bugs traffic, so that part doesn't change.) --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Wed Feb 2 17:27:42 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 2 Feb 2000 12:27:42 -0500 (EST) Subject: [Python-Dev] Python 1.6 timing In-Reply-To: <200002021423.JAA21511@eric.cnri.reston.va.us> References: <200002021419.PAA07712@python.inrialpes.fr> <200002021423.JAA21511@eric.cnri.reston.va.us> Message-ID: <14488.26894.735522.942609@weyr.cnri.reston.va.us> Guido van Rossum writes: > Oops, sorry. The first one is proposing a validate() function for > pathnames. In the second one, the patch code speaks for itself -- it I did have a thought on this, but hadn't gotten back to it. Essentially, I'm not sure how to implement this correctly; things like MAXPATHLEN aren't always easy to root out the "right" way to get the *real* definition from C. pathconf(_PC_PATH_MAX) could be used to get that, and pathconf(_PC_NAME_MAX) can get the maximum length of an individual name within the path, but I don't know if the concepts are even meaningful on all systems. I wouldn't be surprised if validity on some systems is highly specific to the actual filesystem that's being referred to, and that requires the name be valid on the local system. I've noticed that the functions in the os.path implementations fall into two categories: "abstract" functions like join(), split(), and the like, which are bound to the "path algebra" syntax rules, and the "local-access" functions like isfile() and abspath(), which require the paths relate to the local system. This isn't a problem, but something we should probably keep in mind. > makes an exception class conform to the rule that exceptions must > inherit from Exception. I don't think this is valuable for 1.6, but might be interesting for 1.7; the documentation can include a notice that this relationship will be required in the future. That would allow people to define exceptions with the required inheritance before the last minute. On the other hand, I'm not sure it really matters that exceptions inherit from a specific base class. *That* seems unnecessary. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From Moshe Zadka Wed Feb 2 18:37:15 2000 From: Moshe Zadka (Moshe Zadka) Date: Wed, 2 Feb 2000 20:37:15 +0200 (IST) Subject: [Python-Dev] __contains__ hook In-Reply-To: <200002021635.LAA22671@eric.cnri.reston.va.us> Message-ID: On Wed, 2 Feb 2000, Guido van Rossum wrote: > I like the idea of overloading 'in' (and 'not in') with __contains__. > There are several issues with this patch though (apart from the fact > that he left out the disclaimer from > http://www.python.org/1.5/bugrelease.html :-). Sorry: I'd d/l it and mail it later... > Micro-nit: I want a space between 'if' and '('. It just looks better. Sorry: old habits die hard. Change as you will. > But the real issue is what Moshe himself already brings up: contains > should have a slot in the type struct, so extension types can also > define this. > > Moshe, do you feel like doing this right? Yes, but not in the near future. Wouldn't adding a new slot break old extension types? I'm a bit ignorant on the subject -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From guido@CNRI.Reston.VA.US Wed Feb 2 18:49:54 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 02 Feb 2000 13:49:54 -0500 Subject: [Python-Dev] __contains__ hook In-Reply-To: Your message of "Wed, 02 Feb 2000 20:37:15 +0200." References: Message-ID: <200002021849.NAA23092@eric.cnri.reston.va.us> > > But the real issue is what Moshe himself already brings up: contains > > should have a slot in the type struct, so extension types can also > > define this. > > > > Moshe, do you feel like doing this right? > > Yes, but not in the near future. Wouldn't adding a new slot break old > extension types? I'm a bit ignorant on the subject There are some spare slots in PyTypeObject: /* More spares */ long tp_xxx5; long tp_xxx6; long tp_xxx7; long tp_xxx8; These can be used for binary compatibility; old extensions will simply not have the new feature. There's also a more sophisticated feature, implemented through tp_flags, which can indicate that an extension is aware of a particular feature. These comments in object.h may explain this: /* Type flags (tp_flags) These flags are used to extend the type structure in a backwards-compatible fashion. Extensions can use the flags to indicate (and test) when a given type structure contains a new feature. The Python core will use these when introducing new functionality between major revisions (to avoid mid-version changes in the PYTHON_API_VERSION). Arbitration of the flag bit positions will need to be coordinated among all extension writers who publically release their extensions (this will be fewer than you might expect!).. Python 1.5.2 introduced the bf_getcharbuffer slot into PyBufferProcs. Type definitions should use Py_TPFLAGS_DEFAULT for their tp_flags value. Code can use PyType_HasFeature(type_ob, flag_value) to test whether the given type object has a specified feature. */ --Guido van Rossum (home page: http://www.python.org/~guido/) From Moshe Zadka Wed Feb 2 19:03:45 2000 From: Moshe Zadka (Moshe Zadka) Date: Wed, 2 Feb 2000 21:03:45 +0200 (IST) Subject: [Python-Dev] Python 1.6 timing In-Reply-To: <14488.25035.985792.589165@anthem.cnri.reston.va.us> Message-ID: On Wed, 2 Feb 2000, Barry A. Warsaw wrote: > I'm willing to put the entire python.org web site under CVS. This > would at least make it easier for others to send us patches to the .ht > files against the latest revisions. Is there any interest in this? > It wouldn't take me long. Ummmm...would this be the wrong time to ask how the redesign contest is going on? > Probably a separate list. xemacs.org runs a xemacs-patches mailing > list with a replybot on it that scans for patches. It sends back a > different response based on whether it finds a patch or not. Then > there's a group of lieutenants that keep an eye on the patches and > work out their applicability. We could set something like that up > fairly easily. A definite aye vote, though perhaps that's an overkill. As long as we're comparing other Free Software projects, let me just note that on linux-kernel, patches are part of the regular discussion. Whoever feels like it, runs a modified kernel, and reports the result. Patches are then chosen (in part) by the responses of people who have tried them out -- a very good QA mechanism. Just to brainstorm about the process. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From guido@CNRI.Reston.VA.US Wed Feb 2 19:07:07 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 02 Feb 2000 14:07:07 -0500 Subject: [Python-Dev] Python 1.6 timing In-Reply-To: Your message of "Wed, 02 Feb 2000 12:27:42 EST." <14488.26894.735522.942609@weyr.cnri.reston.va.us> References: <200002021419.PAA07712@python.inrialpes.fr> <200002021423.JAA21511@eric.cnri.reston.va.us> <14488.26894.735522.942609@weyr.cnri.reston.va.us> Message-ID: <200002021907.OAA23326@eric.cnri.reston.va.us> > Guido van Rossum writes: > > Oops, sorry. The first one is proposing a validate() function for > > pathnames. In the second one, the patch code speaks for itself -- it > > I did have a thought on this, but hadn't gotten back to it. > Essentially, I'm not sure how to implement this correctly; things like > MAXPATHLEN aren't always easy to root out the "right" way to get the > *real* definition from C. pathconf(_PC_PATH_MAX) could be used to get > that, and pathconf(_PC_NAME_MAX) can get the maximum length of an > individual name within the path, but I don't know if the concepts are > even meaningful on all systems. I wouldn't be surprised if validity > on some systems is highly specific to the actual filesystem that's > being referred to, and that requires the name be valid on the local > system. I personally think there is very little merit in a validate() function -- it's not going to be a very useful predictor of whether an open may succeed or not. > I've noticed that the functions in the os.path implementations fall > into two categories: "abstract" functions like join(), split(), and > the like, which are bound to the "path algebra" syntax rules, and the > "local-access" functions like isfile() and abspath(), which require > the paths relate to the local system. This isn't a problem, but > something we should probably keep in mind. > > > makes an exception class conform to the rule that exceptions must > > inherit from Exception. > > I don't think this is valuable for 1.6, but might be interesting for > 1.7; the documentation can include a notice that this relationship > will be required in the future. That would allow people to define > exceptions with the required inheritance before the last minute. > On the other hand, I'm not sure it really matters that exceptions > inherit from a specific base class. *That* seems unnecessary. This is valuable mostly as an example; plus in that specific case I think he noticed that the module defines an exception class from scratch with functionality that is already present in the standard Exception class. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@CNRI.Reston.VA.US Wed Feb 2 19:12:57 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 02 Feb 2000 14:12:57 -0500 Subject: [Python-Dev] Python 1.6 timing In-Reply-To: Your message of "Wed, 02 Feb 2000 21:03:45 +0200." References: Message-ID: <200002021912.OAA23362@eric.cnri.reston.va.us> > Ummmm...would this be the wrong time to ask how the redesign contest is > going on? I haven't seen any submissions. Maybe Barry has? > > Probably a separate list. xemacs.org runs a xemacs-patches mailing > > list with a replybot on it that scans for patches. It sends back a > > different response based on whether it finds a patch or not. Then > > there's a group of lieutenants that keep an eye on the patches and > > work out their applicability. We could set something like that up > > fairly easily. > > A definite aye vote, though perhaps that's an overkill. As long as we're > comparing other Free Software projects, let me just note that on > linux-kernel, patches are part of the regular discussion. Whoever feels > like it, runs a modified kernel, and reports the result. Patches are then > chosen (in part) by the responses of people who have tried them out -- a > very good QA mechanism. Just to brainstorm about the process. Good point. At the moment I am the sole arbiter for patches, and I'm tired of it. --Guido van Rossum (home page: http://www.python.org/~guido/) From bwarsaw@cnri.reston.va.us Wed Feb 2 19:14:52 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Wed, 2 Feb 2000 14:14:52 -0500 (EST) Subject: [Python-Dev] Python 1.6 timing References: <14488.25035.985792.589165@anthem.cnri.reston.va.us> Message-ID: <14488.33324.334870.763850@anthem.cnri.reston.va.us> >>>>> "MZ" == Moshe Zadka writes: MZ> Ummmm...would this be the wrong time to ask how the redesign MZ> contest is going on? It isn't, AFAICT. I had two people come up to me at the conference and ask a few questions, with a promise to have something "soon". I had one other person email me a few weeks before the conference. But we've seen nothing, so I'm not very hopeful. MZ> A definite aye vote, though perhaps that's an overkill. As MZ> long as we're comparing other Free Software projects, let me MZ> just note that on linux-kernel, patches are part of the MZ> regular discussion. Whoever feels like it, runs a modified MZ> kernel, and reports the result. Patches are then chosen (in MZ> part) by the responses of people who have tried them out -- a MZ> very good QA mechanism. Just to brainstorm about the process. Mine's just one vote, but I really do not want to see patches floated on python-dev. -Barry From bwarsaw@CNRI.Reston.VA.US Wed Feb 2 19:17:30 2000 From: bwarsaw@CNRI.Reston.VA.US (bwarsaw@CNRI.Reston.VA.US) Date: Wed, 2 Feb 2000 14:17:30 -0500 (EST) Subject: [Python-Dev] Python 1.6 timing References: <200002021419.PAA07712@python.inrialpes.fr> <200002021423.JAA21511@eric.cnri.reston.va.us> <14488.25035.985792.589165@anthem.cnri.reston.va.us> <200002021700.MAA22997@eric.cnri.reston.va.us> Message-ID: <14488.33482.899746.619127@anthem.cnri.reston.va.us> >>>>> "Guido" == Guido van Rossum writes: Guido> This sounds like a useful idea. It should also check for Guido> the disclaimer text and send an appropriate apply if a Guido> patch is found without a disclaimer. Guido> Of course, the replybot would need to be smart enough to Guido> find things like patches hiding in BASE64-encoded Guido> attachments... Guido> On the other hand, perhaps it would be better to deal with Guido> patches the same way as with bug reports -- the Jitterbug Guido> database isn't perfect, but it makes it possible to check Guido> regularly whether something has been dealt with or not, Guido> much better than a simple mailing list. (There are already Guido> lieutenants scanning the bugs traffic, so that part doesn't Guido> change.) Maybe then just use Jitterbug to collate patches. That's what a lot of my JPython users do. -Barry From Moshe Zadka Wed Feb 2 19:28:33 2000 From: Moshe Zadka (Moshe Zadka) Date: Wed, 2 Feb 2000 21:28:33 +0200 (IST) Subject: [Python-Dev] Python 1.6 timing In-Reply-To: <14488.33324.334870.763850@anthem.cnri.reston.va.us> Message-ID: On Wed, 2 Feb 2000 bwarsaw@cnri.reston.va.us wrote: > Mine's just one vote, but I really do not want to see patches floated > on python-dev. How 'bout a seperate list with a Reply-To: python-dev? -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From bwarsaw@cnri.reston.va.us Wed Feb 2 21:56:41 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Wed, 2 Feb 2000 16:56:41 -0500 (EST) Subject: [Python-Dev] Python 1.6 timing References: <14488.33324.334870.763850@anthem.cnri.reston.va.us> Message-ID: <14488.43033.642829.483942@anthem.cnri.reston.va.us> >>>>> "MZ" == Moshe Zadka writes: MZ> On Wed, 2 Feb 2000 bwarsaw@cnri.reston.va.us wrote: >> Mine's just one vote, but I really do not want to see patches >> floated on python-dev. MZ> How 'bout a seperate list with a Reply-To: python-dev? That would work for me. I need to hack Mailman a little to add this feature, but it could be done. -Barry From fdrake@acm.org Wed Feb 2 22:14:58 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 2 Feb 2000 17:14:58 -0500 (EST) Subject: [Python-Dev] Python 1.6 timing In-Reply-To: <200002021907.OAA23326@eric.cnri.reston.va.us> References: <200002021419.PAA07712@python.inrialpes.fr> <200002021423.JAA21511@eric.cnri.reston.va.us> <14488.26894.735522.942609@weyr.cnri.reston.va.us> <200002021907.OAA23326@eric.cnri.reston.va.us> Message-ID: <14488.44130.277594.651205@weyr.cnri.reston.va.us> Guido van Rossum writes: > I personally think there is very little merit in a validate() function > -- it's not going to be a very useful predictor of whether an open may > succeed or not. Agreed; I was not trying to push for the addition of the function but to point out that it is unlikely to be correct or meaningful. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From mhammond@skippinet.com.au Wed Feb 2 23:00:41 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Thu, 3 Feb 2000 10:00:41 +1100 Subject: [Python-Dev] Proposal: Registry access for Python on Windows In-Reply-To: <02a501bf6cc9$f1681620$4500a8c0@thomasnotebook> Message-ID: > I propose to include functions to access the windows registry > into the python 1.6 core. Agreed! > I have thrown together some code which I will post hopefully > tomorrow, but I would like to hear some comments before. > > ---------------------------------------------------------------------- > winreg - registry access module Im not convinced that we need either a) a new implementation, or b) a new API. I would propose that we use the existing registry access functions from win32api - as I didnt code them, Im not suggesting this due to my own ego. The reasons for my beliefs are: * The new API does not add any new value. The only changes I can see are that OpenKey() has had a reserved value dropped and QueryValue() has an extra redundant result. Indeed the new API does not appear a new API at all, so should be clearly stated if indeed it is (and in which case, exactly what the changes are) or be documented as a pythonic wrapping of the existing win32 registry API. Further, there is some useful functionality dropped. * Well tested. There are some obscure rules and code that experience has shown we need. It would be a shame to reinvent those wheels. It has taken a few years to get it "just right", and this was an implementation by people who know their stuff! Why not take the existing code as it stands (which _does_ include an auto-closing key) and add a new .py wrapper? At least Python will have a "complete, but low-level" registry API, and a higher level "easier to use, but maybe not complete" layer implemented in Python. Indeed, Guido has stated that the existing win32api functions would be suitable for the core. However, the issue is the "coding style" - and I admit that if I knew exectly what that meant I would make the change - eg, if it only means renaming the identifiers it would be trivial - however, it would also appear to be excessively anal, so Im sure there is more to it than that :-) And-I-think-I-will-stay-out-of-the-ternary-debate-ly, Mark. From guido@CNRI.Reston.VA.US Wed Feb 2 23:14:50 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 02 Feb 2000 18:14:50 -0500 Subject: [Python-Dev] Proposal: Registry access for Python on Windows In-Reply-To: Your message of "Thu, 03 Feb 2000 10:00:41 +1100." References: Message-ID: <200002022314.SAA24664@eric.cnri.reston.va.us> > Im not convinced that we need either a) a new implementation, or b) a new > API. > > I would propose that we use the existing registry access functions from > win32api - as I didnt code them, Im not suggesting this due to my own ego. > The reasons for my beliefs are: > > * The new API does not add any new value. The only changes I can see are > that OpenKey() has had a reserved value dropped and QueryValue() has an > extra redundant result. Indeed the new API does not appear a new API at > all, so should be clearly stated if indeed it is (and in which case, exactly > what the changes are) or be documented as a pythonic wrapping of the > existing win32 registry API. Further, there is some useful functionality > dropped. > > * Well tested. There are some obscure rules and code that experience has > shown we need. It would be a shame to reinvent those wheels. It has taken > a few years to get it "just right", and this was an implementation by people > who know their stuff! > > Why not take the existing code as it stands (which _does_ include an > auto-closing key) and add a new .py wrapper? At least Python will have a > "complete, but low-level" registry API, and a higher level "easier to use, > but maybe not complete" layer implemented in Python. Sounds good to me. > Indeed, Guido has stated that the existing win32api functions would be > suitable for the core. However, the issue is the "coding style" - and I > admit that if I knew exectly what that meant I would make the change - eg, > if it only means renaming the identifiers it would be trivial - however, it > would also appear to be excessively anal, so Im sure there is more to it > than that :-) I only vaguely recollect what caused me to object against the coding style, but I think it was a lot of little things: // comments, lines longer than 78 chars, unusual indentation style. It was also somehow requiring C++ (everything in the core is plain C). I tried to do a style conversion myself, but found it very painful -- plus there were some dependencies on other files or modules that seemed to require me to pull in a lot of other things besides win32api.c. However, the registry functions are only a tiny piece of win32api -- they could probably be cut out of win32api and moved into something I would tentatively call win32reg (unless that's already an existing name), which could then be cleaned up much easier than all of win32api. I still think that it can be done, but evidently it takes someone besides me and Mark to do it. --Guido van Rossum (home page: http://www.python.org/~guido/) From mhammond@skippinet.com.au Wed Feb 2 23:32:09 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Thu, 3 Feb 2000 10:32:09 +1100 Subject: [Python-Dev] Proposal: Registry access for Python on Windows In-Reply-To: <200002022314.SAA24664@eric.cnri.reston.va.us> Message-ID: > I only vaguely recollect what caused me to object against the coding > style, but I think it was a lot of little things: // comments, lines > longer than 78 chars, unusual indentation style. It was also somehow > requiring C++ (everything in the core is plain C). I tried to do a > style conversion myself, but found it very painful -- plus there were > some dependencies on other files or modules that seemed to require me > to pull in a lot of other things besides win32api.c. OK - that is a good enought list for me to get started. If no one else steps up within a week I will make an attempt at a module that meets these requirements.... Mark. From mal@lemburg.com Wed Feb 2 23:33:17 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 03 Feb 2000 00:33:17 +0100 Subject: [Python-Dev] __contains__ hook References: <200002021849.NAA23092@eric.cnri.reston.va.us> Message-ID: <3898BEBD.93DFBCA7@lemburg.com> Guido van Rossum wrote: > > > > But the real issue is what Moshe himself already brings up: contains > > > should have a slot in the type struct, so extension types can also > > > define this. > > > > > > Moshe, do you feel like doing this right? > > > > Yes, but not in the near future. Wouldn't adding a new slot break old > > extension types? I'm a bit ignorant on the subject > > There are some spare slots in PyTypeObject: > > /* More spares */ > long tp_xxx5; > long tp_xxx6; > long tp_xxx7; > long tp_xxx8; > > These can be used for binary compatibility; old extensions will simply > not have the new feature. > > There's also a more sophisticated feature, implemented through > tp_flags, which can indicate that an extension is aware of a > particular feature. These comments in object.h may explain this: > > /* > > Type flags (tp_flags) Shouldn't 'in' be a slot of the sequence methods ? I'd suggest creating a new tp_flag bit and then extending tp_as_sequence with: binaryfunc sq_contains; plus of course add an abstract function to abstract.c: PySequence_Contain(PyObject *container, PyObject *element) which uses the above slot after testing the tp_flag setting. Python instances, lists, tuples should then support this new slot. We could even sneak in support for dictionaries once we decide whether semantics whould be 1. key in dict 2. value in dict or 3. (key,value) in dict :-) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From guido@CNRI.Reston.VA.US Wed Feb 2 23:49:34 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 02 Feb 2000 18:49:34 -0500 Subject: [Python-Dev] __contains__ hook In-Reply-To: Your message of "Thu, 03 Feb 2000 00:33:17 +0100." <3898BEBD.93DFBCA7@lemburg.com> References: <200002021849.NAA23092@eric.cnri.reston.va.us> <3898BEBD.93DFBCA7@lemburg.com> Message-ID: <200002022349.SAA25030@eric.cnri.reston.va.us> > Shouldn't 'in' be a slot of the sequence methods ? I'd suggest > creating a new tp_flag bit and then extending tp_as_sequence > with: > > binaryfunc sq_contains; > > plus of course add an abstract function to abstract.c: > > PySequence_Contain(PyObject *container, PyObject *element) That function already exists, spelled "PySequence_Contains" (currently it does the C equivalent of for i in container: if element == i: return 1 return 0 I'm not entirely sure whether the 'contains' slot should be part of the as_sequence struct, but I suppose it makes sense historically. (The as_number, as_sequece, as_mapping structs don't make sense at all in the grand scheme of things, but we're stuck with them for the time being.) --Guido van Rossum (home page: http://www.python.org/~guido/) From gstein@lyra.org Thu Feb 3 00:21:19 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 2 Feb 2000 16:21:19 -0800 (PST) Subject: [Python-Dev] Python 1.6 timing In-Reply-To: <200002021423.JAA21511@eric.cnri.reston.va.us> Message-ID: On Wed, 2 Feb 2000, Guido van Rossum wrote: >... > > Sharing your collection of suggested patches definitely increases > > your chances to get some help and answer the authors in time! > > Thanks. Apart from Tim Peters who immediately tackled a typo in the > FAQ, no-one else has offered any help with these. I'm glad that > you've at least taken the time to look at them. I'm still being > inundated with patches... But as long as this experiment hasn't shown > more success I will keep them in my inbox. I looked through some of them, but (as I mentioned) would defer any real action until the April timeframe. At that point, I'd be more than happy to dig in and start handling them. Properly responding to them (in a group context) would probably need a bit of coordination infrustructure... > Or is there a better idea? Should I forward all patches I get to > python-dev? To a separate list? A new list. python-dev is very constructive in that it typically has one, maybe two or three, threads going at once. Throwing patches into the mix, and possibly unreasonable/unqualified patches, would seem to be quite a disruptive influence. The (smaller) list could be much more focused, and could hold just the truly active reviewers. People who are "just interested" could wait for the CVS checkin notices or become active :-) Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Thu Feb 3 00:31:41 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 2 Feb 2000 16:31:41 -0800 (PST) Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: <14488.33482.899746.619127@anthem.cnri.reston.va.us> Message-ID: On Wed, 2 Feb 2000 bwarsaw@CNRI.Reston.VA.US wrote: > >>>>> "Guido" == Guido van Rossum writes: >... > Guido> On the other hand, perhaps it would be better to deal with > Guido> patches the same way as with bug reports -- the Jitterbug > Guido> database isn't perfect, but it makes it possible to check > Guido> regularly whether something has been dealt with or not, > Guido> much better than a simple mailing list. (There are already > Guido> lieutenants scanning the bugs traffic, so that part doesn't > Guido> change.) > > Maybe then just use Jitterbug to collate patches. That's what a lot > of my JPython users do. Personally, I'm more comfortable knowing that I can email a patch (rather than dropping it into a bug db), and that will get handled by a human. The patch handlers could certainly use Jitterbug as arbitration, but I would think the list itself would make that reasonably clear. Note that "patches@python.org" could become THE place to submit patches. Sure, Guido would get some patches still, but some of the load will drop from his direct Inbox (yet he'd still get the patch as a subscriber). When the patch handlers had a "final" patch, it would get sent straight to Guido (with some "final" marker on it). etc etc. I'm sure there is a lot discussion that can take place on the exact mechanics. Until people will *volunteer*, I think the discussion of mechanics can be postponed. No need for a peanut gallery here :-) Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Thu Feb 3 00:35:20 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 2 Feb 2000 16:35:20 -0800 (PST) Subject: [Python-Dev] patches reply-to (was: Python 1.6 timing) In-Reply-To: <14488.43033.642829.483942@anthem.cnri.reston.va.us> Message-ID: On Wed, 2 Feb 2000 bwarsaw@cnri.reston.va.us wrote: > >>>>> "MZ" == Moshe Zadka writes: > MZ> On Wed, 2 Feb 2000 bwarsaw@cnri.reston.va.us wrote: > > >> Mine's just one vote, but I really do not want to see patches > >> floated on python-dev. > > MZ> How 'bout a seperate list with a Reply-To: python-dev? > > That would work for me. I need to hack Mailman a little to add this > feature, but it could be done. Note that I requested this feature for Mailman a while back. I'd like to use it for the "checkin" mailing lists that I run. Send to checkins, respond to the discussion list. Currently, my CVS automailer just inserts a Reply-To:, but it would be nice to have it directly on the mailing list itself. (view it more as a Followup-To: for mailers, rather than Reply-To munging) In this particular case, I think the "patches mailing list" would be a self-contained list discussing a *patch*. Sure, it could certainly migrate to python-dev when appropriate, but I think the majority of the discussion should stay on the patches list. Otherwise, we'd just be spamming the -dev list as if the patches list didn't exist. Cheers, -g -- Greg Stein, http://www.lyra.org/ From Moshe Zadka Thu Feb 3 00:32:41 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 3 Feb 2000 02:32:41 +0200 (IST) Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: Message-ID: On Wed, 2 Feb 2000, Greg Stein wrote: > > Maybe then just use Jitterbug to collate patches. That's what a lot > > of my JPython users do. > Note that "patches@python.org" could become THE place to submit patches. OK, I want to know one thing: is everyone comfortable with my suggestion of a seperate list, with a reply-to: python-dev? Barry said he'd be willing to hack it in, and everyone who talked seemed like this idea is similar to what they have in mind. The sooner a consensus is reached, the less patches Guido will have to deal with. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From gstein@lyra.org Thu Feb 3 00:42:57 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 2 Feb 2000 16:42:57 -0800 (PST) Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: Message-ID: On Thu, 3 Feb 2000, Moshe Zadka wrote: >... > OK, I want to know one thing: is everyone comfortable with my suggestion > of a seperate list, with a reply-to: python-dev? Barry said he'd be I don't advocate the reply-to (see my other email). > willing to hack it in, and everyone who talked seemed like this idea is > similar to what they have in mind. The sooner a consensus is reached, the > less patches Guido will have to deal with. It won't reduce his load unless we have actual volunteers that will truly do some handling of the patches. I'm assuming the current volunteer list is: - Guido (he volunteers whether he wants to or not :-) - Moshe (??) - Greg in April Others? Cheers, -g -- Greg Stein, http://www.lyra.org/ From Moshe Zadka Thu Feb 3 00:37:42 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 3 Feb 2000 02:37:42 +0200 (IST) Subject: [Python-Dev] patches reply-to (was: Python 1.6 timing) In-Reply-To: Message-ID: On Wed, 2 Feb 2000, Greg Stein wrote: > In this particular case, I think the "patches mailing list" would be a > self-contained list discussing a *patch*. Sure, it could certainly migrate > to python-dev when appropriate, but I think the majority of the discussion > should stay on the patches list. Otherwise, we'd just be spamming the -dev > list as if the patches list didn't exist. I retract my suggestion. Have python-patch, python-patch-discuss (python-patch would be replied-to: python-patch-discuss) and keep python-dev as a clean list. This way, people could just subscribe to python-patch, and when they get a patch they're interested in, they could subscribe to the discuss mailing list. That way, people could also subscribe to python-patch-discuss without subscribing to python-patch, to avoid the large attachments that would be sent by python-patch. Of course, mailman's new feature would automatically extract those attachments and post them up, so they can be downloaded by non-subscribers. enough-with-the-blabber-let's-just-get-something-going-ly y'rs, Z. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From guido@CNRI.Reston.VA.US Thu Feb 3 00:45:56 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 02 Feb 2000 19:45:56 -0500 Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: Your message of "Thu, 03 Feb 2000 02:32:41 +0200." References: Message-ID: <200002030045.TAA25323@eric.cnri.reston.va.us> > OK, I want to know one thing: is everyone comfortable with my suggestion > of a seperate list, with a reply-to: python-dev? Barry said he'd be > willing to hack it in, and everyone who talked seemed like this idea is > similar to what they have in mind. The sooner a consensus is reached, the > less patches Guido will have to deal with. Yes, let's do it. I actually think the reply-to is unnecessary to start (it's just a nicety). --Guido van Rossum (home page: http://www.python.org/~guido/) From Moshe Zadka Thu Feb 3 00:44:52 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 3 Feb 2000 02:44:52 +0200 (IST) Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: Message-ID: On Wed, 2 Feb 2000, Greg Stein wrote: > I'm assuming the current volunteer list is: > > - Guido (he volunteers whether he wants to or not :-) > - Moshe (??) Part time -- I'll take responsibility for the patches that interest me. There will likely be a few, and I *love* installing patches off the net, just to test my security measures <0.3 wink> > Others? I assume you'll get a similar response from many people: hopefully, for each patch it will either get booed automatically (hey! I just added braces instead of indentation to the parser) or will interest someone. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From gstein@lyra.org Thu Feb 3 00:50:50 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 2 Feb 2000 16:50:50 -0800 (PST) Subject: [Python-Dev] patches reply-to In-Reply-To: Message-ID: On Thu, 3 Feb 2000, Moshe Zadka wrote: >... > I retract my suggestion. Have python-patch, python-patch-discuss > (python-patch would be replied-to: python-patch-discuss) and keep > python-dev as a clean list. This way, people could just subscribe to > python-patch, and when they get a patch they're interested in, they could > subscribe to the discuss mailing list. That way, people could also > subscribe to python-patch-discuss without subscribing to python-patch, to > avoid the large attachments that would be sent by python-patch. Of course, > mailman's new feature would automatically extract those attachments and > post them up, so they can be downloaded by non-subscribers. Guido/Barry can decide on the final structure, but I'd recommend something a bit different: 1) drop the python- prefix. These are @python.org 2) just have "patches@python.org" I'm assuming the mailing list would be Guido-approved and the people on it would be required to "deal with the patch size". I think an open list might generate some noise rather than just "work". But again: the structure is ultimately up to Guido. Oops. I see a post from Guido saying "let's do it." In that case, it is probably best to move this discussion to the new list. I believe we need a statement of subscription policy from Guido. Or at least something to the effect of "python-dev members are free to subscribe, but you are expected to directly/positively contribute." I am presuming in all cases, that it would be administratively closed to non-python-dev members. Cheers, -g -- Greg Stein, http://www.lyra.org/ From guido@CNRI.Reston.VA.US Thu Feb 3 00:50:17 2000 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Wed, 02 Feb 2000 19:50:17 -0500 Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: Your message of "Thu, 03 Feb 2000 02:44:52 +0200." References: Message-ID: <200002030050.TAA25376@eric.cnri.reston.va.us> > I assume you'll get a similar response from many people: hopefully, > for each patch it will either get booed automatically (hey! I just added > braces instead of indentation to the parser) or will interest someone. The thing here that makes me slightly uncomfortable is how to keep track of patches that nobody picks up. a Jitterbug database would nicely do this, but I agree that it's inconvenient and overkill for other reasons. Perhaps we could use the "Linus Torvalds' inbox algorithm"? (When it overflows he deletes everything; "if it was important they'll send it again.") --Guido van Rossum (home page: http://www.python.org/~guido/) From tim_one@email.msn.com Thu Feb 3 00:58:07 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 2 Feb 2000 19:58:07 -0500 Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: Message-ID: <000001bf6de1$bc2d0740$c1a0143f@tim> [Greg Stein asks ...] > ... > It won't reduce his load unless we have actual volunteers that will truly > do some handling of the patches. > > I'm assuming the current volunteer list is: > > - Guido (he volunteers whether he wants to or not :-) > - Moshe (??) > - Greg in April > > Others? Sure -- but can't make a concrete time commitment. The time I've been able to make so far amounted to changing three letters in a FAQ . I hope the Jitterbug idea is adopted: clunky as it is, it's public, searchable, has categories (so supports *some* rudimentary notion of workflow), and doesn't tie up my 28.8 modem . From gstein@lyra.org Thu Feb 3 01:00:19 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 2 Feb 2000 17:00:19 -0800 (PST) Subject: [Python-Dev] unhandled patches? (was: patch handling) In-Reply-To: <200002030050.TAA25376@eric.cnri.reston.va.us> Message-ID: On Wed, 2 Feb 2000, Guido van Rossum wrote: > > I assume you'll get a similar response from many people: hopefully, > > for each patch it will either get booed automatically (hey! I just added > > braces instead of indentation to the parser) or will interest someone. > > The thing here that makes me slightly uncomfortable is how to keep > track of patches that nobody picks up. a Jitterbug database would > nicely do this, but I agree that it's inconvenient and overkill for > other reasons. Perhaps we could use the "Linus Torvalds' inbox > algorithm"? (When it overflows he deletes everything; "if it was > important they'll send it again.") We have a mailing list to archive the patches, so the "delete all" approach becomes a bit harder :-). Note that the approach requires feedback to be successful. The patch author must watch CVS to know if the patch went it. Otherwise, the patch handlers would be required to notify the author one way or another. With the notify route, then we'd have to state something like "you should resend if you haven't heard back within X weeks." But the notify/resend approach also creates an expectation that a patch *will* be handled within a *specific* timeframe. Dunno what I'd think the policy should look like right now. More thought needed on my part, so I'll just defer until a list is set up and discuss there... Cheers, -g -- Greg Stein, http://www.lyra.org/ From Moshe Zadka Thu Feb 3 00:57:41 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 3 Feb 2000 02:57:41 +0200 (IST) Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: <200002030050.TAA25376@eric.cnri.reston.va.us> Message-ID: On Wed, 2 Feb 2000, Guido van Rossum wrote: > > I assume you'll get a similar response from many people: hopefully, > > for each patch it will either get booed automatically (hey! I just added > > braces instead of indentation to the parser) or will interest someone. > > The thing here that makes me slightly uncomfortable is how to keep > track of patches that nobody picks up. a Jitterbug database would > nicely do this, but I agree that it's inconvenient and overkill for > other reasons. Perhaps we could use the "Linus Torvalds' inbox > algorithm"? (When it overflows he deletes everything; "if it was > important they'll send it again.") 1. This discussion is in the (as you put it) niceties are. You are unlikely to get that many patches that it is an *immediate* problem. 2. mailman (what fun to me! I'm dumping work on Barry) could be hacked (or hooked) into doing that: it can keep a list of all patches which never got a reply in whatever list is being "replied to:" (that would neccessitate developers to CC the list, at least on the first post, but that's probably a good idea to do anyway) and send a mail message after a week to a patch-submitter who hasn't gotten a reply with a notice to the effect that nobody seems interested in it so he should make a bit more noise. 3. Like in the CP4E BOF we're getting all geeky again (which is fine, since we're all geeks). Just get something out of the door! Even a mailing list with no policy at all to who sends to it is infinitely better then Guido's mailbox, as much as we all respect that mailbox. We'll argue the fine points of exactly how to score automatically irrelevant patches (and I've got just the algorithm <0.9 wink>) later. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From da@ski.org Thu Feb 3 01:04:46 2000 From: da@ski.org (David Ascher) Date: Wed, 2 Feb 2000 17:04:46 -0800 Subject: [Python-Dev] patch handling (was: Python 1.6 timing) References: <200002030050.TAA25376@eric.cnri.reston.va.us> Message-ID: <01c501bf6de2$a9b54950$c355cfc0@ski.org> From: "Guido van Rossum" To: Sent: Wednesday, February 02, 2000 4:50 PM Subject: Re: [Python-Dev] patch handling (was: Python 1.6 timing) > > I assume you'll get a similar response from many people: hopefully, > > for each patch it will either get booed automatically (hey! I just added > > braces instead of indentation to the parser) or will interest someone. > > The thing here that makes me slightly uncomfortable is how to keep > track of patches that nobody picks up. Sourceforge has a nice patch manager which allows a GUI'ish view of patches in context, discarding/deferring/etc. The code is available and open source. --david From Moshe Zadka Thu Feb 3 01:08:21 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 3 Feb 2000 03:08:21 +0200 (IST) Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: <01c501bf6de2$a9b54950$c355cfc0@ski.org> Message-ID: On Wed, 2 Feb 2000, David Ascher wrote: > Sourceforge has a nice patch manager which allows a GUI'ish view of patches > in context, discarding/deferring/etc. The code is available and open > source. However, I think I'd still like to be able to 1. send patches by e-mail (I don't always want to fire up a browser) 2. receive patches by e-mail (think of it as select() instead of a busy wait ;-) If the patch manager allows that/can do that with a simple addition, I'm for it. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From da@ski.org Thu Feb 3 01:12:37 2000 From: da@ski.org (David Ascher) Date: Wed, 2 Feb 2000 17:12:37 -0800 Subject: [Python-Dev] patch handling (was: Python 1.6 timing) References: <200002030050.TAA25376@eric.cnri.reston.va.us> Message-ID: <01f701bf6de3$c220da80$c355cfc0@ski.org> Another idea is to steal Ping's very cool idea of a 'nosy list'. If you missed his short talk at IPC8, he has a system which is a frontend to their bug database (but I think it would work well with patches). As far as I understood, folks submit bugs via an email message. Each email message has an _implicit_ mailing list, which is made up of all the people who either 'register' interest in the bug/patch, or who reply to the submission. That way there is an automatic broadcasting of the discussion to those parties interested in the topic at hand, and only those. Maybe Ping can explain in more detail how it works. It seemed like a brilliant idea to me. --david From guido@python.org Thu Feb 3 01:25:05 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 02 Feb 2000 20:25:05 -0500 Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: Your message of "Wed, 02 Feb 2000 17:12:37 PST." <01f701bf6de3$c220da80$c355cfc0@ski.org> References: <200002030050.TAA25376@eric.cnri.reston.va.us> <01f701bf6de3$c220da80$c355cfc0@ski.org> Message-ID: <200002030125.UAA25808@eric.cnri.reston.va.us> > Another idea is to steal Ping's very cool idea of a 'nosy list'. I missed it. Sounds an interesting long-term solution. I've heard about a similar concept elsewhere: you never unsubscribe to a list, each subject has its own list, and subjects just die. --Guido van Rossum (home page: http://www.python.org/~guido/) From ping@lfw.org Thu Feb 3 01:49:33 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Wed, 2 Feb 2000 17:49:33 -0800 (PST) Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: <200002030125.UAA25808@eric.cnri.reston.va.us> Message-ID: On Wed, 2 Feb 2000, Guido van Rossum wrote: > > Another idea is to steal Ping's very cool idea of a 'nosy list'. > > I missed it. Sounds an interesting long-term solution. I've heard > about a similar concept elsewhere: you never unsubscribe to a list, > each subject has its own list, and subjects just die. Yup, that's the general approach. The short paper is at http://www.lfw.org/ping/roundup.html; here is an excerpt describing the mechanism: a. New issues are always submitted by sending an e-mail message. This message is saved in a mail spool attached to the newly-created issue record, and copied to the relatively large user community of the application so everyone knows the issue has been raised. b. All e-mail messages sent by Roundup have their "Reply-To" field set to send mail back to Roundup, and have the issue's ID number in the Subject field. So, any replies to the initial announcement and subsequent threads are all received by Roundup and appended to the spool. c. Each issue has a "nosy list" of people interested in the issue. Any mail tagged with the issue's ID number is copied to this list of people, and any users found in the From:, To:, or Cc: fields of e-mail about the issue are automatically added to the nosy list. Whenever a user edits an item in the Web interface, they are also added to the list. The result is that no one ever has to worry about subscribing to anything. Indicating interest in an issue is sufficient, and if you want to bring someone new into the conversation, all you need to do is Cc: a message to them. It turns out that no one ever has to worry about unsubscribing, either: the nosy lists are so specific in scope that the conversation tends to die down by itself when the issue is resolved or people no longer find it sufficiently important. The transparent capture of the mail spool attached to each issue also yields a nice searchable knowledge repository over time. In practice, this has worked fairly well for developers at ILM, and no one has complained about missing mail they wanted or getting mail they didn't want -- which, given the apathetic nature of programmers, i suppose one could interpret as a positive empirical result. -- ?!ng "There's no point in being grown up if you can't be childish sometimes." -- Dr. Who From tim_one@email.msn.com Thu Feb 3 03:25:02 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 2 Feb 2000 22:25:02 -0500 Subject: [Python-Dev] Adding C ternary select (a?b:c) to Python? In-Reply-To: <00d901bf6d85$5577dd60$f29b12c2@secret.pythonware.com> Message-ID: <000101bf6df6$4220ee20$822d153f@tim> [Tim] > Man, you *are* Unfathomable <0.9 wink>! Even if true, you should > never have admitted to it <0.1 wink>. [Greg Wilson] > I think that's supposed to be , isn't it? [Fredrik Lundh] > not in this case. quoting a leading bot implementor: > > "We did requirements and task analysis, iterative > design, and user testing. You'd almost think emails > were an interface between people and bots." > > and I can assure you that proving that human beings don't > like weird but formally well designed syntaxes was the easy > part of that project... (but don't tell the schemers ;-) Right on, effbot! "Bots Helping Bots" is our motto. I'm quite sure the timbot's use of predates the Web's twisted formalization of what originally started life as a typographic device in a snail-mail newsletter, when the timbot discovered that "real people" had no idea what to make of ;-) style emoticons. User testing is exactly on target. Iterative design, too: the timbot's original use of [grin] didn't work nearly as well. The introduction of fractional winkery was actually a minor refinement, yet widely promoted by intuitionists as if it were the key idea. Feh. > > > "Larry Wall should be shot. Along with Bill Joy and Eric Allman." > -- Daniel Finster, on comp.lang.lisp > > "Why, just because you guys frittered away a 20-year headstart?" > -- Larry Wall Say what you will about Perl, but you gotta love Larry! I recently filed a Perl bug that was apparently introduced the day Perl5 hit the streets and somehow went unnoticed for years, and had a nice exchange with him. Looking over other recent bugs, I stumbled into this one first: @array = "0" .. -1; That, of course, computes an array of 100 elements, "0" thru "99": the string "0" gets magically autoincremented, as if it were an integer, until the *length* of the resulting string exceeds the length of the string "-1". That this wasn't justified as "a feature" gives me hope that Guido's presence on earth has done *some* little bit of good . time-for-an-oil-change-ly y'rs - tim From thomas.heller@ion-tof.com Thu Feb 3 13:27:00 2000 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 3 Feb 2000 14:27:00 +0100 Subject: [Python-Dev] Revised proposal (and preliminary implementation): Registry access module for Python on Windows Message-ID: <019301bf6e4a$5a53f470$4500a8c0@thomasnotebook> Ok, at least the first proposal did start the discussion. Here is a revised one: A preliminary implementation is available at http://starship.python.net/crew/theller/ ---------------------------------------------------------------------- winreg - windows registry access module Exception: error - raised when a function fails. Will contain a windows error code and a textual description. Objects: regnode object - represents a open key in the registry. Functions: OpenKey (name) -> regnode object Opens an existing key with the specified access rights and returns a regnode object. name is specified like "HKLM\Software\Python" or "HKEY_LOCAL_MACHINE\Software\Python" CreateKey (name) -> regnode object Creates a new key or opens an existing one and returns a regnode object. For the name format see OpenKey regnode object methods: Values () -> dict Returns a dictionary mapping names to values. The or unnamed value has the key ''. The values are either strings or integers, depending on the REG_* type. GetValue ([name]) -> integer or string Returns a value specified by name or the default value. SetValue ([name,] value) Set a named or the value. Named values must be integers or string (which are stored as REG_DWORD or REG_SZ). Should an optional third parameter be used, allowing to store in other REG_* typecodes? I dont think so. DeleteValue ([name]) Deletes a named or the value. SubKeys () -> sequence Returns a sequence containing the names of all subkeys. DeleteKey (name [,recursive=0]) If recursive is 0, deletes the named key if no subkeys exist. If there are subkeys an error is raised. If recursive is not 0, the named key is deleted including subkeys. OpenKey (name) -> regnode object Openes an existing subkey and returns a regnode object pointing to it. CreateKey (name) -> regnode object Creates a new or openes an existing subkey and returns a regnode object pointing to it. regnode objects have the following properties: name - the name of the RegistryKey, something like "HKLM\Software\Python" hkey - the integer keyhandle ---------------------------------------------------------------------- Thomas Heller From thomas.heller@ion-tof.com Thu Feb 3 13:26:39 2000 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 3 Feb 2000 14:26:39 +0100 Subject: [Python-Dev] Proposal: Registry access for Python on Windows References: Message-ID: <018b01bf6e4a$4db27700$4500a8c0@thomasnotebook> [Thomas] > > I propose to include functions to access the windows registry > > into the python 1.6 core. [Mark] > > Agreed! Fine. Thats the _main_ purpose of the proposal. > > winreg - registry access module > > Im not convinced that we need either a) a new implementation, or b) a new > API. > > I would propose that we use the existing registry access functions from > win32api - as I didnt code them, Im not suggesting this due to my own ego. > The reasons for my beliefs are: > > * The new API does not add any new value. The only changes I can see are > that OpenKey() has had a reserved value dropped and QueryValue() has an > extra redundant result. The redundant result value was a typo in the proposal. > Indeed the new API does not appear a new API at > all, so should be clearly stated if indeed it is (and in which case, exactly > what the changes are) or be documented as a pythonic wrapping of the > existing win32 registry API. Further, there is some useful functionality > dropped. I posted my proposal to start the discussion - so far it has been successfull - not as the final specification for the api. I admit: it has been too low level. I have posted (and implemented) a new proposal describing a more high level interface. In this I followed Gordon's suggestion: Provide the minimum needed. If one wants to do more special things, one probably needs your win32 extensions anyway. > > * Well tested. There are some obscure rules and code that experience has > shown we need. It would be a shame to reinvent those wheels. Already too late! (Programming is fun :-) (Is competition good ?) > It has taken > a few years to get it "just right", and this was an implementation by people > who know their stuff! The only obscure code I can see (having looked at win32api source AFTER writing my code) is the handling of REG_MULTI_SZ. These are returned as one normal string with embedded NULL bytes by my implementation. BTW: This is also how regedit.exe (but not regedt32) handles these. > > Why not take the existing code as it stands (which _does_ include an > auto-closing key) and add a new .py wrapper? At least Python will have a > "complete, but low-level" registry API, and a higher level "easier to use, > but maybe not complete" layer implemented in Python. > > Indeed, Guido has stated that the existing win32api functions would be > suitable for the core. Don't you think that the raw Win32 api functions are much too low level to belong into core python? How should they be documented? Pointers to msdn.microsoft.com? > However, the issue is the "coding style" - and I > admit that if I knew exectly what that meant I would make the change - eg, > if it only means renaming the identifiers it would be trivial - however, it > would also appear to be excessively anal, so Im sure there is more to it > than that :-) It seems (but I may be wrong) that my coding style is nearer to Guido's than that of win32api. Regards, Thomas From fdrake@acm.org Thu Feb 3 14:49:49 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 3 Feb 2000 09:49:49 -0500 (EST) Subject: [Python-Dev] __contains__ hook In-Reply-To: <3898BEBD.93DFBCA7@lemburg.com> References: <200002021849.NAA23092@eric.cnri.reston.va.us> <3898BEBD.93DFBCA7@lemburg.com> Message-ID: <14489.38285.840829.261016@weyr.cnri.reston.va.us> M.-A. Lemburg writes: > Shouldn't 'in' be a slot of the sequence methods ? I'd suggest > creating a new tp_flag bit and then extending tp_as_sequence Only if we want to restrict set-like behavior to sequences, and I don't think that's clear, though it does mirror the current situation. Regardless of the location of the slot, why should a flag be needed? Testing the slot for NULL is necessary to avoid core dumps anyway. > plus of course add an abstract function to abstract.c: > > PySequence_Contain(PyObject *container, PyObject *element) There's already PySequence_In(...); see: http://www.python.org/doc/current/api/sequence.html#l2h-135 I'm inclined to add PyObject_In(...) (or ..._Contains(); I like Contains better than In, but there's precedence for In and that's more important) and define the new slot on the Object using one of the reserved spaces. That allows a clean interface for "pure" sets that don't have to "look like" sequences or mappings. > which uses the above slot after testing the tp_flag setting. > Python instances, lists, tuples should then support this new > slot. We could even sneak in support for dictionaries once we > decide whether semantics whould be Bait! -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From tismer@tismer.com Thu Feb 3 14:47:51 2000 From: tismer@tismer.com (Christian Tismer) Date: Thu, 03 Feb 2000 15:47:51 +0100 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) References: <000301bf652e$f348d1c0$132d153f@tim> Message-ID: <38999517.F32AFBF5@tismer.com> Tim Peters wrote: > > [Jim Fulton] [association objects] > Jim, I've been intrigued by this idea for all the years you've been > suggesting it , but I've never understood what it is you're proposing! > This is the Python-Dev list, so feel encouraged to present it in concrete > implementation terms instead of ambiguous English. Or maybe an interface? My guess is: An association object adds another level of indirection to namespaces and makes global variables be more like true variables, i.e. changing them in one place changes them everywhere. > Does this kind of dictionary have keys? If so, of what type? What type are > the values? Best I can make sense of the above, the values are "association > objects", each of which contains a name and a value, and a key is maybe a > duplicate of the name in the association object to which it maps. "A name" > may or may not be a string -- I can't tell. Or maybe by "dictionary" you > didn't intend Python's current meaning for that word at all. I assume "a > value" is a PyObject*. The whole thrust *appears* to be to get names to map > to a PyObject** instead of PyObject*, but if that's the ticket I don't know > what association objeects have to do with it. I don't believe that the actual implementation matters too much and is still open to be choosen. Here my approach: Let an association object be a pair of a key and a value. The restrictions for keys may be the same as for dict keys. We can now either use dicts as they are, inserting asso-objects as values and sharing the key field, or invent new dictionaries which have no key/value pairs at all, but just references to asso-objects. In either case, we have the advantage that further references by global use from a function or by imports will always add to the asso-object, not to its value. This keeps the value changeable, like a list with one element, kind of boxed object. Since the asso-objects stay alive as long as they are referenced, they are never moved, and it is ok to refer to their address. For a function, this means that it can resolve a global at compile time. If the asso-object exists already, it has a fixed memory address and can be placed into the code object. If it does not exist, it can be created in the global dictionary or special asso-dictionary, whatever we'll use. The value will be NULL in this case, and this is perfect. If we do right, a value will have been inserted before the function is called, or we will raise a name error. The idea is simply to generate fixed slots for global names which never move. By mentioning the name, we create such a slot. The slot is alive as long it is seen, i.e. refcount > 0. There must be a general way to look these things up, either by the per-module dictionary, or by a specialized one. Finally I'd tend to do the latter, since those unitialized key/value asso-objects would give ambiguity what dict.keys() should be then. For consistency, I would hide all asso-objects in a special asso-collection per module. They could be placed into the modules dict, when their value becomes assigned first time. Alternatively, they are not created at compile time but at runtime, when a value is assigned. I'm not sure yet. Now, moving on from globals to all name spaces: If they are all handled by the asso-approach, can we use it to speed up attribute access for classes and instances? I guess we can! But I need more thought. > > An import like: > > > > from foo import spam > > sentence == """ > > would copy the association between the name 'foo' and a > > value from module 'spam' into the current module. > > Where does the idea that 'spam' is a *module* here come from? It doesn't > make sense to me, and I'm so lost I'll spare everyone my further confusions > . """ def swap_words(str, one, two): pieces = string.split(str, one) for i in range(len(pieces)): pieces[i] = string.replace(pieces[i], two, one) return string.join(pieces, two) sentence = swap_words(sentence, "'foo'", "'spam'") ciao - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Düppelstr. 31 : *Starship* http://starship.python.net 12163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From fdrake@acm.org Thu Feb 3 15:05:25 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 3 Feb 2000 10:05:25 -0500 (EST) Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: References: Message-ID: <14489.39221.91861.262217@weyr.cnri.reston.va.us> Moshe Zadka writes: > OK, I want to know one thing: is everyone comfortable with my suggestion > of a seperate list, with a reply-to: python-dev? Barry said he'd be > willing to hack it in, and everyone who talked seemed like this idea is I think this is the right approach. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From fdrake@acm.org Thu Feb 3 15:07:52 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 3 Feb 2000 10:07:52 -0500 (EST) Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: References: Message-ID: <14489.39368.355346.261453@weyr.cnri.reston.va.us> Greg Stein writes: > - Guido (he volunteers whether he wants to or not :-) > - Moshe (??) > - Greg in April I can certainly help out a bit, at least for small patches and anything related to the documentation (including additions of docstrings to module sources and the like). -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From fdrake@acm.org Thu Feb 3 15:09:54 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 3 Feb 2000 10:09:54 -0500 (EST) Subject: [Python-Dev] patches reply-to (was: Python 1.6 timing) In-Reply-To: References: Message-ID: <14489.39490.879151.5888@weyr.cnri.reston.va.us> Moshe Zadka writes: > I retract my suggestion. Have python-patch, python-patch-discuss > (python-patch would be replied-to: python-patch-discuss) and keep > python-dev as a clean list. This way, people could just subscribe to I'd be fine with this as well. If Ping can release his issue-tracker any time soon, that would also be a really nice tool. That was quite impressive. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From jim@digicool.com Thu Feb 3 15:34:54 2000 From: jim@digicool.com (Jim Fulton) Date: Thu, 03 Feb 2000 10:34:54 -0500 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) References: <200001200419.XAA01969@mira.erols.com> <38871665.C3B6FFEE@digicool.com> <38871CE5.53FB9D68@digicool.com> <200001201545.KAA21304@eric.cnri.reston.va.us> <38873C52.29FEAC6D@digicool.com> <200001201720.MAA21534@eric.cnri.reston.va.us> Message-ID: <3899A01E.DE29FC94@digicool.com> Eek, I didn't realized this thread had continued until I happened to notice Christian's post today. I get too much email Guido van Rossum wrote: > > [me] > > > I've never liked this very much, mostly because it breaks simplicity: > > > the idea that a namespace is a mapping from names to values > > > (e.g. {"limit": 100, "doit": , ...}) is beautifully > > > simple, while the idea of inserting an extra level of indirection, no > > > matter how powerful, is much murkier. > > [Jim F] > > How so? It doesn't change the mapping semantics. > > My assumption is that in your version, the dictionary would contain > special objects which then would contain the > referenced objects. E.g. {"limit": , "doit": >}. Thus, d["limit"] would be that object, > while previously it would return 100. No. The idea is to have "association" objects. We can create these directly if we want: a=Association('limit',100) print a.key, a.value # whatever The association value is mutable, but the key is not. A namespace object is a collection of association objects such that no two items have the same key. Internally, this would be very much like the current dictionary except that instead of an array of dictentries, you'd have an array of association object pointers. Effectively, associations are exposed dictentries. Externally, a namspace acts more or less like any mapping object. For example, when someone does a getitem, the namespace object will find the association with the desired key and return it's value. In addition, a namspace object would provide methods along the lines of: associations() Return a sequence of the associations in the namespace addAssociation(assoc) Add the given association to the namsspace. This creates another reference to the association. Changing the association's value also changes the value in the namespace. getAssociation(key) Get the association associated with the key. A setitem on a namespace modifies an existing association if there is already an association for the given key. For example: n1=namespace() n1['limit']=100 n2=namespace() n2.addAssociation(n1.getAssociation('limit')) print n2['limit'] # prints 100 n1['limit']=200 print n2['limit'] # prints 200 When a function is compiled that refers to a global variable, we get the association from the global namespace and store it. The function doesn't need to store the global namespace itself, so we don't create a circular reference. Note that circular references are bad even if we have a more powerful gc. For example, by not storing the global namespace in a function, we don't have to worry about the global namespace being blown away before a destructor is run during process exit. When we use the global variable in the function, we simply get the current value from the association. We don't have to look it up. Namespaces would have other benefits: - improve the semantics of: from spam import foo in that you'd be importing a name binding, not a value - Be useful in any application where it's desireable to share a name binding. > > Again, it would also make function global variable access > > faster and cleaner in some ways. > > But I have other plans for that (if the optional static typing stuff > ever gets implemented). Well, OK, but I argue that the namespace idea is much simpler and more foolproof. > > > however it would break a considerable amount of old code, > > > I think. > > > > Really? I wonder. I bet it would break alot less old > > code that other recent changes. > > Oh? Name some changes that broke a lot of code? The move to class-based exceptions broke alot of our code. Maybe we can drop this point. Do you still think that the namespace idea would break alot of code? Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (888) 344-4332 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From jim@digicool.com Thu Feb 3 15:39:50 2000 From: jim@digicool.com (Jim Fulton) Date: Thu, 03 Feb 2000 10:39:50 -0500 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) References: <000301bf652e$f348d1c0$132d153f@tim> Message-ID: <3899A146.210BD2D9@digicool.com> Tim Peters wrote: > > [Jim Fulton] > > ... > > A change to the way that namespaces are handled > > could make this work and have a number of other benefits, > > like global name usage without namespace lookups. > > > > I've suggested this to Guido in the past. His > > reasonable response is that this would be too big a > > change for Python 1. Maybe this is something to consider > > for Python 2? > > > > The basic idea (borrowed from Smalltalk) is to have a kind > > of dictionary that is a collection of "association" > > objects. An association object is simply a pairing of a > > name with a value. Association objects can be shared among > > multiple namespaces. > > Jim, I've been intrigued by this idea for all the years you've been > suggesting it , but I've never understood what it is you're proposing! > This is the Python-Dev list, so feel encouraged to present it in concrete > implementation terms instead of ambiguous English. Or maybe an interface? > > interface a_kind_of_dictionary_that_is_a_collection_of_\ > association_objects: > # ??? beats me ... > > Or maybe as a C struct? For example, is "an association object" a (char*, > PyObject*) pair? I just responded to Guido in a bit more detail. Hopefully, this will be of sufficient clarity. If not, then I'll be happy to work up a Python demonstration. > Does this kind of dictionary have keys? Yes. > If so, of what type? Whatever you want. Just like a dictionary. > What type are > the values? ditto. > Best I can make sense of the above, the values are "association > objects", each of which contains a name and a value, and a key is maybe a > duplicate of the name in the association object to which it maps. "A name" > may or may not be a string -- I can't tell. Sorry. See my reply to Guido and let me know if I'm still being too vague. > Or maybe by "dictionary" you > didn't intend Python's current meaning for that word at all. I assume "a > value" is a PyObject*. The whole thrust *appears* to be to get names to map > to a PyObject** instead of PyObject*, but if that's the ticket I don't know > what association objeects have to do with it. > > > An import like: > > > > from foo import spam > > > > would copy the association between the name 'foo' and a > > value from module 'spam' into the current module. > > Where does the idea that 'spam' is a *module* here come from? From my lack of careful typing. Sorry. let me try again: An import like: from foo import spam would copy the association between the name 'spam' and a value from module 'foo' into the current module. Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (888) 344-4332 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From fdrake@acm.org Thu Feb 3 15:42:42 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 3 Feb 2000 10:42:42 -0500 (EST) Subject: [Python-Dev] zipfile.py, file system abstractions Message-ID: <14489.41458.735589.927818@weyr.cnri.reston.va.us> --rDGr2BijA3 Content-Type: text/plain; charset=us-ascii Content-Description: message body text Content-Transfer-Encoding: 7bit At one point we discussed file system abstractions on this list. Do we want to pursue the idea? I have an implementation that's been tested on Unix: a Filesystem instance refers to either the entire "native" filesystem, or a subset of a filesystem (either a filesystem object or the native filesystem), based on a "root" that's a directory within the parent filesystem. There'd need to be some work done to make sure it works properly on Windows and the Mac, but I don't think that would be a lot of work. I think this should be available as an abstraction in Python. Implementing this on top of a ZIP/JAR file is also useful. If Jim A.'s zipfile.py will be added to the standard library in 1.6, I'd like to add a Filesystem class that operates on a zipfile object. Any thoughts? If you want this, someone needs to check in and document zipfile.py. ;) I'll add filesys.py & it's documentation. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives --rDGr2BijA3 Content-Type: text/x-python Content-Description: Filesystem abstraction for Python Content-Disposition: inline; filename="filesys.py" Content-Transfer-Encoding: 7bit """FilesystemAPI.Filesystem implementations. These aren't actually connected to ILU in any way. """ __version__ = '$Revision: 1.7 $' # This probably doesn't work anywhere that doesn't use Unix-style # pathnames. class Filesystem: def __init__(self, fs=None, root=None): if fs: self._fs = fs self._path = fs.path self._open = fs.open else: import os, __builtin__ self._fs = os self._path = os.path self._open = __builtin__.open self._cwd = self._fs.sep self.pardir = self._fs.pardir self.curdir = self._fs.curdir self.sep = self._fs.sep if root: self._root = root = self._path.abspath(root) self.path = SubsettingPath(self, self._path, root) else: self._root = '' self.path = SimplePath(self, self._path) def __getinitargs__(self): return (self._fs, self._root) def chdir(self, path): return self.__pathcall(self._fs.chdir, path) def fstat(self, file): try: return self._fs.fstat(file) except IOError, e: # make sure no filename information leaks out if hasattr(file, "name"): e.filename = file.name else: e.filename = None raise e def getcwd(self): return self._cwd def listdir(self, path): return self.__pathcall(self._fs.listdir, path) def lstat(self, path): return self.__pathcall(self._fs.lstat, path) def mkdir(self, path): self.__pathcall(self._fs.mkdir, path) def fopen(self, path, mode="r"): name = self.path._normpath(path) try: f = self._open(self._root + name, mode) except IOError, e: e.filename = path raise e if self._root: f = SubsettingFile(f, path, mode) return f def remove(self, path): return self.__pathcall(self._fs.remove, path) def rmdir(self, path): return self.__pathcall(self._fs.rmdir, path) def stat(self, path): return self.__pathcall(self._fs.stat, path) def unlink(self, path): return self.__pathcall(self._fs.unlink, path) def __pathcall(self, func, path): name = self.path._normpath(path) try: return func(self._root + name) except IOError, e: e.filename = path raise e class SubsettingFile: """Class used to mask a real file when the origin filesystem is being 'subsetted'. This avoids revealing the real file name without restricting the ability to use the file as a replacement sys.stdout (which needs softspace support). """ def __init__(self, file, path, mode): self.__dict__['_file'] = file self.__dict__['_mode'] = mode self.__dict__['_path'] = path self.__dict__['name'] = path def __getattr__(self, name): v = getattr(self._file, name) if callable(v) and not self.__dict__.has_key(name): v = SubsettingMethod(v, self._path).call self.__dict__[name] = v return v def __setattr__(self, name, value): setattr(self._file, name, value) def __repr__(self): oc = self._file.closed and "closed" or "open" return "<%s file %s, mode %s at %x>" \ % (oc, `self._path`, `self._mode`, id(self)) class SubsettingMethod: def __init__(self, method, path): self.method = method self.path = path def call(self, *args, **kw): try: return apply(self.method, args, kw) except IOError, e: e.filename = self.path raise e class BasePath: def __init__(self, fs, realpath): self._fs = fs self._path = realpath self._prefix = '' try: fs.stat(fs.curdir) self._have_stat = 1 except AttributeError: self._have_stat = 0 def __getattr__(self, name): v = getattr(self._path, name) setattr(self, name, v) return v def abspath(self, path): if not self.isabs(path): path = self._path.join(self._fs.getcwd(), path) return self.normpath(path) def basename(self, path): return self._path.basename(path) def commonprefix(self, list): return self._path.commonprefix(list) def dirname(self, path): return self._path.dirname(path) def exists(self, path): name = self._normpath(path) try: return self._path.exists(self._prefix + name) except IOError, e: e.filename = path raise e def isabs(self, path): return self._path.isabs(path) def isdir(self, path): name = self._normpath(path) try: return self._path.isdir(self._prefix + name) except IOError, e: e.filename = name raise e def isfile(self, path): name = self._normpath(path) try: return self._path.isfile(self._prefix + name) except IOError, e: e.filename = path raise e def join(self, *args): return apply(self._path.join, args) def normcase(self, path): return self._path.normcase(path) def normpath(self, path): return self._path.normpath(path) def samefile(self, path1, path2): """Return true if both pathname arguments refer to the same file or directory. If the underlying filesystem supports stat(), the comparison is made using samestat(), otherwise a simple string comparison is used. """ if self._have_stat: return self.samestat(self._fs.stat(path1), self._fs.stat(path2)) else: p1 = self._normpath(path1) p2 = self._normpath(path2) return p1 == p2 def sameopenfile(self, file1, file2): stat1 = self._fs.fstat(file1) stat2 = self._fs.fstat(file2) return self.samestat(stat1, stat2) def split(self, path): return self._path.split(path) def splitdrive(self, path): return self._path.splitdrive(path) def splitext(self, path): return self._path.splitext(path) def walk(self, root, func, arg): queue = [root] while queue: dir = queue[0] del queue[0] if self.isdir(dir): files = self._fs.listdir(dir) func(arg, dir, files) for file in files: path = self.join(dir, file) if self.isdir(path): queue.append(path) class SimplePath(BasePath): def __getinitargs__(self): return (self._fs, self._path) def _normpath(self, path): return self.abspath(path) class SubsettingPath(BasePath): """os.path equivalent that pretends some directory down in the tree is the root. This class works relative to a filesystem and path-math object that are passed to the constructor. """ def __init__(self, fs, realpath, prefix): BasePath.__init__(self, fs, realpath) self._prefix = prefix self._rootref = "%s%s%s" \ % (self._fs.sep, self._fs.pardir, self._fs.sep) def __getinitargs__(self): return (self._fs, self._path, self._prefix) def _normpath(self, path): """Return the absolute pathname for path, relative to the filesystem object.""" p = self.abspath(path) while p[:len(self._rootref)] == self._rootref: # on Unix, /.. refers to /, on Windows, \.. refers to \ (yech!) p = p[len(self._rootref) - len(self._fs.sep):] if not self.isabs(p): raise IOError(errno.ENOENT, "No such file or directory", path) return p --rDGr2BijA3-- From jim@digicool.com Thu Feb 3 15:47:48 2000 From: jim@digicool.com (Jim Fulton) Date: Thu, 03 Feb 2000 10:47:48 -0500 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) References: <000301bf652e$f348d1c0$132d153f@tim> <38999517.F32AFBF5@tismer.com> Message-ID: <3899A324.280FB276@digicool.com> Christian Tismer wrote: > > Tim Peters wrote: > > > > [Jim Fulton] > > [association objects] > > > Jim, I've been intrigued by this idea for all the years you've been > > suggesting it , but I've never understood what it is you're proposing! > > This is the Python-Dev list, so feel encouraged to present it in concrete > > implementation terms instead of ambiguous English. Or maybe an interface? > > My guess is: Ugh. Sorry to make you guess.... > An association object adds another level of indirection > to namespaces and makes global variables be more like true > variables, i.e. changing them in one place changes them > everywhere. Right. > > Does this kind of dictionary have keys? If so, of what type? What type are > > the values? Best I can make sense of the above, the values are "association > > objects", each of which contains a name and a value, and a key is maybe a > > duplicate of the name in the association object to which it maps. "A name" > > may or may not be a string -- I can't tell. Or maybe by "dictionary" you > > didn't intend Python's current meaning for that word at all. I assume "a > > value" is a PyObject*. The whole thrust *appears* to be to get names to map > > to a PyObject** instead of PyObject*, but if that's the ticket I don't know > > what association objeects have to do with it. > > I don't believe that the actual implementation matters too much > and is still open to be choosen. Here my approach: > > Let an association object be a pair of a key and a value. > The restrictions for keys may be the same as for dict keys. > > We can now either use dicts as they are, inserting asso-objects > as values and sharing the key field, or invent new dictionaries > which have no key/value pairs at all, but just references > to asso-objects. Right, Replace dictentries with association object pointers. > In either case, we have the advantage that further references > by global use from a function or by imports will always add to the refcount of > to the asso-object, not to its value. This keeps the value > changeable, like a list with one element, kind of boxed object. > Since the asso-objects stay alive as long as they are referenced, > they are never moved, and it is ok to refer to their address. Yup. > For a function, this means that it can resolve a global at > compile time. If the asso-object exists already, it has a fixed > memory address and can be placed into the code object. > If it does not exist, it can be created in the global dictionary > or special asso-dictionary, whatever we'll use. The value will > be NULL in this case, and this is perfect. If we do right, a > value will have been inserted before the function is called, or we > will raise a name error. exactly. You are a great guesser! :) > The idea is simply to generate fixed slots for global names > which never move. By mentioning the name, we create such > a slot. The slot is alive as long it is seen, i.e. refcount > 0. > > There must be a general way to look these things up, either by > the per-module dictionary, or by a specialized one. Finally > I'd tend to do the latter, since those unitialized key/value > asso-objects would give ambiguity what dict.keys() should > be then. For consistency, I would hide all asso-objects in > a special asso-collection per module. They could be placed > into the modules dict, when their value becomes assigned > first time. > Alternatively, they are not created at compile time but > at runtime, when a value is assigned. I'm not sure yet. > > Now, moving on from globals to all name spaces: > If they are all handled by the asso-approach, can we > use it to speed up attribute access for classes and > instances? I guess we can! But I need more thought. Yes, it needs more thought. > > > An import like: > > > > > > from foo import spam > > > > sentence == """ > > > would copy the association between the name 'foo' and a > > > value from module 'spam' into the current module. > > > > Where does the idea that 'spam' is a *module* here come from? It doesn't > > make sense to me, and I'm so lost I'll spare everyone my further confusions > > . > """ > > def swap_words(str, one, two): > pieces = string.split(str, one) > for i in range(len(pieces)): > pieces[i] = string.replace(pieces[i], two, one) > return string.join(pieces, two) > > sentence = swap_words(sentence, "'foo'", "'spam'") Ooh ooh, you've invented a 'Jim translator bot'! Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (888) 344-4332 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From guido@python.org Thu Feb 3 16:21:37 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 03 Feb 2000 11:21:37 -0500 Subject: [Python-Dev] __contains__ hook In-Reply-To: Your message of "Thu, 03 Feb 2000 09:49:49 EST." <14489.38285.840829.261016@weyr.cnri.reston.va.us> References: <200002021849.NAA23092@eric.cnri.reston.va.us> <3898BEBD.93DFBCA7@lemburg.com> <14489.38285.840829.261016@weyr.cnri.reston.va.us> Message-ID: <200002031621.LAA28950@eric.cnri.reston.va.us> > M.-A. Lemburg writes: > > Shouldn't 'in' be a slot of the sequence methods ? I'd suggest > > creating a new tp_flag bit and then extending tp_as_sequence Fred Drake: > Only if we want to restrict set-like behavior to sequences, and I > don't think that's clear, though it does mirror the current > situation. > Regardless of the location of the slot, why should a flag be needed? Because if we add a slot to the as_sequence struct, old extensions that haven't been recompiled will appear to have garbage in that slot (because they don't actually have it). When we use a spare slot in the main type struct, that problem doesn't exist; but the as_sequence struct and friends don't have spares. > Testing the slot for NULL is necessary to avoid core dumps anyway. > > > plus of course add an abstract function to abstract.c: > > > > PySequence_Contain(PyObject *container, PyObject *element) > > There's already PySequence_In(...); see: That's just a backwards compatibility alias for PySequence_Contains; see abstract.h. (PySequence_In() was a bad name, because it has its arguments reversed with respect to the 'in' operator: PySequence_In(seq, item) is equivalent to item in seq; you would expect PySequence_In(item, seq). The PySequence_Contains name correctly suggests the (seq, item) argument order. > http://www.python.org/doc/current/api/sequence.html#l2h-135 Maybe the docs need to be updated? (Hint, hint.) > I'm inclined to add PyObject_In(...) (or ..._Contains(); I like > Contains better than In, but there's precedence for In and that's more > important) and define the new slot on the Object using one of the > reserved spaces. That allows a clean interface for "pure" sets that > don't have to "look like" sequences or mappings. > > > which uses the above slot after testing the tp_flag setting. > > Python instances, lists, tuples should then support this new > > slot. We could even sneak in support for dictionaries once we > > decide whether semantics whould be > > Bait! Yuck. The same argument for disallowing 'x in dict' applies to the C API. There's already PyMapping_HasKey(). --Guido van Rossum (home page: http://www.python.org/~guido/) From Moshe Zadka Thu Feb 3 16:38:04 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 3 Feb 2000 18:38:04 +0200 (IST) Subject: [Python-Dev] __contains__ hook In-Reply-To: <200002031621.LAA28950@eric.cnri.reston.va.us> Message-ID: I like to be personally CC'ed on mails here, and I assume arbitrarily everyone else is like me. If you don't want to be CC'ed, please mention it personally. On Thu, 3 Feb 2000, Guido van Rossum wrote: > > > which uses the above slot after testing the tp_flag setting. > > > Python instances, lists, tuples should then support this new > > > slot. We could even sneak in support for dictionaries once we > > > decide whether semantics whould be > > > > Bait! > > Yuck. The same argument for disallowing 'x in dict' applies to the C > API. There's already PyMapping_HasKey(). I totally agree with Guido -- for me, the whole point of this hack is to avoid people asking for 'in' in dicts: this way we can code a class 'set' (as I've demonstrated), and have rational semantics to 'in' which is just as efficient as 'dict.has_key'. I'm not quite sure where we want to put the C API version of __contains__ - I'd add a tp_as_set, but the only method seems to be 'in', so it seems like a waste of valuable real-estate before we are driven into non-backwards-compatability. I think I should at least ask permission from the owner before I move over there, trampling everything in my way What does everyone think about that? -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From guido@python.org Thu Feb 3 16:55:44 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 03 Feb 2000 11:55:44 -0500 Subject: [Python-Dev] zipfile.py, file system abstractions In-Reply-To: Your message of "Thu, 03 Feb 2000 10:42:42 EST." <14489.41458.735589.927818@weyr.cnri.reston.va.us> References: <14489.41458.735589.927818@weyr.cnri.reston.va.us> Message-ID: <200002031655.LAA29053@eric.cnri.reston.va.us> [Fred, proposing filesystem.py] I think we should discuss the desired interface here a bit more before talking about specific implementations. This class wants to look like the os module, except that instead of an open method (which returns an integer filedescriptor) it has a fopen method, similar to the built-in open function (which returns a file-like object). The implementation you post would confuse people because it has some specific hooks to deal with subsetting an existing filesystem (sort-of like chroot, but enforced by path algebra). This would be usable in an rexec environment, but not when dealing with a zipfile or a remote filesystem. If we define an API that's a subset of the os module (including os.path), maybe we can make the os module conform to the same API; I think all that's needed is to make a list of methods that are part of the interface, and add os.fopen as an alias for __builtin__.open. --Guido van Rossum (home page: http://www.python.org/~guido/) From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 3 17:07:19 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 3 Feb 2000 12:07:19 -0500 (EST) Subject: [Python-Dev] patch handling (was: Python 1.6 timing) References: <200002030050.TAA25376@eric.cnri.reston.va.us> Message-ID: <14489.46535.246097.304601@anthem.cnri.reston.va.us> >>>>> "MZ" == Moshe Zadka writes: MZ> 2. mailman (what fun to me! I'm dumping work on Barry) could MZ> be hacked (or hooked) into doing that: Not a good strategy right now :) Here's my proposal, from short term/easy to long term: 1. Create patches@python.org which will be the official advertised address to send patches. This will be a Mailman mailing list to which anybody can post, but to which subscriptions are closed. Amount of work: trivial. 2. Create patches-discuss@python.org which will be the address to discuss patches@python.org postings. We make Reply-To: for patches@ be patches-discuss@ Amount of work: relatively small (for me to hack Mailman) 3. If someone will write a Python function to scan a string, returning true or false as to whether it contains a patch, I will integrate this into Mailman so that if a patch is found, it will be forwarded to the Jitterbug database address. Ideally this filter would check to be sure that the disclaimer text is included, but that's not necessary. Guido (or someone) is still going to have to decide whether the small text or a wet signature is necessary. Amount of work: Fairly non-trivial for someone other than Barry Not too bad for Barry to hack this into Mailman 4. When !?ng frees Roundup we look at adopting it. From everything that I heard about it, it's OHS (official hot shit). I want, I want! :) Amount of work: none now, except for !?ng :) Later, some for one of us CNRI'ers 5. Move the whole kit-and-kaboodle to SourceForge (or server51.freshmeat.net which looks to provide many of the same services). We cannot be the only project facing these same issues, so let's leverage off of what others have done. Amount of work: unknown, but I'm looking into it for Mailman -Barry From guido@python.org Thu Feb 3 17:12:51 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 03 Feb 2000 12:12:51 -0500 Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: Your message of "Thu, 03 Feb 2000 12:07:19 EST." <14489.46535.246097.304601@anthem.cnri.reston.va.us> References: <200002030050.TAA25376@eric.cnri.reston.va.us> <14489.46535.246097.304601@anthem.cnri.reston.va.us> Message-ID: <200002031712.MAA29198@eric.cnri.reston.va.us> > 1. Create patches@python.org which will be the official advertised > address to send patches. This will be a Mailman mailing list to > which anybody can post, but to which subscriptions are closed. > Amount of work: trivial. > > 2. Create patches-discuss@python.org which will be the address to > discuss patches@python.org postings. We make Reply-To: for > patches@ be patches-discuss@ > Amount of work: relatively small (for me to hack Mailman) Why do we need two lists? For my own email processing I don't see how it will make a difference. A reply-bot could check whether there's a "Re:" in the subject or not to decide whether to send an acknowledgement. --Guido van Rossum (home page: http://www.python.org/~guido/) From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 3 17:13:49 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 3 Feb 2000 12:13:49 -0500 (EST) Subject: [Python-Dev] patch handling (was: Python 1.6 timing) References: <200002030050.TAA25376@eric.cnri.reston.va.us> <14489.46535.246097.304601@anthem.cnri.reston.va.us> <200002031712.MAA29198@eric.cnri.reston.va.us> Message-ID: <14489.46925.553741.458139@anthem.cnri.reston.va.us> >>>>> "GvR" == Guido van Rossum writes: GvR> Why do we need two lists? For my own email processing I GvR> don't see how it will make a difference. A reply-bot could GvR> check whether there's a "Re:" in the subject or not to decide GvR> whether to send an acknowledgement. Okay, fine with me. From gvwilson@nevex.com Thu Feb 3 18:07:03 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Thu, 3 Feb 2000 13:07:03 -0500 (EST) Subject: [Python-Dev] re: syntax issues / LANL Message-ID: Given current discussions of ternary conditionals, namespaces, etc., would people like me to write up my notes on syntactic/semantic issues that came up during the courses I taught at Los Alamos National Laboratory? I could send 'em directly to interested parties, or post to this list. Greg From fdrake@acm.org Thu Feb 3 18:25:18 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 3 Feb 2000 13:25:18 -0500 (EST) Subject: [Python-Dev] __contains__ hook In-Reply-To: <200002031621.LAA28950@eric.cnri.reston.va.us> References: <200002021849.NAA23092@eric.cnri.reston.va.us> <3898BEBD.93DFBCA7@lemburg.com> <14489.38285.840829.261016@weyr.cnri.reston.va.us> <200002031621.LAA28950@eric.cnri.reston.va.us> Message-ID: <14489.51214.979726.330965@weyr.cnri.reston.va.us> Guido van Rossum writes: > Because if we add a slot to the as_sequence struct, old extensions > that haven't been recompiled will appear to have garbage in that slot > (because they don't actually have it). When we use a spare slot in > the main type struct, that problem doesn't exist; but the as_sequence > struct and friends don't have spares. Good point. I still think a spare slot should be used so sets don't have to look like sequences. > Maybe the docs need to be updated? (Hint, hint.) Done in CVS. > Yuck. The same argument for disallowing 'x in dict' applies to the C > API. There's already PyMapping_HasKey(). Yep! Poorly conceived notions aren't dependent on syntax. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From gmcm@hypernet.com Thu Feb 3 18:33:13 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 3 Feb 2000 13:33:13 -0500 Subject: [Python-Dev] re: syntax issues / LANL In-Reply-To: Message-ID: <1262519719-5685764@hypernet.com> Greg Wilson wrote: > Given current discussions of ternary conditionals, namespaces, etc., would > people like me to write up my notes on syntactic/semantic issues that came > up during the courses I taught at Los Alamos National Laboratory? I could > send 'em directly to interested parties, or post to this list. I would say by all means, post them. Please describe the prior experience of your students, though. I noticed that the HS teacher at IPC8 said that case-sensitivity was *not* a problem, and that his students were Linux users. Randy Pausch said it was a serious problem, but I think it's a safe guess most of his students had only Windows experience. So if your students were mostly Fortran programmers, that should help keep their needs in perspective, (it took me at least 2 years to recover from the trauma of coding Fortran on punch cards). - Gordon From gvwilson@nevex.com Thu Feb 3 18:36:40 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Thu, 3 Feb 2000 13:36:40 -0500 (EST) Subject: [Python-Dev] re: syntax issues / LANL In-Reply-To: <1262519719-5685764@hypernet.com> Message-ID: > I would say by all means, post them. Please describe the prior > experience of your students, though. I noticed that the HS teacher at > IPC8 said that case-sensitivity was *not* a problem, and that his > students were Linux users. Randy Pausch said it was a serious problem, > but I think it's a safe guess most of his students had only Windows > experience. So if your students were mostly Fortran programmers, that > should help keep their needs in perspective, (it took me at least 2 > years to recover from the trauma of coding Fortran on punch cards). I don't know if it's a Windows/Unix thing; however, I do think that anyone (of any age) who's using Linux has long since been assimilated into the case-sensitive world... Luckily, it didn't come up in the feedback... :-) Greg From fdrake@acm.org Thu Feb 3 18:41:47 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 3 Feb 2000 13:41:47 -0500 (EST) Subject: [Python-Dev] __contains__ hook In-Reply-To: References: <200002031621.LAA28950@eric.cnri.reston.va.us> Message-ID: <14489.52203.701872.195189@weyr.cnri.reston.va.us> Moshe Zadka writes: > I totally agree with Guido -- for me, the whole point of this hack is > to avoid people asking for 'in' in dicts: this way we can code a class That's not a good enough reason to add it. > I'm not quite sure where we want to put the C API version of __contains__ > - I'd add a tp_as_set, but the only method seems to be 'in', so it seems > like a waste of valuable real-estate before we are driven into > non-backwards-compatability. I think I should at least ask permission from > the owner before I move over there, trampling everything in my way I suspect there will be fairly few set implementations in C; there will be something like a dictionary (kjSet might be updated, for instance), but that's probably about it. The "in"/"not in" operation can work off the contains slot, and I expect set union would be expressed as +, which is already in the as_number structure. Everything else should probably be implemented as a method or a function rather than as an operator overload. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From guido@python.org Thu Feb 3 18:52:52 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 03 Feb 2000 13:52:52 -0500 Subject: [Python-Dev] re: syntax issues / LANL In-Reply-To: Your message of "Thu, 03 Feb 2000 13:07:03 EST." References: Message-ID: <200002031852.NAA29422@eric.cnri.reston.va.us> > Given current discussions of ternary conditionals, namespaces, etc., would > people like me to write up my notes on syntactic/semantic issues that came > up during the courses I taught at Los Alamos National Laboratory? I could > send 'em directly to interested parties, or post to this list. Sure -- although this almost might make more sense in the edu-sig. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Feb 3 18:54:13 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 03 Feb 2000 13:54:13 -0500 Subject: [Python-Dev] __contains__ hook In-Reply-To: Your message of "Thu, 03 Feb 2000 13:25:18 EST." <14489.51214.979726.330965@weyr.cnri.reston.va.us> References: <200002021849.NAA23092@eric.cnri.reston.va.us> <3898BEBD.93DFBCA7@lemburg.com> <14489.38285.840829.261016@weyr.cnri.reston.va.us> <200002031621.LAA28950@eric.cnri.reston.va.us> <14489.51214.979726.330965@weyr.cnri.reston.va.us> Message-ID: <200002031854.NAA29439@eric.cnri.reston.va.us> > > Because if we add a slot to the as_sequence struct, old extensions > > that haven't been recompiled will appear to have garbage in that slot > > (because they don't actually have it). When we use a spare slot in > > the main type struct, that problem doesn't exist; but the as_sequence > > struct and friends don't have spares. > > Good point. I still think a spare slot should be used so sets don't > have to look like sequences. But they won't have to -- all the other pointers in the as_sequence struct can be NULL. (This used to be not the case, but I've finally given in and added NULL tests everywhere -- it was a recurring complaint from extension writers.) --Guido van Rossum (home page: http://www.python.org/~guido/) From esr@thyrsus.com Fri Feb 4 06:22:28 2000 From: esr@thyrsus.com (esr@thyrsus.com) Date: Thu, 3 Feb 2000 22:22:28 -0800 Subject: [Python-Dev] re: syntax issues / LANL In-Reply-To: ; from gvwilson@nevex.com on Thu, Feb 03, 2000 at 01:07:03PM -0500 References: Message-ID: <20000203222228.A9863@thyrsus.com> gvwilson@nevex.com : > Given current discussions of ternary conditionals, namespaces, etc., would > people like me to write up my notes on syntactic/semantic issues that came > up during the courses I taught at Los Alamos National Laboratory? I could > send 'em directly to interested parties, or post to this list. I would certainly be interested in seeing these. -- Eric S. Raymond The two pillars of `political correctness' are, a) willful ignorance, and b) a steadfast refusal to face the truth -- George MacDonald Fraser From fdrake@acm.org Thu Feb 3 19:19:34 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 3 Feb 2000 14:19:34 -0500 (EST) Subject: [Python-Dev] __contains__ hook In-Reply-To: <200002031854.NAA29439@eric.cnri.reston.va.us> References: <200002021849.NAA23092@eric.cnri.reston.va.us> <3898BEBD.93DFBCA7@lemburg.com> <14489.38285.840829.261016@weyr.cnri.reston.va.us> <200002031621.LAA28950@eric.cnri.reston.va.us> <14489.51214.979726.330965@weyr.cnri.reston.va.us> <200002031854.NAA29439@eric.cnri.reston.va.us> Message-ID: <14489.54470.193434.641817@weyr.cnri.reston.va.us> Guido van Rossum writes: > But they won't have to -- all the other pointers in the as_sequence > struct can be NULL. (This used to be not the case, but I've finally > given in and added NULL tests everywhere -- it was a recurring > complaint from extension writers.) Good enough; sounds like the thing to do is to declare a set to be an sequence that supports sq_contains, sets a flag in tp_flag, and doesn't support the irrelevant slots in the sequence structure. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From dascher@mindspring.com Thu Feb 3 19:39:57 2000 From: dascher@mindspring.com (David Ascher) Date: Thu, 3 Feb 2000 11:39:57 -0800 Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: <14489.46535.246097.304601@anthem.cnri.reston.va.us> Message-ID: <002901bf6e7e$73af5bb0$c355cfc0@ski.org> Barry: > 5. Move the whole kit-and-kaboodle to SourceForge (or > server51.freshmeat.net which looks to provide many of the same > services). We cannot be the only project facing these same issues, > so let's leverage off of what others have done. > Amount of work: unknown, but I'm looking into it for Mailman Given that VA Linux is buying Andover.net (parent of freshmeat.net), I doubt the two will remain as competing solutions. =) http://www.businesswire.com/cgi-bin/f_headline.cgi?day0/200340020&ticker=lnu x|andn --david From guido@python.org Thu Feb 3 19:45:35 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 03 Feb 2000 14:45:35 -0500 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) In-Reply-To: Your message of "Thu, 03 Feb 2000 10:34:54 EST." <3899A01E.DE29FC94@digicool.com> References: <200001200419.XAA01969@mira.erols.com> <38871665.C3B6FFEE@digicool.com> <38871CE5.53FB9D68@digicool.com> <200001201545.KAA21304@eric.cnri.reston.va.us> <38873C52.29FEAC6D@digicool.com> <200001201720.MAA21534@eric.cnri.reston.va.us> <3899A01E.DE29FC94@digicool.com> Message-ID: <200002031945.OAA29682@eric.cnri.reston.va.us> > No. The idea is to have "association" objects. We can create > these directly if we want: > > a=Association('limit',100) > print a.key, a.value # whatever > > The association value is mutable, but the key is not. > > A namespace object is a collection of association objects > such that no two items have the same key. Internally, this > would be very much like the current dictionary except that > instead of an array of dictentries, you'd have an array of > association object pointers. Effectively, associations > are exposed dictentries. > > Externally, a namspace acts more or less like any > mapping object. For example, when someone does a getitem, > the namespace object will find the association with the > desired key and return it's value. In addition, a namspace > object would provide methods along the lines of: > > associations() > > Return a sequence of the associations in the namespace > > addAssociation(assoc) > > Add the given association to the namsspace. This > creates another reference to the association. > Changing the association's value also changes the value > in the namespace. > > getAssociation(key) > > Get the association associated with the key. > > A setitem on a namespace modifies an existing association > if there is already an association for the given key. I presume __setitem__() creates a new association if there isn't one. I also presume that if an association's value is NULL, it doesn't show up in keys(), values() and items() and it doesn't exist for has_key() or __getitem__(). What does a delitem do? Delete the association or set the value to NULL? I suppose the latter. > For example: > > n1=namespace() > n1['limit']=100 > n2=namespace() > n2.addAssociation(n1.getAssociation('limit')) > print n2['limit'] # prints 100 > n1['limit']=200 > print n2['limit'] # prints 200 > > When a function is compiled that refers to a global > variable, we get the association from the global namespace > and store it. The function doesn't need to store the global > namespace itself, so we don't create a circular reference. For this to work we would have to have to change the division of labor between the function object and the code object. The code object is immutable and contains no references to mutable objects; this means that it can easily be marshalled and unmarshalled. (Also, when a code object is compiled or unmarshalled, the globals in which its function will be defined may not exist yet.) The function object currently contains a pointer to the code object and a pointer to the dictionary with the globals. (It also contains the default arg values.) It seems that for associations to work, they need to be placed in the function object, and the code object somehow needs to reference them through the function object. To make this concrete: if a function references globals a, b, and c, these need to be numbered, and the bytecodes should look like this: LOAD_GLOBAL 0 # a STORE_GLOBAL 1 # b DEL_GLOBAL 2 # c (This could be compiled from ``b = a; del c''.) The code object should also contains a list of global names, ordered by their ordinals, e.g. ("a", "b", "c"). Then when the function object is created, it looks in that list and creates a corresponding list of associations, e.g.: L = [] for name in code.co_global_names: L.append(globals.getAssociation(name)) The VM then sticks a pointer to this list into the frame, whenever the function is called (instead of the globals dict which it sticks there now), and the LOAD/STORE/DEL_GLOBAL opcodes reference the associations through this list. Some complications left as exercises: - The built-in functions (and exceptions, etc.) should also be referenced via associations; the loop above would become a bit trickier since it needs to look in two dicts. (We're assuming that the code generator doesn't know which names are globals and which are built-ins.) - If the association for a name doesn't yet exist, it should be created. Note that the semantics are slightly different than currently: the decision whether a name refers to a global or to a built-in is made when the function is defined rather than each time when the name is referenced. This is a bit cleaner -- in the type-sig we're making similar assumptions but the decision is made even earlier. But, overall the necessary changes to the implementation and to the semantics (e.g. of the 'for' statement) seem prohibitive to me. I also think that the namespace implementation will be quite a bit less efficient than a regular dictionary: currently, a dictionary entry is a struct of 12 bytes, and the dictionary has an array of these tightly packed. Your association objects will be "real" objects, which means they have a reference count, a type pointer, a key, and a value, i.e. 16 bytes, without counting the malloc overhead; this probably comes in addition to the 12 bytes in the dict entry. (If you want to have the association objects directly in the hash table, they can't be shared between namespaces, and a namespace couldn't grow -- when a dict grows its hash table is reallocated.) > Note that circular references are bad even if we have > a more powerful gc. I don't understand or believe this statement. > For example, by not storing the global > namespace in a function, we don't have to worry about the > global namespace being blown away before a destructor is run > during process exit. If we had more powerful gc the global namespace wouldn't have to be blown away at all (it would gently dissolve when __main__ was deleted from the interpreter). > When we use the global variable > in the function, we simply get the current value from the > association. We don't have to look it up. > > Namespaces would have other benefits: > > - improve the semantics of: > > from spam import foo > > in that you'd be importing a name binding, not a value But its semantics will be harder to explain, because they will no longer be equivalent to import spam # assume there's no spam already foo = spam.foo del spam Also, we currently *explain* that only objects are shared and name bindings are unique per namespace; this would no longer be true so we would have to explain a much harder rule. ("If you got your foo through an import from another module, assigning to it will affect foo in that other module too; but if you got it through a local assignment, the effect will be local.") All in all, I think these semantics are messy and unacceptable. True, object sharing is hard to explain too (see diagram on Larning Python page 60), but you'll still have to explain that anyway because it still exists within a namespace; but now in addition we'd have to explain that there is an exception to object sharing... Messy, messy. > - Be useful in any application where it's desireable to > share a name binding. I think it's better to explicitly share the namespace -- "foo.bar = 1" makes it clear that whoever else has a reference to foo will see bar similarly changed. > > > Again, it would also make function global variable access > > > faster and cleaner in some ways. > > > > But I have other plans for that (if the optional static typing stuff > > ever gets implemented). > > Well, OK, but I argue that the namespace idea is much simpler > and more foolproof. I claim that it's not foolproof at all -- on the contrary, it creates something that hides in the dark and will bite us in the behind by surprise, long after we thought we knew there were no monsters under the bed. (Yes, I've been re-reading Calvin and Hobbes. :-) > > > > however it would break a considerable amount of old code, > > > > I think. > > > > > > Really? I wonder. I bet it would break alot less old > > > code that other recent changes. > > > > Oh? Name some changes that broke a lot of code? > > The move to class-based exceptions broke alot of our code. It must have been very traumatic that you're still sore over that; it was introduced in 1.5, over two years ago. > Maybe we can drop this point. Do you still think > that the namespace idea would break alot of code? Yes. --Guido van Rossum (home page: http://www.python.org/~guido/) From gvwilson@nevex.com Thu Feb 3 20:00:04 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Thu, 3 Feb 2000 15:00:04 -0500 (EST) Subject: [Python-Dev] re: syntax feedback from LANL course (very long) Message-ID: I taught Python at Los Alamos National Laboratory for the second time two weeks ago, and finally found time last night to go through both sets of feedback forms. Given the current discussion on python-dev of ternary conditionals, namespaces, etc., I figured syntax issues would be most topical. Most of the students were in their 20's or 30's, with a handful in their 40's and 50's. When asked what they used on a daily basis, they split pretty evenly between Fortran and C++ (no C-but-not-C++ responses). They also divided pretty evenly between "numerical and/or visualization" and "systems programming". (At the lab, the latter means everything from making the web archive of past experimental results searchable, through to porting HDF-5 to massively-parallel machines and hacking the Linux kernel to make inter-process communication run faster.) 28 students had Ph.D.'s (all in math, physical science, or engineering); nine had M.Sc.'s (one in C.S.), and the three B.Sc.'s were all in C.S. So here goes... ------------------------------------------------------------------------- 0. "Hey, using indentation to show nesting is cool!" 'Nuff said --- thanks, Guido. ------------------------------------------------------------------------- 1. "What's with 'while 1: break' for normal loops?" This was the most common complaint about syntax --- neither Fortran nor C programmers found it natural to create an infinite loop, then jump out of it. (Several of the C++ programmers said that house style in their groups only allows 'break' for truly exceptional cases.) One student suggested the following loop-and-a-half notation: do: first-half while condition: second-half which has the nice property that both of the degenerate forms: do: first-half while condition # no trailing ':' and while condition: second-half are useful in their own right. Tim Peters tells me that this has been discussed many times before, and that the main obstacle is the introduction of a new keyword. He mentioned: while: first-half and while condition-1: second-half and while condition-2: third-half # [sic] and while final-condition # no trailing ':' as an interesting (but possibly confusing) generalization. ------------------------------------------------------------------------- 2. "Using range() in for loops is clumsy." My audience does a lot of 'for' loops over numerical bounds, and would prefer something slice-ish like: for i in 0:10:2 : body I think they could live with: for i in [0:10:2] : body I believe Brian Harvey mentions somewhere in "Computer Science Logo Style" (the best programming books for kids I've ever seen) that it was important to make basic counting loops very, very simple to write. It's a different audience (10-year-olds instead of Ph.D.'s with 30 years of Fortran behind them), but maybe worth examining. (While we're on the topic: is there an easy way to construct a slice like 3:10, pass it around as a variable, and later use it to index an arbitrary sequence?) ------------------------------------------------------------------------- 3. "Where is '+=' ?" and "Why can't I redefine assignment?" To my surprise, no-one asked why assignment wasn't an operator. However, a lot of people wanted in-place modification --- it's one of the innovations of C and derived languages that scientific programmers appreciate, since it saves them having to write 'pressure_gradient[(ix+1)*3+idx][iy-1-(idx%2)]' on both sides of an assignment, and then remember to update both occurrences when they notice the typo. (The '%' should be a '/'.) Several of the students who ticked the box "Familiar with object-oriented programming" asked why they could redefine addition, membership, etc., but not straight assignment. Guido explained the reasoning to me at IPC8, and pointed out that classes can always define a '.set(...)' method. However, I'm pretty sure that 'set()' will be a non-starter with this crowd (I can explain why if people care). ------------------------------------------------------------------------- 3.1 Min and Max On a related note, I also had requests for redefining "min" and "max" (e.g. to handle objects representing bounding volumes in 3D graphics). As a certain kind of mathematician will point out, these are really binary operators, just like "+", and it's only an historical accident that they aren't treated with the same respect (I'm quoting here). I mentioned this to Guido, and suggested the C* notation: a = x ? y # max He said (rightly) that '?' won't be familiar even to experienced programmers, but that user-definable 'min' and 'max' might be do-able. The only issue then is that if '+=' makes it into the language, something like 'min=' ought to as well... ------------------------------------------------------------------------- 3.2 Case Statement, and Remembering Matches in Conditionals Another related point came up in discussion of regular expressions. The code I showed them was: pat1 = re.compile(...) pat2 = re.compile(...) ...etc... while 1: line = ...get input... m = pat1.match(...) if m: code continue m = pat2.match(...) if m: code continue ...etc... Several students said that it wasn't at all clear that the various matches were intended to be mutually exclusive; one student became very frustrated trying to debug this code after accidentally deleting a 'continue'. If there's a cleaner way to structure this, I'd be grateful for an example. One student (a physicist who now does computer graphics) sent me: if x is: expr1, expr2: code using x (which is either val1 or val2) expr3: code using x (which is guaranteed to be val3) else: code using x (which is something else) which would make the above: while 1: line = ...get input... if m is: pat1.match(...): code using m pat2.match(...): code using m etc. (I'm not advocating, I'm just reporting...) ------------------------------------------------------------------------- 4. "Aren't tuples redundant?" I explained that the const-ness of tuples was needed so that they could be used as dictionary keys. The guy with Perl in his background immediately asked if that's reliable --- a tuple can contain a list, which can be mutated. You've all heard this one more times than I have... ------------------------------------------------------------------------- 5. "Why can't I put an 'except' after an 'if'?" This one surprised me. Several students wanted to be able to say: if (dict['fred'] > 0): code except KeyError: other-code or: for i in seq: i = i ** 2 except ArithmeticError: error-handling-code or even (from one guy with Perl in his background): dict['fred'] = dict['fred'] + 1 except KeyError: dict['fred'] = 1 They pointed out that 'try' does nothing except introduce a block, so why not just use the blocks that other keywords introduce? I'd be interested in knowing whether other people's students think like this... :-) ------------------------------------------------------------------------- 6. "There is no number six." That's really all that came up about Python syntax. None of the students were OO expects, so meta-classes and the like weren't raised. Generic programming did come up once, but the guy who asked hadn't clued in that typing is dynamic. The same guy also asked why he couldn't interleave anonymous and named function arguments, as in "def foo(a, b=3, c, d=5)" also came up, but everybody else who expressed an opinion said they'd find that confusing. Hope it's useful, Greg From dascher@mindspring.com Thu Feb 3 20:14:26 2000 From: dascher@mindspring.com (David Ascher) Date: Thu, 3 Feb 2000 12:14:26 -0800 Subject: [Python-Dev] RE: slice objects In-Reply-To: Message-ID: <002e01bf6e83$45270720$c355cfc0@ski.org> > (While we're on the topic: is there an easy way to construct a slice > like 3:10, pass it around as a variable, and later use it to index an > arbitrary sequence?) Sort of. Slice objects can be created with the slice builtin. slice(1,10,2) is meant to be equivalent to 1:10:2, but only NumPy knows how to deal with slice objects in the getitem slot (use None to express 'omission' of an index). It's on the TODO list for 1.7 at this point, although if someone submits a working patch to 1.6, who knows... JPython does it right, I believe. --david From jim@digicool.com Thu Feb 3 20:21:45 2000 From: jim@digicool.com (Jim Fulton) Date: Thu, 03 Feb 2000 15:21:45 -0500 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) References: <200001200419.XAA01969@mira.erols.com> <38871665.C3B6FFEE@digicool.com> <38871CE5.53FB9D68@digicool.com> <200001201545.KAA21304@eric.cnri.reston.va.us> <38873C52.29FEAC6D@digicool.com> <200001201720.MAA21534@eric.cnri.reston.va.us> <3899A01E.DE29FC94@digicool.com> <200002031945.OAA29682@eric.cnri.reston.va.us> Message-ID: <3899E359.4ED37D66@digicool.com> Guido van Rossum wrote: > > > No. The idea is to have "association" objects. We can create > > these directly if we want: > > > > a=Association('limit',100) > > print a.key, a.value # whatever > > > > The association value is mutable, but the key is not. > > > > A namespace object is a collection of association objects > > such that no two items have the same key. Internally, this > > would be very much like the current dictionary except that > > instead of an array of dictentries, you'd have an array of > > association object pointers. Effectively, associations > > are exposed dictentries. > > > > Externally, a namspace acts more or less like any > > mapping object. For example, when someone does a getitem, > > the namespace object will find the association with the > > desired key and return it's value. In addition, a namspace > > object would provide methods along the lines of: > > > > associations() > > > > Return a sequence of the associations in the namespace > > > > addAssociation(assoc) > > > > Add the given association to the namsspace. This > > creates another reference to the association. > > Changing the association's value also changes the value > > in the namespace. > > > > getAssociation(key) > > > > Get the association associated with the key. > > > > A setitem on a namespace modifies an existing association > > if there is already an association for the given key. > > I presume __setitem__() creates a new association if there isn't one. Yes. > I also presume that if an association's value is NULL, it doesn't show > up in keys(), values() and items() and it doesn't exist for has_key() > or __getitem__(). Right. > What does a delitem do? Delete the association or set the value to > NULL? I suppose the latter. Good question. I'm inclined to think the former. That is, deleting an item from the namespace would delete the name association. I can see arguments both ways. > > For example: > > > > n1=namespace() > > n1['limit']=100 > > n2=namespace() > > n2.addAssociation(n1.getAssociation('limit')) > > print n2['limit'] # prints 100 > > n1['limit']=200 > > print n2['limit'] # prints 200 > > > > When a function is compiled that refers to a global > > variable, we get the association from the global namespace > > and store it. The function doesn't need to store the global > > namespace itself, so we don't create a circular reference. > > For this to work we would have to have to change the division of labor > between the function object and the code object. The code object is > immutable and contains no references to mutable objects; this means > that it can easily be marshalled and unmarshalled. (Also, when a code > object is compiled or unmarshalled, the globals in which its function > will be defined may not exist yet.) The function object currently > contains a pointer to the code object and a pointer to the dictionary > with the globals. (It also contains the default arg values.) > > It seems that for associations to work, they need to be placed in the > function object, and the code object somehow needs to reference them > through the function object. To make this concrete: if a function > references globals a, b, and c, these need to be numbered, and the > bytecodes should look like this: > > LOAD_GLOBAL 0 # a > STORE_GLOBAL 1 # b > DEL_GLOBAL 2 # c > > (This could be compiled from ``b = a; del c''.) > > The code object should also contains a list of global names, ordered > by their ordinals, e.g. ("a", "b", "c"). > > Then when the function object is created, it looks in that list and > creates a corresponding list of associations, e.g.: > > L = [] > for name in code.co_global_names: > L.append(globals.getAssociation(name)) > > The VM then sticks a pointer to this list into the frame, whenever the > function is called (instead of the globals dict which it sticks there > now), and the LOAD/STORE/DEL_GLOBAL opcodes reference the associations > through this list. Looks good to me. :) > Some complications left as exercises: > > - The built-in functions (and exceptions, etc.) should also be > referenced via associations; the loop above would become a bit > trickier since it needs to look in two dicts. (We're assuming that > the code generator doesn't know which names are globals and which are > built-ins.) > > - If the association for a name doesn't yet exist, it should be > created. Yup. > Note that the semantics are slightly different than currently: the > decision whether a name refers to a global or to a built-in is made > when the function is defined rather than each time when the name is > referenced. This is a bit cleaner -- in the type-sig we're making > similar assumptions but the decision is made even earlier. > > But, overall the necessary changes to the implementation and to the > semantics (e.g. of the 'for' statement) seem prohibitive to me. Really? Even for Py3K? > I also think that the namespace implementation will be quite a bit > less efficient than a regular dictionary: Spacewise yes. They'd me much faster in use. This is a space/speed tradeoff. > currently, a dictionary > entry is a struct of 12 bytes, and the dictionary has an array of > these tightly packed. Your association objects will be "real" > objects, which means they have a reference count, a type pointer, a > key, and a value, i.e. 16 bytes, without counting the malloc overhead; > this probably comes in addition to the 12 bytes in the dict entry. Why not replace the key and value pointers with the association pointer. Then you'd get back a little of the space. > (If you want to have the association objects directly in the hash > table, they can't be shared between namespaces, and a namespace > couldn't grow -- when a dict grows its hash table is reallocated.) > > > Note that circular references are bad even if we have > > a more powerful gc. > > I don't understand or believe this statement. This was discussed at length a year or two ago. You added code to print to stderr when an error occured in a destructor. People noticed that they were getting errors when Python exited. The problem occured when a destructor was called after it's globals had been deallocated. You subsequently added alot of extra rules on shutdown to make this much less likely. I don't think you made the problem go away completely. I find circular references to be bad in other ways. For example, they are a pain with deep copy. You can make deep copy do something in the presense of circular references, but the things it does can be quite surprising. > > For example, by not storing the global > > namespace in a function, we don't have to worry about the > > global namespace being blown away before a destructor is run > > during process exit. > > If we had more powerful gc the global namespace wouldn't have to be > blown away at all (it would gently dissolve when __main__ was deleted > from the interpreter). Uh, OK, then we wouldn't have to worry about the global namespace being gently dissolved before a destructor is run during process exit. > > When we use the global variable > > in the function, we simply get the current value from the > > association. We don't have to look it up. > > > > Namespaces would have other benefits: > > > > - improve the semantics of: > > > > from spam import foo > > > > in that you'd be importing a name binding, not a value > > But its semantics will be harder to explain, because they will no > longer be equivalent to > > import spam # assume there's no spam already > foo = spam.foo > del spam Will they really be harder to explain? Why not explain them a different way? "The statement: from spam import foo copies a name binding for foo from module spam to the current module." Eh, I guess I can see why someone would find this harder.... > Also, we currently *explain* that only objects are shared and name > bindings are unique per namespace; this would no longer be true so we > would have to explain a much harder rule. ("If you got your foo > through an import from another module, assigning to it will affect foo > in that other module too; but if you got it through a local > assignment, the effect will be local.") Good point. Perhaps assinging in the client module should break the connection to the other module. This would require some extra magic. > All in all, I think these semantics are messy and unacceptable. True, > object sharing is hard to explain too (see diagram on Larning Python > page 60), but you'll still have to explain that anyway because it > still exists within a namespace; but now in addition we'd have to > explain that there is an exception to object sharing... Messy, messy. Well, I don't have a problem with object sharing, so the notion of sharing namespaces doesn't bother me. I undertand that some folks have a problem with object sharing and I agree that they'd have problems with name sharing. OTOH, I don't think you'd consider the fact that some people have difficulty with object sharing to be sufficient justification for removing the feature from the language. > > - Be useful in any application where it's desireable to > > share a name binding. > > I think it's better to explicitly share the namespace -- "foo.bar = 1" > makes it clear that whoever else has a reference to foo will see bar > similarly changed. > > > > > Again, it would also make function global variable access > > > > faster and cleaner in some ways. > > > > > > But I have other plans for that (if the optional static typing stuff > > > ever gets implemented). > > > > Well, OK, but I argue that the namespace idea is much simpler > > and more foolproof. > > I claim that it's not foolproof at all -- on the contrary, it creates > something that hides in the dark and will bite us in the behind by > surprise, How so? > long after we thought we knew there were no monsters under > the bed. (Yes, I've been re-reading Calvin and Hobbes. :-) > > > > > > however it would break a considerable amount of old code, > > > > > I think. > > > > > > > > Really? I wonder. I bet it would break alot less old > > > > code that other recent changes. > > > > > > Oh? Name some changes that broke a lot of code? > > > > The move to class-based exceptions broke alot of our code. > > It must have been very traumatic that you're still sore over that; > it was introduced in 1.5, over two years ago. I'm not sore. But it was a bigger (IMO) backward incompatibility. Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (888) 344-4332 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From gmcm@hypernet.com Thu Feb 3 20:42:25 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 3 Feb 2000 15:42:25 -0500 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: Message-ID: <1262511971-6152029@hypernet.com> [Greg Wilson] > 4. "Aren't tuples redundant?" > > I explained that the const-ness of tuples was needed so that they > could be used as dictionary keys. The guy with Perl in his background > immediately asked if that's reliable --- a tuple can contain a list, > which can be mutated. You've all heard this one more times than I > have... I guess you didn't try it: >>> t = ([0,1],2) >>> d = {} >>> d[t] = "hooey" Traceback (innermost last): File "", line 1, in ? TypeError: unhashable type >>> To pass the hash test, it needs to have immutable contents as well. To my mind, the "can act as dictionary key" argument is a red herring. Tuples and lists may technically be almost the same thing, but I find I hardly ever need to ponder whether I should use a tuple or a list. Tuples are structures and lists are containers. Yes, you can use a container as a structure. You can also use a bucket as a spoon. or-stick-your-head-straight-into-the-trough-ly y'rs - Gordon From dascher@mindspring.com Thu Feb 3 21:01:48 2000 From: dascher@mindspring.com (David Ascher) Date: Thu, 3 Feb 2000 13:01:48 -0800 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: <1262511971-6152029@hypernet.com> Message-ID: <003701bf6e89$e2a7b5c0$c355cfc0@ski.org> GmCm > [Greg Wilson] > > > 4. "Aren't tuples redundant?" > To my mind, the "can act as dictionary key" argument is a red > herring. Tuples and lists may technically be almost the same > thing, but I find I hardly ever need to ponder whether I should > use a tuple or a list. Tuples are structures and lists are > containers. They're not structures the way a C programmer thinks of a struct or a pascal programmer thinks of a record. Do you have a better way of justifying their existence to novices who haven't reached the zen? It's a nontrivial pedagogical problem in my experience as well. (yes, this is a better topic for edu-sig than for python-dev). --david From guido@python.org Thu Feb 3 21:07:12 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 03 Feb 2000 16:07:12 -0500 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) In-Reply-To: Your message of "Thu, 03 Feb 2000 15:21:45 EST." <3899E359.4ED37D66@digicool.com> References: <200001200419.XAA01969@mira.erols.com> <38871665.C3B6FFEE@digicool.com> <38871CE5.53FB9D68@digicool.com> <200001201545.KAA21304@eric.cnri.reston.va.us> <38873C52.29FEAC6D@digicool.com> <200001201720.MAA21534@eric.cnri.reston.va.us> <3899A01E.DE29FC94@digicool.com> <200002031945.OAA29682@eric.cnri.reston.va.us> <3899E359.4ED37D66@digicool.com> Message-ID: <200002032107.QAA00051@eric.cnri.reston.va.us> > > But, overall the necessary changes to the implementation and to the > > semantics (e.g. of the 'for' statement) seem prohibitive to me. ^^^ (I meant 'from' of course) > Really? Even for Py3K? The implementation wouldn't be a problem for Py3K; I was under the impression that you thought this could be put in earlier. But the change in semantics is very hard to swallow to me. It definitely seems to be come murkier. > > I also think that the namespace implementation will be quite a bit > > less efficient than a regular dictionary: > > Spacewise yes. They'd me much faster in use. This is a space/speed > tradeoff. Agreed; though the speedup comes from circumventing the dictionary altogether. > Why not replace the key and value pointers with the association pointer. > Then you'd get back a little of the space. Yes; assuming that a speedy getitem is not an issue, the hash table could become an array of pointers to associations. > Will they really be harder to explain? Why not explain them > a different way? > > "The statement: > > from spam import foo > > copies a name binding for foo from module spam to the current > module." > > Eh, I guess I can see why someone would find this > harder.... Yes -- the concept of a name binding as an object in itself is hard; and it's hard to understand why it is needed. > Good point. Perhaps assinging in the client module > should break the connection to the other module. This would > require some extra magic. More murkiness. > > All in all, I think these semantics are messy and unacceptable. True, > > object sharing is hard to explain too (see diagram on Larning Python > > page 60), but you'll still have to explain that anyway because it > > still exists within a namespace; but now in addition we'd have to > > explain that there is an exception to object sharing... Messy, messy. > > Well, I don't have a problem with object sharing, so the notion > of sharing namespaces doesn't bother me. I undertand that > some folks have a problem with object sharing and I agree > that they'd have problems with name sharing. OTOH, I don't > think you'd consider the fact that some people have difficulty > with object sharing to be sufficient justification for removing > the feature from the language. Object sharing is something you have to learn very early on; something like "objects drop under gravity". Name binding sharing is something that can effectively be skirted initially, but at some later point it bites you (sort of like mutable default arguments do); this is more comparable to discovering Einstein's relativity. > > I claim that it's not foolproof at all -- on the contrary, it creates > > something that hides in the dark and will bite us in the behind by > > surprise, > > How so? Because the tendency of tutorials will be to avoid mentioning namespaces at all until you get to the appendix at the end titled "Implementation Details." > I'm not sore. But it was a bigger (IMO) backward incompatibility. Sometimes a bigger incompatibility that is easy to explain is more acceptable than a very subtle one that breaks code in very subtle ways. Anyway, let's drop this comparison; you can't objectively measure how backwards incompatible something us. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Feb 3 21:20:29 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 03 Feb 2000 16:20:29 -0500 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: Your message of "Thu, 03 Feb 2000 13:01:48 PST." <003701bf6e89$e2a7b5c0$c355cfc0@ski.org> References: <003701bf6e89$e2a7b5c0$c355cfc0@ski.org> Message-ID: <200002032120.QAA00092@eric.cnri.reston.va.us> > > > 4. "Aren't tuples redundant?" > > > To my mind, the "can act as dictionary key" argument is a red > > herring. Tuples and lists may technically be almost the same > > thing, but I find I hardly ever need to ponder whether I should > > use a tuple or a list. Tuples are structures and lists are > > containers. > > They're not structures the way a C programmer thinks of a struct or a pascal > programmer thinks of a record. That doesn't matter; I'd like to assume that we want to teach students who don't already have a preconceived notion of a Pascal record or C struct. > Do you have a better way of justifying their existence to novices who > haven't reached the zen? It's a nontrivial pedagogical problem in my > experience as well. (yes, this is a better topic for edu-sig than for > python-dev). I have a feeling that introducing optional static typing would help here, because it clarifies the concept. For example, these two have the same type, [int]: (the length is part of the value) [3, 100] [5, 6, 7] On the other hand, these two have different types: (3, 100) # (int, int) (5, 6, 7) # (int, int, int) If we start mixing types, we may get [3, "wow"] # [any]; or perhaps [int|str] (3, "wow") # (int, str) We can then explain that we might have informal names for the parts, and we can use these as local variable names. This is borrowed from ABC, which did at least *some* user testing on issues like this: count, value = t t = count, value (ABC used a different assignment syntax; it would be: PUT t IN count, value PUT count, value IN t I am still friends with the librarian who was taught ABC as a guinea pig.) However, ABC was a statically typed language in the sense of Haskell: there were no declarations but the types were inferenced from initializations and operators. Maybe that *does* make a difference. Also, its tuples were completely unlike sequences: they didn't have indexing or slicing. There were no tuples of lengths 1 or 0. --Guido van Rossum (home page: http://www.python.org/~guido/) From dascher@mindspring.com Thu Feb 3 21:28:11 2000 From: dascher@mindspring.com (David Ascher) Date: Thu, 3 Feb 2000 13:28:11 -0800 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: <200002032120.QAA00092@eric.cnri.reston.va.us> Message-ID: <003801bf6e8d$92659c40$c355cfc0@ski.org> > > They're not structures the way a C programmer thinks of a > struct or a pascal > > programmer thinks of a record. > > That doesn't matter; I'd like to assume that we want to teach students > who don't already have a preconceived notion of a Pascal record or C > struct. For CP4E, absolutely. I was just reporting on my experience with existing programmers not unlike Greg's (also with many fewer PhD's)[*]. > I have a feeling that introducing optional static typing would help > here, because it clarifies the concept. Very interesting point -- however, I think that we need a better answer in general. The question in my experience stems from a desire to minimize memory load. The students grok lists in about 30 seconds. Then I explain tuples, and they wonder why they were "added" given that lists "exist" (in their brain at that time =). --david [*]: I did teach a course at Bank of America where 15 of the 16 people in the room were Vice Presidents. Of course, they were just programmers, but the pay for a competent programmer is high in the hierarchy of bank salaries. =) From bwarsaw@cnri.reston.va.us Thu Feb 3 21:51:27 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Thu, 3 Feb 2000 16:51:27 -0500 (EST) Subject: [Python-Dev] patch handling (was: Python 1.6 timing) References: <14489.46535.246097.304601@anthem.cnri.reston.va.us> <002901bf6e7e$73af5bb0$c355cfc0@ski.org> Message-ID: <14489.63583.351497.24828@anthem.cnri.reston.va.us> >>>>> "DA" == David Ascher writes: DA> Given that VA Linux is buying Andover.net (parent of DA> freshmeat.net), I doubt the two will remain as competing DA> solutions. =) Indeed! From gmcm@hypernet.com Thu Feb 3 21:56:19 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 3 Feb 2000 16:56:19 -0500 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: <003801bf6e8d$92659c40$c355cfc0@ski.org> References: <200002032120.QAA00092@eric.cnri.reston.va.us> Message-ID: <1262507540-6418715@hypernet.com> David Ascher wrote: > [*]: I did teach a course at Bank of America where 15 of the 16 people in > the room were Vice Presidents. Of course, they were just programmers, but > the pay for a competent programmer is high in the hierarchy of bank > salaries. =) From my recollection of the numbers of 10 years ago: if every man, woman and child in my town were a BofA VP, then you could still easily pack all BofA VPs into a dumpster without any two of my fellow residents coming into contact. ow-careful-with-that-cell-phone-ly y'rs - Gordon From mal@lemburg.com Thu Feb 3 18:49:31 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 03 Feb 2000 19:49:31 +0100 Subject: [Python-Dev] __contains__ hook References: <200002021849.NAA23092@eric.cnri.reston.va.us> <3898BEBD.93DFBCA7@lemburg.com> <200002022349.SAA25030@eric.cnri.reston.va.us> Message-ID: <3899CDBB.2A034D5A@lemburg.com> Guido van Rossum wrote: > > > Shouldn't 'in' be a slot of the sequence methods ? I'd suggest > > creating a new tp_flag bit and then extending tp_as_sequence > > with: > > > > binaryfunc sq_contains; > > > > plus of course add an abstract function to abstract.c: > > > > PySequence_Contain(PyObject *container, PyObject *element) > > That function already exists, spelled "PySequence_Contains" (currently > it does the C equivalent of > > for i in container: > if element == i: return 1 > return 0 Hmm, I must have overseen that one... the above only works for sequences, while 'in'ness only need an unordered set to work. Perhaps we do need an abstraction for unordered object containers after all, just like Moshe suggested. I don't think it's top-priority, though... > I'm not entirely sure whether the 'contains' slot should be part of > the as_sequence struct, but I suppose it makes sense historically. > (The as_number, as_sequece, as_mapping structs don't make sense at all > in the grand scheme of things, but we're stuck with them for the time > being.) Doens't really matter where we put it -- the type object is a mess already ;-) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From gvwilson@nevex.com Thu Feb 3 22:52:56 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Thu, 3 Feb 2000 17:52:56 -0500 (EST) Subject: [Python-Dev] Re: syntax - "Aren't tuples redundant?" In-Reply-To: <1262511971-6152029@hypernet.com> Message-ID: > [Greg Wilson] > > > 4. "Aren't tuples redundant?" > > > > I explained that the const-ness of tuples was needed so that they > > could be used as dictionary keys. The guy with Perl in his background > > immediately asked if that's reliable --- a tuple can contain a list, > > which can be mutated. You've all heard this one more times than I > > have... > I guess you didn't try it: I actually had -- the point I was trying to make was that his first reaction was "But what about..." Maybe it's just a reflection of his Perl background :-). Like I said, you've all heard this one more times than I have... Greg From ping@lfw.org Thu Feb 3 23:24:36 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Thu, 3 Feb 2000 17:24:36 -0600 (EST) Subject: [Python-Dev] re: syntax feedback from LANL course (very long) In-Reply-To: Message-ID: On Thu, 3 Feb 2000 gvwilson@nevex.com wrote: > I taught Python at Los Alamos National Laboratory for the second time > two weeks ago, and finally found time last night to go through both > sets of feedback forms. Given the current discussion on python-dev of > ternary conditionals, namespaces, etc., I figured syntax issues would > be most topical. Wow, these were really informative. Thanks for collecting the results. > 1. "What's with 'while 1: break' for normal loops?" I'd probably agree that this is a significant thing. I think it would be very nice to have do...while, and of all your examples i liked do: first-body while condition: second-body best. This very nicely takes care of the common idiom do: line =3D file.readline() while line: body (Once upon a time i even wrote a patch just to handle this case with a strange "while line from file.readline()" syntax. I think the above is much nicer.) > while: > first-half > and while condition-1: > second-half > and while condition-2: > third-half # [sic] > and while final-condition # no trailing ':' This *is* confusing. Does the "third-half" get executed if condition-2 is true, or only if condition-1 and condition-2 are both true? Hmmm... i suppose if i interpret each "and while " as "if not : break" it makes more sense. But i still like the first. "do" doesn't even have to be made into a keyword for the do...while construct to work: there is no other valid construct in which a name would appear followed by a colon as the first thing in a statement or expression, so the parser should be able to figure it out. > 2. "Using range() in for loops is clumsy." >=20 > My audience does a lot of 'for' loops over numerical bounds, and would > prefer something slice-ish like: >=20 > for i in 0:10:2 : > body >=20 > I think they could live with: >=20 > for i in [0:10:2] : > body Wow, i like that. There is a nice symmetry to >>> t =3D (3, 7, 5, 6, 2, 1, 8) >>> print t[0:5] (3, 7, 5, 6, 2) >>> for i in [0:5]: print t[i] ... 3 7 5 6 2 > I believe Brian Harvey mentions somewhere in "Computer Science Logo > Style" (the best programming books for kids I've ever seen) that it > was important to make basic counting loops very, very simple to write. I'd tend to agree with that. This is definitely one of the first things any beginner will see, and it's nice not to be sidetracked into an explanation of "range". The symmetry means that, after they've seen "for i in [0:10]:", they'll already have a feel for what "list[0:10]" ought to do. Oh, and having a slice object work in a "for" would allow us to drop that vaguely irritating range/xrange distinction. That said, however, i don't think i'd make a really big deal out of this. Maybe i'm being too swayed by the cuteness of the idea. > (While we're on the topic: is there an easy way to construct a slice > like 3:10, pass it around as a variable, and later use it to index an > arbitrary sequence?) I was wondering why this doesn't work. Given that slices are built in, the lack of discussion of them in the documentation makes them rather mysterious. It would be nice to make them just do the expected thing. You could then do things like putting list[::-1] in an expression without having to copy it to a temporary, call reverse(), and then use the temporary. > ------------------------------------------------------------------------- >=20 > 3. "Where is '+=3D' ?" and "Why can't I redefine assignment?" +=3D would be nice; not a really big deal to me. Redefining assignment would get way too confusing, i think. __setitem__ has always been good enough for me. > ------------------------------------------------------------------------- >=20 > 3.1 Min and Max Interesting, but doesn't matter much to me. > ------------------------------------------------------------------------- >=20 > 3.2 Case Statement, and Remembering Matches in Conditionals >=20 [...] > One student (a physicist who now does computer graphics) sent me: >=20 > if x is: > expr1, expr2: > code using x (which is either val1 or val2) > expr3: > code using x (which is guaranteed to be val3) > else: > code using x (which is something else) I like this quite a lot! One question: in that last "else" clause, wouldn't "x" be undefined? > ------------------------------------------------------------------------- >=20 > 5. "Why can't I put an 'except' after an 'if'?" Possibly interesting, but it doesn't seem necessary to me. Having the "try:" at the beginning of the block more clearly delimits what's being protected. Wouldn't this look weird? if foo.bar > 0: blah else: blah except AttributeError: blah else: blah Now does the "except" cover just the first "else" or both the bodies of the "if" and the "else"? Alternatively, imagine that there used to be only an "if" and we wanted to insert an "else". It's clear we can't get rid of "try:" altogether, so it seems like more work to remember the specific list of other situations in which "except:" might apply... > ------------------------------------------------------------------------- >=20 > 6. "There is no number six." And six is RIGHT OUT. min-sv=E4vare-=E4r-full-med-=E5lar-ly y'rs, -- ?!ng From Moshe Zadka Thu Feb 3 23:32:05 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 4 Feb 2000 01:32:05 +0200 (IST) Subject: [Python-Dev] patch handling (was: Python 1.6 timing) In-Reply-To: <14489.46535.246097.304601@anthem.cnri.reston.va.us> Message-ID: Let me just note the whole thing sounds great! I'm for it. On Thu, 3 Feb 2000, Barry A. Warsaw wrote: > ... We cannot be the only project facing these same issues, > so let's leverage off of what others have done. As a point to think about, linux-kernel have a very non-high-tech solution that works, but for some reason causes everyone shivers: high volume mailing list, with patches and discussion of patches, and Linus's mailbox, in which patches from known hackers are more streamlined. That's the ultimate in flexibility ;-) I don't know how many people here ever subscribed to linux-kernel, but as someone who compiled a new kernel every week for a few months, I just want to say that it works <0.6 wink> -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From Moshe Zadka Thu Feb 3 23:39:33 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 4 Feb 2000 01:39:33 +0200 (IST) Subject: [Python-Dev] __contains__ hook In-Reply-To: <14489.52203.701872.195189@weyr.cnri.reston.va.us> Message-ID: On Thu, 3 Feb 2000, Fred L. Drake, Jr. wrote: > > I totally agree with Guido -- for me, the whole point of this hack is > > to avoid people asking for 'in' in dicts: this way we can code a class > > That's not a good enough reason to add it. Well, it the metaphorical sense it is -- the reason people were asking for 'in' in dicts were usually because they wanted to use dictionaries as sets. Not having a way to express with 'in' certainly seems like a wart. > > I'm not quite sure where we want to put the C API version of __contains__ > > - I'd add a tp_as_set, but the only method seems to be 'in', so it seems > > like a waste of valuable real-estate before we are driven into > > non-backwards-compatability. I think I should at least ask permission from > > the owner before I move over there, trampling everything in my way > > I suspect there will be fairly few set implementations in C; there > will be something like a dictionary (kjSet might be updated, for > instance), but that's probably about it. > The "in"/"not in" operation can work off the contains slot, and I > expect set union would be expressed as +, which is already in the > as_number structure. Everything else should probably be implemented > as a method or a function rather than as an operator overload. Fred, I'm afraid I didn't understand you /at all/. Can you just say what is it you're offering? There isn't a "contains" slot right now, and what I'm wondering is where to put it. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From gvwilson@nevex.com Thu Feb 3 23:58:43 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Thu, 3 Feb 2000 18:58:43 -0500 (EST) Subject: [Python-Dev] re: LANL syntax feedback (loops) In-Reply-To: Message-ID: > > Greg Wilson wrote: > > while: > > first-half > > and while condition-1: > > second-half > > and while condition-2: > > third-half # [sic] > > and while final-condition # no trailing ':' > Ka-Ping wrote: > This *is* confusing. Does the "third-half" get executed if > condition-2 is true, or only if condition-1 and condition-2 are both > true? The loop breaks out the first time it finds a false condition. I find it confusing as well, and wonder whether the extra generality would ever be used. I'd even want to put: do: first-half while cond: second-half in front of both novices and experts --- is this two loops, or one? > "do" doesn't even have to be made into a keyword for the do...while > construct to work: there is no other valid construct in which a name > would appear followed by a colon as the first thing in a statement or > expression, so the parser should be able to figure it out. Might not be true in future, so probably dangerous to assume now. (Icon, anyone? :-) Thanks, Greg From gvwilson@nevex.com Fri Feb 4 00:02:46 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Thu, 3 Feb 2000 19:02:46 -0500 (EST) Subject: [Python-Dev] re: LANL syntax (assignment) In-Reply-To: Message-ID: > > Greg Wilson wrote: > > 3. "Where is '+=' ?" and "Why can't I redefine assignment?" > Ka-Ping Yee: > += would be nice; not a really big deal to me. > > Redefining assignment would get way too confusing, i think. > __setitem__ has always been good enough for me. It'll make a big difference to NumPy (avoiding temporaries, etc.). Also hard to explain why you can redefine assignment to an array element, but not to a scalar variable. Thanks, Greg From gvwilson@nevex.com Fri Feb 4 00:08:41 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Thu, 3 Feb 2000 19:08:41 -0500 (EST) Subject: [Python-Dev] re: LANL syntax (case/branch) In-Reply-To: Message-ID: > > Greg Wilson wrote: > > One student (a physicist who now does computer graphics) sent me: > > > > if x is: > > expr1, expr2: > > code using x (which is either val1 or val2) > > expr3: > > code using x (which is guaranteed to be val3) > > else: > > code using x (which is something else) > Ka-Ping Yee wrote: > I like this quite a lot! One question: in that last "else" clause, > wouldn't "x" be undefined? (Quick flip through notes): "x is None in the else branch". Looking at it again, it came up as part of the question "Why isn't assignment an operator?" The student in question was used to doing: if (x = foo()) { body } else if (x = bar()) { body } else { body } and wanted to have something in Python that would (a) provide more flexibility than a C/C++ case statement, while (b) making it clear that the alternatives really were mutually exclusive. I think. Or maybe not. He talked really, really fast... Greg From DavidA@ActiveState.com Fri Feb 4 00:11:00 2000 From: DavidA@ActiveState.com (David Ascher) Date: Thu, 3 Feb 2000 16:11:00 -0800 Subject: [Python-Dev] re: LANL syntax (assignment) In-Reply-To: Message-ID: <000a01bf6ea4$513eb820$c355cfc0@ski.org> > > > Greg Wilson wrote: > > > 3. "Where is '+=' ?" and "Why can't I redefine assignment?" > > > Ka-Ping Yee: > > += would be nice; not a really big deal to me. > > > > Redefining assignment would get way too confusing, i think. > > __setitem__ has always been good enough for me. > > It'll make a big difference to NumPy (avoiding temporaries, etc.). Also > hard to explain why you can redefine assignment to an array element, but > not to a scalar variable. FWIW, I never got that question by my students. I did get that question in a consulting context, but it was a very advanced problem having to do with needing control over references to huge external objects, etc. After much trial and error I've come up with a 'method' for describing assignment, references, names, etc., which I've found works well in practice. I demonstrate the concept of names & references on a whiteboard with trivial cases like a = 1 + 3 and then I move on to b = 3 c = 5 a = b + c and then a = [b,c] and then I graduate to the a = [0]*3 and then finally a = [[0]*3]*3 and when they get that I know I've succeeded in getting the concept of assignment in Python across. It's much too hard to describe in email because it requires a blackboard and drawing arrows, erasing arrows, etc., but someday I can demonstrate (SD2000?). Anyway, it's possible that your students are just too smart for their own good, Greg. =) --david From gvwilson@nevex.com Fri Feb 4 00:16:00 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Thu, 3 Feb 2000 19:16:00 -0500 (EST) Subject: [Python-Dev] re: LANL syntax (assignment) In-Reply-To: <000a01bf6ea4$513eb820$c355cfc0@ski.org> Message-ID: > > Greg Wilson wrote: > > It'll make a big difference to NumPy (avoiding temporaries, etc.). Also > > hard to explain why you can redefine assignment to an array element, but > > not to a scalar variable. > David Ascher wrote: > FWIW, I never got that question by my students. I did get that > question in a consulting context, but it was a very advanced problem > having to do with needing control over references to huge external > objects, etc. > > Anyway, it's possible that your students are just too smart for their > own good, Greg. =) I wish I could tell from my notes whether the ones asking to redefine assignment had C++ in their background or not. Setting the consistency issue aside, libraries like NumPy (and the Python futures package I keep meaning to write) will be a lot cleaner from a user's point of view if assignment is redefinable. However, I do understand that there's no easy way to do it without taking an unacceptable performance hit... I'll send anyone who can come up with a clean, workable solution a case of Canadian beer (now how's *that* for a real prize? :-) Greg From ping@lfw.org Fri Feb 4 01:01:54 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Thu, 3 Feb 2000 19:01:54 -0600 (EST) Subject: [Python-Dev] Set data type Message-ID: Okay, so i was going to wait until i had an implementation before bringing this up here, but it seems the other discussion has come round to sets already so this might be a good time. After Eric asked about sets at dinner (e.g. having 'x in dict' do 'dict.has_key(x)') i wondered about what it would take to support sets. Guido's response to 'x in dict' was that it was inconsistent since 'x in list' searches the values of the list, while the proposed 'x in dict' would search the keys. A friend (Mark Miller, designer of E -- see www.erights.org) and i had chatted about this before, and one idea that was suggested was to have the mapping map each member of the set to itself. Then 'in' would give the expected behaviour. E also manages to overload | and & so that they do useful things with mappings-used- as-dictionaries as well as union and intersection for mappings- used-as-sets. I expect, however, that & will be rarely used on mappings-used-as-dictionaries; more common is the desire for |, which Python has solved nicely with dict.update() (also better because it is clearly non-commutative). So i think the clearest thing to do is to make sets a separate built-in type. Here's the interface i was thinking of: >>> s = {1, 5, 7} # no colons means a set >>> type(s) >>> s # sets are printed in hashed order {7, 1, 5} >>> {1: 4, 5, 7} # mixing pairs and singletons is an error File "", line 1 {1: 4, 5, 7} ^ SyntaxError: invalid syntax >>> {3, 4: 5, 7} # mixing pairs and singletons is an error File "", line 1 {3, 4: 5, 7} ^ SyntaxError: invalid syntax >>> 5 in s # membership 1 >>> 6 in s 0 >>> t = s # sets are mutable, so this is an alias >>> t.append(2) # like list.append(), accepts one new member >>> s {7, 5, 2, 1} >>> u = s.copy() # make a copy >>> u.append(9) >>> u.append([3]) # set members must be hashable Traceback (innermost last): File "", line 1, in ? TypeError: unhashable type >>> u {7, 5, 9, 2, 1} >>> u.extend(range(5)) # like list.extend(), accepts a sequence or set >>> u {9, 7, 5, 4, 3, 2, 1, 0} >>> u.remove(7) # like list.remove() >>> s {7, 5, 2, 1} >>> s | {3, 5} # union {7, 5, 3, 2, 1} >>> s & {1, 2, 3} # intersection {2, 1} >>> s <= u # subset 1 >>> s >= s 1 >>> u <= s 0 >>> for i in s: print i # iterate in hashed order 7 5 2 1 >>> l = list(s) >>> l.sort() >>> for i in l: print i # iterate in sorted order 1 2 5 7 >>> s.clear() # for completeness, i suppose >>> s >>> s[3] # probably best not to permit subscripting Traceback (innermost last): File "", line 1, in ? TypeError: unsubscriptable object >>> s[5:7] Traceback (innermost last): File "", line 1, in ? TypeError: unsliceable object In short, sets get append, extend, remove # from lists clear, copy # from dictionaries and the operators &, |, in, <=, >=, <, >, == They could be implemented like dicts with just keys and no values. Two open issues: 1. How to spell the empty set? Possibilities include {,} or {-} or a constant in __builtins__. __builtins__ would probably get a conversion function 'set' (soon to be a type-object/constructor like int, list, etc.), so you could just say 'set()'. 2. How do sets sort? We could insert them somewhere in the hierarchy of types next to lists and tuples. Currently () > [] > {}, so we could have () > [] > {,} > {} for example. More tricky is the question of what to do with the partial ordering created by >, >=, etc. a. The answer is the same as whatever answer we get when Python supports rich comparisons and instances can have partial orderings too. b. We can make >, >= behave like dictionary >, >= so that there is a defined sort order, and put the subset/superset functionality in a method. Option a. would be nicest since there are other good uses for rich comparisons, and in b. we get comparison operators that don't really do anything useful. (Side note: Do we achieve a. just by divorcing __cmp__ from >, >=, etc.? sorting would use __cmp__, while > and < would look for __gt__, __lt__, etc., and if not found, fall back on __cmp__. __cmp__ contracts to be stable [a.__cmp__(b) always has the same outcome for a given a and b], reflexive [if a == b, then a.__cmp__(b) == 0], consistent [if a.__cmp__(c) == 0 and b.__cmp__(d) == 0, then a.__cmp__(b) == c.__cmp__(d)], symmetric [a.__cmp__(b) + b.__cmp__(a) == 0 for all a, b], and transitive [if a.__cmp__(b) > 0 and b.__cmp__(c) > 0, then a.__cmp__(c) > 0; if a.__cmp__(b) == 0 and b.__cmp__(c) == 0, then a.__cmp__(c) == 0], but __gt__, __lt__ need make no such promises. Side side note: which of these promises, if any, should we ask __gt__ et al to make? Stability, at least?) ((Side side side note: in E, Mark Miller also ran into the problem of spelling different kinds of "equals"es. For object identity we have "is", for content comparison we have "==". If we need a new operator for magnitude equality i suggest "<=>", as used in E.)) Well, it may be a lot of words, but it's not much good unless you are going to use it in practice. I think i would have good use for it, but i would like to hear your opinions. Would you use such a thing? -- ?!ng From ping@lfw.org Fri Feb 4 01:27:36 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Thu, 3 Feb 2000 19:27:36 -0600 (EST) Subject: [Python-Dev] Set data type (addendum) In-Reply-To: Message-ID: On Thu, 3 Feb 2000, Ka-Ping Yee wrote: > > In short, sets get > > append, extend, remove # from lists > clear, copy # from dictionaries I picked append() rather than insert() since list.insert() takes two arguments, although you could also argue that "insert" makes more sense since the item being added doesn't "go at the end". > and the operators > > &, |, in, <=, >=, <, >, == Oh! I forgot one! Of course, sets should also support "-": >>> s = {1, 2, 3, 4} >>> t = {2, 4, 5} >>> s - t {3, 1} The "+" operator could do the same as "|". -- ?!ng From gstein@lyra.org Fri Feb 4 01:21:03 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 3 Feb 2000 17:21:03 -0800 (PST) Subject: [Python-Dev] Set data type In-Reply-To: Message-ID: On Thu, 3 Feb 2000, Ka-Ping Yee wrote: >... > So i think the clearest thing to do is to make sets a separate > built-in type. Here's the interface i was thinking of: I might agree with this, but would not recommend special syntax for it. In particular: > >>> s = {1, 5, 7} # no colons means a set Without a lot of hackery or post-parse validation (e.g. at byte-compile time), the above syntax is not possible. I think the follow is just as readable, if not more so: >>> s = set(1, 5, 7) Where set() is a new builtin, taking an arbitrary number of arguments and returning a new Set type. The rest can easily follow once you have a Set object in hand. Cheers, -g -- Greg Stein, http://www.lyra.org/ From DavidA@ActiveState.com Fri Feb 4 00:53:03 2000 From: DavidA@ActiveState.com (David Ascher) Date: Thu, 3 Feb 2000 16:53:03 -0800 Subject: [Python-Dev] Set data type In-Reply-To: Message-ID: <000101bf6eaa$310881c0$c355cfc0@ski.org> Ka-Ping Yee, on sets: I think you forgot one behavior: >>> t.append('Spam!') >>> t.append('Spam!') KeyError: set already contains value 'Spam!' no? FWIW, I don't like the use of the word 'append', which to me implies a serial order. I'd use 'add', but that's a nit. > 2. How do sets sort? [..] > a. The answer is the same as whatever answer we get when > Python supports rich comparisons and instances can have > partial orderings too. [..] > (Side note: Do we achieve a. just by divorcing __cmp__ from > >, >=, etc.? I think we hashed (pun!) all of this out a couple of years ago in one of your hibernation periods =). I'll try to find the deja thread on the topic. > ((Side side side note: in E, Mark Miller also ran into the > problem of spelling different kinds of "equals"es. For > object identity we have "is", for content comparison we > have "==". If we need a new operator for magnitude equality > i suggest "<=>", as used in E.)) Isn't magnitude equality currently spelled len(a) == len(b)? > Well, it may be a lot of words, but it's not much good unless you > are going to use it in practice. I think i would have good use for > it, but i would like to hear your opinions. Would you use such a > thing? I use some of the features of such a thing now by making up dictionaries with None as values for all the keys. I have no strong feeling re: whether we need a new datatype for sets. -david ascher From ping@lfw.org Fri Feb 4 01:55:49 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Thu, 3 Feb 2000 19:55:49 -0600 (EST) Subject: [Python-Dev] Set data type In-Reply-To: <000101bf6eaa$310881c0$c355cfc0@ski.org> Message-ID: On Thu, 3 Feb 2000, David Ascher wrote: > > I think you forgot one behavior: > > >>> t.append('Spam!') > >>> t.append('Spam!') > KeyError: set already contains value 'Spam!' Mmmm... i don't think so. I can't think of a situation in which i would want to get this exception. If i really cared i would ask whether 'Spam!' in t. > no? FWIW, I don't like the use of the word 'append', which to me implies a > serial order. I'd use 'add', but that's a nit. 'append', 'add', 'insert', all okay with me. I initially avoided 'add' due to possible confusion with + and __add__, and 'insert' because list.insert took two arguments. But, as i say, i wouldn't be much bothered by any of these names. > > ((Side side side note: in E, Mark Miller also ran into the > > problem of spelling different kinds of "equals"es. For > > object identity we have "is", for content comparison we > > have "==". If we need a new operator for magnitude equality > > i suggest "<=>", as used in E.)) > > Isn't magnitude equality currently spelled len(a) == len(b)? That's, uh, "size equality" (oh, i don't know, i'll make up the terminology as i go along). That's not the same as "<=>" if you are going to allow arbitrary partial orderings. I suppose it depends how "arbitrary" we are willing to let them get -- a "<=>" operator might turn out to be never needed, though it is truly a third and different kind of equality. a <=> b is equivalent to a <= b and a >= b (...hence the spelling.) -- ?!ng From ping@lfw.org Fri Feb 4 01:57:47 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Thu, 3 Feb 2000 19:57:47 -0600 (EST) Subject: [Python-Dev] Set data type In-Reply-To: Message-ID: On Thu, 3 Feb 2000, Greg Stein wrote: > > I think the follow is just as readable, if not more so: > > >>> s = set(1, 5, 7) > > Where set() is a new builtin, taking an arbitrary number of arguments and > returning a new Set type. Sure. We would certainly need a set() built-in anyway. I just thought the {1, 5, 7} syntax would be quite nice for math-and-science people, as it matches textbooks well. -- ?!ng From Jasbahr@origin.EA.com Fri Feb 4 02:06:49 2000 From: Jasbahr@origin.EA.com (Asbahr, Jason) Date: Thu, 3 Feb 2000 20:06:49 -0600 Subject: [Python-Dev] Set data type Message-ID: <11A17AA2B9EAD111BCEA00A0C9B4179303385EB4@molach.origin.ea.com> >Okay, so i was going to wait until i had an implementation before >bringing this up here, but it seems the other discussion has come >round to sets already so this might be a good time. I like your set proposal, Ping. The lack of a set datatype something that has come up in my organization as new people are exposed to the language, especially those familiar with the STL. It's not a show stopper or anything... Jason Asbahr Origin Systems, Inc. jasbahr@origin.ea.com From ping@lfw.org Fri Feb 4 02:32:28 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Thu, 3 Feb 2000 20:32:28 -0600 (EST) Subject: [Python-Dev] Set data type In-Reply-To: Message-ID: On Thu, 3 Feb 2000, Ka-Ping Yee wrote: > On Thu, 3 Feb 2000, Greg Stein wrote: > > > > I think the follow is just as readable, if not more so: > > > > >>> s = set(1, 5, 7) > > > > Where set() is a new builtin, taking an arbitrary number of arguments and > > returning a new Set type. Oh, waitasec. On further thought, this produces surprising behaviour when compared to list() and tuple(): >>> list((1, 2, 3)) [1, 2, 3] >>> list(1, 2, 3) Traceback (innermost last): File "", line 1, in ? TypeError: list() argument must be a sequence >>> list(3) Traceback (innermost last): File "", line 1, in ? TypeError: list() argument must be a sequence >>> tuple([1, 2, 3]) (1, 2, 3) >>> tuple(3) Traceback (innermost last): File "", line 1, in ? TypeError: tuple() argument must be a sequence Now look... >>> set([1, 2, 3]) # {1, 2, 3} or {[1, 2, 3]}? >>> set(1) # {1} or TypeError? I still think set(x, y) -> {x, y} is nice, but it is unfortunate that set() and list() would work differently. Perhaps if we think of set() in the same category as slice() it makes more sense. -- ?!ng From gvwilson@nevex.com Fri Feb 4 03:20:49 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Thu, 3 Feb 2000 22:20:49 -0500 (EST) Subject: [Python-Dev] Set data type In-Reply-To: Message-ID: FWIW, I favor {3, 5, 7} because it's similar to the notation for dictionaries. Greg On Thu, 3 Feb 2000, Ka-Ping Yee wrote: > On Thu, 3 Feb 2000, Greg Stein wrote: > > > > I think the follow is just as readable, if not more so: > > > > >>> s = set(1, 5, 7) > > > > Where set() is a new builtin, taking an arbitrary number of arguments and > > returning a new Set type. > > Sure. We would certainly need a set() built-in anyway. > I just thought the {1, 5, 7} syntax would be quite nice > for math-and-science people, as it matches textbooks well. > > > -- ?!ng > > > _______________________________________________ > Python-Dev maillist - Python-Dev@python.org > http://www.python.org/mailman/listinfo/python-dev > From gmcm@hypernet.com Fri Feb 4 03:45:20 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 3 Feb 2000 22:45:20 -0500 Subject: [Python-Dev] Set data type In-Reply-To: Message-ID: <1262486605-7678305@hypernet.com> Ka-Ping Yee wrote: > After Eric asked about sets at dinner (e.g. having 'x in dict' > do 'dict.has_key(x)') i wondered about what it would take to > support sets. [huge snip] While the syntax is really, really nice, of all the things in kjbuckets, sets are the only one that can be (almost) transparently faked. It strikes me the really valuable stuff there is in things like set*dict, finding closures, transposing... I know that Guido is not very comfortable with some of the choices Aaron made (or perhaps with defending those choices to the algebraicly impaired), but it seems a shame to just do the easy one and ignore the really powerful algebra that goes with it. - Gordon From gstein@lyra.org Fri Feb 4 03:46:26 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 3 Feb 2000 19:46:26 -0800 (PST) Subject: [Python-Dev] Set data type In-Reply-To: Message-ID: On Thu, 3 Feb 2000 gvwilson@nevex.com wrote: > FWIW, I favor {3, 5, 7} because it's similar to the notation for > dictionaries. > > Greg > > On Thu, 3 Feb 2000, Ka-Ping Yee wrote: > > On Thu, 3 Feb 2000, Greg Stein wrote: > > > > > > I think the follow is just as readable, if not more so: > > > > > > >>> s = set(1, 5, 7) > > > > > > Where set() is a new builtin, taking an arbitrary number of arguments and > > > returning a new Set type. > > > > Sure. We would certainly need a set() built-in anyway. > > I just thought the {1, 5, 7} syntax would be quite nice > > for math-and-science people, as it matches textbooks well. Context was dropped here. I had stated earlier (in my original note) that {3, 5, 7} is syntactically ambiguous. It *can* be resolved, but it would truly be messy. Borrowing Tim's Guido-channeling-headgear, I foresee that Guido wouldn't want to implement sets if it creates this kind of ambiguity and hackery in the parser/compiler. Cheers, -g p.s. Basically, the problem is the parser can't detect certain types of syntax errors in dictionary/set construction, but has to defer them to the byte-compile phase. Unfortunately, this means you can have a syntax tree that represents an invalid Python program. Given the increasing interest in manipulating syntax trees, this would imply that tools will have to be aware of this "feature" of syntax trees. Or we'd have to insert a post-parse tree walk to find these (and report an error before allowing tools to access the parse tree). There's more along this line, but I hope that I've clarified the issue (at least, as I see it :-) -- Greg Stein, http://www.lyra.org/ From tim_one@email.msn.com Fri Feb 4 04:33:00 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 3 Feb 2000 23:33:00 -0500 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: <200002032120.QAA00092@eric.cnri.reston.va.us> Message-ID: <000001bf6ec8$ec53e960$a2a0143f@tim> [Guido] > ... > I have a feeling that introducing optional static typing would help > here, because it clarifies the concept. For example, these two have > the same type, [int]: (the length is part of the value) > > [3, 100] > [5, 6, 7] > > On the other hand, these two have different types: > > (3, 100) # (int, int) > (5, 6, 7) # (int, int, int) > > If we start mixing types, we may get > > [3, "wow"] # [any]; or perhaps [int|str] > (3, "wow") # (int, str) Very much the same applies to functional languages, except (usually) even more so. For example, Haskell has both lists and tuples, but lists are homogenous (and the type system has no union types, at least not of the int|str form -- the list [3, "wow"] can't be expressed (although the tuple form can be): ? [3, "wow"] ERROR: [Char] is not an instance of class "Num" ? (a Haskell string is a list of Char, which is what [Char] means)). At heart, tuples are for Cartesian products of a fixed number of possibly differing types, while lists are for arbitrary-length sequences of a single type. This is good style even in Python! (Think about it: when you see a newbie post on c.l.py that violates your Python intuition wrt list or tuple use, in my experience it almost always violates one of those (hitherto never expressed ) guidelines.) That lists are allowed to be heterogenous should really be considered an advanced feature; ditto list operations (like indexing) on tuples. BTW, despite the recent unpleasantness with the TeachScheme! folks, there's a lot to be learned from DrScheme, part of which is the notion of language "levels": the IDE can be set to restrict the newbie to various (nested) subsets of the full language, so that new learners are shielded from the hairier stuff until their confidence builds. metaclasses-are-a-good-candidate-ly y'rs - tim From tim_one@email.msn.com Fri Feb 4 05:03:41 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 4 Feb 2000 00:03:41 -0500 Subject: [Python-Dev] Set data type In-Reply-To: Message-ID: <000901bf6ecd$34bb3ba0$a2a0143f@tim> [Greg Stein] > Context was dropped here. I had stated earlier (in my original note) > that {3, 5, 7} is syntactically ambiguous. It *can* be resolved, but > it would truly be messy. Borrowing Tim's Guido-channeling-headgear, > I foresee that Guido wouldn't want to implement sets if it creates > this kind of ambiguity and hackery in the parser/compiler. Hey, give that back! I was going to channel Guido to decide what I should eat tonight, but without his ersatz guidance ended up eating a can of rancid Y2K-paranoia sardines. Ooh, the belches. Ah, good, it's back. Ask next time, OK? Guido won't want to implement sets regardless of syntax: the pressure to make builtin types maximally efficient for all possible (reasonable & otherwise) applications is relentless & unproductively contentious, and "the best" representation for sets doesn't exist. My (not so) secret hope is that a combination of interfaces and optional static typing in Python 3000 will enable motivated & intelligent users to pick the most appropriate concrete implementations of such "one size barely fits anyone" data structures, based on their knowledge of their apps' specific needs. the-spirits-have-departed-and-now-i-must-rest-ly y'rs - tim From tim_one@email.msn.com Fri Feb 4 05:33:33 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 4 Feb 2000 00:33:33 -0500 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: Message-ID: <000b01bf6ed1$60959820$a2a0143f@tim> I'm fwd'ing this back to the list, because disagreement is more valuable to share than agreement. The C++, Fortran, Pascal or Java programmer can't *spell* the list [1, "two"] in their languages without playing casting tricks. Of course the elements are of different types, and for that very reason it's better to use a tuple here instead(especially if it's always of length two!). "Different data structures for different purposes" is as Pythonic as "different syntax for different purposes", and paying attention to this can improve your (that's the generic "your") Python programming life. I don't care if it's not immediately obvious to new users (or even to you ). Start from ground zero and try to explain why Python has both ints and floats: there is no *obvious* reason (even less so for having both ints and longs, btw). Python wouldn't fall apart without tuples, but I'd miss them a lot (e.g., in another vein, in an imperative language using immutable objects when possible can greatly aid reasoning about code ("OK, they pass a tuple here, so I don't have to worry at all about the callee mutating it") ...). Python itself violates my "guidelines" in using a tuple to catch a vrbl number of arguments, & I believe that's a minor design flaw (it should use a list for this instead -- although you can't satisfy both "homogenous" & "fixed length" at the same time here). WRT Scheme, I don't believe it's a reasonable choice to teach newbies unless they're CompSci majors; it would certainly be better if they had a language that didn't need to be "layered". DrScheme effectively *did* "re-design their language" by introducing subsets. Whether this is successful for Mathias you'll have to argue with him; I don't see cause to believe SP/k is relevant (neither the "subset" form of Fortran77 -- it was aiming at an entirely different thing). > -----Original Message----- > From: gvwilson@nevex.com [mailto:gvwilson@nevex.com] > Sent: Thursday, February 03, 2000 11:49 PM > To: Tim Peters > Subject: RE: [Python-Dev] re: syntax - "Aren't tuples redundant?" > > > > Tim Peters wrote: > > At heart, tuples are for Cartesian products of a fixed number of > > possibly differing types, while lists are for arbitrary-length > > sequences of a single type. > > Greg Wilson writes (to Tim Peters directly, in order to reduce bandwidth > on the general list --- feel free to re-post there if you think there's > any merit in what I've said below): > > Fooey. Programmers raised on C, Fortran, Pascal, or Java would tell you > that the elements of [1, "two"] have different types. So (I believe) > would most ten-year-olds, and they'd be right: I can't add 7 to "two", or > take a slice of 1. I can grandparent them by fiat with a type "any", but > that smells like I've decided what answer I want, and am now inventing > what I need to get there. > > Footnote: every time someone's told me that a language has a "zen" that > you have to "get", it's turned out that what they've really meant is that > there's a set of "just-so" stories that you have to accept as gospel. I > really don't want to offend anyone, but the justifications I'm hearing for > Python tuples are starting to sound like that. I'd be very grateful if > you could tell me what they actually buy programmers that lists don't, > other than being usable as hash keys, and maintaining backward > compatibility with earlier versions of Python. > > > BTW, despite the recent unpleasantness with the TeachScheme! folks, > > there's a lot to be learned from DrScheme, part of which is the notion > > of language "levels": the IDE can be set to restrict the newbie to > > various (nested) subsets of the full language, so that new learners > > are shielded from the hairier stuff until their confidence builds. > > I lived through another attempt to do this --- the SP/k series of layered > languages based on PL/1. It wasn't generally regarded as a success; when > a Pascal compiler that generated readable error messages came along, SP/k > was abandoned almost immediately. No disrespect to Mathias and his > colleagues, but I think that that if they have to shield learners from > part of the language in order to make it learnable, they ought to > re-design their language... > > Thanks, > Greg From tim_one@email.msn.com Fri Feb 4 07:05:03 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 4 Feb 2000 02:05:03 -0500 Subject: [Python-Dev] Set data type In-Reply-To: Message-ID: <000e01bf6ede$28fe1b00$a2a0143f@tim> [Ka-Ping Yee] > ... > 2. How do sets sort? > > We could insert them somewhere in the hierarchy of types > next to lists and tuples. Currently () > [] > {}, so we > could have () > [] > {,} > {} for example. Mixed non-numeric types currently sort by alphabetical order of type name, so "list" < "set" < "tuple" Since it's utterly arbitrary, there's no reason to depart from this (currently consistent) precedent. > More tricky is the question of what to do with the partial > ordering created by >, >=, etc. > > a. The answer is the same as whatever answer we get when > Python supports rich comparisons and instances can have > partial orderings too. Yes, that's the best. > ... > (Side note: Do we achieve a. just by divorcing __cmp__ from > >, >=, etc.? sorting would use __cmp__, In anticipation of rich comparisons, the 1.5.2 list.sort() already looks only at whether cmp's result is (or isn't) < 0. So it's also now easy to do sorting solely in terms of __lt__. > while > and < would look for __gt__, __lt__, etc., and if not > found, fall back on __cmp__. You're reinventing what rich comparisons will *always* do, here -- leave that to David Ascher, cuz it's a mess <0.9 wink>. > ... > such promises. Side side note: which of these promises, if > any, should we ask __gt__ et al to make? Stability, at least?) The ones that make obvious sense for __gt__ etc, of course. There's no point in introducing a type for which it's not guaranteed that, e.g., a ((Side side side note: in E, Mark Miller also ran into the > problem of spelling different kinds of "equals"es. For > object identity we have "is", for content comparison we > have "==". If we need a new operator for magnitude equality > i suggest "<=>", as used in E.)) I don't know what "magnitude equality" means. If it means comparing by cardinality, people should use explict "len". <=> is likely a poor choice regardless because that's the way Perl spells Python's cmp (or mabye that's what mag. eq. means?). > Well, it may be a lot of words, but it's not much good unless you > are going to use it in practice. I think i would have good use for > it, but i would like to hear your opinions. Would you use such a > thing? Yes, but Guido won't accept it . In my own Set classes, I eschewed operator overloading because I found that, in practice, I routinely need both functional and mutating forms of all the basic operations (inclusion, intersection, union, difference, symmetric difference). I also needed to dream up a scheme to allow Sets of Sets (& so on). A truly useful set type needs to be very rich! Which is another reason (see earlier post for the other) Guido won't want this (the inability to subclass builtin types in Python today means he would have to arbitrate the whole universe of possible set methods, and that's a time sink with small payback). when-desire-meets-reality-guido-wins-ly y'rs - tim From tim_one@email.msn.com Fri Feb 4 08:55:02 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 4 Feb 2000 03:55:02 -0500 Subject: [Python-Dev] re: syntax feedback from LANL course (very long) In-Reply-To: Message-ID: <001301bf6eed$865846e0$a2a0143f@tim> Just picking on the loop issue here. [Ka-Ping Yee, on Greg Wilson's post] > Wow, these were really informative. Thanks for collecting the results. Ditto! It's posts like these that make me glad Guido has so much spare time . >> 1. "What's with 'while 1: break' for normal loops?" > I'd probably agree that this is a significant thing. Me too: it's the ugliest thing in Python newbies hit right away, and while *I* can't really "see" the convolution in "while 1: ... if xxx: break ..." anymore, that's learned behavior; but I get reminded of how grating it is at first each time someone new at work starts learning Python. This one made Andrew's "Python Warts" paper. Enough is enough already <0.5 wink>. > I think it would be very nice to have do...while, and of all your > examples i liked > > do: > first-body > while condition: > second-body > > best. Actually suggested by Guido Himself on Christmas Eve of '98 (see DejaNews; I found the URL for Greg offline but have since lost it). And the IDLE editor already handles it fine . >> while: >> first-half >> and while condition-1: >> second-half >> and while condition-2: >> third-half # [sic] >> and while final-condition # no trailing ':' > This *is* confusing. It's from an old c.l.py thread; don't recall who suggested it; I don't like it myself; "and while xxx:" is just sugar for "if not xxx: break"; its advantage is the absence of new keywords; the "while:" at the top reads poorly; the semantics of "and while" are indeed unclear at first (IIRC, Christian took way too many drugs at the time and tried to generalize it to "and if" as well ...). > ... > "do" doesn't even have to be made into a keyword for the > do...while construct to work: there is no other valid > construct in which a name would appear followed by a colon > as the first thing in a statement or expression, so the > parser should be able to figure it out. Good point! Not the kind of point Guido warms to instantly, but a good point all the same. I had already checked, and there are at least two functions named "do" in Python's own std distribution. So making "do" a keyword is very unattractive. I'm not sure "repeat:" (a la Pascal) or "loop:" (Icon? too lazy to look it up -- *one* of those loser languages I'm always irritating people about ) would actually be better. If hackery could overload "do" (or repeat, or loop), I'm in favor of it. if-syntax-matters-then-syntax-matters-ly y'rs - tim From tismer@tismer.com Fri Feb 4 12:25:04 2000 From: tismer@tismer.com (Christian Tismer) Date: Fri, 04 Feb 2000 13:25:04 +0100 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) References: <200001200419.XAA01969@mira.erols.com> <38871665.C3B6FFEE@digicool.com> <38871CE5.53FB9D68@digicool.com> <200001201545.KAA21304@eric.cnri.reston.va.us> <38873C52.29FEAC6D@digicool.com> <200001201720.MAA21534@eric.cnri.reston.va.us> <3899A01E.DE29FC94@digicool.com> <200002031945.OAA29682@eric.cnri.reston.va.us> <3899E359.4ED37D66@digicool.com> <200002032107.QAA00051@eric.cnri.reston.va.us> Message-ID: <389AC520.96C24321@tismer.com> Guido van Rossum wrote: > > > > But, overall the necessary changes to the implementation and to the > > > semantics (e.g. of the 'for' statement) seem prohibitive to me. > ^^^ (I meant 'from' of course) > > > Really? Even for Py3K? > > The implementation wouldn't be a problem for Py3K; I was under the > impression that you thought this could be put in earlier. Jim proposed adding namespaces to Python 2000. This will be, for my understanding, a complete rewrite and redesign that is allowed to break existing code. It would even be run in parallel to Python 1.6++ for a while, right? > But the change in semantics is very hard to swallow to me. It > definitely seems to be come murkier. > > > > I also think that the namespace implementation will be quite a bit > > > less efficient than a regular dictionary: > > > > Spacewise yes. They'd me much faster in use. This is a space/speed > > tradeoff. > > Agreed; though the speedup comes from circumventing the dictionary > altogether. I do not even believe in the space ineffectiveness. The namespace concept works fine without a dictionary and hashes at all. We can implement this as a linear list of pointers to namespace objects, since they are looked up only once, usually. But even if we would keep a dictionary-like structure as well, it is possible to implement it as an array of pointers, and you get things smaller than now, not bigger. In your analysis, you forgot to take into account that the average dictionary slot overhead gives a factor of about two. 1 dict slot = 3 words n dict entries = average 2n dict slots = 6n words versus 1 asso object = = 4 words n asso dict entries = average 2n words + n asso objects This gives 6n words for the proposed solution, actually as effective as today's 6n solution with dicts. Ahem :-) It can of course be that we also need a hash filed, which can be stored in the asso object. This is another word per element, so we'd have a cost increase of 1/6. As said, the dictionary is not necessary and could be created on demand (that is, if globals are really used like a dict). Without it, I count just n + 4n = 5n, actually a saving. This idea bears a lot of potential to also speed up classes and instances. Further analysis is needed, please let us not drop this idea too early. ciao - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Düppelstr. 31 : *Starship* http://starship.python.net 12163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From gvwilson@nevex.com Fri Feb 4 13:10:27 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Fri, 4 Feb 2000 08:10:27 -0500 (EST) Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: <000b01bf6ed1$60959820$a2a0143f@tim> Message-ID: (Hope no-one minds me keeping this thread alive --- as I said in my first reply to Tim Peters, there's either something very fundamental here, or a "just-so" story...) > > > Tim Peters wrote: > > > At heart, tuples are for Cartesian products of a fixed number of > > > possibly differing types, while lists are for arbitrary-length > > > sequences of a single type. > > Greg Wilson wrote: > > Fooey. Programmers raised on C, Fortran, Pascal, or Java would tell you > > that the elements of [1, "two"] have different types. So (I believe) > > would most ten-year-olds, and they'd be right: I can't add 7 to "two", or > > take a slice of 1. I can grandparent them by fiat with a type "any", but > > that smells like I've decided what answer I want, and am now inventing > > what I need to get there. > Tim Peters wrote: > The C++, Fortran, Pascal or Java programmer can't *spell* the list [1, > "two"] in their languages without playing casting tricks. Greg Wilson wrote: The fact that their current language doesn't allow this is irrelevant to the argument. Show them [1, "two"] and they (a) understand it, and (b) think it's cool; show them (1, "two") as well and they become confused. > Tim Peters wrote: > Of course the elements are of different types, and for that very > reason it's better to use a tuple here instead (especially if it's > always of length two!). Greg Wilson wrote: But *why* is it better? Or to put it another way: If tuples didn't already exist, would anyone ask for them to to be added to the language today? > Tim Peters wrote: > "Different data structures for different purposes" is as Pythonic as > "different syntax for different purposes", and paying attention to > this can improve your (that's the generic "your") Python programming > life. Greg Wilson wrote: Analogic reasoning makes me nervous, as it is most often used to transfuse legitimacy from the defensible to the suspect. > Tim Peters wrote: > Start from ground zero and try to explain why Python has both ints and > floats: there is no *obvious* reason (even less so for having both > ints and longs, btw). Greg Wilson wrote: I've never had any trouble explaining int vs. float to students at any level; I've also never had any trouble explaining int vs. long (memory vs. accuracy). Thanks for your reply, Greg From jcw@equi4.com Fri Feb 4 14:02:19 2000 From: jcw@equi4.com (Jean-Claude Wippler) Date: Fri, 04 Feb 2000 15:02:19 +0100 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" References: Message-ID: <389ADBE1.C325EE0E@equi4.com> gvwilson@nevex.com wrote: > > (Hope no-one minds me keeping this thread alive [...]) Ditto. GV> Show them [1, "two"] and they (a) understand it, and (b) think GV> it's cool; show them (1, "two") as well and they become confused. Because they mean the same thing, I suppose? GV> If tuples didn't already exist, would anyone ask for them to GV> to be added to the language today? Why indeed? They are more space-efficient, and they are immutable, but those are both purely technical reasons. The first reason is likely to become less important (silicon gets faster), the second one *could* be solved by forcing keys to have a refcount of 1 - this means copying when needed, both on creation and on access. That's pretty awkward, copy on write would help a lot, but I doubt that Python can be made to do that (tuples neatly prevent circular immutable structures, btw). GV> I've never had any trouble explaining int vs. float to students at Because ints and floats differ in meaning? GV> any level; I've also never had any trouble explaining int vs. long GV> (memory vs. accuracy). That's interesting. Tuples vs. lists are a similar tradeoff, though both memory-savings and immutability are CS-type issues, whereas non- programmers are more likely to consider accuracy a meaningful tradeoff? -- Jean-Claude From gvwilson@nevex.com Fri Feb 4 14:15:54 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Fri, 4 Feb 2000 09:15:54 -0500 (EST) Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: <389ADBE1.C325EE0E@equi4.com> Message-ID: Hi, Jean-Claude; thanks for your mail. > GV> Show them [1, "two"] and they (a) understand it, and (b) think > GV> it's cool; show them (1, "two") as well and they become confused. > Because they mean the same thing, I suppose? Redundancy seems to confuse people (did someone say "Perl" or "PL/1"?) > GV> If tuples didn't already exist, would anyone ask for them to > GV> to be added to the language today? > > Why indeed? They are more space-efficient, and they are immutable, > but those are both purely technical reasons. Agreed --- I could understand having tuples as an internal data structure, but do not understand why they are exposed to users. If Python had been a type-checked, type-inferenced langauge from the beginning, I guess I could see it... As for the immutability: >>>> x = [1, 2] >>>> y = (10, x, 11) >>>> y[1][0] = 999 >>>> y (10, [999, 2], 11) > GV> I've never had any trouble explaining int vs. float to students at > Because ints and floats differ in meaning? People are taught "whole numbers" vs. "fractions" at an early age. > GV> I've also never had any trouble explaining int vs. long > GV> (memory vs. accuracy). > > That's interesting. Tuples vs. lists are a similar tradeoff, though > both memory-savings and immutability are CS-type issues, whereas non- > programmers are more likely to consider accuracy a meaningful tradeoff? I just show them the range of values that can be represented in 8, 16, 32, or 64 bits (for ints); 'float' vs. 'double' follows naturally from that. Again, I've never had any trouble with this one... Interestingly, I've also never had trouble with strings being immutable. I point out to people that the number 123 isn't mutable, and that supposedly-mutable strings in other languages are really allocating/deallocating memory behind your back; everybody nods, and we carry on. Greg From guido@python.org Fri Feb 4 14:26:09 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 04 Feb 2000 09:26:09 -0500 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) In-Reply-To: Your message of "Fri, 04 Feb 2000 13:25:04 +0100." <389AC520.96C24321@tismer.com> References: <200001200419.XAA01969@mira.erols.com> <38871665.C3B6FFEE@digicool.com> <38871CE5.53FB9D68@digicool.com> <200001201545.KAA21304@eric.cnri.reston.va.us> <38873C52.29FEAC6D@digicool.com> <200001201720.MAA21534@eric.cnri.reston.va.us> <3899A01E.DE29FC94@digicool.com> <200002031945.OAA29682@eric.cnri.reston.va.us> <3899E359.4ED37D66@digicool.com> <200002032107.QAA00051@eric.cnri.reston.va.us> <389AC520.96C24321@tismer.com> Message-ID: <200002041426.JAA09301@eric.cnri.reston.va.us> (Math about sapce savings gracefully accepted.) > This idea bears a lot of potential to also speed up > classes and instances. Further analysis is needed, > please let us not drop this idea too early. I will gladly use it as an implementation strategy "under the hood" if it makes sense -- I think that with some code analysis (e.g. "what are the globals here") it could be made to work well. But I don't think that changing the "from M import v" semantics so that local assignment to v changes the binding of M.v as well is defensible. --Guido van Rossum (home page: http://www.python.org/~guido/) From tismer@tismer.com Fri Feb 4 14:30:54 2000 From: tismer@tismer.com (Christian Tismer) Date: Fri, 04 Feb 2000 15:30:54 +0100 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) References: <200001200419.XAA01969@mira.erols.com> <38871665.C3B6FFEE@digicool.com> <38871CE5.53FB9D68@digicool.com> <200001201545.KAA21304@eric.cnri.reston.va.us> <38873C52.29FEAC6D@digicool.com> <200001201720.MAA21534@eric.cnri.reston.va.us> <3899A01E.DE29FC94@digicool.com> <200002031945.OAA29682@eric.cnri.reston.va.us> <3899E359.4ED37D66@digicool.com> <200002032107.QAA00051@eric.cnri.reston.va.us> <389AC520.96C24321@tismer.com> <200002041426.JAA09301@eric.cnri.reston.va.us> Message-ID: <389AE29E.BD2A4235@tismer.com> Guido van Rossum wrote: > > (Math about sapce savings gracefully accepted.) > > > This idea bears a lot of potential to also speed up > > classes and instances. Further analysis is needed, > > please let us not drop this idea too early. > > I will gladly use it as an implementation strategy "under the hood" if > it makes sense -- I think that with some code analysis (e.g. "what are > the globals here") it could be made to work well. Pleased to hear this. Thank you for keeping it alive. > But I don't think that changing the "from M import v" semantics so > that local assignment to v changes the binding of M.v as well is > defensible. Of course this may be the weakest point yet, while it was the reason to reasoning at all in the first place, it is less important now. :-) ciao - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Düppelstr. 31 : *Starship* http://starship.python.net 12163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From jim@digicool.com Fri Feb 4 14:40:04 2000 From: jim@digicool.com (Jim Fulton) Date: Fri, 04 Feb 2000 09:40:04 -0500 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) References: <200001200419.XAA01969@mira.erols.com> <38871665.C3B6FFEE@digicool.com> <38871CE5.53FB9D68@digicool.com> <200001201545.KAA21304@eric.cnri.reston.va.us> <38873C52.29FEAC6D@digicool.com> <200001201720.MAA21534@eric.cnri.reston.va.us> <3899A01E.DE29FC94@digicool.com> <200002031945.OAA29682@eric.cnri.reston.va.us> <3899E359.4ED37D66@digicool.com> <200002032107.QAA00051@eric.cnri.reston.va.us> <389AC520.96C24321@tismer.com> <200002041426.JAA09301@eric.cnri.reston.va.us> <389AE29E.BD2A4235@tismer.com> Message-ID: <389AE4C3.D950C081@digicool.com> Christian Tismer wrote: > (snip) > Of course this may be the weakest point yet, while it was > the reason to reasoning at all in the first place, it is > less important now. :-) Actually, I've been talking about this in various venues for about 4 years. :) This thread is the first time I've mentioned it the context of import. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From jim@digicool.com Fri Feb 4 14:47:05 2000 From: jim@digicool.com (Jim Fulton) Date: Fri, 04 Feb 2000 09:47:05 -0500 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) References: <200001200419.XAA01969@mira.erols.com> <38871665.C3B6FFEE@digicool.com> <38871CE5.53FB9D68@digicool.com> <200001201545.KAA21304@eric.cnri.reston.va.us> <38873C52.29FEAC6D@digicool.com> <200001201720.MAA21534@eric.cnri.reston.va.us> <3899A01E.DE29FC94@digicool.com> <200002031945.OAA29682@eric.cnri.reston.va.us> <3899E359.4ED37D66@digicool.com> <200002032107.QAA00051@eric.cnri.reston.va.us> <389AC520.96C24321@tismer.com> <200002041426.JAA09301@eric.cnri.reston.va.us> Message-ID: <389AE669.72A127CA@digicool.com> Guido van Rossum wrote: > (snip) > But I don't think that changing the "from M import v" semantics so > that local assignment to v changes the binding of M.v as well is > defensible. I agree, however, I think that having: from M import v causing a name binding that is broken by local assigment to v *is* defensible and reasonably implementable. Changes to 'v' in M (including by reload of M) would be reflected locally unless someone did: v=something locally. Local assignment would negate an import, as it does now. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From fredrik@pythonware.com Fri Feb 4 14:56:41 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Fri, 4 Feb 2000 15:56:41 +0100 Subject: [Python-Dev] Re: [Py-Consortium] Unicode snapshot available References: <200002040030.TAA07687@eric.cnri.reston.va.us> Message-ID: <032901bf6f20$208f44c0$f29b12c2@secret.pythonware.com> Guido wrote: > There's one thing missing (but not for long!): Fredrik Lundh's sre > module, which implements re-compatible-but-faster regular expressions > for 8-bit characters and Unicode, hasn't been integrated yet. We're > waiting for Fredrik, who's just back from the conference -- it > shouldn't be long now. a snapshot will be posted real soon now (should have appeared yesterday, but some business stuff got into the way, and now I've had too much champagne for my own good ;-). anyway, to grab it, take the last part of MAL's URL (just the filename), and append it to: http://w1.132.telia.com/~u13208596/ (nothing there right now) if you have questions, use the effbot@telia.com address (until further notice, at least -- might reorganize my mail accounts in a near future). From guido@python.org Fri Feb 4 15:19:52 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 04 Feb 2000 10:19:52 -0500 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) In-Reply-To: Your message of "Fri, 04 Feb 2000 09:47:05 EST." <389AE669.72A127CA@digicool.com> References: <200001200419.XAA01969@mira.erols.com> <38871665.C3B6FFEE@digicool.com> <38871CE5.53FB9D68@digicool.com> <200001201545.KAA21304@eric.cnri.reston.va.us> <38873C52.29FEAC6D@digicool.com> <200001201720.MAA21534@eric.cnri.reston.va.us> <3899A01E.DE29FC94@digicool.com> <200002031945.OAA29682@eric.cnri.reston.va.us> <3899E359.4ED37D66@digicool.com> <200002032107.QAA00051@eric.cnri.reston.va.us> <389AC520.96C24321@tismer.com> <200002041426.JAA09301@eric.cnri.reston.va.us> <389AE669.72A127CA@digicool.com> Message-ID: <200002041519.KAA09803@eric.cnri.reston.va.us> [me] > > But I don't think that changing the "from M import v" semantics so > > that local assignment to v changes the binding of M.v as well is > > defensible. > > I agree, however, I think that having: > > from M import v > > causing a name binding that is broken by local > assigment to v *is* defensible and reasonably > implementable. > > Changes to 'v' in M (including by reload of M) would be > reflected locally unless someone did: > > v=something > > locally. Local assignment would negate an import, as it > does now. Hm, but it still wouldn't have the same semantics as currently, and that's still a monster hiding under the bed until you're nearly asleep. Consider this example: # in M: verbose = 1 # in __main__: from M import verbose # somewhere else: M.verbose = 0 Under the current semantics, that would have no effect on verbose in __main__; but with your semantics it would. I think that is very hard to explain; even more so if you say that assigning a different value to __main__.verbose does not change M.verbose and furthermore breaks the connection. This means that if I add verbose = verbose to the __main__ code the semantics are different! I don't understand why you wanted these semantics in the first place. --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik@pythonware.com Fri Feb 4 14:44:13 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Fri, 4 Feb 2000 15:44:13 +0100 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" References: Message-ID: <031f01bf6f1e$51fe8540$f29b12c2@secret.pythonware.com> Greg Wilson wrote: > > Because ints and floats differ in meaning? > People are taught "whole numbers" vs. "fractions" at an early age. sure, but are they taught that what looks like fractions are treated as integers, and that small integers don't auto- matically become large integers when necessary? ;-) > Interestingly, I've also never had trouble with strings being = immutable. I > point out to people that the number 123 isn't mutable, and that > supposedly-mutable strings in other languages are really > allocating/deallocating memory behind your back; everybody nods, and = we > carry on. interesting indeed. seems to me as if the best way to avoid list/tuple confusion is to start by introducing the basic types (numbers, strings, tuples, which are all immutable), *before* explaining that python also supports mutable container types (lists, dictionaries, instances, etc)? From jim@digicool.com Fri Feb 4 17:22:20 2000 From: jim@digicool.com (Jim Fulton) Date: Fri, 04 Feb 2000 12:22:20 -0500 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) References: <200001200419.XAA01969@mira.erols.com> <38871665.C3B6FFEE@digicool.com> <38871CE5.53FB9D68@digicool.com> <200001201545.KAA21304@eric.cnri.reston.va.us> <38873C52.29FEAC6D@digicool.com> <200001201720.MAA21534@eric.cnri.reston.va.us> <3899A01E.DE29FC94@digicool.com> <200002031945.OAA29682@eric.cnri.reston.va.us> <3899E359.4ED37D66@digicool.com> <200002032107.QAA00051@eric.cnri.reston.va.us> <389AC520.96C24321@tismer.com> <200002041426.JAA09301@eric.cnri.reston.va.us> <389AE669.72A127CA@digicool.com> <200002041519.KAA09803@eric.cnri.reston.va.us> Message-ID: <389B0ACC.30071942@digicool.com> Guido van Rossum wrote: > > [me] > > > But I don't think that changing the "from M import v" semantics so > > > that local assignment to v changes the binding of M.v as well is > > > defensible. > > > > I agree, however, I think that having: > > > > from M import v > > > > causing a name binding that is broken by local > > assigment to v *is* defensible and reasonably > > implementable. > > > > Changes to 'v' in M (including by reload of M) would be > > reflected locally unless someone did: > > > > v=something > > > > locally. Local assignment would negate an import, as it > > does now. > > Hm, but it still wouldn't have the same semantics as currently, Agreed. Think Python 3000. I think that the semantics differ in boundary cases though. > and that's still a monster hiding under the bed until you're nearly > asleep. Consider this example: > > # in M: > verbose = 1 > > # in __main__: > from M import verbose > > # somewhere else: > M.verbose = 0 > > Under the current semantics, that would have no effect on verbose in > __main__; but with your semantics it would. Yup. > I think that is very hard > to explain; even more so if you say that assigning a different value > to __main__.verbose does not change M.verbose and furthermore breaks > the connection. This means that if I add > > verbose = verbose > > to the __main__ code the semantics are different! I'm suggesting a model where from "M import x" has a different meaning than it does now. I think the notion of sharing a name is useful. I'll admit that using "M.x" achieves the same thing, although at a higher performance cost (and, OK, typing cost ;). > I don't understand why you wanted these semantics in the first place. First, let me say that this isn't super important to me. It does solve a problem with reload, which is the context in which I brought it up. Now, consider: from M import x ..... use(x) Many people would (wrongly) consider this to be equivalent to: import M ..... use(M.x) In fact, I'd *prefer* these to be equivalent even in the face of changes to M (e.g. reload). I'd prefer different semantics. Note that if I had: from M import x ..... x=y ..... use(x) I'd no longer exprect x to have any connection to M. Of course: x=x or x=M.x would be a bit more puzzling, but then they're meant to be. ;) They are addressed by a simple rule, which is that assignment in a module overrides imported name definition. Hm...ooh ooh A better solution would be to disallow assignments to imported names, as they are very likely to be errors. This could be detected without any fancy type inferencing. In fact, we could also decide to disallow an import-from to override an existing name binding. Ahhhhhh. :) In any case, I'd feel comfortable explaining a system in which from M import x # reference semantics wrt name had a different meaning from: import M x=M.x # copy semantics since I expect an attribute access to give me a value, not a name, whereas: from M import x seems more to me like it's talking about names. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From skip@mojam.com (Skip Montanaro) Fri Feb 4 19:02:53 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 4 Feb 2000 13:02:53 -0600 (CST) Subject: [Python-Dev] Python 1.6 timing In-Reply-To: <14488.33324.334870.763850@anthem.cnri.reston.va.us> References: <14488.25035.985792.589165@anthem.cnri.reston.va.us> <14488.33324.334870.763850@anthem.cnri.reston.va.us> Message-ID: <14491.8797.503366.14398@beluga.mojam.com> >>>>> "MZ" == Moshe Zadka writes: MZ> Ummmm...would this be the wrong time to ask how the redesign contest MZ> is going on? BAW> It isn't, AFAICT. .... Perhaps the occasional reminder posted to c.l.py would stimulate a bit more activity. I don't recall seeing anything. I stumbled upon the menu item on the PSA web site awhile ago. Otherwise I probably wouldn't have known what Moshe was referring to. Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ 847-971-7098 "Languages that change by catering to the tastes of non-users tend not to do so well." - Doug Landauer From gvwilson@nevex.com Fri Feb 4 19:00:29 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Fri, 4 Feb 2000 14:00:29 -0500 (EST) Subject: [Python-Dev] re: Programming Paradigms paper Message-ID: I've put a copy of a paper by Marian Petre on the Software Carpentry web site at: http://www.software-carpentry.com/extern/petre_paradigms.html I hope it will be of interest to people on this list. From the introduction: ...language designers everywhere aspire to a Kuhnian ideal that paradigms are absolute: complete and exclusive, dominating and defining every aspect of an approach... This paper presents an alternative view of programming paradigms based on studies of expert programmer behaviour: a view of paradigms as selectable reference models...that evolve from expert practice and are shaped within a programming culture. From that vantage, it considers the implications of programming paradigms for teaching programming. The author is with the Open University in the United Kingdom; her email is in the paper, and she would be grateful if you could let her know if you cite or reference the paper. Greg From tismer@tismer.com Fri Feb 4 19:11:11 2000 From: tismer@tismer.com (Christian Tismer) Date: Fri, 04 Feb 2000 20:11:11 +0100 Subject: [Python-Dev] re: syntax feedback from LANL course (very long) References: <001301bf6eed$865846e0$a2a0143f@tim> Message-ID: <389B244F.8395233D@tismer.com> Tim Peters wrote: > > Just picking on the loop issue here. ... > >> while: > >> first-half > >> and while condition-1: > >> second-half > >> and while condition-2: > >> third-half # [sic] > >> and while final-condition # no trailing ':' > > > This *is* confusing. > > It's from an old c.l.py thread; don't recall who suggested it; I don't like > it myself; "and while xxx:" is just sugar for "if not xxx: break"; its > advantage is the absence of new keywords; the "while:" at the top reads > poorly; the semantics of "and while" are indeed unclear at first (IIRC, > Christian took way too many drugs at the time and tried to generalize > it to "and if" as well ...). Yes I'm still ashamed on that. It may be that Guido was about to say "perhaps" (as worthy as a woman's "perhaps", you know), but I went too far and killed it by "and if". Sigh... SORRY! > > "do" doesn't even have to be made into a keyword for the > > do...while construct to work: there is no other valid > > construct in which a name would appear followed by a colon > > as the first thing in a statement or expression, so the > > parser should be able to figure it out. > > Good point! Not the kind of point Guido warms to instantly, but a good > point all the same. I had already checked, and there are at least two > functions named "do" in Python's own std distribution. So making "do" a > keyword is very unattractive. I'm not sure "repeat:" (a la Pascal) or > "loop:" (Icon? too lazy to look it up -- *one* of those loser languages I'm > always irritating people about ) would actually be better. If hackery > could overload "do" (or repeat, or loop), I'm in favor of it. Oh, much simpler: We have it all ready :) Just make "while 1:" into a keyword, like the #end block delimiters. ciao - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Düppelstr. 31 : *Starship* http://starship.python.net 12163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From Moshe Zadka Fri Feb 4 19:42:22 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 4 Feb 2000 21:42:22 +0200 (IST) Subject: [Python-Dev] random.py weirdness Message-ID: I'll refer you all to the copy of random.py you have on your hard-disks, to line 68, where a function new_generator(a=None) is defined, merely returning generator(a=None). I wonder who put it there, and why. At least to simplify the weirdness, we could change it to # for backwards compatability new_generator = generator Which would work for all but the most obscure hacks. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From guido@python.org Fri Feb 4 19:57:41 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 04 Feb 2000 14:57:41 -0500 Subject: [Python-Dev] random.py weirdness In-Reply-To: Your message of "Fri, 04 Feb 2000 21:42:22 +0200." References: Message-ID: <200002041957.OAA14995@eric.cnri.reston.va.us> I have no idea; new_generator() was introduced at the same time as the generator class. I would leave it alone -- it's an example of a factory function. --Guido van Rossum (home page: http://www.python.org/~guido/) From Vladimir.Marangozov@inrialpes.fr Fri Feb 4 21:09:05 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 4 Feb 2000 22:09:05 +0100 (CET) Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) In-Reply-To: <200002041519.KAA09803@eric.cnri.reston.va.us> from "Guido van Rossum" at Feb 04, 2000 10:19:52 AM Message-ID: <200002042109.WAA12722@python.inrialpes.fr> [Jim, Chris & Guido discussing a namespace idea] Guys, I'm lost. Please help me understanding this idea from the start. After rereading this whole thread, I have only a vague intuition of what Jim has proposed, but I fail to understand it; and believe me, I'm very interested in being in sync with you on the subject. Please filter the concept from the consequences and resubmit it once again (in english, through examples, ascii art, whatever). Thanks. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Moshe Zadka Fri Feb 4 21:27:54 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 4 Feb 2000 23:27:54 +0200 (IST) Subject: [Python-Dev] python-patches, anyone? Message-ID: Have we decided not to do anything to take the load off of our poor Guido? Where have the discussion gone? Perhaps someone should just do something? (hint, hint, nudge, nudge, people whose name starts with B ) -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From dan@cgsoftware.com Fri Feb 4 21:37:25 2000 From: dan@cgsoftware.com (Daniel Berlin) Date: Fri, 4 Feb 2000 13:37:25 -0800 (PST) Subject: [Python-Dev] python-patches, anyone? In-Reply-To: Message-ID: Well, i sent barry a first run at a patchhunter yesterday (where first run is defined as "worked on every patch encapsulated in a mail message on guido's page, but wasn't commented") that, when given a message, will tell you if it has a patch, and if it does, if that patch is missing a disclaimer. Handles MIME (IE multipart messages, mime encodings, etc), uuencode, gzipped, and all variants thereof. So, hopefully, we will see something soon. --Dan On Fri, 4 Feb 2000, Moshe Zadka wrote: > Have we decided not to do anything to take the load off of our poor Guido? > Where have the discussion gone? > Perhaps someone should just do something? (hint, hint, nudge, nudge, > people whose name starts with B ) > > -- > Moshe Zadka . > INTERNET: Learn what you know. > Share what you don't. > > > _______________________________________________ > Python-Dev maillist - Python-Dev@python.org > http://www.python.org/mailman/listinfo/python-dev > From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Fri Feb 4 21:38:21 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 4 Feb 2000 16:38:21 -0500 (EST) Subject: [Python-Dev] python-patches, anyone? References: Message-ID: <14491.18125.327512.56257@anthem.cnri.reston.va.us> >>>>> "MZ" == Moshe Zadka writes: MZ> Have we decided not to do anything to take the load off of our MZ> poor Guido? Where have the discussion gone? Perhaps someone MZ> should just do something? (hint, hint, nudge, nudge, people MZ> whose name starts with B ) http://www.python.org/mailman/listinfo/patches but since GvR's the list admin, I was going to let him announce it. -Barry From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Fri Feb 4 21:39:11 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 4 Feb 2000 16:39:11 -0500 (EST) Subject: [Python-Dev] python-patches, anyone? References: Message-ID: <14491.18175.558832.443339@anthem.cnri.reston.va.us> >>>>> "DB" == Daniel Berlin writes: DB> Well, i sent barry a first run at a patchhunter yesterday DB> (where first run is defined as "worked on every patch DB> encapsulated in a mail message on guido's page, but wasn't DB> commented") that, when given a message, will tell you if it DB> has a patch, and if it does, if that patch is missing a DB> disclaimer. Handles MIME (IE multipart messages, mime DB> encodings, etc), uuencode, gzipped, and all variants thereof. DB> So, hopefully, we will see something soon. Yup, I just haven't had time to look at it yet. Some last minute deadlines came up. -Barry From Moshe Zadka Fri Feb 4 21:43:09 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 4 Feb 2000 23:43:09 +0200 (IST) Subject: [Python-Dev] python-patches, anyone? In-Reply-To: <14491.18125.327512.56257@anthem.cnri.reston.va.us> Message-ID: [Moshe Zadka gets impatient with the patches list] [Barry A. Warsaw] > http://www.python.org/mailman/listinfo/patches > > but since GvR's the list admin, I was going to let him announce it. 1. My sincere apologies for ruining GvR's announcement 2. Lots of thanks to Barry. in-any-case-the-best-/late/-christmas-present-I-got-this-year-ly y'rs, Z. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From guido@python.org Fri Feb 4 22:12:06 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 04 Feb 2000 17:12:06 -0500 Subject: [Python-Dev] New patch submission guidelines Message-ID: <200002042212.RAA16517@eric.cnri.reston.va.us> We've started a new way of dealing with patches. From now on, patches must be mailed to patches@python.org, and must conform to a set of requirements set forth in http://www.python.org/patches/ Patches sent to me personally and non-conforming will be bounced; see the above webpage for details. Developers (python-dev): feel free to subscribe to the patches mailing list! --Guido van Rossum (home page: http://www.python.org/~guido/) From Moshe Zadka Fri Feb 4 22:44:56 2000 From: Moshe Zadka (Moshe Zadka) Date: Sat, 5 Feb 2000 00:44:56 +0200 (IST) Subject: [Python-Dev] Naming rules for function types Message-ID: In Include/object.h, around line 140, many pointer to functions are typedef'ed. I need a new function type: int (*f)(PyObject *, PyObject *) And I'm at a bit of a los what to call it. Anyone has any idea what the naming convention is? Is objobjproc the right thing? Thanks for your attention. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From Moshe Zadka Sat Feb 5 00:15:37 2000 From: Moshe Zadka (Moshe Zadka) Date: Sat, 5 Feb 2000 02:15:37 +0200 (IST) Subject: [Python-Dev] __contains__ hook: follow-up Message-ID: I've sent a patch to patches@python.org which does the 'in' hook in the pseudo-right way: adding a slot to the sequence methods structure, and adding a flag to support it for backwards compatability. Now that it's there, I've also changed instance method to use it, and to map it to __contains__. The code there also has special-case for strings, so if someone (maybe me) adds such a slot to strings, then the code will only have to do the C equivalent of try: obj.__contains__(whatever) except AttributeError: for element in obj: if element == whatever: return 1 return 0 Is this desirable? It shouldn't be too hard... -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From mhammond@skippinet.com.au Sat Feb 5 02:53:20 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Sat, 5 Feb 2000 13:53:20 +1100 Subject: [Python-Dev] Proposal: Registry access for Python on Windows In-Reply-To: <018b01bf6e4a$4db27700$4500a8c0@thomasnotebook> Message-ID: [Thomas writes] > I admit: it has been too low level. I have posted > (and implemented) a new proposal describing a more > high level interface. Your high-level interface looks fine (except it should be coded in Python ;-) > In this I followed Gordon's > suggestion: Provide the minimum needed. I took that to mean "for the high-level interface" > If one wants to do more special things, one probably needs > your win32 extensions anyway. Im not really convinced about this. I would still rather see the complete win32api registry support added, seeing as the code exists. > Already too late! (Programming is fun :-) (Is competition good ?) Of course it is good - it means I will get my implementation done quicker now ;-) > Don't you think that the raw Win32 api functions are much too low > level to belong into core python? How should they be documented? You said you have seen the sources, so you should have seen there is also copious documentation - Im converting them to docstrings. Mark. From tim_one@email.msn.com Sat Feb 5 08:01:26 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 5 Feb 2000 03:01:26 -0500 Subject: [Python-Dev] re: syntax feedback from LANL course (very long) In-Reply-To: <389B244F.8395233D@tismer.com> Message-ID: <001801bf6faf$33e0e580$182d153f@tim> [about "and while", and generalization to "and if"] [Christian Tismer] > Yes I'm still ashamed on that. It may be that Guido was about > to say "perhaps" (as worthy as a woman's "perhaps", you know), > but I went too far and killed it by "and if". Sigh... > SORRY! Don't apologize to me! I *thank* you -- "and while" is excessively novel (a phrase I'm using excessively often lately , but one that I think captures the things about Python I like least). The world has many conventional, perfectly clear ways to spell this kind of thing already. The only thing stopping Python from adopting one is Keyword Fear (which I share, but not to the extent of eternal paralysis <0.9 wink>). > ... > Oh, much simpler: We have it all ready :) > > Just make "while 1:" into a keyword, like the #end block delimiters. It's a needless initial barrier to language acceptance, but more importantly it's plainly bad pedagogics to model a "loop and a half" via a construct that *says* "do this while true", i.e. do this forever. Not a disaster, but surely deserving of the "wart" Andrew gives it. at-heart-it's-a-simple-problem-with-a-simple-fix-ly y'rs - tim From tim_one@email.msn.com Sat Feb 5 08:01:34 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 5 Feb 2000 03:01:34 -0500 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: Message-ID: <001a01bf6faf$384b92a0$182d153f@tim> [Greg Wilson] > (Hope no-one minds me keeping this thread alive --- as I said in my first > reply to Tim Peters, there's either something very fundamental here, or a > "just-so" story...) Fundamental but not obvious, and possibly a matter of taste. I can only repeat myself at this point, but I'll try minor rewording : I write my code deliberately to follow the guidelines I mentioned (tuples for fixed heterogenous products, lists for indefinite homogenous sequences). Perhaps I see it that way because I love Haskell too, where those "guidelines" are absolute requirements (btw, is Haskell being silly here too in your view?). In Python, I find that following them voluntarily is a truly effective aid to both reasoning and clarity. Give it a try! The distinction between ints and floats is much more a "just so" story to me: your students never questioned it because their previous languages (Fortran and C++ and ...) told them the same story. Now they suck on it for comfort . But, e.g., Perl got along fine for years without a distinct "int" type, and added one (well, added a funky "use int" pragma) purely for optimization. At the language level there's really little sense to this distinction -- it's "play nice with the guts of the machine" cruft. Now given Python's use to script various C interfaces more or less directly, I'd actually be loathe to see Python give up the distinction entirely. But, if you think about it *hard* (accept my "start from ground zero" invitation), I expect you'll find there's far less justification for it than you may currently believe. Heck, floating point is even faster than ints on some platforms . > ... > If tuples didn't already exist, would anyone ask for them to > to be added to the language today? I probably would, because I grew to like the distinction so much in Haskell, and would *expect* the Haskell benefits to carry over to Python as well. Note that I've never made the "dict key" argument here, because I don't think it's fundamental. However, if you hate tuples you're going to have to come up with a reasonable alternative (if that's the deepest use you can see for them now, fine, then at least address it for real *at* that level ...). > Show them [1, "two"] and they (a) understand it, and (b) think > it's cool; show them (1, "two") as well and they become confused. So don't show people [1, "two"] at first <0.5 wink>. > ... > I've never had any trouble explaining int vs. float to students at any > level; I've also never had any trouble explaining int vs. long (memory > vs. accuracy). That last tradeoff is an artifact of the current implementation; there's no fundamental reason for this tension. Python already has different concrete implementations of a single "integer" interface, and essentially the only things needed to integrate int and long fully are changing the literal parsers to ignore "L", and changing the guts of the "if (overflow) {}" bits of intobject.c to return a long instead of raising an exception (a nice refinement would be also to change the guts of longobject.c to return an int for "small" longs). Note that, e.g., high end HP calculators use about a dozen(!) different internal representations for its one visible "number" type (to save precious space), and users aren't even aware of this. It's an old implementation trick. and-a-good-one-ly y'rs - tim From mal@lemburg.com Sat Feb 5 11:10:46 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 05 Feb 2000 12:10:46 +0100 Subject: [Python-Dev] Naming rules for function types References: Message-ID: <389C0536.ABC057F2@lemburg.com> Moshe Zadka wrote: > > In Include/object.h, around line 140, many pointer to functions are > typedef'ed. I need a new function type: > > int (*f)(PyObject *, PyObject *) > > And I'm at a bit of a los what to call it. > Anyone has any idea what the naming convention is? > > Is objobjproc the right thing? I'd suggest creating a new type which is related to your particular need rather than coming up with a new generic name. The generic names bypass type safety. Since you probably want this for the __contains__ slot, how about "containsfunc" ? -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Sat Feb 5 11:40:46 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 05 Feb 2000 12:40:46 +0100 Subject: [Python-Dev] Coercion and slots handling mixed types Message-ID: <389C0C3E.37BD4CDD@lemburg.com> Greetings everybody, During the course of implementing the Unicode proposal we discussed here in early December I stumbled into an old problem which has caused me some troubles already in the early stages of writing mx.DateTime: Most of the C slot functions are not capable of dealing with mixed type arguments. Even worse, most expect to be passed "their" type of arguments without even checking them. In Python things are much better: you have the __r***___ methods, can do mixed type arguments, etc. In short: we need the same things for the C level ! To start this discussion, I would like to point you to an old patch I wrote for Python 1.5. It focusses mainly on the numeric slots, but does provide some insights into possible ways out of the current problems with C level coercions (the business of dealing with mixed type argument): http://starship.skyport.net/~lemburg/CoercionProposal.html Here are some starters for the discussion. I think we need: · slots which are able to deal with mixed type argument · slots which do not expect to find a certain type without checking for it · coercion implemented at two levels: 1. a generic mixed type abstract operation mechanism (such as the one included in the above patch for numeric types) 2. slots which can handle mixed type argument and have a way to signal "I can't handle this type combination" (the patch uses a special singleton for this instead of exceptions because the latter cause too much of a perfomance hit -- yes I've tested this). · all of the above for the numeric slots as well as most of the other slots dealing with PyObject* arguments Note that these things are also needed for David Ascher's rich comparisons. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Sat Feb 5 11:55:23 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 05 Feb 2000 12:55:23 +0100 Subject: [Python-Dev] Python Unicode Implementation Message-ID: <389C0FAB.242C4418@lemburg.com> Guido already said it all: Python is going Unicode in 1.6 and things are moving fast into that direction. But... to make the integration rock solid, I need your participation: please test the patch in every way you can so that I can get it ready for prime time as soon as possible. I am most interested in hearing about its behaviour in mixed string/Unicode situations. The idea behind the implementation is that strings and Unicode should inter- operate to the greatest extent possible. Note that the patch also includes a completely new core feature: the codec APIs and registry. You can write stackable streams, data converters, etc. in a generic way and use them throughout Python. This should be especially interesting for people in the web business who want to provide content using many different encodings, e.g. Latin-1, UTF-8, etc. Encryption and compression codecs should also be possible using the new APIs. To make things more convenient for you, I'll also put a complete Python source distribution on the web page for you to download -- this eliminates the need to patch the CVS sources. Thanks, -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Sat Feb 5 12:59:50 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 05 Feb 2000 13:59:50 +0100 Subject: [Python-Dev] Python Unicode Implementation References: <389C0FAB.242C4418@lemburg.com> Message-ID: <389C1EC6.4436F9A9@lemburg.com> "M.-A. Lemburg" wrote: > > To make things more convenient for you, I'll also put a > complete Python source distribution on the web page for > you to download -- this eliminates the need to patch the > CVS sources. This is now available from the web site. Along with a slightly new version of the patch set. I will upload new versions as they come along... -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From gvwilson@nevex.com Sat Feb 5 15:24:50 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Sat, 5 Feb 2000 10:24:50 -0500 (EST) Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: <001a01bf6faf$384b92a0$182d153f@tim> Message-ID: [Question for Guido at the bottom of the message...] > Tim Peters wrote: > I write my code deliberately to follow the guidelines I mentioned > (tuples for fixed heterogenous products, lists for indefinite > homogenous sequences). Perhaps I see it that way because I love > Haskell too, where those "guidelines" are absolute requirements (btw, > is Haskell being silly here too in your view?). In Python, I find that > following them voluntarily is a truly effective aid to both reasoning > and clarity. I'm a big fan of Haskell; if Python enforced the distinction you've made, I would probably never have questioned it. However: 1) As long as it's just a convention, that only a handful of people strictly conform to, it's a pedagogic wart --- every Python book or tutorial I've read spends at least a paragraph or two justifying tuples' existence. 2) I tried in my first class at LANL to say "tuples are like records". One guy put his hand up and said, "Records that you access using integer indices instead of names? *laugh* Well, it's good to see that Fortran-66 is alive and well!" *general laughter* The point is a serious one --- Pascal taught us to give meaningful names to the fields in structures, and then tuples take us back to "oh, I think the day of the month is in the fourth field --- or is it the fifth?" > The distinction between ints and floats is much more a "just so" story > to me: your students never questioned it because their previous > languages (Fortran and C++ and ...) told them the same story. That's part of it --- but again, I think the Logo community found that novice non-programmers understood "whole numbers vs. fractions" without any trouble. Don't remember if rounding (assigning float to int) was a problem or not; I'll ask Brian H. > > If tuples didn't already exist, would anyone ask for them to > > to be added to the language today? > I probably would, because I grew to like the distinction so much in > Haskell, and would *expect* the Haskell benefits to carry over to > Python as well. Given enforced typing (fixed-length heterogeneous vs. variable-length homogeneous), I'd agree. Guido, if you're still on this thread, can you please tell us about the history here --- were list and tuples both put into the language with this kind of distinction in mind? Thanks for your patience, Greg From gmcm@hypernet.com Sat Feb 5 16:11:11 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Sat, 5 Feb 2000 11:11:11 -0500 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: References: <001a01bf6faf$384b92a0$182d153f@tim> Message-ID: <1262355436-5521195@hypernet.com> Greg Wilson wrote: > 1) As long as it's just a convention, that only a handful of people > strictly conform to, it's a pedagogic wart --- every Python book > or tutorial I've read spends at least a paragraph or two justifying > tuples' existence. > > 2) I tried in my first class at LANL to say "tuples are like records". > One guy put his hand up and said, "Records that you access using > integer indices instead of names? *laugh* Well, it's good to see > that Fortran-66 is alive and well!" *general laughter* > > The point is a serious one --- Pascal taught us to give meaningful > names to the fields in structures, and then tuples take us back to > "oh, I think the day of the month is in the fourth field --- or is > it the fifth?" When this comes up from newbies on the list (which is *much* less often than a number of other so-called warts), I explain the difference, and then say, "If you don't know which to use, use a list. One day it will become obvious." Now, experience (not a priori reasoning) tells me that this is safe: x, y = tpl and this is not: x, y = lst There's not much use in arguing about it, because both require trust in the programmer. It's just the in the first case, you *can* trust the programmer, and in the second you *can't*. Even when the programmer is yourself. The fact that you don't like "zen" arguments doesn't mean you have to make them. Don't defend it at all. Just point out that most Python programmers consider tuples very valuable and move on. In general, it's very hard to "defend" Python on theoretical grounds. The newsgroup is littered with posts from OO cultists berating Python for it's primitive object model. They either move on, or shut up once they realize that Python's object model is a lot cleaner *in practice* than the theoretically correct crap they cut their teeth on. (What astounds me is the number of functional programmers who are sure that Python is modeled after a functional language.) - Gordon From esr@thyrsus.com Sat Feb 5 16:42:25 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Sat, 5 Feb 2000 11:42:25 -0500 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: <1262355436-5521195@hypernet.com>; from Gordon McMillan on Sat, Feb 05, 2000 at 11:11:11AM -0500 References: <001a01bf6faf$384b92a0$182d153f@tim> <1262355436-5521195@hypernet.com> Message-ID: <20000205114225.A6375@thyrsus.com> Gordon McMillan : > (What astounds me is the number of functional programmers > who are sure that Python is modeled after a functional > language.) Speaking as a functional programmer, it's always been quite clear to me that Python was *not* modeled after a functional language. OTOH, it resembles one sometimes because here are certain functional notions that *must* be covered by any language with a sufficiently broad expressive range -- and Guido was certainly trying for broad expressive range. -- Eric S. Raymond "A system of licensing and registration is the perfect device to deny gun ownership to the bourgeoisie." -- Vladimir Ilyich Lenin From tismer@tismer.com Sat Feb 5 16:57:47 2000 From: tismer@tismer.com (Christian Tismer) Date: Sat, 05 Feb 2000 17:57:47 +0100 Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing class instances) References: <200002042109.WAA12722@python.inrialpes.fr> Message-ID: <389C568B.6CEBAC05@tismer.com> Howdy, lemme try... Vladimir Marangozov wrote: > > [Jim, Chris & Guido discussing a namespace idea] > > Guys, I'm lost. > > Please help me understanding this idea from the start. > > After rereading this whole thread, I have only a vague intuition of > what Jim has proposed, but I fail to understand it; and believe me, > I'm very interested in being in sync with you on the subject. > > Please filter the concept from the consequences and resubmit it > once again (in english, through examples, ascii art, whatever). Naming it different than before, I think this formulation hits the nail on its top: Jim proposes a construction that yields early binding of names, while late binding of values. Sample layout of an association object: with the same key semantics as for dicts. The key object is assigned when the association object is created, that is when the name is seen the first time. The value is still NULL until assigned. Now assume a namespace object as a collection of pointers to asso objects, and assume that it is only extended, nothing deleted. Then, a code object can refer to such a namespace object by giving the index of the asso object. Since the construction of these objects will occour in the same order after marshal/pickling, the offsets in a code object will always be correct. This means: There is a construction that allows to settle a name as soon as it is seen, without necessarily assigning a value. When a function at compile time sees a global, it tries to resolve it by finding an association object in the module's global scope that contains this name. If not found, it is created. From now on, the function still doesn't know the value, but the name has been resolved already, and it is clear where the value slot will be, later. There is no more lookup necessary. Jim proposed this primarily to make it easier to reload modules, after they have been used in "from module import" style. The "from" imported objects would not be inserted into the module's dict as new key/value entry, but instead a namespace object would be inserted that adds another level of indirection here. If the value of module "module"'s object "a" in "from module import a" changes, this change would also be visible in our client module. While the general idea of early name/late value binding seems to be a promising concept to speed up/avoid dicionary lookups in certain places, Guido has already pointed out that the proposed semantic change to "from .. import" seems to have hard to explain effects and is not considered a good thing. I agree that the idea is hard to defend for the import case. For classes and instances, I believe it can make attribute lookups very fast (saving dictionary lookups in almost all cases), while the space requirement can be made equal to dictionaries. hoping it was clear enough - ciao - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Düppelstr. 31 : *Starship* http://starship.python.net 12163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From akuchlin@mems-exchange.org Sat Feb 5 22:01:56 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Sat, 5 Feb 2000 17:01:56 -0500 (EST) Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: <001a01bf6faf$384b92a0$182d153f@tim> References: <001a01bf6faf$384b92a0$182d153f@tim> Message-ID: <14492.40404.735599.774901@newcnri.cnri.reston.va.us> Tim Peters writes: >fundamental reason for this tension. Python already has different concrete >implementations of a single "integer" interface, and essentially the only >things needed to integrate int and long fully are changing the literal >parsers to ignore "L", and changing the guts of the "if (overflow) {}" bits >of intobject.c to return a long instead of raising an exception (a Not quite *that* simple; you'd also have to change various bits of the core that currently do PyInt_Check(whatever) to also accept longs. I just sent off a patch to make long*sequence legal, but there's still slicing and indexing to take care of: list[0L:5L] isn't currently legal. The Solaris large-file patch that makes .tell() return a long is probably going to turn up more errors like this. Making ints and long ints integerchangeable would be an *excellent* idea. Possible for 1.6? Maybe, if GvR indicates it should be a priority. 'grep -l PyInt_Check' over the Python source code lists lots of Modules, most of the files in Objects/, and 5 files in Python/ (ceval, marshal, pythonrun, structmember, and traceback). -- A.M. Kuchling http://starship.python.net/crew/amk/ "Doctor, we did good, didn't we?" "Perhaps. Time will tell. Always does." -- Ace and the Doctor, in Ben Aaronovitch's _Remembrance of the Daleks_ From da@ski.org Sat Feb 5 22:57:29 2000 From: da@ski.org (David Ascher) Date: Sat, 5 Feb 2000 14:57:29 -0800 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" References: <001a01bf6faf$384b92a0$182d153f@tim> <1262355436-5521195@hypernet.com> Message-ID: <011d01bf702c$6102d230$0100000a@ski.org> From: Gordon McMillan > The fact that you don't like "zen" arguments doesn't mean you > have to make them. Don't defend it at all. Just point out that > most Python programmers consider tuples very valuable and > move on. Doesn't really work with argumentative, opinionated and poorly informed students, which are majority in the computer field =). > In general, it's very hard to "defend" Python on theoretical > grounds. The point is not theoretical grounds, but, at least in my case, backing up the claim that Python has an elegant, spare design. Tuples show up as challenging that claim, as do some of the other warts on AMK's pages. I expect some of those to go away naturally (e.g. apply(Base.__init__, (self,)+args, kw) will mutate naturally to: Base.__init__(self, *args, **kw) but the tuples will stay. I'm just looking for a better pedagogical trick, not arguing against them on theoretical grounds. --david From mhammond@skippinet.com.au Sun Feb 6 00:35:41 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Sun, 6 Feb 2000 11:35:41 +1100 Subject: [Python-Dev] win32 specific exception in the core? Message-ID: As part of the registry work, Im toying with the idea of a standard "win32 exception" error in the Python core. The reasons why I think a standard exception would be useful are: * Single body of code that knows how to raise these exceptions. Eg, whenever an extension module needs to report that a Win32 API function failed, they can use the core routines to create the exception object - just like they can with OSError. This ensures consistency, and saves each extension module author from reinventing the wheel. * No confusion about what exception Python programmers should catch. In the same way that "OSError" will be thrown any time the C runtime library fails, the "Win32Error" exception will be thrown whenever a Win32 API function (called directly) fails. This has come up now that I am looking at a "win32reg" (or "_win32reg") module for inclusion in Python 1.6. It makes sense to me that rather than creating specific exceptions for each new module of this ilk, we define a single exception able to be shared by any core or 3rd party module that needs it. Ideally, this exception should derive from "EnvironmentError" - however, this is pretty specific to the OSError mechanics. The other alternatives are to create a Win32Error that derives directly from "StandardError", or to introduce an intermediate "PlatformError". The Win32 extensions have been doing this for ages (ie, every win32*.pyd module shares and raises the same exception object) and IMO it is a good thing. If we do go this route, the Win32 exceptions could also share this exception object, which is, IMO, also a good thing (in the same way that extension modules now can already raise OSError without a need to redefine the semantics each time) Does anyone have thoughts on this? Is it a good thing or a bad thing? Have-I-made-the-comparison-to-OSError-enough?-ly, Mark. From gstein@lyra.org Sun Feb 6 00:37:14 2000 From: gstein@lyra.org (Greg Stein) Date: Sat, 5 Feb 2000 16:37:14 -0800 (PST) Subject: [Python-Dev] longs, too? (was: syntax - "Aren't tuples redundant?") In-Reply-To: <14492.40404.735599.774901@newcnri.cnri.reston.va.us> Message-ID: On Sat, 5 Feb 2000, Andrew Kuchling wrote: >... > Not quite *that* simple; you'd also have to change various bits of the > core that currently do PyInt_Check(whatever) to also accept longs. I > just sent off a patch to make long*sequence legal, but there's still > slicing and indexing to take care of: list[0L:5L] isn't currently > legal. > > The Solaris large-file patch that makes .tell() return a long is > probably going to turn up more errors like this. Making ints and long > ints integerchangeable would be an *excellent* idea. Possible for > 1.6? Maybe, if GvR indicates it should be a priority. 'grep -l > PyInt_Check' over the Python source code lists lots of Modules, most > of the files in Objects/, and 5 files in Python/ (ceval, marshal, > pythonrun, structmember, and traceback). Priority or not, it won't happen if the patch is not available :-) As we say in Apache-land, "+1 on concept" Cheers, -g ps. Python-land translation: I agree with the concept ... go for it -- Greg Stein, http://www.lyra.org/ From gvwilson@nevex.com Sun Feb 6 01:39:10 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Sat, 5 Feb 2000 20:39:10 -0500 (EST) Subject: [Python-Dev] re: scientific Python software Message-ID: Hi, folks. A couple of people have suggested that I ought to start building up a page full of links to Python modules (and Python-friendly packages like VTK) that would be of interest to scientific and engineering programmers. It could eventually be part of the new-look python.org, stay at software-carpentry.com, or...? Pointers much appreciated... Thanks, Greg From akuchlin@mems-exchange.org Sun Feb 6 02:19:06 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Sat, 5 Feb 2000 21:19:06 -0500 (EST) Subject: [Python-Dev] re: scientific Python software In-Reply-To: References: Message-ID: <14492.55835.1198.459734@newcnri.cnri.reston.va.us> gvwilson@nevex.com writes: >Hi, folks. A couple of people have suggested that I ought to start >building up a page full of links to Python modules (and Python-friendly >packages like VTK) that would be of interest to scientific and engineering >programmers. It could eventually be part of the new-look python.org, stay David Ascher's topic guide for scientific programming is at http://www.python.org/topics/scicomp/ . I don't know if David still has time to maintain it, or interest in doing so; perhaps you could ask to take it over. -- A.M. Kuchling http://starship.python.net/crew/amk/ "Are you OK, dressed like that? You don't seem to notice the cold." "I haven't come ten thousand miles to discuss the weather, Mr Moberly." -- Moberly and the Doctor, in "The Seeds of Doom" From da@ski.org Sun Feb 6 03:17:32 2000 From: da@ski.org (David Ascher) Date: Sat, 5 Feb 2000 19:17:32 -0800 Subject: [Python-Dev] re: scientific Python software References: <14492.55835.1198.459734@newcnri.cnri.reston.va.us> Message-ID: <017401bf7050$b5547950$0100000a@ski.org> > David Ascher's topic guide for scientific programming is at > http://www.python.org/topics/scicomp/ . I don't know if David still > has time to maintain it, or interest in doing so; perhaps you could > ask to take it over. If you want it, Greg, it's yours. Interest, of course, time, no way. --da From ping@lfw.org Sun Feb 6 21:03:02 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Sun, 6 Feb 2000 13:03:02 -0800 (PST) Subject: [Python-Dev] Re: [Edu-sig] Rational Division In-Reply-To: Message-ID: This is from a response to a question about rational division on the Edu-SIG list. On Sat, 5 Feb 2000, Ka-Ping Yee wrote: > > In E, there are two operators for division: > > 2 / 3 yields 0.666666 > > 2 _/ 3 yields 0 > > Read "_/" as "floor-divide". > (See http://www.eright.org/elang/grammar/expr.html.) > > Note that in the current Python, if a and b are integers and > f is the floating-point result of dividing a by b, then > a/b is *not* equal to int(f). It's equal to math.floor(f). Then i realized that math.floor returns a float. So, really, in order to replace the old /, a / b becomes import math int(math.floor(a / b)) As long as we're going to push forward into the realm of sanity (and i'm in support of this change!) do we need an easier way to spell this to help folks convert? I rather like E's "_/" operator, but that may be harder to swallow here since "_" goes in Python identifiers. E allows "_" in identifiers too, but you can use spaces to avoid ambiguity, e.g.: a _/ b # parses as 'a' floor-divide 'b' a_/b # parses as 'a_' float-divide 'b' But perhaps that is just too arcane for Python. "//" was also considered for a floor-divide operator for a while, but Java-style comments won out. So do we go for a new operator, or put a 'div()' in __builtins__, or what? > int() truncates, always rounding towards zero (an abomination > in my opinion!), while floor() rounds down. This is also possibly an Edu-SIG question, but don't you think it would be nice to have a round() built-in? I can see it being invaluable for classroom and scientific work. Something like: def round(x): return int(math.floor(x + 0.5)) or even: def round(x, unit=1): return int(math.floor(x / unit + 0.5))*unit -- ?!ng "If I have not seen as far as others, it is because giants were standing on my shoulders." -- Hal Abelson From tim_one@email.msn.com Mon Feb 7 06:42:16 2000 From: tim_one@email.msn.com (Tim Peters) Date: Mon, 7 Feb 2000 01:42:16 -0500 Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: Message-ID: <001001bf7136$78f41ce0$152d153f@tim> [Mark Hammond] > As part of the registry work, Im toying with the idea of a > standard "win32 exception" error in the Python core. Sounds good to me, but if & only if it's a subclass of OSError (why? because nothing else makes sense ). I added UnboundLocalError as a subclass of NameError (in the CVS tree), so you can mimic what that did. IIRC, for the benefit of the old "string-based exceptions" option hack, UnboundLocalError reverts to NameError if class-based exceptions are disabled ... From tim_one@email.msn.com Mon Feb 7 06:42:04 2000 From: tim_one@email.msn.com (Tim Peters) Date: Mon, 7 Feb 2000 01:42:04 -0500 Subject: [Python-Dev] Re: [Edu-sig] Rational Division In-Reply-To: Message-ID: <000a01bf7136$72459a40$152d153f@tim> [Ping] > ... > Then i realized that math.floor returns a float. > So, really, in order to replace the old /, > > a / b > > becomes > > import math > int(math.floor(a / b)) Defining this in terms of math.floor is a mistake, because long ints can easily exceed the precision of a C double. > As long as we're going to push forward into the realm > of sanity (and i'm in support of this change!) do we > need an easier way to spell this to help folks convert? We need an easier way to spell this just so that it works. > I rather like E's "_/" operator, but that may be harder > to swallow here since "_" goes in Python identifiers. > E allows "_" in identifiers too, but you can use spaces > to avoid ambiguity, e.g.: > > a _/ b # parses as 'a' floor-divide 'b' > > a_/b # parses as 'a_' float-divide 'b' > > But perhaps that is just too arcane for Python. Python uses "maximal munch" lexing, so it would naturally do these same things; but Guido has avoided *relying* on MM so far as possible (e.g., it's relied on to parse """a""" as a single (triple-quoted) string instead of as the catenation of 3 single-quoted strings). > "//" was also considered for a floor-divide operator > for a while, but Java-style comments won out. Looks natural for Python! I'm not sure what "/" should *do* in the Brave New World, though. Floating-point has been the conventional answer so far, but I'd like to take another look at what Scheme does (not sure what the std sez, but every Scheme I've ever used treated integer division as returning a rational). The *prime* motivation here seems to be that "7/3" not lose catastrophic amounts of information silently; other clear choices are "lose none" (rationals) and "maybe lose a little in a way that's very hard to explain" (floating point). > .. > This is also possibly an Edu-SIG question, but don't > you think it would be nice to have a round() built-in? Guido hopped in his time machine and added this to Python 1.0 : >>> round >>> print round.__doc__ round(number[, ndigits]) -> floating point number Round a number to a given precision in decimal digits (default 0 digits). This always returns a floating point number. Precision may be negative. >>> > I can see it being invaluable for classroom and > scientific work. Indeed . From tim_one@email.msn.com Mon Feb 7 06:42:19 2000 From: tim_one@email.msn.com (Tim Peters) Date: Mon, 7 Feb 2000 01:42:19 -0500 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: <14492.40404.735599.774901@newcnri.cnri.reston.va.us> Message-ID: <001101bf7136$7ad1b680$152d153f@tim> [Tim, blissfully minimizes the difficulties of erasing the int/long distinction] [Andrew Kuchling] > Not quite *that* simple; you'd also have to change various bits of the > core that currently do PyInt_Check(whatever) to also accept longs. I had in mind a vague scheme to cheat. I have since recovered from the delusions. > I just sent off a patch to make long*sequence legal, but there's > still slicing and indexing to take care of: list[0L:5L] isn't > currently legal. Note that MS has already decided to leave sizeof(long) == 4 in 64-bit Windows, but sizeof(void*) will jump to 8. Python is remarkably free of dubious assumptions here, but, as you point out for Solaris, "large" files are likely going to make problems here on an increasing # of platforms. > The Solaris large-file patch that makes .tell() return a long is > probably going to turn up more errors like this. Making ints and long > ints integerchangeable would be an *excellent* idea. Possible for > 1.6? Maybe, if GvR indicates it should be a priority. 'grep -l > PyInt_Check' over the Python source code lists lots of Modules, most > of the files in Objects/, and 5 files in Python/ (ceval, marshal, > pythonrun, structmember, and traceback). The idea is very old, and has come up several times, but I don't recall Guido ever saying anything about it. So it's safe to conclude it *hasn't* been a priority for him. I can't channel him on this issue. I'm personally in favor of merging them, but along with Konrad (Hinsen) am also in favor of doing a lot more "numeric merging" in Python 3000. It's really unclear to me whether the distinction in Python1 can be erased without breaking programs -- but can't make time to think about it now, either. Sorry! From esr@thyrsus.com Mon Feb 7 06:54:16 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 7 Feb 2000 01:54:16 -0500 Subject: [Python-Dev] Re: [Edu-sig] Rational Division In-Reply-To: <000a01bf7136$72459a40$152d153f@tim>; from Tim Peters on Mon, Feb 07, 2000 at 01:42:04AM -0500 References: <000a01bf7136$72459a40$152d153f@tim> Message-ID: <20000207015416.B22447@thyrsus.com> Tim Peters : > Looks natural for Python! I'm not sure what "/" should *do* in the Brave > New World, though. Floating-point has been the conventional answer so far, > but I'd like to take another look at what Scheme does (not sure what the std > sez, but every Scheme I've ever used treated integer division as returning a > rational). The *prime* motivation here seems to be that "7/3" not lose > catastrophic amounts of information silently; other clear choices are "lose > none" (rationals) and "maybe lose a little in a way that's very hard to > explain" (floating point). I agree with you in preferring the "let's do rationals" answer. Seems to me I recall one of the other points in the Alice presentation was that non-techies find floating point obscure and think of simple fractions as atoms. -- Eric S. Raymond The saddest life is that of a political aspirant under democracy. His failure is ignominious and his success is disgraceful. -- H.L. Mencken From tim_one@email.msn.com Mon Feb 7 08:03:55 2000 From: tim_one@email.msn.com (Tim Peters) Date: Mon, 7 Feb 2000 03:03:55 -0500 Subject: [Python-Dev] Re: [Edu-sig] Rational Division In-Reply-To: <20000207015416.B22447@thyrsus.com> Message-ID: <001301bf7141$e1f1db00$152d153f@tim> [posted & mailed] [Tim, on integer division] > ... I'd like to take another look at what Scheme does ... I have since been reminded that the Scheme std isn't helpful: may return a rational, may return a float, may return darn near anything at all -- and may even vary depending on the specific arguments. There's no language std I know of more carefully crafted to ensure programs will be unportable except for Fortran's <1/4 wink>. > The *prime* motivation here seems to be that "7/3" not lose > catastrophic amounts of information silently; other clear > choices are "lose none" (rationals) and "maybe lose a little > in a way that's very hard to explain" (floating point). [Eric S. Raymond] > I agree with you in preferring the "let's do rationals" answer. Seems > to me I recall one of the other points in the Alice presentation was that > non-techies find floating point obscure and think of simple fractions > as atoms. I didn't say what I preferred . Seriously, Guido & I corresponded about Python's numerics when the language was being designed, and agreed that ABC's default use of rationals didn't work out well: "simple little" numeric programs suffered wildly unpredictable space and time use (rationals can "blow up" very quickly). Have to say that hasn't been my experience in other languages, though -- so I want to go back & see if there was something else about ABC that conspired to produce this unhappy outcome. Offhand, I seem to recall that fp in ABC had a "tacked on" look & feel that made it clumsy to get at; or maybe the docs just sucked; or maybe ... The horrid thing about fp is that it gets *more* obscure the more you learn about it -- until reaching "fp expert" level, which is a journey that takes years, deliberate study, and lots of numeric scars. REXX's decimal fp is also an idea (it's my impression that most newbie problems with fp are due to the "accidental complexity" introduced by using bounded binary fractions to represent bounded decimal fractions: this loses info for reasons that "make no sense" in a world of decimal hand calculators and paper arithmetic). we-can't-win-but-we-can-choose-how-to-lose-ly y'rs - tim From fredrik@pythonware.com Mon Feb 7 08:25:13 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Mon, 7 Feb 2000 09:25:13 +0100 Subject: [Python-Dev] win32 specific exception in the core? References: Message-ID: <004c01bf7144$dc139950$f29b12c2@secret.pythonware.com> Mark Hammond wrote: > As part of the registry work, Im toying with the idea of a standard = "win32 > exception" error in the Python core. >=20 > The reasons why I think a standard exception would be useful are: >=20 > * Single body of code that knows how to raise these exceptions. Eg, > whenever an extension module needs to report that a Win32 API function > failed, they can use the core routines to create the exception object = - just > like they can with OSError. This ensures consistency, and saves each > extension module author from reinventing the wheel. >=20 > * No confusion about what exception Python programmers should catch. = In the > same way that "OSError" will be thrown any time the C runtime library = fails, > the "Win32Error" exception will be thrown whenever a Win32 API = function > (called directly) fails. any reason you can't just throw OSError exceptions? (after all, Windows is an operating system, right? ;-) From mhammond@skippinet.com.au Mon Feb 7 08:25:03 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Mon, 7 Feb 2000 19:25:03 +1100 Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: <004c01bf7144$dc139950$f29b12c2@secret.pythonware.com> Message-ID: Fredrik writes: > any reason you can't just throw OSError exceptions? Mainly cos the error numbers are different - (eg, compare "errno.h" with "winerror.h" OSError really should be named CRTError. I dont think it makes much sense to have 2 different error numbering schemes use the same exception... I guess we _could_ tho - I still lean towards a new one, but other opinions would be good... Mark. From Moshe Zadka Mon Feb 7 08:52:00 2000 From: Moshe Zadka (Moshe Zadka) Date: Mon, 7 Feb 2000 10:52:00 +0200 (IST) Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: Message-ID: On Mon, 7 Feb 2000, Mark Hammond wrote: > Mainly cos the error numbers are different - (eg, compare "errno.h" with > "winerror.h" OSError really should be named CRTError. I dont think it makes > much sense to have 2 different error numbering schemes use the same > exception... > > I guess we _could_ tho - I still lean towards a new one, but other opinions > would be good... Um...my Windows programing experience is thankfully little, but I seem to remember every "big collection" of function works with a different error handler (the sockets with WSA...error...() is the example which comes to mind). The OSError is very UNIX-oriented, in that it assumes a monolithic OS with consistent errors. This problem will probably also come up when (in a century or so) I manage to get Python working on Hurd. How about, for windows, to derive "many" error classes from OSError on Win32, and always return the right one? On the other hand, Mark will correct me if I'm talking nonsense Windows-wise -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From tim_one@email.msn.com Mon Feb 7 09:11:02 2000 From: tim_one@email.msn.com (Tim Peters) Date: Mon, 7 Feb 2000 04:11:02 -0500 Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: Message-ID: <000001bf714b$4152e220$712d153f@tim> [/F] > any reason you can't just throw OSError exceptions? [Mark] > Mainly cos the error numbers are different - (eg, compare "errno.h" with > "winerror.h" OSError really should be named CRTError. There's nothing to stop a new CRTError class being derived from OSError, leaving OSError platform-neutral. This would just be vanilla OO refactoring in the face of new requirements, and existing code wouldn't know the difference. You should subclass from OSError so that code that cares about portability can catch OSError specifically instead of needing to be modified every time someone with a new OS wants to add a class of error specific to that OS. > I dont think it makes much sense to have 2 different error numbering > schemes use the same exception... No, I agree that does not. But it makes perfect sense for a subclass to override its parent's numbering scheme (and even more sense to make "numbering scheme" an abstract property of the parent to be implemented by its subclasses). From mhammond@skippinet.com.au Mon Feb 7 10:56:51 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Mon, 7 Feb 2000 21:56:51 +1100 Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: Message-ID: [Moshe writes:] > How about, for windows, to derive "many" error classes from OSError on > Win32, and always return the right one? Another alternative would be to derive from EnvironmentError. I would then like to change the __str__ method from: ... return '[Errno %s] %s: %s' % (self.errno, self.strerror, ... to return '[%s %s] %s: %s' % (self.getdesc(), self.errno, self.strerror, Where getdesc() returns "OS Error", "IO Error" or "Win32 error" respectively. This all comes down to Guido tho. I believe he is away, so if he doesnt respond Ill simply send him a summary of the discussions and he can decide lickety-split! Mark. From gvwilson@nevex.com Mon Feb 7 12:36:30 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Mon, 7 Feb 2000 07:36:30 -0500 (EST) Subject: [Python-Dev] Re: [Edu-sig] Rational Division (fwd) Message-ID: [Brian Harvey is the author of "Computer Science Logo Style", and a long-time Logophile and educator] ---------- Forwarded message ---------- Date: Sun, 6 Feb 2000 17:21:44 -0800 (PST) From: Brian Harvey To: gvwilson@nevex.com Subject: Re: [Python-Dev] Re: [Edu-sig] Rational Division Hi. There were some early Logo versions in which 2/3 was 0, and 2.0/3.0 was 0.6666, but sure enough, everyone was confused. These days 2/3 is 0.6666, and there's an INT operation that truncates to integer. Oh, there was a later time when 2/3 was 0.6666 but QUOTIENT 2 3 was 0. That confused people too. I think some dialects have an INTQUOTIENT operation that's the same as INT QUOTIENT. :-) From mal@lemburg.com Mon Feb 7 15:19:55 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 07 Feb 2000 16:19:55 +0100 Subject: [Python-Dev] New Unicode Snapshot Message-ID: <389EE29B.52B48DED@lemburg.com> Hi everybody, I've just uploaded a new Unicode snapshot. It includes a brand new UTF-16 codec which is BOM mark aware, meaning that it recognizes BOM marks on input and adjusts the byte order accordingly. On output you can choose to have BOM marks written or specifically define a byte order to use. Also new in this snapshot is configuration code which figures out the byte order on the installation machine... I looked everywhere in the Python source code but couldn't find any hint whether this was already done in some place, so I simply added some autoconf magic to have two new symbols defined: BYTEORDER_IS_LITTLE_ENDIAN and BYTEORDER_IS_BIG_ENDIAN (mutually exclusive of course). BTW, I changed the hash method of Unicode objects to use the UTF-8 string as basis for the hash code. This means that u'abc' and 'abc' will now be treated as the same dictionary key ! Some documentation also made into the snapshot. See the file Misc/unicode.txt for all the interesting details about the implementation. Note that the web page provides a prepatched version of the interpreter for your convenience... just download, run ./configure and make and your done. Could someone with access to a MS VC compiler please update the project files and perhaps post me some feedback about any glitches ?! I have never compiled Python on Windows myself and don't have the time to figure out just now :-/. Thanks :-) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Mon Feb 7 15:26:04 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 7 Feb 2000 10:26:04 -0500 (EST) Subject: [Python-Dev] win32 specific exception in the core? References: <001001bf7136$78f41ce0$152d153f@tim> Message-ID: <14494.58380.75343.607049@anthem.cnri.reston.va.us> >>>>> "TP" == Tim Peters writes: TP> Sounds good to me, but if & only if it's a subclass of OSError I agree (and I think Mark's been given some ideas about how to accomplish this). TP> (why? because nothing else makes sense ). I added TP> UnboundLocalError as a subclass of NameError (in the CVS TP> tree), so you can mimic what that did. IIRC, for the benefit TP> of the old "string-based exceptions" option hack, TP> UnboundLocalError reverts to NameError if class-based TP> exceptions are disabled ... I wouldn't worry about string-based standard exceptions. Didn't we decide to get rid of these (and -X) for 1.6 at IPC8? -Barry From Moshe Zadka Mon Feb 7 15:26:20 2000 From: Moshe Zadka (Moshe Zadka) Date: Mon, 7 Feb 2000 17:26:20 +0200 (IST) Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: <14494.58380.75343.607049@anthem.cnri.reston.va.us> Message-ID: On Mon, 7 Feb 2000, Barry A. Warsaw wrote: > I wouldn't worry about string-based standard exceptions. Didn't we > decide to get rid of these (and -X) for 1.6 at IPC8? > In that case, installations should freeze exceptions.py into the binary, otherwise all hell *will* break loose... -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From fdrake@acm.org Mon Feb 7 15:41:33 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 7 Feb 2000 10:41:33 -0500 (EST) Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: <001001bf7136$78f41ce0$152d153f@tim> References: <001001bf7136$78f41ce0$152d153f@tim> Message-ID: <14494.59309.967871.506115@weyr.cnri.reston.va.us> Tim Peters writes: > IIRC, for the benefit of the old "string-based exceptions" option hack, > UnboundLocalError reverts to NameError if class-based exceptions are > disabled ... Which reminds us once again of the old -X problem. Will this still be present in 1.6? I'd vote to remove it if this weren't a dictatorship. I don't see that it serves any useful purpose at this point. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From Moshe Zadka Mon Feb 7 15:52:48 2000 From: Moshe Zadka (Moshe Zadka) Date: Mon, 7 Feb 2000 17:52:48 +0200 (IST) Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: <14494.59309.967871.506115@weyr.cnri.reston.va.us> Message-ID: On Mon, 7 Feb 2000, Fred L. Drake, Jr. wrote: > Which reminds us once again of the old -X problem. Will this still > be present in 1.6? I'd vote to remove it if this weren't a > dictatorship. I don't see that it serves any useful purpose at this > point. Don't think of it as "-X", think of it as "could not find exceptions.py, reverting to string exceptions" (or whatever the error message is). Be sure it won't change to "could not find exceptions.py, I don't think I feel like running" -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From guido@python.org Mon Feb 7 16:08:06 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 07 Feb 2000 11:08:06 -0500 Subject: [Python-Dev] re: syntax - "Aren't tuples redundant?" In-Reply-To: Your message of "Mon, 07 Feb 2000 01:42:19 EST." <001101bf7136$7ad1b680$152d153f@tim> References: <001101bf7136$7ad1b680$152d153f@tim> Message-ID: <200002071608.LAA17904@eric.cnri.reston.va.us> > > The Solaris large-file patch that makes .tell() return a long is > > probably going to turn up more errors like this. Making ints and long > > ints integerchangeable would be an *excellent* idea. Possible for > > 1.6? Maybe, if GvR indicates it should be a priority. 'grep -l > > PyInt_Check' over the Python source code lists lots of Modules, most > > of the files in Objects/, and 5 files in Python/ (ceval, marshal, > > pythonrun, structmember, and traceback). > > The idea is very old, and has come up several times, but I don't recall > Guido ever saying anything about it. So it's safe to conclude it *hasn't* > been a priority for him. I can't channel him on this issue. I'm personally > in favor of merging them, but along with Konrad (Hinsen) am also in favor of > doing a lot more "numeric merging" in Python 3000. It's really unclear to > me whether the distinction in Python1 can be erased without breaking > programs -- but can't make time to think about it now, either. Sorry! I'm not sure I want to make this a priority given the accellerated 1.6 schedule, but I certainly think this is the way of the future, and I don't expect many backwards compatibility problems... --Guido van Rossum (home page: http://www.python.org/~guido/) From gmcm@hypernet.com Mon Feb 7 16:16:56 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Mon, 7 Feb 2000 11:16:56 -0500 Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: References: <14494.58380.75343.607049@anthem.cnri.reston.va.us> Message-ID: <1262182293-3258653@hypernet.com> Moshe Zadka wrote: > On Mon, 7 Feb 2000, Barry A. Warsaw wrote: > > > I wouldn't worry about string-based standard exceptions. Didn't we > > decide to get rid of these (and -X) for 1.6 at IPC8? > > > > In that case, installations should freeze exceptions.py into the binary, > otherwise all hell *will* break loose... I think we got agreement on that too (in some import related thread on this list). Hmmm, well I can't find exceptions.py mentioned specifically. But I do find the BDFL advocating writing as much as possible of the start-up code in Python (eg, the equivalent of getpath.c) and freezing it in. In his absence, it makes sense for us to include exceptions.py in that statement . - Gordon From jeremy@cnri.reston.va.us Mon Feb 7 16:49:46 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Mon, 7 Feb 2000 11:49:46 -0500 (EST) Subject: [Python-Dev] Re: Python 2 namespace change? In-Reply-To: <389B0ACC.30071942@digicool.com> References: <200001200419.XAA01969@mira.erols.com> <38871665.C3B6FFEE@digicool.com> <38871CE5.53FB9D68@digicool.com> <200001201545.KAA21304@eric.cnri.reston.va.us> <38873C52.29FEAC6D@digicool.com> <200001201720.MAA21534@eric.cnri.reston.va.us> <3899A01E.DE29FC94@digicool.com> <200002031945.OAA29682@eric.cnri.reston.va.us> <3899E359.4ED37D66@digicool.com> <200002032107.QAA00051@eric.cnri.reston.va.us> <389AC520.96C24321@tismer.com> <200002041426.JAA09301@eric.cnri.reston.va.us> <389AE669.72A127CA@digicool.com> <200002041519.KAA09803@eric.cnri.reston.va.us> <389B0ACC.30071942@digicool.com> Message-ID: <14494.63402.397312.863803@goon.cnri.reston.va.us> >>>>> "JF" == Jim Fulton writes: JF> I'm suggesting a model where from "M import x" has a different JF> meaning than it does now. I think the notion of sharing a name JF> is useful. I'll admit that using "M.x" achieves the same thing, JF> although at a higher performance cost (and, OK, typing cost ;). This seems to contradict the 2nd Pythonic principle: Explicit is better than implicit. I don't literally mean to argue that "The Python Way" should be used to make design decisions, but it captures exactly what makes me uncomfortable with the proposed change. [someone else, who could have been channeling me, wrote:] >> I don't understand why you wanted these semantics in the first >> place. JF> First, let me say that this isn't super important to me. It Glad to hear it <0.3 wink>! JF> does solve a problem with reload, which is the context in which JF> I brought it up. I don't think the reload problem is important enough to justify a change to name binding rules. [much omitted] JF> In any case, I'd feel comfortable explaining a system in which JF> from M import x # reference semantics wrt name JF> had a different meaning from: JF> import M x=M.x # copy semantics JF> since I expect an attribute access to give me a value, not a JF> name, whereas: JF> from M import x JF> seems more to me like it's talking about names. I think the proposed change muddies the semantics of assignment, and I would not feel comfortable trying to explain it. I don't have the same impression vis a vis import and names; I think this is why I disagree and have heretofore been puzzled about why you want this. Assignment binds a name to an object; import is just a variant of assignment. There is no need for a special case. One other worry: How would it create a copy of an object in general? How do you copy a class object or a file or a socket? Since (1) you can't restrict which types of objects are exported by a module and (2) there is no clear definition of copy that applies to any type of object, I don't see how these semantics could be defined. Jeremy From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Mon Feb 7 19:07:12 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 7 Feb 2000 14:07:12 -0500 (EST) Subject: [Python-Dev] win32 specific exception in the core? References: <14494.59309.967871.506115@weyr.cnri.reston.va.us> Message-ID: <14495.6112.441151.155238@anthem.cnri.reston.va.us> >>>>> "MZ" == Moshe Zadka writes: MZ> Don't think of it as "-X", think of it as "could not find MZ> exceptions.py, reverting to string exceptions" (or whatever MZ> the error message is). Be sure it won't change to "could not MZ> find exceptions.py, I don't think I feel like running" Absolutely. If string-based standard exceptions are removed for 1.6, we need a fool-proof way of getting exceptions.py. Remember, my first attempt at this stuff made that Python code a BACS[*] that got exec'd. I'm sure there are better ways of doing that now, and I think exceptions.py has settled down enough to warrant cooling it for 1.6. -Barry [*] Big Ass C String From Moshe Zadka Mon Feb 7 22:26:55 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 8 Feb 2000 00:26:55 +0200 (IST) Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: <1262182293-3258653@hypernet.com> Message-ID: On Mon, 7 Feb 2000, Gordon McMillan wrote: > Hmmm, well I can't find exceptions.py mentioned specifically. > But I do find the BDFL advocating writing as much as possible > of the start-up code in Python (eg, the equivalent of getpath.c) > and freezing it in. In his absence, it makes sense for us to > include exceptions.py in that statement . Of course then you have a bootstapping issue (how to write the first interpreter. Probably, need some C code to get it started: but that C code would only be used in compiling the interpreter, not in the interpreter that gets installed...) -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From akuchlin@mems-exchange.org Tue Feb 8 03:27:50 2000 From: akuchlin@mems-exchange.org (A.M. Kuchling) Date: Mon, 7 Feb 2000 22:27:50 -0500 Subject: [Python-Dev] Long/int unification patch 1 Message-ID: <200002080327.WAA02190@207-172-57-86.s86.tnt2.ann.va.dialup.rcn.com> I've got the first version of a patch against the current CVS tree to treat ints and longs identically at the Python programmer's level: ftp://starship.python.net/pub/crew/amk/new/long-python-patch-1 As an acid test, this patch changes PyInt_FromLong() to just call PyLong_FromLong(). After some minor changes to the test suite to skip tests of OverflowError, and to account for the string representation of integers having an 'L' on the end, Python gets through the test_builtin test suite. (test_pickle and test_cpickle fail with this aggressive patch; I won't bother to fix that.) Obviously, these changes won't be in any final version. The patch adds a new API function: extern DL_IMPORT(int) PyIntOrLong_AsLong Py_PROTO((PyObject *, long *)); It returns -1 if the PyObject is a long integer that's too large to fit into a C long, so it's used like this: if (PyIntOrLong_AsLong(fno, &value)) { PyErr_SetString(... The pystone results are puzzling; original Python gets 3154.57 pystones on my machine, and the patched version without PyInt_FromLong returning longs gets a spurious value of 3367! With the PyInt_FromLong change, it gets 2450.98 pystones, which is what you'd expect. Mostly I want comments on the approach, and am not claiming this version is reliable. Still to do: * Write test cases that exercise the new code. * Start converting Python/structmember.c, and a bunch of files in Modules/ * Begin changing places that raise OverflowError, to create long integers instead. (Or maybe this is a bad idea; certainly it might break existing code.) -- A.M. Kuchling http://starship.python.net/crew/amk/ I'm an excuse for medical experiments and art theory. You must get me out of here and out of the hospital. -- Peter Greenaway, _A Zed and Two Noughts_ (1986) From tim_one@email.msn.com Tue Feb 8 06:42:37 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 8 Feb 2000 01:42:37 -0500 Subject: [Python-Dev] Re: Python 2 namespace change? In-Reply-To: <14494.63402.397312.863803@goon.cnri.reston.va.us> Message-ID: <000301bf71ff$b0085b00$d92d153f@tim> [Jeremy Hylton, on JimF's association objects wrt "from M import x" semantics] > This seems to contradict the 2nd Pythonic principle: > Explicit is better than implicit. > > I don't literally mean to argue that "The Python Way" should be used > to make design decisions, but it captures exactly what makes me > uncomfortable with the proposed change. Go ahead & argue it: they were *intended* to be used to guide design decisions! They were my best shot at summarizing what I've learned in a decade of (mostly successful) Guido-channeling. But note that I only listed 19 of the 20 Pythonic Theses: the 20th was left blank, for Guido to fill in however he likes whenever the other 19 suggest a direction he dislikes . Other relevant theses here include the ones about whether the implementation is, or isn't, easy to explain. I'm suffering an email backlog and haven't yet studied the latest batch on this topic, but a quick skim sure suggests that a concrete implementation isn't self-evident, and its implications perhaps downright subtle regardless. not-a-conclusion-just-a-concern-ly y'rs - tim From tim_one@email.msn.com Tue Feb 8 06:42:39 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 8 Feb 2000 01:42:39 -0500 Subject: Int vs long (was: RE: [Python-Dev] re: syntax - "Aren't tuples redundant?") In-Reply-To: <200002071608.LAA17904@eric.cnri.reston.va.us> Message-ID: <000401bf71ff$b1920840$d92d153f@tim> [Guido] > I'm not sure I want to make this a priority given the accellerated 1.6 > schedule, but I certainly think this is the way of the future, and I > don't expect many backwards compatibility problems... I think more than one issue is on the table here: 1. Whether internal implementation code that currently relies on PyInt_Check should be liberalized to allow "int-sized longs" too. 2. Whether Python language semantics should be changed, so that e.g. int * int never overflows, but returns a long when appropriate. I was mostly talking about #2 but I think Andrew's enthusiastic agreement was really wrt #1. You may also believe I was talking about #1. Regardless, *just* tackling #1 at this time would be a good foundation for later decisions about #2, and has real value on its own (with, I agree, few backward-compatibility implications, and likely none serious (people would no longer get exceptions on stuff like [42]*42L)). Besides, I'm sure I heard Andrew volunteer to complete all the work by Wednesday . From tim_one@email.msn.com Tue Feb 8 06:42:45 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 8 Feb 2000 01:42:45 -0500 Subject: [Python-Dev] New Unicode Snapshot In-Reply-To: <389EE29B.52B48DED@lemburg.com> Message-ID: <000601bf71ff$b52d3060$d92d153f@tim> [M.-A. Lemburg] > ... > Also new in this snapshot is configuration code which figures > out the byte order on the installation machine... I looked > everywhere in the Python source code but couldn't find any > hint whether this was already done in some place, There's a tiny bit of inline code for this in the "host byte order" case of structmodule.c's function whichtable. It's easy to figure out, so probably better to copy that than create new ifdef symbols for autoconf to screw up . From gstein@lyra.org Tue Feb 8 11:53:10 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 8 Feb 2000 03:53:10 -0800 (PST) Subject: [Python-Dev] freezing code (was: win32 specific exception in the core?) In-Reply-To: <14495.6112.441151.155238@anthem.cnri.reston.va.us> Message-ID: On Mon, 7 Feb 2000, Barry A. Warsaw wrote: > >>>>> "MZ" == Moshe Zadka writes: > > MZ> Don't think of it as "-X", think of it as "could not find > MZ> exceptions.py, reverting to string exceptions" (or whatever > MZ> the error message is). Be sure it won't change to "could not > MZ> find exceptions.py, I don't think I feel like running" > > Absolutely. If string-based standard exceptions are removed for 1.6, > we need a fool-proof way of getting exceptions.py. Remember, my first > attempt at this stuff made that Python code a BACS[*] that got > exec'd. I'm sure there are better ways of doing that now, and I think > exceptions.py has settled down enough to warrant cooling it for 1.6. To possibly improve startup time and heap-memory usage, I added a feature into 1.5.2 that applies to freezing code. Specifically, a code object no longer requires a PyStringObject for the bytecodes. Any object exporting the buffer interface will work. This means that you don't have to copy the bytecodes onto the heap before using them. However, before this part can work properly, some special un-marshalling would need to happen. Currently, frozen code is a marshalled code object. When that is unmarshalled, the bytes will be copied. Instead, it would be nice to call PyCode_New() manually with all the various parameters and a "code" argument that is a PyBufferObject that points to the BACS. The alternative is to beef up the PyMarshal stuff so that it knows it can create buffers that point into a passed-in PyStringObject. Just some rambling thoughts... Cheers, -g -- Greg Stein, http://www.lyra.org/ From fredrik@pythonware.com Tue Feb 8 13:39:03 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 8 Feb 2000 14:39:03 +0100 Subject: [Python-Dev] win32 specific exception in the core? References: <14494.59309.967871.506115@weyr.cnri.reston.va.us> <14495.6112.441151.155238@anthem.cnri.reston.va.us> Message-ID: <006601bf7239$e282d2b0$f29b12c2@secret.pythonware.com> Barry A. Warsaw wrote: > Absolutely. If string-based standard exceptions are removed for 1.6, > we need a fool-proof way of getting exceptions.py. Remember, my first > attempt at this stuff made that Python code a BACS[*] that got > exec'd. I'm sure there are better ways of doing that now, and I think > exceptions.py has settled down enough to warrant cooling it for 1.6. let's just make it a built-in module. I have a 95% complete "exceptions.c" implementation somewhere. I'll take a look. (for more info, cc effbot@telia.com. GvR won't add the effbot to this list before SRE is finished ;-) From fredrik@pythonware.com Tue Feb 8 13:35:19 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 8 Feb 2000 14:35:19 +0100 Subject: [Python-Dev] New Unicode Snapshot References: <000601bf71ff$b52d3060$d92d153f@tim> Message-ID: <006501bf7239$e2769db0$f29b12c2@secret.pythonware.com> Tim Peters wrote: > [M.-A. Lemburg] > > ... > > Also new in this snapshot is configuration code which figures > > out the byte order on the installation machine... I looked > > everywhere in the Python source code but couldn't find any > > hint whether this was already done in some place, >=20 > There's a tiny bit of inline code for this in the "host byte order" = case of > structmodule.c's function whichtable. It's easy to figure out, so = probably > better to copy that than create new ifdef symbols for autoconf to = screw up > . otoh, figuring out the byte order is one of the things autoconf do very well. if they're not there already, Python's autoconf should include the basic "platform metrics" macros: AC_HEADER_STDC AC_C_INLINE AC_C_BIGENDIAN AC_CHECK_SIZEOF(char) AC_CHECK_SIZEOF(short) AC_CHECK_SIZEOF(int) AC_CHECK_SIZEOF(long) AC_CHECK_SIZEOF(float) AC_CHECK_SIZEOF(double) AC_C_CONST (think "extension writers", not necessarily "python core") From gstein@lyra.org Tue Feb 8 13:53:08 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 8 Feb 2000 05:53:08 -0800 (PST) Subject: [Python-Dev] frozen exceptions.py (was: win32 specific exception in the core?) In-Reply-To: <006601bf7239$e282d2b0$f29b12c2@secret.pythonware.com> Message-ID: On Tue, 8 Feb 2000, Fredrik Lundh wrote: > Barry A. Warsaw wrote: > > Absolutely. If string-based standard exceptions are removed for 1.6, > > we need a fool-proof way of getting exceptions.py. Remember, my first > > attempt at this stuff made that Python code a BACS[*] that got > > exec'd. I'm sure there are better ways of doing that now, and I think > > exceptions.py has settled down enough to warrant cooling it for 1.6. > > let's just make it a built-in module. I have a 95% complete > "exceptions.c" implementation somewhere. I'll take a look. > > (for more info, cc effbot@telia.com. GvR won't > add the effbot to this list before SRE is finished ;-) Why C? Implement it in Python and freeze the sucker. We all know that Python is much more maintainable. With a few simple changes, we can also make this freeze process very straight-forward and part of the standard build process. There are quite a few things that could be done in Python, then just frozen into the binary. Just wait until I start arguing for the parser and compiler to be written in Python, frozen in, and we dump their C equivalents... :-) Cheers, -g -- Greg Stein, http://www.lyra.org/ From Moshe Zadka Tue Feb 8 13:55:31 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 8 Feb 2000 15:55:31 +0200 (IST) Subject: [Python-Dev] frozen exceptions.py (was: win32 specific exception in the core?) In-Reply-To: Message-ID: On Tue, 8 Feb 2000, Greg Stein wrote: > There are quite a few things that could be done in Python, then just > frozen into the binary. Just wait until I start arguing for the parser and > compiler to be written in Python, frozen in, and we dump their C > equivalents... :-) For the record, I'm for it... -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From gstein@lyra.org Tue Feb 8 14:08:28 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 8 Feb 2000 06:08:28 -0800 (PST) Subject: [Python-Dev] frozen exceptions.py Message-ID: probably intended for this forum, too... ---------- Forwarded message ---------- Date: Tue, 08 Feb 2000 09:04:27 -0500 From: James C. Ahlstrom To: Greg Stein Subject: Re: [Python-Dev] frozen exceptions.py (was: win32 specific exception in the core?) Greg Stein wrote: > Why C? Implement it in Python and freeze the sucker. We all know that > Python is much more maintainable. With a few simple changes, we can also > make this freeze process very straight-forward and part of the standard > build process. > > There are quite a few things that could be done in Python, then just > frozen into the binary. Just wait until I start arguing for the parser and > compiler to be written in Python, frozen in, and we dump their C > equivalents... :-) I agree completely. We need the ability to freeze in Python code easily without damaging the user's ability to use freeze too. Lets get this fixed right instead of just writing more C. JimA From gstein@lyra.org Tue Feb 8 14:10:35 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 8 Feb 2000 06:10:35 -0800 (PST) Subject: [Python-Dev] frozen exceptions.py In-Reply-To: Message-ID: On Tue, 8 Feb 2000, Jim Ahlstrom wrote: >... > I agree completely. We need the ability to freeze in Python code > easily without damaging the user's ability to use freeze too. Lets > get this fixed right instead of just writing more C. Actually, that is a great way to phrase it, and hadn't occurred to me before: "... instead of just writing more C." Damn straight. Python is a great development language. Why are we avoiding it? Cheers, -g -- Greg Stein, http://www.lyra.org/ From mal@lemburg.com Tue Feb 8 14:24:12 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 08 Feb 2000 15:24:12 +0100 Subject: [Python-Dev] New Unicode Snapshot References: <000601bf71ff$b52d3060$d92d153f@tim> <006501bf7239$e2769db0$f29b12c2@secret.pythonware.com> Message-ID: <38A0270C.B013C9FD@lemburg.com> Fredrik Lundh wrote: > > Tim Peters wrote: > > [M.-A. Lemburg] > > > ... > > > Also new in this snapshot is configuration code which figures > > > out the byte order on the installation machine... I looked > > > everywhere in the Python source code but couldn't find any > > > hint whether this was already done in some place, > > > > There's a tiny bit of inline code for this in the "host byte order" case of > > structmodule.c's function whichtable. It's easy to figure out, so probably > > better to copy that than create new ifdef symbols for autoconf to screw up > > . > > otoh, figuring out the byte order is one of the things > autoconf do very well. if they're not there already, > Python's autoconf should include the basic "platform > metrics" macros: > > AC_HEADER_STDC > AC_C_INLINE > AC_C_BIGENDIAN > AC_CHECK_SIZEOF(char) > AC_CHECK_SIZEOF(short) > AC_CHECK_SIZEOF(int) > AC_CHECK_SIZEOF(long) > AC_CHECK_SIZEOF(float) > AC_CHECK_SIZEOF(double) > AC_C_CONST > > (think "extension writers", not necessarily "python core") Should I add these, Guido ? -- I'd rather stick with predefined macros than cook my own. The AC_C_INLINE would be esp. interesting here: I think this could be used a lot for those tiny function which just apply a type check and then return some object attribute value. The AC_C_CONST frightens me a bit: the Unicode code uses "const" a lot to make sure compilers can do the right optimizations. Are there compilers out there which do not handle "const" correctly ? -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Tue Feb 8 14:18:58 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 08 Feb 2000 15:18:58 +0100 Subject: [Python-Dev] New Unicode Snapshot References: <000601bf71ff$b52d3060$d92d153f@tim> Message-ID: <38A025D1.3A58D0CB@lemburg.com> Tim Peters wrote: > > [M.-A. Lemburg] > > ... > > Also new in this snapshot is configuration code which figures > > out the byte order on the installation machine... I looked > > everywhere in the Python source code but couldn't find any > > hint whether this was already done in some place, > > There's a tiny bit of inline code for this in the "host byte order" case of > structmodule.c's function whichtable. It's easy to figure out, so probably > better to copy that than create new ifdef symbols for autoconf to screw up > . I looked there, but only found that it uses native byte order by means of "letting the compiler do the right thing" -- there doesn't seem to be any code which actually tests for it. The autoconf stuff is pretty simple, BTW. The following code is used for the test: main() { long x = 0x34333231; /* == "1234" on little endian machines */ char *y = (char *)&x; if (strncmp(y,"1234",4)) exit(0); /* big endian */ else exit(1); /* little endian */ } This should be ok on big endian machines... even though I haven't tested it due to lack of access to such a beast. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From fredrik@pythonware.com Tue Feb 8 14:58:44 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 8 Feb 2000 15:58:44 +0100 Subject: [Python-Dev] frozen exceptions.py (was: win32 specific exception in the core?) References: Message-ID: <003901bf7244$ff9daf90$f29b12c2@secret.pythonware.com> > Why C? Implement it in Python and freeze the sucker. size and performance. exception classes are installed during initialization of the python interpreter, and they all need to be made available for C code anyway. (in fact, the new code isn't that much larger than the code needed to copy stuff from exceptions.py) From fredrik@pythonware.com Tue Feb 8 15:04:42 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 8 Feb 2000 16:04:42 +0100 Subject: [Python-Dev] New Unicode Snapshot References: <000601bf71ff$b52d3060$d92d153f@tim> <006501bf7239$e2769db0$f29b12c2@secret.pythonware.com> <38A0270C.B013C9FD@lemburg.com> Message-ID: <003c01bf7245$d4c54070$f29b12c2@secret.pythonware.com> M.-A. Lemburg wrote: > Should I add these, Guido ? -- I'd rather stick with predefined > macros than cook my own. > The AC_C_INLINE would be esp. interesting here: > I think this could be used a lot for those tiny function which > just apply a type check and then return some object attribute > value. umm. since inline isn't really part of ANSI C, that means that you'll end up having possibly non-inlined code in header files, right? (I use inline agressively inside modules, except for really critical things that absolutely definitively must be inlined -- look in PIL to see what I mean...) > The AC_C_CONST frightens me a bit: the Unicode code uses "const" > a lot to make sure compilers can do the right optimizations. Are > there compilers out there which do not handle "const" correctly ? not sure about this; I just copied the list from PIL, and should probably have left this one out. I've don't think I've ever used it, and afaik, 1.6 will no longer support non-ANSI compilers anyway... From mal@lemburg.com Tue Feb 8 15:55:33 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 08 Feb 2000 16:55:33 +0100 Subject: [Python-Dev] New Unicode Snapshot References: <000601bf71ff$b52d3060$d92d153f@tim> <006501bf7239$e2769db0$f29b12c2@secret.pythonware.com> <38A0270C.B013C9FD@lemburg.com> <003c01bf7245$d4c54070$f29b12c2@secret.pythonware.com> Message-ID: <38A03C75.C8B7C3D1@lemburg.com> Fredrik Lundh wrote: > > M.-A. Lemburg wrote: > > Should I add these, Guido ? -- I'd rather stick with predefined > > macros than cook my own. > > > The AC_C_INLINE would be esp. interesting here: > > > I think this could be used a lot for those tiny function which > > just apply a type check and then return some object attribute > > value. > > umm. since inline isn't really part of ANSI C, that > means that you'll end up having possibly non-inlined > code in header files, right? > > (I use inline agressively inside modules, except for > really critical things that absolutely definitively must > be inlined -- look in PIL to see what I mean...) Hmm, it would probably cause code to go into header files -- not really good style but perhaps C++ has leveraged this a bit recently ;-) > > The AC_C_CONST frightens me a bit: the Unicode code uses "const" > > a lot to make sure compilers can do the right optimizations. Are > > there compilers out there which do not handle "const" correctly ? > > not sure about this; I just copied the list from PIL, and > should probably have left this one out. > > I've don't think I've ever used it, and afaik, 1.6 will no longer > support non-ANSI compilers anyway... Uff, glad you said that :-) BTW, has anyone tried to compile the Unicode stuff on Windows yet ? -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From tim_one@email.msn.com Wed Feb 9 00:34:34 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 8 Feb 2000 19:34:34 -0500 Subject: [Python-Dev] New Unicode Snapshot In-Reply-To: <38A025D1.3A58D0CB@lemburg.com> Message-ID: <000201bf7295$70a1aac0$372d153f@tim> [M.-A. Lemburg] > ... > Also new in this snapshot is configuration code which figures > out the byte order on the installation machine... I looked > everywhere in the Python source code but couldn't find any > hint whether this was already done in some place, [Tim] > There's a tiny bit of inline code for this in the "host byte > order" case of structmodule.c's function whichtable. ... [MAL] > I looked there, but only found that it uses native byte order > by means of "letting the compiler do the right thing" -- there > doesn't seem to be any code which actually tests for it. Here's the "tiny bit of (etc)": int n = 1; char *p = (char *) &n; if (*p == 1) ... > The autoconf stuff is pretty simple, BTW. The following code > is used for the test: > > main() { > long x = 0x34333231; /* == "1234" on little endian machines */ > char *y = (char *)&x; > if (strncmp(y,"1234",4)) > exit(0); /* big endian */ > else > exit(1); /* little endian */ > } No, no, no -- that's one "no" for each distinct way I know of that can fail on platforms where sizeof(long) == 8 . Don't *ever* use longs to test endianness; besides the obvious problems, it also sucks you into illusions unique to "mixed endian" architectures. "ints" are iffy too, but less so. Test what you're actually concerned about, as directly and simply as possible; e.g., if you're actually concerned about how the machine stores shorts, do what structmodule does but use a short instead of an int. And if it's important, explicitly verify that sizeof(short)==2 (& raise an error if it's not). From tim_one@email.msn.com Wed Feb 9 00:39:51 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 8 Feb 2000 19:39:51 -0500 Subject: [Python-Dev] FW: Redundant elements in sys.path Message-ID: <000401bf7296$2cf0c260$372d153f@tim> Someone who thinks the Windows installation is a delight may wish to reply to this msg from c.l.py . -----Original Message----- From: python-list-admin@python.org [mailto:python-list-admin@python.org] On Behalf Of Steve Holden Sent: Tuesday, February 08, 2000 10:06 AM To: python-list@python.org Subject: Redundant elements in sys.path I noticed that both PythonWin and Idle have the same items, although with different case representations, in sys.path. Since the filestore isn't case-sensitive on Windows NT I'm wondering a) How these duplicates came about (multiple installs?), and b) What I can do to get rid of them (registry scoured, but nothing probable found). The code I used to detect this situation is no great shakes: >>> s=list(sys.path) >>> s.sort() >>> for p in s: ... print p Output from PythonWin: C:\Program Files\Python C:\Program Files\Python\DLLs C:\Program Files\Python\DLLs C:\Program Files\Python\Lib C:\Program Files\Python\Lib\lib-tk C:\Program Files\Python\Lib\plat-win C:\Program Files\Python\Pythonwin C:\Program Files\Python\Pythonwin C:\Program Files\Python\lib C:\Program Files\Python\lib\lib-tk C:\Program Files\Python\lib\plat-win C:\Program Files\Python\win32 C:\Program Files\Python\win32\lib Output from Idle: C:\PROGRA~1\PYTHON C:\PROGRA~1\Python\Tools\idle C:\Program Files\Python C:\Program Files\Python C:\Program Files\Python\DLLs C:\Program Files\Python\DLLs C:\Program Files\Python\Lib C:\Program Files\Python\Lib\lib-tk C:\Program Files\Python\Lib\plat-win C:\Program Files\Python\Pythonwin C:\Program Files\Python\lib C:\Program Files\Python\lib\lib-tk C:\Program Files\Python\lib\plat-win C:\Program Files\Python\win32 C:\Program Files\Python\win32\lib Since I'm not running from source distribution I'd appreciate it if someone could explain how PYTHONPATH is initialized and how I can simplify my environment to remove this duplication. regards Steve Holden -- "If computing ever stops being fun, I'll stop doing it" -- http://www.python.org/mailman/listinfo/python-list From tim_one@email.msn.com Wed Feb 9 01:29:24 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 8 Feb 2000 20:29:24 -0500 Subject: [Python-Dev] FW: 64-bit port of Python Message-ID: <000501bf729d$1a1b6c60$372d153f@tim> Raises some interesting issues wrt 64-bit Windows. -----Original Message----- From: Trent Mick [mailto:trentm@ActiveState.com] Sent: Tuesday, February 08, 2000 9:35 AM To: Tim Peters; python-list@python.org Subject: RE: 64-bit port of Python [Trent and Tim talk about 64-bit porting issues in the Python code. Trent also sneakily changes email addresses.] [Tim Peters]: > It's not what you suspect . Almost everything boiled down > to mistaken > and unintended assumptions that sizeof(int) == sizeof(long), in and around > the implementations of (unbounded) long arithmetic, and > overflow-checking of > int arithmetic. All that stuff was fixed then. AFAIK, core Python code > *never* casts a pointer to any sort of int, or vice versa, either > explicitly or implicitly. A couple of example where I think the Python core does just that: "Modules/arraymodule.c::728": static PyObject * array_buffer_info(self, args) arrayobject *self; PyObject *args; { return Py_BuildValue("ll", (long)(self->ob_item), (long)(self->ob_size)); } where 'ob_item' is a pointer. "Python/bltinmodule.c::899": static PyObject * builtin_id(self, args) PyObject *self; PyObject *args; { PyObject *v; if (!PyArg_ParseTuple(args, "O:id", &v)) return NULL; return PyInt_FromLong((long)v); } Python sort of relies on C's 'long' to be the largest native integer. This is evidenced by all the use of PyInt_FromLong() above. There are no format specifiers in the PyArg_Parse*() and Py_BuildValue() functions for converting a pointer. This was fine when 'long' would do. On Win64 sizeof(long)==4 and size(void*)==8. I think this also brings up some wider issues in the Python source. For instance, the python integer type uses the C 'long' type. Was it the implicit intention that this be the system's largest native integral type, i.e. 32-bits on a 32 sys and 64-bits on a 64-bit system? If so, then the representation of the Python integer type will have to change (i.e. the use of 'long' cannot be relied upon). One should then carry through and change (or obselete) the *_AsLong(), *_FromLong() Python/C API functions to become something like *_AsLargestNativeInt(), *_FromLargestNativeInt() (or some less bulky name). Alternatively, if the python integer type continues to use the C 'long' type for 64-bit systems then the following ugly thing happens: - A Python integer on a 64-bit Intel chip compiled with MSVC is 32-bits wide. - A Python integer on a 64-bit Intel chip compiled with gcc is 64-bits wide. That cannot be good. From tim_one@email.msn.com Wed Feb 9 04:45:30 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 8 Feb 2000 23:45:30 -0500 Subject: [Python-Dev] Long/int unification patch 1 In-Reply-To: <200002080327.WAA02190@207-172-57-86.s86.tnt2.ann.va.dialup.rcn.com> Message-ID: <000701bf72b8$7e06ff80$372d153f@tim> [A.M. Kuchling] > ... > * Begin changing places that raise OverflowError, to create long > integers instead. (Or maybe this is a bad idea; certainly it might > break existing code.) It's a darned interesting question! This is the kind of thing I was worried about. I have a lot of int code in try blocks that catches OverflowError, but, when it happens, I redo the whole darn algorithm from scratch after promoting oodles of stuff to long! This is in situations where I expect that faster int arithmetic will usually work, but don't mind slower long arithmetic when necessary. All that would still work fine (it simply wouldn't trigger OverflowError anymore). Note this is done routinely in Demo/classes/Rat.py (which isn't mine, so I'm not the only one ). At least a dozen copies of this are floating around my modules: def chop(n, int=int): """Return int(n) if no overflow, else n. """ try: return int(n) except OverflowError: return n This is usually just to get rid of the trailing "L" in output whenever possible, and sometimes to speed later operations. Etc. I think all my code would work fine. But then there's Guido's faqwiz.py: try: cutoff = now - days * 24 * 3600 except OverflowError: cutoff = 0 The intent there isn't at all obvious, although in context I think it would continue to work. After a quick but not entirely careless scan, I didn't see anything in the std distribution that's likely to break other than the OverflowError tests. It would usually *allow* rewriting to simpler, clearer, and probably faster (if long ops automagically cut back small-enough results to internal ints) code. glad-i'm-not-the-dictator-ly y'rs - tim From tim_one@email.msn.com Wed Feb 9 05:41:44 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 9 Feb 2000 00:41:44 -0500 Subject: [Python-Dev] FW: 64-bit port of Python Message-ID: <000c01bf72c0$593201c0$372d153f@tim> Ack -- meant to cc PythonDev on this but forgot. -----Original Message----- From: python-list-admin@python.org [mailto:python-list-admin@python.org] On Behalf Of Tim Peters Sent: Tuesday, February 08, 2000 11:26 PM To: Trent Mick; python-list@python.org; Mark Hammond Cc: Andrew Kuchling Subject: RE: 64-bit port of Python [posted & mailed] [Trent and Tim talk about 64-bit porting issues in the Python code. Trent also sneakily changes email addresses.] [Tim] > AFAIK, core Python code *never* casts a pointer to any sort of > int, or vice versa, either explicitly or implicitly. [Trent Mick [mailto:trentm@ActiveState.com]] > A couple of example where I think the Python core does just that: Good eye, Trent! Thank you. I'm also sending this to Mark Hammond, since ActiveState now pays him to worry about Python's Windows story -- and perhaps pays you too . > "Modules/arraymodule.c::728": > > static PyObject * > array_buffer_info(self, args) > arrayobject *self; > PyObject *args; > { > return Py_BuildValue("ll", > (long)(self->ob_item), (long)(self->ob_size)); > } > > where 'ob_item' is a pointer. Yes, the author of the new buffer interface code is being shot for many reasons . > "Python/bltinmodule.c::899": > > static PyObject * > builtin_id(self, args) > PyObject *self; > PyObject *args; > { > PyObject *v; > > if (!PyArg_ParseTuple(args, "O:id", &v)) > return NULL; > return PyInt_FromLong((long)v); > } Oh yes. Been there forever, and won't work at all (while nothing promises that id returns an address, it's crucial that "id(x) == id(y)" iff "x is y" in Python). > Python sort of relies on C's 'long' to be the largest native integer. Don't forget that Python was written pre-ANSI, and this was a common (universal?) assumption in the fuzzier K&R flavor of C. ANSI C went on to guarantee the existence of *some* integral type such that a pointer could be cast to that type and back again without loss of info -- but one committee member told me that at least he was surprised as all heck when it was pointed out that the std neglected to say that must be a *standard* integral type. The notion that "long isn't long enough" is a loophole in the std, and I'm not sure it was an intentional one. Nevertheless, it's an official one now, so that's that. > This is evidenced by all the use of PyInt_FromLong() above. There are > no format specifiers in the PyArg_Parse*() and Py_BuildValue() > functions for converting a pointer. This was fine when 'long' would > do. On Win64 sizeof(long)==4 and size(void*)==8. > > I think this also brings up some wider issues in the Python source. > For instance, the python integer type uses the C 'long' type. Was it > the implicit intention that this be the system's largest native > integral type, i.e. 32-bits on a 32 sys and 64-bits on a 64-bit > system? More the explicit intention that it be the longest standard integral type, back in the days that was believed to "mean something non-trivial". It's been a darned good bet for a decade . The advertised semantics at the Python level promise only that it's at least 32 bits. > If so, then the representation of the Python integer type will have > to change (i.e. the use of 'long' cannot be relied upon). One should > then carry through and change (or obselete) the *_AsLong(), > *_FromLong() Python/C API functions to becomesomething like > AsLargestNativeInt(), *_FromLargestNativeInt() (or some > less bulky name). > > Alternatively, if the python integer type continues to use the C > 'long' type for 64-bit systems then the following ugly thing > happens: > - A Python integer on a 64-bit Intel chip compiled with MSVC is > 32-bits wide. > - A Python integer on a 64-bit Intel chip compiled with gcc is > 64-bits wide. > That cannot be good. Two things work against all that: 1. In 1.5.2, and more in the current CVS tree, there's already grudging support for "longer than long" via the config LONG_LONG macro (e.g., under MS Windows that's already #defined as __int64). That may spread more, although it's ugly so will be resisted (Guido hates #ifdef'ing code, and platform #ifdef'ed macros aren't exactly liked -- each one is that much more for new ports to wrestle with, and everyone to trip over forever after). 2. It's already not good that int size can matter across platforms with grosser differences than the above. For that & other reasons, the sharp Python-level distinction between (bounded) ints and (unbounded) longs is slated for (backward compatible) death. Andrew Kuchling already has much of the work for that in hand, but unclear whether it will make it into 1.6 (it's not a high priority now, although I expect you just boosted it a bit ...). Once it's in, "id" can return million-bit ints as easily as it returns C longs now. or-if-activestate-solves-this-for-perl-first-we'll-just-rewrite-python- in-that-ly y'rs - ti -- http://www.python.org/mailman/listinfo/python-list From mal@lemburg.com Wed Feb 9 09:11:14 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 09 Feb 2000 10:11:14 +0100 Subject: [Python-Dev] ./configure support for the New Unicode Snapshot References: <000201bf7295$70a1aac0$372d153f@tim> Message-ID: <38A12F32.7A9F03F1@lemburg.com> Tim Peters wrote: > > [M.-A. Lemburg] > > ... > > Also new in this snapshot is configuration code which figures > > out the byte order on the installation machine... I looked > > everywhere in the Python source code but couldn't find any > > hint whether this was already done in some place, > > [Tim] > > There's a tiny bit of inline code for this in the "host byte > > order" case of structmodule.c's function whichtable. ... > > [MAL] > > I looked there, but only found that it uses native byte order > > by means of "letting the compiler do the right thing" -- there > > doesn't seem to be any code which actually tests for it. > > Here's the "tiny bit of (etc)": > > int n = 1; > char *p = (char *) &n; > if (*p == 1) > ... Hmm, haven't noticed that one (but Jean posted the same idea in private mail ;). > > The autoconf stuff is pretty simple, BTW. The following code > > is used for the test: > > > > main() { > > long x = 0x34333231; /* == "1234" on little endian machines */ > > char *y = (char *)&x; > > if (strncmp(y,"1234",4)) > > exit(0); /* big endian */ > > else > > exit(1); /* little endian */ > > } > > No, no, no -- that's one "no" for each distinct way I know of that can fail > on platforms where sizeof(long) == 8 . Don't *ever* use longs to test > endianness; besides the obvious problems, it also sucks you into illusions > unique to "mixed endian" architectures. "ints" are iffy too, but less so. > > Test what you're actually concerned about, as directly and simply as > possible; e.g., if you're actually concerned about how the machine stores > shorts, do what structmodule does but use a short instead of an int. And if > it's important, explicitly verify that sizeof(short)==2 (& raise an error if > it's not). I've turned to the autoconf predefined standard macro as suggested by Fredrik. It does the above plus some other magic as well to find out endianness. On big endian machines the configure script now defines WORDS_BIGENDIAN. The sizeof(Py_UNICODE)==2 assertion is currently tested at init time of the Unicode implementation. I would like to add Fredriks proposed sizeof checks to the configure script too, but there's a catch: the config.h in PC/ is hand generated and would need some updates for the various PC targets. Any volunteer ? We'd need the following extra data: /* The number of bytes in a char. */ #define SIZEOF_CHAR 1 /* The number of bytes in a double. */ #define SIZEOF_DOUBLE 8 /* The number of bytes in a float. */ #define SIZEOF_FLOAT 4 /* The number of bytes in a short. */ #define SIZEOF_SHORT 2 plus maybe /* Endianness. PCs are usually little endian, so we don't define this here... */ /* #undef WORDS_BIGENDIAN */ -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From gstein@lyra.org Wed Feb 9 14:12:47 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 9 Feb 2000 06:12:47 -0800 (PST) Subject: [Python-Dev] FW: 64-bit port of Python In-Reply-To: <000c01bf72c0$593201c0$372d153f@tim> Message-ID: On Wed, 9 Feb 2000, Tim Peters wrote: >... > [Trent Mick [mailto:trentm@ActiveState.com]] > > A couple of example where I think the Python core does just that: > > Good eye, Trent! Thank you. I'm also sending this to Mark Hammond, since > ActiveState now pays him to worry about Python's Windows story -- and > perhaps pays you too . In any case where Python needs to cast a pointer back/forth with an "integer", there are two new routines in Python 1.5.2. From longobject.h: extern DL_IMPORT(PyObject *) PyLong_FromVoidPtr Py_PROTO((void *)); extern DL_IMPORT(void *) PyLong_AsVoidPtr Py_PROTO((PyObject *)); I supplied the patch for these while I was also adding the 'P' format code for the "struct" module. The functions return a PyIntObject or a PyLongObject depending on whether the size of void pointer matches the size of a C long value. If a pointer fits in a long, you get an Integer. Otherwise, you get a Long. > > "Modules/arraymodule.c::728": > > > > static PyObject * > > array_buffer_info(self, args) > > arrayobject *self; > > PyObject *args; > > { > > return Py_BuildValue("ll", > > (long)(self->ob_item), (long)(self->ob_size)); > > } > > > > where 'ob_item' is a pointer. > > Yes, the author of the new buffer interface code is being shot for many > reasons . Yah. That function is quite insane. *shudder* It should use PyLong_FromVoidPtr for the first item, and PyInt_FromLong for the second (IMO, it is reasonable to assume ob_size fits in a C long). However, I'd simply argue that the function should probably be punted. What is it for? > > "Python/bltinmodule.c::899": > > > > static PyObject * > > builtin_id(self, args) > > PyObject *self; > > PyObject *args; > > { > > PyObject *v; > > > > if (!PyArg_ParseTuple(args, "O:id", &v)) > > return NULL; > > return PyInt_FromLong((long)v); > > } > > Oh yes. Been there forever, and won't work at all (while nothing promises > that id returns an address, it's crucial that "id(x) == id(y)" iff "x is y" > in Python). Assuming that we can say that id() is allowed to return a PyLongObject, then this should just use PyLong_FromVoidPtr. On most platforms, it will still return an Integer. For Win64 (and some other platforms), it will return a Long. >... > > This is evidenced by all the use of PyInt_FromLong() above. There are > > no format specifiers in the PyArg_Parse*() and Py_BuildValue() > > functions for converting a pointer. This was fine when 'long' would > > do. On Win64 sizeof(long)==4 and size(void*)==8. Similar to the structmodule, I might suggest adding a 'P' code. Exercise for the reader... However, I might counter that it is an uncommon operation, so I would argue against adding the code. People that need to do this can resort to manually using the PyLong_* functions. >... > > If so, then the representation of the Python integer type will have > > to change (i.e. the use of 'long' cannot be relied upon). One should > > then carry through and change (or obselete) the *_AsLong(), > > *_FromLong() Python/C API functions to becomesomething like > > AsLargestNativeInt(), *_FromLargestNativeInt() (or some > > less bulky name). > > > > Alternatively, if the python integer type continues to use the C > > 'long' type for 64-bit systems then the following ugly thing > > happens: > > - A Python integer on a 64-bit Intel chip compiled with MSVC is > > 32-bits wide. > > - A Python integer on a 64-bit Intel chip compiled with gcc is > > 64-bits wide. > > That cannot be good. The problem already solved (it's so much fun to borrow Guido's time machine!). Some of the C code just needs to catch up and use the new functionality, though. > ... other Tim remarks ... Tim! Wake up! New functions snuck in behind your back! hehe... Cheers, -g -- Greg Stein, http://www.lyra.org/ From Vladimir.Marangozov@inrialpes.fr Wed Feb 9 15:07:50 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Wed, 9 Feb 2000 16:07:50 +0100 (CET) Subject: [Python-Dev] Malloc interface (was: [Patches] Re: Garbage collection patches for Python (fwd) Message-ID: <200002091507.QAA02647@python.inrialpes.fr> I'm bringing the issue on python-dev, because it may change your coding habits in the future. If you have any comments or something... ---------------------------------------------------------------------- Subject: Re: [Patches] Re: Garbage collection patches for Python To: nascheme@enme.ucalgary.ca Date: Wed, 9 Feb 2000 15:59:48 +0100 (CET) Cc: Vladimir.Marangozov@inrialpes.fr, toby@puckish.demon.co.uk, pygc@freemail.hu, patches@python.org (Python Patches), guido@cnri.reston.va.us (Guido van Rossum) In-Reply-To: <20000205092822.A2204@acs.ucalgary.ca> from "nascheme@enme.ucalgary.ca" at Feb 05, 2000 09:28:23 AM Reply-To: Vladimir.Marangozov@inrialpes.fr X-Mailer: ELM [version 2.5 PL1] Content-Length: 6224 nascheme@enme.ucalgary.ca wrote: > > On Sat, Feb 05, 2000 at 03:00:59PM +0100, Vladimir Marangozov wrote: > > Neil, could you please resubmit your 1st GC patch with the disclaimer, > > so that we can focus on this 1st step, test the patch and fold it into > > CVS? > > No problem. I've included my Boehm-Demers-Weiser garbage > collector patch too as I have improved it to use autoconfig. The > gc patch is small compared to the malloc cleanup. Thanks. After a closer look at the patch, I see a couple of problems with it. In particular, it mixes different API families. Let me start from the start: The goal is to remove Python's dependency on the standard POSIX interface (malloc/realloc/free) so that we can cleanly and easily plug in the future a "proprietary" mem manager, other than the one in the C library. For this purpose, the Python core should be patched and "cleaned" to use one or more of the following APIs: 1) PyMem_MALLOC 2) PyMem_NEW PyMem_REALLOC ==> PyMem_RESIZE PyMem_FREE PyMem_DEL PyMem_XDEL The proposed Guido's augmented version of 1) which is raw mem interface. more "Pythonic" and contains some safety additions, like _PyMem_EXTRA). This one (in mymalloc.h) should be defined in terms of 1). 3) Py_Malloc 4) PyMem_Malloc 5) _PyObject_New Py_Realloc PyMem_Realloc PyObject_NEW Py_Free PyMem_Free PyObject_NEW_VAR, ... These are implemented using 1) and/or 2) It seems to me that presently nobody uses the wrappers 3) and 4), (except one single occurence of Py_Malloc, lost in _localemodule.c), because these wrappers cause an additional func call overhead... All these APIs are certainly redundant and the added value of each of them has to be weighted one again, but for now, let's assume that we have them all (without more argumentation - we could throw away some of them later). The rule of thumb in this situation is: Every chunk of memory must be manupulated via the same malloc family. That is, if one gets some piece of mem through PyMem_MALLOC (1), s/he must release that memory with PyMem_FREE (1). Accordingly, if one gets a chunk via PyMem_MALLOC (1), that chunk *should not* be released with PyMem_DEL (2). (which is what Neil's patch does, not to mention that (2) is not defined in terms of (1). So we must be careful here. In particular, when patching a file, we must figure out which family (-ies) is (are) used in order to perform the "malloc cleanup" the right way. That is, we have to inspect every file one by one, once again. ----------- With the above theory in mind, I see that there are a couple of "problematic" files in the current snapshot. I remember that I have identified them with my allocator (pymalloc) either. (I'll use some excerpts of Neil's patch to illustrate the problems I'm talking about) 1) pypcre.c This one is "buggy" for sure. The PCRE code allocates memory through the proprietary functions "pcre_malloc/pcre_free" (which default to malloc/free), so I really don't see why there's code inside mentioning "free" and not "pcre_free". a) Would this code be dropped with the inclusion of /F's re engine in 1.6? b) If the answer to a) is negative, I hope AMK & the String-SIG will help us on this (it's a hairy code :-). > diff -cr Python-cvs/Modules/pypcre.c Python-gc/Modules/pypcre.c > *** Python-cvs/Modules/pypcre.c Sat Jul 17 12:13:10 1999 > --- Python-gc/Modules/pypcre.c Sat Feb 5 09:00:47 2000 > *************** > *** 3057,3068 **** > static int free_stack(match_data *md) > { > /* Free any stack space that was allocated by the call to match(). */ > ! if (md->off_num) free(md->off_num); > ! if (md->offset_top) free(md->offset_top); > ! if (md->r1) free(md->r1); > ! if (md->r2) free(md->r2); > ! if (md->eptr) free((char *)md->eptr); > ! if (md->ecode) free((char *)md->ecode); > return 0; > } > > --- 3057,3068 ---- > static int free_stack(match_data *md) > { > /* Free any stack space that was allocated by the call to match(). */ > ! if (md->off_num) PyMem_DEL(md->off_num); > ! if (md->offset_top) PyMem_DEL(md->offset_top); > ! if (md->r1) PyMem_DEL(md->r1); > ! if (md->r2) PyMem_DEL(md->r2); > ! if (md->eptr) PyMem_DEL((char *)md->eptr); > ! if (md->ecode) PyMem_DEL((char *)md->ecode); > return 0; > } 2) _tkinter.c I remember from my experiments that this one was really a mess from a malloc point of view because it caused lots of mixed API calls (once the core allocator was changed). I have to look at it carefully once again... 3) readline.c Neil, what's this? Could you elaborate on this one? > > diff -cr Python-cvs/Modules/readline.c Python-gc/Modules/readline.c > *** Python-cvs/Modules/readline.c Sat Feb 5 09:00:14 2000 > --- Python-gc/Modules/readline.c Sat Feb 5 09:00:47 2000 > *************** > *** 41,46 **** > --- 41,63 ---- > extern int (*PyOS_InputHook)(); > extern char *(*PyOS_ReadlineFunctionPointer) Py_PROTO((char *)); > > + /* convert a string allocated by malloc to one allocated by PyMem_MALLOC */ > + static char * > + malloc2PyMem(char *p) > + { > + char *py; > + int n; > + if (p == NULL) { > + return p; > + } else { > + n = strlen(p); > + py = PyMem_MALLOC(n); > + strncpy(py, p, n+1); > + free(p); > + return py; > + } > + } > + All in all, this patch isn't acceptable for me and I suggest that we prepare another one which respects "the rule", after demystifying some of the issues mentioned above. Note that introducing PyMem_MALLOC would constitute an additional constraint for C Python coders, who're used to malloc/free. In order to pave the way for alternatives to libc's malloc, we must propose a clean solution. Hence it would be good to collect as much opinions/reactions on the subject as possible and settle on an interface which would be convenient for everybody. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From gvwilson@nevex.com Wed Feb 9 15:28:35 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Wed, 9 Feb 2000 10:28:35 -0500 (EST) Subject: [Python-Dev] final note on LANL syntax suggestions Message-ID: Just to wrap up this thread (as if): a couple of people have pointed out that two of the suggestions that came up w.r.t. Python syntax are really about eliminating need for assignment-as-an-operator: do: line = readline() while line: print line gets rid of a need for: while (line = readline()): print line without the confusing-for-newcomers: while 1: line = readline() if not line: break print line and the generalized case statement: if x is: expr1, expr2: x is the result of expr1 or expr2 expr3: x is the result of expr3 else: x is undefined or None does: if ((x = expr1) or (x = expr2)): code using x elif x = expr3: code using x else: code (Don't know what other people's experience is, but this comes up a lot in my code when I'm doing Awk-style coding, i.e. when RE matching drives execution.) Looking through some C++ code I wrote last year, these two cases (loop control and capturing successful conditions) account for at least three quarters of my uses of assignment-as-operator. My thanks to people for pointing this out, Greg From paul@prescod.net Wed Feb 9 15:36:33 2000 From: paul@prescod.net (Paul Prescod) Date: Wed, 09 Feb 2000 07:36:33 -0800 Subject: [Python-Dev] Expat -- it keeps going, and going, and going ... Message-ID: <38A18981.CED6F585@prescod.net> Apologies for the long message. There are a lot of issues to address: There was a clear concensus at the XML-SIG developer's day discussion that Expat should become part of the standard distribution. Admittedly the audience was biased and Fredrik wasn't in the room at that point but it was clear that everyone was in agreement (in contrast to the doc-sig discussion!). I think Andrew had some reservations (he was probably subconsciously channeling Guido) but almost everyone in the room was strongly behind the idea -- and the room was overfull. Insofar as this is not a democracy, I feel the need to channel some of the crowd's opinions and some of my own. The crowd (and I) obviously thought that XML support is an important part of coming with "batteries included" on the modern Web. There are four basic specs maintained by the W3C and IETF that underly the Web: URLs, HTTP, HTML and now XML. In fact, modern versions of HTML (XHTML) and HTTP (WebDAV) depend upon XML. Microsoft is also trying to establish XML-based protocols as replacements for CORBA and as the basis of their entire Web object model. Luckily, the important things to know about XML are very simple. It's a way of encoding hierarchical structures in text using a standard, language-independent syntax that happens to be compatible with document markup syntaxes. Some other things to know about it are: * it is very rigorously defined * there are test suites to verify implementations * it has enough nooks and crannies to be hard to implement * xmllib doesn't implement enough of it * and thus isn't a conforming XML parser xmllib was pretty cool when it was the first XML parser in a general purpose language. Now it is out of date. It is, however, what we present to the world as our "XML support." Whatever we do about expat, we need to decide what to do about the fact that xmllib is not a real XML processor (plus it is slow as hell!). Writing an XML processor is harder than it should be and very few people have the patience to pour over the spec and get it right. Okay, so of course you know where I am leading. Perl, Apache, Mozilla and most other C-coded open source software projects embed expat. This is because expat is blazingly fast, Unicode aware and highly conformant. It's written in ANSI-C and seems stable as a rock. It changes slowly and doesn't have a lot of extra features. Best of all, someone else maintains it and we have wrapped it in a pretty thin C layer which is easy to maintain. The layer is roughly the same size as xmllib. Guido astonished me at IPC8 with a level of humility and honesty that is very rare in this business -- especially coming from a successful language designer. He said that part of why Python didn't grab a bigger part of the CGI market was because he didn't understand the importance of CGI to the Web in the early days. He has also not been shy in saying he doesn't know much about XML. Many of us think that it will be much bigger than CGI. One opinion expressed during the meeting is that XML is a big draw for business, development money and publicity. Okay, having XML in a separate package is not the same as ignoring it altogether but people expect these fundamental technologies to be built in. As soon as you split them out you run into versioning and distribution issues. Yes, distutils will help, but I don't think it will do everything. I don't know of any package management system that can automatically correct version skew problems. The only "system" that works is full-distribution testing. Some feel that we should install PyExpat but not expat. The problem is installation, especially on Windows. It is demonstrably the case that windows programmers are ALREADY nervous about installing the XML toolkit. I got two personal emails about how to install last week (where do people get my email address??) and the XML-SIG list got one or two also. If we install pyexpat without expat, we'll have versioning problems, path problems, multiple DLL problems and so forth. If we statically bind expat and pyexpat the problems go away (on windows at least). There are rumours that some Unixes are not smart in the same situation. This can be solved by renaming symbols before building. This can be accomplished with the C pre-processor. Expat+Pyexpat is about 100K. My Python directory is 35MB so I'm not too worried. I think that the compressed Python tarball is more than 5MB now, isn't it? I'm not big on the idea of multiple Python "distributions" because in practice there will be only two: the portable one and the Windows-specific one. We'll still have to write emails like this imploring the (two) maintainers to support XML or whatever and we may have divergence between the two versions. Distributions make sense in the Linux case because there is a lot of money going around, there is money to be made on shrink-wrapped boxes and it is important to optimize for different cases. For Python, the freebsd model of "the same everywhere" is more appropriate. If that means a more distributed standard library maintenance mechanism, then fine, let's work that out. I don't expect Guido to maintain PyExpat or Expat any more than Larry Wall maintains the Perl XML parser layer or Brian B. maintains the XML support in Apache himself. If we can get concensus on this issue, I will approach James Clark for a more Pythonic license. Right now it has an MPL license but I suspect that James will be flexible. Therefore the concrete proposal is: * add expat, pyexpat and a thin SAX layer to the standard Python distribution * rename symbols in expat if necessary * deprecate xmllib * continue development of the XML toolkit for non-core tools -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made the modern world possible. - The Advent of the Algorithm (pending), by David Berlinski From Moshe Zadka Wed Feb 9 15:52:58 2000 From: Moshe Zadka (Moshe Zadka) Date: Wed, 9 Feb 2000 17:52:58 +0200 (IST) Subject: [Python-Dev] Expat -- it keeps going, and going, and going ... In-Reply-To: <38A18981.CED6F585@prescod.net> Message-ID: On Wed, 9 Feb 2000, Paul Prescod wrote: > more Pythonic license. Right now it has an MPL license but I suspect > that James will be flexible. Hey Paul! Since you probably won't want my hastily written XML-subset parser <3000 wink>, I guess I'll have to vote for expat. My seriosu issue is the license, but you seem optimistic about that... One thing you didn't mention (or maybe I missed it) is whether expeat validates. Shouldn't we have a validator too? -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From paul@prescod.net Wed Feb 9 17:22:08 2000 From: paul@prescod.net (Paul Prescod) Date: Wed, 09 Feb 2000 09:22:08 -0800 Subject: [Python-Dev] Expat -- it keeps going, and going, and going ... References: Message-ID: <38A1A240.6836C382@prescod.net> Moshe Zadka wrote: > > Hey Paul! Since you probably won't want my hastily written XML-subset > parser <3000 wink>, I guess I'll have to vote for expat. My seriosu > issue is the license, but you seem optimistic about that... Yes, but I want Python-world buy-in before I talk to James. > One thing you didn't mention (or maybe I missed it) is whether expeat > validates. Shouldn't we have a validator too? Expat does not validate because it is totally focused on performance. We could eventually build a validating level on top of expat -- especially once XML schemas "firm up". -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made possible the modern world." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml From Vladimir.Marangozov@inrialpes.fr Wed Feb 9 17:45:01 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Wed, 9 Feb 2000 18:45:01 +0100 (CET) Subject: [Python-Dev] Expat -- it keeps going, and going, and going ... In-Reply-To: <38A18981.CED6F585@prescod.net> from "Paul Prescod" at Feb 09, 2000 07:36:33 AM Message-ID: <200002091745.SAA02955@python.inrialpes.fr> Paul Prescod wrote: > > There was a clear concensus at the XML-SIG developer's day discussion > that Expat should become part of the standard distribution. ... > If we can get concensus on this issue, Paul, after this excellent argumentation, I have no choice but voting for the concensus :) > I will approach James Clark for a more Pythonic license. > Right now it has an MPL license but I suspect that James will be flexible. I expat ^H^H^Hect that from him too. > > Therefore the concrete proposal is: > > * add expat, pyexpat and a thin SAX layer to the standard Python > distribution no objections from me > * rename symbols in expat if necessary ok > * deprecate xmllib Why "deprecate"? xmllib is a solution which tends to be replaced by a new one, so chances are that in the long run, having pyexpat, development will shift to additional XML tools which have more added value than xmllib. But leave this choice to the user. I'm not a proponent of deprecating existing solutions. xmllib may perfectly fit some people's needs. In short, let the time deprecate xmllib. Okay for pyexpat, but leave xmllib as is. Oh yes, I know that for a purist, xmllib is already deprecated ;-). > * continue development of the XML toolkit for non-core tools This would be reasonable. If a better core techno exists, glue it, and build additional services ot top of that. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Vladimir.Marangozov@inrialpes.fr Wed Feb 9 18:41:03 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Wed, 9 Feb 2000 19:41:03 +0100 (CET) Subject: Python 2 namespace change? (was Re: [Python-Dev] Changing existing In-Reply-To: <389C568B.6CEBAC05@tismer.com> from "Christian Tismer" at Feb 05, 2000 05:57:47 PM Message-ID: <200002091841.TAA03001@python.inrialpes.fr> [Chris comes to my rescue on Jim's namespace idea] Christian Tismer wrote: > > Naming it different than before, I think this formulation > hits the nail on its top: > > Jim proposes a construction that yields early binding of > names, while late binding of values. > Ahaa. Got it. Thank you Chris! So naming is the same. Binding and name resolution are different. This is certainly a valuable idea in some foreseeble situations (like the globals pre-binding for a code object you're describing -- sort of a cache/array for globals, with initially invalidated entries). But the problem is that this indirection has so much power in it, that generalizing it to all namespaces seems to hide all kinds of surprises. I'm not in a position even to figure out what the implications could be (it smells "out of bounds"), but it certainly needs more digging. I suspect that if it turns out that these intermediate contexts cannot be generalized, their implementation may be compromised for the few identified cases where they are expected to be useful. > > hoping it was clear enough - ciao - chris > Yes, but embracing it all is still a "so-so"... -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From paul@prescod.net Wed Feb 9 19:50:21 2000 From: paul@prescod.net (Paul Prescod) Date: Wed, 09 Feb 2000 11:50:21 -0800 Subject: [Python-Dev] Expat -- it keeps going, and going, and going ... References: <200002091745.SAA02955@python.inrialpes.fr> Message-ID: <38A1C4FD.6BFFCD47@prescod.net> Vladimir Marangozov wrote: > > ... > Why "deprecate"? xmllib is a solution which tends to be replaced by a new one, > so chances are that in the long run, having pyexpat, development will shift > to additional XML tools which have more added value than xmllib. We don't want to encourage programmers to use the old xmllib API. I was actually one of a minority that said that we should put the xmllib API on top of expat. Most people said: "don't do that or people will keep using that API." The argument for deprecating (not removing) PyExpat goes like this: * having multiple ways to do the same thing is not the Python way * the APIs are close enough that moving to SAX should be easy * SAX is a good marketing buzzword and it makes moving from other languages easier. * the xmllib API has a little bit of weird stuff inherited from sgmllib. It's basically about causing confusion by offering too many choices. -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made possible the modern world." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml From Moshe Zadka Wed Feb 9 19:48:56 2000 From: Moshe Zadka (Moshe Zadka) Date: Wed, 9 Feb 2000 21:48:56 +0200 (IST) Subject: [Python-Dev] Expat -- it keeps going, and going, and going ... In-Reply-To: <200002091745.SAA02955@python.inrialpes.fr> Message-ID: On Wed, 9 Feb 2000, Vladimir Marangozov wrote: > > * deprecate xmllib > > Why "deprecate"? For the same reason regex was deperecated in favour of "re" -- it's a better solution. If anyone will *really* want xmllib's interface, she'll probably write it as a front end to expat, and most users wants to know right away what is the recommended solution. Dropping xmllib out of the distribution is *not* an option, however. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From gvwilson@nevex.com Wed Feb 9 19:52:53 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Wed, 9 Feb 2000 14:52:53 -0500 (EST) Subject: [Python-Dev] Expat -- it keeps going, and going, and going ... In-Reply-To: <38A1C4FD.6BFFCD47@prescod.net> Message-ID: > > Why "deprecate" xmllib? > We don't want to encourage programmers to use the old xmllib API. > ... > * having multiple ways to do the same thing is not the Python way > ... > It's basically about causing confusion by offering too many choices. I concur --- the people I'm dealing with want one answer to any question. If by chance it happens to be the best one, so much the better... :-) Greg From akuchlin@mems-exchange.org Wed Feb 9 19:55:34 2000 From: akuchlin@mems-exchange.org (Andrew M. Kuchling) Date: Wed, 9 Feb 2000 14:55:34 -0500 (EST) Subject: [Python-Dev] Expat -- it keeps going, and going, and going ... In-Reply-To: <38A1C4FD.6BFFCD47@prescod.net> References: <200002091745.SAA02955@python.inrialpes.fr> <38A1C4FD.6BFFCD47@prescod.net> Message-ID: <14497.50742.999560.8984@amarok.cnri.reston.va.us> Paul Prescod writes: >We don't want to encourage programmers to use the old xmllib API. I was >actually one of a minority that said that we should put the xmllib API >on top of expat. Most people said: "don't do that or people will keep >using that API." There's definitely no point in having multiple ways to do the same thing that produce different results, though having multiple interfaces is OK. That would be an argument in favor of building xmllib on top of Expat, but the problem is, should xmllib.py work at all without the Expat interface? The cruel-to-be-kind answer would be "no"; no Expat, no xmllib either! But that's so harsh that I'd bet GvR would never go for it, since he either doesn't understand or doesn't agree with the arguments for strict XML parsing. So the gentler answer is to leave xmllib.py as is, mark it as deprecated, and encourage use of the new Expat interface. --amk From paul@prescod.net Wed Feb 9 20:10:51 2000 From: paul@prescod.net (Paul Prescod) Date: Wed, 09 Feb 2000 12:10:51 -0800 Subject: [Python-Dev] Expat -- it keeps going, and going, and going ... References: <200002091745.SAA02955@python.inrialpes.fr> <38A1C4FD.6BFFCD47@prescod.net> <14497.50742.999560.8984@amarok.cnri.reston.va.us> Message-ID: <38A1C9CB.65965DA7@prescod.net> "Andrew M. Kuchling" wrote: > > ... > > So the gentler answer is to leave xmllib.py as is, mark it as > deprecated, and encourage use of the new (Expat) interface. #define Expat SAX -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made possible the modern world." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml From fdrake@acm.org Wed Feb 9 21:15:27 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 9 Feb 2000 16:15:27 -0500 (EST) Subject: [Python-Dev] Expat -- it keeps going, and going, and going ... In-Reply-To: <38A1C4FD.6BFFCD47@prescod.net> References: <200002091745.SAA02955@python.inrialpes.fr> <38A1C4FD.6BFFCD47@prescod.net> Message-ID: <14497.55535.25486.352402@weyr.cnri.reston.va.us> Paul Prescod writes: > We don't want to encourage programmers to use the old xmllib API. I was > actually one of a minority that said that we should put the xmllib API I too would be fine with building xmllib on top of expat; the most-used part of the API is quite reasonable. > * the APIs are close enough that moving to SAX should be easy And this has two edges. > * SAX is a good marketing buzzword and it makes moving from other > languages easier. > * the xmllib API has a little bit of weird stuff inherited from > sgmllib. That can & should be fixed, regardless. > It's basically about causing confusion by offering too many choices. This is the kicker. I really don't like deprecating xmllib, but having two very similar APIs (SAX & xmllib) is bad for exactly this reason. It's better to change this now than to wait. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From paul@prescod.net Thu Feb 10 00:10:33 2000 From: paul@prescod.net (Paul Prescod) Date: Wed, 09 Feb 2000 16:10:33 -0800 Subject: [Python-Dev] Python -t Message-ID: <38A201F9.6222BAD5@prescod.net> What are the chances of making python -t the default in Python 1.6? It isn't pythonic to silently allow people to do something that almost everyone agrees is wrong. -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made possible the modern world." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml From gstein@lyra.org Thu Feb 10 00:13:42 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 9 Feb 2000 16:13:42 -0800 (PST) Subject: [Python-Dev] Python -t In-Reply-To: <38A201F9.6222BAD5@prescod.net> Message-ID: On Wed, 9 Feb 2000, Paul Prescod wrote: > What are the chances of making python -t the default in Python 1.6? It > isn't pythonic to silently allow people to do something that almost > everyone agrees is wrong. I don't see a need to be so restrictive. Our views do not always match others'. I see no harm in continuing to allow mixed tabs/spaces. It doesn't hurt them, it doesn't hurt us. In fact, turning that on by default can/will cause pain. Okay, so people go and fix up their scripts. To what end? It hasn't improved anything, or given them more functionality. All it has done is to stop some warnings. Cheers, -g -- Greg Stein, http://www.lyra.org/ From paul@prescod.net Thu Feb 10 00:18:57 2000 From: paul@prescod.net (Paul Prescod) Date: Wed, 09 Feb 2000 16:18:57 -0800 Subject: [Python-Dev] Python -t References: Message-ID: <38A203F1.DD5A439D@prescod.net> Newbies are the only people who don't know to avoid tab/space mixing and they are the people we should be trying to help. -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made possible the modern world." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml From trentm@ActiveState.com Wed Feb 9 16:49:17 2000 From: trentm@ActiveState.com (Trent Mick) Date: Wed, 9 Feb 2000 16:49:17 -0000 Subject: [Python-Dev] FW: 64-bit port of Python In-Reply-To: Message-ID: Let me define two different Python interpreter systems to help the discussion. - System A: Compiled with MSVC on a 64-bit Intel chip (i.e. LLP64 data model, long is 32-bits). - System B: Compiled with gcc on a 64-bit Intel chip (i.e. LP64 data model, long is 64-bits). Same hardware. Just different compiler (and possibly different OS). First a couple of responses: [Greg Stein]: > In any case where Python needs to cast a pointer back/forth with an > "integer", there are two new routines in Python 1.5.2. From longobject.h: > > extern DL_IMPORT(PyObject *) PyLong_FromVoidPtr Py_PROTO((void *)); > extern DL_IMPORT(void *) PyLong_AsVoidPtr Py_PROTO((PyObject *)); > > I supplied the patch for these while I was also adding the 'P' format code > for the "struct" module. > > The functions return a PyIntObject or a PyLongObject depending on whether > the size of void pointer matches the size of a C long value. If a pointer > fits in a long, you get an Integer. Otherwise, you get a Long. > > > > "Python/bltinmodule.c::899": > > > > > > static PyObject * > > > builtin_id(self, args) > > > PyObject *self; > > > PyObject *args; > > > { > > > PyObject *v; > > > > > > if (!PyArg_ParseTuple(args, "O:id", &v)) > > > return NULL; > > > return PyInt_FromLong((long)v); > > > } > > > Assuming that we can say that id() is allowed to return a PyLongObject, > then this should just use PyLong_FromVoidPtr. On most platforms, it will > still return an Integer. For Win64 (and some other platforms), it will > return a Long. This means that my System A and System B (above) get different resultant object types for id() just because the compiler used for their Python interpreter uses a different data model. That sounds dangerous. Are there pickling portability issues or external interface issues? I know that noone should really need to be passing converted pointer results between platforms, but... shouldn't two Python interpreters running on identical hardware behave identically. This seems to me the only (or safest) way to guarantee portability. [Trent Mick]: > > > If so, then the representation of the Python integer type will have > > > to change (i.e. the use of 'long' cannot be relied upon). One should > > > then carry through and change (or obselete) the *_AsLong(), > > > *_FromLong() Python/C API functions to becomesomething like > > > AsLargestNativeInt(), *_FromLargestNativeInt() (or some > > > less bulky name). > > > > > > Alternatively, if the python integer type continues to use the C > > > 'long' type for 64-bit systems then the following ugly thing > > > happens: > > > - A Python integer on a 64-bit Intel chip compiled with MSVC is > > > 32-bits wide. > > > - A Python integer on a 64-bit Intel chip compiled with gcc is > > > 64-bits wide. > > > That cannot be good. [Greg Stein]: > The problem already solved (it's so much fun to borrow Guido's time > machine!). Some of the C code just needs to catch up and use the new > functionality, though. How so? Do you mean with PyLong_{As|From}VoidPtr()? I want to make a couple of suggestions about this 64-bit compatibility stuff. I will probably sound like I am on glue but please bear with me and let me try and convince you that I am not. * * * PyInt was tied to C's 'long' based on the (reasonable) assumption that this would represent the largest native integral type on which Python was running (or, at least, I think that PyInt *should* be the largest native int, otherwise it is arbitrarily limited). Hence, things like holding a pointer and printing its value came for free. However, with the LLP64 data model (that Microsoft has assumed for WIN64) this intention is bastardized: sizeof(long)==4 and size(void*)==8. Taking it as a given that Python should be made to run on the various 64-bit platforms, there are two ways to deal with this: 1. Continue to base PyInt on 'long' and bolt on things like LONG_LONG, PyLong_FromVoidPtr with return types of either PyInt or PyLong as necessary; or 2. Add a level of typedef abstraction to decouple Python from the no-longer-really-valid wish that 'long' is that largest native integer and couple PyInt with the actual largest native integral value. 3. (I know I said there were only two. Spanish Iqui...:) Andrew Kuchling has this all under control (as Tim intimated might be the case) or I am really missing something. I would like to argue for option number 2. C programmers use the various integral types for different reasons. Simple uses: Use 'int' when you just want a typical integer. Use 'long' when you need the range. Use 'short' when you know the range is limited to 64k and you need to save space. More specific: Use 'long' to store a pointer or cast to 'long' to print the decimal value of the pointer with printf(). These uses all make assumptions that can bite you when the data model (i.e. type sizes) changes. What is needed is a level of abstraction away from the fundamental C types. ANSI has defined some of this already (but maybe not enough). If you want to store a pointer, use 'intptr_t' (or 'uintptr_t'). If you know the range is limited 64k, then use 'int16_t'. If you want the largest native integral type, use something like 'intlongest_t'. If you know that range is limited to 64k, but you don't take the time hit for sign extension that 'int16_t' may imply, then use 'int16fast_t'. 'int16fast_t' and its kin (the ugly name is mine) would be guaranteed to be at least as wide as the name implies (i.e. 16-bits wide here), but could be larger if that would be faster for the current system. It is these meanings that I think C programmers are really trying to express when they use short, and int, and long. On the Python/C API side, use things like: - PyInt would be tied to intlongest_t - extern DL_IMPORT(PyObject *) PyInt_FromLongest Py_PROTO((intlongest_t)); "What?!," you say. "Trent, are you nuts? Why not just use 'int' then instead of this ugly 'int16fast_t'?" Well, just using 'int' carries the implicit assumption that 'int' is at least 16-bits wide. I know that it *is* for any reasonable system that Python is going to run on but: (1) the explicit specification of the range is self documenting as to the intentions of the author; and (2) the same argument applies to int*fast_t of other sizes where the size assumption about 'int' may not be so cut-and-dry. This opens up a can of worms. Your first impression is to raise your hands and say that everything from printf formatters, to libc functions, to external libraries, to PyArg_Parse() and Py_BuildValue() is based upon the fundamental C types. Hence, it is not possible to slip in a level of data type abstraction. I suppose I could be proven wrong, but I think it is possible. The printf formatters can be manhandled to use the formatter you want. The libc functions, on quick perusal, painfully try to do something like what I am suggesting anyway so they map fairly well. PyArg_Parse(), etc *could* be changed if that was necessary (*now* I *know* Guido thinks I am nuts). * * * This, I think, is the idea for general data model portability. However, because (1) it would require a lot of little patches and (2) it may require some backward incompatibilities, I realize that it would never be considered until at least Python 2.0. If you are skeptical because it sounds like I am just talking and asking for a volunteer to make these changes, it might help to know that I am volunteering to work on this. (Yes, Tim. ActiveState *is* paying me to look at this stuff.) I just want to see what the general reaction is to this: You are going about this in the wrong way? Go for it? Yes, but...? > or-if-activestate-solves-this-for-perl-first-we'll-just-rewrite- > python-in-that-ly y'rs - tim not-on-your-life-ly y'rs - Trent Trent trentm@ActiveState.com From jcw@equi4.com Thu Feb 10 00:46:13 2000 From: jcw@equi4.com (Jean-Claude Wippler) Date: Thu, 10 Feb 2000 01:46:13 +0100 Subject: [Python-Dev] Python -t References: <38A203F1.DD5A439D@prescod.net> Message-ID: <38A20A55.C48EE8F@equi4.com> Paul Prescod wrote: > > Newbies are the only people who don't know to avoid tab/space mixing > and they are the people we should be trying to help. Long term, there are far more newbies than current users. This intermediate-bie votes yes to -t as default. -jcw From gvwilson@nevex.com Thu Feb 10 01:20:25 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Wed, 9 Feb 2000 20:20:25 -0500 (EST) Subject: [Python-Dev] Python -t In-Reply-To: <38A20A55.C48EE8F@equi4.com> Message-ID: > > Newbies are the only people who don't know to avoid tab/space mixing > > and they are the people we should be trying to help. > Long term, there are far more newbies than current users. > This intermediate-bie votes yes to -t as default. I believe Randy Pausch mentioned this (briefly) in his talk --- people become confused when things that look identical (mixed tabs and spaces) don't behave identically. Can anyone who was at IPC8 confirm/refute? Thanks, Greg From skip@mojam.com (Skip Montanaro) Thu Feb 10 01:39:14 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 9 Feb 2000 19:39:14 -0600 (CST) Subject: [Python-Dev] Python -t In-Reply-To: <38A201F9.6222BAD5@prescod.net> References: <38A201F9.6222BAD5@prescod.net> Message-ID: <14498.5826.622563.708676@beluga.mojam.com> Paul> What are the chances of making python -t the default in Python 1.6? It Paul> isn't pythonic to silently allow people to do something that almost Paul> everyone agrees is wrong. What is the definition of "inconsistent usage"? I just ran "python -tt .../compileall.py over my local library and got plenty of complaints. The first example I checked seemed okay to me. If I have if foo: foo() where the first line is indented using four spaces and the second using a single tab, is that considered inconsistent usage? I'd offer that perhaps the Make targets that use compileall.py should either invoke "python -t" or python -tt". Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ "Languages that change by catering to the tastes of non-users tend not to do so well." - Doug Landauer From tim_one@email.msn.com Thu Feb 10 04:30:08 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 9 Feb 2000 23:30:08 -0500 Subject: [Python-Dev] Python -t In-Reply-To: <38A201F9.6222BAD5@prescod.net> Message-ID: <000301bf737f$831fbf80$8e2d153f@tim> [Paul Prescod] > What are the chances of making python -t the default in Python 1.6? My guess is slim. Guido explictly chose not to make it the default when he introduced it, and I doubt it would be possible to dream up an argument he hasn't heard before <0.5 wink>. > It isn't pythonic to silently allow people to do something that > almost everyone agrees is wrong. FWIW, I agree. That and 3 bucks will get you a doughnut . Note that, in the pre-Barry pre-enlightenment days, the Emacs pymode routinely mixed tabs and spaces when people changed (as most did!) the default indent level (which Guido then insisted be 8, and which I strongly encouraged people to change to 4 in the pymode comments ). That may have something to do with this, and if so "the new argument" is simply that we're another year removed from code produced under the old pymode regime. From tim_one@email.msn.com Thu Feb 10 04:40:49 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 9 Feb 2000 23:40:49 -0500 Subject: [Python-Dev] Python -t In-Reply-To: <14498.5826.622563.708676@beluga.mojam.com> Message-ID: <000501bf7381$01151ec0$8e2d153f@tim> [Skip Montanaro] > What is the definition of "inconsistent usage"? I just ran > "python -tt .../compileall.py over my local library and got > plenty of complaints. The first example I checked seemed > okay to me. If I have > > if foo: > foo() > > where the first line is indented using four spaces and the second > using a single tab, is that considered inconsistent usage? Yes, and for a reason that would be obvious if you were me : my mail reader shows those lines lining up exactly. -t looks for places where the compiler's INDENT and DEDENT decisions differ under tab settings 4 and 8. tabnanny.py does the same, but considering all tab settings in 1 to infinity "simultaneously". 4 and 8 are the "practical" ones to check, since a tab setting of 4 is as common on the platform I'm using as a tab setting of 8 is on the one you're using. Note that Guido's style guide says "all spaces" is the std for distributed code. From paul@prescod.net Thu Feb 10 05:29:40 2000 From: paul@prescod.net (Paul Prescod) Date: Wed, 09 Feb 2000 21:29:40 -0800 Subject: [Python-Dev] Python -t References: <000301bf737f$831fbf80$8e2d153f@tim> Message-ID: <38A24CC4.39C460A8@prescod.net> Tim Peters wrote: > > [Paul Prescod] > > What are the chances of making python -t the default in Python 1.6? > > My guess is slim. Guido explictly chose not to make it the default when he > introduced it, and I doubt it would be possible to dream up an argument he > hasn't heard before <0.5 wink>. I see it as a series of steps. First someone suggests tab mixing is a bad idea. Then msot people come to agree. Then we add a flag for those people to enforce the rule. The flag acts as a powerful rhetorical rebuke against those that would transgress, so everyone comes around to our way of thinking (other than Skip). Then we declare the old behavior sin and charge a fee (an extra flag) for those that would behave in such and unnatural way. > > It isn't pythonic to silently allow people to do something that > > almost everyone agrees is wrong. > > FWIW, I agree. That and 3 bucks will get you a doughnut . No, basking in your agreement is worth at least $4.00. Maybe even 4 euros. -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made possible the modern world." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml From mal@lemburg.com Thu Feb 10 08:49:30 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 10 Feb 2000 09:49:30 +0100 Subject: [Python-Dev] Python -t References: Message-ID: <38A27B9A.819AF43@lemburg.com> Greg Stein wrote: > > On Wed, 9 Feb 2000, Paul Prescod wrote: > > What are the chances of making python -t the default in Python 1.6? It > > isn't pythonic to silently allow people to do something that almost > > everyone agrees is wrong. > > I don't see a need to be so restrictive. Our views do not always match > others'. I see no harm in continuing to allow mixed tabs/spaces. It > doesn't hurt them, it doesn't hurt us. > > In fact, turning that on by default can/will cause pain. Okay, so people > go and fix up their scripts. To what end? It hasn't improved anything, or > given them more functionality. All it has done is to stop some warnings. Is there a command line tool out there to untabify existing Python scripts ? I recently wanted to apply such a tool to all my stuff, but couldn't find any suitable Python script for the job... there are lots of tools to catch those tab/space mixes, so no finding the cure to the problem kind of surprised me ;-) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From gstein@lyra.org Thu Feb 10 10:42:42 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 10 Feb 2000 02:42:42 -0800 (PST) Subject: [Python-Dev] FW: 64-bit port of Python In-Reply-To: Message-ID: On Wed, 9 Feb 2000, Trent Mick wrote: >... description of Systems A and B > [Greg Stein]: >... > > Assuming that we can say that id() is allowed to return a PyLongObject, > > then this should just use PyLong_FromVoidPtr. On most platforms, it will > > still return an Integer. For Win64 (and some other platforms), it will > > return a Long. > > This means that my System A and System B (above) get different resultant > object types for id() just because the compiler used for their Python > interpreter uses a different data model. That sounds dangerous. Are there > pickling portability issues or external interface issues? I know that noone > should really need to be passing converted pointer results between > platforms, but... shouldn't two Python interpreters running on identical > hardware behave identically. This seems to me the only (or safest) way to > guarantee portability. No issues that I foresee. id() is only useful as an unique identifier, and it should *only* be used as that. It should not be used as an access point for an object's address. Within a particular interpreter invocation, "id(x)==id(y)" IFF "x is y" (as Tim Peters pointed out). This will hold for System A or B if id() can return a long value. Within a given pickle, an id() can only be used to match up references to a single object. Again, this will continue to match. For external interfaces, it is possible that people are passing id() to something and that target is expecting an "int" rather than possible a "long". IMO, that possibility is negligible. I've never seen it. People do use the id() value when they are printing a repr() of objects. Those uses may overflow, though, because people are using '%d' or '%x' format codes. It should be %s. >... > > The problem already solved (it's so much fun to borrow Guido's time > > machine!). Some of the C code just needs to catch up and use the new > > functionality, though. > > How so? Do you mean with PyLong_{As|From}VoidPtr()? Yah. >... > Taking it as a given that Python should be made to run on the various 64-bit > platforms, Yes, that is a given. Python has been running on Alpha processors for a long time now; I'd say you found bugs rather than semantic problems. > there are two ways to deal with this: > 1. Continue to base PyInt on 'long' and bolt on things like LONG_LONG, > PyLong_FromVoidPtr with return types of either PyInt or PyLong as necessary; > or > 2. Add a level of typedef abstraction to decouple Python from the > no-longer-really-valid wish that 'long' is that largest native integer and > couple PyInt with the actual largest native integral value. > 3. (I know I said there were only two. Spanish Iqui...:) Andrew Kuchling has > this all under control (as Tim intimated might be the case) or I am really > missing something. >... discussion of options and intlongest_t and stuff ... I think that you're concentrating on the wrong problem. PyInt and PyLong are fine integer abstractions (and Andrew is working to minimize the apparent differences). You're looking at changing Int/Long to solve the "store a pointer into an integer-like thing." Instead, I think it is much more straight-forward to look at why the pointer needs to be stored and whether it matters that you use an Int or Long. IMO, it doesn't matter what type is used to store pointer-values, as long as you can go back/forth. There is no other operation needed. In other words, rather than change the numeric model for what is a small problem, just change the small problem. I would agree with you if there was a fundamental, underlying problem, but I don't believe there is. Integers store at least 32 signed bits of data, Longs store arbitrary precision. Python doesn't deal with pointers, so I don't think we need to design the integers around the capability of holding them. Note that a C extension can also use PyCObject to store an arbitrary pointer. >... > If you are skeptical because it sounds like I am just talking and asking for > a volunteer to make these changes, it might help to know that I am > volunteering to work on this. (Yes, Tim. ActiveState *is* paying me to look > at this stuff.) I just want to see what the general reaction is to this: You > are going about this in the wrong way? Go for it? Yes, but...? Understood, but it seems like you would be applying your efforts for little gain. IMO, the problem of storing pointers into integers is a red herring. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gward@cnri.reston.va.us Thu Feb 10 13:39:02 2000 From: gward@cnri.reston.va.us (Greg Ward) Date: Thu, 10 Feb 2000 08:39:02 -0500 Subject: [Python-Dev] Python -t In-Reply-To: <38A27B9A.819AF43@lemburg.com>; from mal@lemburg.com on Thu, Feb 10, 2000 at 09:49:30AM +0100 References: <38A27B9A.819AF43@lemburg.com> Message-ID: <20000210083901.A2066@cnri.reston.va.us> On 10 February 2000, M.-A. Lemburg said: > Is there a command line tool out there to untabify existing > Python scripts ? I recently wanted to apply such a tool to all > my stuff, but couldn't find any suitable Python script for the job... > there are lots of tools to catch those tab/space mixes, so no > finding the cure to the problem kind of surprised me ;-) On Solaris: User Commands expand(1) NAME expand, unexpand - expand TAB characters to SPACE charac- ters, and vice versa SYNOPSIS expand [ -t tablist ] [ file... ] expand [ -tabstop ] [ -tab1,tab2,...,tabn ] [ file... ] unexpand [ -a ] [ -t tablist ] [ file... ] DESCRIPTION expand copies files (or the standard input) to the standard output, with TAB characters expanded to SPACE characters. [...] And on Linux: EXPAND(1) EXPAND(1) NAME expand - convert tabs to spaces SYNOPSIS expand [-tab1[,tab2[,...]]] [-t tab1[,tab2[,...]]] [-i] [--tabs=tab1[,tab2[,...]]] [--initial] [--help] [--ver- sion] [file...] DESCRIPTION [...] This manual page documents the GNU version of expand. expand writes the contents of each given file, or the standard input if none are given or when a file named `-' is given, to the standard output, with tab characters con- verted to the appropriate number of spaces. By default, expand converts all tabs to spaces. [...] I expect the latter, which is of course GNU expand, is available for (maybe part of) Cygwin. I don't think it's written in Python, though. ;-) Greg From skip@mojam.com (Skip Montanaro) Thu Feb 10 13:47:42 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 10 Feb 2000 07:47:42 -0600 (CST) Subject: [Python-Dev] Python -t In-Reply-To: <000501bf7381$01151ec0$8e2d153f@tim> References: <14498.5826.622563.708676@beluga.mojam.com> <000501bf7381$01151ec0$8e2d153f@tim> Message-ID: <14498.49534.96402.252445@beluga.mojam.com> Thanks all for the explanation on inconsistent whitespace use. Tim> Note that Guido's style guide says "all spaces" is the std for Tim> distributed code. Which would suggest that "make test" and/or compileall.py should run with -t or -tt but don't... hmmm... seems like a submission to patches@python.org is in order... S From skip@mojam.com (Skip Montanaro) Thu Feb 10 14:09:58 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 10 Feb 2000 08:09:58 -0600 (CST) Subject: [Python-Dev] Python -t In-Reply-To: <38A24CC4.39C460A8@prescod.net> References: <000301bf737f$831fbf80$8e2d153f@tim> <38A24CC4.39C460A8@prescod.net> Message-ID: <14498.50870.979144.694410@beluga.mojam.com> Paul> I see it as a series of steps. .... so everyone comes around to Paul> our way of thinking (other than Skip). I resemble that remark... Actually, I suspect that many people are in the same boat I'm in. I rarely need to move code I write off of Unix systems. I had to execute "python --help" yesterday to learn what -t means. Modules that I made public ages ago with no consideration of the tab devil have elicited nary a peep about indentation problems from anyone. (Of course, maybe nobody uses them and I'm simply deluding myself thinking they might be of interest to someone... ;-) Still, if all tabs or all spaces is the way to go and we can be reasonably sure that most/all people will have an indentation-friendly editor at their disposal, then perhaps after a period of time -t should be the default. tab-devil-be-gone-ly y'rs, Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ "Languages that change by catering to the tastes of non-users tend not to do so well." - Doug Landauer From fredrik@pythonware.com Thu Feb 10 14:31:28 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 10 Feb 2000 15:31:28 +0100 Subject: [Python-Dev] FW: 64-bit port of Python References: Message-ID: <008501bf73d4$83d664b0$f29b12c2@secret.pythonware.com> Greg Stein wrote: > People do use the id() value when they are printing a repr() of = objects. > Those uses may overflow, though, because people are using '%d' or '%x' > format codes. It should be %s. %s doesn't hexify. cannot be that hard to fix %d and %x to work also for longs, can it? > > Taking it as a given that Python should be made to run on the = various 64-bit > > platforms, >=20 > Yes, that is a given. Python has been running on Alpha processors for = a > long time now; I'd say you found bugs rather than semantic problems. footnote: assert Unix in (LP32, LP64), according to the single Unix specification (and if you dig up their rationale, you'll see why everything else is totally braindead -- I'm usually no Microsoft basher, but this really pisses me off) > In other words, rather than change the numeric model for what is a = small > problem, just change the small problem. agreed. From fredrik@pythonware.com Thu Feb 10 14:38:35 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 10 Feb 2000 15:38:35 +0100 Subject: [Python-Dev] Malloc interface (was: [Patches] Re: Garbage collection patches for Python (fwd) References: <200002091507.QAA02647@python.inrialpes.fr> Message-ID: <008601bf73d4$83de05d0$f29b12c2@secret.pythonware.com> Vladimir Marangozov wrote: > The goal is to remove Python's dependency on the standard POSIX = interface > (malloc/realloc/free) so that we can cleanly and easily plug in the = future > a "proprietary" mem manager, other than the one in the C library. (hmm. I've been forced to use interfaces like this a lot, but never ever stumbled upon a situation where we couldn't just tweak malloc/free to mean what we wanted, either by relinking or via the preprocessor. I'm sceptical, in other words...) > For this purpose, the Python core should be patched and "cleaned" to = use > one or more of the following APIs: >=20 > 1) PyMem_MALLOC 2) PyMem_NEW =20 > PyMem_REALLOC =3D=3D> PyMem_RESIZE > PyMem_FREE PyMem_DEL > PyMem_XDEL is XDEL simply a "if (p) free(p)" variant? if so, don't forget that ANSI C requires that free() does the right thing if handled a NULL pointer. since Python 1.6 will be ANSI C, it's probably better to force users of broken platforms to work around bugs in PyMem_FREE, rather than expose two different alternatives. I'm pretty sure there are standard macros for auto- conf that tests for this. in other words,=20 > > ! if (md->off_num) free(md->off_num);=20 > > ! if (md->offset_top) free(md->offset_top);=20 > > ! if (md->r1) free(md->r1);=20 > > ! if (md->r2) free(md->r2);=20 > > ! if (md->eptr) free((char *)md->eptr);=20 > > ! if (md->ecode) free((char *)md->ecode); would become: > > ! PyMem_FREE(md->off_num);=20 > > ! PyMem_FREE(md->offset_top);=20 > > ! PyMem_FREE(md->r1);=20 > > ! PyMem_FREE(md->r2);=20 > > ! PyMem_FREE(md->eptr);=20 > > ! PyMem_FREE(md->ecode); From gmcm@hypernet.com Thu Feb 10 14:45:58 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 10 Feb 2000 09:45:58 -0500 Subject: [Python-Dev] Python -t In-Reply-To: <38A27B9A.819AF43@lemburg.com> Message-ID: <1261928537-18521181@hypernet.com> M.-A. Lemburg wrote: > Is there a command line tool out there to untabify existing > Python scripts ? I recently wanted to apply such a tool to all > my stuff, but couldn't find any suitable Python script for the job... > there are lots of tools to catch those tab/space mixes, so no > finding the cure to the problem kind of surprised me ;-) Contrib/system/tabcleaner.py - Gordon From skip@mojam.com (Skip Montanaro) Thu Feb 10 15:00:08 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 10 Feb 2000 09:00:08 -0600 (CST) Subject: [Python-Dev] Python -t In-Reply-To: <38A2C9E1.2DC297A@equi4.com> References: <000301bf737f$831fbf80$8e2d153f@tim> <38A24CC4.39C460A8@prescod.net> <14498.50870.979144.694410@beluga.mojam.com> <38A2C9E1.2DC297A@equi4.com> Message-ID: <14498.53880.273577.627351@beluga.mojam.com> me> I resemble that remark... jc> ^^^^^^^^ jc> Resent? My apologies. That was Bad American Humor(tm). It comes from the Garfield comic strip. The main character is a very fat cat (Garfield) who uses that response when he thinks he's being insulted but he's not sure (typically when some other character in the strip comments on his culinary habits or his waistline). Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ "Languages that change by catering to the tastes of non-users tend not to do so well." - Doug Landauer From Vladimir.Marangozov@inrialpes.fr Thu Feb 10 15:08:54 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Thu, 10 Feb 2000 16:08:54 +0100 (CET) Subject: [Python-Dev] Malloc interface (was: [Patches] Re: Garbage collection patches for Python (fwd) In-Reply-To: <008601bf73d4$83de05d0$f29b12c2@secret.pythonware.com> from "Fredrik Lundh" at Feb 10, 2000 03:38:35 PM Message-ID: <200002101508.QAA04484@python.inrialpes.fr> Fredrik Lundh wrote: > > Vladimir Marangozov wrote: > > The goal is to remove Python's dependency on the standard POSIX interface > > (malloc/realloc/free) so that we can cleanly and easily plug in the future > > a "proprietary" mem manager, other than the one in the C library. > > (hmm. I've been forced to use interfaces like this a lot, but > never ever stumbled upon a situation where we couldn't just > tweak malloc/free to mean what we wanted, either by relinking > or via the preprocessor. I'm sceptical, in other words...) /F, I agree that it's "doable" via the pre-processor or by relinking. But the truth is that this is not easy with the current code, and "managing" to do it reveals special cases, spread all over the distrib. It's about cleaning this up and settle on some rules which basically make things explicit, than implicit. Beleive me, if you change Python's malloc to something else, all kinds of troubles show up, especially with extension modules. And all you're rewarded with is a segfault... > > > For this purpose, the Python core should be patched and "cleaned" to use > > one or more of the following APIs: > > > > 1) PyMem_MALLOC 2) PyMem_NEW > > PyMem_REALLOC ==> PyMem_RESIZE > > PyMem_FREE PyMem_DEL > > PyMem_XDEL > > is XDEL simply a "if (p) free(p)" variant? Yes, see mymalloc.h. > > if so, don't forget that ANSI C requires that free() does the right > thing if handled a NULL pointer. since Python 1.6 will be ANSI C, > it's probably better to force users of broken platforms to work > around bugs in PyMem_FREE, rather than expose two different > alternatives. I'm pretty sure there are standard macros for auto- > conf that tests for this. I'm aware of this, but I think Guido has included PyMem_XDEL for 2 reasons: a) it makes us foresee (and be explicit) that the pointer may be NULL b) if the pointer is NULL, the cost of the function call is avoided. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From gvwilson@nevex.com Thu Feb 10 17:24:14 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Thu, 10 Feb 2000 12:24:14 -0500 (EST) Subject: [Python-Dev] Python publishing Message-ID: Hi, folks. A former colleague of mine is now editing a magazine devoted to scientific computing, and is looking for articles. If you're doing something scientific with Python, and want to tell the world about it, please give me a shout, and I'll forward more information. Thanks, Greg From paul@prescod.net Thu Feb 10 19:49:30 2000 From: paul@prescod.net (Paul Prescod) Date: Thu, 10 Feb 2000 11:49:30 -0800 Subject: [Python-Dev] Python -t References: <000301bf737f$831fbf80$8e2d153f@tim> <38A24CC4.39C460A8@prescod.net> <14498.50870.979144.694410@beluga.mojam.com> Message-ID: <38A3164A.E7B7147E@prescod.net> Skip Montanaro wrote: > > Actually, I suspect that many people are in the same boat I'm in. I rarely > need to move code I write off of Unix systems. I had to execute "python > --help" yesterday to learn what -t means. Modules that I made public ages > ago with no consideration of the tab devil have elicited nary a peep about > indentation problems from anyone. (Of course, maybe nobody uses them and > I'm simply deluding myself thinking they might be of interest to > someone... ;-) Mixing tabs and spaces is not all that likely to cause problems, I admit. It's mostly in the hands of newbies starting from scratch (and anti-whitespace zealots) that it will be a problem. > Still, if all tabs or all spaces is the way to go and we can be reasonably > sure that most/all people will have an indentation-friendly editor at their > disposal, then perhaps after a period of time -t should be the default. Well everyone in the world has an editor that either does tabs or spaces. Even "modern" (and I use the term VERY LOOSELY) versions of DOS EDIT.COM seem to preserve spaces. It didn't before. Can you imagine that there is still someone at Microsoft tweaking that code? -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made possible the modern world." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml From Jasbahr@origin.EA.com Thu Feb 10 20:42:24 2000 From: Jasbahr@origin.EA.com (Asbahr, Jason) Date: Thu, 10 Feb 2000 14:42:24 -0600 Subject: [Python-Dev] Python -t Message-ID: <11A17AA2B9EAD111BCEA00A0C9B4179305CB4F90@molach.origin.ea.com> FYI, cross platform coding using MSDev on Windows and Emacs on Unix does seem to turn up more problems with tabs and spaces. It's a standard here to use a 2 space indent level for tabs (even if it's a tab, MSDev presents it as a two-space indent), not 4 or 8 space indents. Users new to the language have reacted amazingly negatively to errors caused by mix tab/space problems. So we've standardized on spaces and every developer has his or her editor set to generate two actual spaces when the tab key is pressed. Tab-free code. ;-) The tab/space issue is a subtle problem, and as Paul mentiond, we don't need errors caused by it to draw the fire of the anti-whitespace camp. Jason Asbahr Origin Systems, Inc. jasbahr@origin.ea.com -----Original Message----- From: Paul Prescod [mailto:paul@prescod.net] Sent: Thursday, February 10, 2000 1:50 PM To: python-dev@python.org Subject: Re: [Python-Dev] Python -t Skip Montanaro wrote: > > Actually, I suspect that many people are in the same boat I'm in. I rarely > need to move code I write off of Unix systems. I had to execute "python > --help" yesterday to learn what -t means. Modules that I made public ages > ago with no consideration of the tab devil have elicited nary a peep about > indentation problems from anyone. (Of course, maybe nobody uses them and > I'm simply deluding myself thinking they might be of interest to > someone... ;-) Mixing tabs and spaces is not all that likely to cause problems, I admit. It's mostly in the hands of newbies starting from scratch (and anti-whitespace zealots) that it will be a problem. > Still, if all tabs or all spaces is the way to go and we can be reasonably > sure that most/all people will have an indentation-friendly editor at their > disposal, then perhaps after a period of time -t should be the default. Well everyone in the world has an editor that either does tabs or spaces. Even "modern" (and I use the term VERY LOOSELY) versions of DOS EDIT.COM seem to preserve spaces. It didn't before. Can you imagine that there is still someone at Microsoft tweaking that code? -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made possible the modern world." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml _______________________________________________ Python-Dev maillist - Python-Dev@python.org http://www.python.org/mailman/listinfo/python-dev From mal@lemburg.com Thu Feb 10 17:46:28 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 10 Feb 2000 18:46:28 +0100 Subject: [Python-Dev] Python -t References: <1261928537-18521181@hypernet.com> Message-ID: <38A2F974.C832F6E2@lemburg.com> Gordon McMillan wrote: > > M.-A. Lemburg wrote: > > > Is there a command line tool out there to untabify existing > > Python scripts ? I recently wanted to apply such a tool to all > > my stuff, but couldn't find any suitable Python script for the job... > > there are lots of tools to catch those tab/space mixes, so no > > finding the cure to the problem kind of surprised me ;-) > > Contrib/system/tabcleaner.py Or: Tools/scripts/untabify.py as Fredrik suggested in private mail. The latter nukes the file permissions though... which is not very elegant :-( Thanks for the pointers, -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From skip@mojam.com (Skip Montanaro) Thu Feb 10 23:07:04 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 10 Feb 2000 17:07:04 -0600 Subject: [Python-Dev] Weekly posting of FAQ index? Message-ID: <200002102307.RAA24856@beluga.mojam.com> Apologies in advance. This isn't really the right place to bring this up, but I didn't want to open up discussion to the entire c.l.py without a little feedback first. I volunteer to take this off-list (maybe there's a more appropriate list already set up meta-sig?) and come back with a recommendation to the assembled masses here or elsewhere. Many newsgroups post FAQs on a regular basis. I wonder if it wouldn't be such a bad idea to try and figure out how to do that with the Python FAQ. A few issues come to mind: 1. Availability in plain text. Is it? I only every use the FAQ wizard any more, so I don't know. 2. How to keep it from percolating to the mailing list? I doubt people are going to want weekly or monthly mailings of the form "Python FAQ part 1/13" on a regular basis. I suspect the mail/news gateway could be trained to recognize this special message, perhaps if it was cross-posted to one other special newsgroup (misc.answers or whatever catches all the FAQs in the known universe). 3. Maybe a completely different form is needed for periodic posting - a Mini-FAQ - which answers the most egregious questions like "why whitespace indentation?" and "how do I run this thing I just downloaded?", then refers to the real FAQ URL for everything else. Comments? Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ "Languages that change by catering to the tastes of non-users tend not to do so well." - Doug Landauer From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 10 23:27:55 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 10 Feb 2000 18:27:55 -0500 (EST) Subject: [Python-Dev] Weekly posting of FAQ index? References: <200002102307.RAA24856@beluga.mojam.com> Message-ID: <14499.18811.550745.556263@anthem.cnri.reston.va.us> >>>>> "SM" == Skip Montanaro writes: SM> 2. How to keep it from percolating to the mailing list? I SM> doubt people are going to want weekly or monthly mailings of SM> the form "Python FAQ part 1/13" on a regular basis. I suspect SM> the mail/news gateway could be trained to recognize this SM> special message, perhaps if it was cross-posted to one other SM> special newsgroup (misc.answers or whatever catches all the SM> FAQs in the known universe). Since he's not here, let me step into Guido's time machine... .................. Kids, don't try /that/ at home. But hey, it worked. If your message has a header 'X-BeenThere: python-list@python.org' it will not be forwarded on the mailing list, 'cause it'll look like it /came/ from the mailing list. Okay, so that's a bit crufty, but you'll forgive me if I didn't quite understand what the blue knob did. -Barry From tim_one@email.msn.com Fri Feb 11 01:32:42 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 10 Feb 2000 20:32:42 -0500 Subject: [Python-Dev] Python -t In-Reply-To: <11A17AA2B9EAD111BCEA00A0C9B4179305CB4F90@molach.origin.ea.com> Message-ID: <000201bf742f$e42247c0$d6a0143f@tim> [Jason Asbahr] > ... > The tab/space issue is a subtle problem, and as Paul mentiond, we > don't need errors caused by it to draw the fire of the anti- > whitespace camp. There are two people on c.l.py who actually use Python and routinely complain about whitespace. One doesn't appear to know how to use his editor; the other never responds to counterpoints. That is, "the anti-whitespace camp" is overwhelmingly composed of people who don't use the language and make crap up as they go along. Note that, in the last three incarnations of this silliness, not a one of them mentioned the "cross platform coding using MSDev on Windows and Emacs on Unix" *actual* clumsiness you mentioned! Facts have no bearing on this argument, so there's nothing rational we can do to avoid "drawing fire". Guido (like us!) is a bit schizophrenic here: he wants to be a benevolent dictator, but also wants to treat people like grownups. This probably worked better before Python got a large American audience <0.9 wink>. The sure way to end this forever is the same way you deal with a child who can't yet deal with a certain freedom responsibly: as some later languages have done, just decree that, henceforth, tab characters in source code are illegal. Period. The howling would be deafening, but-- surprise! --after the first time, the *only* complaint to be made is "Guido won't let me use tab characters bwah bwah"; and every stupid tool ever written to "deal with" this non-problem could be trashed. let's-see-what-the-old-boy-is-made-of-ly y'rs - tim From paul@prescod.net Fri Feb 11 01:48:53 2000 From: paul@prescod.net (Paul Prescod) Date: Thu, 10 Feb 2000 17:48:53 -0800 Subject: [Python-Dev] Python -t References: <000201bf742f$e42247c0$d6a0143f@tim> Message-ID: <38A36A85.9E28C5A5@prescod.net> > as some later languages have > done, just decree that, henceforth, tab characters in source code are > illegal. Period. Why not disallow spaces instead of tabs? Space separated source code is so cramped anyhow. I'm trying to think what "later" languages you are describing. -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made possible the modern world." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml From gvwilson@nevex.com Fri Feb 11 01:50:30 2000 From: gvwilson@nevex.com (gvwilson@nevex.com) Date: Thu, 10 Feb 2000 20:50:30 -0500 (EST) Subject: [Python-Dev] Python -t In-Reply-To: <000201bf742f$e42247c0$d6a0143f@tim> Message-ID: > There are two people on c.l.py who actually use Python and routinely > complain about whitespace... "the anti-whitespace camp" is > overwhelmingly composed of people who don't use the language and make > crap up as they go along. Whic is how many people? How are you counting the people who try Python, become frustrated, give up, and go elsewhere to satisfy their programming needs? ...as was well answered by Diagoras to him that showed him in Neptune's temple the great number of pictures of such as had escaped shipwreck, and had paid their vows to Neptune, saying, 'Advise now, you that think it folly to invocate Neptune in tempest.' 'Yea, but,' saith Diagoras, 'Where are they painted that prayed but drowned?' -- Sir Francis Bacon, "The Advancement of Learning" (1605) From tim_one@email.msn.com Fri Feb 11 02:55:44 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 10 Feb 2000 21:55:44 -0500 Subject: [Python-Dev] Python -t In-Reply-To: <38A36A85.9E28C5A5@prescod.net> Message-ID: <000b01bf743b$7d9d0c40$d6a0143f@tim> [Paul] > Why not disallow spaces instead of tabs? > Space separated source code is > so cramped anyhow. I was thinking maybe more the character "x", to avoid just this argument . > I'm trying to think what "later" languages you > are describing. Have mentioned them on c.l.py as they've popped up; if the answer were of any conceivable interest, I'd pee away time digging it up again <0.1 wink>. From tim_one@email.msn.com Fri Feb 11 02:55:46 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 10 Feb 2000 21:55:46 -0500 Subject: [Python-Dev] Python -t In-Reply-To: Message-ID: <000c01bf743b$7e880880$d6a0143f@tim> [Tim] >> There are two people on c.l.py who actually use Python and routinely >> complain about whitespace... "the anti-whitespace camp" is >> overwhelmingly composed of people who don't use the language and make >> crap up as they go along. [Greg Wilson] > Which is how many people? 3,763. > How are you counting the people who try Python, become frustrated, > give up, and go elsewhere to satisfy their programming needs? One at a time . If they're invisible, they're not part of "the anti-whitespace camp". What's your point? All languages drive thousands of programmers away for all sorts of reasons. I give whitespace no special consideration in this; e.g., Python has likely lost far more potential users in the past for lack of an ANSI or ISO std (which is an official requirement for adoption in many corporations, although the Internet is making such rules a bad joke -- more power to it!). Now that we've repeated every argument made on c.l.py 500 times in the last year, how about let's stop. From tim_one@email.msn.com Fri Feb 11 04:59:00 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 10 Feb 2000 23:59:00 -0500 Subject: [Python-Dev] FW: 64-bit port of Python In-Reply-To: Message-ID: <000f01bf744c$b5821180$d6a0143f@tim> [leaving out areas of universal harmony] >>> "Modules/arraymodule.c::728": >>> >>> static PyObject * >>> array_buffer_info(self, args) >>> arrayobject *self; >>> PyObject *args; >>> { >>> return Py_BuildValue("ll", >>> (long)(self->ob_item), >>> (long)(self->ob_size)); >>> } [Greg Stein] > Yah. That function is quite insane. *shudder* > > It should use PyLong_FromVoidPtr for the first item, and > PyInt_FromLong for the second (IMO, it is reasonable to assume > ob_size fits in a C long). Until someone whines otherwise, ya. > However, I'd simply argue that the function should probably be > punted. What is it for? Haven't used it myself. From the (array module) docs: buffer_info() Return a tuple (address, length) giving the current memory address and the length in bytes of the buffer used to hold array's contents. This is occasionally useful when working with low-level (and inherently unsafe) I/O interfaces that require memory addresses, such as certain ioctl() operations. The returned numbers are valid as long as the array exists and no length-changing operations are applied to it. So we have to assume it's used, and returns what it says it does. [on "id"] > Assuming that we can say that id() is allowed to return a > PyLongObject, then this should just use PyLong_FromVoidPtr. Yes, that would be fine. The docs say it returns "an integer", which generally means it's not specified whether it returns a (Python) int or (Python) long. Andrew is eradicating that distinction anyway. > Tim! Wake up! New functions snuck in behind your back! hehe... Oh, indeed they do! I have been living in a fool's paradise, populated with loving memories. That is, since I stopped writing compilers for a living, the only attention I've given to Python *internals* is when I've had to, either to fix a bug (very rare) or speed something up (even rarer -- it's so bloody fast already ). I'm delighted to see *you're* up to date! [Trent Mick] Trent, you should be on the Python-Dev list if you're going to (as I sure hope you are!) be working on Python internals. This requires Guido's approval, but in his temporary absence I'm channeling his approval for you. Guido can take it up with David Ascher if he doesn't like that (*nobody* messes with David -- the guy is a hard core psycho). [on id] > This means that my System A and System B (above) get different > resultant object types for id() just because the compiler used > for their Python interpreter uses a different data model. So long as they're using different memory models, and so long as Python distinguishes between int and long, I'd say that's *expected*. id()'s are valid only for the life of a single run, and, as Greg said, the only thing you can do with them that's guaranteed to work is compare them for equality (well, you can use cmp on 'em too, but that's unusual and will work fine anyway). > ... > I know that noone should really need to be passing converted > pointer results between platforms, but... id() is not guaranteed to return an address to begin with, so anyone relying on that is hosed regardless. > ... > I want to make a couple of suggestions about this 64-bit > compatibility stuff. I will probably sound like I am on glue > but please bear with me and let me try and convince you that > I am not. It did not sound like you're on glue. It did sound like you have strong opinions about how C "should be" used that don't coincide with the bulk of C programmers' views, and that's enough to stop it right there: part of why extending & embedding Python is so popular is that the API caters to "lowest common denominator" views of C. Non-experts hooking up ancient legacy code is *no problem* now; they need to learn the Python API, but the C part still looks like C . So, in the absence of evidence of *widespread* Win64 problems with Python, I expect everyone here will favor the "find the handful of problems and just fix 'em" approach implicit in what I've said and explicit in what Greg's said. Massive declaration changes don't seem *needed*, and would likely be much more destabilizing (some of us here remember the Great Renaming without fondness ...). > ... > PyInt was tied to C's 'long' based on the (reasonable) assumption > that this would represent the largest native integral type on which > Python was running Na, it was C's "long" specifically, something every C programmer was & is comfortable with. It makes little sense to expose platform-specific extensions to C's set of types as if they were somehow "std". long isn't always good enough anymore for Win64, but it's still good enough everywhere else, and I'd be amazed to see others following MS's strange decision here. Tail, dog, wag . Summary: > On the Python/C API side, use things like: > - PyInt would be tied to intlongest_t > - extern DL_IMPORT(PyObject *) PyInt_FromLongest > Py_PROTO((intlongest_t)); This is exactly the kind of thing that will make the API instantly repellent to the people it's trying to attract. I understand (& agree!) that your scheme is better, but Python is more interested in getting used. > ... > Well, just using 'int' carries the implicit assumption that 'int' > is at least 16-bits wide. ANSI C guarantees that it is, BTW. > (Yes, Tim. ActiveState *is* paying me to look at this stuff.) Heh heh -- I know everything . [back to Greg] > People do use the id() value when they are printing a repr() of > objects. Those uses may overflow, though, because people are > using '%d' or '%x' format codes. It should be %s. %d and %x should get fixed -- the int/long distinction mostly just creates stumbling blocks. [and on to Fredrik] > footnote: assert Unix in (LP32, LP64), according to the single Unix > specification (and if you dig up their rationale, you'll see why > everything else is totally braindead -- I'm usually no Microsoft > basher, but this really pisses me off) Strangely enough, when KSR was doing its 64-bit Unix, it, DEC and Cray were the *only* ones pushing LP64. Everyone else was announcing MS-style plans. By the time they got around to actually building HW and porting code, they changed their minds. Unfortunately, MS has dozens of millions of lines of its *own* cheating code to port, and sizeof(long) == sizeof(DWORD) is a universal bad assumption in that code. I understood more of it a couple years ago (when it was being planned), and expected they would take this short-term easiest (for them) way out; I'm not sure they had a realistic alternative; and *they're* sure they didn't. BTW, when KSR folded, a 128-bit machine was on the drawing board. Nothing lasts forever . From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Fri Feb 11 13:19:14 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Fri, 11 Feb 2000 08:19:14 -0500 (EST) Subject: [Python-Dev] Python -t References: <000201bf742f$e42247c0$d6a0143f@tim> <38A36A85.9E28C5A5@prescod.net> Message-ID: <14500.3154.931818.200641@anthem.cnri.reston.va.us> >>>>> "PP" == Paul Prescod writes: >> as some later languages have done, just decree that, >> henceforth, tab characters in source code are illegal. Period. PP> Why not disallow spaces instead of tabs? Space separated PP> source code is so cramped anyhow. whynotjustgetridofbothspacesandtabsandwhileyoureatitallpunctuationcapitallettersthosedontworkforsomestrangenonamuhrikinlanguagesanywayandheckwhileyoureatitdigitstoomathisforstoopidpeopleanywayscrewthemoronsthatcanthandleitivedoneuseabilityteststhatshowevensixmontholdkidscanwritedesignrulecheckersintheirsleepwithalanguageaskoolastheoneiproposeijustdonthavethemonlinebutillfaxthemtoyouifyousendmeahunnertbucksbarry From Vladimir.Marangozov@inrialpes.fr Fri Feb 11 13:47:31 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 11 Feb 2000 14:47:31 +0100 (CET) Subject: [Python-Dev] Python -t In-Reply-To: <14500.3154.931818.200641@anthem.cnri.reston.va.us> from "Barry A. Warsaw" at Feb 11, 2000 08:19:14 AM Message-ID: <200002111347.OAA14553@python.inrialpes.fr> Barry A. Warsaw wrote: > > whynotjustgetridofbothspacesandtabsandwhileyoureatitallpunctuationcapitallettersthosedontworkforsomestrangenonamuhrikinlanguagesanywayandheckwhileyoureatitdigitstoomathisforstoopidpeopleanywayscrewthemoronsthatcanthandleitivedoneuseabilityteststhatshowevensixmontholdkidscanwritedesignrulecheckersintheirsleepwithalanguageaskoolastheoneiproposeijustdonthavethemonlinebutillfaxthemtoyouifyousendmeahunnertbucksbarry > There's definitely too much information in this very legible text. Theory of Information says "Barry can do much better with a lossless comm channel like pydev". -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From gstein@lyra.org Fri Feb 11 15:27:42 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 11 Feb 2000 07:27:42 -0800 (PST) Subject: [Python-Dev] fyi: tcl/tk 8.3 released Message-ID: http://core.freshmeat.net/news/2000/02/11/950278798.html It may have an impact on the Tkinter stuff... Cheers, -g p.s. the first beta of Perl 5.6 was released a couple days ago, too. but we have no dependencies there :-) -- Greg Stein, http://www.lyra.org/ From skip@mojam.com (Skip Montanaro) Fri Feb 11 20:42:44 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 11 Feb 2000 14:42:44 -0600 Subject: [Python-Dev] non-integer slice indices? Message-ID: <200002112042.OAA03792@beluga.mojam.com> Viorel Preoteasa asked on python-help today about supporting slices with non-integer indexes, e.g.: foo['a':'abc'] = some_sequence Currently the Python interpreter (in the slice_index function of ceval.c) enforces integer slice indices. I won't pretend to provide motivation for non-integral slice indices. Instead, I've CC'd Viorel and will let him chime in if he feels the need. It does seem to me that if the __setslice__ programmer is willing to do the type checking and provide the semantics of "from X to Y" for aribtrary X and Y that the interpreter should let non-integer indices pass. Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ "Languages that change by catering to the tastes of non-users tend not to do so well." - Doug Landauer From trentm@ActiveState.com Fri Feb 11 14:01:50 2000 From: trentm@ActiveState.com (Trent Mick) Date: Fri, 11 Feb 2000 14:01:50 -0000 Subject: [Python-Dev] FW: 64-bit port of Python In-Reply-To: <000f01bf744c$b5821180$d6a0143f@tim> Message-ID: [Tim Peters]: > Trent, you should be on the Python-Dev list if you're going to (as I sure > hope you are!) be working on Python internals. This requires Guido's > approval, but in his temporary absence I'm channeling his > approval for you. > Guido can take it up with David Ascher if he doesn't like that (*nobody* > messes with David -- the guy is a hard core psycho). Thank you Tim! I went through the mailman subsription procedure and got the expected "Subscription was deferred..." message. Will that sit quietly in Guido's or Barry's mailbox? > > [on id] > > This means that my System A and System B (above) get different > > resultant object types for id() just because the compiler used > > for their Python interpreter uses a different data model. > > So long as they're using different memory models, and so long as Python > distinguishes between int and long, I'd say that's *expected*. id()'s are > valid only for the life of a single run, and, as Greg said, the only thing > you can do with them that's guaranteed to work is compare them > for equality > (well, you can use cmp on 'em too, but that's unusual and will work fine > anyway). Unfortunately, the couple of examples that I picked to show that some improper sizeof() assumptions still existed in the code included the id() function. I did not mean to dwell on id() but rather that I thought it would be reasonable that the Python core should run identically on (A) Trillian (Linux64) on itanium (Intel64) and on (B) Win64 on itanium. Maybe not. [Tim's good explanation of why a data typename abstraction layer is a bad thing for Python] Okay, I am convinced. My idealism is showing through. > > Well, just using 'int' carries the implicit assumption that 'int' > > is at least 16-bits wide. > > ANSI C guarantees that it is, BTW. Surprise! I should know that. I suppose I was basing by assumptions on a port of gcc to the HC11 (8-bit microprocessor) that I onced worked with that had 8-bit ints. Trent From pf@artcom-gmbh.de Fri Feb 11 22:12:14 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Fri, 11 Feb 2000 23:12:14 +0100 (MET) Subject: [Python-Dev] doc patch handling (was: patch handling (which was Python 1.6 timing Message-ID: Hi all! First sorry for the intrusion, since I'm not subscribed to this list. I've takeen the opportunity to review the discussion via the pipermail archive and now have a small annotation: AFAIK Fred L. Drake does the actual work on the /Doc sub tree. There is a list python-docs@python.org which should be used for comments (patches?) on the Python documentation. A week ago I submitted several small patches to this address and one of them found its way into the CVS immediately, where the other patches were ignored or delayed? I don't know. (May be bad english? I'm no native speaker) However the new patch submission guide lines on python.org are fine. But additionally the page should point out, *where* to submit patches for the documentation subtree: Either to patches@python.org or to python-docs@python.org as before. It should definitely be the same address, which will be referenced by the mailto:-links on each page of the upcoming Python-1.6-HTML-documentation. Regards from Germany and have nice weekend, Peter -- Peter Funk, Oldenburger Str.86, 27777 Ganderkesee, Tel: 04222 9502 70, Fax: -60 From gstein@lyra.org Fri Feb 11 23:24:07 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 11 Feb 2000 15:24:07 -0800 (PST) Subject: [Python-Dev] FW: 64-bit port of Python In-Reply-To: Message-ID: On Fri, 11 Feb 2000, Trent Mick wrote: >... > Unfortunately, the couple of examples that I picked to show that some > improper sizeof() assumptions still existed in the code included the id() > function. I did not mean to dwell on id() but rather that I thought it would > be reasonable that the Python core should run identically on (A) Trillian > (Linux64) on itanium (Intel64) and on (B) Win64 on itanium. Maybe not. Not a problem. If you find more, then please bring them up! As I mentioned before, Python has been running on 64-bit platforms for at least three years (at Microsoft, we had to get it running on the Alpha in Fall, 1996, so that we could ship Microsoft Merchant Server 1.0). Apparently, we never used the id() function :-) So... if you find anything, then they're bugs to be fixed! Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Fri Feb 11 23:32:45 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 11 Feb 2000 15:32:45 -0800 (PST) Subject: [Python-Dev] non-integer slice indices? In-Reply-To: <200002112042.OAA03792@beluga.mojam.com> Message-ID: On Fri, 11 Feb 2000, Skip Montanaro wrote: > Viorel Preoteasa asked on python-help today about supporting slices with > non-integer indexes, e.g.: > > foo['a':'abc'] = some_sequence > > Currently the Python interpreter (in the slice_index function of ceval.c) > enforces integer slice indices. I won't pretend to provide motivation for > non-integral slice indices. Instead, I've CC'd Viorel and will let him > chime in if he feels the need. It does seem to me that if the __setslice__ > programmer is willing to do the type checking and provide the semantics of > "from X to Y" for aribtrary X and Y that the interpreter should let > non-integer indices pass. Currently, a person can do the following: foo[slice('a','abc')] = some_sequence In other words, you have to first wrap the thing into a slice object. Then, it calls the __setitem__ method with the slice object, which can extract the values using the .start, .stop, and .step attributes. Now... altering the syntax and semantic restrictions (to make it easier) is surely possible, but yah: let's hear some motivations from Viorel. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Fri Feb 11 23:36:31 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 11 Feb 2000 15:36:31 -0800 (PST) Subject: [Python-Dev] doc patch handling In-Reply-To: Message-ID: On Fri, 11 Feb 2000, Peter Funk wrote: >... > AFAIK Fred L. Drake does the actual work on the /Doc sub tree. There is a > list python-docs@python.org which should be used for comments (patches?) > on the Python documentation. > > A week ago I submitted several small patches to this address and one > of them found its way into the CVS immediately, where the other patches > were ignored or delayed? I don't know. (May be bad english? I'm no native > speaker) However the new patch submission guide lines on python.org are fine. > > But additionally the page should point out, *where* to submit patches > for the documentation subtree: Either to patches@python.org > or to python-docs@python.org as before. It should definitely be the same > address, which will be referenced by the mailto:-links on each > page of the upcoming Python-1.6-HTML-documentation. I'd have to go with the idea of submitting doc patches to patches@python.org. If that alias starts to get some automation associated with it, then the doc patches can begin to use that. Personally, I'm not about to interfere with the great doc job that Fred is doing, but using patches@ could distribute his load. Well... whatever Fred would like to do, but that's my thought... :-) Cheers, -g -- Greg Stein, http://www.lyra.org/ From mhammond@skippinet.com.au Sat Feb 12 01:44:53 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Sat, 12 Feb 2000 12:44:53 +1100 Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: <000001bf714b$4152e220$712d153f@tim> Message-ID: Im looking for closure on the Win32 specific exception. Summary for those that missed the thread: * I would like a standard exception for Win32 specific errors. The new hopefully-to-be-standard "win32reg" module could make use of it, the win32 extensions could switch to it. It is, in my mind at least, analogous to OSError. * I proposed deriving the new exception from EnvironmentError, like OSError is. * Fredrik replied with "why not use OSError then?". I answered "due to the numbering systems being different" * Tim agreed that they should be different exceptions if they use different numbering, but pointed out it could still be OSError with creative exception hierarchies. So, in summary, no one disagreed, and the only open issue was the exception hierarchy. My initial proposal called for: StandardError: -> EnvironmentError: -> IOError -> OSError -> Win32Error Which would generally lead to people writing "except Win32Error:" Tim's suggestion (as I understand it) was: StandardError: -> EnvironmentError: -> IOError -> OSError -> CRTError (new - is what os.error would use) -> Win32Error (new) which would lead to people writing "except IOError:" I still like my idea better - fits better with the existing "IOError", and doesnt cause confusion with the change from os.error being OSError to os.error being CRTError. Guido: Can you first make a "yes/no" decision, then (if necessary ) an either/or one? Thanks, Mark. From skip@mojam.com (Skip Montanaro) Sat Feb 12 11:49:33 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Sat, 12 Feb 2000 05:49:33 -0600 (CST) Subject: [Python-Dev] non-integer slice indices? In-Reply-To: References: <200002112042.OAA03792@beluga.mojam.com> Message-ID: <14501.18637.637227.834040@beluga.mojam.com> Greg> Currently, a person can do the following: Greg> foo[slice('a','abc')] = some_sequence Well, I'll be damned! To wit: import types, string, UserDict class SliceableDict(UserDict.UserDict): def __setitem__(self, index, val): if type(index) == types.SliceType: # val must be a sequence. if it's too short, the last # value is replicated. if it's too long, the extra values # are ignored. # keys between index.start and index.stop are assigned elements # of val - index.step is ignored start = index.start stop = index.stop keys = self.data.keys() keys.sort() j = 0 vl = len(val) for k in keys: if index.start <= k < index.stop: self.data[k] = val[j] j = min(j+1, vl-1) else: self.data[index] = val def init_range(self, keys, val=None): for k in keys: self.data[k] = val d = SliceableDict() d.init_range(string.lowercase[0:13], 7) d[slice('a', 'g')] = [12] print d.data Now, about that motivation... Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ "Languages that change by catering to the tastes of non-users tend not to do so well." - Doug Landauer From tismer@tismer.com Sat Feb 12 16:23:36 2000 From: tismer@tismer.com (Christian Tismer) Date: Sat, 12 Feb 2000 17:23:36 +0100 Subject: [Python-Dev] non-integer slice indices? References: <200002112042.OAA03792@beluga.mojam.com> <14501.18637.637227.834040@beluga.mojam.com> Message-ID: <38A58908.6AF91FBE@tismer.com> Hi Skip, Skip Montanaro wrote: > > Greg> Currently, a person can do the following: > > Greg> foo[slice('a','abc')] = some_sequence > > Well, I'll be damned! To wit: [nice implementation cut] > Now, about that motivation... usually I'm not the one to argue against a new feature, but I think this extension to slicing is too much and not consistent enough. When we write x[low:high] = some_sequence then we imply that there is a sequence on the left hand that can be indexed by the implicit ordered set of integers in the range [low, high), and we allow this assignment to change the sequence's length arbitrarily. Speaking of mapping objects, you specify a set of values by an expression of their keys, but you have no way to invent new keys, only deletion applies. Appears a bit twisted to do this to a mapping. A different approach would be to require a mapping object on the right hand. The assignment would have to 1) check that all keys on the right are inside the lice's range 2) delete the entries in that range from the left 3) insert the new keys/values. Indexing a mapping by a slice should return a mapping again. Well, I don't like any of these so much. They make dicts look like something ordered, that rings a bell about too much cheating. Or we could be consequent and provide a sequence protocol for mappings as well, with all that sort-on-demand consequences necessary. But this is not possible since integers can be keys, and it would be undecidable wether we want sequence indexing or mapping indexing. This would only make sense for typed dictionaries, which allow string keys only for instance. I'd say better drop it - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Düppelstr. 31 : *Starship* http://starship.python.net 12163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From bckfnn@worldonline.dk Sat Feb 12 19:38:43 2000 From: bckfnn@worldonline.dk (Finn Bock) Date: Sat, 12 Feb 2000 19:38:43 GMT Subject: [Python-Dev] non-integer slice indices? In-Reply-To: References: Message-ID: <38a5b5f6.268395@smtp.worldonline.dk> [Skip asks about noninteger slices] > foo['a':'abc'] = some_sequence > [Greg writes] >Now... altering the syntax and semantic restrictions (to make it easier) >is surely possible, but yah: let's hear some motivations from Viorel. What about getting full compatibility with jpython as motivation: JPython 1.1 on java1.3.0rc1 (JIT: null) Copyright (C) 1997-1999 Corporation for National Research Initiatives >>> class A: ... def __setitem__(self, idx, value): ... print "__setitem__", `idx`, value ... >>> A()['a':'d'] = '1234' __setitem__ slice('a', 'd', 1) 1234 >>> regards, finn From dascher@mindspring.com Sun Feb 13 03:59:15 2000 From: dascher@mindspring.com (David Ascher) Date: Sat, 12 Feb 2000 19:59:15 -0800 Subject: [Python-Dev] Perl 5.6 beta feature list Message-ID: This is from ActiveState, but I believe that all of the new features in Perl are not ActivePerl-specific. FYI. -----Original Message----- Greetings All, ActiveState is pleased to announce the first beta of ActivePerl 5.6. ActivePerl, including PPM, now supports the following platforms: - Windows - Linux - Solaris This beta release includes all the major features that have been discussed for Perl 5.6. - Globalization and Unicode - Concurrent Interpreters - granularity of warnings - New regex construct - Lvalue subroutines - Subroutine attributes - 64-bit platforms - Large File Systems - Additional documentation and tutorials. The following additional new features are available on the Windows platform: - Unicode system calls - Windows Installer - CPAN extensions - fork() For further information see the release notes or readme file. ActiveState will also shortly make available beta versions of the upcoming new releases of the Perl Dev Kit and PerlEx, which will take advantage of the new features in ActivePerl 5.6. ActiveState cautions that the ActivePerl 5.6 beta release may contain certain incomplete features and known bugs. It is made available for testing purposes only. Download a copy of the ActivePerl Beta from: "http://www.ActiveState.com/cgibin/ActivePerl/download.pl" You can submit bug reports at: "http://bugs.ActiveState.com/ActivePerl" You can send feedback on the new features to: "ActivePerl-Beta@ActiveState.com" Problems downloading the beta? Send mail to: "WebMaster@ActiveState.com" -- The Activators From dascher@mindspring.com Sun Feb 13 04:06:35 2000 From: dascher@mindspring.com (David Ascher) Date: Sat, 12 Feb 2000 20:06:35 -0800 Subject: [Python-Dev] FW: [Patches] Complaints new patches list. Message-ID: Greg: > On Fri, 11 Feb 2000, Gerrit Holl wrote: > > More than a week ago, I sent a patch for some scripts in the Tools/ > > directory. > [...] > There is *NO* guaranteed response time. If you believe so, then go rethink > things. [...] While I agree with Greg's reaction mostly (Gerrit was being childish, which maybe is not so unappropriate for a 14-year old =), I do think that the patches mailing list mechanism is only appropriate in the short-term. In the longer term, a database backend with automatic notification of interested parties when something 'happens' to a patch is needed. Distributed INBOX management just doesn't work (witness python-help). That said, I'm not able to contribute to the development of such a database anymore than anyone else currently, so the last thing I want to do is complain about the status quo. I just wanted to point out that Gerrit's complaint, while poorly phrased, is real, and is likely to be felt by others in the future. I know I've felt similar frustration sending things to the Scriptics folks and seeing patches dissapear into a black hole. I've just learned diplomacy over the years =). --david From gstein@lyra.org Sun Feb 13 06:05:00 2000 From: gstein@lyra.org (Greg Stein) Date: Sat, 12 Feb 2000 22:05:00 -0800 (PST) Subject: [Python-Dev] FW: [Patches] Complaints new patches list. In-Reply-To: Message-ID: On Sat, 12 Feb 2000, David Ascher wrote: >... > While I agree with Greg's reaction mostly (Gerrit was being childish, which > maybe is not so unappropriate for a 14-year old =), I do think that the > patches mailing list mechanism is only appropriate in the short-term. In > the longer term, a database backend with automatic notification of > interested parties when something 'happens' to a patch is needed. > Distributed INBOX management just doesn't work (witness python-help). Our lack of infrastructure is no excuse for Gerrit's position/attitude. As I mentioned in my response, his email came on the heels of an equally inflammatory email sent to webmaster@list.org (me). I was a bit tweaked :-) Sure: he has a point. Sure: we recognize that we could have better facilities. Yes: I didn't like his attitude (for right or wrong). Note: SourceForge has a "Patch Manager". >... > I just wanted to point out that Gerrit's > complaint, while poorly phrased, is real, and is likely to be felt by others > in the future. Recognized. But as you point out: there is only so much we can do. We'll get there, but I don't believe that it gives people license. > I know I've felt similar frustration sending things to the > Scriptics folks and seeing patches dissapear into a black hole. I've just > learned diplomacy over the years =). I know diplomacy, too (although, I'm not as good as David :-). I gave Gerrit the benefit of the doubt [in the list.org mail] and assumed his use of "lie" was a mis-translation from Dutch to English. I said so in my response. He replied, stating that he *did* intend to use that word. Accept abuse once: things are fine; accept it twice, and you encourage continued, future behavior. We're all nice, great people: none of us want to deal with that. Cheers, -g -- Greg Stein, http://www.lyra.org/ From tim_one@email.msn.com Sun Feb 13 07:44:55 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 13 Feb 2000 02:44:55 -0500 Subject: [Python-Dev] Malloc interface (was: [Patches] Re: Garbage collection patches for Python (fwd) In-Reply-To: <200002091507.QAA02647@python.inrialpes.fr> Message-ID: <000501bf75f6$37fe98e0$962d153f@tim> [Vladimir Marangozov] > ... > In order to pave the way for alternatives to libc's malloc, we must > propose a clean solution. Hence it would be good to collect as much > opinions/reactions on the subject as possible and settle on an > interface which would be convenient for everybody. I can't imagine a rational *objection* to cleaning this stuff up. BTW, at work we bumped into a horrible problem with malloc under Linux: Linux "overcommits", meaning that malloc may return success, yet subsequent reference to the malloc'ed memory may segfault! Absolutely sucks, and the Linux developers aren't inclined to fix it (they made an analogy to overbooking plane flights -- even MS wouldn't be so lame ). This showed up on a "small platform" project, where swap space was so small as to be essentially not there. For that and a gazillion other reasons, I (like you) would be much happier if it were easy to plug in a malloc replacement, and I agree that requires reworking of the current code. just-do-it-ly y'rs - tim From gstein@lyra.org Sun Feb 13 08:34:24 2000 From: gstein@lyra.org (Greg Stein) Date: Sun, 13 Feb 2000 00:34:24 -0800 (PST) Subject: [Python-Dev] non-integer slice indices? In-Reply-To: <38a5b5f6.268395@smtp.worldonline.dk> Message-ID: On Sat, 12 Feb 2000, Finn Bock wrote: >... > What about getting full compatibility with jpython as motivation: > > JPython 1.1 on java1.3.0rc1 (JIT: null) > Copyright (C) 1997-1999 Corporation for National Research Initiatives > >>> class A: > ... def __setitem__(self, idx, value): > ... print "__setitem__", `idx`, value > ... > >>> A()['a':'d'] = '1234' > __setitem__ slice('a', 'd', 1) 1234 > >>> Typically, CPython is the reference platform. This would indicate that JPython has a bug. Second, section 5.3.3 of the Language Reference states that the upper and lower bound of a slice must be integers. Again, this would indicate that JPython has a bug. :-) -- Greg Stein, http://www.lyra.org/ From Fredrik Lundh" Message-ID: <00e901bf760c$6df4ad20$34aab5d4@hagrid> David Ascher wrote: > I know I've felt similar frustration sending things to the Scriptics > folks and seeing patches dissapear into a black hole. yeah, but wasn't gerrit's problem that his suggestion *was* implemented, but not by using his exact patch? if submitting patches turns into some kind of land- grabbing game ("hey, I now own that part of the code, since I submitted a patch"), we're in for a really bumpy ride on our way to Py3K. (no, I'm not worried ;-) From bckfnn@worldonline.dk Sun Feb 13 11:54:28 2000 From: bckfnn@worldonline.dk (Finn Bock) Date: Sun, 13 Feb 2000 11:54:28 GMT Subject: [Python-Dev] non-integer slice indices? In-Reply-To: References: Message-ID: <38a69aac.4513920@smtp.worldonline.dk> On Sun, 13 Feb 2000 00:34:24 -0800 (PST), you wrote: >On Sat, 12 Feb 2000, Finn Bock wrote: >>... >> What about getting full compatibility with jpython as motivation: >> >> JPython 1.1 on java1.3.0rc1 (JIT: null) >> Copyright (C) 1997-1999 Corporation for National Research Initiatives >> >>> class A: >> ... def __setitem__(self, idx, value): >> ... print "__setitem__", `idx`, value >> ... >> >>> A()['a':'d'] = '1234' >> __setitem__ slice('a', 'd', 1) 1234 >> >>> >Typically, CPython is the reference platform. I think your are confusing a large and rich legacy with beeing the reference <0.7 wink>. >This would indicate that JPython has a bug. At worst, I would call it an overgeneralization. >Second, section 5.3.3 of the Language Reference states that the upper and >lower bound of a slice must be integers. Again, this would indicate that >JPython has a bug. Is it not only "simple slicing" which calls for integer expressions? So IMHO, it seems that JPython does follows the text of 5.3.3. regards, finn From klm@digicool.com Sun Feb 13 15:40:01 2000 From: klm@digicool.com (Ken Manheimer) Date: Sun, 13 Feb 2000 10:40:01 -0500 (EST) Subject: [Python-Dev] FW: [Patches] Complaints new patches list. In-Reply-To: Message-ID: On Sat, 12 Feb 2000, David Ascher wrote: > maybe is not so unappropriate for a 14-year old =), I do think that the > patches mailing list mechanism is only appropriate in the short-term. In > the longer term, a database backend with automatic notification of > interested parties when something 'happens' to a patch is needed. > Distributed INBOX management just doesn't work (witness python-help). > > That said, I'm not able to contribute to the development of such a database > anymore than anyone else currently, so the last thing I want to do is > complain about the status quo. I just wanted to point out that Gerrit's > complaint, while poorly phrased, is real, and is likely to be felt by others > in the future. I know I've felt similar frustration sending things to the > Scriptics folks and seeing patches dissapear into a black hole. I've just > learned diplomacy over the years =). One of the four items in the software carpentry competition is an issue tracker, which might fit the bill. Depending on what value of "short-term" we're able to tolerate, and the success of the software-carpentry exercise. Maybe we can wait for the results, or is year too long? In fact, i should have an entry based on a zope-based tracker that we're starting to use here, which i believe does address some of the issues. We should have my initial version going for some trial projects on zope.org soon, so all could see - other commitments and quirks of fate prevent me from concentrating on packaging it up for, eg, python.org folks to evaluate. (My tracker only uses email for notices about activities on issues, actions are all done via the web, i seem to recall some people explicitly stating email actions are needed.) I'll put a notice where to look, when it's viewable, in case it's of interest. Ken klm@digicool.com From akuchlin@mems-exchange.org Sun Feb 13 17:12:29 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Sun, 13 Feb 2000 12:12:29 -0500 (EST) Subject: [Python-Dev] FW: [Patches] Complaints new patches list. In-Reply-To: References: Message-ID: <14502.58877.344880.56489@newcnri.cnri.reston.va.us> Greg Stein writes: >Our lack of infrastructure is no excuse for Gerrit's position/attitude. As >I mentioned in my response, his email came on the heels of an equally >inflammatory email sent to webmaster@list.org (me). I was a bit Erm... let me stick up for Gerrit here and point out that www.python.org/patches/ says: Each submitted patch (except spam) will receive at least one mail back regarding the status of the patch. If you don't hear from us within two weeks, it's possible that your mail didn't arrive or that it got lost; please ping us politely. We don't guarantee a turn-around time for patches (sometimes it's really hard to make a decision). This text mentions a time span ("two weeks") and says "*will* receive" -- not "may", not "might", but "will". While Gerrit's e-mails have been a bit immoderate, frankly so was Greg Stein's immediate response. Can we just draw a line here and bring this thread to a close? --amk From Moshe Zadka Sun Feb 13 17:46:30 2000 From: Moshe Zadka (Moshe Zadka) Date: Sun, 13 Feb 2000 19:46:30 +0200 (IST) Subject: [Python-Dev] FW: [Patches] Complaints new patches list. In-Reply-To: <14502.58877.344880.56489@newcnri.cnri.reston.va.us> Message-ID: On Sun, 13 Feb 2000, Andrew Kuchling wrote: > This text mentions a time span ("two weeks") and says "*will* receive" > -- not "may", not "might", but "will". While Gerrit's e-mails have > been a bit immoderate, frankly so was Greg Stein's immediate response. > Can we just draw a line here and bring this thread to a close? I guess that would be a Bad Time(TM) for me to ask whether anyone had a look at my newest overloading-in patch, which it does it "right"? I'd love to hear some comments... -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From fredrik@pythonware.com Mon Feb 14 09:35:47 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Mon, 14 Feb 2000 10:35:47 +0100 Subject: [Python-Dev] Malloc interface (was: [Patches] Re: Garbage collection patches for Python (fwd) References: <200002101508.QAA04484@python.inrialpes.fr> Message-ID: <008801bf76ce$e1165160$f29b12c2@secret.pythonware.com> Vladimir Marangozov wrote: > > is XDEL simply a "if (p) free(p)" variant? >=20 > Yes, see mymalloc.h. >=20 > > if so, don't forget that ANSI C requires that free() does the right > > thing if handled a NULL pointer. since Python 1.6 will be ANSI C, > > it's probably better to force users of broken platforms to work > > around bugs in PyMem_FREE, rather than expose two different > > alternatives. I'm pretty sure there are standard macros for auto- > > conf that tests for this. >=20 > I'm aware of this, but I think Guido has included PyMem_XDEL for 2 = reasons: >=20 > a) it makes us foresee (and be explicit) that the pointer may be NULL > b) if the pointer is NULL, the cost of the function call is avoided. but if the pointer is not NULL, you end up with an extra test. and having read lots of python source code lately, I'm pretty sure that's a far more common case. given this, and an ANSI compiler, XDEL is pretty meaning- less. and since Python 1.6 won't compile under a non-ANSI compiler, I think we can require ANSI libraries too. (if it's important to make that test inline, spell it out). From Vladimir.Marangozov@inrialpes.fr Mon Feb 14 10:18:27 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Mon, 14 Feb 2000 11:18:27 +0100 (CET) Subject: [Python-Dev] Malloc interface (was: [Patches] Re: Garbage collection patches for Python (fwd) In-Reply-To: <008801bf76ce$e1165160$f29b12c2@secret.pythonware.com> from "Fredrik Lundh" at Feb 14, 2000 10:35:47 AM Message-ID: <200002141018.LAA10708@python.inrialpes.fr> Fredrik Lundh wrote: > > Vladimir Marangozov wrote: > > > is XDEL simply a "if (p) free(p)" variant? > > > > Yes, see mymalloc.h. > > > > > if so, don't forget that ANSI C requires that free() does the right > > > thing if handled a NULL pointer. since Python 1.6 will be ANSI C, > > > it's probably better to force users of broken platforms to work > > > around bugs in PyMem_FREE, rather than expose two different > > > alternatives. I'm pretty sure there are standard macros for auto- > > > conf that tests for this. > > > > I'm aware of this, but I think Guido has included PyMem_XDEL for 2 reasons: > > > > a) it makes us foresee (and be explicit) that the pointer may be NULL > > b) if the pointer is NULL, the cost of the function call is avoided. > > but if the pointer is not NULL, you end up with an > extra test. and having read lots of python source > code lately, I'm pretty sure that's a far more common > case. PyMem_XDEL is very rare in the distribution... > > given this, and an ANSI compiler, XDEL is pretty meaning- > less. and since Python 1.6 won't compile under a non-ANSI > compiler, I think we can require ANSI libraries too. Okay, so you can use PyMem_DEL directly. > > (if it's important to make that test inline, spell it out). > > > PyMem_XDEL spells it out, but the choice is to the programmer. If your point is to remove it, or to rename it to PyMem_DEL in the distribution, well, Guido has to speak up (but I doubt he'll remove it, for backwards compatibility reasons -- 3rd party extensions may have used it). -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From fredrik@pythonware.com Mon Feb 14 11:37:10 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Mon, 14 Feb 2000 12:37:10 +0100 Subject: [Python-Dev] Malloc interface (was: [Patches] Re: Garbage collection patches for Python (fwd) References: <200002141018.LAA10708@python.inrialpes.fr> Message-ID: <003f01bf76df$d50b5ee0$f29b12c2@secret.pythonware.com> Vladimir Marangozov wrote: > PyMem_XDEL is very rare in the distribution... "if (p) free(p)" is quite common, though. > Okay, so you can use PyMem_DEL directly. which means that PyMem_DEL works just fine for NULL pointers on almost all plat- forms, while PyMem_XDEL works on all platforms, but is slower if the pointer is usually not NULL. seems a bit messy to me... > If your point is to remove it, or to rename it to PyMem_DEL in > the distribution my proposal is to have only one of these, and force people on non-ANSI platforms to provide a workaround. From gstein@lyra.org Mon Feb 14 11:33:28 2000 From: gstein@lyra.org (Greg Stein) Date: Mon, 14 Feb 2000 03:33:28 -0800 (PST) Subject: [Python-Dev] Malloc interface In-Reply-To: <003f01bf76df$d50b5ee0$f29b12c2@secret.pythonware.com> Message-ID: On Mon, 14 Feb 2000, Fredrik Lundh wrote: >... > my proposal is to have only one of these, > and force people on non-ANSI platforms > to provide a workaround. I'm with Fredrik. Have a single DEL function. If a non-conformant platform is being used, then ./configure can easily detect it and insert a workaround. And don't argue about a memory test introduced by ./configure. Realize that ANSI's free() has a memory test inside it, too. Somebody, somewhere, has to check for that NULL pointer possibility. configure will just determine whether free() does it, or Python does it. Cheers, -g -- Greg Stein, http://www.lyra.org/ From Vladimir.Marangozov@inrialpes.fr Mon Feb 14 11:43:19 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Mon, 14 Feb 2000 12:43:19 +0100 (CET) Subject: [Python-Dev] Malloc interface (was: [Patches] Re: Garbage collection patches for Python (fwd) In-Reply-To: <003f01bf76df$d50b5ee0$f29b12c2@secret.pythonware.com> from "Fredrik Lundh" at Feb 14, 2000 12:37:10 PM Message-ID: <200002141143.MAA11231@python.inrialpes.fr> Fredrik Lundh wrote: > > Vladimir Marangozov wrote: > > PyMem_XDEL is very rare in the distribution... > > "if (p) free(p)" is quite common, though. > > > Okay, so you can use PyMem_DEL directly. > > which means that PyMem_DEL works just > fine for NULL pointers on almost all plat- > forms, while PyMem_XDEL works on all > platforms, but is slower if the pointer is > usually not NULL. > > seems a bit messy to me... > > > If your point is to remove it, or to rename it to PyMem_DEL in > > the distribution > > my proposal is to have only one of these, > and force people on non-ANSI platforms > to provide a workaround. > > Agreed. So the concrete proposal is this, right? (mymalloc.h) ... #define PyMem_FREE(p) free((ANY *)p) ... #define PyMem_DEL(p) PyMem_FREE(p) #define PyMem_XDEL(p) PyMem_FREE(p) /* obsolete */ -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From gstein@lyra.org Mon Feb 14 11:48:41 2000 From: gstein@lyra.org (Greg Stein) Date: Mon, 14 Feb 2000 03:48:41 -0800 (PST) Subject: [Python-Dev] Malloc interfaced In-Reply-To: <200002141143.MAA11231@python.inrialpes.fr> Message-ID: On Mon, 14 Feb 2000, Vladimir Marangozov wrote: >... > Agreed. So the concrete proposal is this, right? > > (mymalloc.h) > > ... > #define PyMem_FREE(p) free((ANY *)p) Small nit: #define PyMem_FREE(p) free((ANY *)(p)) (yes, the "bug" exists in 1.5.2, too) > ... > > #define PyMem_DEL(p) PyMem_FREE(p) > #define PyMem_XDEL(p) PyMem_FREE(p) /* obsolete */ Both of these would be obsolete. Note that the PyMem_NEW/RESIZE/FREE macros are intended for internal use by Python only. (move to a private header?) Py_Malloc and friends, and PyMem_Malloc and friends are to be used by C extensions (or embeddors). Cheers, -g -- Greg Stein, http://www.lyra.org/ From fredrik@pythonware.com Mon Feb 14 13:28:44 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Mon, 14 Feb 2000 14:28:44 +0100 Subject: [Python-Dev] FW: [Patches] Complaints new patches list. References: Message-ID: <004d01bf76ef$6bc6df80$f29b12c2@secret.pythonware.com> David wrote: > I know I've felt similar frustration sending things to the > Scriptics folks and seeing patches dissapear into a black > hole. fwiw, the "Tcl Developer Xchange" just opened: http://dev.scriptics.com/ among other things, they claim that Python is not embeddable (!), has no i18n support, and isn't thread safe. we're better off than Perl, though... ;-) From Vladimir.Marangozov@inrialpes.fr Mon Feb 14 13:28:46 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Mon, 14 Feb 2000 14:28:46 +0100 (CET) Subject: [Python-Dev] Malloc interfaced In-Reply-To: from "Greg Stein" at Feb 14, 2000 03:48:41 AM Message-ID: <200002141328.OAA11353@python.inrialpes.fr> Greg Stein wrote: > > > ... > > > > #define PyMem_DEL(p) PyMem_FREE(p) > > #define PyMem_XDEL(p) PyMem_FREE(p) /* obsolete */ > > Both of these would be obsolete. This makes a lot of sense to me. (Oh boy, I've spent big time this weekend trying to clean up the distrib, discovering subtle errors, etc, and now that I think I have something ready, you suggest me rethink the interface and rename DEL to FREE in the whole distribution? :-) > > Note that the PyMem_NEW/RESIZE/FREE macros are intended for internal use > by Python only. (move to a private header?) > > Py_Malloc and friends, and PyMem_Malloc and friends are to be used by C > extensions (or embeddors). I couldn't agree more. (modulo that hiding the macros in a pvt header may cause gobs of feedback like "Why PyMem_DEL isn't available anymore? I use it to release my objects allocated with PyObject_NEW"...). Let me give you a status: Currently, malloc/realloc/free/PyMem_NEW/RESIZE/DEL & PyObject_NEW are used in the core and in the extensions without respecting any particular rules, because such rules didn't exist. Everything relies on the assumption that these primitives boil down to malloc/realloc/free. And this is what prevents Python from being user-malloc friendly (although "hostile" would be a better word ;) After extensive malloc figthing, I have a working patch which corrects this aspect of the code (core + extensions in Modules/*), but preserves the PyMem_MALLOC/REALLOC/FREE and PyMem_NEW/RESIZE/DEL families entirely, assuming they're public macros. With Fredrik's and Greg's remarks included, mymalloc.h would look like: ----------- /* Python's raw memory interface. To make the interpreter user-malloc friendly, all other memory or object APIs are implemented on top of this one. These can be changed to make the interpreter use another allocator. */ ... #define PyMem_MALLOC(n) malloc(n) #define PyMem_REALLOC(p, n) realloc((ANY *)(p), (n)) #define PyMem_FREE(p) free((ANY *)(p)) /* Type-oriented memory interface. */ #define PyMem_NEW(type, n) \ ( (type *) PyMem_MALLOC(_PyMem_EXTRA + (n) * sizeof(type)) ) #define PyMem_RESIZE(p, type, n) \ if ((p) == NULL) \ (p) = (type *) PyMem_MALLOC(_PyMem_EXTRA + (n) * sizeof(type)); \ else \ (p) = (type *) PyMem_REALLOC((p), \ _PyMem_EXTRA + (n) * sizeof(type)) #define PyMem_DEL(p) PyMem_FREE(p) /* obsolete */ #define PyMem_XDEL(p) PyMem_FREE(p) /* obsolete */ ---------- Indeed, I'm inclined to zap all PyMem_DEL and rename them to PyMem_FREE in the core, thus leaving PyMem_NEW & PyMem_RESIZE as handy shotcuts. I'm okay to put all these macros in a private header too, but... Problem: There's a lot of code (core + Modules/* + perhaps 3rd party) which says: o = PyObject_NEW(, &); then: Mem_DEL(o); This is messy. This is the wrong NEW/DEL pair for extension modules. It's acceptable to use this for the core, but for C extensions, the right pair is PyObject_NEW()/PyMem_Free(). PyObject_NEW is a public C interface for creating objects (defined as _PyObject_New, which calls PyMem_MALLOC). The public destructor function should be named PyObject_DEL, which should be defined as PyMem_Free (which in turn calls PyMem_FREE). And this is where we are at the moment. No PyObject_DEL, spaghetti code. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From fdrake@acm.org Mon Feb 14 16:02:04 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 14 Feb 2000 11:02:04 -0500 (EST) Subject: [Python-Dev] doc patch handling (was: patch handling (which was Python 1.6 timing In-Reply-To: References: Message-ID: <14504.9980.588564.440787@weyr.cnri.reston.va.us> Peter Funk writes: > A week ago I submitted several small patches to this address and one > of them found its way into the CVS immediately, where the other patches > were ignored or delayed? I don't know. (May be bad english? I'm no native > speaker) However the new patch submission guide lines on python.org are fine. Peter, They aren't being ignored, and I do intend to integrate them. They were just longer than the one I did handle right away, and I didn't have the time to deal with them at the moment. > But additionally the page should point out, *where* to submit patches > for the documentation subtree: Either to patches@python.org > or to python-docs@python.org as before. It should definitely be the same > address, which will be referenced by the mailto:-links on each > page of the upcoming Python-1.6-HTML-documentation. Or even the more immediate 1.5.2p2 documentation. ;) I'll see about getting that page fixed, either the hard way or by coercing Guido to make the edits. > Regards from Germany and have nice weekend, Peter Thanks, I did. Even the kids got tired. ;) -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From fdrake@acm.org Mon Feb 14 19:52:07 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 14 Feb 2000 14:52:07 -0500 (EST) Subject: [Python-Dev] Documentation patches Message-ID: <14504.23783.932818.209084@weyr.cnri.reston.va.us> I've just updated the page at python.org/patches/ to ask that documentation patches be sent to python-docs@python.org, with specific bug reports to be submitted through the bugs database (which I monitor). Hopefully this will clarify what should happen for documentation issues. It also means that the published documentation on the Web site isn't out of date! -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From fdrake@cnri.reston.va.us Mon Feb 14 20:56:34 2000 From: fdrake@cnri.reston.va.us (Fred L. Drake, Jr.) Date: Mon, 14 Feb 2000 15:56:34 -0500 (EST) Subject: [Python-Dev] doc patch handling In-Reply-To: References: Message-ID: <14504.27650.860605.525498@weyr.cnri.reston.va.us> Greg Stein writes: > Personally, I'm not about to interfere with the great doc job that Fred is > doing, but using patches@ could distribute his load. Are you volunteering to take over? ;) Or are you just saying that your little documentation assignment is ready to hand in? I'll be waiting for *that* patch! -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Tue Feb 15 00:28:58 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Mon, 14 Feb 2000 19:28:58 -0500 (EST) Subject: [Python-Dev] non-integer slice indices? References: <38a5b5f6.268395@smtp.worldonline.dk> Message-ID: <14504.40394.901801.69623@anthem.cnri.reston.va.us> >>>>> "GS" == Greg Stein writes: GS> Typically, CPython is the reference platform. This would GS> indicate that JPython has a bug. GS> Second, section 5.3.3 of the Language Reference states that GS> the upper and lower bound of a slice must be integers. Again, GS> this would indicate that JPython has a bug. I think instead you have a case where JimH was (not-so?)subtly trying to push Guido in a certain direction. :) -Barry From gstein@lyra.org Tue Feb 15 00:34:12 2000 From: gstein@lyra.org (Greg Stein) Date: Mon, 14 Feb 2000 16:34:12 -0800 (PST) Subject: [Python-Dev] non-integer slice indices? In-Reply-To: <14504.40394.901801.69623@anthem.cnri.reston.va.us> Message-ID: On Mon, 14 Feb 2000, Barry A. Warsaw wrote: > >>>>> "GS" == Greg Stein writes: > GS> Typically, CPython is the reference platform. This would > GS> indicate that JPython has a bug. > > GS> Second, section 5.3.3 of the Language Reference states that > GS> the upper and lower bound of a slice must be integers. Again, > GS> this would indicate that JPython has a bug. > > I think instead you have a case where JimH was (not-so?)subtly trying > to push Guido in a certain direction. :) Yah... :-) Guido always reserves the right to change the language definition. At the moment, though, it is integers only (if you want to be portable across Python implementations). As MarkH would say: I'm not fussed about it. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Tue Feb 15 00:59:45 2000 From: gstein@lyra.org (Greg Stein) Date: Mon, 14 Feb 2000 16:59:45 -0800 (PST) Subject: [Python-Dev] doc patch handling In-Reply-To: <14504.27650.860605.525498@weyr.cnri.reston.va.us> Message-ID: On Mon, 14 Feb 2000, Fred L. Drake, Jr. wrote: > Greg Stein writes: > > Personally, I'm not about to interfere with the great doc job that Fred is > > doing, but using patches@ could distribute his load. > > Are you volunteering to take over? ;) Take over? No. Assist with doc patches? Sure. I see them as similar to any other patch. If they arrived at patches@, then I'd process them just like code patches. > Or are you just saying that > your little documentation assignment is ready to hand in? I'll be > waiting for *that* patch! Tomorrow. I released two of my apps last Friday (mod_dav and ViewCVS; deferring an edna release for now), and the doc that I need to write is the next project on my plate. I'd normally say that you'd get some doc tonite, but it's Valentine's Day. *wink wink nudge nudge* Cheers, -g -- Greg Stein, http://www.lyra.org/ From fdrake@acm.org Tue Feb 15 15:09:09 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 15 Feb 2000 10:09:09 -0500 (EST) Subject: [Python-Dev] doc patch handling In-Reply-To: References: <14504.27650.860605.525498@weyr.cnri.reston.va.us> Message-ID: <14505.27669.516809.649849@weyr.cnri.reston.va.us> Greg Stein writes: > Take over? No. Assist with doc patches? Sure. I see them as similar to > any other patch. If they arrived at patches@, then I'd process them just > like code patches. Actually, I wonder how many people would want to be seriously involved in the documentation work. If others besides myself & fellow CNRI staff could make check-ins, how many of us would be interested over the long haul? > Tomorrow. I released two of my apps last Friday (mod_dav and ViewCVS; > deferring an edna release for now), and the doc that I need to write is > the next project on my plate. I'd normally say that you'd get some doc > tonite, but it's Valentine's Day. *wink wink nudge nudge* Yeah, I even remembered before the last minute this year. ;) -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From Moshe Zadka Tue Feb 15 15:55:36 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 15 Feb 2000 17:55:36 +0200 (IST) Subject: [Python-Dev] doc patch handling In-Reply-To: <14505.27669.516809.649849@weyr.cnri.reston.va.us> Message-ID: On Tue, 15 Feb 2000, Fred L. Drake, Jr. wrote: > Greg Stein writes: > > Take over? No. Assist with doc patches? Sure. I see them as similar to > > any other patch. If they arrived at patches@, then I'd process them just > > like code patches. > > Actually, I wonder how many people would want to be seriously > involved in the documentation work. If others besides myself & fellow > CNRI staff could make check-ins, how many of us would be interested > over the long haul? I'd like to. brief-ly y'rs, Z. -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From mal@lemburg.com Wed Feb 16 10:36:31 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 16 Feb 2000 11:36:31 +0100 Subject: [Python-Dev] Python on 64-bit AIX Message-ID: <38AA7DAF.B7C48EC6@lemburg.com> Hi everybody, I've noted some discussion about Python on 64-bit platforms on this mailing list lately. The reason I'm posting is that I am currently installing a Python 1.5 interpreter at a client's site which is running AIX 4.3.2 on a 2-processor PowerPC machine. It will use my mx Extensions as add-ons. The compiler used is gcc 2.95.1. Linking is done with the AIX linker. Since I can't possibly check all situations where this setup might fail, I would like to know if anybody else has made some experience with similar setups. Python does compile (I had to disable the fcntl module though) and seems to run fine. My mx Extensions also compile and import properly, but I haven't run any extended tests yet. Has Python already been tested extensively on 64-bit machines ? Thanks for any hints, -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From fredrik@pythonware.com Wed Feb 16 10:49:48 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Wed, 16 Feb 2000 11:49:48 +0100 Subject: [Python-Dev] Python on 64-bit AIX References: <38AA7DAF.B7C48EC6@lemburg.com> Message-ID: <007f01bf786b$8c105e60$f29b12c2@secret.pythonware.com> > Has Python already been tested extensively on 64-bit machines ? python works perfectly fine on the AXP platform (and has done so since 1.2, at least). should work fine on any LP64 platform. (given good enough compilers, that is. we don't use gcc on AXP -- earlier gcc's didn't work well, and DEC's own tools are not only excellent, but they're also shipped with the OS) From gstein@lyra.org Wed Feb 16 11:01:12 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 16 Feb 2000 03:01:12 -0800 (PST) Subject: [Python-Dev] Python on 64-bit AIX In-Reply-To: <38AA7DAF.B7C48EC6@lemburg.com> Message-ID: On Wed, 16 Feb 2000, M.-A. Lemburg wrote: >... > Has Python already been tested extensively on 64-bit machines ? Back in 1996, Microsoft used Python 1.4 on Alpha machines for the Microsoft Merchant Server 1.0 product. The thing was seriously stress-tested and code coverage was applied to the Python source (not the C interpreter!). It ran quite well for us. Any 64-bit issues that we found were sent to Mark/Guido long ago. In 1997, we shipped Site Server 2.0; the Commerce Server portion used Python to implement some COM objects. Again: that was well-tested on Alpha machines. In 1998, when Site Server 3.0 came out, the use of Python was quite minimal by that point, and the Alpha testing was also minimal. We used a Python 1.4 base for all three product releases. The Site Server products also had my "free threading patches" applied, and used a snapshot of the win32com stuff to implement the COM stuff. Of course, we didn't use all of Python. The particular subset that we used worked great. So... Your Mileage May Vary. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Wed Feb 16 11:57:05 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 16 Feb 2000 03:57:05 -0800 (PST) Subject: [Python-Dev] doc patch handling In-Reply-To: <14505.27669.516809.649849@weyr.cnri.reston.va.us> Message-ID: On Tue, 15 Feb 2000, Fred L. Drake, Jr. wrote: >... > Actually, I wonder how many people would want to be seriously > involved in the documentation work. If others besides myself & fellow > CNRI staff could make check-ins, how many of us would be interested > over the long haul? I posited a while back that I think a few more people may be inclined to assist if they could make direct checkins. I know that it would certainly be easier for others to assist in many cases, and it could reduce the burden on the CNRI staff. [ the XML-SIG moved their repository "off-site" so that AMK wouldn't be the gating factor for checkins from all the contributors... ] Cheers, -g -- Greg Stein, http://www.lyra.org/ From petrilli@amber.org Wed Feb 16 14:59:08 2000 From: petrilli@amber.org (Christopher Petrilli) Date: Wed, 16 Feb 2000 09:59:08 -0500 Subject: [Python-Dev] Python on 64-bit AIX In-Reply-To: <007f01bf786b$8c105e60$f29b12c2@secret.pythonware.com>; from fredrik@pythonware.com on Wed, Feb 16, 2000 at 11:49:48AM +0100 References: <38AA7DAF.B7C48EC6@lemburg.com> <007f01bf786b$8c105e60$f29b12c2@secret.pythonware.com> Message-ID: <20000216095908.A19455@trump.amber.org> Fredrik Lundh [fredrik@pythonware.com] wrote: > > Has Python already been tested extensively on 64-bit machines ? > > python works perfectly fine on the AXP platform > (and has done so since 1.2, at least). should work > fine on any LP64 platform. > > (given good enough compilers, that is. we don't use gcc on > AXP -- earlier gcc's didn't work well, and DEC's own tools are > not only excellent, but they're also shipped with the OS) I think this is an important ote. GCC has some serious flaws in its 64-bit support on various platforms (and I don't believe supports PowerII/III and the 64 bit architecture). My experience has been that on AXP, PPC and POWER systems that the vendor compilers are not only more reliable, but substantially faster (on POWER, it's 50% or more in many cases). The joy with AIX compilers is in finding the right combination of switches to get it doing what you want. Start with 'c98' rather than 'cc' and you'll be much happier :-) Chris -- | Christopher Petrilli | petrilli@amber.org From fdrake@acm.org Wed Feb 16 14:50:18 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 16 Feb 2000 09:50:18 -0500 (EST) Subject: [Python-Dev] Python on 64-bit AIX In-Reply-To: <007f01bf786b$8c105e60$f29b12c2@secret.pythonware.com> References: <38AA7DAF.B7C48EC6@lemburg.com> <007f01bf786b$8c105e60$f29b12c2@secret.pythonware.com> Message-ID: <14506.47402.56596.290048@weyr.cnri.reston.va.us> Fredrik Lundh writes: > (given good enough compilers, that is. we don't use gcc on > AXP -- earlier gcc's didn't work well, and DEC's own tools are > not only excellent, but they're also shipped with the OS) That depends on the OS, doesn't it? ;) I doubt RedHat for the alpha ships DEC compilers, and I understand that Linux is selling more AXP chips than Tru64 Unix (as it's called these days). -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From mal@lemburg.com Wed Feb 16 15:18:19 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 16 Feb 2000 16:18:19 +0100 Subject: [Python-Dev] Python on 64-bit AIX References: Message-ID: <38AABFBB.4DC4C5BA@lemburg.com> Greg Stein wrote: > > On Wed, 16 Feb 2000, M.-A. Lemburg wrote: > >... > > Has Python already been tested extensively on 64-bit machines ? > > Back in 1996, Microsoft used Python 1.4 on Alpha machines for the > Microsoft Merchant Server 1.0 product. The thing was seriously > stress-tested and code coverage was applied to the Python source (not the > C interpreter!). It ran quite well for us. Any 64-bit issues that we found > were sent to Mark/Guido long ago. > > In 1997, we shipped Site Server 2.0; the Commerce Server portion used > Python to implement some COM objects. Again: that was well-tested on Alpha > machines. In 1998, when Site Server 3.0 came out, the use of Python was > quite minimal by that point, and the Alpha testing was also minimal. We > used a Python 1.4 base for all three product releases. The Site Server > products also had my "free threading patches" applied, and used a snapshot > of the win32com stuff to implement the COM stuff. > > Of course, we didn't use all of Python. The particular subset that we used > worked great. So... Your Mileage May Vary. What I'm particularly interested in is the socket module, which is being used rather heavily for inter process communication. Anyway, your comments do sound promising -- after all Python 1.5.x has gone a long way since the release of Python 1.4. Thanks for the Good News ;-), -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Wed Feb 16 15:15:58 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 16 Feb 2000 16:15:58 +0100 Subject: [Python-Dev] Python on 64-bit AIX References: <38AA7DAF.B7C48EC6@lemburg.com> <007f01bf786b$8c105e60$f29b12c2@secret.pythonware.com> <20000216095908.A19455@trump.amber.org> Message-ID: <38AABF2E.34F00A40@lemburg.com> Christopher Petrilli wrote: > > Fredrik Lundh [fredrik@pythonware.com] wrote: > > > Has Python already been tested extensively on 64-bit machines ? > > > > python works perfectly fine on the AXP platform > > (and has done so since 1.2, at least). should work > > fine on any LP64 platform. > > > > (given good enough compilers, that is. we don't use gcc on > > AXP -- earlier gcc's didn't work well, and DEC's own tools are > > not only excellent, but they're also shipped with the OS) > > I think this is an important ote. GCC has some serious flaws in its 64-bit > support on various platforms (and I don't believe supports PowerII/III > and the 64 bit architecture). My experience has been that on AXP, PPC > and POWER systems that the vendor compilers are not only more reliable, > but substantially faster (on POWER, it's 50% or more in many cases). > > The joy with AIX compilers is in finding the right combination of switches > to get it doing what you want. Start with 'c98' rather than 'cc' and you'll > be much happier :-) Well, it does seem to compile linkable programs and I even got Python and my extensions all running. I would have liked to use the native OS compilers but the client has a rather strange attitude towards: the argument is to use GNU tools out- side the firewall and AIX tools on the inside -- no idea what this buys for him, but that's the reason why I'm stuck to gcc and a few undefined symbols in some libs :-( Thanks for your comments, -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From Fredrik Lundh" <007f01bf786b$8c105e60$f29b12c2@secret.pythonware.com> <14506.47402.56596.290048@weyr.cnri.reston.va.us> Message-ID: <007b01bf78ab$96003540$34aab5d4@hagrid> Fred L. Drake, Jr. wrote: > That depends on the OS, doesn't it? ;) I doubt RedHat for the > alpha ships DEC compilers, and I understand that Linux is selling more = > AXP chips than Tru64 Unix (as it's called these days). yeah, but serious developers use DEC C also on Linux: http://www.unix.digital.com/linux/compaq_c/index.html From fdrake@acm.org Wed Feb 16 18:58:45 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 16 Feb 2000 13:58:45 -0500 (EST) Subject: [Python-Dev] Python on 64-bit AIX In-Reply-To: <007b01bf78ab$96003540$34aab5d4@hagrid> References: <38AA7DAF.B7C48EC6@lemburg.com> <007f01bf786b$8c105e60$f29b12c2@secret.pythonware.com> <14506.47402.56596.290048@weyr.cnri.reston.va.us> <007b01bf78ab$96003540$34aab5d4@hagrid> Message-ID: <14506.62309.377480.224978@weyr.cnri.reston.va.us> Fredrik Lundh writes: > yeah, but serious developers use DEC C also on Linux: > http://www.unix.digital.com/linux/compaq_c/index.html Cool! This is good to know. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From Fredrik Lundh" <007f01bf786b$8c105e60$f29b12c2@secret.pythonware.com><14506.47402.56596.290048@weyr.cnri.reston.va.us><007b01bf78ab$96003540$34aab5d4@hagrid> <14506.62309.377480.224978@weyr.cnri.reston.va.us> Message-ID: <00ff01bf78bc$ae913580$34aab5d4@hagrid> http://msdn.microsoft.com/vstudio/nextgen/language.asp New object oriented programming features: Inheritance Encapsulation=20 Overloading=20 Polymorphism=20 Parameterized Constructors=20 Additional modernized language features: Free Threading=20 Structured Exception Handling Type Safety=20 Shared Members=20 Initializers From gstein@lyra.org Wed Feb 16 23:41:57 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 16 Feb 2000 15:41:57 -0800 (PST) Subject: [Python-Dev] tp_flags (was: __contains__ hook, done right) In-Reply-To: <200002162006.PAA28892@eric.cnri.reston.va.us> Message-ID: On Wed, 16 Feb 2000, Guido van Rossum wrote: >... > > > > /* PyBufferProcs contains bf_getcharbuffer */ > > #define Py_TPFLAGS_HAVE_GETCHARBUFFER (1L<<0) > > + #define Py_TPFLAGS_HAVE_SEQUENCE_IN (1L<<1) If this flag is going to be defined, then it needs a comment about it. The above code seems to imply that HAVE_SEQUENCE_IN is related to the PyBufferProces. > > > > #define Py_TPFLAGS_DEFAULT (Py_TPFLAGS_HAVE_GETCHARBUFFER) > > I would modify this to include Py_TPFLAGS_HAVE_SEQUENCE_IN by default. > The flag means that the code knows that the sq_contains field exists; > not that this particular object has a non-NULL value in it. So it can > always be on in code compiled with this version of the header file. Guido: adding new flags is *only* necessary when you want to avoid changes in the PYTHON_API_VERSION. If the API has already changed between 1.5 and 1.6, then PYTHON_API_VERSION should be bumped, and this new tp_flags value is not necessary. In fact, when you bump the VERSION, it can even be argued that these specific flags get obsoleted (since an extension must be compiled with the new VERSION to be properly loaded, which makes it pick up the new slot). So... I'd pose this question to you, Guido: will the API version be bumped for Python 1.6? If so, then we have some potential cleanup that can occur. (note: tp_flags is not *reserved* for slot extensions; it is simply that we haven't discovered any other flags to put in there yet) > > *************** > > *** 1405,1410 **** > > --- 1432,1439 ---- > > 0, /*tp_str*/ > > (getattrofunc)instance_getattr, /*tp_getattro*/ > > (setattrofunc)instance_setattr, /*tp_setattro*/ > > + 0, /* tp_as_buffer */ > > + Py_TPFLAGS_HAVE_SEQUENCE_IN, /* tp_flags */ > > This could be Py_TPFLAGS_DEFAULT. I'd rephrase as *should*. Remember: the flag bits (as used today) are to determine whether a slot exists -- in lieu of changing the PYTHON_API_VERSION. Once you compile under the new definition of PyBufferProcs or PySequenceMethods, then the slots will definitely exist; therefore, Py_TPFLAGS_DEFAULT should be used. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Thu Feb 17 00:03:31 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 16 Feb 2000 16:03:31 -0800 (PST) Subject: [Python-Dev] new httplib? Message-ID: Hi all, I've been using my HTTP/1.1 httplib for quite a while now. It seems quite stable in everything that I've tried. I need to port over the recent changes for SSL stuff, but it is ready to go. The library contains a deprecated, backwards-compat class (HTTP), and its replacement: HTTPConnection. Responses are now handled through HTTPResponse class. Doc for the new class will also be needed, but (of course) the old doc still applies. Any comments on the module and/or its inclusion into 1.6? Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Thu Feb 17 00:06:46 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 16 Feb 2000 16:06:46 -0800 (PST) Subject: [Python-Dev] davlib.py Message-ID: One more module for consideration... My DAV client module has been around for a while, has been used in several contexts, and seems to do its job quite fine. However, I'm a bit concerned that it will see continued development over the next year as more WebDAV specs become finalized and turned into RFCs. So the question is: do we add the module now (it is useful), or do we wait for a while? Cheers, -g -- Greg Stein, http://www.lyra.org/ From guido@python.org Thu Feb 17 00:55:54 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Feb 2000 19:55:54 -0500 Subject: [Python-Dev] Re: [Patches] tp_flags (was: __contains__ hook, done right) In-Reply-To: Your message of "Wed, 16 Feb 2000 15:41:57 PST." References: Message-ID: <200002170055.TAA29872@eric.cnri.reston.va.us> > > > /* PyBufferProcs contains bf_getcharbuffer */ > > > #define Py_TPFLAGS_HAVE_GETCHARBUFFER (1L<<0) > > > + #define Py_TPFLAGS_HAVE_SEQUENCE_IN (1L<<1) > > If this flag is going to be defined, then it needs a comment about it. The > above code seems to imply that HAVE_SEQUENCE_IN is related to the > PyBufferProces. Good catch! > > > #define Py_TPFLAGS_DEFAULT (Py_TPFLAGS_HAVE_GETCHARBUFFER) > > > > I would modify this to include Py_TPFLAGS_HAVE_SEQUENCE_IN by default. > > The flag means that the code knows that the sq_contains field exists; > > not that this particular object has a non-NULL value in it. So it can > > always be on in code compiled with this version of the header file. > > Guido: adding new flags is *only* necessary when you want to avoid changes > in the PYTHON_API_VERSION. If the API has already changed between 1.5 and > 1.6, then PYTHON_API_VERSION should be bumped, and this new tp_flags value > is not necessary. > > In fact, when you bump the VERSION, it can even be argued that these > specific flags get obsoleted (since an extension must be compiled with the > new VERSION to be properly loaded, which makes it pick up the new slot). > > So... I'd pose this question to you, Guido: will the API version be bumped > for Python 1.6? If so, then we have some potential cleanup that can occur. I'd like to keep the API version unchanged from 1.5.2, if at all possible. That will depend on how we end up doing new coercions and rich comparisons. > (note: tp_flags is not *reserved* for slot extensions; it is simply that > we haven't discovered any other flags to put in there yet) Indeed. > > > + 0, /* tp_as_buffer */ > > > + Py_TPFLAGS_HAVE_SEQUENCE_IN, /* tp_flags */ > > > > This could be Py_TPFLAGS_DEFAULT. > > I'd rephrase as *should*. > > Remember: the flag bits (as used today) are to determine whether a slot > exists -- in lieu of changing the PYTHON_API_VERSION. Once you compile > under the new definition of PyBufferProcs or PySequenceMethods, then the > slots will definitely exist; therefore, Py_TPFLAGS_DEFAULT should be used. Yeah, sure. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Feb 17 00:59:32 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Feb 2000 19:59:32 -0500 Subject: [Python-Dev] new httplib? In-Reply-To: Your message of "Wed, 16 Feb 2000 16:03:31 PST." References: Message-ID: <200002170059.TAA29886@eric.cnri.reston.va.us> > I've been using my HTTP/1.1 httplib for quite a while now. It seems quite > stable in everything that I've tried. I need to port over the recent > changes for SSL stuff, but it is ready to go. > > The library contains a deprecated, backwards-compat class (HTTP), and its > replacement: HTTPConnection. Responses are now handled through > HTTPResponse class. > > Doc for the new class will also be needed, but (of course) the old doc > still applies. > > Any comments on the module and/or its inclusion into 1.6? Doesn't Jeremy have one too? I would suggest putting the new classes in a new module, e.g. httplib2.py, and leaving the old httplib unchanged. BTW, where can we look at this code? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Feb 17 01:00:53 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Feb 2000 20:00:53 -0500 Subject: [Python-Dev] davlib.py In-Reply-To: Your message of "Wed, 16 Feb 2000 16:06:46 PST." References: Message-ID: <200002170100.UAA29898@eric.cnri.reston.va.us> > One more module for consideration... My DAV client module has been around > for a while, has been used in several contexts, and seems to do its job > quite fine. However, I'm a bit concerned that it will see continued > development over the next year as more WebDAV specs become finalized and > turned into RFCs. > > So the question is: do we add the module now (it is useful), or do we wait > for a while? Given the expected further development, I'd say don't include it yet. As long as it's listed in the vaults of parnassus, I don't see why it needs to be in the standard library -- people will be able to find it without much effort. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Feb 17 01:27:47 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Feb 2000 20:27:47 -0500 Subject: [Python-Dev] frozen exceptions.py (was: win32 specific exception in the core?) In-Reply-To: Your message of "Tue, 08 Feb 2000 15:58:44 +0100." <003901bf7244$ff9daf90$f29b12c2@secret.pythonware.com> References: <003901bf7244$ff9daf90$f29b12c2@secret.pythonware.com> Message-ID: <200002170127.UAA00046@eric.cnri.reston.va.us> > > Why C? Implement it in Python and freeze the sucker. > > size and performance. exception classes are installed > during initialization of the python interpreter, and they > all need to be made available for C code anyway. > > (in fact, the new code isn't that much larger than the > code needed to copy stuff from exceptions.py) Actually, in this case, I'd vote for C too. My reason is stability. The process to freeze exceptions.py into the core will always be more fragile than the process to compile C code. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Feb 17 01:32:57 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Feb 2000 20:32:57 -0500 Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: Your message of "Sat, 12 Feb 2000 12:44:53 +1100." References: Message-ID: <200002170132.UAA00067@eric.cnri.reston.va.us> > Im looking for closure on the Win32 specific exception. Summary for those > that missed the thread: My apologies. I *did* miss the thread. I think this is a reasonable request. > * I would like a standard exception for Win32 specific errors. The new > hopefully-to-be-standard "win32reg" module could make use of it, the win32 > extensions could switch to it. It is, in my mind at least, analogous to > OSError. > > * I proposed deriving the new exception from EnvironmentError, like OSError > is. > > * Fredrik replied with "why not use OSError then?". I answered "due to the > numbering systems being different" > > * Tim agreed that they should be different exceptions if they use different > numbering, but pointed out it could still be OSError with creative exception > hierarchies. > > So, in summary, no one disagreed, and the only open issue was the exception > hierarchy. My initial proposal called for: > > StandardError: > -> EnvironmentError: > -> IOError > -> OSError > -> Win32Error > > Which would generally lead to people writing "except Win32Error:" > > Tim's suggestion (as I understand it) was: > StandardError: > -> EnvironmentError: > -> IOError > -> OSError > -> CRTError (new - is what os.error would use) > -> Win32Error (new) I see no merit in adding CRTError -- that's an abstraction that's only relevant to C developers on Windows. I do like it better if Win32Error derives from OSError. Shouldn't it be just WinError or WindowsError? I don't see why a different exception should be used for Win64. > which would lead to people writing "except IOError:" I don't understand this -- Win32Error isn't derived from IOError. > I still like my idea better - fits better with the existing "IOError", and > doesnt cause confusion with the change from os.error being OSError to > os.error being CRTError. > > Guido: Can you first make a "yes/no" decision, then (if necessary ) > an either/or one? OK, this is a definite Yes. The either/or I'm still open on. --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Thu Feb 17 01:33:58 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 16 Feb 2000 20:33:58 -0500 (EST) Subject: [Python-Dev] davlib.py In-Reply-To: References: Message-ID: <14507.20486.696059.633607@weyr.cnri.reston.va.us> Greg Stein writes: > One more module for consideration... My DAV client module has been around > for a while, has been used in several contexts, and seems to do its job > quite fine. However, I'm a bit concerned that it will see continued > development over the next year as more WebDAV specs become finalized and > turned into RFCs. In spite of Guido's misgivings, I think that if the core RFCs have proven effective and useful and the implementation conforms and is stable with respect to those, it's a good idea to include it, especially as DAV servers really start to show up. My question is with regard to the continued development you mention: Would this change the existing API, or augment & extend it? Or is it too early to say? If it would change the API, it should wait. So, are you working on a "Using WebDAV" book yet, with all the interesting client & server availability & configuration info we'd actually need to be able to use it? ;) *That's* what WebDAV needs -- accessibility! -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From mhammond@skippinet.com.au Thu Feb 17 01:50:58 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Thu, 17 Feb 2000 12:50:58 +1100 Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: <200002170132.UAA00067@eric.cnri.reston.va.us> Message-ID: [Guido] > > I do like it better if Win32Error derives from OSError. > > Shouldn't it be just WinError or WindowsError? I don't see why a > different exception should be used for Win64. > > > which would lead to people writing "except IOOError:" > > I don't understand this -- Win32Error isn't derived from IOError. Oops - I meant OSError. What I meant is that people will say: try: win32reg.Something() except IOError, ... Rather than the specific error. This makes our specific Windows exception, basically, an internal mechanism to get the error messages lined up with the error number. Not that this is a problem, but I just thought it worth mentioning. OK - so lets go with: WindowsError(OSError): ... If no one complains, I will submit the relevant patches to Exception.py along with my win32reg.c (I can leave string-based exceptions out, yes?) Mark. From tim_one@email.msn.com Thu Feb 17 02:50:02 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 16 Feb 2000 21:50:02 -0500 Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: Message-ID: <001001bf78f1$aff84c80$dea0143f@tim> [Mark Hammond] > ... > OK - so lets go with: > > WindowsError(OSError): > ... Yes! Note that you're the joker who brought up CRTError: I only mentioned the *possibility* of refactoring to OSError CRTError WindowsError to shut you up . > If no one complains, I will submit the relevant patches to > Exception.py along with my win32reg.c (I can leave string-based > exceptions out, yes?) Is the -X switch going away? If so, sure. If not, you probably can't. I'd be delighted if -X went away (the purpose it served has expired). From guido@python.org Thu Feb 17 03:15:25 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Feb 2000 22:15:25 -0500 Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: Your message of "Wed, 16 Feb 2000 21:50:02 EST." <001001bf78f1$aff84c80$dea0143f@tim> References: <001001bf78f1$aff84c80$dea0143f@tim> Message-ID: <200002170315.WAA00191@eric.cnri.reston.va.us> Yes, -X is going away. --Guido van Rossum (home page: http://www.python.org/~guido/) From gstein@lyra.org Thu Feb 17 08:05:47 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Feb 2000 00:05:47 -0800 (PST) Subject: [Python-Dev] new httplib? In-Reply-To: <200002170059.TAA29886@eric.cnri.reston.va.us> Message-ID: On Wed, 16 Feb 2000, Guido van Rossum wrote: > > I've been using my HTTP/1.1 httplib for quite a while now. It seems quite > > stable in everything that I've tried. I need to port over the recent > > changes for SSL stuff, but it is ready to go. > > > > The library contains a deprecated, backwards-compat class (HTTP), and its > > replacement: HTTPConnection. Responses are now handled through > > HTTPResponse class. > > > > Doc for the new class will also be needed, but (of course) the old doc > > still applies. > > > > Any comments on the module and/or its inclusion into 1.6? > > Doesn't Jeremy have one too? Yes, we talked about it at IPC7. As I recall, Jeremy wasn't happy with the result... it was request/response based with a lot of helper classes. My httplib is connection-oriented (like the current httplib), and I think Jeremy had said that was good (empirically, after his prototype). But that's just let my memory... Jeremy? > I would suggest putting the new classes in a new module, > e.g. httplib2.py, and leaving the old httplib unchanged. Why? The compatibility class exports the same attributes, the same methods, the same constructor, etc. Creating a new module is needless duplication and confusing for users ("which one do I use?" "how is that one different?" "do I have to switch modules to use HTTP/1.1?") A single module that reuses code is going to be much more maintainable than two, possibly diverging, modules. I also feel having one module will help to gently prod people towards using HTTP/1.1 and the new HTTPConnection class (which I think is *subjectively* a nice goal). > BTW, where can we look at this code? Oops. Sorry about that... As usual, it is with the rest of my Python code at http://www.lyra.org/greg/python/ (the davlib is there, too) Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Thu Feb 17 08:07:16 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Feb 2000 00:07:16 -0800 (PST) Subject: [Python-Dev] frozen exceptions.py In-Reply-To: <200002170127.UAA00046@eric.cnri.reston.va.us> Message-ID: On Wed, 16 Feb 2000, Guido van Rossum wrote: > > > Why C? Implement it in Python and freeze the sucker. > > > > size and performance. exception classes are installed > > during initialization of the python interpreter, and they > > all need to be made available for C code anyway. > > > > (in fact, the new code isn't that much larger than the > > code needed to copy stuff from exceptions.py) > > Actually, in this case, I'd vote for C too. My reason is stability. > The process to freeze exceptions.py into the core will always be more > fragile than the process to compile C code. Heretic! :-) All right... I guess that I'm just gonna have to see if I can disprove your assertion :-) Cheers, -g -- Greg Stein, http://www.lyra.org/ From Fredrik Lundh" <003901bf7244$ff9daf90$f29b12c2@secret.pythonware.com> <200002170127.UAA00046@eric.cnri.reston.va.us> Message-ID: <00d601bf7923$084f6360$34aab5d4@hagrid> Guido van Rossum wrote: > > > Why C? Implement it in Python and freeze the sucker. > >=20 > > size and performance. exception classes are installed > > during initialization of the python interpreter, and they > > all need to be made available for C code anyway. > >=20 > > (in fact, the new code isn't that much larger than the > > code needed to copy stuff from exceptions.py) >=20 > Actually, in this case, I'd vote for C too. My reason is stability. > The process to freeze exceptions.py into the core will always be more > fragile than the process to compile C code. incomplete proof-of-concept implementation available here. see webpage and FIXME's in the code for more info: http://w1.132.telia.com/~u13208596/exceptions.htm From gstein@lyra.org Thu Feb 17 08:18:57 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Feb 2000 00:18:57 -0800 (PST) Subject: [Python-Dev] davlib.py In-Reply-To: <14507.20486.696059.633607@weyr.cnri.reston.va.us> Message-ID: On Wed, 16 Feb 2000, Fred L. Drake, Jr. wrote: > Greg Stein writes: > > One more module for consideration... My DAV client module has been around > > for a while, has been used in several contexts, and seems to do its job > > quite fine. However, I'm a bit concerned that it will see continued > > development over the next year as more WebDAV specs become finalized and > > turned into RFCs. > > In spite of Guido's misgivings, I think that if the core RFCs have > proven effective and useful and the implementation conforms and is > stable with respect to those, it's a good idea to include it, > especially as DAV servers really start to show up. Yes, to all points. > My question is with regard to the continued development you mention: > Would this change the existing API, or augment & extend it? Or is it > too early to say? Well, things that I'm considering are something like this: class DAV(...): def get(self, url): ... The above works very well for doing a GET on a resource. But what if the user wants to set an If: header? Or a If-Modified-Since: header? Or set some authentication values on the connection? Stuff like that. I'm thinking that changes would augment/extend the existing functionality. What is in there right now is great for the simple cases, so I wouldn't want to lose that. The provided functionality just doesn't scale up as well to the harder cases as well. It's possible, but it would look something like: hdrs = { 'If' : '(["entity-tag"])' } response = connection._request('GET', '/some/url', extra_hdrs=hdrs) > If it would change the API, it should wait. It is possible that it could change in a backwards-incompatible fashion if we started to load in support for the harder cases. It kind of depends on whether that extra support would make the simple stuff harder (I hope not). > So, are you working on a "Using WebDAV" book yet, with all the > interesting client & server availability & configuration info we'd > actually need to be able to use it? ;) *That's* what WebDAV needs -- > accessibility! I've tried my hand at a book. And at writing doc :-). I'm just not very well cut out for it. While there isn't a book, the website has a lot of material and pointers and stuff: http://www.webdav.org/ ------------------------ Anyhow... I might suggest adding a simple davlib to the distro. We'd certainly be well ahead of the other languages :-). It would be nice to cover the 90% case for people interacting with a DAV server. And maybe it would help in displacing some of that yucky FTP stuff ;-) Cheers, -g -- Greg Stein, http://www.lyra.org/ From Fredrik Lundh" <200002170100.UAA29898@eric.cnri.reston.va.us> Message-ID: <013201bf7927$8b9beb40$34aab5d4@hagrid> Guido van Rossum wrote: > > So the question is: do we add the module now (it is useful), or do = we wait > > for a while? >=20 > Given the expected further development, I'd say don't include it yet. > As long as it's listed in the vaults of parnassus, I don't see why it > needs to be in the standard library -- people will be able to find it > without much effort. hmm. I cannot find it in the vaults. does that mean that we have to add it to the standard distribution? :-) (personally, I'd like to have xmlrpclib.py and soaplib.py in the core, but that can wait for 1.6.1) From Vladimir.Marangozov@inrialpes.fr Thu Feb 17 09:22:58 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Thu, 17 Feb 2000 10:22:58 +0100 (CET) Subject: [Python-Dev] davlib.py In-Reply-To: from "Greg Stein" at Feb 17, 2000 12:18:57 AM Message-ID: <200002170922.KAA17759@python.inrialpes.fr> Greg Stein wrote: > > > Anyhow... I might suggest adding a simple davlib to the distro. We'd > certainly be well ahead of the other languages :-). It would be nice to > cover the 90% case for people interacting with a DAV server. I'm inclined to favor some Python pioneering here, despite that all this stuff is still young. Besides, Greg (at least) knows what he's doing and follows the domain closely. So if he writes the doc ;-) and a test_webdav, I'd say: let it be. People will jump in subsequently. Remember: Batteries included. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From gward@cnri.reston.va.us Thu Feb 17 14:48:43 2000 From: gward@cnri.reston.va.us (Greg Ward) Date: Thu, 17 Feb 2000 09:48:43 -0500 Subject: [Python-Dev] * and ? in fnmatch Message-ID: <20000217094843.A10767@cnri.reston.va.us> Hi all -- I have recently been playing with the fnmatch module, and learned that * and ? as considered by 'fnmatch.translate()' match *all* characters, including slashes, colons, backslashes -- in short, whatever happens to be "special" characters for pathnames on the current platform. In other words, "foo?bar.py" matches both "foo_bar.py" and "foo/bar.py". This is not the way any Unix shells that I know of work, nor is it how the wildcard-expanding MS-DOS system calls that I dimly remember from a decade or so back worked. I dunno how wildcard expansion is done under Windows nowadays, but I wouldn't expect * and ? to match colons or backslashes there any more than I expect them to match slash under a Unix shell. So is this a bug or a feature? Seems to me that a good fix would be to extend 'fnmatch.translate()' to have some (maybe all?) of the flags that the standard Unix library 'fnmatch()' supports. The flag in question here is FNM_PATHNAME, which is described in the Solaris manual as FNM_PATHNAME If set, a slash (/) character in string will be explicitly matched by a slash in pattern; it will not be matched by either the asterisk (*) or question-mark (?) special characters, nor by a bracket ([]) expression. If not set, the slash charac- ter is treated as an ordinary character. and in the GNU/Linux manual as FNM_PATHNAME If this flag is set, match a slash in string only with a slash in pattern and not, for example, with a [] - sequence containing a slash. To adapt this to Python's 'fnmatch.translate()', I think "slash" would have to be generalized to "special character", which is platform dependent: Unix / DOS/Windows : \ (and maybe / too?) Mac : I propose changing the signature of 'fnmatch.translate()' from def translate(pat) to at least def translate(pat,pathname=0) and possibly to def translate(pat, pathname=0, noescape=0, period=0, leading_dir=0, casefold=0) which follows the lead of GNU 'fnmatch()'. (Solaris 'fnmatch()' only supports the PATHNAME, NOESCAPE, and PERIOD flags; the GNU man page says LEADING_DIR and CASEFOLD are GNU extensions. I like GNU extensions.) Similar optional parameters would be added to 'fnmatch()' and 'fnmatchcase()', possibly dropping the 'casefold' argument since it's covered by which function you're calling. I have yet to fully grok the meaning of those other four flags, though, so I'm not sure how easy it would be to hack them into 'fnmatch.translate()'. Opinions? Greg From Fredrik Lundh" Message-ID: <003a01bf7958$86e068c0$34aab5d4@hagrid> > I have recently been playing with the fnmatch module, and learned that = * > and ? as considered by 'fnmatch.translate()' match *all* characters, > including slashes, colons, backslashes -- in short, whatever happens = to > be "special" characters for pathnames on the current platform. >=20 > In other words, "foo?bar.py" matches both "foo_bar.py" and = "foo/bar.py". this is documented behaviour: Note that the filename separator ('/' on Unix) is not special to this module. See module glob for pathname expansion (glob uses fnmatch() to match filename=20 segments).=20 > def translate(pat, > pathname=3D0,=20 > noescape=3D0, > period=3D0, > leading_dir=3D0, > casefold=3D0) sure looks like feature creep to me. is anyone actually using this module directly? From guido@python.org Thu Feb 17 15:01:08 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 17 Feb 2000 10:01:08 -0500 Subject: [Python-Dev] * and ? in fnmatch In-Reply-To: Your message of "Thu, 17 Feb 2000 09:48:43 EST." <20000217094843.A10767@cnri.reston.va.us> References: <20000217094843.A10767@cnri.reston.va.us> Message-ID: <200002171501.KAA00533@eric.cnri.reston.va.us> > I have recently been playing with the fnmatch module, and learned that * > and ? as considered by 'fnmatch.translate()' match *all* characters, > including slashes, colons, backslashes -- in short, whatever happens to > be "special" characters for pathnames on the current platform. > > In other words, "foo?bar.py" matches both "foo_bar.py" and "foo/bar.py". > > This is not the way any Unix shells that I know of work, nor is it how > the wildcard-expanding MS-DOS system calls that I dimly remember from a > decade or so back worked. I dunno how wildcard expansion is done under > Windows nowadays, but I wouldn't expect * and ? to match colons or > backslashes there any more than I expect them to match slash under a > Unix shell. > > So is this a bug or a feature? It's a feature. As I recall, I carefully implemented the standard fnmatch() as it existed 10 years ago. Use the glob module for matching Unix pathname syntax -- or use os.path.split(). I'm not wildly fond of the GNU way of adding 10 more options to each function... KISS. (And yes, I'm in a bad mood today. :-( ) --Guido van Rossum (home page: http://www.python.org/~guido/) From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 17 15:47:18 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 17 Feb 2000 10:47:18 -0500 (EST) Subject: [Python-Dev] frozen exceptions.py (was: win32 specific exception in the core?) References: <003901bf7244$ff9daf90$f29b12c2@secret.pythonware.com> <200002170127.UAA00046@eric.cnri.reston.va.us> <00d601bf7923$084f6360$34aab5d4@hagrid> Message-ID: <14508.6150.406456.69720@anthem.cnri.reston.va.us> I haven't looked at /F's code yet, but I've been thinking about doing something very similar for JPython. I think it'll be easier to freeze it in Java than in C. -Barry From fdrake@acm.org Thu Feb 17 17:08:06 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 17 Feb 2000 12:08:06 -0500 (EST) Subject: [Python-Dev] davlib.py In-Reply-To: <013201bf7927$8b9beb40$34aab5d4@hagrid> References: <200002170100.UAA29898@eric.cnri.reston.va.us> <013201bf7927$8b9beb40$34aab5d4@hagrid> Message-ID: <14508.10998.495303.415806@weyr.cnri.reston.va.us> Fredrik Lundh writes: > (personally, I'd like to have xmlrpclib.py and soaplib.py > in the core, but that can wait for 1.6.1) These make a lot of sense, especially once we coerce Guido to accept Paul's expat bindings so that we can have some real XML support. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From guido@python.org Thu Feb 17 17:29:22 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 17 Feb 2000 12:29:22 -0500 Subject: [Python-Dev] davlib.py In-Reply-To: Your message of "Thu, 17 Feb 2000 12:08:06 EST." <14508.10998.495303.415806@weyr.cnri.reston.va.us> References: <200002170100.UAA29898@eric.cnri.reston.va.us> <013201bf7927$8b9beb40$34aab5d4@hagrid> <14508.10998.495303.415806@weyr.cnri.reston.va.us> Message-ID: <200002171729.MAA01355@eric.cnri.reston.va.us> > Fredrik Lundh writes: > > (personally, I'd like to have xmlrpclib.py and soaplib.py > > in the core, but that can wait for 1.6.1) > > These make a lot of sense, especially once we coerce Guido to accept > Paul's expat bindings so that we can have some real XML support. I keep hearing noises about this. I'm not against improving XML support -- does the XML SIG have a decent proposal? Once we agree on what we are going to do, you can just go and check it in... --Guido van Rossum (home page: http://www.python.org/~guido/) From paul@prescod.net Thu Feb 17 18:33:58 2000 From: paul@prescod.net (Paul Prescod) Date: Thu, 17 Feb 2000 10:33:58 -0800 Subject: [Python-Dev] davlib.py References: <200002170100.UAA29898@eric.cnri.reston.va.us> <013201bf7927$8b9beb40$34aab5d4@hagrid> <14508.10998.495303.415806@weyr.cnri.reston.va.us> <200002171729.MAA01355@eric.cnri.reston.va.us> Message-ID: <38AC3F16.EDD77198@prescod.net> Guido van Rossum wrote: > > ... > > I keep hearing noises about this. I'm not against improving XML > support -- does the XML SIG have a decent proposal? Once we agree on > what we are going to do, you can just go and check it in... I think that you missed a thread on that. Please check it out and comment: http://www.python.org/pipermail/python-dev/2000-February/003627.html -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made possible the modern world." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml From jeremy@cnri.reston.va.us Thu Feb 17 18:57:56 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Thu, 17 Feb 2000 13:57:56 -0500 (EST) Subject: [Python-Dev] new httplib? In-Reply-To: References: <200002170059.TAA29886@eric.cnri.reston.va.us> Message-ID: <14508.17588.185414.152197@goon.cnri.reston.va.us> >>>>> "GS" == Greg Stein writes: >> Doesn't Jeremy have one too? GS> Yes, we talked about it at IPC7. As I recall, Jeremy wasn't GS> happy with the result... it was request/response based with a GS> lot of helper classes. My httplib is connection-oriented (like GS> the current httplib), and I think Jeremy had said that was good GS> (empirically, after his prototype). GS> But that's just let my memory... Jeremy? Sounds right to me. I haven't worked on http code in years. I seem to recall liking the basic architecture, but not the implementation. That said, I have no objections to including your modifications: It preserves the current interface and improves its support for http/1.1. I still think the httplib interface could be improved, but I don't have any time to work on it. The one thing I don't see how to do with the new code is pipeline requests. Is that possible? It's not a big deal if pipelining isn't support, since the current implementation doesn't support it at all. Jeremy From Moshe Zadka Thu Feb 17 18:59:36 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 17 Feb 2000 20:59:36 +0200 (IST) Subject: [Python-Dev] davlib.py In-Reply-To: <14508.10998.495303.415806@weyr.cnri.reston.va.us> Message-ID: On Thu, 17 Feb 2000, Fred L. Drake, Jr. wrote: > Fredrik Lundh writes: > > (personally, I'd like to have xmlrpclib.py and soaplib.py > > in the core, but that can wait for 1.6.1) > > These make a lot of sense, especially once we coerce Guido to accept > Paul's expat bindings so that we can have some real XML support. While I do enjoy having all this stuff with the core distribution, I have one nit to pick. (Relevant more to 1.7 then 1.6, but worth keeping in mind). Can we please, pretty please with cream on top, start using packages in the standard distribution? We can do it backwards compatibly, with something like internet/ protocol/ httplib.py and httplib.py which does from internet.protocol.httplib import *? Put new stuff only in packages, and that way we could have a large distribution, without a lot of top-level name clutter. Anyone for? Against? Going to add me to their kill-files? -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From fdrake@acm.org Thu Feb 17 19:11:48 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 17 Feb 2000 14:11:48 -0500 (EST) Subject: [Python-Dev] davlib.py In-Reply-To: References: <14508.10998.495303.415806@weyr.cnri.reston.va.us> Message-ID: <14508.18420.637014.518700@weyr.cnri.reston.va.us> Moshe Zadka writes: > While I do enjoy having all this stuff with the core distribution, I have > one nit to pick. (Relevant more to 1.7 then 1.6, but worth keeping in > mind). Can we please, pretty please with cream on top, start using I think this is really a matter for Py3K; the 1.x series has a lot of backward-compatibility issues. There's no need to move things to packages if the old names are still supported; that just increases the clutter (all the old names *plus* the new packages). Until the library is reorganized into packages, there's no need to change the structural approach, since the re-organization will break lots of code anyway. > packages in the standard distribution? We can do it backwards compatibly, > with something like > > internet/ > protocol/ > httplib.py I wouldn't make it so deep; perhaps internet/httplib, but that's it. > Put new stuff only in packages, and that way we could have a large > distribution, without a lot of top-level name clutter. The XML stuff will be in the xml package; this matches what the XML-SIG has been doing all along. And please don't confuse distribution clutter with module namespace clutter; the first is merely annoying, while the latter is actually the important issue. > Anyone for? Against? Going to add me to their kill-files? Remember, the PSU reads names from my kill-file; be careful where you suggest we put yours. ;) -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From skip@mojam.com (Skip Montanaro) Thu Feb 17 19:23:16 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 17 Feb 2000 13:23:16 -0600 (CST) Subject: [Python-Dev] davlib.py In-Reply-To: References: <14508.10998.495303.415806@weyr.cnri.reston.va.us> Message-ID: <14508.19108.642097.101173@beluga.mojam.com> Moshe> Can we please, pretty please with cream on top, start using Moshe> packages in the standard distribution? We can do it backwards Moshe> compatibly, with something like Moshe> internet/ Moshe> protocol/ Moshe> httplib.py Moshe> and httplib.py which does Moshe> from internet.protocol.httplib import *? Moshe> Put new stuff only in packages, and that way we could have a large Moshe> distribution, without a lot of top-level name clutter. Moshe> Anyone for? Against? Going to add me to their kill-files? For. Besides the reasons Moshe indicated, it also serves as an educational tool. New Python programmers will get more comfortable with writing packages faster if they are already comfortable using them. Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ "Languages that change by catering to the tastes of non-users tend not to do so well." - Doug Landauer From skip@mojam.com (Skip Montanaro) Thu Feb 17 19:29:56 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 17 Feb 2000 13:29:56 -0600 (CST) Subject: [Python-Dev] davlib.py In-Reply-To: <14508.18420.637014.518700@weyr.cnri.reston.va.us> References: <14508.10998.495303.415806@weyr.cnri.reston.va.us> <14508.18420.637014.518700@weyr.cnri.reston.va.us> Message-ID: <14508.19508.735191.750118@beluga.mojam.com> Fred> I think this is really a matter for Py3K; the 1.x series has a lot Fred> of backward-compatibility issues. There's no need to move things Fred> to packages if the old names are still supported; that just Fred> increases the clutter (all the old names *plus* the new packages). Doesn't worry me a bit... ;-) Lots of other stuff has been deprecated in the relatively recent past: regex, regexp, the <>, the assert statement. Some things (regexp and assert come to mind) have actually even been deleted (oh, the horror of it all!). The relative danger of actual code breakage only lengthens the time between deprecation and deletion. If you have an internet.protocols package, you can deprecate the non-packaged versions of whatever you stick in there. Skip From mal@lemburg.com Thu Feb 17 19:32:21 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 17 Feb 2000 20:32:21 +0100 Subject: [Python-Dev] davlib.py References: <14508.10998.495303.415806@weyr.cnri.reston.va.us> <14508.18420.637014.518700@weyr.cnri.reston.va.us> Message-ID: <38AC4CC5.6656C6F4@lemburg.com> "Fred L. Drake, Jr." wrote: > > Moshe Zadka writes: > > While I do enjoy having all this stuff with the core distribution, I have > > one nit to pick. (Relevant more to 1.7 then 1.6, but worth keeping in > > mind). Can we please, pretty please with cream on top, start using > > I think this is really a matter for Py3K; the 1.x series has a lot > of backward-compatibility issues. There's no need to move things to > packages if the old names are still supported; that just increases the > clutter (all the old names *plus* the new packages). > Until the library is reorganized into packages, there's no need to > change the structural approach, since the re-organization will break > lots of code anyway. > > > packages in the standard distribution? We can do it backwards compatibly, > > with something like > > > > internet/ > > protocol/ > > httplib.py > > I wouldn't make it so deep; perhaps internet/httplib, but that's it. Python 1.6 will start using packages in the core... the Unicode patches add an "encodings" package. > > Put new stuff only in packages, and that way we could have a large > > distribution, without a lot of top-level name clutter. > > The XML stuff will be in the xml package; this matches what the > XML-SIG has been doing all along. "mx" is also taken -- its just that nobody knows yet :-) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From gstein@lyra.org Thu Feb 17 19:40:46 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Feb 2000 11:40:46 -0800 (PST) Subject: [Python-Dev] new httplib? In-Reply-To: <14508.17588.185414.152197@goon.cnri.reston.va.us> Message-ID: On Thu, 17 Feb 2000, Jeremy Hylton wrote: >... > The one thing I don't see how to do with the new code is pipeline > requests. Is that possible? It's not a big deal if pipelining isn't > support, since the current implementation doesn't support it at all. Pipelining is very important... it is once of the big advances in HTTP/1.1 (well, the advance is really the "chunked" transfer encoding which *allows* pipelining to occur more often). To do pipelining, just keep feeding requests into the thing: ------------------------- conn = httplib.HTTPConnection('www.python.org') conn.request('GET', '/') code, msg, response = conn.getreply() print response.read() conn.request('GET', '/sigs/') code, msg, response = conn.getreply() print response.read() ------------------------- Note that the code requires you to consume the response before issuing another request (it will raise an error otherwise). The connection object will connect automatically connect to the host. It will keep the connection open per HTTP/1.1 guidelines (or close it when told to). If an "broken pipe" occurs during the sending of the request (the other end closed the socket), then it will close, reconnect, and re-issue the request (just once). The broken pipe typically occurs if you wait too long between requests. The client can force a connect or a close thru the connection object API. In addition to the simple request() method above, there is the standard putrequest, putheader, endheader style. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Thu Feb 17 19:49:03 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Feb 2000 11:49:03 -0800 (PST) Subject: [Python-Dev] packaging (was: davlib.py) In-Reply-To: <14508.18420.637014.518700@weyr.cnri.reston.va.us> Message-ID: [ can people start updating the subject line! this stopped being about davlib a while ago ] On Thu, 17 Feb 2000, Fred L. Drake, Jr. wrote: >... > > packages in the standard distribution? We can do it backwards compatibly, > > with something like > > > > internet/ > > protocol/ > > httplib.py > > I wouldn't make it so deep; perhaps internet/httplib, but that's it. +1 on using a *shallow* package, as Fred states. I would also rename it to "network" and shove other networking stuff in there. The name "protocol" might work, but "urllib" doesn't fit that very well (and I think urllib probably would go in there: network.urllib). Cheers, -g -- Greg Stein, http://www.lyra.org/ From Fredrik Lundh" Message-ID: <004201bf7980$afa27e60$34aab5d4@hagrid> Moshe Zadka wrote: > While I do enjoy having all this stuff with the core distribution, I = have > one nit to pick. (Relevant more to 1.7 then 1.6, but worth keeping in > mind). Can we please, pretty please with cream on top, start using > packages in the standard distribution?=20 I'd say wait for Py3K. cleaning up the standard library is a *huge* project, and should be done right. "packetizing" small portions of it are likely to cause more trouble than it's worth. (moving the entire thing into a "std" package would be fine, though. or should that be "org.python.std"?) From gstein@lyra.org Thu Feb 17 19:51:01 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Feb 2000 11:51:01 -0800 (PST) Subject: [Python-Dev] packaging and backwards-compat (was: davlib.py) In-Reply-To: <14508.19508.735191.750118@beluga.mojam.com> Message-ID: On Thu, 17 Feb 2000, Skip Montanaro wrote: >... > Fred> I think this is really a matter for Py3K; the 1.x series has a lot > Fred> of backward-compatibility issues. There's no need to move things > Fred> to packages if the old names are still supported; that just > Fred> increases the clutter (all the old names *plus* the new packages). > > Doesn't worry me a bit... ;-) Doesn't worry me either. > Lots of other stuff has been deprecated in the relatively recent past: > regex, regexp, the <>, the assert statement. Some things (regexp and assert Nit: the "access" statement. > If you have an internet.protocols package, you can deprecate the > non-packaged versions of whatever you stick in there. Yes: deprecate them, and use the "from foo import *" in their placeholders. Cheers, -g -- Greg Stein, http://www.lyra.org/ From fdrake@acm.org Thu Feb 17 19:50:20 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 17 Feb 2000 14:50:20 -0500 (EST) Subject: [Python-Dev] new httplib? In-Reply-To: References: <14508.17588.185414.152197@goon.cnri.reston.va.us> Message-ID: <14508.20732.604181.414586@weyr.cnri.reston.va.us> Greg Stein writes: > Pipelining is very important... it is once of the big advances in HTTP/1.1 > (well, the advance is really the "chunked" transfer encoding which > *allows* pipelining to occur more often). You should be able to pipeline requests even if responses don't use chunking; if I parse the first HTML page while still receiving, I should be able to request non-cached images or other dependencies before the first HTML is completely received. This allows the server to locate the resources while still waiting for me finish pulling the first response off the network. It improves the possibility of scheduling resource location at the server (possibly important if there's a dynamic backend) as well as avoiding establishing new TCP connections. (Yes, some of this can be done with keep-alive stuff, but I don't know that httplib supports that at all.) -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From fdrake@acm.org Thu Feb 17 19:52:31 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 17 Feb 2000 14:52:31 -0500 (EST) Subject: [Python-Dev] packaging (was: davlib.py) In-Reply-To: References: <14508.18420.637014.518700@weyr.cnri.reston.va.us> Message-ID: <14508.20863.996377.452144@weyr.cnri.reston.va.us> Greg Stein writes: > I would also rename it to "network" and shove other networking stuff in > there. The name "protocol" might work, but "urllib" doesn't fit that very > well (and I think urllib probably would go in there: network.urllib). I'd prefer "network" over "protocol". Protocol can also mean other things. (Ok, ok: network can too. Can we call it "neural"? ;) -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From gstein@lyra.org Thu Feb 17 20:01:33 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Feb 2000 12:01:33 -0800 (PST) Subject: [Python-Dev] new httplib? In-Reply-To: <14508.20732.604181.414586@weyr.cnri.reston.va.us> Message-ID: On Thu, 17 Feb 2000, Fred L. Drake, Jr. wrote: > Greg Stein writes: > > Pipelining is very important... it is once of the big advances in HTTP/1.1 > > (well, the advance is really the "chunked" transfer encoding which > > *allows* pipelining to occur more often). > > You should be able to pipeline requests even if responses don't use > chunking; Not in all cases. You can pipeline only if the server sends a Content-Length: header or uses chunking. There are a few other conditions (look at the httplib.HTTPResponse constructor in my httplib), but the Content-Length/chunking is the major point. > if I parse the first HTML page while still receiving, I > should be able to request non-cached images or other dependencies > before the first HTML is completely received. This allows the server > to locate the resources while still waiting for me finish pulling the > first response off the network. It improves the possibility of > scheduling resource location at the server (possibly important if > there's a dynamic backend) as well as avoiding establishing new TCP > connections. This would be nice, but my httplib currently enforces consumption before a new request. I do not recall the specific reasons why, but I imagine it would be possible to change it. [ today: you could certainly use a second connection object to improve client response, with the corresponding impact on the server ] > (Yes, some of this can be done with keep-alive stuff, but I don't > know that httplib supports that at all.) My httplib deals with the keep-alive header, when applicable (you are supposed to use Connection: in HTTP/1.1; it only looks at keep-alive if you aren't using 1.1). Cheers, -g -- Greg Stein, http://www.lyra.org/ From jeremy@cnri.reston.va.us Thu Feb 17 20:04:59 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Thu, 17 Feb 2000 15:04:59 -0500 (EST) Subject: [Python-Dev] new httplib? In-Reply-To: References: <14508.17588.185414.152197@goon.cnri.reston.va.us> Message-ID: <14508.21611.538025.587013@goon.cnri.reston.va.us> This sounds good enough for me! I endorse it for inclusion as long as their is accompanying documentation <0.25 wink>. Jeremy From gstein@lyra.org Thu Feb 17 20:07:16 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Feb 2000 12:07:16 -0800 (PST) Subject: [Python-Dev] davlib.py In-Reply-To: <200002170922.KAA17759@python.inrialpes.fr> Message-ID: On Thu, 17 Feb 2000, Vladimir Marangozov wrote: >... > I'm inclined to favor some Python pioneering here, despite that all this > stuff is still young. Besides, Greg (at least) knows what he's doing and > follows the domain closely. So if he writes the doc ;-) and a test_webdav, > I'd say: let it be. People will jump in subsequently. Consider doc and testing a given, if this stuff goes in. Cheers, -g -- Greg Stein, http://www.lyra.org/ From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 17 20:08:28 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 17 Feb 2000 15:08:28 -0500 (EST) Subject: [Python-Dev] davlib.py References: <14508.10998.495303.415806@weyr.cnri.reston.va.us> Message-ID: <14508.21820.227329.666405@anthem.cnri.reston.va.us> >>>>> "MZ" == Moshe Zadka writes: MZ> Anyone for? Against? Going to add me to their MZ> kill-files? Very much for, if we can come up with some reasonable package hierarchy. -Barry From gstein@lyra.org Thu Feb 17 20:12:23 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Feb 2000 12:12:23 -0800 (PST) Subject: [Python-Dev] concrete proposal (was: davlib.py) In-Reply-To: <14508.10998.495303.415806@weyr.cnri.reston.va.us> Message-ID: On Thu, 17 Feb 2000, Fred L. Drake, Jr. wrote: > Fredrik Lundh writes: > > (personally, I'd like to have xmlrpclib.py and soaplib.py > > in the core, but that can wait for 1.6.1) > > These make a lot of sense, especially once we coerce Guido to accept > Paul's expat bindings so that we can have some real XML support. Agreed. Concrete proposal: * add Expat and PyExpat (I'm +0 on other XML items) * add "network" package * add network.davlib, network.xmlrpclib, network.soaplib * upgrade httplib * move: ftplib, gopherlib, httplib, imaplib, nntplib, poplib, smtplib, telnetlib, urllib * +0 on moving: BaseHTTPServer, CGIHTTPServer, SimpleHTTPServer, SocketServer, asynchat, asyncore Cheers, -g -- Greg Stein, http://www.lyra.org/ From jeremy@cnri.reston.va.us Thu Feb 17 20:20:18 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Thu, 17 Feb 2000 15:20:18 -0500 (EST) Subject: [Python-Dev] concrete proposal (was: davlib.py) In-Reply-To: References: <14508.10998.495303.415806@weyr.cnri.reston.va.us> Message-ID: <14508.22530.932416.357269@goon.cnri.reston.va.us> I'm all for re-organizing the library using packages. I don't think I'm in favoring of doing some of it now, and some of it later. Rather than moving modules into the network package (good name), I'd hold off and do full packaging for 1.7. Jeremy From gstein@lyra.org Thu Feb 17 20:26:39 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Feb 2000 12:26:39 -0800 (PST) Subject: [Python-Dev] package reorg in 1.6 (was: concrete proposal) In-Reply-To: <14508.22530.932416.357269@goon.cnri.reston.va.us> Message-ID: On Thu, 17 Feb 2000, Jeremy Hylton wrote: > I'm all for re-organizing the library using packages. I don't think > I'm in favoring of doing some of it now, and some of it later. Rather > than moving modules into the network package (good name), I'd > hold off and do full packaging for 1.7. Python 1.6 is on a short release schedule, mostly just waiting for Unicode and distutils to arrive. If we have time/inclination to get stuff in before that occurs, and we believe it is the Right Thing, then why wait? That seems rather artificial. One argument (of course) is stability. I don't see that as a problem in the case of moving modules. Personally, I'm okay with a partial move. I don't think that we'll be able to find proper packaging for everything. Let's shift the modules, review what's left, shift some more, etc. Lather/rinse/repeat. Even if we don't "complete the move", then we've still *improved* the situation. Cheers, -g -- Greg Stein, http://www.lyra.org/ From skip@mojam.com (Skip Montanaro) Thu Feb 17 20:24:55 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 17 Feb 2000 14:24:55 -0600 (CST) Subject: [Python-Dev] packaging (was: davlib.py) In-Reply-To: <14508.20863.996377.452144@weyr.cnri.reston.va.us> References: <14508.18420.637014.518700@weyr.cnri.reston.va.us> <14508.20863.996377.452144@weyr.cnri.reston.va.us> Message-ID: <14508.22807.650629.824174@beluga.mojam.com> Fred> I'd prefer "network" over "protocol". Protocol can also mean Fred> other things. (Ok, ok: network can too. Can we call it "neural"? Fred> ;) No, no, no! That would be network.neural, network.wires, network.wireless, network.vapor, etc. ;-) Skip From gstein@lyra.org Thu Feb 17 20:33:20 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Feb 2000 12:33:20 -0800 (PST) Subject: [Python-Dev] new httplib? In-Reply-To: <14508.21611.538025.587013@goon.cnri.reston.va.us> Message-ID: On Thu, 17 Feb 2000, Jeremy Hylton wrote: > This sounds good enough for me! I endorse it for inclusion as long as > their is accompanying documentation <0.25 wink>. Nowadays, I consider it a prerequisite, along with an updated test suite. In the past, I've submitted stuff and the doc stuff has hung over my head for a long while... I'd rather get it done up front :-) Cheers, -g -- Greg Stein, http://www.lyra.org/ From bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Thu Feb 17 20:33:47 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) (Barry A. Warsaw) Date: Thu, 17 Feb 2000 15:33:47 -0500 (EST) Subject: [Python-Dev] package reorg in 1.6 (was: concrete proposal) References: <14508.22530.932416.357269@goon.cnri.reston.va.us> Message-ID: <14508.23339.745005.553973@anthem.cnri.reston.va.us> >>>>> "GS" == Greg Stein writes: GS> Personally, I'm okay with a partial move. I don't think that GS> we'll be able to find proper packaging for everything. Let's GS> shift the modules, review what's left, shift some more, GS> etc. Lather/rinse/repeat. GS> Even if we don't "complete the move", then we've still GS> *improved* the situation. I agree. There will be some modules which can clearly be grouped together, and others that'll take some head scratching (or worse, artificial conglomerificationating). Let's do the ones that make sense and see how far we get. We'll hold the nice fat `misc' package for 1.7 :) -Barry From fdrake@acm.org Thu Feb 17 20:40:56 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 17 Feb 2000 15:40:56 -0500 (EST) Subject: [Python-Dev] new httplib? In-Reply-To: References: <14508.21611.538025.587013@goon.cnri.reston.va.us> Message-ID: <14508.23768.741045.969557@weyr.cnri.reston.va.us> Greg Stein writes: > Nowadays, I consider it a prerequisite, along with an updated test suite. > In the past, I've submitted stuff and the doc stuff has hung over my head > for a long while... I'd rather get it done up front :-) And Greg knows how heavy it can hang! ;) Thanks for the buffer documentation, Greg! -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From Vladimir.Marangozov@inrialpes.fr Thu Feb 17 21:03:06 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Thu, 17 Feb 2000 22:03:06 +0100 (CET) Subject: [Python-Dev] RFC: malloc cleanup Message-ID: <200002172103.WAA22685@python.inrialpes.fr> Since nobody has had a spontaneous comment on the malloc problem I was referring to in my previous message, here is a concrete proposal. BTW, I was thinking on how to deal with *all* malloc problems and solve them definitively starting from by staying backwards compatible on top of that! Please comment on it, so that I could proceed on working on a patch (which involves lots of files in the distribution, so it'll take me some time verifying the files once again, and document the changes). --------------- I'll start with the public malloc interfaces, exported by libpython.a (and Python.h) for third party development. As a second step, I'll attack the "core". The "core" for me are the files in Parser/*, Python/* and Objects/*. I'm considering the files in Modules/* as extensions to the core. (although as long as they are in the standard distribution, one may want to see them as part of the core, but this is a secondary issue). -------------------------------------------------------------------------- Public Malloc (Object) Interface ================================ Functions: 1) PyMem_Malloc / Realloc / Free -- existing wrappers around the Python core malloc - don't call anything on failure. 2) Py_Malloc / Realloc / Free -- existing wrappers around the Python core malloc, calling PyErr_NoMemory on failure 3) _PyObject_New -- object creation, uses Python's core malloc Macros: 4) PyMem_NEW / PyMem_RESIZE -- the existing type-oriented macros *but* implemented in terms of 1). (PyMem_Malloc & PyMem_Realloc) 5) PyObject_NEW -- existing, implemented in terms of 3) 6) PyMem_DEL/XDEL -- existing, *but* pointing to PyMem_Free. PyMem_XDEL is deprecated, but is left for backwards compatibility. Note: everything is implemented on top of 1), the PyMem_Malloc wrappers. This will ensure that compiled C extensions can be used "as is" by new versions of the core (even if the latter has another malloc) ==> no need to recompile them. Questions: a) What about PyMem_DEL? Should it really be deprecated as a "handy alias" for PyMem_Free? Strictly speaking, I'd probably keep it so that PyMem_NEW/DEL form a pair. b) What about introducing PyObject_DEL as the buddy of PyObject_NEW? (also an alias for PyMem_Free). This is really an interface issue, so please adopt an "end-user" point of view. Personally, I'd feel more comfortable writing code like this one, voluntarily made to use the different interfaces (checks omitted). Object initialization: o = PyObject_NEW(spam_struct, &spam_type); /* new spam object */ o->foo = 1; o->bar = PyMem_Malloc(10); /* requesting Python memory */ o->my_buff = malloc(20); /* for personal needs */ Finalization: free(o->my_buff); /* release private mem */ PyMem_Free(o->bar); /* release Python mem */ PyObject_DEL(o); /* release the object */ That is, as a programmer, I don't really want to know what's behind the macros, as long as they come in pairs. I want things to be easier for me as much as possible. Note that as long as I allocate/release my objects with PyObject_NEW/DEL, I can use _any_ malloc I want for private purposes. --------------------------------------------------------------------------- Given the above, the malloc interfaces used by the core (the private ones) should be easier to understand. Basically, we'll have the the same macros (with the same questions a) & b) but everything here is implemented on top of PyMem_MALLOC and friends: Private/Core Malloc (Object) Interface ======================================= 1) PyMem_MALLOC / REALLOC / FREE - raw memory interface, core malloc. 2) _PyObject_New - uses 1) 3) PyMem_NEW / RESIZE / DEL(?) - use 1) PyMem_XDEL is out here. 4) PyObject_NEW / DEL (?) - use 2) & 1) -------------------------------------------------------------------------- The thing is: the public interfaces are always redirected through the wrappers, while the private ones are used (are visible) only by the core, and are based on a raw malloc. This is, IMO, the right approach for avoiding all troubles (past & future) Implementation issue ==================== Solution 1) If the exported and the core malloc interfaces preserve the same (macro) names, we need to introduce some preprocessor magic such that the same macros have different definitions depending on the type of the interface they belong to (public or core). Solution 2) The alternative is to avoid the name clashes by renaming the macros of the _core_ interface (the exported one cannot be changed for backwards compatibility reasons). That is, rename 3) and 4). (Note: 1) is new, 2) is a function) Question: which one is better? (assuming that both of them require the same amount of work ;-) ------------------------------------------------------------------------ That's it. Don't hesitate to ask me for clarifications. All undecided (not approved) parts of the proposal have been marked as such --> Questions. If we manage to agree on the differents questions, I believe that the implementation of this proposal would solve the issue permanently. Thanks for your patience! I'm sick either <0.5 wink> -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From viorel.preoteasa@abo.fi Thu Feb 17 21:33:39 2000 From: viorel.preoteasa@abo.fi (Viorel Preoteasa) Date: Thu, 17 Feb 2000 23:33:39 +0200 Subject: [Python-Dev] non-integer slice indices? References: <200002112042.OAA03792@beluga.mojam.com> <14501.18637.637227.834040@beluga.mojam.com> <38A58908.6AF91FBE@tismer.com> Message-ID: <38AC6933.A8E974A1@abo.fi> Dear All, Thank you very much for your answers. I will try to give answers to all problems that arise from my question on non-integer slices. First I will comment on any idea that arises from it, and then I will give my example. 1. peter> Hmmmm.... I was very astonished to read this, since it broke a frozen peter> model in my ---wannabe a python guru--- brain. peter> >>> class HyperSeq: peter> ... def __setslice__(self, i, j, sequence): peter> ... print "Just kidding. index i =", i, "j =", j, "seq =", sequence peter> >>> t = HyperSeq() peter> >>> t['a':'abc'] = "does this work?" peter> Traceback (innermost last): peter> File "", line 1, in ? peter> TypeError: slice index must be int peter> >>> peter> Now I think, that the model in my brain was not so wrong and that at peter> least in Python 1.5.2 slicing can't work with non-integer indices. skip> Indeed. (Obviously I didn't read the manual or perform the concrete skip> experiment that Peter did.) Before posting my message to python-hep, I have read the manual, I have tried a version of the above example, and I have read the python-FAQ. I have asked the question because I have guessed that may be there is a way to get __setslice__ called even when the indexes are not integers. Any way it is also useful to find out that there is not. I guess that this "frozen model" guides some answers that I got. Why this when somebody can give a consistent semantic for his program, and when there is a function that should deal with this? (Skip's idea) 2. greg> Currently, a person can do the following: greg> foo[slice('a','abc')] = some_sequence Yes he/she can, but it is easier and nicer to have something like foo.slice('a', 'abc', some_sequence) and much nicer foo['a':'abc'] = some_sequence when there is an appropriate semantic for it. 3. skip> Well, I'll be damned! To wit: skip> class SliceableDict(UserDict.UserDict): skip> ... Yes. I also don't have now a good example of data structure like dictionaries that can support such operation. But I have an example of data structure like lists. I guess it has a good semantic for the range between the non-integer indexes. See the example from the end. 4. Christian> When we write Christian> x[low:high] = some_sequence Christian> Christian> then we imply that there is a sequence on the left hand that Christian> can be indexed by the implicit ordered set of integers in Christian> the range [low, high), and we allow this assignment to change Christian> the sequence's length arbitrarily. Almost yes. But why by "implicit ordered set of ...". Why this order cannot depend also on the data that is stored in x. See my example. Christian> Well, I don't like any of these so much. They make dicts look Christian> like something ordered, that rings a bell about too much Christian> cheating. Yes, may be your example is not appropriate for such operation. But <> then why the interpreter could not let <>. As Skip suggested. My example: My example is very simple. I want to have an object that has as data structure lines of, for example, characters. So the basic data type is a list of strings, or list of list of characters. The indexes are pairs of line, column (how Tk Text Widget works). The order between indexes is given by the lexicographic order, i.e. (x,y)<=(u,v) iff x Message-ID: <010701bf799c$7a288c40$34aab5d4@hagrid> [Greg] > One argument (of course) is stability. I don't see that as a problem = in > the case of moving modules. >=20 > Personally, I'm okay with a partial move. I don't think that we'll be = able > to find proper packaging for everything. Let's shift the modules, = review > what's left, shift some more, etc. Lather/rinse/repeat. you obviously don't plan to write Python books in the foreseeable future :-) From gstein@lyra.org Thu Feb 17 23:10:24 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Feb 2000 15:10:24 -0800 (PST) Subject: [Python-Dev] package reorg in 1.6 In-Reply-To: <010701bf799c$7a288c40$34aab5d4@hagrid> Message-ID: On Fri, 18 Feb 2000, Fredrik Lundh wrote: > [Greg] > > One argument (of course) is stability. I don't see that as a problem in > > the case of moving modules. > > > > Personally, I'm okay with a partial move. I don't think that we'll be able > > to find proper packaging for everything. Let's shift the modules, review > > what's left, shift some more, etc. Lather/rinse/repeat. > > you obviously don't plan to write Python books in > the foreseeable future :-) hehe... The examples in the book will continue to work. But we'll just be doing it in A Better Way from now on :-) And no... I'm not going to write any books... (despite numerous inquiries from publishers) Cheers, -g -- Greg Stein, http://www.lyra.org/ From paul@prescod.net Thu Feb 17 23:19:00 2000 From: paul@prescod.net (Paul Prescod) Date: Thu, 17 Feb 2000 15:19:00 -0800 Subject: [Python-Dev] package reorg in 1.6 (was: concrete proposal) References: Message-ID: <38AC81E4.A3A8082C@prescod.net> Greg Stein wrote: > > Even if we don't "complete the move", then we've still *improved* the > situation. I feel like the reorg is more likely to get done if we do it as we can rather than waiting for someone to come along and do everything at once. The standard library already feels so "messy" to me that I don't think that use of packages in one place and not in another will confuse anyone. On the other hand, maybe we should only packagize things when we are also going to make sure that they make sense as a collective. For instance I would resist packaging binhex, uu, and binascii while they have radically different interfaces. I will leave it up to those more in the know to decide whether the "network" package would be coherent or just a bag of related but not really integrated modules. A brief perusal suggests that there is no such thing as a Gopher class (but then who cares!). And should urlparse be part of urllib? etc. -- Paul Prescod - ISOGEN Consulting Engineer speaking for himself "The calculus and the rich body of mathematical analysis to which it gave rise made modern science possible, but it was the algorithm that made possible the modern world." - from "Advent of the Algorithm" David Berlinski http://www.opengroup.com/mabooks/015/0151003386.shtml From mal@lemburg.com Thu Feb 17 23:39:17 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 18 Feb 2000 00:39:17 +0100 Subject: [Python-Dev] Unicode Implementation Snapshot 2000-02-18 Message-ID: <38AC86A5.6421F41D@lemburg.com> Hi everybody, I've just uploaded a new snapshot to the secret URL. New in this snapshot is a generic character mapping codec which can decode and encode a large number of code pages used on PCs and Macs. I used a Unicode mapping file parser to automatically generate the codecs from the mapping files available at http://www.unicode.org/ and then included all those files which use less than 10k for the Python source code (with comments). These codecs are thus available and need some serious testing: cp855.py iso_8859_6.py cp856.py iso_8859_7.py ascii.py cp857.py iso_8859_8.py charmap.py cp860.py iso_8859_9.py cp1006.py cp861.py koi8_r.py cp1250.py cp862.py latin_1.py cp1251.py cp863.py mac_cyrillic.py cp1252.py cp864.py mac_greek.py cp1253.py cp865.py mac_iceland.py cp1254.py cp866.py mac_latin2.py cp1255.py cp869.py mac_roman.py cp1256.py cp874.py mac_turkish.py cp1257.py iso_8859_10.py raw_unicode_escape.py cp1258.py iso_8859_13.py unicode_escape.py cp424.py iso_8859_14.py unicode_internal.py cp437.py iso_8859_15.py utf_16.py cp737.py iso_8859_2.py utf_16_be.py cp775.py iso_8859_3.py utf_16_le.py cp850.py iso_8859_4.py utf_8.py cp852.py iso_8859_5.py All these codecs are stored in the encodings package of the standard lib and directly useable via the unicode(input, encoding) and u"abc".encode(encoding) APIs. I would like some feedback on which of these code pages are really in common use... we could make all not so common ones available as separate package then. Also, I'm curious if we should rename the cpXXX.py files to cp_XXX.py or not (or whether to just add aliases to the encodings/aliases.py file for them). The naming scheme usually defines letters-numbers-etc. but for code pages the above names are quite common. Another feature of the patch is that it has some optimizations for short Unicode strings. Unfortunately, the implementation still has some bugs, so it is currently disabled. To reenable it, edit the file Objects/unicodeobject.c and set e.g. #define STAYALIVE_SIZE_LIMIT 5 This will cause to the Unicode objects on the free list having a size below or equal to this limit to stay alive even when on the free list. Note that this is the final patch for the next week. I'll be offline until 2000-02-28 and then hope to make some serious progress on documenting the different parts (most docs are still buried in the C and header files and the unicode proposal which is included in the file Misc/unicode.txt). Now it's up to you to give the code the final swirl... :-) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From da@ski.org Sun Feb 20 18:42:12 2000 From: da@ski.org (David Ascher) Date: Sun, 20 Feb 2000 10:42:12 -0800 Subject: [Python-Dev] Borland release free C++ compiler Message-ID: <004a01bf7bd2$3401e030$0100000a@ski.org> They've made their compiler free-as-in-beer. http://www.borland.com/about/press/2000/bcppcompiler.html It would be nice to have relevant makefiles in the source tree by the next release. --david From da@ski.org Fri Feb 18 22:27:56 2000 From: da@ski.org (David Ascher) Date: Fri, 18 Feb 2000 14:27:56 -0800 Subject: [Python-Dev] TomC Message-ID: <01ae01bf7a5f$67afcec0$0100000a@ski.org> Seen on perl5-porters: >...and thus providing a rationale for C, as opposed to >C which is merely checking for an array element >containing a perfectly valid scalar value... Exacerbating the number of ways you can count the disparate but indistinguishable sets of angels dancing on the head of a pin serves no legitimate purpose save to further elevate the priesthood of the triviamasters above neophyte and acolyte alike by needlessly injecting the abstruse into the quotidian, thereby reducing the already debatable accessibility of Perl by those common masses who need it most--all because of the irrepressible urge by the destructively clever to continue unfettered these labyrinthine embellishments, gratuitous nuances of surpassing subletly and dubious utility. Let your true be true and your false be false, and add not to the already confusing number of falsehoods, lest the resulting eminently confusable shadings be forever beyond the ken of mortal men trying to get their jobs done without first subjecting themselves to decades of monastic study. --tom From Vladimir.Marangozov@inrialpes.fr Fri Feb 18 21:45:47 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 18 Feb 2000 22:45:47 +0100 (CET) Subject: [Python-Dev] Re: RFC: malloc cleanup In-Reply-To: from "marangoz" at Feb 17, 2000 10:03:06 PM Message-ID: <200002182145.WAA28785@python.inrialpes.fr> [Detailed RFC on a bunch of malloc interfaces] OK, I'll comment first as I'm probably one of the few who understands (what I'm talking about :-) what's this mess is all about in its deepest details while having an overall vision on the issue. So I feel obliged to expose a more friendly explanation. (This turned out to be only 2 screens long, so you may leave it for tomorrow morning ;-) I'll make it simple: What really happens if the RFC gets implemented and how it affects you? ----------------------------------------------------------------------- 1) All C extension modules, without exceptions, would be silently redirected to use the Python malloc wrappers. Silently means that no one line of code would need to be modified and the modules will continue to compile and run as before. (i.e. all macros and functions used in C code remain the same) Consequences: a) You *won't* be required to modify anything, to change your programming habits or to adopt new mandatory interfaces. You could continue the same way as before. b) All *user-defined* Python objects (not the "core" ones, like ints, dicts or strings) would start using the wrappers automatically, which may result in a tiny degradation in their performance. This is the price to pay if we want to make Python user-malloc friendly -- that is, decouple the core from its strong dependency on libc malloc and make 3rd party extensions depend only on Python's memory interfaces when they manipulate Python objects. Currently, the extensions depend on libc malloc (they use "public" memory offered by the system) and there's no way to make them use "private Python memory" for Python objects. BTW, user-defined objects are used far less frequently than the "core" ones. 2) All "core" objects will run as before at the same speed, but they'll use "private Python memory". 3) There would be an opportunity to make the code fancier. This is the whole point of the discussion about the NEW/DEL pairs. Fancier == more logical from the programmer's point of view. This is not mandatory, but it would be desirable to adopt these pairs so that mallocs don't mess again. And I'm volunteering to rename the core and the modules in the distribution so that they look fancier and that they serve as examples for future Python development. But all this remains optional. 4) Last, but not least, this will open the way to change (optionally) Python's core allocator to specialized and more appropriate mallocs than libc malloc. There's only one I know of for the moment (mine ) and it's not "so good". But it looks promising, because it already serves Python better than libc malloc. The current state, however, prevents people to work in this area and to experiment alternatives, because it's very hard to get things working without an expert level knowledge of the internals. I personally find this regrettable and am willing to "repair" it, because I think that Python can make a noticeable profit from a specialized malloc (which may well outperform the tiny degradation caused by the wrappers). ------------ Currently I have a patch which does this the "easy way" -- it does not truly decouple the core from the extensions through the wrappers. However, one can already make experiments by recompiling Python *and* all user extensions with a new malloc. With the patch, Python doesn't dump core anymore, which may be a little gain in your eyes, but a huge one in mine . FYI, I've corresponded in private with Neil Shemenauer who's working on GC, and he tested & validated the present "easy" version of the patch by saying: > On Sun, Feb 13, 2000 at 04:06:05PM +0100, Vladimir Marangozov wrote: > > I'm appending another patch suite, so that you could test it directly > > on your setup with your malloc. Let me know how it goes. Hope we're > > making progress on this. > > It works for me. It passes the regression test and also does not > leak memory when cycles are created. :) > > > Neil > ------------ So what I'm looking for here is your approval or your objections to the principles I've exposed in this msg and in the RFC. I can presently send a whole bunch of patched files affected by the "easy" way, but it does not solve the real problem. The "easy way" does not redirect the extensions to the wrappers. i.e. it doesn't have the "private Python memory" concept. It *is* a step in the right direction, though. Final word: I'm okay for deferring this issue for future Python releases, if there's no time, no resources, no understanding, whatever (lately, I see a big increase in Python's contributions since the announce of the tight schedule for 1.6). That's why I won't push too much and I don't want to burded the atmosphere with still undecided interfaces -- better take some more time and get them right. OTOH, the sooner we get them right, the better. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From guido@python.org Mon Feb 21 17:30:46 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 21 Feb 2000 12:30:46 -0500 Subject: [Python-Dev] Borland release free C++ compiler In-Reply-To: Your message of "Sun, 20 Feb 2000 10:42:12 PST." <004a01bf7bd2$3401e030$0100000a@ski.org> References: <004a01bf7bd2$3401e030$0100000a@ski.org> Message-ID: <200002211730.MAA02796@eric.cnri.reston.va.us> > They've made their compiler free-as-in-beer. > > http://www.borland.com/about/press/2000/bcppcompiler.html > > It would be nice to have relevant makefiles in the source tree by the > next release. That would be nice indeed. Too bad I don't have time to do more stuff. Care to hustle for volunteers on c.l.py? --Guido van Rossum (home page: http://www.python.org/~guido/) From DavidA@ActiveState.com Mon Feb 21 17:34:18 2000 From: DavidA@ActiveState.com (David Ascher) Date: Mon, 21 Feb 2000 09:34:18 -0800 Subject: [Python-Dev] Borland release free C++ compiler In-Reply-To: <200002211730.MAA02796@eric.cnri.reston.va.us> Message-ID: <000901bf7c91$e149de40$c355cfc0@ski.org> > That would be nice indeed. Too bad I don't have time to do more > stuff. > > Care to hustle for volunteers on c.l.py? Will do. --da From fdrake@acm.org Mon Feb 21 18:42:15 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 21 Feb 2000 13:42:15 -0500 (EST) Subject: [Python-Dev] Re: RFC: malloc cleanup In-Reply-To: <200002182145.WAA28785@python.inrialpes.fr> References: <200002182145.WAA28785@python.inrialpes.fr> Message-ID: <14513.34567.396636.374654@weyr.cnri.reston.va.us> Vladimir, Frankly, I think this is a good idea and should be implemented for 1.6, especially the conversion of the extension modules that ship with Python (Modules/*.c). Having solid examples of "the right way" is very important in making the right way accepted and *used*. I'd also like to see the various interfaces documented. Since you seem to be the expert on this, I'm asking you to work up the needed content for the Python/C API document! (You should have been expecting this! ;) If you can mail me the text to use, I'd really appreciate it! I can handle the LaTeX-ification if you'd rather not. I'm planning to release the final 1.5.2 documentation in a few weeks.... ;) -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From bckfnn@worldonline.dk Fri Feb 18 21:11:36 2000 From: bckfnn@worldonline.dk (Finn Bock) Date: Fri, 18 Feb 2000 21:11:36 GMT Subject: [Python-Dev] Unicode Implementation in JPython Message-ID: <38adb510.17806223@smtp.worldonline.dk> Hi, My feeling on the unicode proposal and its implementation is that most of the changes can be integrated directly into JPython without breaking any existing JPython code. One thing concerns me though: open("out", "wb").write(u"hello") This writes a 10 bytes to the file "out". I have two problems with that: 1. In java, files are always byte-based. To move from unicode chars to bytes some kind of encoder must always be applied. It is also strange to see the actual byte layout of the data, which in my "out" file seems to be platform dependent. Is that the case? If it is, then the write(u"..") strikes me as somewhat random (unknown). 2. To get this behavior under JPython, it is necessary to introduce a new string type which in all other aspects are equal to the existing string type. Only when passed to file.write should the new string type returned a faked representation of its memory. When a normal string is passed to .write, some byte representation of the string is written to the file. I would prefer that in jpython a unicode string is the same as a normal string (type("") == type(u"")). Perhaps the real reason for my dislike of this feature of the unicode implementation is based on my (from java) assumption that a unicode character is an atomic data type. regards, finn From guido@python.org Mon Feb 21 20:13:19 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 21 Feb 2000 15:13:19 -0500 Subject: [Python-Dev] Unicode Implementation in JPython In-Reply-To: Your message of "Fri, 18 Feb 2000 21:11:36 GMT." <38adb510.17806223@smtp.worldonline.dk> References: <38adb510.17806223@smtp.worldonline.dk> Message-ID: <200002212013.PAA03712@eric.cnri.reston.va.us> > My feeling on the unicode proposal and its implementation is that most > of the changes can be integrated directly into JPython without breaking > any existing JPython code. One thing concerns me though: > > open("out", "wb").write(u"hello") (Note that the file is opened in *binary* mode; in text mode, this would write the 5 bytes or "hello".) > This writes a 10 bytes to the file "out". > > I have two problems with that: > > 1. In java, files are always byte-based. To move from unicode chars to > bytes some kind of encoder must always be applied. It is also strange to > see the actual byte layout of the data, which in my "out" file seems to > be platform dependent. Is that the case? If it is, then the > write(u"..") strikes me as somewhat random (unknown). > > 2. To get this behavior under JPython, it is necessary to introduce a > new string type which in all other aspects are equal to the existing > string type. Only when passed to file.write should the new string type > returned a faked representation of its memory. When a normal string is > passed to .write, some byte representation of the string is written to > the file. I would prefer that in jpython a unicode string is the same as > a normal string (type("") == type(u"")). > > Perhaps the real reason for my dislike of this feature of the unicode > implementation is based on my (from java) assumption that a unicode > character is an atomic data type. Hm, I agree that it's not a great feature. On the other hand it's hard to decide what to do instead without breaking other corners of the Unicode design. Could we leave this implementation-dependent? --Guido van Rossum (home page: http://www.python.org/~guido/) From tim_one@email.msn.com Tue Feb 22 07:35:23 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 22 Feb 2000 02:35:23 -0500 Subject: [Python-Dev] Re: RFC: malloc cleanup In-Reply-To: <200002182145.WAA28785@python.inrialpes.fr> Message-ID: <000a01bf7d07$614de080$e82d153f@tim> [Vladimir Marangozov] > OK, I'll comment first as I'm probably one of the few who understands > (what I'm talking about :-) what's this mess is all about in its deepest > details while having an overall vision on the issue. So I feel obliged to > expose a more friendly explanation. > ... I said it before & I'll say it again: "just do it"! Was there some opposition to this cleanup I've failed to notice on Python-Dev? The "overcommitting Linux malloc" work issue I mentioned before looks like it will require us to fiddle Python to use a homegrown malloc (that touches every allocated byte to make sure the memory "is really there" ), so with or without an official patch we're going to have to get the effect of it. So far, my boss is highly in favor of Vladimir doing the work instead of me <0.6 wink>. under-the-law-silence-implies-consent-ly y'rs - tim From jack@oratrix.nl Tue Feb 22 11:31:17 2000 From: jack@oratrix.nl (Jack Jansen) Date: Tue, 22 Feb 2000 12:31:17 +0100 Subject: [Python-Dev] win32 specific exception in the core? In-Reply-To: Message by "Mark Hammond" , Sat, 12 Feb 2000 12:44:53 +1100 , Message-ID: <20000222113118.0371F371868@snelboot.oratrix.nl> > Im looking for closure on the Win32 specific exception. Summary for those > that missed the thread: I'm sorry that I'm so late, but I was busy with other things. Anyway: > * I would like a standard exception for Win32 specific errors. The new > hopefully-to-be-standard "win32reg" module could make use of it, the win32 > extensions could switch to it. It is, in my mind at least, analogous to > OSError. Agreed. > My initial proposal called for: > > StandardError: > -> EnvironmentError: > -> IOError > -> OSError > -> Win32Error > > Which would generally lead to people writing "except Win32Error:" > > Tim's suggestion (as I understand it) was: > StandardError: > -> EnvironmentError: > -> IOError > -> OSError > -> CRTError (new - is what os.error would use) > -> Win32Error (new) I also like Mark's idea better. MacOSError has the same problems as Win32Error (different numbering scheme). If OSError can suddenly return numbers from one of three sets we're bound to break a lot of code. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From jack@oratrix.nl Tue Feb 22 16:04:08 2000 From: jack@oratrix.nl (Jack Jansen) Date: Tue, 22 Feb 2000 17:04:08 +0100 Subject: [Python-Dev] Re: RFC: malloc cleanup In-Reply-To: Message by "Fred L. Drake, Jr." , Mon, 21 Feb 2000 13:42:15 -0500 (EST) , <14513.34567.396636.374654@weyr.cnri.reston.va.us> Message-ID: <20000222160408.846D1371868@snelboot.oratrix.nl> One point to think of, with the malloc cleanup: how many places are there where Python allocates something that is passed to some packages which then takes ownership of it (and ultimately free()s it)? And the other way around? For these cases we should probably continue to use malloc()/free() to be on the safe side. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From guido@python.org Tue Feb 22 16:44:57 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 22 Feb 2000 11:44:57 -0500 Subject: [Python-Dev] concrete proposal (was: davlib.py) In-Reply-To: Your message of "Thu, 17 Feb 2000 15:20:18 EST." <14508.22530.932416.357269@goon.cnri.reston.va.us> References: <14508.10998.495303.415806@weyr.cnri.reston.va.us> <14508.22530.932416.357269@goon.cnri.reston.va.us> Message-ID: <200002221644.LAA04972@eric.cnri.reston.va.us> > I'm all for re-organizing the library using packages. I don't think > I'm in favoring of doing some of it now, and some of it later. Rather > than moving modules into the network package (good name), I'd > hold off and do full packaging for 1.7. Amen. --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Tue Feb 22 16:50:08 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 22 Feb 2000 11:50:08 -0500 (EST) Subject: [Python-Dev] Re: RFC: malloc cleanup In-Reply-To: <20000222160408.846D1371868@snelboot.oratrix.nl> References: <14513.34567.396636.374654@weyr.cnri.reston.va.us> <20000222160408.846D1371868@snelboot.oratrix.nl> Message-ID: <14514.48704.285544.986043@weyr.cnri.reston.va.us> Jack Jansen writes: > One point to think of, with the malloc cleanup: how many places are there > where Python allocates something that is passed to some packages which then > takes ownership of it (and ultimately free()s it)? And the other way around? > > For these cases we should probably continue to use malloc()/free() to be on > the safe side. Yes; that would be the only way to handle that if it doesn't provide any way to control the deallocation. And if it does, you can still use malloc & free. Good enough for me! -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From guido@python.org Tue Feb 22 17:08:44 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 22 Feb 2000 12:08:44 -0500 Subject: [Python-Dev] Expat -- it keeps going, and going, and going ... In-Reply-To: Your message of "Wed, 09 Feb 2000 07:36:33 PST." <38A18981.CED6F585@prescod.net> References: <38A18981.CED6F585@prescod.net> Message-ID: <200002221708.MAA05028@eric.cnri.reston.va.us> On Feb 9, Paul Prescod wrote (amongst other things): > If we can get concensus on this issue, I will approach James Clark for a > more Pythonic license. Right now it has an MPL license but I suspect > that James will be flexible. I'm coming late to the XML discussion. I haven't looked at Expat or its license or PyExpat or any of the other stuff that you are suggesting that should be added, but if you all want this in Python, and if we can get the license to do it, I'd say let's go for it. The license issue is potentially a showstopper though. Have you talked to James Clark about it yet? Assuming we can get the license to work, I propose something like the following (all work to be done by members of the XML-SIG or other volunteers!): - On Windows, incorporate Expat in an extension (a .PYD file, i.e. a DLL in disguise), statically linked. The VC++ project files in the PCbuild directory should be updated to build this, assuming the expat source tree is elsewhere on the disk (we do this for the zlib, bsddb and _tkinter extensions too, so it's no big deal). - On Unix, if you want Expat you have to download and build it yourself, then change a few lines in Python's Modules/Setup file, and rebuild Python and you're done. Linux repackagers can add Expat support to their list. - On all platforms, the Python modules for XML support are always available. Please use a reasonable package structure for everything that's not already in the current distribution. I'd say a toplevel name with "xml" in it should be fine. - Leave xmllib.py and friends alone. Add a note to their documentation that says that better solutions are available. - It needs to come with documentation and testing. Without these, it won't go into 1.6, 1.7 *or* 3000. --Guido van Rossum (home page: http://www.python.org/~guido/) From bckfnn@worldonline.dk Tue Feb 22 19:48:45 2000 From: bckfnn@worldonline.dk (Finn Bock) Date: Tue, 22 Feb 2000 19:48:45 GMT Subject: [Python-Dev] Unicode Implementation in JPython In-Reply-To: <200002212013.PAA03712@eric.cnri.reston.va.us> References: <38adb510.17806223@smtp.worldonline.dk> <200002212013.PAA03712@eric.cnri.reston.va.us> Message-ID: <38b2e80f.10980749@smtp.worldonline.dk> [me] > My feeling on the unicode proposal and its implementation is that most > of the changes can be integrated directly into JPython without breaking > any existing JPython code. One thing concerns me though: > > open("out", "wb").write(u"hello") > > This writes a 10 bytes to the file "out". [snip] [Guido van Rossum] >Hm, I agree that it's not a great feature. On the other hand it's >hard to decide what to do instead without breaking other corners of >the Unicode design. Could we leave this implementation-dependent? Absolutely. In fact, that was exactly what I wanted to hear. Thanks. regards, finn From Vladimir.Marangozov@inrialpes.fr Tue Feb 22 20:07:13 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Tue, 22 Feb 2000 21:07:13 +0100 (CET) Subject: [Python-Dev] Re: RFC: malloc cleanup In-Reply-To: <20000222160408.846D1371868@snelboot.oratrix.nl> from "Jack Jansen" at Feb 22, 2000 05:04:08 PM Message-ID: <200002222007.VAA12975@python.inrialpes.fr> Jack Jansen wrote: > > One point to think of, with the malloc cleanup: how many places are there > where Python allocates something that is passed to some packages which then > takes ownership of it (and ultimately free()s it)? None that I know of. > And the other way around? > One. PyOS_Readline, when the interpreter is compiled with the readline module. (i.e when the PyOS_ReadlineFunctionPointer hook is activated, see myreadline.c) Actually PyOS_Readline gets a malloc'ed chunk from the readline module which is subsequently freed by Python. The right way to do it is to require that the buffer passed to PyOS_Readline is allocated through PyMem_Malloc. This should/will be documented, and is corrected in the patch. The other way around would be wrong, i.e. Python should not release the incoming buffer with free(). We can't know the malloc used on the other side, so it's not our job to guess it. (think embedding apps) Note that the 2 cases you're describing are bad design. > For these cases we should probably continue to use malloc()/free() to be on > the safe side. For now, nobody will notice anything, because Python's malloc will default to malloc/free. So we're still on the safe side. It's about preparing the code to be able to switch core malloc, eventually, optionally. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Vladimir.Marangozov@inrialpes.fr Tue Feb 22 20:14:56 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Tue, 22 Feb 2000 21:14:56 +0100 (CET) Subject: [Python-Dev] Re: RFC: malloc cleanup In-Reply-To: <20000221180728.9984@trixie.triqs.com> from "Jean-Claude Wippler" at Feb 21, 2000 12:07:27 PM Message-ID: <200002222014.VAA13031@python.inrialpes.fr> Jean-Claude Wippler wrote: > > Hello Vladimir, > > >1) All C extension modules, without exceptions, would be silently redirected > > to use the Python malloc wrappers. Silently means that no one line of code > > would need to be modified and the modules will continue to compile and run > > as before. (i.e. all macros and functions used in C code remain the same) > > Have you taken C++ into account (i.e. new and new[] in the C++ runtime > calling malloc)? As long as Python objects are allocated and freed with PyObject_NEW/DEL, and Python memory is manipulated with the Python malloc interfaces, I don't see a problem, even if the core, or some extensions are compiled with a C++ compiler. C++ constructors and destructors use malloc (usually), but remain completely symmetrical. That is what's allocated via malloc() is released via free(). Why, should I be aware of something special? -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From pf@artcom-gmbh.de Thu Feb 24 15:41:32 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Thu, 24 Feb 2000 16:41:32 +0100 (MET) Subject: [Python-Dev] Re: [Patches] Patch to src/Modules/mpzmodule.c In-Reply-To: <200002241534.KAA09725@eric.cnri.reston.va.us> from Guido van Rossum at "Feb 24, 2000 10:34:18 am" Message-ID: Hi! I wrote: > > >This patch allows building the Python 'mpzmodule' under SuSE Linux > > >without having to install the source package of the GMP-libary. > > >The gmp-mparam.h seems to be an internal header file. The patch > > >shouldn't hurt any other platforms. in src/Modules/mpzmodule.c > > >+ #if defined __GLIBC__ && __GLIBC__ >= 2 /*aid building on Linux distributions*/ > > >+ #define BITS_PER_MP_LIMB mp_bits_per_limb > > >+ #else > > > #include "gmp-mparam.h" > > >+ #endif > > Andrew replies: > > This seems like the wrong fix, tying GMP to the version of glibc! > > (They haven't rolled GMP into glibc, have they?) If GMP 2.0 exposes > > an mp_bits_per_limb constant, then the correct patch may just be > > changing BITS_PER_MP_LIMB to mp_bits_per_limb, inside an #ifdef GMP2 > > if necessary. Guido van Rossum wrote: > Darn. I just checked it in. Well, if something better comes up, I'll > check that in. I personally don't care much about this module (have > never bothered to download gmp even so I can't even compile it). I will investigate the source code of earlier versions of GMP in more detail later. May be Andrew is right, that my patch will hurt people, that happen to use a recent version GLIBC together with in ancient version of GMP. However also I believe this is rather unlikely, it shouldn't break. Sorry for the confusion I've caused. Regards, Peter -- Peter Funk, Oldenburger Str.86, 27777 Ganderkesee, Tel: 04222 9502 70, Fax: -60 From gstein@lyra.org Thu Feb 24 21:52:31 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 24 Feb 2000 13:52:31 -0800 (PST) Subject: [Python-Dev] using ":name" in ParseTuple() Message-ID: If anyone out there is looking for some straight-forward changes to help with Python development, then here is a chance... I'll get around to submitting patches for this, but it would be much later. So, if somebody can beat me to it... Cheers, -g -- Greg Stein, http://www.lyra.org/ ---------- Forwarded message ---------- Date: Thu, 24 Feb 2000 16:42:12 -0500 From: Guido van Rossum To: Greg Stein Cc: python-checkins@python.org Subject: Re: [Python-checkins] CVS: python/dist/src/Objects listobject.c,2.63, 2.64 > On a separate track, how about using the ":Name" form in the ParseTuple > calls? It improves the error response a bit more... > > Is that something you'd want to see in all the ParseTuple calls? Ah, darn. I tried ";message" but it's inferior; I'd forgotten about ":name". Yes, that should be standard... Patches, please? --Guido van Rossum (home page: http://www.python.org/~guido/) _______________________________________________ Python-checkins mailing list Python-checkins@python.org http://www.python.org/mailman/listinfo/python-checkins From skip@mojam.com (Skip Montanaro) Thu Feb 24 22:56:31 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 24 Feb 2000 16:56:31 -0600 (CST) Subject: [Python-Dev] using ":name" in ParseTuple() In-Reply-To: References: Message-ID: <14517.46879.267016.843990@beluga.mojam.com> Greg> If anyone out there is looking for some straight-forward changes Greg> to help with Python development, then here is a chance... Greg, Can you give some brief instructions for those of us who haven't doing other things in school when we should been paying attention? Is it just adding ":public_name" to any and all format strings we can find? I'm going to have a little evening time free off-net while I'm out of town for the next week. I will freshen up my CVS copy and try doing a bunch. Skip From skip@mojam.com (Skip Montanaro) Thu Feb 24 23:03:46 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 24 Feb 2000 17:03:46 -0600 (CST) Subject: [Python-Dev] using ":name" in ParseTuple() In-Reply-To: <14517.46879.267016.843990@beluga.mojam.com> References: <14517.46879.267016.843990@beluga.mojam.com> Message-ID: <14517.47314.80115.130405@beluga.mojam.com> Skip> Can you give some brief instructions for those of us who haven't Skip> doing other things in school when we should been paying attention? Yiee! You know what I was doing! Should have been: Can you give some brief instructions for those of us who were doing other things in school when we should have been paying attention? i-gotta-clean-my-glasses-ly y'rs, S From gstein@lyra.org Thu Feb 24 23:35:47 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 24 Feb 2000 15:35:47 -0800 (PST) Subject: [Python-Dev] using ":name" in ParseTuple() In-Reply-To: <14517.46879.267016.843990@beluga.mojam.com> Message-ID: On Thu, 24 Feb 2000, Skip Montanaro wrote: >... > Is it just adding > ":public_name" to any and all format strings we can find? I'm going to have > a little evening time free off-net while I'm out of town for the next week. > I will freshen up my CVS copy and try doing a bunch. Yup. That's all there is. For example, consider line 1013 in stringobject.c (string_lower function): today: if (!PyArg_ParseTuple(args, "")) change to: if (!PyArg_ParseTuple(args, ":lower")) The :name part should be the user-visible name of the function or method. It does get tricky with something like the string_rfind_internal function -- the ParseTuple call is used for several functions. Cheers, -g -- Greg Stein, http://www.lyra.org/ From akuchlin@mems-exchange.org Fri Feb 25 19:31:37 2000 From: akuchlin@mems-exchange.org (Andrew M. Kuchling) Date: Fri, 25 Feb 2000 14:31:37 -0500 (EST) Subject: [Python-Dev] Whither cursesmodule? Message-ID: <200002251931.OAA07324@amarok.cnri.reston.va.us> On comp.lang.python, "Juergen A. Erhard" wrote about cursesmodule: > Why two versions? Did Oliver forget to submit his patches to Guido > (et al)? Or did Guido not accept them? If so, why not? > > What needs to be done to synchronize the canonical Python and the > Python RPMs? For python-dev readers: Oliver Andrich's Python RPMs contain his enhanced cursesmodule, which supports many ncurses features. The cursesmodule in the Python distribution supports only plain curses. Question: what should be done about this? The problem is that Oliver's enhanced module probably won't work on systems that support only BSD curses. I haven't verified this, though. On the other hand, ncurses implements the SYSV curses API, and maybe there are no platforms left that only have plain curses. Options: 1) Forget about it and leave things as they are. 2) Include the ncurses version of the module, backward compatibility be damned. 3) Split the curses module out of the standard distribution, and distribute it separately; users then download the plain or ncurses version as they see fit. 4) Attempt to make patches for Oliver's module that will make it work with plain curses. I don't like #1; if the code is going to be unmaintained in the future, why leave it in at all? #2 might be OK, if it's the case that the SYSV curses API is widespread these days; is it? I'd be willing to take a crack at #4, but have no idea where I could find a system with only plain curses. (Apparently OpenBSD, at least, includes the old BSD curses as libocurses.) -- A.M. Kuchling http://starship.python.net/crew/amk/ When a man tells you that he got rich through hard work, ask him *whose*? -- Don Marquis From guido@python.org Fri Feb 25 19:37:15 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 25 Feb 2000 14:37:15 -0500 Subject: [Python-Dev] Whither cursesmodule? In-Reply-To: Your message of "Fri, 25 Feb 2000 14:31:37 EST." <200002251931.OAA07324@amarok.cnri.reston.va.us> References: <200002251931.OAA07324@amarok.cnri.reston.va.us> Message-ID: <200002251937.OAA20125@eric.cnri.reston.va.us> > On comp.lang.python, "Juergen A. Erhard" wrote about > cursesmodule: > > Why two versions? Did Oliver forget to submit his patches to Guido > > (et al)? Or did Guido not accept them? If so, why not? > > > > What needs to be done to synchronize the canonical Python and the > > Python RPMs? > > For python-dev readers: Oliver Andrich's Python RPMs contain his > enhanced cursesmodule, which supports many ncurses features. The > cursesmodule in the Python distribution supports only plain curses. > Question: what should be done about this? > > The problem is that Oliver's enhanced module probably won't work on > systems that support only BSD curses. I haven't verified this, > though. On the other hand, ncurses implements the SYSV curses API, > and maybe there are no platforms left that only have plain curses. > > Options: > > 1) Forget about it and leave things as they are. > > 2) Include the ncurses version of the module, backward compatibility > be damned. > > 3) Split the curses module out of the standard distribution, and > distribute it separately; users then download the plain or ncurses > version as they see fit. > > 4) Attempt to make patches for Oliver's module that will make it work > with plain curses. > > I don't like #1; if the code is going to be unmaintained in the > future, why leave it in at all? #2 might be OK, if it's the case that > the SYSV curses API is widespread these days; is it? I'd be willing > to take a crack at #4, but have no idea where I could find a system > with only plain curses. (Apparently OpenBSD, at least, includes the > old BSD curses as libocurses.) I vote for #3 -- I have zero interest in curses, and it is probably better off having its own website, Vaults of Parnassus entry, etc., than being in the core and utterly unmaintained. (Also note that long ago, someone gave me patches for support of color curses; there was absolutely no interest, so I haven't integrated them.) Note that we have a similar situation with the BSDDB module: the distribution contains a wrapper for BSDDB 1.85, while someone else maintains a wrapper for Sleepycat's BSDDB 3.x. The reasons for the fork are a bit different there: BSDDB 3.x, while superior, isn't as free as 1.85, so some people must use 1.85 (if they want to use it commercially but don't want to license 3.x from Sleepycat). --Guido van Rossum (home page: http://www.python.org/~guido/) From Moshe Zadka Fri Feb 25 20:11:33 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 25 Feb 2000 22:11:33 +0200 (IST) Subject: [Python-Dev] Whither cursesmodule? In-Reply-To: <200002251931.OAA07324@amarok.cnri.reston.va.us> Message-ID: On Fri, 25 Feb 2000, Andrew M. Kuchling wrote: > 2) Include the ncurses version of the module, backward compatibility > be damned. I think I'm for it. I'm not sure if the backward compatibility problem would be so hard. (I actually did use curses, with the Python module) -- Moshe Zadka . INTERNET: Learn what you know. Share what you don't. From akuchlin@mems-exchange.org Fri Feb 25 22:39:24 2000 From: akuchlin@mems-exchange.org (Andrew M. Kuchling) Date: Fri, 25 Feb 2000 17:39:24 -0500 (EST) Subject: [Python-Dev] Whither cursesmodule? In-Reply-To: <200002251937.OAA20125@eric.cnri.reston.va.us> References: <200002251931.OAA07324@amarok.cnri.reston.va.us> <200002251937.OAA20125@eric.cnri.reston.va.us> Message-ID: <14519.1180.432464.376558@amarok.cnri.reston.va.us> Guido van Rossum writes: >> 3) Split the curses module out of the standard distribution, and >> distribute it separately; users then download the plain or ncurses >> version as they see fit. >I vote for #3 -- I have zero interest in curses, and it is probably >better off having its own website, Vaults of Parnassus entry, etc., >than being in the core and utterly unmaintained. (Also note that long Fine. I'll post to c.l.p and ask if anyone wants to maintain it. If no one expresses an interest, I'll happily wrap the module up neatly and put it someplace (python.org? my Starship pages? www.mems-exchange.org?). But I'm *not* interested in maintaining the plain curses module in future. -- A.M. Kuchling http://starship.python.net/crew/amk/ [John] Dalton's records, carefully preserved for a century, were destroyed during the World War II bombing of Manchester. It is not only the living who are killed in war. -- Isaac Asimov From guido@python.org Sat Feb 26 16:20:43 2000 From: guido@python.org (Guido van Rossum) Date: Sat, 26 Feb 2000 11:20:43 -0500 Subject: [Python-Dev] Re: [Patches] Readline replacement under QNX in myreadline.c In-Reply-To: Your message of "Sat, 26 Feb 2000 12:39:16 +0200." <38B7AD54.57498D82@elisra.com> References: <38B7AD54.57498D82@elisra.com> Message-ID: <200002261620.LAA21156@eric.cnri.reston.va.us> > This patch for Python 1.52 , /Parser/myreadline.c on QNX using Watcom > C++. > Readline does not work properly . > Using QNX input_line function instead of Linux readline. > ------------------------------------------------------------ > 30,38d29 > < > < #ifdef __QNX__ > < p = input_line( fp, buf, len ); > < if( p ) { > < int n = strlen(p); > < p[n] = '\n'; > < p[n+1] = 0; > < } > < #else > 40d30 > < #endif > ------------------------------------------------------------- I seem to recall that this came up recently but I don't remember where. Can anybody jog my memory? What did we decide in the end? --Guido van Rossum (home page: http://www.python.org/~guido/) From Fredrik Lundh" <200002261620.LAA21156@eric.cnri.reston.va.us> Message-ID: <003901bf811a$68136ea0$34aab5d4@hagrid> when hacking on SRE's substitution code, I stumbled upon a problem. to do a substitution, SRE needs to merge slices from the target strings and from the sub- stitution pattern. here's a simple example: re.sub( "(perl|tcl|java)", "python (not \\1)", "perl rules" ) contains a "substitution pattern" consisting of three parts: "python (not " (a slice from the substitution string) group 1 (a slice from the target string) ")" (a slice from the substitution string) PCRE implements this by doing the slicing (thus creating three new strings), and then doing a "join" by hand into a PyString buffer. this isn't very efficient, and it also doesn't work for uni- code strings. in other words, this needs to be fixed. but how? ... here's one proposal, off the top of my head: 1. introduce a PySliceListObject, which behaves like a simple sequence of strings, but stores them as slices. the type structure looks something like this: typedef struct { PyObject* string; int start; int end; } PySliceListItem; typedef struct { PyObject_VAR_HEAD PySliceListItem item[1]; } PySliceListObject; where start and end are normalized (0..len(string)) __len__ returns self->ob_size __getitem__ calls PySequence_GetSlice() PySliceListObjects are only used internally; they have no Python-level interface. 2. tweak string.join and unicode.join to look for PySliceListObject's, and have special code that copies slices directly from the source strings. (note that a slice list can still be used with any method that expects a sequence of strings, but at a cost) ... give the above, the substitution engine can now create a slice list by combining slices from the match object and the substitution object, and hand the result off to the string implementation; e.g: sep =3D PySequence_GetSlice(subst_string, 0, 0): result =3D PyObject_CallMethod(sep, "join", "O", slice_list) Py_DECREF(sep); (can anyone come up with something more elegant than the [0:0] slice?) comments? better ideas? From ping@lfw.org Sun Feb 27 12:22:16 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Sun, 27 Feb 2000 04:22:16 -0800 (PST) Subject: [Python-Dev] RFD: how to build strings from lots of slices? In-Reply-To: <003901bf811a$68136ea0$34aab5d4@hagrid> Message-ID: On Sun, 27 Feb 2000, Fredrik Lundh wrote: > here's one proposal, off the top of my head: > > 1. introduce a PySliceListObject, which behaves like a > simple sequence of strings, but stores them as slices. It occurred to me when i read this that *all* slices could be references within the original string, since strings are immutable. That is, s[x:y] could return a PyStringRefObject that behaves just like a string, but contains a length y - x and a pointer to &(s->ob_sval) + x instead of the character data itself. The creation of this PyStringRefObject would increment the reference count of s by 1. Perhaps this has been suggested before. The string methods could transparently work on such PyStringRefObjects, and any extensions that were polite enough to use only the Python API for playing with strings could continue to work fine; but things which directly manipulated the ob_sval field would break. Perhaps a possibility for Python 3K? Anyway -- as a solution for your particular problem, Fredrik, the PySliceListObject sounds like a good idea. -- ?!ng "To be human is to continually change. Your desire to remain as you are is what ultimately limits you." -- The Puppet Master, Ghost in the Shell From jcw@equi4.com Sun Feb 27 12:23:41 2000 From: jcw@equi4.com (Jean-Claude Wippler) Date: Sun, 27 Feb 2000 13:23:41 +0100 Subject: [Python-Dev] RFD: how to build strings from lots of slices? References: Message-ID: <38B9174D.63508240@equi4.com> Ka-Ping Yee wrote: > > On Sun, 27 Feb 2000, Fredrik Lundh wrote: > > here's one proposal, off the top of my head: > > > > 1. introduce a PySliceListObject, which behaves like a > > simple sequence of strings, but stores them as slices. > > It occurred to me when i read this that *all* slices > could be references within the original string, since > strings are immutable. That is, > > s[x:y] > > could return a PyStringRefObject that behaves just like > a string, but contains a length y - x and a pointer to > &(s->ob_sval) + x instead of the character data itself. > The creation of this PyStringRefObject would increment > the reference count of s by 1. The general approach is "cords" (in Hans Boehm's GC, garbage-collected), and "ropes" (in SGI's STL, http://www.sgi.com/Technology/STL/Rope.html, reference-counted). It's a great idea, IMO. Why create and copy strings all the time? -jcw From Fredrik Lundh" Message-ID: <001101bf811f$ec023480$34aab5d4@hagrid> Ka-Ping Yee wrote: > > 1. introduce a PySliceListObject, which behaves like a > > simple sequence of strings, but stores them as slices. >=20 > It occurred to me when i read this that *all* slices > could be references within the original string, since > strings are immutable. That is, >=20 > s[x:y] >=20 > could return a PyStringRefObject that behaves just like > a string, but contains a length y - x and a pointer to > &(s->ob_sval) + x instead of the character data itself. > The creation of this PyStringRefObject would increment > the reference count of s by 1. >=20 > Perhaps this has been suggested before. as an experiment, I actually implemented this for the original unicode string type (where "split" and "slice" returned slice references, not string copies). here are some arguments against it: a) bad memory behaviour if you slice small strings out of huge input strings -- which may surprise newbies. b) harder to interface to underlying C libraries -- the current string implementation guarantees that a Python string is also a C string (with a trailing null). personally, I don't care much about (a) (e.g. match objects already keep references to the input string, and if this is a real problem, you can always use a more elaborate data structure...). (b) is a bit harder to ignore, though. From guido@python.org Sun Feb 27 15:40:09 2000 From: guido@python.org (Guido van Rossum) Date: Sun, 27 Feb 2000 10:40:09 -0500 Subject: [Python-Dev] RFD: how to build strings from lots of slices? In-Reply-To: Your message of "Sun, 27 Feb 2000 13:23:41 +0100." <38B9174D.63508240@equi4.com> References: <38B9174D.63508240@equi4.com> Message-ID: <200002271540.KAA21750@eric.cnri.reston.va.us> > The general approach is "cords" (in Hans Boehm's GC, garbage-collected), > and "ropes" (in SGI's STL, http://www.sgi.com/Technology/STL/Rope.html, > reference-counted). > > It's a great idea, IMO. Why create and copy strings all the time? Each of these "improvements" slows things down in the common case. Believe me, I thought about this a lot when I designed Python's string object. ABC had an extremely complicated string implementation that used tricks like this and was proven to be asymptotically optimal. Unfortunately, the constant factor was large, and it was very slow for typical string ops. KISS, folks! --Guido van Rossum (home page: http://www.python.org/~guido/) From tismer@tismer.com Sun Feb 27 16:29:56 2000 From: tismer@tismer.com (Christian Tismer) Date: Sun, 27 Feb 2000 17:29:56 +0100 Subject: [Python-Dev] RFD: how to build strings from lots of slices? References: <001101bf811f$ec023480$34aab5d4@hagrid> Message-ID: <38B95104.F24934A@tismer.com> Fredrik Lundh wrote: > > Ka-Ping Yee wrote: > > > 1. introduce a PySliceListObject, which behaves like a > > > simple sequence of strings, but stores them as slices. > > > > It occurred to me when i read this that *all* slices > > could be references within the original string, since > > strings are immutable. That is, > > > > s[x:y] ... [b strings with NULL] > (b) is a bit harder to ignore, though. I think the explicit string slice object is a good thing, but changing [:] behavior is not. With special stuff made explicit, it is easy to write optimized code, while the average user is not affected. cheers - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From tismer@tismer.com Sun Feb 27 16:30:25 2000 From: tismer@tismer.com (Christian Tismer) Date: Sun, 27 Feb 2000 17:30:25 +0100 Subject: [Python-Dev] Safe destruction of recursive objects Message-ID: <38B95121.48769171@tismer.com> Hi Guido, When a user does the following with standard Python: tup = () for i in xrange(100000): tup = (tup, i) del tup # ka-boom He will get a core dump due to stack limitations. Recently, I changed Stackless Python to be safe for any recursive object built from lists, tuples, dictionaries, tracebacks and frames. The implementation is not Stackless Python dependant and very efficient (for my eyes at least). For efficiency, locality and minimum changes to five modules, it is implemented as two embracing macroes which are stuffed around the bodies of the deallocator methods, that makes just 3-4 lines of change for every module. (Well, the macro *can* be expanded if you like that more) I can submit patches, but please have a look at the example below, to save me the time in case you don't like it. It works great for SLP. cheers - chris -------------------------------------- Example of modified list deallocator: /* Methods */ static void list_dealloc(op) PyListObject *op; { int i; Py_TRASHCAN_SAFE_BEGIN(op) if (op->ob_item != NULL) { /* Do it backwards, for Christian Tismer. There's a simple test case where somehow this reduces thrashing when a *very* large list is created and immediately deleted. */ i = op->ob_size; while (--i >= 0) { Py_XDECREF(op->ob_item[i]); } free((ANY *)op->ob_item); } free((ANY *)op); Py_TRASHCAN_SAFE_END(op) } This is the original 1.5.2+ code, with two macro lines added. -------------------------------------- Here the macro code (which may of course be expanded) #define PyTrash_UNWIND_LEVEL 50 #define Py_TRASHCAN_SAFE_BEGIN(op) \ { \ ++_PyTrash_delete_nesting; \ if (_PyTrash_delete_nesting < PyTrash_UNWIND_LEVEL) { \ #define Py_TRASHCAN_SAFE_END(op) \ ;} \ else { \ if (!_PyTrash_delete_later) \ _PyTrash_delete_later = PyList_New(0); \ if (_PyTrash_delete_later) \ PyList_Append(_PyTrash_delete_later, (PyObject *)op); \ } \ --_PyTrash_delete_nesting; \ while (_PyTrash_delete_later && _PyTrash_delete_nesting <= 0) { \ PyObject *shredder = _PyTrash_delete_later; \ _PyTrash_delete_later = NULL; \ ++_PyTrash_delete_nesting; \ Py_DECREF(shredder); \ --_PyTrash_delete_nesting; \ } \ } \ extern DL_IMPORT(int) _PyTrash_delete_nesting; extern DL_IMPORT(PyObject *) _PyTrash_delete_later; -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From gstein@lyra.org Sun Feb 27 18:42:08 2000 From: gstein@lyra.org (Greg Stein) Date: Sun, 27 Feb 2000 10:42:08 -0800 (PST) Subject: [Python-Dev] RFD: how to build strings from lots of slices? In-Reply-To: Message-ID: On Sun, 27 Feb 2000, Ka-Ping Yee wrote: > On Sun, 27 Feb 2000, Fredrik Lundh wrote: > > here's one proposal, off the top of my head: > > > > 1. introduce a PySliceListObject, which behaves like a > > simple sequence of strings, but stores them as slices. > > It occurred to me when i read this that *all* slices > could be references within the original string, since > strings are immutable. That is, > > s[x:y] > > could return a PyStringRefObject that behaves just like > a string, but contains a length y - x and a pointer to > &(s->ob_sval) + x instead of the character data itself. > The creation of this PyStringRefObject would increment > the reference count of s by 1. This is exactly what the PyBufferObject does. I just documented the thing in api.tex a week ago or so. Regardless, the thing can operate exactly like a lightweight slice object. It it very similar at the Python level to a string, but it doesn't have the new string methods (yet) :-( If you want a temporary object for your slices (before recomposition with a "".join), then you should be able to use the buffer objects. [ unfortunately, the "".join method is nowhere near as optimal as it could be... it converts elems to string objects during the concatenation; it should have a variant that uses the buffer interface to precalculate the joined size, then use the interface to fetch the data ] Cheers, -g -- Greg Stein, http://www.lyra.org/ From tim_one@email.msn.com Sun Feb 27 23:19:48 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 27 Feb 2000 18:19:48 -0500 Subject: [Python-Dev] Safe destruction of recursive objects In-Reply-To: <38B95121.48769171@tismer.com> Message-ID: <000701bf8179$24d18500$f0a2143f@tim> [Christian Tismer] > ... > Recently, I changed Stackless Python to be safe > for any recursive object built from > lists, tuples, dictionaries, tracebacks and frames. > > The implementation is not Stackless Python dependant > and very efficient (for my eyes at least). > > For efficiency, locality and minimum changes to five > modules, it is implemented as two embracing macroes > which are stuffed around the bodies of the deallocator > methods, that makes just 3-4 lines of change for > every module. > (Well, the macro *can* be expanded if you like that more) It's a nice approach, but I'd rather see you put the bulk of the Py_TRASHCAN_SAFE_END macro into a real function or two, invoked from the macro. This code is only going to get hairier if GregS takes up his free-threading quest again. Like #define Py_TRASHCAN_SAFE_END(op) \ ;} \ else \ /* call a function to enqueue op, & maybe create list */ \ --_PyTrash_delete_nesting; \ if (_PyTrash_delete_later && _PyTrash_delete_nesting <= 0) \ /* call a function to (maybe) clean up */ \ } \ The first function only gets called when the nesting level hits (exactly) 50, and the 2nd function only when something got enqueued and the stack has completely unwound again. These should be infrequent enough that inline expansion doesn't buy much -- except the inability to set a useful breakpoint when the code fails to work . From tim_one@email.msn.com Sun Feb 27 23:19:52 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 27 Feb 2000 18:19:52 -0500 Subject: [Python-Dev] RFD: how to build strings from lots of slices? In-Reply-To: <001101bf811f$ec023480$34aab5d4@hagrid> Message-ID: <000801bf8179$267522e0$f0a2143f@tim> [/F, upon the reinvention of substring descriptors] > ... > a) bad memory behaviour if you slice small strings out > of huge input strings -- which may surprise newbies. Experts too. Dragon has gobs of code that copies little strings via loops in Java and C++, because Java's and MFC's descriptor-based string classes routinely keep a megabyte string alive after you've sliced out the 3 bytes <0.5 wink> you needed. Last year my group finally wrote its own string classes, to just copy the damn things. Performance improvement was significant (both space & time). Boehm's "cords"/"ropes" (he's the primary author of both pkgs JC mentioned) were specifically designed to support efficient random & repeated editing of giant mutable strings -- agree with Guido that it's overall major loss for pedestrian uses. Heck, why not implement strings as giant B-trees like the Tcl text widget does . > b) harder to interface to underlying C libraries -- the > current string implementation guarantees that a Python > string is also a C string (with a trailing null). c) For apps that use oodles of short strings, the space overhead of maintaining descriptors exceeds that of making copies. A buddy in Sun's Java development group tells me Java is despised for this by Major Players in the DB world; so don't be surprised if Java eventually drops the descriptor idea too (or, more Java-like, introduces 5 new flavors of strings <0.7 wink>). So there's no pure win here. Python's current scheme is at least predictable, and by everyone, with finite effort. Agree you have a particular good but limited use it for it, though, and Greg's suggestion of using buffer objects under the covers is almost certainly "the right" idea. From Fredrik Lundh" Message-ID: <006601bf817e$d9dc0240$34aab5d4@hagrid> Tim Peters wrote: > So there's no pure win here. Python's current scheme is at least > predictable, and by everyone, with finite effort. Agree you have a > particular good but limited use it for it, though, and Greg's = suggestion of > using buffer objects under the covers is almost certainly "the right" = idea. hmm. I'm not so sure about that... with Greg's scheme, SRE needs to create a list full of buffer objects, = while the SliceList scheme involves creating *one* object per substitution -- = and to create that object, SRE only needs to copy slots from the match and substitution objects, and bump the reference counts. (...and btw, using raw buffer objects to point into a set of strings of = mixed types doesn't sound right to me...) I think I'll stick to the SliceList model, with or without explicit = support in the string join methods. From mal@lemburg.com Mon Feb 28 10:06:42 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 28 Feb 2000 11:06:42 +0100 Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs References: <20000224170031.8245B1CE13@dinsdale.python.org> Message-ID: <38BA48B2.FB9D2662@lemburg.com> > Message: 7 > Date: Thu, 24 Feb 2000 10:23:05 -0500 (EST) > From: Guido van Rossum > To: python-checkins@python.org > Subject: [Python-checkins] CVS: python/dist/src/Objects listobject.c,2.63,2.64 > > Update of /projects/cvsroot/python/dist/src/Objects > In directory eric:/projects/python/develop/guido/src/Objects > > Modified Files: > listobject.c > Log Message: > Made all list methods use PyArg_ParseTuple(), for more accurate > diagnostics. > > *** INCOMPATIBLE CHANGE: This changes append(), remove(), index(), and > *** count() to require exactly one argument -- previously, multiple > *** arguments were silently assumed to be a tuple. Not sure about remove(), index() and count(), but the change to .append() will break *lots* of code ! -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From fredrik@pythonware.com Mon Feb 28 11:50:53 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Mon, 28 Feb 2000 12:50:53 +0100 Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs References: <20000224170031.8245B1CE13@dinsdale.python.org> <38BA48B2.FB9D2662@lemburg.com> Message-ID: <001901bf81e2$94c5db90$f29b12c2@secret.pythonware.com> MAL wrote: > Not sure about remove(), index() and count(), but the change > to .append() will break *lots* of code ! ouch. missed this checkin. I thought all the append fixes in the standard library were made to make it clearer that this use was deprecated. some random grepping through our code repositories makes me think this should be left for Py3K. From guido@python.org Mon Feb 28 14:45:43 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 09:45:43 -0500 Subject: [Python-Dev] Safe destruction of recursive objects In-Reply-To: Your message of "Sun, 27 Feb 2000 18:19:48 EST." <000701bf8179$24d18500$f0a2143f@tim> References: <000701bf8179$24d18500$f0a2143f@tim> Message-ID: <200002281445.JAA22881@eric.cnri.reston.va.us> I encourage Christian to submit a patch, taking Tim's modification into account. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Feb 28 14:47:53 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 09:47:53 -0500 Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs In-Reply-To: Your message of "Mon, 28 Feb 2000 11:06:42 +0100." <38BA48B2.FB9D2662@lemburg.com> References: <20000224170031.8245B1CE13@dinsdale.python.org> <38BA48B2.FB9D2662@lemburg.com> Message-ID: <200002281447.JAA22894@eric.cnri.reston.va.us> > > *** INCOMPATIBLE CHANGE: This changes append(), remove(), index(), and > > *** count() to require exactly one argument -- previously, multiple > > *** arguments were silently assumed to be a tuple. > > Not sure about remove(), index() and count(), but the change > to .append() will break *lots* of code ! Agreed. But 1.6 is as good a point to break it as any -- what's the point in putting this off? This isn't big enough to wait for 2.0. I could've done it in the 1.5.x series, but decided not to. --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Mon Feb 28 16:53:25 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 28 Feb 2000 17:53:25 +0100 Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs References: <20000224170031.8245B1CE13@dinsdale.python.org> <38BA48B2.FB9D2662@lemburg.com> <200002281447.JAA22894@eric.cnri.reston.va.us> Message-ID: <38BAA805.825C6002@lemburg.com> Guido van Rossum wrote: > > > > *** INCOMPATIBLE CHANGE: This changes append(), remove(), index(), and > > > *** count() to require exactly one argument -- previously, multiple > > > *** arguments were silently assumed to be a tuple. > > > > Not sure about remove(), index() and count(), but the change > > to .append() will break *lots* of code ! > > Agreed. But 1.6 is as good a point to break it as any -- what's the > point in putting this off? This isn't big enough to wait for 2.0. I > could've done it in the 1.5.x series, but decided not to. Hmm, I'd say it doesn't hurt leaving .append() as it is until 2.0. This small change will cause lots of trouble because it's hard to find (just like the x = 2L; print x[:-1] thingie, btw)... even though it's easy to fix. Note: the CVS cgi.py uses list.append(x,y) too. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From guido@python.org Mon Feb 28 16:58:40 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 11:58:40 -0500 Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs In-Reply-To: Your message of "Mon, 28 Feb 2000 17:53:25 +0100." <38BAA805.825C6002@lemburg.com> References: <20000224170031.8245B1CE13@dinsdale.python.org> <38BA48B2.FB9D2662@lemburg.com> <200002281447.JAA22894@eric.cnri.reston.va.us> <38BAA805.825C6002@lemburg.com> Message-ID: <200002281658.LAA26909@eric.cnri.reston.va.us> > > > > *** INCOMPATIBLE CHANGE: This changes append(), remove(), index(), and > > > > *** count() to require exactly one argument -- previously, multiple > > > > *** arguments were silently assumed to be a tuple. > > > > > > Not sure about remove(), index() and count(), but the change > > > to .append() will break *lots* of code ! > > > > Agreed. But 1.6 is as good a point to break it as any -- what's the > > point in putting this off? This isn't big enough to wait for 2.0. I > > could've done it in the 1.5.x series, but decided not to. > > Hmm, I'd say it doesn't hurt leaving .append() as it is until > 2.0. This small change will cause lots of trouble because it's > hard to find (just like the x = 2L; print x[:-1] thingie, btw)... > even though it's easy to fix. Sigh. It will be just as hard to fix later... I'd really like to take a hard line stance on this one. After all the docs are quite clear. > Note: the CVS cgi.py uses list.append(x,y) too. Already fixed in CVS. --Guido van Rossum (home page: http://www.python.org/~guido/) From Fredrik Lundh" <38BA48B2.FB9D2662@lemburg.com> <200002281447.JAA22894@eric.cnri.reston.va.us> <38BAA805.825C6002@lemburg.com> Message-ID: <01d501bf8210$3a398900$34aab5d4@hagrid> MAL: > Hmm, I'd say it doesn't hurt leaving .append() as it is until > 2.0. fwiw, I definitely agree. I've spotted too many places where this change cause a program to silently misbehave, rather than blow up (PIL's JPEG plugin is just one example...). I suggest adding an explicit warning in the 1.6 docs, and removing it in the first post-1.6 release. appendnanny.py, anyone? From guido@python.org Mon Feb 28 17:19:06 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 12:19:06 -0500 Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs In-Reply-To: Your message of "Mon, 28 Feb 2000 18:21:12 +0100." <01d501bf8210$3a398900$34aab5d4@hagrid> References: <20000224170031.8245B1CE13@dinsdale.python.org> <38BA48B2.FB9D2662@lemburg.com> <200002281447.JAA22894@eric.cnri.reston.va.us> <38BAA805.825C6002@lemburg.com> <01d501bf8210$3a398900$34aab5d4@hagrid> Message-ID: <200002281719.MAA26980@eric.cnri.reston.va.us> > MAL: > > Hmm, I'd say it doesn't hurt leaving .append() as it is until > > 2.0. /F: > fwiw, I definitely agree. I've spotted too many > places where this change cause a program to > silently misbehave, rather than blow up (PIL's > JPEG plugin is just one example...). Sigh... This smells of a too-inclusive except clause... Otherwise the program should have raised a clear exception. I suggest to fix it rather than whine... Am I responsible for everybody else's bad coding style? If it's not in the docs, where does everybody get the idea that this is legal? (The few cases in the std library are unlikely to be the only source; they were in pretty obscure places.) --Guido van Rossum (home page: http://www.python.org/~guido/) From akuchlin@mems-exchange.org Mon Feb 28 17:20:11 2000 From: akuchlin@mems-exchange.org (Andrew M. Kuchling) Date: Mon, 28 Feb 2000 12:20:11 -0500 (EST) Subject: [Python-Dev] Re: A warning switch? In-Reply-To: <01d501bf8210$3a398900$34aab5d4@hagrid> References: <20000224170031.8245B1CE13@dinsdale.python.org> <38BA48B2.FB9D2662@lemburg.com> <200002281447.JAA22894@eric.cnri.reston.va.us> <38BAA805.825C6002@lemburg.com> <01d501bf8210$3a398900$34aab5d4@hagrid> Message-ID: <14522.44619.590309.476@amarok.cnri.reston.va.us> Fredrik Lundh writes ( about .append(x,y) ): >I suggest adding an explicit warning in the 1.6 docs, >and removing it in the first post-1.6 release. What about adding a command-line switch for enabling warnings, as has been suggested long ago? The .append() change could then print a warning in 1.6alphas (and betas?), but still run, and be turned into an error later. -- A.M. Kuchling http://starship.python.net/crew/amk/ Well, that's a little thing -- the specification. -- Guido van Rossum at IPC7 From guido@python.org Mon Feb 28 17:35:12 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 12:35:12 -0500 Subject: [Python-Dev] Re: A warning switch? In-Reply-To: Your message of "Mon, 28 Feb 2000 12:20:11 EST." <14522.44619.590309.476@amarok.cnri.reston.va.us> References: <20000224170031.8245B1CE13@dinsdale.python.org> <38BA48B2.FB9D2662@lemburg.com> <200002281447.JAA22894@eric.cnri.reston.va.us> <38BAA805.825C6002@lemburg.com> <01d501bf8210$3a398900$34aab5d4@hagrid> <14522.44619.590309.476@amarok.cnri.reston.va.us> Message-ID: <200002281735.MAA27771@eric.cnri.reston.va.us> > Fredrik Lundh writes ( about .append(x,y) ): > >I suggest adding an explicit warning in the 1.6 docs, > >and removing it in the first post-1.6 release. > > What about adding a command-line switch for enabling warnings, as has > been suggested long ago? The .append() change could then print a > warning in 1.6alphas (and betas?), but still run, and be turned into > an error later. That's better. I propose that the warnings are normally on, and that there are flags to turn them off or thrn them into errors. Alternatively, the default should be warnings == errors, with flags to turn them into warnings or turn them off. Turning them off by default seems to defeat the purpose -- only the most zealous will use them. The warning messages should show a source file and line number and display the source code line if available, just like in a traceback; but no traceback info should be printed. Also, a warning should only be printed once per message/file/lineno combination, per Python invocation. A dictionary could take care of this. Anybody care to further specify and then code up such a mechanism? --Guido van Rossum (home page: http://www.python.org/~guido/) From da@ski.org Mon Feb 28 17:42:52 2000 From: da@ski.org (David Ascher) Date: Mon, 28 Feb 2000 09:42:52 -0800 Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs References: <20000224170031.8245B1CE13@dinsdale.python.org> <38BA48B2.FB9D2662@lemburg.com> <200002281447.JAA22894@eric.cnri.reston.va.us> <38BAA805.825C6002@lemburg.com> <01d501bf8210$3a398900$34aab5d4@hagrid> <200002281719.MAA26980@eric.cnri.reston.va.us> Message-ID: <027d01bf8213$3d2a6000$0100000a@ski.org> > Am I responsible for everybody else's bad coding style? > > If it's not in the docs, where does everybody get the idea that this > is legal? (The few cases in the std library are unlikely to be the > only source; they were in pretty obscure places.) I think you're responsible for a little bit of the confusion. Most people read the docs at the beginning to learn the basics, and then experiment with the interpreter. The fact that it worked is naturally interpreted by users as meaning that it should work. Very few of Python's semantics are accidental, so people "believe" the interpreter. Teaching people when ()'s are necessary in Python and when they're not is not a trivial task if you're talking to someone who'se never heard the difference between a parse tree and a pear tree. In my courses I typically say "()'s are sometimes optional" and leave it at that -- I expect the students' experience to guide them. That experience will be interaction-driven, not doc-driven. max/min() is another example, btw. What's the "right" way? To call them with N arguments where N > 1, or with a sequence argument? If I look it up in the doc, I can tell (it's the latter, folks) -- but it seems arbitrary. After all, the max/min calls/macros typically used in other languages require 2 arguments, so extending that to N arguments is conceptually at least as easy as shrinking it to a sequence argument. --david From guido@python.org Mon Feb 28 18:32:00 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 13:32:00 -0500 Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs In-Reply-To: Your message of "Mon, 28 Feb 2000 09:42:52 PST." <027d01bf8213$3d2a6000$0100000a@ski.org> References: <20000224170031.8245B1CE13@dinsdale.python.org> <38BA48B2.FB9D2662@lemburg.com> <200002281447.JAA22894@eric.cnri.reston.va.us> <38BAA805.825C6002@lemburg.com> <01d501bf8210$3a398900$34aab5d4@hagrid> <200002281719.MAA26980@eric.cnri.reston.va.us> <027d01bf8213$3d2a6000$0100000a@ski.org> Message-ID: <200002281832.NAA27907@eric.cnri.reston.va.us> > > Am I responsible for everybody else's bad coding style? > > > > If it's not in the docs, where does everybody get the idea that this > > is legal? (The few cases in the std library are unlikely to be the > > only source; they were in pretty obscure places.) > > I think you're responsible for a little bit of the confusion. Most > people read the docs at the beginning to learn the basics, and then > experiment with the interpreter. The fact that it worked is naturally > interpreted by users as meaning that it should work. Very few of > Python's semantics are accidental, so people "believe" the interpreter. It's a fair cop. > Teaching people when ()'s are necessary in Python and when they're not > is not a trivial task if you're talking to someone who'se never heard > the difference between a parse tree and a pear tree. In my courses I > typically say "()'s are sometimes optional" and leave it at that -- I > expect the students' experience to guide them. That experience will be > interaction-driven, not doc-driven. Yes -- and this is one reason why I want to fix append(). I should've fixed it years ago. > max/min() is another example, btw. What's the "right" way? To call > them with N arguments where N > 1, or with a sequence argument? If I > look it up in the doc, I can tell (it's the latter, folks) -- but it > seems arbitrary. After all, the max/min calls/macros typically used in > other languages require 2 arguments, so extending that to N arguments is > conceptually at least as easy as shrinking it to a sequence argument. This is different. Maybe the docs are wrong; I always intended for both max(a, b, ...) and max(seq) to be valid. (BTW, perhaps the __contains__ changes should be extended to __max__ and __min__? They share many of the same issues.) --Guido van Rossum (home page: http://www.python.org/~guido/) From pf@artcom-gmbh.de Mon Feb 28 18:09:37 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Mon, 28 Feb 2000 19:09:37 +0100 (MET) Subject: [Python-Dev] tracking memory consumption Message-ID: Hi! In PR#214 Martin v. Loewis suggests a sizeof function as result of request to python-help. I've followed the thread silently until now. On platforms with virtual memory subsystem this is usually a not an issue. On embedded systems and ancient OSes (like MS-Dos) it is often useful, if applications can estimate how much memory their data consumes. The sizeof() function proposed by Martin is only one possible approach I can think of. Another approach would be encapsulationg the 'malloc/free'-logic into a wrapper, that traces all allocations und deallocations in a special private 'usedmem' variable, which could be queried by a function sys.usedmem() returning an integer. Very often this is more convinient than a sizeof() function, because you don't need to embed the summing into a maybe complicated nested object data structure. Although 'usedmem' wouldn't return a precise measure, it is often sufficient to estimate and it also should be rather easy to implement. We have implemented this approach years ago here in a Modula-2 based system, where we however had one great advantage: the Modula-2 Storage.DEALLOCATE procedure has a second parameter giving the size of the data, which is missing from the signature of the C-library free() function. So a wrapper around 'free()' would have to use an additional hash or has to know something about the internals of the underlying malloc library. The former of course would hurt portability. Regards from Germany, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen) From ping@lfw.org Mon Feb 28 18:46:16 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Mon, 28 Feb 2000 12:46:16 -0600 (CST) Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs In-Reply-To: <200002281832.NAA27907@eric.cnri.reston.va.us> Message-ID: On Mon, 28 Feb 2000, Guido van Rossum wrote: > > This is different. Maybe the docs are wrong; I always intended for > both max(a, b, ...) and max(seq) to be valid. I suppose in this case it's clear what you mean just from the number of arguments. But there is a potential surprise if someone who expects to have to say max(a, b, ...) then writes apply(max, tuple) and tuple turns out to only have one element. (I don't think i've ever realized that we could use min() or max() on a sequence.) > (BTW, perhaps the __contains__ changes should be extended to __max__ > and __min__? They share many of the same issues.) Indeed -- but then who do you trust? The first element of the sequence? Is it acceptable for max(a, b, c, d) to read as "a, please tell me which is the maximum among yourself, b, c, and d" ? Does 'a' then have to take care of the type-comparison logic for consistency with everything else? What if 'a' happens to be a built-in type but 'c' is a user-defined instance? -- ?!ng From guido@python.org Mon Feb 28 18:54:34 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 13:54:34 -0500 Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs In-Reply-To: Your message of "Mon, 28 Feb 2000 12:46:16 CST." References: Message-ID: <200002281854.NAA28102@eric.cnri.reston.va.us> > On Mon, 28 Feb 2000, Guido van Rossum wrote: > > > > This is different. Maybe the docs are wrong; I always intended for > > both max(a, b, ...) and max(seq) to be valid. (BTW, I was wrong about the docs. The docs explain quite clearly that max(a) is different from max(a, b, ...). Learning Python and Python Essential Reference also document both forms.) > I suppose in this case it's clear what you mean just from the > number of arguments. But there is a potential surprise if someone > who expects to have to say max(a, b, ...) then writes > > apply(max, tuple) > > and tuple turns out to only have one element. (I don't think > i've ever realized that we could use min() or max() on a sequence.) Yes, but there simply isn't any need to do this, so it won't occur in practice. > > (BTW, perhaps the __contains__ changes should be extended to __max__ > > and __min__? They share many of the same issues.) > > Indeed -- but then who do you trust? The first element of the > sequence? Is it acceptable for > > max(a, b, c, d) > > to read as > > "a, please tell me which is the maximum among yourself, b, c, and d" > > ? Does 'a' then have to take care of the type-comparison logic > for consistency with everything else? What if 'a' happens to be > a built-in type but 'c' is a user-defined instance? No, that's not what I meant. __min__ and __max__ would be methods of sequences. The min() and max() functions would catch the special case of multiple arguments and translate to min((a, b, ...)) etc. before looking for __min__. --Guido van Rossum (home page: http://www.python.org/~guido/) From da@ski.org Mon Feb 28 18:57:25 2000 From: da@ski.org (David Ascher) Date: Mon, 28 Feb 2000 10:57:25 -0800 Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs References: <20000224170031.8245B1CE13@dinsdale.python.org> <38BA48B2.FB9D2662@lemburg.com> <200002281447.JAA22894@eric.cnri.reston.va.us> <38BAA805.825C6002@lemburg.com> <01d501bf8210$3a398900$34aab5d4@hagrid> <200002281719.MAA26980@eric.cnri.reston.va.us> <027d01bf8213$3d2a6000$0100000a@ski.org> <200002281832.NAA27907@eric.cnri.reston.va.us> Message-ID: <02d901bf821d$a74395b0$0100000a@ski.org> From: Guido van Rossum > (BTW, perhaps the __contains__ changes should be extended to __max__ > and __min__? They share many of the same issues.) I suppose so, although I think the uses of a smart __contains__ are much more frequent than the uses of a smart __max__. On the other hand, I do think that it might be nice to have that sort of hook in the rich array world... On the topic of rich comparisons, I think I have a complete game plan in my head, if not in code. I had to do some figuring out of the mods to the compilation phase to allow short-circuiting with minimal performance impact, as you and Jim H. discussed on the list way back when. But, as you can guess, I'm a bit short on time. [For those of you who don't know, I have a 4-day old daughter at home, and, more relevantly, she has an older brother =)]. I would really like a bit more discussion and decision on coercions before finalizing the rich comparison patches, as I think a coherent coercion strategy will help simplify the patches. Marc-Andre is short on time due to the Unicode stuff, and he posted a teaser to spark some discussion, which got no response at all. I'm not surprised, it's an ugly problem. Did anyone have thoughts that they'd want to share on the topic? --david From guido@python.org Mon Feb 28 19:04:55 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 14:04:55 -0500 Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs In-Reply-To: Your message of "Mon, 28 Feb 2000 10:57:25 PST." <02d901bf821d$a74395b0$0100000a@ski.org> References: <20000224170031.8245B1CE13@dinsdale.python.org> <38BA48B2.FB9D2662@lemburg.com> <200002281447.JAA22894@eric.cnri.reston.va.us> <38BAA805.825C6002@lemburg.com> <01d501bf8210$3a398900$34aab5d4@hagrid> <200002281719.MAA26980@eric.cnri.reston.va.us> <027d01bf8213$3d2a6000$0100000a@ski.org> <200002281832.NAA27907@eric.cnri.reston.va.us> <02d901bf821d$a74395b0$0100000a@ski.org> Message-ID: <200002281904.OAA28130@eric.cnri.reston.va.us> > > (BTW, perhaps the __contains__ changes should be extended to __max__ > > and __min__? They share many of the same issues.) > > I suppose so, although I think the uses of a smart __contains__ are much > more frequent than the uses of a smart __max__. That's probably a reflection of the fact that min/max are less frequently used than 'in'. (Which is reflected in making min/max "mere" functions while 'in' is a built-in operator.) I was thinking of any sequence representation that keeps its items sorted (like the old ABC "lists"). Of course, if you're using a hash table, 'in' is trivially answered, but min/max aren't. > On the other hand, I do think that it might be nice to have that sort of > hook in the rich array world... Really? The min/max functions already do all their looping in C. > On the topic of rich comparisons, I think I have a complete game plan in > my head, if not in code. I had to do some figuring out of the mods to > the compilation phase to allow short-circuiting with minimal performance > impact, as you and Jim H. discussed on the list way back when. But, as > you can guess, I'm a bit short on time. [For those of you who don't > know, I have a 4-day old daughter at home, and, more relevantly, she has > an older brother =)]. [I guess you get to worry about the older brother while your wife takes care of the newborn? :-)] > I would really like a bit more discussion and decision on coercions > before finalizing the rich comparison patches, as I think a coherent > coercion strategy will help simplify the patches. Marc-Andre is short on > time due to the Unicode stuff, and he posted a teaser to spark some > discussion, which got no response at all. I'm not surprised, it's an > ugly problem. Did anyone have thoughts that they'd want to share on the > topic? I have no children [yet], but Python is my baby -- and I'm way overcommitted to other Python projects. :-( --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Mon Feb 28 18:06:02 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 28 Feb 2000 19:06:02 +0100 Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs References: <20000224170031.8245B1CE13@dinsdale.python.org> <38BA48B2.FB9D2662@lemburg.com> <200002281447.JAA22894@eric.cnri.reston.va.us> <38BAA805.825C6002@lemburg.com> <01d501bf8210$3a398900$34aab5d4@hagrid> <200002281719.MAA26980@eric.cnri.reston.va.us> Message-ID: <38BAB90A.789407B3@lemburg.com> Guido van Rossum wrote: > > > MAL: > > > Hmm, I'd say it doesn't hurt leaving .append() as it is until > > > 2.0. > > /F: > > fwiw, I definitely agree. I've spotted too many > > places where this change cause a program to > > silently misbehave, rather than blow up (PIL's > > JPEG plugin is just one example...). > > Sigh... I think you've got a wrong impression here: this is not so much a design question, it's a timing problem: two months are not enough to get all our software ready for 1.6 ... -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Mon Feb 28 18:15:15 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 28 Feb 2000 19:15:15 +0100 Subject: [Python-Dev] RFD: how to build strings from lots of slices? References: <38B7AD54.57498D82@elisra.com> <200002261620.LAA21156@eric.cnri.reston.va.us> <003901bf811a$68136ea0$34aab5d4@hagrid> Message-ID: <38BABB33.11324515@lemburg.com> Fredrik Lundh wrote: > > when hacking on SRE's substitution code, I stumbled > upon a problem. to do a substitution, SRE needs to > merge slices from the target strings and from the sub- > stitution pattern. > > here's a simple example: > > re.sub( > "(perl|tcl|java)", > "python (not \\1)", > "perl rules" > ) > > contains a "substitution pattern" consisting of three > parts: > > "python (not " (a slice from the substitution string) > group 1 (a slice from the target string) > ")" (a slice from the substitution string) > > PCRE implements this by doing the slicing (thus creating > three new strings), and then doing a "join" by hand into > a PyString buffer. > > this isn't very efficient, and it also doesn't work for uni- > code strings. Why not ? The Unicode implementation has an API PyUnicode_Join() which does eaxctly this: extern DL_IMPORT(PyObject*) PyUnicode_Join( PyObject *separator, /* Separator string */ PyObject *seq /* Sequence object */ ); Note that the PyUnicode_Join() API takes a sequence of Unicode objects, strings or objects providing the charbuf interface, coerces all of these into a Unicode object and then does the joining. There is also a _PyUnicode_Resize() API. It is currently not exported though... but that's easy to fix. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From da@ski.org Mon Feb 28 19:57:17 2000 From: da@ski.org (David Ascher) Date: Mon, 28 Feb 2000 11:57:17 -0800 Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs References: <20000224170031.8245B1CE13@dinsdale.python.org> <38BA48B2.FB9D2662@lemburg.com> <200002281447.JAA22894@eric.cnri.reston.va.us> <38BAA805.825C6002@lemburg.com> <01d501bf8210$3a398900$34aab5d4@hagrid> <200002281719.MAA26980@eric.cnri.reston.va.us> <027d01bf8213$3d2a6000$0100000a@ski.org> <200002281832.NAA27907@eric.cnri.reston.va.us> <02d901bf821d$a74395b0$0100000a@ski.org> <200002281904.OAA28130@eric.cnri.reston.va.us> Message-ID: <033a01bf8226$04365160$0100000a@ski.org> > > On the other hand, I do think that it might be nice to have that sort of > > hook in the rich array world... > > Really? The min/max functions already do all their looping in C. Right, but it might make sense to define min() on an array to mean something different than what min-the-builtin could guess from whatever __lt__ returns -- In fact, IIRC, min() on an array which implemented rich comparisons would raise an exception. I don't want to specify the semantics now (arrays are weird sequences), I just appreciate the hook. The semantics would probably depend on the flavor of array one wanted to use. > [I guess you get to worry about the older brother while your wife > takes care of the newborn? :-)] Got it it one. --david From guido@python.org Mon Feb 28 20:01:30 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Feb 2000 15:01:30 -0500 Subject: [Python-Dev] Re: Python-checkins digest, Vol 1 #370 - 8 msgs In-Reply-To: Your message of "Mon, 28 Feb 2000 11:57:17 PST." <033a01bf8226$04365160$0100000a@ski.org> References: <20000224170031.8245B1CE13@dinsdale.python.org> <38BA48B2.FB9D2662@lemburg.com> <200002281447.JAA22894@eric.cnri.reston.va.us> <38BAA805.825C6002@lemburg.com> <01d501bf8210$3a398900$34aab5d4@hagrid> <200002281719.MAA26980@eric.cnri.reston.va.us> <027d01bf8213$3d2a6000$0100000a@ski.org> <200002281832.NAA27907@eric.cnri.reston.va.us> <02d901bf821d$a74395b0$0100000a@ski.org> <200002281904.OAA28130@eric.cnri.reston.va.us> <033a01bf8226$04365160$0100000a@ski.org> Message-ID: <200002282001.PAA03616@eric.cnri.reston.va.us> [DA] > > > On the other hand, I do think that it might be nice to have that > > > sort of hook in the rich array world... [me] > > Really? The min/max functions already do all their looping in C. [DA] > Right, but it might make sense to define min() on an array to mean > something different than what min-the-builtin could guess from whatever > __lt__ returns -- In fact, IIRC, min() on an array which implemented > rich comparisons would raise an exception. I don't want to specify the > semantics now (arrays are weird sequences), I just appreciate the hook. > The semantics would probably depend on the flavor of array one wanted to > use. You're right. I bet Moshe is already coding... (Tip: you could reuse the same flag bit and simply rename it a bit, rather than having separate flag bits. The binary compatibility requirement is only between released versions, not between CVS versions.) --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Mon Feb 28 22:02:17 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 28 Feb 2000 23:02:17 +0100 Subject: [Python-Dev] Unicode codec names Message-ID: <38BAF069.D0287157@lemburg.com> Hi everybody, As you may have noticed, the latest Unicode snapshot contains a large number of new codecs. Most of them are based on a generic mapping codec which makes adding new codecs a very simple (even automated) task. I've gotten some feedback on the compatibility of the JPython Unicode implementation (actually the underlying Java one) and the new CPython code. Finn Bock mentioned that Java uses a slightly different naming scheme and also has some differences in the code-page-to-Unicode mappings. * Could someone provide a list of all default code pages and other encodings that Java supports ? It would be ideal to provide the same set for CPython, IMHO. So far I've got these encodings: cp852.py iso_8859_5.py cp855.py iso_8859_6.py ascii.py cp856.py iso_8859_7.py charmap.py cp857.py iso_8859_8.py cp037.py cp860.py iso_8859_9.py cp1006.py cp861.py koi8_r.py cp1250.py cp862.py latin_1.py cp1251.py cp863.py mac_cyrillic.py cp1252.py cp864.py mac_greek.py cp1253.py cp865.py mac_iceland.py cp1254.py cp866.py mac_latin2.py cp1255.py cp869.py mac_roman.py cp1256.py cp874.py mac_turkish.py cp1257.py iso_8859_10.py raw_unicode_escape.py cp1258.py iso_8859_13.py unicode_escape.py cp424.py iso_8859_14.py unicode_internal.py cp437.py iso_8859_15.py utf_16.py cp737.py iso_8859_2.py utf_16_be.py cp775.py iso_8859_3.py utf_16_le.py cp850.py iso_8859_4.py utf_8.py Encoding names map to these module names in the following way: 1. convert all hyphens to underscores 2. convert all chars to lowercase 3. apply an alias dictionary to the resulting name Thus u"abc".encode('KOI8-R') and u"abc".encode('koi8_r') will result in the same codec being used. * There's also another issue: code pages with names cpXXXX come from two sources: IBM and MS. Unfortunately, some of these pages don't match even though they carry the same name. Could someone verify whether the included maps work on Windows, DOS and Mac platforms as intended ? (Finn reported some divergence between the Java view of things and the maps I created from the ftp.unicode.org site ones.) Thanks, -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From gstein@lyra.org Mon Feb 28 22:55:24 2000 From: gstein@lyra.org (Greg Stein) Date: Mon, 28 Feb 2000 14:55:24 -0800 (PST) Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib ConfigParser.py,1.14,1.15 In-Reply-To: <200002282059.PAA24383@weyr.cnri.reston.va.us> Message-ID: On Mon, 28 Feb 2000, Fred L. Drake wrote: >... > Index: ConfigParser.py > =================================================================== > RCS file: /projects/cvsroot/python/dist/src/Lib/ConfigParser.py,v > retrieving revision 1.14 > retrieving revision 1.15 > diff -C2 -r1.14 -r1.15 > *** ConfigParser.py 1999/10/12 16:12:48 1.14 > --- ConfigParser.py 2000/02/28 20:59:03 1.15 > *************** > *** 293,302 **** > SECTCRE = re.compile( > r'\[' # [ > ! r'(?P
[-\w]+)' # `-', `_' or any alphanum > r'\]' # ] > ) > OPTCRE = re.compile( > ! r'(?P