From guido@zope.com Wed Aug 1 00:12:32 2001 From: guido@zope.com (Guido van Rossum) Date: Tue, 31 Jul 2001 19:12:32 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Your message of "Tue, 31 Jul 2001 13:31:01 MDT." <15207.1909.395000.123189@gargle.gargle.HOWL> References: <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1909.395000.123189@gargle.gargle.HOWL> Message-ID: <200107312312.TAA17483@cj20424-a.reston1.va.home.com> > Guido van Rossum writes: > > Yeah, but the runtime could offer a choice of data types -- for Python > > code the constants table would contain Python ints and strings etc., for > > Perl code it would contain Perl string-number objects. Maybe. > > A perl6 value have a vtable, essentially an array of function pointers > which comprises the standard operations on that value. I talked to > Dan (the perl6 internals guy, dan@sidhe.org) about an impedence > mismatch between Perl and Python data types, and he pointed out that > you can have Perl values and Python values, each with their own > semantics, simply by having separate vtables (and thus separate > functions to implement the behaviour of those types). Code can work > with either type because the type carries around (in its vtable) the > knowledge of how it should behave. The vtable looks a lot like Python's type object. Is Perl's vtable an object in its own right? In Python, it is: in C, obj->ob_type is the type object, in Python type(obj) or (in 2.2) obj.__class__ gives the type object. > Feel free to grill Dan about these things if you want. No thanks. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@zope.com Wed Aug 1 00:15:03 2001 From: guido@zope.com (Guido van Rossum) Date: Tue, 31 Jul 2001 19:15:03 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Your message of "Tue, 31 Jul 2001 04:14:43 EDT." <20010731041443.A26075@thyrsus.com> References: <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1393.232974.785433@beluga.mojam.com> <20010731041443.A26075@thyrsus.com> Message-ID: <200107312315.TAA17507@cj20424-a.reston1.va.home.com> > In the conservative version of this vision, Perl and Python have > different runtimes dynamically linked to an instance of the same > opcode interpreter. Memory allocation/GC and scheduling/threading are > handled inside the opcode interpreter but the OS and environment > binding is (mostly) in the libraries. Hm, the Python bytecode interpreter doesn't do the memory allocation/GC, and only does the scheduling/threading indirectly (by releasing and immediately reacquiring the global interpreter lock). > Things Python would bring to this party: our serious-cool GC, our > C extension/embedding system (*much* nicer than XS). Things Perl would > bring: blazingly fast regexps, taint, flexitypes, references. I'm not sure I want references, and I don't know what flexitypes are. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@zope.com Wed Aug 1 00:26:13 2001 From: guido@zope.com (Guido van Rossum) Date: Tue, 31 Jul 2001 19:26:13 -0400 Subject: [Python-Dev] Revised decimal type PEP In-Reply-To: Your message of "Tue, 31 Jul 2001 03:27:11 EDT." <01073103271101.02004@fermi.eeel.nist.gov> References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073023115207.02466@fermi.eeel.nist.gov> <3B66DE8C.C9C62012@lemburg.com> <01073103271101.02004@fermi.eeel.nist.gov> Message-ID: <200107312326.TAA17540@cj20424-a.reston1.va.home.com> > Also, why do you consider a float to be a "larger" value type than decimal? > Do you mean that a float is less precise? (Warning: I think the following is a sound model, but I'm still practicing how to explain it right.) I have this ordering of the types in mind: int/long < decimal < rational < float < complex \---------------------------/ \-------------/ exact inexact This is different from the Scheme numeric "tower" -- I no longer agree with the Scheme model any more. The ordering is only to determine what happens on mixed arithmetic: the result has the rightmost type in the diagram (or a type further on the right in some cases). The ints are a subset of the decimal numbers, and the decimal numbers (in this view) are a subset of the rational numbers. Ints and decimals aren't closed under division -- the result of division on these (in general) is a rational. While the exact values of floats are a subset of the rationals, the inexactness property (which I give all floats) makes that each float stands for an infinite set of numbers *including* the exact value. When a binary operation involves an exact and an inexact operand, the result is inexact. Tim's "numeric context" contains a bunch of flags controlling detailed behavior of numeric operations. It could specify that mixing exact and inexact numbers is illegal, and that would be Michael's pedantic mode. It could also specify warnings. (I would never call a mode that issues warnings "safe" :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@zope.com Wed Aug 1 00:32:07 2001 From: guido@zope.com (Guido van Rossum) Date: Tue, 31 Jul 2001 19:32:07 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Your message of "Tue, 31 Jul 2001 05:18:08 EDT." <20010731051808.A27187@thyrsus.com> References: <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1909.395000.123189@gargle.gargle.HOWL> <20010731225445.B626@xs4all.nl> <20010731051808.A27187@thyrsus.com> Message-ID: <200107312332.TAA17611@cj20424-a.reston1.va.home.com> > Making this work is going to take strong leadership from Larry and > Guido. We're laying some of the technical groundwork now. More will > have to be done. But I think the key moment, if it happens, will be > the one at which Guido and Larry, each flanked by their three or four > chief lieutenants, shake hands for the cameras and issue a joint ukase > to their tribes. Larry, are you following this? What's *your* view? --Guido van Rossum (home page: http://www.python.org/~guido/) From dan@sidhe.org Wed Aug 1 00:34:00 2001 From: dan@sidhe.org (Dan Sugalski) Date: Tue, 31 Jul 2001 19:34:00 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <200107312315.TAA17507@cj20424-a.reston1.va.home.com> References: <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1393.232974.785433@beluga.mojam.com> <20010731041443.A26075@thyrsus.com> Message-ID: <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48> At 07:15 PM 7/31/2001 -0400, Guido van Rossum wrote: > > In the conservative version of this vision, Perl and Python have > > different runtimes dynamically linked to an instance of the same > > opcode interpreter. Memory allocation/GC and scheduling/threading are > > handled inside the opcode interpreter but the OS and environment > > binding is (mostly) in the libraries. > >Hm, the Python bytecode interpreter doesn't do the memory >allocation/GC, and only does the scheduling/threading indirectly (by >releasing and immediately reacquiring the global interpreter lock). The Parrot interpreter does (or will at least, as there's not much code behind it yet) handle GC and intercept memory allocation, at least if you want to use managed memory. (Though that's only of interest to people who are writing C/C++/Fortran/Whatever extensions to the interpreter) GC & object destruction are going to be dealt with separately by the interpreter, FWIW. Which might not be much. As for threading, well, that's where things get interesting. Perl's tried it two ways (multiple threads in the same interpreter, and one thread per interpreter, with cloned interpreters) both of which aren't very good. And the global lock thing's not that keen either. The plan for parrot is to allow multiple interpreters to run simultaneously, with shared data protected on access time. (And *only* shared data protected, so there's no penalty to access non-shared data) Along with a scheme to allow only partial (or no) visibility of the spawning thread's variables. We plan on basing the secure sandbox scheme around this as well. > > Things Python would bring to this party: our serious-cool GC, our > > C extension/embedding system (*much* nicer than XS). Things Perl would > > bring: blazingly fast regexps, taint, flexitypes, references. > >I'm not sure I want references, You certainly don't have to have them. The parser doesn't have to necessarily expose all the features of the interpreter. What you'd expose to the programmer would depend on the language grammar--if you can't write valid Python code to take references, well, you don't get them. :) > and I don't know what flexitypes are. I presume this is perl's "I'll make it any type I darned well need" basic scalar, or possibly it's auto-resizing arrays and hashes. Neither of which you have to have either. Since that behaviour's a function of the type of a variable, we'd just not give python variables perl variable types. (Or the vtables that go with them) Now, what happens when a Python routine is passed a perl variable is an interesting question. Personally I'd presume that it Did The Right Thing, but I'm not 100% sure what that is. Probably maintain its core behavior. (So if I passed in a Magic Morphing Perl Scalar to a Python routine that needed an integer value, it'd get one, but if I tried to use a Python integer in string context in a perl sub, I'd get yelled at. (Assuming that's the correct behaviour, of course. If not, well, it'd do something else)) Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From paulp@ActiveState.com Wed Aug 1 00:42:57 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 31 Jul 2001 16:42:57 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? References: <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1393.232974.785433@beluga.mojam.com> <20010731041443.A26075@thyrsus.com> <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48> Message-ID: <3B674281.6962A99D@ActiveState.com> Dan Sugalski wrote: > >... > > As for threading, well, that's where things get interesting. Perl's tried > it two ways (multiple threads in the same interpreter, and one thread per > interpreter, with cloned interpreters) both of which aren't very good. And > the global lock thing's not that keen either. What is the downside of the global lock on the average single processor machine? I tend to think that the "default" threading model should allow simple and easy, everything-shared multi-threading on ordinary machines. Having a multi-processor-friendly advanced mode is a great extension for the wizards. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From aahz@rahul.net Wed Aug 1 00:55:26 2001 From: aahz@rahul.net (Aahz Maruch) Date: Tue, 31 Jul 2001 16:55:26 -0700 (PDT) Subject: [Python-Dev] Revised decimal type PEP In-Reply-To: <200107312326.TAA17540@cj20424-a.reston1.va.home.com> from "Guido van Rossum" at Jul 31, 2001 07:26:13 PM Message-ID: <20010731235526.8E8ED99C82@waltz.rahul.net> Guido van Rossum wrote: > > int/long < decimal < rational < float < complex > \---------------------------/ \-------------/ > exact inexact Note that in Cowlishaw's implementation of decimal numbers, decimals are *not* exact. Truncation (rounding), overflow, and underflow errors can occur under addition, subtraction, and multiplication. It's trivial to set them to be unbounded, but then Cowlishaw provides no mechanism for determining the truncation of division. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From aahz@rahul.net Wed Aug 1 01:04:52 2001 From: aahz@rahul.net (Aahz Maruch) Date: Tue, 31 Jul 2001 17:04:52 -0700 (PDT) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48> from "Dan Sugalski" at Jul 31, 2001 07:34:00 PM Message-ID: <20010801000452.560D999C82@waltz.rahul.net> Dan Sugalski wrote: > > As for threading, well, that's where things get interesting. Perl's > tried it two ways (multiple threads in the same interpreter, and one > thread per interpreter, with cloned interpreters) both of which aren't > very good. And the global lock thing's not that keen either. The plan > for parrot is to allow multiple interpreters to run simultaneously, > with shared data protected on access time. (And *only* shared data > protected, so there's no penalty to access non-shared data) Along with > a scheme to allow only partial (or no) visibility of the spawning > thread's variables. We plan on basing the secure sandbox scheme around > this as well. This is where calling what Python uses "variables" gets you into real problems. Many people think that the terms for "variable" and "reference" should be "name" and "binding"; it makes things far less confusing. See, Python objects are *global*. Period. No ifs, ands, or buts. Any object can be shared at any time. More than that, most of the time Python threads are bound to class instances, so causing another thread to get a binding to an object is as simple as setting an attribute of that thread's instance. And, as Paul pointed out, the global interpreter lock only kills you if you're running SMP in the first place. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From DavidA@ActiveState.com Wed Aug 1 01:03:41 2001 From: DavidA@ActiveState.com (David Ascher) Date: Tue, 31 Jul 2001 17:03:41 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? References: <20010731223259.A626@xs4all.nl> Message-ID: <3B67475D.76FDBE4F@ActiveState.com> Thomas Wouters wrote: > The Perl version of the compiled code could of course be > > PUSH "1" > COERCE_INT > PUSH "2" > COERCE_INT > BINARY_ADD > > for Perl's > > "1" + "2" > > and > > PUSH "1" > PUSH "2" > BINARY_ADD > > for it's > > "1" . "2" I hate to bring up the fact that in Perl6 the . will be used for attribute access (or at least method dereferencing), and some other operator will be used for string concatenation =). =) From greg@cosc.canterbury.ac.nz Wed Aug 1 01:17:14 2001 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 01 Aug 2001 12:17:14 +1200 (NZST) Subject: [Python-Dev] Revised decimal type PEP In-Reply-To: <20010731235526.8E8ED99C82@waltz.rahul.net> Message-ID: <200108010017.MAA00798@s454.cosc.canterbury.ac.nz> aahz@rahul.net (Aahz Maruch): > Truncation (rounding), overflow, and underflow errors can > occur under addition, subtraction, and multiplication. It's trivial to > set them to be unbounded, but then Cowlishaw provides no mechanism for > determining the truncation of division. If you allow for the representation of repeating parts in your unbounded decimals, they could be closed under division. (I think -- does the division of one repeating decimal by another always lead to a third repeating decimal? Yes, it must, because every rational can be expressed as a repeating decimal and vice versa, IIRC. Hmmm, that means we'd just be implementing rationals another way...) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From dan@sidhe.org Wed Aug 1 01:13:33 2001 From: dan@sidhe.org (Dan Sugalski) Date: Tue, 31 Jul 2001 20:13:33 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <3B674281.6962A99D@ActiveState.com> References: <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1393.232974.785433@beluga.mojam.com> <20010731041443.A26075@thyrsus.com> <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48> Message-ID: <5.1.0.14.0.20010731200907.02747f58@24.8.96.48> At 04:42 PM 7/31/2001 -0700, Paul Prescod wrote: >Dan Sugalski wrote: > > > >... > > > > As for threading, well, that's where things get interesting. Perl's tried > > it two ways (multiple threads in the same interpreter, and one thread per > > interpreter, with cloned interpreters) both of which aren't very good. And > > the global lock thing's not that keen either. > >What is the downside of the global lock on the average single processor >machine? I tend to think that the "default" threading model should allow >simple and easy, everything-shared multi-threading on ordinary machines. >Having a multi-processor-friendly advanced mode is a great extension for >the wizards. If you hold the lock during an I/O operation, you'll lose time you could have otherwise used. Getting and releasing a global lock frequently also costs performance you might otherwise have used in other places. Mutex releases require memory coherency, which will force your CPU to flush any pending writes that might be hanging about, which will tend to drop it's efficiency, especially on heavily out-of-order machines like the Alpha. Also, that is a zillion and a half mutex aquisition and releases, most of which you probably have no need of. Even cutting out, say 5-10% of them will be noticeable. (I do work on SMP machines as a rule, so I am a little biased against things that single-thread me when I don't need it--what's the point of 500% idle time?) Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From dan@sidhe.org Wed Aug 1 01:19:41 2001 From: dan@sidhe.org (Dan Sugalski) Date: Tue, 31 Jul 2001 20:19:41 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <20010801000452.560D999C82@waltz.rahul.net> References: <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48> Message-ID: <5.1.0.14.0.20010731201359.02719598@24.8.96.48> At 05:04 PM 7/31/2001 -0700, Aahz Maruch wrote: >Dan Sugalski wrote: > > > > As for threading, well, that's where things get interesting. Perl's > > tried it two ways (multiple threads in the same interpreter, and one > > thread per interpreter, with cloned interpreters) both of which aren't > > very good. And the global lock thing's not that keen either. The plan > > for parrot is to allow multiple interpreters to run simultaneously, > > with shared data protected on access time. (And *only* shared data > > protected, so there's no penalty to access non-shared data) Along with > > a scheme to allow only partial (or no) visibility of the spawning > > thread's variables. We plan on basing the secure sandbox scheme around > > this as well. > >This is where calling what Python uses "variables" gets you into real >problems. Many people think that the terms for "variable" and >"reference" should be "name" and "binding"; it makes things far less >confusing. I'm not sure that's the appropriate translation for what perl considers variables and references, but that's OK. >See, Python objects are *global*. Period. No ifs, ands, or >buts. Is this global in the "all variables are in one big pool and can be looked up by name", more or less the equivalent of externing all your C variables, or global in the "Any thread might be able to see it" sense? (I thought Python had lexically scoped variables, but I might be wrong here. I'm an internals guy--the actual language being run is just an implementation detail... :) If it's "All objects are visible to all threads" thing, that's fine, you can still handle threading with some care. Perl's done it wrong once, and I know how to do it much less wrong with Parrot. (It's not, strictly speaking, possible to do it right without the programmer taking an active hand) >Any object can be shared at any time. More than that, most of >the time Python threads are bound to class instances, so causing another >thread to get a binding to an object is as simple as setting an >attribute of that thread's instance. Ah. Doable, and not much of a problem. It'll require tweaking the thread model a touch (for which I appreciate you bringing this up) but better now than later. >And, as Paul pointed out, the global interpreter lock only kills you if >you're running SMP in the first place. Yep, well, I do. :) Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From paulp@ActiveState.com Wed Aug 1 01:27:48 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 31 Jul 2001 17:27:48 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? References: <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48> <5.1.0.14.0.20010731201359.02719598@24.8.96.48> Message-ID: <3B674D04.54F2E83E@ActiveState.com> Dan Sugalski wrote: > >... > > Is this global in the "all variables are in one big pool and can be looked > up by name", more or less the equivalent of externing all your C variables, > or global in the "Any thread might be able to see it" sense? (I thought > Python had lexically scoped variables, but I might be wrong here. I'm an > internals guy--the actual language being run is just an implementation > detail... :) Python certainly has HIGHLY SCOPED variables. Hardly anything is global in that sense. Even a two-line script will not really have truly global, global variables. I wasn't so happy with Aahz's choice of wording there. :) Only the namespace of module names is truly global. i.e. "import sys" gets you the same module object everywhere so you can use modules as containers for other kinds of globals. > If it's "All objects are visible to all threads" thing, that's fine, you > can still handle threading with some care. Perl's done it wrong once, and I > know how to do it much less wrong with Parrot. (It's not, strictly > speaking, possible to do it right without the programmer taking an active hand) Agreed. >... > >And, as Paul pointed out, the global interpreter lock only kills you if > >you're running SMP in the first place. > > Yep, well, I do. :) Still, I would encourage you to optimize for the common case. People have been predicting multiprocessor would be the common case for years but Moore's law keeps delaying the need to pump up all of our servers with a dozen processors. The Python threading model is one interpreter for all threads unless you really need multiple interpreters in which case you are sort of on your own. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From paul@pfdubois.com Wed Aug 1 01:19:45 2001 From: paul@pfdubois.com (Paul F. Dubois) Date: Tue, 31 Jul 2001 17:19:45 -0700 Subject: [Python-Dev] numerical models and numpy Message-ID: I'm a little worried that you all may be assuming that I'm keeping up with this flood of stuff on python-dev about the numerical model. I have tried but I just can't manage that much volume. My silence doesn't mean approval. Whatever "native" types you decide on there will be considerable pressure on us to include corresponding types in Numerical Python, but I don't think that can happen. At the moment there seem to be few people with any time to spare to help me. I won't be able to do such work myself as my job just has no use for decimals, rationals, etc. My community would like a native Python type corresponding to a C float (and a corresponding complex made out of two floats). Numpy already has those. I was considering just hacking the source file for the native Python type corresponding to a C double -- until I looked at it. What a complicated beast it is. I have room in my draft implementation of PEP 242 for the beast and I was going to do it in there. To those of you thinking about this that haven't read PEP 242, please do so. The Fortran committee thought hard about these issues and concluded that some numeric "types" are not really different types but different "kinds" of the same basic type. -- Paul From DavidA@ActiveState.com Wed Aug 1 01:33:15 2001 From: DavidA@ActiveState.com (David Ascher) Date: Tue, 31 Jul 2001 17:33:15 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? References: <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1393.232974.785433@beluga.mojam.com> <20010731041443.A26075@thyrsus.com> <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48> <5.1.0.14.0.20010731200907.02747f58@24.8.96.48> Message-ID: <3B674E4B.1B849A06@ActiveState.com> To bring in yet someone else to this discussion, Mark Hammond and I were chatting with Brendan Eich (of JavaScript and Mozilla fame) last week about interpreter locks and the like, and Brendan mentioned that he had done a lot of possibly reusable work in optimizing the various kinds of thread lock strategies in JavaScript, as JS is used a lot server-side. Specifically he was talking about some 'multi-level' locking stuff which sounded impressive to me at least. I doubt if Brendan has time to dig into either of Perl or Python's internals, but I'm sure he'll provide general pointers if folks ask nicely. =) Brendan, do you care to give an executive overview of the locking strategies used in JS? --david ascher PS: Brendan: If you want to know the context of this discussion, see: http://aspn.activestate.com/ASPN/Mail/Browse/Threaded/python-dev From paulp@ActiveState.com Wed Aug 1 01:36:43 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 31 Jul 2001 17:36:43 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? References: <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1393.232974.785433@beluga.mojam.com> <20010731041443.A26075@thyrsus.com> <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48> <5.1.0.14.0.20010731200907.02747f58@24.8.96.48> Message-ID: <3B674F1B.4CD32765@ActiveState.com> Dan Sugalski wrote: > >... > > If you hold the lock during an I/O operation, you'll lose time you could > have otherwise used. "Doctor it hurts when I do that." :) We just don't hold the lock during I/O or other computations that do not affect Python interpreter data structures. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From aahz@rahul.net Wed Aug 1 01:49:38 2001 From: aahz@rahul.net (Aahz Maruch) Date: Tue, 31 Jul 2001 17:49:38 -0700 (PDT) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <5.1.0.14.0.20010731200907.02747f58@24.8.96.48> from "Dan Sugalski" at Jul 31, 2001 08:13:33 PM Message-ID: <20010801004938.67EA499C80@waltz.rahul.net> Dan Sugalski wrote: > > If you hold the lock during an I/O operation, you'll lose time you could > have otherwise used. Python extensions can (and do, for all standard I/O operations) release the Global Interpreter Lock. For more information, check out http://starship.python.net/crew/aahz/ -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From aahz@rahul.net Wed Aug 1 01:59:12 2001 From: aahz@rahul.net (Aahz Maruch) Date: Tue, 31 Jul 2001 17:59:12 -0700 (PDT) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <5.1.0.14.0.20010731201359.02719598@24.8.96.48> from "Dan Sugalski" at Jul 31, 2001 08:19:41 PM Message-ID: <20010801005913.3B5F999C80@waltz.rahul.net> Dan Sugalski wrote: > At 05:04 PM 7/31/2001 -0700, Aahz Maruch wrote: >>Dan Sugalski wrote: >>> >>> As for threading, well, that's where things get interesting. Perl's >>> tried it two ways (multiple threads in the same interpreter, and one >>> thread per interpreter, with cloned interpreters) both of which aren't >>> very good. And the global lock thing's not that keen either. The plan >>> for parrot is to allow multiple interpreters to run simultaneously, >>> with shared data protected on access time. (And *only* shared data >>> protected, so there's no penalty to access non-shared data) Along with >>> a scheme to allow only partial (or no) visibility of the spawning >>> thread's variables. We plan on basing the secure sandbox scheme around >>> this as well. >> >>This is where calling what Python uses "variables" gets you into real >>problems. Many people think that the terms for "variable" and >>"reference" should be "name" and "binding"; it makes things far less >>confusing. > > I'm not sure that's the appropriate translation for what perl considers > variables and references, but that's OK. Paul evidently disagrees with me, but I'll write a rebuttal to that right after I finish this. ;-) >>See, Python objects are *global*. Period. No ifs, ands, or >>buts. > > Is this global in the "all variables are in one big pool and can be > looked up by name", more or less the equivalent of externing all your > C variables, or global in the "Any thread might be able to see it" > sense? (I thought Python had lexically scoped variables, but I might > be wrong here. I'm an internals guy--the actual language being run is > just an implementation detail... :) Somewhere in between. Python names are scoped; Python objects are not. Here's a trival example: given a global list "l" (and "global" in this phrase almost always means "module global"), and the following code fragment running in one thread def foo(): x = 1 l.append(x) any other threads created in that module will immediately have access to the locally-created object "1". Beyond that, though, given Python's introspection capabilities, it is theoretically correct to say that all variables can be looked up by name. > If it's "All objects are visible to all threads" thing, that's fine, > you can still handle threading with some care. Perl's done it wrong > once, and I know how to do it much less wrong with Parrot. (It's not, > strictly speaking, possible to do it right without the programmer > taking an active hand) Yup. Which is why the GIL represents a pretty good compromise, most of the time, particularly the way it's implemented in Python. >>And, as Paul pointed out, the global interpreter lock only kills you if >>you're running SMP in the first place. > > Yep, well, I do. :) Should have added, "...and only if you're running pure Python code." -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From aahz@rahul.net Wed Aug 1 02:06:00 2001 From: aahz@rahul.net (Aahz Maruch) Date: Tue, 31 Jul 2001 18:06:00 -0700 (PDT) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <3B674D04.54F2E83E@ActiveState.com> from "Paul Prescod" at Jul 31, 2001 05:27:48 PM Message-ID: <20010801010600.2764699C80@waltz.rahul.net> Paul Prescod wrote: > Dan Sugalski wrote: >> >> Is this global in the "all variables are in one big pool and can be looked >> up by name", more or less the equivalent of externing all your C variables, >> or global in the "Any thread might be able to see it" sense? (I thought >> Python had lexically scoped variables, but I might be wrong here. I'm an >> internals guy--the actual language being run is just an implementation >> detail... :) > > Python certainly has HIGHLY SCOPED variables. Hardly anything is global > in that sense. Even a two-line script will not really have truly global, > global variables. I wasn't so happy with Aahz's choice of wording there. > :) No, but it will have global, global objects. Any time one creates an object in Python, it has the potential to be bound to any name. Consider the module foo with attribute bar; foo.bar can be set at any time in any scope to any other object existing somewhere in Python's heap. And that change will be visible everwhere that the name foo.bar is visible. It gets even trickier with multiple threads created in a single module namespace combined with mutable objects and global names (as I illustrated in my response to Dan). -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From aahz@rahul.net Wed Aug 1 02:07:50 2001 From: aahz@rahul.net (Aahz Maruch) Date: Tue, 31 Jul 2001 18:07:50 -0700 (PDT) Subject: [Python-Dev] Revised decimal type PEP In-Reply-To: <200108010017.MAA00798@s454.cosc.canterbury.ac.nz> from "Greg Ewing" at Aug 01, 2001 12:17:14 PM Message-ID: <20010801010750.E045299C80@waltz.rahul.net> Greg Ewing wrote: > aahz@rahul.net (Aahz Maruch): >> >> Truncation (rounding), overflow, and underflow errors can >> occur under addition, subtraction, and multiplication. It's trivial to >> set them to be unbounded, but then Cowlishaw provides no mechanism for >> determining the truncation of division. > > If you allow for the representation of repeating parts in your > unbounded decimals, they could be closed under division. (I think -- > does the division of one repeating decimal by another always lead to > a third repeating decimal? Yes, it must, because every rational can > be expressed as a repeating decimal and vice versa, IIRC. Hmmm, that > means we'd just be implementing rationals another way...) Tell ya what, *you* write the algorithm and I'll think about it sometime in the next five years. ;-) -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From ping@lfw.org Wed Aug 1 02:12:55 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Tue, 31 Jul 2001 18:12:55 -0700 (PDT) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <20010731041443.A26075@thyrsus.com> Message-ID: On Tue, 31 Jul 2001, Eric S. Raymond wrote: > Things Python would bring to this party: our serious-cool GC, our > C extension/embedding system (*much* nicer than XS). Things Perl would > bring: blazingly fast regexps, taint, flexitypes, references. I don't really understand the motivation. Do we want any of those things? Are our current regexps not good enough? What are flexitypes, and why would we really want them? Don't we already have references? Taint might be worth something, and i would see the regexp engine as incrementally better perhaps, but that doesn't seem enough to justify a massive re-engineering of both languages. -- ?!ng From paulp@ActiveState.com Wed Aug 1 02:11:34 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 31 Jul 2001 18:11:34 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? References: <20010801010600.2764699C80@waltz.rahul.net> Message-ID: <3B675746.F060EAEC@ActiveState.com> Aahz Maruch wrote: > > ... > > No, but it will have global, global objects. Any time one creates an > object in Python, it has the potential to be bound to any name. > Consider the module foo with attribute bar; foo.bar can be set at any > time in any scope to any other object existing somewhere in Python's > heap. Right, but what language is otherwise? In what language are variables in one module firewalled from others so that references can't go back and forth? Only functional languages and perhaps security-constrained languages like JavaScript. > .... And that change will be visible everwhere that the name foo.bar > is visible. It gets even trickier with multiple threads created in a > single module namespace combined with mutable objects and global names > (as I illustrated in my response to Dan). > ... I don't see that as having much to do with "globality." Python objects are shared between threads including the module object and the dictionary of module objects. But variables in Python are seldom global in the sense of C's extern or Perl's default scope. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From ping@lfw.org Wed Aug 1 02:23:39 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Tue, 31 Jul 2001 18:23:39 -0700 (PDT) Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: <200107310622.CAA11742@cj20424-a.reston1.va.home.com> Message-ID: On Tue, 31 Jul 2001, Guido van Rossum wrote: > Why not add this to cgi.py instead? Because that would seem to defeat most of the point. The point was to provide an instant, effortless improvement for all of the Python CGI scripts out there. If programmers have to manually edit all of their CGI scripts to insert import sys, cgitb sys.excepthook = cgitb.excepthook then it's just as annoying as inserting import sys sys.stderr = sys.stdout I don't want people to have to edit every single script. > I don't like new additions that are > irrelevant for most apps (CGI is a tiny niche for Python IMO). I think this is where our perceptions differ. I think of CGI as the application that totally "made" Perl, and as the quickest, easiest way that many beginners get early payoff from a scripting language. My impression is that it has been a big "hook" for bringing people to Perl and Python -- it's the shortest path to building and deploying something useful to a huge and unlimited audience. Wouldn't you say there are more Python CGI programmers out there than, say, Zope developers? Or think of it like this: what fraction of Web developers are CGI programmers, and what fraction of those use Python? Maybe i'm wrong? I welcome more opinions from others -- how do you see people coming to Python? What's the first "real" thing they do with Python that motivates them to try it? -- ?!ng From ping@lfw.org Wed Aug 1 02:25:32 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Tue, 31 Jul 2001 18:25:32 -0700 (PDT) Subject: [Python-Dev] cgitb.py for Python 2.2 In-Reply-To: <15206.36478.421953.437702@anthem.wooz.org> Message-ID: On Tue, 31 Jul 2001, Barry A. Warsaw wrote: > I'm at best +0 on adding it to site.py too. E.g. for performance > reasons Mailman's cgi wrappers invoke Python with -S to avoid the > expensive overhead of importing site.py for each cgi hit. This is just one environment variable check -- doesn't touch the disk at all. Most of site.py's time is spent scanning paths and such, i believe. -- ?!ng From esr@thyrsus.com Wed Aug 1 03:06:27 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Tue, 31 Jul 2001 22:06:27 -0400 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: ; from ping@lfw.org on Tue, Jul 31, 2001 at 06:23:39PM -0700 References: <200107310622.CAA11742@cj20424-a.reston1.va.home.com> Message-ID: <20010731220627.G1708@thyrsus.com> Ka-Ping Yee : > I think this is where our perceptions differ. I think of CGI as > the application that totally "made" Perl, and as the quickest, easiest > way that many beginners get early payoff from a scripting language. > My impression is that it has been a big "hook" for bringing people to > Perl and Python -- it's the shortest path to building and deploying > something useful to a huge and unlimited audience. I strongly agree with Ping on this. My experience is that CGI is an extremely important application category for Python, and one worth significant effort to serve. -- Eric S. Raymond Good intentions will always be pleaded for every assumption of authority. It is hardly too strong to say that the Constitution was made to guard the people against the dangers of good intentions. There are men in all ages who mean to govern well, but they mean to govern. They promise to be good masters, but they mean to be masters. -- Daniel Webster From gnat@oreilly.com Wed Aug 1 03:26:47 2001 From: gnat@oreilly.com (Nathan Torkington) Date: Tue, 31 Jul 2001 20:26:47 -0600 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <3B668608.F68B5953@ActiveState.com> References: <20010730051831.B1122@thyrsus.com> <20010731012432.G20676@xs4all.nl> <15206.8708.811000.468489@gargle.gargle.HOWL> <20010731115521.I20676@xs4all.nl> <3B668608.F68B5953@ActiveState.com> Message-ID: <15207.26855.65000.718694@gargle.gargle.HOWL> Paul Prescod writes: > One of the things I picked up from the Perl conference is that Perl > users *seem* (to me) to have a higher tolerance for code breakage than > Python users. (and Python users have a higher tolerance than (let's say) > Java users) Even if we put aside Perl 6, Perlers talk pretty glibly > about ripping little used features out in Perl 5.8.0 and Perl 5.10 and > so forth. (off-topic, but he started it!) That's a really interesting comment. In many ways it's probably true, although make no mistake about it--we take it in the neck for every language change we make! The features we're talking about removing are either deprecated or experimental. With that said, we've probably been less rigorous than we should be about segregating experimental features from blessed ones. I suspect that because experimental features are the only way to do something, they still get used no matter how much we caution that they may go away. I've been eyeing your 'import from the future' system of getting experimental features. I'll be interested to see how this holds up. Perl and Python believe in active maintenance and development with many small incremental releases (unlike Java with occasional bursty releases, and C++ and C which are to all intents and purposes static). Changes to the language cause problems we share, and the other mainstream languages don't provide us with any direction on how to approach the problem. We in the Perl camp have bandied around the idea of "use perl 5.004" as a way to get all the behaviour of the 5.004 release. That's a noble idea, but forces us to keep a lot of crap around when one of the goals of active development is to be able to shed obstructive missteps in language evolution. Nat From pedroni@inf.ethz.ch Wed Aug 1 03:36:48 2001 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Wed, 1 Aug 2001 04:36:48 +0200 Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12 References: <200107301959.VAA11733@core.inf.ethz.ch> <2m4rrt78pn.fsf@starship.python.net> Message-ID: <000f01c11a32$f24241c0$525821c0@newmexico> Hi. [Michael Hudson] > Samuele Pedroni writes: > > > ... > > > > > > > > > > Does codeop currently work in Jython? The solution should continue to > > > > > work in Jython then. > > > > We have our interface compatible version of codeop that works. > > > > > > Would implementing the new interfaces I sketched out for codeop.py be > > > possible in Jython? That's the bit I care about, not so much the > > > interface to __builtin__.compile. > > Yes, it's of possible. > > Good; hopefully we can get somewhere then. > > > > > > Does Jython support the same flag bit values as > > > > > CPython? If not, Paul Prescod's suggestion to use keyword arguments > > > > > becomes very relevant. > > > > we support a subset of the co_flags, CO_NESTED e.g. is there with the same > > > > value. > > > > > > > > But the embedding API is very different, my implementation of nested > > > > scopes does not define any Py_CF... flags, we have an internal CompilerFlags > > > > object but is more similar to PyFutureFeatures ... > > > > > > Is this object exposed to Python code at all? > > Not publicily, but in Jython the separating line is a bit different, > > because public java classes are always accessible from jython, > > even most of the internals. That does not mean and every use of that > > is welcome and supported. > > Ah, of course. I'd forgotten how cool Jython was in some ways. > > > > One approach would be > > > PyObject-izing PyFutureFlags and making *that* the fourth argument to > > > compile... > > > > > > class Compiler: > > > def __init__(self): > > > self.ff = ff.new() # or whatever > > > def __call__(self, source, filename, start_symbol): > > > code = compile(source, filename, start_symbol, self.ff) > > > self.ff.merge(code.co_flags) > > > return code > > I see, "internally" we already have a compiler_flags function > > that do the same of: > > > code = compile(source, filename, start_symbol, self.ff) > > > self.ff.merge(code.co_flags) > > > > where self.ff is a CompuilerFlags object. > > > > I can re-arrange things for any interface, > > Well, I don't want to make more work for you - I imagine Guido's doing > enough of that for two! > > > I was only trying to explain our approach and situation and a > > possible way to avoid duplicating some internal code in Python. > > Can you point me to the code in CVS that implements this sort of > thing? I don't really know Java but I can probably muddle through to > some extent. We might as well have CPython copy Jython for once... I don't know if this can help you but the relevant stuff is in: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jython/jython/org/python/modules /codeop.java http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jython/jython/org/python/core/Co mpilerFlags.java http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jython/jython/org/python/core/Py .java see compile_flags http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jython/jython/org/python/compile r/Module.java see compile and PyCode http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jython/jython/org/python/compile r/CodeCompiler.java http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jython/jython/org/python/compile r/Future.java However implementing this approach in CPython is a bit more demanding because we can import any java class from jython code, in CPython you cannot import a C struct directly (for the obvious reasons) you need an Extension Type :(. regards, Samuele Pedroni. From guido@zope.com Wed Aug 1 03:51:30 2001 From: guido@zope.com (Guido van Rossum) Date: Tue, 31 Jul 2001 22:51:30 -0400 Subject: [Python-Dev] Revised decimal type PEP In-Reply-To: Your message of "Tue, 31 Jul 2001 16:55:26 PDT." <20010731235526.8E8ED99C82@waltz.rahul.net> References: <20010731235526.8E8ED99C82@waltz.rahul.net> Message-ID: <200108010251.WAA18015@cj20424-a.reston1.va.home.com> > Guido van Rossum wrote: > > > > int/long < decimal < rational < float < complex > > \---------------------------/ \-------------/ > > exact inexact > > Note that in Cowlishaw's implementation of decimal numbers, decimals are > *not* exact. Truncation (rounding), overflow, and underflow errors can > occur under addition, subtraction, and multiplication. It's trivial to > set them to be unbounded, but then Cowlishaw provides no mechanism for > determining the truncation of division. I know all that. Cowlishaw's numbers are a replacement for float (and for the real/imag parts of complex). I was toying with a *different* idea, more appropriate perhaps for accountants, that guarantees exact results for decimal numbers, giving decimal results when possible, rational otherwise. This is based on my guess about Michael's target audience. Michael makes a big deal of not allowing "decimalness" to be contaminated by mixing with floats. Possible this could be implemented using a subclass of your decimals that uses unbounded precision but overrides division to produce a rational number if needed. But it's possible that Michael's audience would in fact be very happy with your/Cowlishaw's decimal numbers. --Guido van Rossum (home page: http://www.python.org/~guido/) From pedroni@inf.ethz.ch Wed Aug 1 03:52:28 2001 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Wed, 1 Aug 2001 04:52:28 +0200 Subject: R: [Python-Dev] Parrot -- should life imitate satire? References: <20010731041443.A26075@thyrsus.com> <20010731095941.E1708@thyrsus.com> Message-ID: <001f01c11a35$01bc6fc0$525821c0@newmexico> [Eric S. Raymond] > No, but we want to be able to interoperate with Perl and have if possible > have just one back end on which efforts to do things like native code > compilation can be concentrated. >From my experience of the relative pain of a dynamic language over a static typed VM, and from the fact that native (dynamic?) compilation of Python/Perl would reduce the amount of code that must/is perceived to have to be written C, if the goal is to work on the long run on native compilation, that's a *great* goal. Sharing C extesion would be also a good thing, no more duplicated efforts for writing wrapping code for important libraries. Sharing Perl libraries I feel is somehow different, using a library needs often to read the code, what would be the sentiment if a large part of the code a python program calls is written in Perl. IMHO also the fact of not merging the type "ontologies" but carrying somehow both around is a bit scary. regards, Samuele Pedroni. From guido@zope.com Wed Aug 1 03:56:06 2001 From: guido@zope.com (Guido van Rossum) Date: Tue, 31 Jul 2001 22:56:06 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Your message of "Tue, 31 Jul 2001 17:33:15 PDT." <3B674E4B.1B849A06@ActiveState.com> References: <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1393.232974.785433@beluga.mojam.com> <20010731041443.A26075@thyrsus.com> <5.1.0.14.0.20010731192136.0268c9a0@24.8.96.48> <5.1.0.14.0.20010731200907.02747f58@24.8.96.48> <3B674E4B.1B849A06@ActiveState.com> Message-ID: <200108010256.WAA18063@cj20424-a.reston1.va.home.com> BTW, for a VM flexible enough to deal with Python and Perl, JavaScript should be a piece of cake. (JavaScript is much closer to Python than to Perl in its semantic model, for the most part.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@zope.com Wed Aug 1 04:00:17 2001 From: guido@zope.com (Guido van Rossum) Date: Tue, 31 Jul 2001 23:00:17 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Your message of "Tue, 31 Jul 2001 18:12:55 PDT." References: Message-ID: <200108010300.XAA18095@cj20424-a.reston1.va.home.com> > On Tue, 31 Jul 2001, Eric S. Raymond wrote: > > Things Python would bring to this party: our serious-cool GC, our > > C extension/embedding system (*much* nicer than XS). Things Perl would > > bring: blazingly fast regexps, taint, flexitypes, references. > > I don't really understand the motivation. Do we want any of those things? > > Are our current regexps not good enough? > What are flexitypes, and why would we really want them? > Don't we already have references? > > Taint might be worth something, and i would see the regexp engine as > incrementally better perhaps, but that doesn't seem enough to justify > a massive re-engineering of both languages. Python won't be reengineered. If the Perl VM happens to be able to retargetable for Python, maybe some synergy might happen (like reusable C extensions). But I don't see Python making any compromises in its semantic model, and the new VM would only be used for something like Python 3.0 (or more likely 4.0) if it really offers significant advantages. Dan's recent contributions make me a lot less optimistic. Don't ask me to explain -- I don't have the time, sorry. The other participants in the thread are doing a good job. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@zope.com Wed Aug 1 04:06:14 2001 From: guido@zope.com (Guido van Rossum) Date: Tue, 31 Jul 2001 23:06:14 -0400 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: Your message of "Tue, 31 Jul 2001 18:23:39 PDT." References: Message-ID: <200108010306.XAA18133@cj20424-a.reston1.va.home.com> > On Tue, 31 Jul 2001, Guido van Rossum wrote: > > Why not add this to cgi.py instead? > > Because that would seem to defeat most of the point. > > The point was to provide an instant, effortless improvement for all > of the Python CGI scripts out there. If programmers have to manually > edit all of their CGI scripts to insert > > import sys, cgitb > sys.excepthook = cgitb.excepthook > > then it's just as annoying as inserting > > import sys > sys.stderr = sys.stdout > > I don't want people to have to edit every single script. You misunderstand. You proposed a few lines that would automatically do this in site.py, which is always imported. I propose to add those same lines to cgi.py, so that any code that imports cgi.py *automatically* has he feature enabled. I assume that all CGI scripts have an import of cgi. There are non-CGI scripts that import cgi, but those will be protected by the check for an environment variable that you propose. > > I don't like new additions that are > > irrelevant for most apps (CGI is a tiny niche for Python IMO). > > I think this is where our perceptions differ. I think of CGI as > the application that totally "made" Perl, and as the quickest, easiest > way that many beginners get early payoff from a scripting language. Yeah. But it's not doing that for Python IMO. Most Python apps (even those that do web stuff) are not CGI apps. > My impression is that it has been a big "hook" for bringing people to > Perl and Python -- it's the shortest path to building and deploying > something useful to a huge and unlimited audience. > > Wouldn't you say there are more Python CGI programmers out there > than, say, Zope developers? Or think of it like this: what fraction > of Web developers are CGI programmers, and what fraction of those > use Python? I don't want Python to become a wannabe CGI language, and I don't want to make choices that benefit CGI programmers to the detriment of others (CGI is actually a pretty lame way of producing active web content). Python is a decent language for CGI, but Perl is the established standard and then there's PHP which also has way more users than Python. > Maybe i'm wrong? I welcome more opinions from others -- how do you > see people coming to Python? What's the first "real" thing they do > with Python that motivates them to try it? All sorts of stuff. Using NumPy. GUI apps. Database apps. Unix scripting. App steering. --Guido van Rossum (home page: http://www.python.org/~guido/) From dan@sidhe.org Wed Aug 1 04:16:05 2001 From: dan@sidhe.org (Dan Sugalski) Date: Tue, 31 Jul 2001 23:16:05 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: References: <20010731041443.A26075@thyrsus.com> Message-ID: <5.1.0.14.0.20010731231443.02298fd8@24.8.96.48> At 06:12 PM 7/31/2001 -0700, Ka-Ping Yee wrote: >On Tue, 31 Jul 2001, Eric S. Raymond wrote: > > Things Python would bring to this party: our serious-cool GC, our > > C extension/embedding system (*much* nicer than XS). Things Perl would > > bring: blazingly fast regexps, taint, flexitypes, references. > >I don't really understand the motivation. Do we want any of those things? > >Taint might be worth something, and i would see the regexp engine as >incrementally better perhaps, but that doesn't seem enough to justify >a massive re-engineering of both languages. Honestly, neither language should need any reengineering. If it does, then that means I've failed as a designer. The point isn't to redesign Python to run on the Parrot back-end--it's to design the Parrot back end such that Python runs just fine on it. Which doesn't mean you *can't* redesign, just that you shouldn't have to. Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From guido@zope.com Wed Aug 1 04:21:58 2001 From: guido@zope.com (Guido van Rossum) Date: Tue, 31 Jul 2001 23:21:58 -0400 Subject: [Python-Dev] numerical models and numpy In-Reply-To: Your message of "Tue, 31 Jul 2001 17:19:45 PDT." References: Message-ID: <200108010321.XAA18226@cj20424-a.reston1.va.home.com> > I'm a little worried that you all may be assuming that I'm keeping up with > this flood of stuff on python-dev about the numerical model. I have tried > but I just can't manage that much volume. My silence doesn't mean approval. > > Whatever "native" types you decide on there will be considerable pressure on > us to include corresponding types in Numerical Python, but I don't think > that can happen. At the moment there seem to be few people with any time to > spare to help me. I won't be able to do such work myself as my job just has > no use for decimals, rationals, etc. > > My community would like a native Python type corresponding to a C float (and > a corresponding complex made out of two floats). Numpy already has those. I > was considering just hacking the source file for the native Python type > corresponding to a C double -- until I looked at it. What a complicated > beast it is. I have room in my draft implementation of PEP 242 for the beast > and I was going to do it in there. I have read your PEP and see it as a way to provide control over very different aspects of numbers. I expect that most of it is motivated out of a desire to control the range, size and accuracy of floating point more precisely, *and* to have scalar types matching the NumPy types. I think we can give you what you want for float and complex, because I have no great plans to change those (a decimal type might be added, but we can't remove binary floats). I'm nor sure if you really need various width integer types once the native Python integer uses unbounded precision (the int/long unification). > To those of you thinking about this that haven't read PEP 242, > please do so. The Fortran committee thought hard about these issues > and concluded that some numeric "types" are not really different > types but different "kinds" of the same basic type. Yeah, but Python is not Fortran, and programmer needs do differ. I do like the terminological distinction, it allows us to talk with more clarity about these aspects of our numbers! --Guido van Rossum (home page: http://www.python.org/~guido/) From dan@sidhe.org Wed Aug 1 04:31:07 2001 From: dan@sidhe.org (Dan Sugalski) Date: Tue, 31 Jul 2001 23:31:07 -0400 Subject: R: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <001f01c11a35$01bc6fc0$525821c0@newmexico> References: <20010731041443.A26075@thyrsus.com> <20010731095941.E1708@thyrsus.com> Message-ID: <5.1.0.14.0.20010731231925.022a68e0@24.8.96.48> At 04:52 AM 8/1/2001 +0200, Samuele Pedroni wrote: >[Eric S. Raymond] > > No, but we want to be able to interoperate with Perl and have if possible > > have just one back end on which efforts to do things like native code > > compilation can be concentrated. > > From my experience of the relative pain of a dynamic language over a static >typed VM, and from the fact that native (dynamic?) compilation of >Python/Perl would reduce the amount of code that must/is perceived to have >to be written C, if the goal is to work on the long run on native >compilation, that's a *great* goal. This is one of the goals, yes. It's next on the list after getting a working interpreter. There will probably be a TIL version on the road to native compilation. >IMHO also the fact of not merging the type "ontologies" but carrying somehow >both around is a bit scary. It's mildly scary, sure. On the other hand, it's not really any more scary than, say, writing code in COBOL, C++, Basic, or Fortran and calling them from C. Or vice versa. While that's a tad odd, you can pretty easily get used to it, and it works out well. You also don't have to mix languages. That's certainly one of the advantages, but it's far from required. (This is assuming, of course, that there's a performance gain to be had in targeting the parrot back end--if there isn't, then it's reasonably pointless to do so) Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From dan@sidhe.org Wed Aug 1 04:41:55 2001 From: dan@sidhe.org (Dan Sugalski) Date: Tue, 31 Jul 2001 23:41:55 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <20010731080128.B2757@ute.cnri.reston.va.us> References: <15206.8511.147000.832644@gargle.gargle.HOWL> <20010730051831.B1122@thyrsus.com> <20010731012432.G20676@xs4all.nl> <20010730205657.A2298@ute.cnri.reston.va.us> <15206.8511.147000.832644@gargle.gargle.HOWL> Message-ID: <5.1.0.14.0.20010731233118.022a6640@24.8.96.48> At 08:01 AM 7/31/2001 -0400, Andrew Kuchling wrote: >On Mon, Jul 30, 2001 at 08:08:47PM -0700, Nathan Torkington wrote: > >Andrew Kuchling writes: > >The core loop would just be the usual opcode dispatch loop ("call the > >function for the current operation, which returns the next > >operation"). The only difference is that some of the opcodes would be > >specific to RE matches. (I'm unclear on how much special logic RE > >The big difference I see between regex opcodes and language opcodes is >that regexes need to backtrack and language ones don't. Backtracking in regexes isn't all that much different from some of the things you do at the language level, and certainly it's a straightforward mapping of regexes onto a series of string & integer register manipulations with simple branch-and-test code, with perhaps some stack space thrown in for deeply nested regexes. >Unless the >idea is to compile a regex to actual VM code similar to that generated >by Python/Perl code, but then wouldn't that sacrifice efficiency? The idea is (currently, barring bad performance) to compile down to VM code similar to what Ptyhon or perl would generate. There's the potential to slow things down, which I'm rather nervous about. On the other hand, the sort of code that would get emitted for regexes maps very well onto the instruction sets of hardware CPUs, so you'd get a corresponding boost in speed once translated to machine code. (Though granted you'd need to be able to do the conversion on the fly, as the regex itself isn't guaranteed to be known at compile time) Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From guido@zope.com Wed Aug 1 04:46:50 2001 From: guido@zope.com (Guido van Rossum) Date: Tue, 31 Jul 2001 23:46:50 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Your message of "Tue, 31 Jul 2001 23:41:55 EDT." <5.1.0.14.0.20010731233118.022a6640@24.8.96.48> References: <15206.8511.147000.832644@gargle.gargle.HOWL> <20010730051831.B1122@thyrsus.com> <20010731012432.G20676@xs4all.nl> <20010730205657.A2298@ute.cnri.reston.va.us> <15206.8511.147000.832644@gargle.gargle.HOWL> <5.1.0.14.0.20010731233118.022a6640@24.8.96.48> Message-ID: <200108010346.XAA18484@cj20424-a.reston1.va.home.com> Dan, Have you looked at the byte code used in SRE, Python's (mostly) Perl-compatible regex package? Worth a look! --Guido van Rossum (home page: http://www.python.org/~guido/) From dan@sidhe.org Wed Aug 1 04:49:02 2001 From: dan@sidhe.org (Dan Sugalski) Date: Tue, 31 Jul 2001 23:49:02 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <200108010300.XAA18095@cj20424-a.reston1.va.home.com> References: Message-ID: <5.1.0.14.0.20010731234359.022a1298@24.8.96.48> At 11:00 PM 7/31/2001 -0400, Guido van Rossum wrote: >Dan's recent contributions make me a lot less optimistic. >Don't ask me to explain -- I don't have the time, sorry. The other >participants in the thread are doing a good job. I would appreciate it if, when you have a chance, you (or someone) would clue me in. I've apparently offended, and that was not, in any way, my intent. My apologies all around. I think I've overstayed my welcome, so I'll bow out here before muddying things up any worse. Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From dan@sidhe.org Wed Aug 1 04:52:01 2001 From: dan@sidhe.org (Dan Sugalski) Date: Tue, 31 Jul 2001 23:52:01 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <200108010346.XAA18484@cj20424-a.reston1.va.home.com> References: <15206.8511.147000.832644@gargle.gargle.HOWL> <20010730051831.B1122@thyrsus.com> <20010731012432.G20676@xs4all.nl> <20010730205657.A2298@ute.cnri.reston.va.us> <15206.8511.147000.832644@gargle.gargle.HOWL> <5.1.0.14.0.20010731233118.022a6640@24.8.96.48> Message-ID: <5.1.0.14.0.20010731234925.02298ae8@24.8.96.48> At 11:46 PM 7/31/2001 -0400, Guido van Rossum wrote: >Have you looked at the byte code used in SRE, Python's (mostly) >Perl-compatible regex package? Worth a look! Nope, not yet, but I will. (You folks will end up making a Python programmer out of me yet, I expect... :) This and the C extension interface are the two big Python things I've got on my "must peer deeply into" list. Are there other places I should be concentrating on? (I'm poking about with the interpreter loop and associated code, but that's the sort of thing an expert with 20 minutes free will get me much farther than a week or two with the source) Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From pedroni@inf.ethz.ch Wed Aug 1 04:51:23 2001 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Wed, 1 Aug 2001 05:51:23 +0200 Subject: [Python-Dev] Parrot -- should life imitate satire? References: <20010731041443.A26075@thyrsus.com><20010731095941.E1708@thyrsus.com> <5.1.0.14.0.20010731231925.022a68e0@24.8.96.48> Message-ID: <003c01c11a3d$3cb1d860$665821c0@newmexico> [Dan Sugalski] > > >IMHO also the fact of not merging the type "ontologies" but carrying somehow > >both around is a bit scary. > > It's mildly scary, sure. On the other hand, it's not really any more scary > than, say, writing code in COBOL, C++, Basic, or Fortran and calling them > from C. Or vice versa. While that's a tad odd, you can pretty easily get > used to it, and it works out well. > > You also don't have to mix languages. That's certainly one of the > advantages, but it's far from required. (This is assuming, of course, that > there's a performance gain to be had in targeting the parrot back end--if > there isn't, then it's reasonably pointless to do so)O If one have to call conversion function from Python/Perl code to deal with the other ontology or deal with unnatural behavior avoiding mixing languages will remain the best thing, also for the problem of reading code in the other language. But for C extensions the situation is a bit different, they should target one ontology (or not?) and then using the extension from both languages will be not that natural. And C extension in Python typically extend the ontology, introduce new types ... regards, Samuele Pedroni. From dan@sidhe.org Wed Aug 1 04:43:45 2001 From: dan@sidhe.org (Dan Sugalski) Date: Tue, 31 Jul 2001 23:43:45 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <200107312312.TAA17483@cj20424-a.reston1.va.home.com> References: <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1909.395000.123189@gargle.gargle.HOWL> Message-ID: <5.1.0.14.0.20010731231748.0228f808@24.8.96.48> At 07:12 PM 7/31/2001 -0400, Guido van Rossum wrote: > > Guido van Rossum writes: > > > Yeah, but the runtime could offer a choice of data types -- for Python > > > code the constants table would contain Python ints and strings etc., for > > > Perl code it would contain Perl string-number objects. Maybe. > > > > A perl6 value have a vtable, essentially an array of function pointers > > which comprises the standard operations on that value. I talked to > > Dan (the perl6 internals guy, dan@sidhe.org) about an impedence > > mismatch between Perl and Python data types, and he pointed out that > > you can have Perl values and Python values, each with their own > > semantics, simply by having separate vtables (and thus separate > > functions to implement the behaviour of those types). Code can work > > with either type because the type carries around (in its vtable) the > > knowledge of how it should behave. > >The vtable looks a lot like Python's type object. Is Perl's vtable an >object in its own right? Depends on your definition of object, I suppose. There is a mapping from the low-level bits the interpreter needs to interpreted-language level things--while the vtable won't actually be directly manipulatable at the HLL level, manipulating HLL constructs will alter the vtable, so it's effectively the same thing. Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From guido@zope.com Wed Aug 1 05:03:58 2001 From: guido@zope.com (Guido van Rossum) Date: Wed, 01 Aug 2001 00:03:58 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Your message of "Tue, 31 Jul 2001 23:43:45 EDT." <5.1.0.14.0.20010731231748.0228f808@24.8.96.48> References: <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1909.395000.123189@gargle.gargle.HOWL> <5.1.0.14.0.20010731231748.0228f808@24.8.96.48> Message-ID: <200108010403.AAA18645@cj20424-a.reston1.va.home.com> > >The vtable looks a lot like Python's type object. Is Perl's vtable an > >object in its own right? > > Depends on your definition of object, I suppose. There is a mapping from > the low-level bits the interpreter needs to interpreted-language level > things--while the vtable won't actually be directly manipulatable at the > HLL level, manipulating HLL constructs will alter the vtable, so it's > effectively the same thing. In Python, the type pointer really does point to a full-fledged object. Which probably means something quite different than it means in Perl. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From pedroni@inf.ethz.ch Wed Aug 1 05:02:58 2001 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Wed, 1 Aug 2001 06:02:58 +0200 Subject: [Python-Dev] Parrot -- should life imitate satire? References: <20010731041443.A26075@thyrsus.com><20010731095941.E1708@thyrsus.com> <5.1.0.14.0.20010731231925.022a68e0@24.8.96.48> <003c01c11a3d$3cb1d860$665821c0@newmexico> Message-ID: <006801c11a3e$dac4a720$665821c0@newmexico> > > But for C extensions the situation is a bit different, they should target one > ontology (or not?) > and then using the extension from both languages will be not that natural. > And C extension in Python typically extend the ontology, introduce new types > ... One more thing, with Python 2.2 you will be able to subclass those type from C Extensions through Python code. > regards, Samuele Pedroni. From michel@digicool.com Wed Aug 1 05:10:56 2001 From: michel@digicool.com (Michel Pelletier) Date: Tue, 31 Jul 2001 21:10:56 -0700 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 References: Message-ID: <3B678150.B8462B7B@digicool.com> Ka-Ping Yee wrote: > > The point was to provide an instant, effortless improvement for all > of the Python CGI scripts out there. If programmers have to manually > edit all of their CGI scripts to insert > > import sys, cgitb > sys.excepthook = cgitb.excepthook > > then it's just as annoying as inserting > > import sys > sys.stderr = sys.stdout But Zope would have to do: import sys _sys_excepthook = sys.excepthook from cgi import FieldStorage, escape sys.excepthook = _sys_excepthook and I don't think we'd be the only one that would have to do that. Zope has the environment variable set when accessed from CGI, but its stderr gets redirected to a log file for plain text human consumption. importing cgi would have an unintended side effect. I would prefer an explicit interface than a side effect. +1 on it being in the standard lib, or course. -Michel From dan@sidhe.org Wed Aug 1 05:16:43 2001 From: dan@sidhe.org (Dan Sugalski) Date: Wed, 01 Aug 2001 00:16:43 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <200108010403.AAA18645@cj20424-a.reston1.va.home.com> References: <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1909.395000.123189@gargle.gargle.HOWL> <5.1.0.14.0.20010731231748.0228f808@24.8.96.48> Message-ID: <5.1.0.14.0.20010801000522.0238d970@24.8.96.48> At 12:03 AM 8/1/2001 -0400, Guido van Rossum wrote: > > >The vtable looks a lot like Python's type object. Is Perl's vtable an > > >object in its own right? > > > > Depends on your definition of object, I suppose. There is a mapping from > > the low-level bits the interpreter needs to interpreted-language level > > things--while the vtable won't actually be directly manipulatable at the > > HLL level, manipulating HLL constructs will alter the vtable, so it's > > effectively the same thing. > >In Python, the type pointer really does point to a full-fledged >object. Which probably means something quite different than it means >in Perl. :-) Ah. There's going to be a level of indirection there with Parrot. That'll introduce some speed hits, but only when altering the behaviour of the objects. It's been my experience that altering behaviour happens rather less frequently than the behaviour itself happens, so it makes sense to optimize both ends (the interface presented to the HLL programmer and the interface presented to the interpreter engine) and let the middle layer be a bit slower. In this case, the vtable interface doesn't cover general method calls (except to the extent that a variable will have a vtable entry to handle method calls, in case you want to override the inheritance behaviour on a per-object basis, or enable multimethod dispatch for some objects), which are another beast entirely. The vtable interface, at the moment, covers the operators you'd expect to be able to override--assignment, simple math, most string operations, conversion duties, and suchlike things. So this code: a = b + c would call the add vtable method for b, and the assign vtable method for a, while this: a.foo() would make a generic method call on a looking for the foo method, and the vtables generally wouldn't be involved other than handling the (possibly cached) lookup of foo. The point is to put the commonly called things in the vtable in a way that you can avoid as much conditional code as possible, while less common things get dealt with in ways you'd generally expect. (Dynamic lookups with caching and suchlike things) Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From tim.one@home.com Wed Aug 1 05:26:30 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 1 Aug 2001 00:26:30 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <3B667087.EBBE8938@ActiveState.com> Message-ID: [Paul Prescod] > Also, the .NET CLR is standardized at ECMA so we could (at least in > theory!) go to the meetings and try to influence version 2. [Guido] > Notice the addition "in theory". In practice, this is BS. [Paul Prescod] > It depends on the rules and politics of each particular standards group. > It is fundamentally a social activity. It also depends how much effort > you are willing to put into promoting your cause. And cash: being an active member (== the only way to be an *effective* member) of a stds committee is expensive and time-consuming. In my 15-year compiler-geek career, I worked for companies who put major money behind language development, and wrote off several person-years per annum to doing language committee work. Somehow I don't picture your boss springing for that -- which isn't a knock, since I can't picture the people up my chain of command even letting me finish explaining the idea . > ... > Working within a standards body is a gamble. It can pay off big or > it can completely fail. I don't know how ECMA works; indeed, I never heard of it before it agreed to fast-track ECMAScript (nee JavaScript). ANSI/ISO committees work by consensus, and one intransigent member can paralyze the entire effort. Lots of power there! The breadth and depth of compromises made to make progress in the end also account for the "a camel is a horse designed by a committee" outcomes. Note: I was an active member of ANSI X3J17 for a couple of years. This was tasked to come up with a high-level x-language model for expressing parallel computation, plus bindings for a few languages. In order to protect an ancient implementation of Fortran, the IBM representative vetoed any idea that required so much as a runtime *stack* -- or any other dynamic feature. In the end, X3J17 did the honorable thing: it voted to put itself out of its misery . had-enough-of-that-myself-ly y'rs - tim From dan@sidhe.org Wed Aug 1 05:42:28 2001 From: dan@sidhe.org (Dan Sugalski) Date: Wed, 01 Aug 2001 00:42:28 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <003c01c11a3d$3cb1d860$665821c0@newmexico> References: <20010731041443.A26075@thyrsus.com> <20010731095941.E1708@thyrsus.com> <5.1.0.14.0.20010731231925.022a68e0@24.8.96.48> Message-ID: <5.1.0.14.0.20010801002628.0228b9a0@24.8.96.48> At 05:51 AM 8/1/2001 +0200, Samuele Pedroni wrote: >[Dan Sugalski] > > > > >IMHO also the fact of not merging the type "ontologies" but carrying > somehow > > >both around is a bit scary. > > > > It's mildly scary, sure. On the other hand, it's not really any more scary > > than, say, writing code in COBOL, C++, Basic, or Fortran and calling them > > from C. Or vice versa. While that's a tad odd, you can pretty easily get > > used to it, and it works out well. > > > > You also don't have to mix languages. That's certainly one of the > > advantages, but it's far from required. (This is assuming, of course, that > > there's a performance gain to be had in targeting the parrot back end--if > > there isn't, then it's reasonably pointless to do so)O >If one have to call conversion function from Python/Perl code to deal with >the other ontology or deal with unnatural behavior avoiding mixing languages >will remain the best thing, also for the problem of reading code in the other >language. I'm not entirely sure there'll be as much of a mismatch between the languages as you might think. One thing that'll help on the perl end is the introduction of optionally typed data--some scalars may be tagged as integer, or arrays as string only, so a lot of the pain of needing to handle conversion and impedance matching will already have been dealt with. >But for C extensions the situation is a bit different, they should target one >ontology (or not?) >and then using the extension from both languages will be not that natural. >And C extension in Python typically extend the ontology, introduce new types >... It's possible there'll be a bigger mismatch, but I'm not that sure here, either. (Though I'm stumbling around a bit here, as I've no experience writing Python extensions in C) Some classes of extension will be essentially language neutral--an array's an array's an array and, while some of the details are interesting (like whether it's typed or sized) those are, ultimately details. Important ones, to be sure, but far from an obstacle. New types are a bit more of a challenge. Some things, like declaring methods or overloaded behaviour, are also reasonably small details. Other things, like things I'm not thinking of, are less so. Don't forget I'm also looking at this from the bottom up, not the top down. Parrot's going to be a dynamic language friendly, mildly OO CPU. From that level, these (And I'd get fancy here, but I don't know how to do so without embarrassment in Python): $foo->bar($a, $b, $c) foo.bar(a, b, c) are pretty much syntactically equivalent, and the fact that perl declares methods in packages and handles inheritance on a per-class basis via the @ISA array and Python does something else entirely syntactically is pretty straightforwardly abstractable, and isn't hugely important *for the execution engine* as long as that abstraction is sufficiently powerful. While Perl and Python aren't quite just ALGOL with funny hats, they're really not that different from each other from the level I'm working at. Certainly far less different than they are together from, say, LISP or Forth, or even .NET. (I expect we face similar performance issues with .NET, and the JVM, as neither seem particularly fond of dynamic languages. Though I do see that Jython gets pretty good performance from some of the JIT JVMs) Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From tim.one@home.com Wed Aug 1 05:42:42 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 1 Aug 2001 00:42:42 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <200107311708.NAA16497@cj20424-a.reston1.va.home.com> Message-ID: [Guido] > Actually, this may not be as big a deal as I thought before. The PVM > doesn't have a lot of knowledge about types built into its instruction > set. It knows a bit about classes, lists, dicts, but not e.g. about > ints and strings. The opcodes are mostly very abstract: BINARY_ADD etc. I agree overall, but you picked an unfortunate example. BINARY_DIVIDE is a perfect example: case BINARY_DIVIDE: w = POP(); v = POP(); x = PyNumber_Divide(v, w); Py_DECREF(v); Py_DECREF(w); PUSH(x); if (x != NULL) continue; break; (For Perl'ers, an operation in Python returns a PyObject*, and returns NULL iff the operation wants to raise an exception; the "continue" if x != NULL tells it to go straight back to the top of the eval loop (to fetch the next opcode) if there is no exception; else control flows out of the switch stmt, and masses of code figure out what to *do* with the exception.) That code works unchanged for any pair of objects whatsoever that think they know what to do with an infix "/", since all the intelligence is in PyNumber_Divide(). But BINARY_ADD knows everything there is to know about Python ints, and that's an important speed optimization: case BINARY_ADD: w = POP(); v = POP(); if (PyInt_Check(v) && PyInt_Check(w)) { /* INLINE: int + int */ register long a, b, i; a = PyInt_AS_LONG(v); b = PyInt_AS_LONG(w); i = a + b; if ((i^a) < 0 && (i^b) < 0) { PyErr_SetString(PyExc_OverflowError, "integer addition"); x = NULL; } else x = PyInt_FromLong(i); } else x = PyNumber_Add(v, w); Py_DECREF(v); Py_DECREF(w); PUSH(x); if (x != NULL) continue; break; While we don't peek under the covers often in the eval loop, the places we do were huge benefit/effort wins. From tim.one@home.com Wed Aug 1 05:47:35 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 1 Aug 2001 00:47:35 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Message-ID: [Moshe Zadka] > PUSH "1" > PUSH "2" > BINARY_ADD > > In Python that gives "12". In Perl that gives 3. > Unless you suggest a PERL_BINARY_ADD and a PYTHON_BINARY_ADD, I > don't see how you can around these things. Perl needs distinct flavors of BINARY_ADD for its infix "+" and "." operators. Once you accept that, there's no real problem here (it simply means that Python's and Perl's "+" operators would need to map to different opcodes). From tim.one@home.com Wed Aug 1 06:23:59 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 1 Aug 2001 01:23:59 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <5.1.0.14.0.20010731200907.02747f58@24.8.96.48> Message-ID: [Paul Prescod] > What is the downside of the global lock on the average single > processor machine? I tend to think that the "default" threading model > should allow simple and easy, everything-shared multi-threading on > ordinary machines. Having a multi-processor-friendly advanced mode is > a great extension for the wizards. [Dan Sugalski] > If you hold the lock during an I/O operation, you'll lose time you > could have otherwise used. Getting and releasing a global lock > frequently also costs performance you might otherwise have used in other > places. Mutex releases require memory coherency, which will force your > CPU to flush any pending writes that might be hanging about, which will > tend to drop it's efficiency, especially on heavily out-of-order > machines like the Alpha. > > Also, that is a zillion and a half mutex aquisition and releases, most > of which you probably have no need of. Python doesn't actually suffer from either of these problems: while there's a pair of acquire/release-global-lock macros around potentially blocking I/O calls in Python's runtime (ditto sleep(), etc), no mutex is actually allocated before *somebody* calls PyEval_InitThreads: void PyEval_InitThreads(void) { if (interpreter_lock) return; _PyThread_Started = 1; interpreter_lock = PyThread_allocate_lock(); PyThread_acquire_lock(interpreter_lock, 1); main_thread = PyThread_get_thread_ident(); } In most uses of Python, no thread other than the main thread ever gets created, that routine never gets called, interpreter_lock remains NULL, and all the global-lock acquire/release code reduces to a cheap test against NULL. However, Python calls the platform's thread-safe libraries regardless, and *that* can be a huge speed hit. A minor example is that system malloc() is more expensive in Microsoft's thread-safe version of libc. A monster example is speed of line-at-a-time input: we only recently discovered that Python's getc()-in-a-loop was killing us on many platforms because the platform threadsafe library implementation locked and unlocked the stream for each character. Worming around that brought our input speed much closer to Perl's (up to 50x faster on Tru64 Unix). It's still slower on most boxes, though, because we're still threadsafe, but, last I looked, Perl's line-at-a-time input tricks mucked with stdio structs directly without benefit of exclusion (and are not threadsafe). > ... > (I do work on SMP machines as a rule, so I am a little biased against > things that single-thread me when I don't need it--what's the point of > 500% idle time?) Greg Stein is the fellow to talk with about about "free threading" of Python. He had that at least mostly working several years ago, but it was a major project, that patch is way out of date now, and Python is much more elegant now. Oops! I didn't mean "elegant", I meant "bigger" . From dan@sidhe.org Wed Aug 1 07:20:10 2001 From: dan@sidhe.org (Dan Sugalski) Date: Wed, 01 Aug 2001 02:20:10 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: References: <5.1.0.14.0.20010731200907.02747f58@24.8.96.48> Message-ID: <5.1.0.14.0.20010801020813.022a56b8@24.8.96.48> At 01:23 AM 8/1/2001 -0400, Tim Peters wrote: >[Paul Prescod] > > What is the downside of the global lock on the average single > > processor machine? I tend to think that the "default" threading model > > should allow simple and easy, everything-shared multi-threading on > > ordinary machines. Having a multi-processor-friendly advanced mode is > > a great extension for the wizards. > >[Dan Sugalski] > > If you hold the lock during an I/O operation, you'll lose time you > > could have otherwise used. Getting and releasing a global lock > > frequently also costs performance you might otherwise have used in other > > places. Mutex releases require memory coherency, which will force your > > CPU to flush any pending writes that might be hanging about, which will > > tend to drop it's efficiency, especially on heavily out-of-order > > machines like the Alpha. > > > > Also, that is a zillion and a half mutex aquisition and releases, most > > of which you probably have no need of. > >Python doesn't actually suffer from either of these problems: while there's >a pair of acquire/release-global-lock macros around potentially blocking I/O >calls in Python's runtime (ditto sleep(), etc), no mutex is actually >allocated before *somebody* calls PyEval_InitThreads: Yeah, I figured you didn't initialize or aquire the mutex until it was actually needed. One of the nice side-benefits of the global opcode-aquired lock. >However, Python calls the platform's thread-safe libraries regardless, and >*that* can be a huge speed hit. A minor example is that system malloc() is >more expensive in Microsoft's thread-safe version of libc. Everywhere else too, I'd bet. I've been considering thread-specific memory pools because of this. (Well, this is one reason, at least) >A monster >example is speed of line-at-a-time input: we only recently discovered that >Python's getc()-in-a-loop was killing us on many platforms because the >platform threadsafe library implementation locked and unlocked the stream >for each character. Ouch, I'd bet that hurts. Has anyone timed the difference between making lots of getc calls and making a few larger reads and managing the buffers internally? I can see it going either way, and another data point would be useful to have. >Worming around that brought our input speed much closer >to Perl's (up to 50x faster on Tru64 Unix). FWIW, at least on Tru64 and VMS, there are a number of faster thread calls if you don't mind bypassing some library error checking. (Which is OK if you're guaranteed to have correct parameters) I can dig them up if you like. A tweak of pthread_get_specific made a depressingly large performance boost for me on my VMS box. (I did say perl's threading models weren't that good... :) Not worth it if you don't make many calls, though. >It's still slower on most >boxes, though, because we're still threadsafe, but, last I looked, Perl's >line-at-a-time input tricks mucked with stdio structs directly without >benefit of exclusion (and are not threadsafe). Yep, perl's pthread-based threading model doesn't guarantee threadsafe I/O on many platforms. The alternative threading model doesn't use that codepath. (It's currently primarily windows-based, and we don't do the buffer lookbehind stuff there) > > ... > > (I do work on SMP machines as a rule, so I am a little biased against > > things that single-thread me when I don't need it--what's the point of > > 500% idle time?) > >Greg Stein is the fellow to talk with about about "free threading" of >Python. He had that at least mostly working several years ago, but it was a >major project, that patch is way out of date now, and Python is much more >elegant now. Oops! I didn't mean "elegant", I meant "bigger" . :) I'm as much (or more, which is generally an odd thing, but threads are almost inherently odd) interested in how things look at the programmer level and what guarantees are made as how things look under the hood. (It's all just a SMOP, right?) Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From tim.one@home.com Wed Aug 1 06:03:18 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 1 Aug 2001 01:03:18 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <5.1.0.14.0.20010731234359.022a1298@24.8.96.48> Message-ID: [Guido van Rossum] > Dan's recent contributions make me a lot less optimistic. > Don't ask me to explain -- I don't have the time, sorry. The other > participants in the thread are doing a good job. > [Dan Sugalski] > I would appreciate it if, when you have a chance, you (or someone) > would clue me in. I've apparently offended, and that was not, in any > way, my intent. My apologies all around. Ah. My primary job in the Python Community is channeling Guido: By "the other participants are doing a good job", he doesn't mean you *aren't* doing a good job. Not at all! He means that they're doing a good enough job of expressing his concerns that he doesn't feel terrible about not having enough time of his own to participate. No offense taken! You're doing a good job too. > I think I've overstayed my welcome, Not at all -- it's a good discussion, and perfectly appropriate for Python-Dev. I wish I had more time for it too. > so I'll bow out here before muddying things up any worse. Please stay. Just try to avoid mentioning Perl, lest you offend Guido again . From mal@lemburg.com Wed Aug 1 08:48:10 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 01 Aug 2001 09:48:10 +0200 Subject: [Python-Dev] Revised decimal type PEP References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073023115207.02466@fermi.eeel.nist.gov> <3B66DE8C.C9C62012@lemburg.com> <01073103271101.02004@fermi.eeel.nist.gov> <200107312326.TAA17540@cj20424-a.reston1.va.home.com> Message-ID: <3B67B43A.D29AF194@lemburg.com> Guido van Rossum wrote: > > > Also, why do you consider a float to be a "larger" value type than decimal? > > Do you mean that a float is less precise? > > (Warning: I think the following is a sound model, but I'm still > practicing how to explain it right.) > > I have this ordering of the types in mind: > > int/long < decimal < rational < float < complex > \---------------------------/ \-------------/ > exact inexact > > This is different from the Scheme numeric "tower" -- I no longer agree > with the Scheme model any more. > > The ordering is only to determine what happens on mixed arithmetic: > the result has the rightmost type in the diagram (or a type further on > the right in some cases). Interesting. Here's what I use in mxNumber: mx.Number.Float ^ | --------> Python float | ^ | | | mx.Number.Rational | ^ | | Python long --> mx.Number.Integer ^ ^ | | -------- Python integer > The ints are a subset of the decimal numbers, and the decimal numbers > (in this view) are a subset of the rational numbers. Ints and > decimals aren't closed under division -- the result of division on > these (in general) is a rational. While the exact values of floats > are a subset of the rationals, the inexactness property (which I give > all floats) makes that each float stands for an infinite set of > numbers *including* the exact value. When a binary operation involves > an exact and an inexact operand, the result is inexact. > > Tim's "numeric context" contains a bunch of flags controlling detailed > behavior of numeric operations. It could specify that mixing exact > and inexact numbers is illegal, and that would be Michael's pedantic > mode. It could also specify warnings. (I would never call a mode > that issues warnings "safe" :-) Could you perhaps write this coercion scheme up as informational PEP ? I think it would help a lot as reference to what Python should do and serve well for numeric extension writers as basis for their coercion decisions. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From mal@lemburg.com Wed Aug 1 08:50:48 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 01 Aug 2001 09:50:48 +0200 Subject: [Python-Dev] Revised decimal type PEP References: <20010801010750.E045299C80@waltz.rahul.net> Message-ID: <3B67B4D8.2B319336@lemburg.com> Aahz Maruch wrote: > > Greg Ewing wrote: > > aahz@rahul.net (Aahz Maruch): > >> > >> Truncation (rounding), overflow, and underflow errors can > >> occur under addition, subtraction, and multiplication. It's trivial to > >> set them to be unbounded, but then Cowlishaw provides no mechanism for > >> determining the truncation of division. > > > > If you allow for the representation of repeating parts in your > > unbounded decimals, they could be closed under division. (I think -- > > does the division of one repeating decimal by another always lead to > > a third repeating decimal? Yes, it must, because every rational can > > be expressed as a repeating decimal and vice versa, IIRC. Hmmm, that > > means we'd just be implementing rationals another way...) > > Tell ya what, *you* write the algorithm and I'll think about > it sometime in the next five years. ;-) Decimals are just a different way of displaying Rationals and depending on their preset precision show a different behaviour in numeric operations. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From mal@lemburg.com Wed Aug 1 08:55:58 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 01 Aug 2001 09:55:58 +0200 Subject: [Python-Dev] Revised decimal type PEP References: <20010731235526.8E8ED99C82@waltz.rahul.net> <200108010251.WAA18015@cj20424-a.reston1.va.home.com> Message-ID: <3B67B60E.201CA0C@lemburg.com> Guido van Rossum wrote: > > > Guido van Rossum wrote: > > > > > > int/long < decimal < rational < float < complex > > > \---------------------------/ \-------------/ > > > exact inexact > > > > Note that in Cowlishaw's implementation of decimal numbers, decimals are > > *not* exact. Truncation (rounding), overflow, and underflow errors can > > occur under addition, subtraction, and multiplication. It's trivial to > > set them to be unbounded, but then Cowlishaw provides no mechanism for > > determining the truncation of division. > > I know all that. Cowlishaw's numbers are a replacement for float (and > for the real/imag parts of complex). I was toying with a *different* > idea, more appropriate perhaps for accountants, that guarantees exact > results for decimal numbers, giving decimal results when possible, > rational otherwise. This is based on my guess about Michael's target > audience. Michael makes a big deal of not allowing "decimalness" to > be contaminated by mixing with floats. Possible this could be > implemented using a subclass of your decimals that uses unbounded > precision but overrides division to produce a rational number if > needed. > > But it's possible that Michael's audience would in fact be very happy > with your/Cowlishaw's decimal numbers. I believe that Rationals should form the basis for Decimal types -- the different possible semantics can then be added either by subclassing Rationals or by providing some context or specialized methods for different operations, e.g. for special rounding requirements like you have in financial applications. Note that I implemented mxNumber for just this reason (the float and long integer support is merely a side effect of GMP providing it ;-). -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From mal@lemburg.com Wed Aug 1 09:11:37 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 01 Aug 2001 10:11:37 +0200 Subject: [Python-Dev] Revised decimal type PEP References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073023115207.02466@fermi.eeel.nist.gov> <3B66DE8C.C9C62012@lemburg.com> <01073103271101.02004@fermi.eeel.nist.gov> Message-ID: <3B67B9B9.37FF7237@lemburg.com> Michael McLay wrote: > > On Tuesday 31 July 2001 12:36 pm, M.-A. Lemburg wrote: > > > I'd suggest to follow the rules for the SQL definitions > > of DECIMAL(,). > > > Well, there are several options. I support that the IBM paper > > on decimal types has good hints as to what the type should do. > > Again, SQL is probably a good source for inspiration too, since > > it deals with decimals a lot. > > Ok, I know about the IBM paper. is there online document on the SQL > semantics that can be referenced in the PEP? The spec is not available online since it is an ISO paper (you pay per page), but I did find a review draft dated July 1992 for SQL-92: http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt which has all the relevant information in text format (section 4.4.). > > I see, the small 'b' still looks funny to me though. Wouldn't > > 1.23f and 25i be more intuitive ? > > I originally used 'f' for both the integer and float. The use of 'b' was > suggested by Guido. There were two reasons not to use 'i' for integers. The > first has to do with how the tokenizer works. It doesn't distringuish > between float and int when the token string is passed to parsenumber(). Both > float and int are processed by the same function. I could have got around > this problem by having the switch statement in parsenumber recognize both 'i' > and 'f', but there is another problem with using 'i'. The 25i would be > confusing for someone if they was trying to use an imaginary numbers If they > accidentally typed 25i instead of 25j they would get an integer instead of an > imaginary number. The error might not be detected since 3.0 + 4i would > evaluate properly. Well, if we manage to have a registry of some sort for these number literal modifiers, I think that the issues in which code to choose are secondary. > > > > I'd rather have this explicit in the sense that you define which > > > > assumptions will be made and what issues arise (rounding, truncation, > > > > loss of precision, etc.). > > > > > > Can you give an example of how this might be implemented. > > > > You would typically first coerce the types to the "larger" > > type, e.g. float + decimal -> float + float -> float, so > > you'd only have to document how the conversion is done and > > which accuracy to expect. > > I would be concerned about the float + decimal automatically generating a > float. Would it generate an error message if the pedantic flag was set? > Would it generate a warning in safe mode? > > Also, why do you consider a float to be a "larger" value type than decimal? > Do you mean that a float is less precise? See Guido's post on this. "Larger" usually means two things: - more precise - larger range This does not necessarily make the decision any easier, though, since the two terms don't discriminate numeric values too well, e.g. how do complex and longs fit into the picture ? As a result, this becomes a language design question. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From m@moshez.org Wed Aug 1 09:37:47 2001 From: m@moshez.org (Moshe Zadka) Date: Wed, 01 Aug 2001 11:37:47 +0300 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: References: Message-ID: [Moshe Zadka] > PUSH "1" > PUSH "2" > BINARY_ADD > > In Python that gives "12". In Perl that gives 3. > Unless you suggest a PERL_BINARY_ADD and a PYTHON_BINARY_ADD, I > don't see how you can around these things. [Tim] > Perl needs distinct flavors of BINARY_ADD for its infix "+" and "." > operators. Once you accept that, there's no real problem here (it simply > means that Python's and Perl's "+" operators would need to map to different > opcodes). And if you call Python's + opcode PYTHON_BINARY_ADD and Perl's PERL_BINARY_ADD, it's exactly what I said, isn't it? Are you agreeing or disagreeing with me? ;-) And more importantly, what do you think it means for the feasibility of such a project? -- gpg --keyserver keyserver.pgp.com --recv-keys 46D01BD6 54C4E1FE Secure (inaccessible): 4BD1 7705 EEC0 260A 7F21 4817 C7FC A636 46D0 1BD6 Insecure (accessible): C5A5 A8FA CA39 AB03 10B8 F116 1713 1BCF 54C4 E1FE Learn Python! http://www.ibiblio.org/obp/thinkCSpy From thomas@xs4all.net Wed Aug 1 09:45:09 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 1 Aug 2001 10:45:09 +0200 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <15207.26855.65000.718694@gargle.gargle.HOWL> References: <20010730051831.B1122@thyrsus.com> <20010731012432.G20676@xs4all.nl> <15206.8708.811000.468489@gargle.gargle.HOWL> <20010731115521.I20676@xs4all.nl> <3B668608.F68B5953@ActiveState.com> <15207.26855.65000.718694@gargle.gargle.HOWL> Message-ID: <20010801104509.D626@xs4all.nl> On Tue, Jul 31, 2001 at 08:26:47PM -0600, Nathan Torkington wrote: > I've been eyeing your 'import from the future' system of getting > experimental features. I'll be interested to see how this holds up. It's really not that different form Perl's "use". They're both scanned for by the parser/tokenizer, they're both used to change the semantics of an operation, and they also double as a normal 'get and use a package' operation (which is, in fact, the primary task of 'import' and 'use' both, right ? :) The main difference is that changing the behaviour of the Python interpreter is a lot less common than changing the behaviour of the Perl one, and Guido wants to keep it that way. The purpously 'magic' spelling of the future-import statement is part of that. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Wed Aug 1 09:48:22 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 1 Aug 2001 10:48:22 +0200 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <5.1.0.14.0.20010731234925.02298ae8@24.8.96.48> Message-ID: <20010801104822.E626@xs4all.nl> On Tue, Jul 31, 2001 at 11:52:01PM -0400, Dan Sugalski wrote: > At 11:46 PM 7/31/2001 -0400, Guido van Rossum wrote: > >Have you looked at the byte code used in SRE, Python's (mostly) > >Perl-compatible regex package? Worth a look! > Nope, not yet, but I will. (You folks will end up making a Python > programmer out of me yet, I expect... :) Well, at least you know it's happening... most people don't realize until it's too late :) > This and the C extension interface are the two big Python things I've got > on my "must peer deeply into" list. Are there other places I should be > concentrating on? (I'm poking about with the interpreter loop and > associated code, but that's the sort of thing an expert with 20 minutes > free will get me much farther than a week or two with the source) What do you say I write up a longish email explaining quickly how Python objects work, and how the Python interpreter and extension code works with them ? I think it'd help with getting us talking about the same thing when we say 'interpreter' or 'type' :-) And the beauty of my being in a different timezone is that noone else is posting right now, so I can take the time without someone else doing it spread out over four posts in separate subthreads :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Wed Aug 1 09:58:13 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 1 Aug 2001 10:58:13 +0200 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Message-ID: <20010801105813.F626@xs4all.nl> On Wed, Aug 01, 2001 at 11:37:47AM +0300, Moshe Zadka wrote: > [Moshe Zadka] > > PUSH "1" > > PUSH "2" > > BINARY_ADD > > > > In Python that gives "12". In Perl that gives 3. > > Unless you suggest a PERL_BINARY_ADD and a PYTHON_BINARY_ADD, I > > don't see how you can around these things. > > [Tim] > > Perl needs distinct flavors of BINARY_ADD for its infix "+" and "." > > operators. Once you accept that, there's no real problem here (it simply > > means that Python's and Perl's "+" operators would need to map to different > > opcodes). > And if you call Python's + opcode PYTHON_BINARY_ADD and Perl's > PERL_BINARY_ADD, it's exactly what I said, isn't it? > Are you agreeing or disagreeing with me? ;-) He's disagreeing. It's not a PERL vs. PYTHON ADD at all; it's a "string concatenation add" vs. "numerical add". Perl code using the string concatenation operator (apparently, it's not going to be ".", which scares me shitless: Perl6 gets a new string concat operator, but it isn't going to be "+" ? If it is going to be '+', how does it flexitype ?) would use the string-concat add opcode, and Perl code and Python code doing '+' would get the normal BINARY_ADD. The string-to-int conversion in Perl's '+' would be put into the 'Scalar' type. Channeling-Tim--probably-wrong--ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From m@moshez.org Wed Aug 1 10:21:52 2001 From: m@moshez.org (Moshe Zadka) Date: Wed, 01 Aug 2001 12:21:52 +0300 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <20010801105813.F626@xs4all.nl> References: <20010801105813.F626@xs4all.nl> Message-ID: On Wed, 1 Aug 2001 10:58:13 +0200, Thomas Wouters wrote: > He's disagreeing. It's not a PERL vs. PYTHON ADD at all; it's a "string > concatenation add" vs. "numerical add". Perl code using the string > concatenation operator (apparently, it's not going to be ".", which scares > me shitless: Perl6 gets a new string concat operator, but it isn't going to > be "+" ? If it is going to be '+', how does it flexitype ?) would use the > string-concat add opcode, and Perl code and Python code doing '+' would get > the normal BINARY_ADD. The string-to-int conversion in Perl's '+' would be > put into the 'Scalar' type. I think I've lost you. But if you're thinking about Perl compiled to something like PUSH "1" CONVERT_TO_INT PUSH "2" CONVERT_TO_INT BINARY_ADD Then it's only a matter of time before the Perl guys find out it's hella slow and optimize it to: PUSH "1" PUSH "2" CONVERT_TO_INT_TOP_2 BINARY_ADD And then to PUSH "1" PUSH "2" CONVERT_TO_INT_TOP_2_AND_BINARY_ADD And then a simple #define CONVERT_TO_INT_TOP_2_AND_BINARY_ADD PERL_ADD Would finish the 2*PI circle to land exactly where we started with. ;-) -- gpg --keyserver keyserver.pgp.com --recv-keys 46D01BD6 54C4E1FE Secure (inaccessible): 4BD1 7705 EEC0 260A 7F21 4817 C7FC A636 46D0 1BD6 Insecure (accessible): C5A5 A8FA CA39 AB03 10B8 F116 1713 1BCF 54C4 E1FE Learn Python! http://www.ibiblio.org/obp/thinkCSpy From thomas@xs4all.net Wed Aug 1 10:53:17 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 1 Aug 2001 11:53:17 +0200 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: References: <20010801105813.F626@xs4all.nl> Message-ID: <20010801115317.F624@xs4all.nl> On Wed, Aug 01, 2001 at 12:21:52PM +0300, Moshe Zadka wrote: > On Wed, 1 Aug 2001 10:58:13 +0200, Thomas Wouters wrote: > > > He's disagreeing. It's not a PERL vs. PYTHON ADD at all; it's a "string > > concatenation add" vs. "numerical add". Perl code using the string > > concatenation operator (apparently, it's not going to be ".", which scares > > me shitless: Perl6 gets a new string concat operator, but it isn't going to > > be "+" ? If it is going to be '+', how does it flexitype ?) would use the > > string-concat add opcode, and Perl code and Python code doing '+' would get > > the normal BINARY_ADD. The string-to-int conversion in Perl's '+' would be > > put into the 'Scalar' type. > I think I've lost you. > But if you're thinking about Perl compiled to something like > PUSH "1" > CONVERT_TO_INT > PUSH "2" > CONVERT_TO_INT > BINARY_ADD No, I'm not. I was, but I'm not. Okay, Moshe, lets try this again. Imagine a Python object, "Scalar", that behaves like a Perl $calar. We're not going to talk about merging the Perl and Python VM here, just about a Python object that behaves somewhat like a Perl scalar, and (for the sake of the discussion) just the string<->int part of the flexitype. The Scalar holds its value as a string, unless it's been stored as an int or 'modified' into an int. Pretend we have an operator "@" to explicitly string-concatenate, exactly like Perl's ".". Now look at this code snippet: one = Scalar("1") two = Scalar("2") three_num = one + two three_s = one @ two "one + two" would simply all BINARY_ADD, and the Scalar type's tp_as_number->nb_add would convert its value internally to an int before handling the operation. "one @ two" would call BINARY_CONCAT (or whatever), which would call a different method to calculate the result. Effectively a different operation. But this isn't "perl add versus python add". "perl add" is just the same as "python add" with a slightly different type, we just get a new opcode for the "perl string concat" operator. Like I (and Eric) said before, there's a lot to be done, and there's bound to be a lot of issues (like how to get *both* scalars to convert their value to a number before arithmatic, above, without resulting to something like PyObject_AsInt() for all binary arithmatic) but those are most likely not insurmountable. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From dan@sidhe.org Wed Aug 1 12:21:47 2001 From: dan@sidhe.org (Dan Sugalski) Date: Wed, 01 Aug 2001 07:21:47 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <20010801104822.E626@xs4all.nl> References: <5.1.0.14.0.20010731234925.02298ae8@24.8.96.48> Message-ID: <5.1.0.14.0.20010801072126.022f2400@24.8.96.48> At 10:48 AM 8/1/2001 +0200, Thomas Wouters wrote: >On Tue, Jul 31, 2001 at 11:52:01PM -0400, Dan Sugalski wrote: > > This and the C extension interface are the two big Python things I've got > > on my "must peer deeply into" list. Are there other places I should be > > concentrating on? (I'm poking about with the interpreter loop and > > associated code, but that's the sort of thing an expert with 20 minutes > > free will get me much farther than a week or two with the source) > >What do you say I write up a longish email explaining quickly how Python >objects work, and how the Python interpreter and extension code works with >them ? I think it'd help with getting us talking about the same thing when >we say 'interpreter' or 'type' :-) And the beauty of my being in a different >timezone is that noone else is posting right now, so I can take the time >without someone else doing it spread out over four posts in separate >subthreads :) That'd be most helpful, thanks. Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From gmcm@hypernet.com Wed Aug 1 14:11:36 2001 From: gmcm@hypernet.com (Gordon McMillan) Date: Wed, 1 Aug 2001 09:11:36 -0400 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: References: <200107310622.CAA11742@cj20424-a.reston1.va.home.com> Message-ID: <3B67C7C8.13312.16DF23AD@localhost> Ka-Ping Yee wrote: [Guido] > > Why not add this to cgi.py instead? > > Because that would seem to defeat most of the point. [snip] > Wouldn't you say there are more Python CGI programmers out there > than, say, Zope developers? Or think of it like this: what > fraction of Web developers are CGI programmers, and what fraction > of those use Python? > > Maybe i'm wrong? I welcome more opinions from others -- how do > you see people coming to Python? What's the first "real" thing > they do with Python that motivates them to try it? Web stuff is certainly a major doorway. Not sure about cgi.py itself. Web stuff accounts for (very, very roughly) 20% of what I do. I have never used cgi.py. In fact, the only web related module in the std lib I use regularly is urllib (mostly for the quote stuff). So neither site.py nor cgi.py would do me any good. (In fact, cgitb.py wouldn't either, since my source is not on the server and sys.std* have nothing to do with requests/responses). Judging by the list at http://www.paul.boddie.net/Python/web_modules.html I don't think I'm all that unusual in this regard. (BTW, line 133 should be filename = tempfile.mktemp('.html') ). - Gordon From gmcm@hypernet.com Wed Aug 1 14:11:36 2001 From: gmcm@hypernet.com (Gordon McMillan) Date: Wed, 1 Aug 2001 09:11:36 -0400 Subject: [Python-Dev] Revised decimal type PEP In-Reply-To: <200108010251.WAA18015@cj20424-a.reston1.va.home.com> References: Your message of "Tue, 31 Jul 2001 16:55:26 PDT." <20010731235526.8E8ED99C82@waltz.rahul.net> Message-ID: <3B67C7C8.5814.16DF229F@localhost> [Guido] > .... I was toying with a > *different* idea, more appropriate perhaps for accountants, that > guarantees exact results for decimal numbers, giving decimal > results when possible, rational otherwise. My experience with accountants (~5 yrs in another life) is that they're perfectly happy with fixed point decimal. They may make you do things to the 1/100 of a penny & then round in funny ways that favor their bosses, but they want every calculation to come out the same as on their calculator. - Gordon From guido@zope.com Wed Aug 1 14:22:22 2001 From: guido@zope.com (Guido van Rossum) Date: Wed, 01 Aug 2001 09:22:22 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Your message of "Wed, 01 Aug 2001 00:42:42 EDT." References: Message-ID: <200108011322.JAA19671@cj20424-a.reston1.va.home.com> [Tim] > But BINARY_ADD knows everything there is to know about Python ints, and > that's an important speed optimization: Which makes it a perfect example IMO: the abstract *semantics* of the opcode are as neutral as can be, and the *implementation* can grease the path for a common case. A complete redesign can still do that. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@zope.com Wed Aug 1 14:25:46 2001 From: guido@zope.com (Guido van Rossum) Date: Wed, 01 Aug 2001 09:25:46 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Your message of "Wed, 01 Aug 2001 00:16:43 EDT." <5.1.0.14.0.20010801000522.0238d970@24.8.96.48> References: <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1909.395000.123189@gargle.gargle.HOWL> <5.1.0.14.0.20010731231748.0228f808@24.8.96.48> <5.1.0.14.0.20010801000522.0238d970@24.8.96.48> Message-ID: <200108011325.JAA19689@cj20424-a.reston1.va.home.com> > The vtable interface, at the moment, covers the > operators you'd expect to be able to override--assignment, simple math, > most string operations, conversion duties, and suchlike things. So this code: > > a = b + c > > would call the add vtable method for b, and the assign vtable method for a, But in Python, this is not an operation on a at all! It's an operation on the namespace containing a, and a cannot override it. I'm sure your VM design can accommodate this, but it points out the fundamental difference between the languages in their ideas of what a "variable" is. Python has at least two types of namespaces in this context: some namespaces (like module globals) are dictionaries, others (like function locals) have mapped the variable names to an array of object references. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@zope.com Wed Aug 1 14:33:37 2001 From: guido@zope.com (Guido van Rossum) Date: Wed, 01 Aug 2001 09:33:37 -0400 Subject: [Python-Dev] Revised decimal type PEP In-Reply-To: Your message of "Wed, 01 Aug 2001 09:50:48 +0200." <3B67B4D8.2B319336@lemburg.com> References: <20010801010750.E045299C80@waltz.rahul.net> <3B67B4D8.2B319336@lemburg.com> Message-ID: <200108011333.JAA19785@cj20424-a.reston1.va.home.com> > Decimals are just a different way of displaying Rationals and > depending on their preset precision show a different behaviour > in numeric operations. Not necessarily. If decimals are used for (in-memory) representation, they can't represent all rationals. --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas@xs4all.net Wed Aug 1 14:36:07 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 1 Aug 2001 15:36:07 +0200 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <5.1.0.14.0.20010731234925.02298ae8@24.8.96.48> Message-ID: <20010801153607.G626@xs4all.nl> On Tue, Jul 31, 2001 at 11:52:01PM -0400, Dan Sugalski wrote: > [SRE] and the C extension interface are the two big Python things I've got > on my "must peer deeply into" list. I can't say much about SRE, but the C extension interface should be on anyone's "must peer deeply into" list, except that there isn't a real need to peer deeply into it :) Lets peer into it. I'm going to discuss the pre-2.2 state of affairs, mostly because I know jack shit about the 2.2-type-dichotomy-healing :) In case you aren't aware, we are currently into the 2.2 alphas, where this story is slightly different because you can now subclass types. If Guido or anyone else involved in the descr-branch wants to tune in and correct the explanation for Python 2.2, that's fine by me. As you might have heard, everything in Python is an object. This is true even for the C side of things. A Python object, any Python object, is essentially a struct like this: stuct PyExample { int ob_refcnt; /* reference count */ struct _typeobject *ob_type; /* type of this object */ /* type-specific data goes here */ } The common header is made available as a #define, so that we can add extra info in severe-debug mode (though this breaks binary compatibility, so it should not be done in production builds.) For instance, the struct to hold a (normal, unbounded) Python integer is typedef struct { PyObject_HEAD long ob_ival; } PyIntObject; The struct _typeobject pointer is a pointer to a Type object, which also acts as a vtable. It contains all the information about the type, including how to operate on it. The typeobject is itself also an object! Its type is the PyType type, which is its own type. The typeobject is actually what you get back when you do 'type(object)' in Python code. All this is also explained in Include/object.h, by the way. The type object holds things like the type name, how to print it, howmuch memory should be allocated, a couple of flags to implement binary compatibility, and a whole boatload of function pointers. The function pointers for operations are spread out over the type struct and several separate structs, to which the type struct points: tp_as_number, tp_as_sequence, and tp_as_mapping. Each of these can be NULL if the type doesn't implement the operations. Some of the operations that are defined on the type struct itself are get-attribute, set-attribute, comparison, get-hashvalue, convert-to-string and call-as-function. The PyNumberMethods struct (tp_as_number) contains arithmatic operations (such as adding, mulitplying, shifting, XORing, and in-place variations of each for use with augmented assignment), truth-value-testing, and converting to various other integer types. The PySequenceMethods struct (tp_as_sequence) contains sequence operations such as membership-test, item- and slice-retrieval and -assignment, and sequence concatenation and repetition. The latter two are somewhat of an odd duck, since they implement the same Python operation ("+" and "*") as the 'add' and 'multiply' methods from the PyNumberMethods struct. They're mainly so the Python runtime can properly handle "number * list" as well as "list * number" without attempting to convert list to an integer. Lastly there's PyMappingMethods. I'm not at all sure why this is a separate sturct, as the three operations it contains (length inquiry, item-retrieval and item-assignment) are also covered by the PySequenceMethods, but I suspect some kind of hysterical raisin crept in here :) To see this construct in action, take a look at Objects/intobject.c, in the Python source tree (doesn't really matter which version.) It defines a bunch of functions to do the arithmatic, fills a PyNumberMethods struct with the appropriate pointers (leaving unsupported ones NULL), and fills a new PyTypeObject with the right info for the "int" type. To contrast, the PyFunction type is nearly empty. It defines getattr and setattr, and a 'repr' function (convert to string). You can't compare it (it will always compare false to anything but itself), add it, index it, or get its length, but you can get attributes from it. And the average function has some interesting attributes: you can, from Python code and C code alike, easily get at things like the bytecode it consists of, the number of arguments it expects, the names of those arguments, etc. Each of those is, of course, a Python object in its own right :) Function objects are defined in Objects/funcobject.c, though the actual execution of function objects is done elsewhere. An 'instance' object is a slightly different case in this story (more so in the post-2.2 story, but I'm not sure howmuch more.) An instance object is a Python class, defined in Python code by the user/programmer. It can choose to implement any of the above operations by defining magically named methods on the class. For instance, to implement the 'getitem' protocol, it defines a function __getitem__, and to implement addition, __add__. The Type object for instance objects acts as a proxy for the Python class, though this does represent some problems: instance objects need to be treated in some areas of the interpreter (but this is almost certain to have been changed in 2.2.) But back to generic Python objects. Obviously, when writing C code, you don't want to manipulate Python objects by directly accessing all those layers of C structs all the time. So each builtin type defines useful C functions and macros to help. For instance, to create a new list, you call PyList_New(size). To test whether something is a list, PyList_Check(object). To index a list, PyList_GetItem(list, index), or a macro version of that, PyList_GET_ITEM. The macro version doesn't do type or boundary checks, so it should only be used when you already did those checks. Likewise, there is PyInt_AsLong (convert a PyInt to a C 'long') and PyInt_AS_LONG. But even that isn't that helpful in a dynamically typed language, since you have to know an object's type in order to call the type-specific functions. So we also have two more abstraction layers. The most abstract one is the Object layer. Basically, regardless of the type of object, you can always manipulate it using PyObject_*() functions. Some PyObject_ functions have a direct relation to Python code, others are only useful to C code. There are PyObject_ functions such as PyObject_IsTrue, PyObject_Compare, PyObject_GetAttr, PyObject_CallObject, etcetera. All these work regardless of the type. If the type (or instance) does not implement the operation, an exception is raised (see below.) The other layer is the slightly less abstract layer that differentiates in the same way as the PyNumberMethods, PySequenceMethods, PyMappingMethods structs. To add two objects, you call PyNumber_Add(o1, o2). To explicitly concatenate two objects, you call PySequence_Concat(o1, o2). All of these work for any type that defines the behaviour, including instance types. Note that calling one of these functions can end up executing arbitrary Python code, so they shouldn't be called in any time-critical situation :-) As a last part of the API, there is the reference counting. As I showed a couple of pages back, all objects contain a reference count. Operations that copy a reference should increment the reference count, using Py_INCREF, and deleting a reference should be done using Py_DECREF. The latter also does the deallocation and cleanup if the DECREF causes the refcount to drop to 0. All API functions that return a reference define whether they return a new reference, which the caller has to DECREF when throwing it away, or a borrowed reference, which shouldn't be DECREF'ed, or kept around without INCREFing it. Exceptions, by the way, are set by setting a global (or thread-local, in a threaded interpreter) variable to contain the Python exception object, and then returning an error flag (either NULL, -1 or 0, depending on the function) to the caller. All code should check all return values and always return an error value if a called function does so, unless it's prepared to handle the exception itself. To compare errors with a particular error 'class', there is PyErr_ExceptionMatches (which also handles 'subclasses' of exception types.) If the error value returned by a particular function is also a real possible value (such as the value returned by PyNumber_AsLong), there is PyErr_Occured() to see if it was a real exception, or a legitimate -1 value. So that's it for the API. All Python objects can be manipulated, created and destroyed this way. Most extension types provide their functionality either in builtin-operations (in the type object) or by providing methods that can be called (using PyObject_GetAttr and PyObject_CallObject), and only rarely need to export an explicit API of their own. (And as you probably know, exporting an API from a shared library to another shared library is... slightly tricky :) Now, the Python bytecode-interpreter itself is very small. All it does is interpret bytecode, and then call out to the API to make it do the actual work. It has some special knowledge about some types, for optimzation, and it does all the real exception handling and namespace handling (name lookup and such), but the actual manipulation of objects is left to the API. Even so, it has enough to do. In the case of exceptions, it has to search up the function for a 'catching' block (a try/except or try/finally statement), or break out of the function to let the error percolate to the caller. Namespaces are a complete story in their own right. Python has three distinct namespaces: the 'local' namespace inside a function (or 'code block'), the 'global' or 'module-level' namespace at the toplevel of a file, and the 'builtin' namespace which holds builtin functions and names. The search order is always local->global->builtin, though for code that is executed at the module level (not inside a function), local is the same as global. In Python, variables are defined by assignment. As a consequence, any variable that is assigned to in a code block, is local to that block. Any variable that isn't assigned to but is just referenced thus has to be defined in the global namespace. The compiler keeps track of this, and generates 'LOAD_FAST' (for loading from the local namespace, inside a function), 'LOAD_GLOBAL' (for loading from the global or builtin namespace, skipping the local namespace), and 'LOAD_NAME' (for occurances where it's truly not known where a variable comes from, and both local and global namespaces have to be searched.) The latter is necessary because Python has a few operations that can modify a local namespace at runtime (such as 'from module import *', and 'exec'.) Since Python 2.1, Python has 'nested scopes', which makes code blocks defined inside other code blocks slightly special: the 'parent' scope is also searched for any variable that isn't local to the nested scope. Most of this, but not all, is done at compiletime, IIRC. (Jeremy can correct me if I'm wrong :) I tried to make the distinction between a 'function' and a 'code block', since the VM really deals with the second, not the first. I suspect that the current Python VM's mechanics to deal with nested scopes could be generalized so that Perl's lexical closures could fit as well (I assume that won't change in perl6 :). Then again, Perl already knows where a variable comes from, so the VM will probably want to put that to good use... The main reason Python does the name-searching at runtime is that a global namespace can be altered from outside the currently-executing codeblock (threads, or other modules, can do 'module.range = my_fake_range', and suddenly the range() function does something completely different throughout the entire module.) A good overview of how the VM uses the API can be obtained by looking at Python/ceval.c, and looking for "switch (opcode)". You'll end up at the top of the main bytecode switch. Most of the opcode names are pretty descriptive, but if you want a better overview, take a look at the 'dis' module's documentation. (www.python.org is still down, but you can use one of the mirrors: http://python.mirrors.netnumina.com/doc/current/lib/module-dis.html http://python.mirrors.netnumina.com/doc/current/lib/bytecodes.html And an easy way to see how a Python operation is actually implemented is using the 'dis' module interactively: >>> import dis >>> def spam(x): ... print "Hello Perl World!" >>> dis.dis(spam) 0 SET_LINENO 1 3 SET_LINENO 2 6 SETUP_LOOP 44 (to 53) >> 9 SET_LINENO 2 12 LOAD_FAST 0 (x) 15 JUMP_IF_FALSE 33 (to 51) 18 POP_TOP 19 SET_LINENO 3 22 LOAD_CONST 1 ('Hello ') 25 LOAD_CONST 2 ('Perl ') 28 BINARY_ADD 29 LOAD_CONST 3 ('World!') 32 BINARY_ADD 33 PRINT_ITEM 34 PRINT_NEWLINE 35 SET_LINENO 4 38 LOAD_FAST 0 (x) 41 LOAD_CONST 4 (1) 44 INPLACE_SUBTRACT 45 STORE_FAST 0 (x) 48 JUMP_ABSOLUTE 9 >> 51 POP_TOP 52 POP_BLOCK >> 53 LOAD_CONST 0 (None) 56 RETURN_VALUE Keep in mind that this is stack-based, and all operations manipulate the stack. (for instance, LOAD_FAST pushes the variable onto the stack, and BINARY_ADD pops two values, adds them, and pushes the result.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido@zope.com Wed Aug 1 14:48:30 2001 From: guido@zope.com (Guido van Rossum) Date: Wed, 01 Aug 2001 09:48:30 -0400 Subject: [Python-Dev] Revised decimal type PEP In-Reply-To: Your message of "Wed, 01 Aug 2001 09:48:10 +0200." <3B67B43A.D29AF194@lemburg.com> References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073023115207.02466@fermi.eeel.nist.gov> <3B66DE8C.C9C62012@lemburg.com> <01073103271101.02004@fermi.eeel.nist.gov> <200107312326.TAA17540@cj20424-a.reston1.va.home.com> <3B67B43A.D29AF194@lemburg.com> Message-ID: <200108011348.JAA19995@cj20424-a.reston1.va.home.com> > Could you perhaps write this coercion scheme up as informational > PEP ? I think it would help a lot as reference to what Python should > do and serve well for numeric extension writers as basis for their > coercion decisions. I'm not sure which coercion scheme you are referring to. I'm also overwhelmed with other stuff so I think I'll pass, but I'll gladly help steer a draft by someone else in the right direction. --Guido van Rossum (home page: http://www.python.org/~guido/) From mclay@nist.gov Wed Aug 1 02:50:57 2001 From: mclay@nist.gov (Michael McLay) Date: Tue, 31 Jul 2001 21:50:57 -0400 Subject: [Python-Dev] Revised decimal type PEP In-Reply-To: <200107312326.TAA17540@cj20424-a.reston1.va.home.com> References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073103271101.02004@fermi.eeel.nist.gov> <200107312326.TAA17540@cj20424-a.reston1.va.home.com> Message-ID: <01073121505701.02964@fermi.eeel.nist.gov> On Tuesday 31 July 2001 07:26 pm, Guido van Rossum wrote: > > Also, why do you consider a float to be a "larger" value type than > > decimal? Do you mean that a float is less precise? > > (Warning: I think the following is a sound model, but I'm still > practicing how to explain it right.) > > > I have this ordering of the types in mind: > > int/long < decimal < rational < float < complex > \---------------------------/ \-------------/ > exact inexact > > This is different from the Scheme numeric "tower" -- I no longer agree > with the Scheme model any more. > > The ordering is only to determine what happens on mixed arithmetic: > the result has the rightmost type in the diagram (or a type further on > the right in some cases). > > The ints are a subset of the decimal numbers, and the decimal numbers > (in this view) are a subset of the rational numbers. Ints and > decimals aren't closed under division -- the result of division on > these (in general) is a rational. While the exact values of floats > are a subset of the rationals, the inexactness property (which I give > all floats) makes that each float stands for an infinite set of > numbers *including* the exact value. When a binary operation involves > an exact and an inexact operand, the result is inexact. Hmm, am I understanding you explaination? Here is a rational expression: 9/4 * 4/3 =3D 3 With floats this ends up being close, but with rounding errors. >>> 2.25*1.333333 2.9999992500000001 >>> If this is expressed as a product of 2.25b * 1.333333d the result would b= e an=20 inexact value. A binary number would be returned, instead of the decimal= =20 number 3. > Tim's "numeric context" contains a bunch of flags controlling detailed > behavior of numeric operations. It could specify that mixing exact > and inexact numbers is illegal, and that would be Michael's pedantic > mode. It could also specify warnings. (I would never call a mode > that issues warnings "safe" :-) Where is Tim's "numeric context" located? =20 M.-A. Lemburg" suggested looking at the SQL specification for Decimal=20 datatypes. A decimal type is also defined as a type in XML Schema. Sinc= e=20 this is an XML datatype there isn't a definition for how these numbers ar= e=20 created. NOTE: All =B7minimally conforming=B7 processors =B7must=B7 support deci= mal numbers with a minimum of 18 decimal digits (i.e., with a =B7totalDigits=B7 of = 18). However, =B7minimally conforming=B7 processors =B7may=B7 set an applica= tion-defined limit on the maximum number of decimal digits they are prepared to supp= ort, in which case that application-defined maximum number =B7must=B7 be cle= arly documented. - http://www.w3.org/TR/xmlschema-2/#decimal From guido@zope.com Wed Aug 1 15:55:02 2001 From: guido@zope.com (Guido van Rossum) Date: Wed, 01 Aug 2001 10:55:02 -0400 Subject: [Python-Dev] Revised decimal type PEP In-Reply-To: Your message of "Tue, 31 Jul 2001 21:50:57 EDT." <01073121505701.02964@fermi.eeel.nist.gov> References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073103271101.02004@fermi.eeel.nist.gov> <200107312326.TAA17540@cj20424-a.reston1.va.home.com> <01073121505701.02964@fermi.eeel.nist.gov> Message-ID: <200108011455.KAA20439@cj20424-a.reston1.va.home.com> > Hmm, am I understanding you explaination? > > Here is a rational expression: > > 9/4 * 4/3 = 3 > > With floats this ends up being close, but with rounding errors. > > >>> 2.25*1.333333 > 2.9999992500000001 > >>> > > If this is expressed as a product of 2.25b * 1.333333d the result would be an > inexact value. A binary number would be returned, instead of the decimal > number 3. Correct. > Where is Tim's "numeric context" located? In his mind. :-) I believe it is typically global per thread, but that's up to the langage binding. A Java binding for Cowlishaw's decimals apparently requires passing in a context as a third argument on each operation. > M.-A. Lemburg" suggested looking at the SQL specification for Decimal > datatypes. A decimal type is also defined as a type in XML Schema. Since > this is an XML datatype there isn't a definition for how these numbers are > created. Do these say anything about semantics under numeric operations? That would seem to be outside the realm of XML and possibly even outside SQL. So I'm not sure how these help. > NOTE: All ·minimally conforming· processors ·must· support decimal numbers > with a minimum of 18 decimal digits (i.e., with a ·totalDigits· of 18). > However, ·minimally conforming· processors ·may· set an application-defined > limit on the maximum number of decimal digits they are prepared to support, > in which case that application-defined maximum number ·must· be clearly > documented. > - http://www.w3.org/TR/xmlschema-2/#decimal I followed the URL and found only external representation issues, nothing that can help us. --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Wed Aug 1 16:16:41 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 1 Aug 2001 11:16:41 -0400 (EDT) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <200108011325.JAA19689@cj20424-a.reston1.va.home.com> References: <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1909.395000.123189@gargle.gargle.HOWL> <5.1.0.14.0.20010731231748.0228f808@24.8.96.48> <5.1.0.14.0.20010801000522.0238d970@24.8.96.48> <200108011325.JAA19689@cj20424-a.reston1.va.home.com> Message-ID: <15208.7513.169067.995475@slothrop.digicool.com> >>>>> "GvR" == Guido van Rossum writes: GvR> But in Python, this is not an operation on a at all! It's an GvR> operation on the namespace containing a, and a cannot override GvR> it. I'm sure your VM design can accommodate this, but it GvR> points out the fundamental difference between the languages in GvR> their ideas of what a "variable" is. Python has at least two GvR> types of namespaces in this context: some namespaces (like GvR> module globals) are dictionaries, others (like function locals) GvR> have mapped the variable names to an array of object GvR> references. In the future, I expect module namespaces will be implemented more like function namespaces. That's the "low-hanging fruit" optimization idea we've kicked around for a while. I hope to work on it for 2.3. This discussion of variables also raises a question for the Perl guys: Is there a Perl language reference ala the Python reference manual? Jeremy From dan@sidhe.org Wed Aug 1 16:22:11 2001 From: dan@sidhe.org (Dan Sugalski) Date: Wed, 01 Aug 2001 11:22:11 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <15208.7513.169067.995475@slothrop.digicool.com> References: <200108011325.JAA19689@cj20424-a.reston1.va.home.com> <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1909.395000.123189@gargle.gargle.HOWL> <5.1.0.14.0.20010731231748.0228f808@24.8.96.48> <5.1.0.14.0.20010801000522.0238d970@24.8.96.48> <200108011325.JAA19689@cj20424-a.reston1.va.home.com> Message-ID: <5.1.0.14.0.20010801112115.0240c7a8@24.8.96.48> At 11:16 AM 8/1/2001 -0400, Jeremy Hylton wrote: >This discussion of variables also raises a question for the Perl guys: >Is there a Perl language reference ala the Python reference manual? Not for perl 6, nope. (Larry's working on that) The Camel and the implementation are pretty much what we've got for perl 5. Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From mal@lemburg.com Wed Aug 1 17:54:35 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 01 Aug 2001 18:54:35 +0200 Subject: [Python-Dev] Revised decimal type PEP References: <20010801010750.E045299C80@waltz.rahul.net> <3B67B4D8.2B319336@lemburg.com> <200108011333.JAA19785@cj20424-a.reston1.va.home.com> Message-ID: <3B68344B.9BFBB666@lemburg.com> Guido van Rossum wrote: > > > Decimals are just a different way of displaying Rationals and > > depending on their preset precision show a different behaviour > > in numeric operations. > > Not necessarily. If decimals are used for (in-memory) representation, > they can't represent all rationals. I know, that's what I wanted to say with "depending on their preset precision"; if you have a decimal with precision 2 then 1/3 will come out as "0.33" -- my point was that by implementing a Rational type instead of a decimal type we can have all semantics of decimals by simply subclassing the Rational type. This setup will be more flexible than choosing one of the many different variants of "how to add two decimal numbers" (e.g. calculator style, with hidden extra digits, with truncation, mathematical rounding, (one of the various) financial rounding modes, etc.). -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From m@moshez.org Wed Aug 1 18:01:32 2001 From: m@moshez.org (Moshe Zadka) Date: Wed, 01 Aug 2001 20:01:32 +0300 Subject: [Python-Dev] Revised decimal type PEP In-Reply-To: <200108011333.JAA19785@cj20424-a.reston1.va.home.com> References: <200108011333.JAA19785@cj20424-a.reston1.va.home.com>, <20010801010750.E045299C80@waltz.rahul.net> <3B67B4D8.2B319336@lemburg.com> Message-ID: [Marc] > Decimals are just a different way of displaying Rationals and > depending on their preset precision show a different behaviour > in numeric operations. [Guido] > Not necessarily. If decimals are used for (in-memory) representation, > they can't represent all rationals. Different definitions of decimals? Marc was thinking repeating decimals, if I understood him correctly. -- gpg --keyserver keyserver.pgp.com --recv-keys 46D01BD6 54C4E1FE Secure (inaccessible): 4BD1 7705 EEC0 260A 7F21 4817 C7FC A636 46D0 1BD6 Insecure (accessible): C5A5 A8FA CA39 AB03 10B8 F116 1713 1BCF 54C4 E1FE Learn Python! http://www.ibiblio.org/obp/thinkCSpy From fdrake@acm.org Wed Aug 1 18:23:20 2001 From: fdrake@acm.org (Fred L. Drake) Date: Wed, 1 Aug 2001 13:23:20 -0400 (EDT) Subject: [Python-Dev] [development doc updates] Message-ID: <20010801172320.BE5A72879B@cj42289-a.reston1.va.home.com> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ Incremental updates, including some useful improvements in the docs for the re module. From DavidA@ActiveState.com Wed Aug 1 18:32:22 2001 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 01 Aug 2001 10:32:22 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? References: <200108011325.JAA19689@cj20424-a.reston1.va.home.com> <20010730051831.B1122@thyrsus.com> <15206.51329.561652.565480@beluga.mojam.com> <200107311708.NAA16497@cj20424-a.reston1.va.home.com> <15206.61296.958360.72700@beluga.mojam.com> <200107311900.PAA17062@cj20424-a.reston1.va.home.com> <15207.1909.395000.123189@gargle.gargle.HOWL> <5.1.0.14.0.20010731231748.0228f808@24.8.96.48> <5.1.0.14.0.20010801000522.0238d970@24.8.96.48> <200108011325.JAA19689@cj20424-a.reston1.va.home.com> <5.1.0.14.0.20010801112115.0240c7a8@24.8.96.48> Message-ID: <3B683D26.5ECEFAF2@ActiveState.com> Dan Sugalski wrote: > > At 11:16 AM 8/1/2001 -0400, Jeremy Hylton wrote: > >This discussion of variables also raises a question for the Perl guys: > >Is there a Perl language reference ala the Python reference manual? > > Not for perl 6, nope. (Larry's working on that) The Camel and the > implementation are pretty much what we've got for perl 5. For Perl5, don't forget perlguts and perlguts illustrated, which may be useful to Pythonistas: http://aspn.activestate.com/ASPN/Products/ActivePerl/lib/Pod/perlguts.html http://gisle.aas.no/perl/illguts/ --david (don't blame me!) ascher From Samuele Pedroni Wed Aug 1 20:03:33 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Wed, 1 Aug 2001 21:03:33 +0200 (MET DST) Subject: [Python-Dev] Parrot -- should life imitate satire? Message-ID: <200108011903.VAA06122@core.inf.ethz.ch> Hi. > > The point is to put the commonly called things in the vtable in a way that > you can avoid as much conditional code as possible, while less common > things get dealt with in ways you'd generally expect. (Dynamic lookups with > caching and suchlike things) If I'm right, you're designing a object-based VM. I don't how typical is OO programming in Perl, but in Python that plays a central role, if your long run goal is to compile to native code you should have a "hard-wired" concept of classes and them like, because an operation like getting the class of an instance should be as direct and fast as possible if you want to use any of the "nice" optimization for a VM for a OO dynamic language: inline polymorphic caches customization or if/when possible: direct inlining. etc. If any of the high-level OO system can not be used, you have to choose a lower level one and map these to it, a vtable bytecode mapping model is too underspecified. regards, Samuele Pedroni. From mclay@nist.gov Wed Aug 1 09:26:58 2001 From: mclay@nist.gov (Michael McLay) Date: Wed, 1 Aug 2001 04:26:58 -0400 Subject: [Python-Dev] Revised decimal type PEP In-Reply-To: <200108011455.KAA20439@cj20424-a.reston1.va.home.com> References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073121505701.02964@fermi.eeel.nist.gov> <200108011455.KAA20439@cj20424-a.reston1.va.home.com> Message-ID: <01080104265804.02964@fermi.eeel.nist.gov> On Wednesday 01 August 2001 10:55 am, Guido van Rossum wrote: > > I believe it is typically global per thread, but that's up to the > langage binding. A Java binding for Cowlishaw's decimals apparently > requires passing in a context as a third argument on each operation. I'm trying to understand how a decimal number context would work. Is the= =20 context a variable and/or flag that defines the rounding rules and precis= ion=20 of a number when it is used in a calculation? How is it associated with = a=20 number or a calculation? The "global per thread" description seems to=20 associate the context with threads. Can the context be altered inside th= e=20 thread? Is it possible to change the context at different levels in a=20 stackframe?=20 I would assume there is a default context will be used until the context = is=20 changed. If this is the case I would expect a default context would be=20 defined at startup. Would it make sense to have a simple decimal type with no features that c= an=20 be modified (a fixed context)? This simple type could be extended by deri= ving=20 a new numerical type from the base decimal type. This base decimal type=20 would be targeted at the newbie user. It would have no surprises. It wo= uld=20 have a default precision of 18 and the rules for rounding would emulate t= he=20 typical hand held calculator. Accountants who need special rounding rules= =20 would use a derived type that allowed the default rules to be overridden. It would be possible to round numbers of the simple based type, but it wo= uld=20 be an explicit step to remove insignificant digits. An accounting decima= l=20 type might automatically round calculations to the smallest denomination.= =20 For instance, an accounting context might have automatically managed the=20 final rounding in the following calculation: p>>> quantity =3D 6 >>> tax =3D .06 >>> price =3D 2.99 >>> total =3D price * quantity * (1 + tax) >>> total 19.0164 >>> round(total,2) 19.02 >>> > > M.-A. Lemburg" suggested looking at the SQL specification for Decima= l > > datatypes. A decimal type is also defined as a type in XML Schema.=20 > > Since this is an XML datatype there isn't a definition for how these > > numbers are created. > > Do these say anything about semantics under numeric operations? That > would seem to be outside the realm of XML and possibly even outside > SQL. So I'm not sure how these help. You are correct that it doesn't deal with numeric operations. It does de= fine=20 a minimum precision requirement. I am only referencing it here because i= t is=20 another instance where having a decimal type in Python would be useful an= d=20 because they have set a minimum requirement. Setting this minmum as a=20 default behavior would probably make newbies comfortable with the languag= e. =20 > > NOTE: All =B7minimally conforming=B7 processors =B7must=B7 support = decimal > > numbers with a minimum of 18 decimal digits (i.e., with a =B7totalDig= its=B7 > > of 18). However, =B7minimally conforming=B7 processors =B7may=B7 set = an > > application-defined limit on the maximum number of decimal digits the= y > > are prepared to support, in which case that application-defined maxim= um > > number =B7must=B7 be clearly documented. > > - http://www.w3.org/TR/xmlschema-2/#decimal From guido@zope.com Wed Aug 1 22:44:37 2001 From: guido@zope.com (Guido van Rossum) Date: Wed, 01 Aug 2001 17:44:37 -0400 Subject: [Python-Dev] Revised decimal type PEP In-Reply-To: Your message of "Wed, 01 Aug 2001 04:26:58 EDT." <01080104265804.02964@fermi.eeel.nist.gov> References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073121505701.02964@fermi.eeel.nist.gov> <200108011455.KAA20439@cj20424-a.reston1.va.home.com> <01080104265804.02964@fermi.eeel.nist.gov> Message-ID: <200108012144.RAA27076@cj20424-a.reston1.va.home.com> > I'm trying to understand how a decimal number context would work. Is the > context a variable and/or flag that defines the rounding rules and precision > of a number when it is used in a calculation? Yes -- I believe advanced HP calculators have such functionality. So does IEEE 754 for binary floating point. > How is it associated with a > number or a calculation? As I wrote, this depends on the language binding. I believe it's associated with a calculation, not with a number. > The "global per thread" description seems to > associate the context with threads. Can the context be altered inside the > thread? Is it possible to change the context at different levels in a > stackframe? Again, this would depend on the language. I believe typically you can change it but it's not stacked (you'd have to do that yourself). > I would assume there is a default context will be used until the context is > changed. If this is the case I would expect a default context would be > defined at startup. Me too. > Would it make sense to have a simple decimal type with no features that can > be modified (a fixed context)? That's like providing IEEE 754 floating point without the controls. That's what C does, but at times it's painful. For MOST users this would be fine, but for advanced use you need the control, and claiming "IEEE 754 std" is unfair without the controls. > This simple type could be extended by deriving > a new numerical type from the base decimal type. This base decimal type > would be targeted at the newbie user. It would have no surprises. It's hard to avoid surprises of the kind (1/3)*3 != 1. My calculator gives 0.99999999, but it's still a surprise. On the other hand for someone who thinks they know how a calculator does it, returning 1 would be the surprise! What kind of surprises do you specifically want to avoid? > It would > have a default precision of 18 and the rules for rounding would emulate the > typical hand held calculator. Accountants who need special rounding rules > would use a derived type that allowed the default rules to be overridden. > > It would be possible to round numbers of the simple based type, but it would > be an explicit step to remove insignificant digits. An accounting decimal > type might automatically round calculations to the smallest denomination. > For instance, an accounting context might have automatically managed the > final rounding in the following calculation: > > p>>> quantity = 6 > >>> tax = .06 > >>> price = 2.99 > >>> total = price * quantity * (1 + tax) > >>> total > 19.0164 > >>> round(total,2) > 19.02 > >>> Looks good to me. This would be a nice goal to strive for. > > > M.-A. Lemburg" suggested looking at the SQL specification for > > > Decimal datatypes. A decimal type is also defined as a type in > > > XML Schema. Since this is an XML datatype there isn't a > > > definition for how these numbers are created. > > > > Do these say anything about semantics under numeric operations? > > That would seem to be outside the realm of XML and possibly even > > outside SQL. So I'm not sure how these help. > > You are correct that it doesn't deal with numeric operations. It > does define a minimum precision requirement. I am only referencing > it here because it is another instance where having a decimal type > in Python would be useful and because they have set a minimum > requirement. Setting this minmum as a default behavior would > probably make newbies comfortable with the language. Good point. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@zope.com Wed Aug 1 22:49:19 2001 From: guido@zope.com (Guido van Rossum) Date: Wed, 01 Aug 2001 17:49:19 -0400 Subject: [Python-Dev] Revised decimal type PEP In-Reply-To: Your message of "Wed, 01 Aug 2001 17:44:37 EDT." <200108012144.RAA27076@cj20424-a.reston1.va.home.com> References: <0107301106520A.02216@fermi.eeel.nist.gov> <01073121505701.02964@fermi.eeel.nist.gov> <200108011455.KAA20439@cj20424-a.reston1.va.home.com> <01080104265804.02964@fermi.eeel.nist.gov> <200108012144.RAA27076@cj20424-a.reston1.va.home.com> Message-ID: <200108012149.RAA27144@cj20424-a.reston1.va.home.com> > > I'm trying to understand how a decimal number context would work. > > Is the context a variable and/or flag that defines the rounding > > rules and precision of a number when it is used in a calculation? > > Yes -- I believe advanced HP calculators have such functionality. So > does IEEE 754 for binary floating point. I forgot: also which exceptional conditions raise exceptions, and a bunch of (resettable?) state flags that tell you whether the last calculation did certain things like lose precision or overflow etc. I know nothing about this stuff, but Tim talks about it a lot and I seem to learn by osmosis. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From greg@cosc.canterbury.ac.nz Thu Aug 2 00:09:12 2001 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 02 Aug 2001 11:09:12 +1200 (NZST) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <5.1.0.14.0.20010801000522.0238d970@24.8.96.48> Message-ID: <200108012309.LAA00931@s454.cosc.canterbury.ac.nz> Dan Sugalski : > In this case, the vtable interface doesn't cover general method > calls ... The vtable interface, at the moment, covers the > operators you'd expect to be able to override--assignment, simple math, > most string operations, conversion duties, and suchlike things. It sounds like a vtable is very similar to a Python TypeObject, the main difference being that TypeObjects are first-class objects. TypeObjects are immutable, though - you can't use Python code to change one of it's method slots. Whereas, if I understand you correctly, you can use Perl code to change a method in a vtable? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.one@home.com Thu Aug 2 05:28:47 2001 From: tim.one@home.com (Tim Peters) Date: Thu, 2 Aug 2001 00:28:47 -0400 Subject: [Python-Dev] descr-branch is dead; long live the trunk Message-ID: I completed merging descr-branch back into the trunk. Checkins should no longer occur on descr-branch. The trunk was tagged before and after the merge: before tag: date2001-08-01 after tag: after-descr-branch-merge After a few quiet months have passed, I intend to get rid of the assorted dateYYYY-MM-DD tags added for repeated merging of the trunk back into the branch. One problem: test_weakref.py fails now. It didn't fail in the trunk before the merge, or in descr-branch. All the weakref files (_weakref.c, weakref.py, test_weakref.py) are the same in both trunk and branch, so there's no obvious accounting for this. Most of test_weakref works fine: C:\Code\python\dist\src\PCbuild>python ../lib/test/test_weakref.py test_basic_callback (__main__.ReferencesTestCase) ... FAIL test_basic_proxy (__main__.ReferencesTestCase) ... ok test_basic_ref (__main__.ReferencesTestCase) ... ok test_callable_proxy (__main__.ReferencesTestCase) ... ok test_getweakrefcount (__main__.ReferencesTestCase) ... ok test_getweakrefs (__main__.ReferencesTestCase) ... ok test_multiple_callbacks (__main__.ReferencesTestCase) ... ok Make sure all references are invalidated before callbacks ... ok test_proxy_ref (__main__.ReferencesTestCase) ... ok test_proxy_reuse (__main__.ReferencesTestCase) ... ok test_ref_reuse (__main__.ReferencesTestCase) ... ok ====================================================================== FAIL: test_basic_callback (__main__.ReferencesTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "../lib/test/test_weakref.py", line 52, in test_basic_callback self.check_basic_callback(create_unbound_method) File "../lib/test/test_weakref.py", line 114, in check_basic_callback "callback did not properly set 'cbcalled'") File "C:\CODE\PYTHON\DIST\SRC\lib\unittest.py", line 249, in failUnless if not expr: raise self.failureException, msg AssertionError: callback did not properly set 'cbcalled' ---------------------------------------------------------------------- Ran 11 tests in 0.000s FAILED (failures=1) Traceback (most recent call last): File "../lib/test/test_weakref.py", line 410, in ? run_unittest(ReferencesTestCase) File "../lib/test\test_support.py", line 146, in run_unittest raise TestFailed(str(err[1])) test_support.TestFailed: callback did not properly set 'cbcalled' C:\Code\python\dist\src\PCbuild> Fred, would you take a look at that and see whether anything punches you in the eye? CVS made the merge back into the trunk much harder than it should have been, and I don't to want hold things up for what's *probably* a shallow problem I'm just too fried now to pick up on. gamblingly y'rs - tim From tim.one@home.com Thu Aug 2 05:46:51 2001 From: tim.one@home.com (Tim Peters) Date: Thu, 2 Aug 2001 00:46:51 -0400 Subject: [Python-Dev] descr-branch is dead; long live the trunk In-Reply-To: Message-ID: [Tim said] > test_weakref.py fails now. Never mind -- I flubbed resolving one of "the conflicts" CVS insisted existed in classobject.c. Fixed now. All tests pass (but 24 are skipped on Windows). Still, keep your eyes open! Lots of code just changed. From tim.one@home.com Thu Aug 2 06:16:55 2001 From: tim.one@home.com (Tim Peters) Date: Thu, 2 Aug 2001 01:16:55 -0400 Subject: [Python-Dev] PyOS_snprintf() / PyOS_vsnprintf() In-Reply-To: <3B66E96C.FBAB8A62@lemburg.com> Message-ID: [MAL] > How about always enabling our version in the alpha cycle and then > reverting back to the native one in the betas ? If we have to ship our own code for it anyway, why ever revert to the native one? Historically, all that gives us is boundless opportunities to catalog and #ifdef our way around gratuitous discrepancies among platform C libraries. since-we-switched-to-our-own-getopt-everywhere-we-no-longer-get- getopt-bug-reports-anywhere-ly y'rs - tim From paulp@ActiveState.com Thu Aug 2 17:51:02 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 02 Aug 2001 09:51:02 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? References: <20010730051831.B1122@thyrsus.com> <20010731012432.G20676@xs4all.nl> <15206.8708.811000.468489@gargle.gargle.HOWL> <20010731115521.I20676@xs4all.nl> <3B668608.F68B5953@ActiveState.com> <15207.26855.65000.718694@gargle.gargle.HOWL> Message-ID: <3B6984F6.1AF3853F@ActiveState.com> Nathan Torkington wrote: > > ... > > I've been eyeing your 'import from the future' system of getting > experimental features. I'll be interested to see how this holds up. Actually, Python doesn't usually have experimental features. "import from future" is about introducing new features that will break code "early". On the one hand, some people want the feature as soon as possible. On the other, people are afraid of the code breakage (e.g. adding a keyword in Python breaks code) and need time to adjust. The future construct is the compromise. Python has no real mechanism for experimental features other than documentation, patches and so forth. I can only imagine one or two experimental features in the history of the language, though. >... > We in the Perl camp have bandied around the idea of "use perl 5.004" > as a way to get all the behaviour of the 5.004 release. That's a > noble idea, but forces us to keep a lot of crap around when one of the > goals of active development is to be able to shed obstructive missteps > in language evolution. Yes, Guido has rejected that idea for Python as too much effort. Imagine the regression testing problems alone! -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From mal@lemburg.com Thu Aug 2 10:58:26 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 02 Aug 2001 11:58:26 +0200 Subject: [Python-Dev] PyOS_snprintf() / PyOS_vsnprintf() References: Message-ID: <3B692442.E0C14931@lemburg.com> Tim Peters wrote: > > [MAL] > > How about always enabling our version in the alpha cycle and then > > reverting back to the native one in the betas ? > > If we have to ship our own code for it anyway, why ever revert to the native > one? Historically, all that gives us is boundless opportunities to catalog > and #ifdef our way around gratuitous discrepancies among platform C > libraries. > > since-we-switched-to-our-own-getopt-everywhere-we-no-longer-get- > getopt-bug-reports-anywhere-ly y'rs - tim Well, the emulation is not as robust and fast as the native implementation is, plus it cannot recover from a buffer overrun; such an overrun is likely to cause a core dump due to sprintf() writing into the heap -- still better than allowing a cracker to hack your system by exploiting a stack overrun, but not as good as a true snprintf() implementation will do. If we do get complaints about snprintf() failures on systems which do have a native API, then we can still switch to the emulation for all platforms. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From barry@zope.com Thu Aug 2 13:20:08 2001 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 2 Aug 2001 08:20:08 -0400 Subject: [Python-Dev] post descr-branch merge results Message-ID: <15209.17784.192118.671324@anthem.wooz.org> Everything looks pretty good from here (Linux, RH6.1-ish). 143 tests passed, and the usual 17 skipped. gcc spit out a bunch of warnings in socketmodule.c, all of which look to be openssl related. It's certainly possible that I have an old openssl package laying around. Other than that, it looks good. Thanks Tim! -Barry gcc -g -O3 -Wall -Wstrict-prototypes -fPIC -DUSE_SSL=1 -I/usr/local/ssl/include -I. -I/home/barry/projects/python/./Include -I/usr/local/include -IInclude/ -c /home/barry/projects/python/Modules/socketmodule.c -o build/temp.linux-i686-2.2/socketmodule.o In file included from /home/barry/projects/python/Modules/socketmodule.c:208: /usr/local/ssl/include/openssl/rsa.h:219: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/rsa.h:220: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/rsa.h:221: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/rsa.h:223: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/rsa.h:224: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/rsa.h:227: warning: function declaration isn't a prototype In file included from /usr/local/ssl/include/openssl/objects.h:960, from /usr/local/ssl/include/openssl/evp.h:138, from /usr/local/ssl/include/openssl/x509.h:67, from /home/barry/projects/python/Modules/socketmodule.c:210: /usr/local/ssl/include/openssl/asn1.h:377: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:378: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:379: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:380: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:753: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:755: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:806: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:806: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:809: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:809: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:810: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:817: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:817: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:818: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:854: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:856: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:858: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/asn1.h:859: warning: function declaration isn't a prototype In file included from /usr/local/ssl/include/openssl/x509.h:67, from /home/barry/projects/python/Modules/socketmodule.c:210: /usr/local/ssl/include/openssl/evp.h:267: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/evp.h:268: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/evp.h:269: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/evp.h:271: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/evp.h:272: warning: function declaration isn't a prototype In file included from /home/barry/projects/python/Modules/socketmodule.c:210: /usr/local/ssl/include/openssl/x509.h:122: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/x509.h:123: warning: function declaration isn't a prototype In file included from /usr/local/ssl/include/openssl/x509_vfy.h:69, from /usr/local/ssl/include/openssl/x509.h:520, from /home/barry/projects/python/Modules/socketmodule.c:210: /usr/local/ssl/include/openssl/lhash.h:90: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/lhash.h:91: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/lhash.h:123: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/lhash.h:123: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/lhash.h:128: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/lhash.h:129: warning: function declaration isn't a prototype In file included from /home/barry/projects/python/Modules/socketmodule.c:210: /usr/local/ssl/include/openssl/x509.h:967: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/x509.h:970: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/x509.h:973: warning: function declaration isn't a prototype In file included from /home/barry/projects/python/Modules/socketmodule.c:211: /usr/local/ssl/include/openssl/pem.h:491: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/pem.h:493: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/pem.h:505: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/pem.h:507: warning: function declaration isn't a prototype In file included from /usr/local/ssl/include/openssl/ssl.h:63, from /home/barry/projects/python/Modules/socketmodule.c:212: /usr/local/ssl/include/openssl/comp.h:15: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/comp.h:16: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/comp.h:17: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/comp.h:18: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/comp.h:19: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/comp.h:20: warning: function declaration isn't a prototype In file included from /home/barry/projects/python/Modules/socketmodule.c:212: /usr/local/ssl/include/openssl/ssl.h:224: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/ssl.h:225: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/ssl.h:226: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/ssl.h:442: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/ssl.h:445: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/ssl.h:467: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/ssl.h:574: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/ssl.h:653: warning: function declaration isn't a prototype In file included from /home/barry/projects/python/Modules/socketmodule.c:212: /usr/local/ssl/include/openssl/ssl.h:1032: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/ssl.h:1068: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/ssl.h:1070: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/ssl.h:1143: warning: function declaration isn't a prototype /usr/local/ssl/include/openssl/ssl.h:1144: warning: function declaration isn't a prototype From dan@sidhe.org Thu Aug 2 15:02:42 2001 From: dan@sidhe.org (Dan Sugalski) Date: Thu, 02 Aug 2001 10:02:42 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <200108011903.VAA06122@core.inf.ethz.ch> Message-ID: <5.1.0.14.0.20010802095102.021f9e68@24.8.96.48> At 09:03 PM 8/1/2001 +0200, Samuele Pedroni wrote: >Hi. > > > > > The point is to put the commonly called things in the vtable in a way that > > you can avoid as much conditional code as possible, while less common > > things get dealt with in ways you'd generally expect. (Dynamic lookups > with > > caching and suchlike things) > >If I'm right, you're designing a object-based VM. More or less, yep, with a semi-generic two-arg dispatch for the binary methods. The vtable associated with each variable has a fixed set of required functions--add with 2nd arg an int, add with second arg a bigint, add with second arg a generic variable, for example. Basically all the things one would commonly do on data (add, subtract, multiply, divide, modulus, some string ops, get integer/real/string representation, copy, destroy, etc) have entries, with a catchall "generic method call" to handle, well, generic method calls. It's all designed with high-speed dispatch and minimal conditional branch requirements in mind, as well as encapsulating all the "what do I do on data" functions. Basically the opcodes generally handle control flow, register/stack ops, and VM management, while actual operations on variables is left to the vtable methods attached to the variables. I expect things to get mildly incestuous for speed reasons, but I'm OK with that. :) >I don't how >typical is OO programming in Perl, but in Python that plays a central role, >if your long run goal is to compile to native code you should have >a "hard-wired" concept of classes and them like, Yep. There's a fast "get name of object's class" and "get pointer to object's class stash/variable table/methods/subs/". > because an operation >like getting the class of an instance should be as direct and fast as possible >if you want to use any of the "nice" optimization for a VM for a OO dynamic >language: > inline polymorphic caches Yup. I'm leaning towards a class based cache for inherited methods and suchlike things, but I'm not sure that's sufficient if we're going to have objects whose inheritance tree is handled on a per-object basis. > customization Details? I'm not sure what you're talking about here. > or if/when possible: direct inlining. Yep, though the potential to mess with a class' methods at runtime tends to shoot this one down. (Perl's got similar issues with this, plus a few extra real nasty optimizer killers) I'm pondering a "check_if_changed" branch opcode that'll check a method/function/sub's definition to see if it's changed since the code was generated, and do the inline stuff if it hasn't. Though that still limits what you can do with code motion and other optimizations. >If any of the high-level OO system can not be used, you have to choose >a lower level one and map these to it, >a vtable bytecode mapping model is too underspecified. Oh, sure, saying "vtable bytecode mapping model" is an awful lot like saying "procedural language"--it's informative without actually being useful... :) Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From skip@pobox.com (Skip Montanaro) Thu Aug 2 15:07:21 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 2 Aug 2001 09:07:21 -0500 Subject: [Python-Dev] test results Message-ID: <15209.24217.765601.582838@beluga.mojam.com> On my Mandrake 8.0 laptop I ran make test after grabbing the latest stuff from cvs and wound up with 145 tests OK. 1 test failed: test_linuxaudiodev 14 tests skipped: test_al test_cd test_cl test_dl test_gl test_imgfile test_largefile test_nis test_ntpath test_socketserver test_sunaudiodev test_unicode_file test_winreg test_winsound The linuxaudiodev test never seems to work for me (though I do hear an agonizing wail (is that supposed to be Homer Simpson?) from my laptop's speakers when it's run). The result is always: test test_linuxaudiodev crashed -- linuxaudiodev.error: (16, 'Device or resource busy') The socketserver test works when run manually. Skip From guido@zope.com Thu Aug 2 15:20:44 2001 From: guido@zope.com (Guido van Rossum) Date: Thu, 02 Aug 2001 10:20:44 -0400 Subject: [Python-Dev] testing Message-ID: <200108021420.KAA32192@cj20424-a.reston1.va.home.com> I have noticed PSU activity on this channel. This is a test message to check wh From Samuele Pedroni Thu Aug 2 15:32:34 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Thu, 2 Aug 2001 16:32:34 +0200 (MET DST) Subject: [Python-Dev] Parrot -- should life imitate satire? Message-ID: <200108021432.QAA28270@core.inf.ethz.ch> Hi. > > > > > > The point is to put the commonly called things in the vtable in a way that > > > you can avoid as much conditional code as possible, while less common > > > things get dealt with in ways you'd generally expect. (Dynamic lookups > > with > > > caching and suchlike things) > > > >If I'm right, you're designing a object-based VM. > > More or less, yep, with a semi-generic two-arg dispatch for the binary > methods. The vtable associated with each variable has a fixed set of > required functions--add with 2nd arg an int, add with second arg a bigint, > add with second arg a generic variable, for example. Basically all the > things one would commonly do on data (add, subtract, multiply, divide, > modulus, some string ops, get integer/real/string representation, copy, > destroy, etc) have entries, with a catchall "generic method call" to > handle, well, generic method calls. A question: when you say variable you mean variable (perl sense of that) or object. It has already been pointed out but it's really confusing from the point of view of python terminology. Will perl6 have only variables which contain objects, truly references to objects like Python or ...? I should repeat that your explanation that assigment is somehow performed calling a method on the "variable" is quiet a strange notion in general, I can imagine having a slot called on assigment that eventually does a copy or return just the object but assigment as an operation on the lvalue is something very peculiar. I know that perl5 assignment is an operator returning an lvalue, is this related? > It's all designed with high-speed dispatch and minimal conditional branch > requirements in mind, as well as encapsulating all the "what do I do on > data" functions. Basically the opcodes generally handle control flow, > register/stack ops, and VM management, while actual operations on variables > is left to the vtable methods attached to the variables. > > I expect things to get mildly incestuous for speed reasons, but I'm OK with > that. :) > > >I don't how > >typical is OO programming in Perl, but in Python that plays a central role, > >if your long run goal is to compile to native code you should have > >a "hard-wired" concept of classes and them like, > > Yep. There's a fast "get name of object's class" and "get pointer to > object's class stash/variable table/methods/subs/ terminology here>". > > > because an operation > >like getting the class of an instance should be as direct and fast as possible > >if you want to use any of the "nice" optimization for a VM for a OO dynamic > >language: > > inline polymorphic caches > > Yup. I'm leaning towards a class based cache for inherited methods and > suchlike things, but I'm not sure that's sufficient if we're going to have > objects whose inheritance tree is handled on a per-object basis. Make sense for the interpreted version, but for speed call-site caches is far more promising when native compiling, but also more complicated. I imagine you already know e.g. the Self project literature. > > customization > > Details? I'm not sure what you're talking about here. Compiling different versions of the same method for example wrt to the receiver type in single dispatch case. See below > > or if/when possible: direct inlining. > > Yep, though the potential to mess with a class' methods at runtime tends to > shoot this one down. (Perl's got similar issues with this, plus a few extra > real nasty optimizer killers) I'm pondering a "check_if_changed" branch > opcode that'll check a method/function/sub's definition to see if it's > changed since the code was generated, and do the inline stuff if it hasn't. > Though that still limits what you can do with code motion and other > optimizations. > > >If any of the high-level OO system can not be used, you have to choose > >a lower level one and map these to it, > >a vtable bytecode mapping model is too underspecified. > > Oh, sure, saying "vtable bytecode mapping model" is an awful lot like > saying "procedural language"--it's informative without actually being > useful... :) I imagine you already know: especially when compiling to native code, it is all a matter of optimizing for the common case, and be prepared to be quiet slow otherwise, especially when dealing with dynamic changes, both in Perl and Python there is no clear distiction from normal ops and dynamic changes to methods... but in any case they are rare compared to the rest. Threading adds complexity to the problem. For the rest is just speed vs. memory, customization is a clear example for that. regards, Samuele Pedroni. From nas@python.ca Thu Aug 2 19:42:13 2001 From: nas@python.ca (Neil Schemenauer) Date: Thu, 2 Aug 2001 11:42:13 -0700 Subject: [Python-Dev] test results In-Reply-To: <15209.24217.765601.582838@beluga.mojam.com>; from skip@pobox.com on Thu, Aug 02, 2001 at 09:07:21AM -0500 References: <15209.24217.765601.582838@beluga.mojam.com> Message-ID: <20010802114213.B17147@glacier.fnational.com> Skip Montanaro wrote: > test test_linuxaudiodev crashed -- linuxaudiodev.error: (16, 'Device or > resource busy') Something has /dev/dsp or /dev/audio open. Are you playing MP3s or running something like esd? Neil From gward@python.net Thu Aug 2 15:15:00 2001 From: gward@python.net (Greg Ward) Date: Thu, 2 Aug 2001 10:15:00 -0400 Subject: [Python-Dev] Good news about ExtensionClass and Python 2.2a1 In-Reply-To: <20010731135457.A15139@mems-exchange.org>; from nascheme@mems-exchange.org on Tue, Jul 31, 2001 at 01:54:57PM -0400 References: <20010731135457.A15139@mems-exchange.org> Message-ID: <20010802101500.A4616@gerg.ca> On 31 July 2001, Neil Schemenauer said: > After a few tweaks to ExtensionClass and a few small fixes to some of > our introspection code I'm happy to say that Python 2.2a1 passes our > unit test suite. This is significant since there are about 45000 lines > of code (counted by "wc -l") tested by 3569 test cases. Since we use > ZODB ExtensionClasses are quite widely used. Merging descr_branch into > HEAD sounds like a good idea to me. Well done Guido. I hate to be a wet blanket, Neil, but we (the MEMS Exchange virtual fab project) really have more like 18000 lines of formally tested code, 6700 lines of test code, and 10000 lines of UI code (which isn't formally tested). It's still a pretty big pile, though, and the fact that it can be beaten into submission to work with 2.2 is great news. (LoC figures are courtesy of Dinu Gherman's pycount, which distinguishes code, docs, comments, and blank lines. I'm just reporting the "code" figure.) > I'm going to spend a bit of time trying to rewrite the ZODB Persistent > class as a type. Excellent! Greg -- Greg Ward - Unix geek gward@python.net http://starship.python.net/~gward/ Drive defensively -- buy a tank. From skip@pobox.com (Skip Montanaro) Thu Aug 2 20:29:12 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 2 Aug 2001 14:29:12 -0500 Subject: [Python-Dev] test results In-Reply-To: <20010802114213.B17147@glacier.fnational.com> References: <15209.24217.765601.582838@beluga.mojam.com> <20010802114213.B17147@glacier.fnational.com> Message-ID: <15209.43528.631408.210878@beluga.mojam.com> >> test test_linuxaudiodev crashed -- linuxaudiodev.error: (16, 'Device >> or resource busy') Neil> Something has /dev/dsp or /dev/audio open. Are you playing MP3s Neil> or running something like esd? Nope, not that I'm aware of. I poked around ps auxww output but didn't find anything that looked like an audio server. /dev/dsp and /dev/audio* are owned by user "skip", group "audio". My usual login is "skip" and I am in group "audio". The test succeeds outside of X but no sound is produced. Skip From fdrake@acm.org Thu Aug 2 20:26:24 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 2 Aug 2001 15:26:24 -0400 (EDT) Subject: [Python-Dev] test results In-Reply-To: <20010802114213.B17147@glacier.fnational.com> References: <15209.24217.765601.582838@beluga.mojam.com> <20010802114213.B17147@glacier.fnational.com> Message-ID: <15209.43360.641499.705190@cj42289-a.reston1.va.home.com> Neil Schemenauer writes: > Something has /dev/dsp or /dev/audio open. Are you playing MP3s or > running something like esd? This is exactly the problem I was having several months ago. I "solved" it by changing the permissions on the audio device: cj42289-a(.../python/linux); ls -lF /dev/dsp crw-rw---- 1 root audio 14, 3 Sep 27 2000 /dev/dsp cj42289-a(.../python/linux); ./python ../Lib/test/regrtest.py test_linuxaudiodevtest_linuxaudiodev test test_linuxaudiodev skipped -- (13, 'Permission denied', '/dev/dsp') 1 test skipped: test_linuxaudiodev As a side effect, the machine no longer makes that horrid screeching sound. ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Thu Aug 2 20:29:13 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 2 Aug 2001 15:29:13 -0400 (EDT) Subject: [Python-Dev] Good news about ExtensionClass and Python 2.2a1 In-Reply-To: <20010802101500.A4616@gerg.ca> References: <20010731135457.A15139@mems-exchange.org> <20010802101500.A4616@gerg.ca> Message-ID: <15209.43529.943039.929906@cj42289-a.reston1.va.home.com> Greg Ward writes: > (LoC figures are courtesy of Dinu Gherman's pycount, which distinguishes > code, docs, comments, and blank lines. I'm just reporting the "code" You don't test your documentation? Shame on you! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From cgw@alum.mit.edu Thu Aug 2 20:48:20 2001 From: cgw@alum.mit.edu (Charles G Waldman) Date: Thu, 2 Aug 2001 14:48:20 -0500 Subject: [Python-Dev] linuxaudiodev In-Reply-To: References: Message-ID: <15209.44676.292017.955013@nyx.dyndns.org> Skip Montanaro write: > The linuxaudiodev test never seems to work for me (though I do hear an > agonizing wail (is that supposed to be Homer Simpson?) from my laptop's > speakers when it's run). The result is always: > > test test_linuxaudiodev crashed -- linuxaudiodev.error: (16, 'Device or > resource busy') > > The socketserver test works when run manually. Neil Schemenauer replied: > Something has /dev/dsp or /dev/audio open. Are you playing MP3s or > running something like esd? If another process was holding /dev/whatever open, it's hard to understand how the agonizing wail was produced. The code in linuxaudiodev does its initializations in an order which is explicitly proscribed by the OSS programming guide. I have a SWIG-based linux audio module that I use in my code, not wanting anything to do with the (to me) highly suspect linuxaudiodev module. Should I dust off this code and contribute it? Of course, if we really want to provide high-quality audio support on Linux, we have to account for a number of different options - at least, OSS and ALSA... this is sort of a can-of-worms. But I do believe that the current linuxaudiodev module should be deprecated. From guido@zope.com Thu Aug 2 20:48:39 2001 From: guido@zope.com (Guido van Rossum) Date: Thu, 02 Aug 2001 15:48:39 -0400 Subject: [Python-Dev] post descr-branch merge results In-Reply-To: Your message of "Thu, 02 Aug 2001 08:20:08 EDT." <15209.17784.192118.671324@anthem.wooz.org> References: <15209.17784.192118.671324@anthem.wooz.org> Message-ID: <200108021948.f72JmeU11410@odiug.digicool.com> > gcc spit out a bunch of warnings in socketmodule.c, all of which look > to be openssl related. It's certainly possible that I have an old > openssl package laying around. It does that to me too. I think our gcc option requiring function prototypes is too picky for the openssl header files. Is there a way to teach setup.py to turn off this option when openssl is detected? It's already a separate case in setup.py. --Guido van Rossum (home page: http://www.python.org/~guido/) From cgw@alum.mit.edu Thu Aug 2 20:49:51 2001 From: cgw@alum.mit.edu (Charles G Waldman) Date: Thu, 2 Aug 2001 14:49:51 -0500 Subject: [Python-Dev] linuxaudiodev, ps Message-ID: <15209.44767.10607.967674@nyx.dyndns.org> Skip - did you try "fuser /dev/dsp" ? From guido@zope.com Thu Aug 2 20:52:32 2001 From: guido@zope.com (Guido van Rossum) Date: Thu, 02 Aug 2001 15:52:32 -0400 Subject: [Python-Dev] test results In-Reply-To: Your message of "Thu, 02 Aug 2001 11:42:13 PDT." <20010802114213.B17147@glacier.fnational.com> References: <15209.24217.765601.582838@beluga.mojam.com> <20010802114213.B17147@glacier.fnational.com> Message-ID: <200108021952.f72JqW411441@odiug.digicool.com> > Skip Montanaro wrote: > > test test_linuxaudiodev crashed -- linuxaudiodev.error: (16, 'Device or > > resource busy') > > Something has /dev/dsp or /dev/audio open. Are you playing MP3s or > running something like esd? Fred experiences this too with Mandrake on his Dell laptop, and before someone hacked the extension module I experienced it on a Dell desktop with Red Hat too -- no MP# players. The wail that Skip hears is the first second of "Nobody expects the Spanish inquisition". I think that different audio hardware has different Linux drivers that act differently, and the extension module doesn't know how to properly wait until enough buffer space is available with some drivers. --Guido van Rossum (home page: http://www.python.org/~guido/) From mwh@python.net Thu Aug 2 20:55:40 2001 From: mwh@python.net (Michael Hudson) Date: 02 Aug 2001 15:55:40 -0400 Subject: [Python-Dev] test results In-Reply-To: Skip Montanaro's message of "Thu, 2 Aug 2001 09:07:21 -0500" References: <15209.24217.765601.582838@beluga.mojam.com> Message-ID: <2mvgk6jkxv.fsf@starship.python.net> Skip Montanaro writes: > The linuxaudiodev test never seems to work for me (though I do hear > an agonizing wail (is that supposed to be Homer Simpson?) Cardinal Biggles, I think. > from my laptop's speakers when it's run). Sorry I can't be of more use. Cheers, M. -- While preceding your entrance with a grenade is a good tactic in Quake, it can lead to problems if attempted at work. -- C Hacking -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From guido@zope.com Thu Aug 2 22:13:28 2001 From: guido@zope.com (Guido van Rossum) Date: Thu, 02 Aug 2001 17:13:28 -0400 Subject: [Python-Dev] linuxaudiodev In-Reply-To: Your message of "Thu, 02 Aug 2001 14:48:20 CDT." <15209.44676.292017.955013@nyx.dyndns.org> References: <15209.44676.292017.955013@nyx.dyndns.org> Message-ID: <200108022113.f72LDSi11865@odiug.digicool.com> > The code in linuxaudiodev does its initializations in an order which > is explicitly proscribed by the OSS programming guide. > > I have a SWIG-based linux audio module that I use in my code, not > wanting anything to do with the (to me) highly suspect linuxaudiodev > module. Should I dust off this code and contribute it? > > Of course, if we really want to provide high-quality audio support on > Linux, we have to account for a number of different options - at > least, OSS and ALSA... this is sort of a can-of-worms. But I do > believe that the current linuxaudiodev module should be deprecated. Sure. Do you want to take the lead here? Nobody at PythonLabs has any interest in this, and without someone putting a stake into the ground, there's no point in even talking about it. It sounds you know what you're talking about, and you have CVS permission, so if you want to do it, go ahead. (For b/w compatibility we may have to keep the linuxaudio extension around for a while, in a deprecated state.) --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Thu Aug 2 22:18:07 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 2 Aug 2001 17:18:07 -0400 (EDT) Subject: [Python-Dev] linuxaudiodev In-Reply-To: <200108022113.f72LDSi11865@odiug.digicool.com> References: <15209.44676.292017.955013@nyx.dyndns.org> <200108022113.f72LDSi11865@odiug.digicool.com> Message-ID: <15209.50063.245244.530635@cj42289-a.reston1.va.home.com> Guido van Rossum writes: > Sure. Do you want to take the lead here? Nobody at PythonLabs has > any interest in this, and without someone putting a stake into the > ground, there's no point in even talking about it. It sounds you know > what you're talking about, and you have CVS permission, so if you want > to do it, go ahead. > > (For b/w compatibility we may have to keep the linuxaudio extension > around for a while, in a deprecated state.) Would it make sense to include a Python implementation that wraps aronud whatever low-level modules are needed? Or to use the same API and just replace the implementation? I don't know enough about the potential replacements, but it would be nice to maintain the availability of the API; it's been available for several versions now. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From dan@sidhe.org Thu Aug 2 22:54:46 2001 From: dan@sidhe.org (Dan Sugalski) Date: Thu, 02 Aug 2001 17:54:46 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <200108021432.QAA28270@core.inf.ethz.ch> Message-ID: <5.1.0.14.0.20010802165055.0221dd00@24.8.96.48> At 04:32 PM 8/2/2001 +0200, Samuele Pedroni wrote: >Hi. > > > > > > > > > The point is to put the commonly called things in the vtable in a > way that > > > > you can avoid as much conditional code as possible, while less common > > > > things get dealt with in ways you'd generally expect. (Dynamic lookups > > > with > > > > caching and suchlike things) > > > > > >If I'm right, you're designing a object-based VM. > > > > More or less, yep, with a semi-generic two-arg dispatch for the binary > > methods. The vtable associated with each variable has a fixed set of > > required functions--add with 2nd arg an int, add with second arg a bigint, > > add with second arg a generic variable, for example. Basically all the > > things one would commonly do on data (add, subtract, multiply, divide, > > modulus, some string ops, get integer/real/string representation, copy, > > destroy, etc) have entries, with a catchall "generic method call" to > > handle, well, generic method calls. >A question: when you say variable you mean variable (perl sense of that) >or object. It has already been pointed out but it's really confusing >from the point of view of python terminology. Will perl6 have only >variables which contain objects, truly references to objects like Python >or ...? Well, it sort of looks like: name--->variable--->value The name is a symbol table entry that points to the variable. The variable is a data structure that has the vtable info, some pointer data, some flags, and general whatnots. (The docs refer to this as a PMC, or Parrot Magic Cookie, FWIW. Which isn't much) The value is the actual contents. We also have to track properties on both a variable and a value basis. For example: declare c, b declare a is const = 0 but true b = a c = \a dereference(c) = 1 In this case, the variable represented by a has the property 'const', and a value of 0 which has the property 'true'. The assignment to b copies the value, and the properties on the value (true in this case), but not the properties on the variable, so b isn't const. c is a reference to the variable a represents, so the attempt to assign to the dereference of c will fail, since the variable c points to has the const property. We can't depend on the properties to be attached to the *name* a, since (in my crude attempt to use python-style blocking) the name a has gone out of scope by the time we dereference c, and so would any properties attached to the name as such. I think the variable here would correspond to a Python object, but I'll need to go digging into Python semantics more to be sure. Currently there's not a lot planned around the actual symbol table entry, since it's not required for variable existence, and there's not much I can think of to attach to it anyway. (References mean we can't use it for much, really) My bet, and this is just an off-hand thing, is that the parser would spit out bytecode tying the symbol table entry and the PMC structure more closely than it might for other languages. At least do a lot more symbolic fetch/store ops, if need be, which we might not. >I should repeat that your explanation that assigment is somehow performed >calling a method on the "variable" is quiet a strange notion in general, >I can imagine having a slot called on assigment that eventually does a copy >or return just the object but assigment as an operation on the lvalue >is something very peculiar. I know that perl5 assignment is an operator >returning an lvalue, is this related? I don't think so, but I'm betting we're talking terminologically cross-wise. Assume, for example, the following pseudoish code: declare kitchen isa Thermostat kitchen = 20 In this case the kitchen variable is active, and assigning to it changes the temperature the thermostat in the kitchen is set to. The magic properties aren't attached to the name per se, at least in perl--it's just a handy way to get the PMC that underlies the thing. I can get an indirect handle on the data that the name 'kitchen' refers to and update it that way, or if I have that handle the 'kitchen' name can go completely out of scope and the only way to refer to the object representing (and allowing control over) my kitchen temperature is via the handle I've got on what's now an anonymous object. (Or, if I'm feeling really wacky, have two or more names that look like separate things but actually all point to the same PMC structure) I think it's the potential existence of references (or pointers, or whatever you want to call them) that necessitate the three-level split. It sounds like the things I'm calling 'name' and 'variable' are much more tightly bound in Python. There are several reasons to call a PMC assignment operator, rather than just unconditionally slam a new PMC into the name slot: 1) In the kitchen example above, assigning 20 to kitchen doesn't suddenly make kitchen an integer variable--it needs to stay a Thermostat, and just do something because we assigned to it. (And I understand this might not be the way Python would do things, in which case Python variable vtables might just have an "overwrite myself with the incoming info" function, rather than allow it to be overridden) 2) Since we can't really assign properties like constant-ness to a name (in which case we could get a reference to a variable and bypass the properties on the name by referring to it by reference) that needs to be enforced in other ways. In this case with a custom assignment vtable function which pitches a fit if you try to alter the variable. 3) Some other thing or other I apparently can't think of. It's a really good reason, though. > > > because an operation > > >like getting the class of an instance should be as direct and fast as >possible > > >if you want to use any of the "nice" optimization for a VM for a OO > dynamic > > >language: > > > inline polymorphic caches > > > > Yup. I'm leaning towards a class based cache for inherited methods and > > suchlike things, but I'm not sure that's sufficient if we're going to have > > objects whose inheritance tree is handled on a per-object basis. >Make sense for the interpreted version, but for speed call-site >caches is far more promising when native compiling, but also more >complicated. I >imagine you already know e.g. the Self project literature. Self I've not dug into. Despite all the "I haven't see that" that keeps popping up in my conversation, I *have* done a lot of research. (The pile of digested books at one point outweighed my kids. They've grown, but the new research pile's about to grow too...) > > > customization > > > > Details? I'm not sure what you're talking about here. >Compiling different versions of the same method for example wrt to >the receiver type in single dispatch case. See below Ah, aggressive inlining of a sort. I think. Yep, on the list of things to do with the optimizer. (Not on the first version, probably, but certainly later) > > > or if/when possible: direct inlining. > > > > Yep, though the potential to mess with a class' methods at runtime > tends to > > shoot this one down. (Perl's got similar issues with this, plus a few > extra > > real nasty optimizer killers) I'm pondering a "check_if_changed" branch > > opcode that'll check a method/function/sub's definition to see if it's > > changed since the code was generated, and do the inline stuff if it > hasn't. > > Though that still limits what you can do with code motion and other > > optimizations. > > > > >If any of the high-level OO system can not be used, you have to choose > > >a lower level one and map these to it, > > >a vtable bytecode mapping model is too underspecified. > > > > Oh, sure, saying "vtable bytecode mapping model" is an awful lot like > > saying "procedural language"--it's informative without actually being > > useful... :) > >I imagine you already know: especially when compiling to native code, >it is all a matter of optimizing for the common case, and be prepared >to be quiet slow otherwise, especially when dealing with dynamic changes, >both in Perl and Python there is no clear distiction from normal ops and >dynamic changes to methods... but in any case they are rare compared to the >rest. Yup. I am profoundly tempted, and I think it will go in, to have a "stable code" switch that can be given to the optimizer which guarantees that, once the compilation phase is done and we finally get to run for real, code will not change. And then apply all the advanced optimization techniques at that point. >Threading adds complexity to the problem. Nah, not too badly. A little synchronization issue, but that's reasonably trivial. >For the rest is just speed vs. memory, customization is a clear example >for that. Sure. The one nice thing that both perl and python have going for them is the ultimate presence of all the source (or a high-enough level representation of it to feed to the optimizer) at run time. We can tell with more surety what side-effects various methods and functions in external code libraries will have, and we can get more aggressive (maybe) because of it. Unlike, say, C, where passing a pointer to a routine outside what the compiler can see will hobble the optimizer a whole lot in many cases... Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From dan@sidhe.org Thu Aug 2 23:01:42 2001 From: dan@sidhe.org (Dan Sugalski) Date: Thu, 02 Aug 2001 18:01:42 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <20010801153607.G626@xs4all.nl> References: <5.1.0.14.0.20010731234925.02298ae8@24.8.96.48> Message-ID: <5.1.0.14.0.20010802180025.0232e130@24.8.96.48> At 03:36 PM 8/1/2001 +0200, Thomas Wouters wrote: > > [SRE] and the C extension interface are the two big Python things I've got > > on my "must peer deeply into" list. > >I can't say much about SRE, but the C extension interface should be on >anyone's "must peer deeply into" list, except that there isn't a real need >to peer deeply into it :) Lets peer into it. I'm going to discuss the >pre-2.2 state of affairs, mostly because I know jack shit about the >2.2-type-dichotomy-healing :) Ah, I see. (and thanks heaps) Well, I just rejigged my thinking a bit on what we need to export, then. My take's been both more agressively black-box about things, and much less OO. Too much C on the brain, I expect. Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From skip@pobox.com (Skip Montanaro) Thu Aug 2 23:06:40 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 2 Aug 2001 17:06:40 -0500 Subject: [Python-Dev] Re: linuxaudiodev, ps In-Reply-To: <15209.44767.10607.967674@nyx.dyndns.org> References: <15209.44767.10607.967674@nyx.dyndns.org> Message-ID: <15209.52976.938845.241434@beluga.mojam.com> Charles> Skip - did you try "fuser /dev/dsp" ? Not until you suggested it. Didn't even know it existed. It displayed nothing for either /dev/dsp or /dev/audio0: % fuser -a -v /dev/audio0 /dev/dsp USER PID ACCESS COMMAND /dev/audio0 No process references; use -v for the complete list /dev/dsp No process references; use -v for the complete list Skip From guido@zope.com Thu Aug 2 23:04:45 2001 From: guido@zope.com (Guido van Rossum) Date: Thu, 02 Aug 2001 18:04:45 -0400 Subject: [Python-Dev] linuxaudiodev In-Reply-To: Your message of "Thu, 02 Aug 2001 17:18:07 EDT." <15209.50063.245244.530635@cj42289-a.reston1.va.home.com> References: <15209.44676.292017.955013@nyx.dyndns.org> <200108022113.f72LDSi11865@odiug.digicool.com> <15209.50063.245244.530635@cj42289-a.reston1.va.home.com> Message-ID: <200108022204.SAA32634@cj20424-a.reston1.va.home.com> > I don't know enough about the potential replacements, but it would > be nice to maintain the availability of the API; it's been available > for several versions now. Yup. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com (Skip Montanaro) Thu Aug 2 23:11:23 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 2 Aug 2001 17:11:23 -0500 Subject: [Python-Dev] test results In-Reply-To: <200108021952.f72JqW411441@odiug.digicool.com> References: <15209.24217.765601.582838@beluga.mojam.com> <20010802114213.B17147@glacier.fnational.com> <200108021952.f72JqW411441@odiug.digicool.com> Message-ID: <15209.53259.443303.220952@beluga.mojam.com> Guido> Fred experiences this too with Mandrake on his Dell laptop, and Guido> before someone hacked the extension module I experienced it on a Guido> Dell desktop with Red Hat too -- no MP# players. Nice to know I'm not alone. I also have Mandrake 8.0 on a Dell laptop. Since it's something the wizards know about, I won't pursue it further. Skip From skip@pobox.com (Skip Montanaro) Thu Aug 2 23:17:31 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 2 Aug 2001 17:17:31 -0500 Subject: [Python-Dev] linuxaudiodev In-Reply-To: <15209.44676.292017.955013@nyx.dyndns.org> References: <15209.44676.292017.955013@nyx.dyndns.org> Message-ID: <15209.53627.333118.300560@beluga.mojam.com> Charles> I have a SWIG-based linux audio module that I use in my code, Charles> not wanting anything to do with the (to me) highly suspect Charles> linuxaudiodev module. Should I dust off this code and Charles> contribute it? Charles> Of course, if we really want to provide high-quality audio Charles> support on Linux, we have to account for a number of different Charles> options - at least, OSS and ALSA... this is sort of a Charles> can-of-worms. But I do believe that the current linuxaudiodev Charles> module should be deprecated. Charles, is your SWIG-based thing still Linux-specific? (I sort of suspect it is.) While having to support multiple options will be a pain, I think they will each tend to be more platform-neutral than the current approach. Skip From akuchlin@mems-exchange.org Thu Aug 2 23:16:37 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Thu, 02 Aug 2001 18:16:37 -0400 Subject: [Python-Dev] Outdated Misc/cheatsheet Message-ID: Misc/cheatsheet is way out of date (the 'access' keyword?). I'd like to replace it with the text version of Simon Brunning and Richard Gruet's Python cheatsheet. Issues: 1) Simon, do you grant permission to do that? Should I ask Richard, too? 2) The new file is significantly larger: 100K instead of just 22K. Is that OK? 3) The reference will need updating for 2.1 and 2.2 changes, but slightly obsolete is better than waaay obsolete. Simon, do you want help updating it for 2.1/2.2? --amk From nas@python.ca Thu Aug 2 23:35:26 2001 From: nas@python.ca (Neil Schemenauer) Date: Thu, 2 Aug 2001 15:35:26 -0700 Subject: [Python-Dev] Good news about ExtensionClass and Python 2.2a1 In-Reply-To: <20010802101500.A4616@gerg.ca>; from gward@python.net on Thu, Aug 02, 2001 at 10:15:00AM -0400 References: <20010731135457.A15139@mems-exchange.org> <20010802101500.A4616@gerg.ca> Message-ID: <20010802153526.A18271@glacier.fnational.com> Greg Ward wrote: > Neil: > > I'm going to spend a bit of time trying to rewrite the ZODB Persistent > > class as a type. > > Excellent! Unfortunately it's looks to be too much work for me to tackle right now. cPersistent is subclass by lots of other ZODB extension classes. My idea of doing think piecemeal is not going to work. Neil From barry@zope.com Thu Aug 2 23:37:08 2001 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 2 Aug 2001 18:37:08 -0400 Subject: [Python-Dev] linuxaudiodev References: <15209.44676.292017.955013@nyx.dyndns.org> <200108022113.f72LDSi11865@odiug.digicool.com> Message-ID: <15209.54804.552103.106351@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> Sure. Do you want to take the lead here? Nobody at GvR> PythonLabs has any interest in this, and without someone GvR> putting a stake into the ground, there's no point in even GvR> talking about it. It sounds you know what you're talking GvR> about, and you have CVS permission, so if you want to do it, GvR> go ahead. I have interest, but no time. Although I'm sure Charles can do everything necessary on his own, if he needs help from inside Pythonlabs I'll volunteer. -Barry From guido@zope.com Fri Aug 3 00:43:35 2001 From: guido@zope.com (Guido van Rossum) Date: Thu, 02 Aug 2001 19:43:35 -0400 Subject: [Python-Dev] Good news about ExtensionClass and Python 2.2a1 In-Reply-To: Your message of "Thu, 02 Aug 2001 15:35:26 PDT." <20010802153526.A18271@glacier.fnational.com> References: <20010731135457.A15139@mems-exchange.org> <20010802101500.A4616@gerg.ca> <20010802153526.A18271@glacier.fnational.com> Message-ID: <200108022343.TAA00415@cj20424-a.reston1.va.home.com> > > Neil: > > > I'm going to spend a bit of time trying to rewrite the ZODB Persistent > > > class as a type. > > Unfortunately it's looks to be too much work for me to tackle right now. > cPersistent is subclass by lots of other ZODB extension classes. My > idea of doing think piecemeal is not going to work. Don't worry about it. Eventually this will become a Zope Corp. project. (If you do get to this, I'd like to hear about it of course. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From esr@thyrsus.com Fri Aug 3 03:36:32 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Thu, 2 Aug 2001 22:36:32 -0400 Subject: [Python-Dev] Proposed new module for the Python library Message-ID: <20010802223632.A31724@thyrsus.com> There's a kind of boilerplate code I've written way too often. The most recent time, I got fed up and wrote a candidate library module so I'd never have to do it again. Here's the header comment: --------------------------------------------------------------------------- ccframe -- framework code for building compiler-like programs. There is a common `compiler-like' pattern in Unix scripts which is useful for translation utilities of all sorts. A program following this pattern behaves as a filter when no argument files are specified on the command line, but otherwise transforms each file individually into a corresponding output file. This module provides framework and glue code to make such programs easy to write. You supply a function to massage the file data; depending on which entry point you use, it can take input and output file pointers, or it can take a string consisting of the entire file's data and return a replacement, or it can take in succession strings consisting of each of the file's lines and return a translated line for each. Argument files are transformed in left to right order on the command line. A filename consisting of a dash is interpreted as a directive to read from standard input (this can be useful in pipelines). Replacement of each file is atomic and doesn't occur until the translation of that file has completed. Any tempfiles are removed automatically on any exception thrown by the translation function, and the exception is then passed upwards. The entry points return 0 on success, 1 to signal a failed file open, and 2 to signal a failed tempfile open or rename. Error messages are emitted to stderr. --------------------------------------------------------------------------- Design comments? Critiques? Code on request. I'm already considering throwing exceptions on open and rename errors instead of complaining to stderr and returning an error status. That would be more Pythonic, though slightly less convenient in the commonest cases. -- Eric S. Raymond The two pillars of `political correctness' are, a) willful ignorance, and b) a steadfast refusal to face the truth -- George MacDonald Fraser From paulp@ActiveState.com Fri Aug 3 04:13:26 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 02 Aug 2001 20:13:26 -0700 Subject: [Python-Dev] Proposed new module for the Python library References: <20010802223632.A31724@thyrsus.com> Message-ID: <3B6A16D6.1DB6C964@ActiveState.com> Sounds "import fileinput-ish" to me. Are there fixes to fileinput that will make it applicable to your problems? -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From esr@thyrsus.com Fri Aug 3 04:30:45 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Thu, 2 Aug 2001 23:30:45 -0400 Subject: [Python-Dev] Proposed new module for the Python library In-Reply-To: <3B6A16D6.1DB6C964@ActiveState.com>; from paulp@ActiveState.com on Thu, Aug 02, 2001 at 08:13:26PM -0700 References: <20010802223632.A31724@thyrsus.com> <3B6A16D6.1DB6C964@ActiveState.com> Message-ID: <20010802233045.A32400@thyrsus.com> Paul Prescod : > Sounds "import fileinput-ish" to me. Are there fixes to fileinput that > will make it applicable to your problems? No. The existing code in the fileinput module is intended to treat a sequence of files as one continuous input source. My module is designed to iterate over files, transforming each one to a corresponsing named output file. They're both input frameworks, but otherwise quite different. The fileinput model is appropriate for filters like cat(1) and tr(1). The ccframe model, on the other hand, is appropriate for compilers and file-format converters. -- Eric S. Raymond Our society won't be truly free until "None of the Above" is always an option. From simon@netthink.co.uk Thu Aug 2 23:01:09 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Thu, 2 Aug 2001 15:01:09 -0700 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: <20010731220627.G1708@thyrsus.com> Message-ID: <20010802150109.B2033@netthink.co.uk> On Tue, Jul 31, 2001 at 10:06:27PM -0400, Eric S. Raymond wrote: > I strongly agree with Ping on this. My experience is that CGI is an > extremely important application category for Python, and one worth > significant effort to serve. I also strongly agree. CGI's one application that draws a lot of people into programming, and many of them for the first time - you get a lot of web designers and the like wanting to go a bit further, and it'd be nice to get them into the good kind of snake (Python) instead of the bad kind of snake. (ASP) Damn, I'm a traitor to my own language. Simon From simon@netthink.co.uk Thu Aug 2 18:33:15 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Thu, 2 Aug 2001 10:33:15 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <15206.8918.603000.448728@gargle.gargle.HOWL> Message-ID: <20010802103315.I1667@netthink.co.uk> On Mon, Jul 30, 2001 at 08:15:34PM -0700, Nathan Torkington wrote: > The people designing and implementing perl6 have already agreed on a > "do it clean, then make it faster" approach. We can all see the > problems with the current Perl internals, and have no desire to repeat > the mistakes of the past. May I just say, as one of those people, yea amen to that. Simon From simon@netthink.co.uk Thu Aug 2 18:22:48 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Thu, 2 Aug 2001 10:22:48 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <3B683D26.5ECEFAF2@ActiveState.com> Message-ID: <20010802102248.F1667@netthink.co.uk> On Wed, Aug 01, 2001 at 10:32:22AM -0700, David Ascher wrote: > For Perl5, don't forget perlguts and perlguts illustrated, which may be > useful to Pythonistas: > > http://aspn.activestate.com/ASPN/Products/ActivePerl/lib/Pod/perlguts.html > http://gisle.aas.no/perl/illguts/ Let's kick this in again because people might have missed it: http://www.netthink.co.uk/downloads/ I spent a lot of time working on that, so use it. :) From simon@netthink.co.uk Thu Aug 2 18:20:25 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Thu, 2 Aug 2001 10:20:25 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <20010731225445.B626@xs4all.nl> Message-ID: <20010802102025.E1667@netthink.co.uk> On Tue, Jul 31, 2001 at 10:54:45PM +0200, Thomas Wouters wrote: > Python objects all have vtables too (though they're structs, not arrays... > I'm not sure why you'd use arrays He means structs. :) Simon From simon@netthink.co.uk Thu Aug 2 17:54:09 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Thu, 2 Aug 2001 09:54:09 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <20010801153607.G626@xs4all.nl> Message-ID: <20010802095409.A1667@netthink.co.uk> On Wed, Aug 01, 2001 at 03:36:07PM +0200, Thomas Wouters wrote: > A Python object, any Python object, is essentially a struct like this: > > stuct PyExample { > int ob_refcnt; /* reference count */ > struct _typeobject *ob_type; /* type of this object */ > > /* type-specific data goes here */ > > } This looks rather like a Perl SV (scalar value): struct STRUCT_SV { /* struct sv { */ void* sv_any; /* pointer to something */ U32 sv_refcnt; /* how many references to us */ U32 sv_flags; /* what we are */ }; > For instance, the struct to hold a (normal, unbounded) Python integer is > > typedef struct { > PyObject_HEAD > long ob_ival; > } PyIntObject; We use double indirection so that we can switch between types very quickly; the sv_any pointer points to another structure or an integer/float. > The struct _typeobject pointer is a pointer to a Type object, which also > acts as a vtable. That's sounding more like how we plan to do things for Perl 6. > >>> def spam(x): > ... print "Hello Perl World!" > >>> dis.dis(spam) > 0 SET_LINENO 1 [...] % ./perl -Dts -e 'print "Hello Python people!\n"' => (-e:1) null => (-e:1) const(PV("Hello Python people!\12"\0)) => PV("Hello Python people!\12"\0) (-e:1) stringify [ That was constant folding, BTW. - SC] EXECUTING... => (-e:0) enter => (-e:0) nextstate => (-e:1) pushmark => * (-e:1) const(PV("Hello Python people!\12"\0)) => * PV("Hello Python people!\12"\0) (-e:1) print Hello Python people! => SV_YES (-e:1) leave perl -MO=Bytecode,-S -e 'print "Hello Python people!\n"' will give you a complete disassembly of the bytecode, but that's the version that can be serialised to disk, so it's a lot more verbose than it need be - it sets up all the opcodes, their flags, their sequence numbers, their connections to other nodes in the opcode tree and so on. If you really want to get stuck in, read the internals tutorial at http://www.netthink.co.uk/downloads/ Simon From simon@netthink.co.uk Thu Aug 2 18:06:06 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Thu, 2 Aug 2001 10:06:06 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <200108011903.VAA06122@core.inf.ethz.ch> Message-ID: <20010802100606.B1667@netthink.co.uk> On Wed, Aug 01, 2001 at 09:03:33PM +0200, Samuele Pedroni wrote: > If I'm right, you're designing a object-based VM. I don't how > typical is OO programming in Perl, but in Python that plays a central role, In Perl, the OO-ness is central at the *implementation* level, regardless of how OO the end user's code is. For instance, "$a = $b + $c" is essentially calling "a.store(b.get_integer() + c.get_integer())" deep down, but the end user doesn't have to know or care about that. Simon From simon@netthink.co.uk Thu Aug 2 18:24:14 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Thu, 2 Aug 2001 10:24:14 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <20010731223259.A626@xs4all.nl> Message-ID: <20010802102414.G1667@netthink.co.uk> On Tue, Jul 31, 2001 at 10:32:59PM +0200, Thomas Wouters wrote: > PUSH "1" > PUSH "2" > BINARY_ADD > > for it's > > "1" . "2" *cough*. Hell no! Addition and string concatenation would be separate opcodes. (I think.) Simon From simon@netthink.co.uk Thu Aug 2 22:54:18 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Thu, 2 Aug 2001 14:54:18 -0700 Subject: [Python-Dev] Python API version & optional features In-Reply-To: <200107301427.f6UERW802779@odiug.digicool.com> Message-ID: <20010802145418.A2033@netthink.co.uk> On Mon, Jul 30, 2001 at 10:27:32AM -0400, Guido van Rossum wrote: > I'm trying to convince myself that the magic number patch is okay, and > here's what I come up with. If someone builds a Python with a > non-standard Unicode width and accidentally uses a directory full of > extensions built for the standard Unicode width on his platform, he > deserves a warning. Shouldn't Python automatically Do The Right Thing in that case? That would mean wrapping the UCS4 calls in a conversion layer, which isn't hard. I mean, it's just a question of adding or taking away zeroes, right? :) Simon From simon@netthink.co.uk Thu Aug 2 18:30:06 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Thu, 2 Aug 2001 10:30:06 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Message-ID: <20010802103006.H1667@netthink.co.uk> On Wed, Aug 01, 2001 at 12:42:42AM -0400, Tim Peters wrote: > case BINARY_DIVIDE: case? Wowsers. Hey, Ruby does that too. We use function pointers, FWIW. Oh, and here's our divide, for comparison: PP(pp_divide) { dSP; dATARGET; tryAMAGICbin(div,opASSIGN); { dPOPPOPnnrl; NV value; if (right == 0.0) DIE(aTHX_ "Illegal division by zero"); value = left / right; PUSHn( value ); RETURN; } } See, this is why we need a new interpreter. :) In slightly more pseudo code: PP(pp_divide) { get_the_stack_pointer; find_somewhere_to_put_result; check_if_overloaded; { NV right = SvNV(POP); NV left = SvNV(POP); NV value; if (right == 0.0) DIE(aTHX_ "Illegal division by zero"); value = left / right; PUSHn( value ); RETURN; } } From greg@cosc.canterbury.ac.nz Fri Aug 3 05:14:07 2001 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 03 Aug 2001 16:14:07 +1200 (NZST) Subject: [Python-Dev] Proposed new module for the Python library In-Reply-To: <20010802233045.A32400@thyrsus.com> Message-ID: <200108030414.QAA01131@s454.cosc.canterbury.ac.nz> "Eric S. Raymond" > The existing code in the fileinput module is intended to treat a > sequence of files as one continuous input source. My module is > designed to iterate over files, transforming each one to a > corresponsing named output file. Still, they sound similar enough that the functionality should perhaps be merged into one module, with options for getting "concatenated" or "separate" input files. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From sdm7g@Virginia.EDU Fri Aug 3 05:20:42 2001 From: sdm7g@Virginia.EDU (Steven D. Majewski) Date: Fri, 3 Aug 2001 00:20:42 -0400 (EDT) Subject: [Python-Dev] Proposed new module for the Python library In-Reply-To: <20010802233045.A32400@thyrsus.com> Message-ID: On Thu, 2 Aug 2001, Eric S. Raymond wrote: > No. The existing code in the fileinput module is intended to treat a > sequence of files as one continuous input source. My module is > designed to iterate over files, transforming each one to a > corresponsing named output file. I did a FileUtil module for MacPython -- perhaps it was more complicated because it did all of the GUI also -- progress dialog with info lines and a cancel button, message boxes on errors, etc. You could drop a folder full of files on it and it would iterate thru them. But I found it difficult to generalize sufficiently -- there was not only different file processing methods to be defined, there were different input/output filename transformation rules, different message templates, different file selection rules, etc. Still -- it was useful, but I think I kept fiddling with it almost every time I used it. I think I started with a function that was passed a process callback function. It ended up a class that could be extended for each particular file processing job. But I think if I did it again now, I'ld use some sort of generator. There was some discussion of this: that the os.path.walk way of iterating with callbacks is really inside-out. Nested generators might be a handier building block for this sort of job -- kind of the python equivalent of a unix pipeline command. -- Steve From esr@thyrsus.com Fri Aug 3 06:45:29 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Fri, 3 Aug 2001 01:45:29 -0400 Subject: [Python-Dev] Proposed new module for the Python library In-Reply-To: <200108030414.QAA01131@s454.cosc.canterbury.ac.nz>; from greg@cosc.canterbury.ac.nz on Fri, Aug 03, 2001 at 04:14:07PM +1200 References: <20010802233045.A32400@thyrsus.com> <200108030414.QAA01131@s454.cosc.canterbury.ac.nz> Message-ID: <20010803014528.A1743@thyrsus.com> Greg Ewing : > > The existing code in the fileinput module is intended to treat a > > sequence of files as one continuous input source. My module is > > designed to iterate over files, transforming each one to a > > corresponsing named output file. > > Still, they sound similar enough that the functionality > should perhaps be merged into one module, with options > for getting "concatenated" or "separate" input files. I'd be open to that -- it's why I said "existing" code :-). It doesn't matter to me whether my functions live in the fileinput module or in a sepate ccframe module. I wrote the documentation page this morning. -- Eric S. Raymond Faith may be defined briefly as an illogical belief in the occurrence of the improbable...A man full of faith is simply one who has lost (or never had) the capacity for clear and realistic thought. He is not a mere ass: he is actually ill. -- H. L. Mencken From thomas@xs4all.net Fri Aug 3 07:11:44 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 3 Aug 2001 08:11:44 +0200 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <20010802102414.G1667@netthink.co.uk> References: <20010802102414.G1667@netthink.co.uk> Message-ID: <20010803081144.J626@xs4all.nl> On Thu, Aug 02, 2001 at 10:24:14AM -0700, Simon Cozens wrote: > On Tue, Jul 31, 2001 at 10:32:59PM +0200, Thomas Wouters wrote: > > PUSH "1" > > PUSH "2" > > BINARY_ADD > > > > for it's > > > > "1" . "2" > *cough*. Hell no! Addition and string concatenation would be > separate opcodes. (I think.) Note that I later in the same thread change my mind and say that string-concatenation should be a separate opcode :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Fri Aug 3 07:21:37 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 3 Aug 2001 08:21:37 +0200 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: Message-ID: <20010803082137.K626@xs4all.nl> On Tue, Jul 31, 2001 at 06:23:39PM -0700, Ka-Ping Yee wrote: > On Tue, 31 Jul 2001, Guido van Rossum wrote: > > Why not add this to cgi.py instead? > Because that would seem to defeat most of the point. > The point was to provide an instant, effortless improvement for all > of the Python CGI scripts out there. If programmers have to manually > edit all of their CGI scripts to insert > I don't want people to have to edit every single script. I'm sorry, Ping, but I have to disagree with you there... You might not want people to edit scripts, but I'm not sure howmany people want the functionality of their CGI script(s) significantly changed without editing their scripts. I know I wouldn't, even though nothing would break; I wouldn't feel comfortable displaying the innards of some of the CGI scripts to whoever manages to feed it faulty data. I would strongly suggest making people add a single line, along the lines of cgi.enable_debug() or cgi.install_reporter() and doing the exception-hook replacement in there. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From martin@loewis.home.cs.tu-berlin.de Fri Aug 3 07:24:02 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Fri, 3 Aug 2001 08:24:02 +0200 Subject: [Python-Dev] Python API version & optional features Message-ID: <200108030624.f736O2B00890@mira.informatik.hu-berlin.de> > Shouldn't Python automatically Do The Right Thing in that case? That > would mean wrapping the UCS4 calls in a conversion layer, which > isn't hard. I mean, it's just a question of adding or taking away > zeroes, right? :) Given the smiley at the end of your message, I don't know whether you meant your point seriously or not. I believe it is impossible for Python to do what you seem to think as the "Right Thing". E.g. modules that use PyUnicode_AS_UNICODE will behave differently when sizeof(Py_UNICODE) changes. Since this is a macro, you cannot change their behavior at run-time; they have assumption on the size of Py_UNICODE compiled-in. Regards, Martin From SBrunning@trisystems.co.uk Fri Aug 3 08:53:39 2001 From: SBrunning@trisystems.co.uk (Simon Brunning) Date: Fri, 3 Aug 2001 08:53:39 +0100 Subject: [Python-Dev] RE: Outdated Misc/cheatsheet Message-ID: <31575A892FF6D1118F5800600846864D78BF67@intrepid> > From: Andrew Kuchling [SMTP:akuchlin@mems-exchange.org] > Misc/cheatsheet is way out of date (the 'access' keyword?). I'd like > to replace it with the text version of Simon Brunning and Richard > Gruet's Python cheatsheet. Issues: Andy, Sounds like a good idea to me. > 1) Simon, do you grant permission to do that? Should I ask Richard, > too? Yes and yes. I've copied Richard in on this. > 2) The new file is significantly larger: 100K instead of just 22K. > Is that OK? > > 3) The reference will need updating for 2.1 and 2.2 changes, but > slightly obsolete is better than waaay obsolete. Simon, do you > want help updating it for 2.1/2.2? Thanks, Andy, but I'm on the Python 2.1 version - it's no more than a week away. Honest. I'll also do a 2.2 version as soon as I know what's going to be in it! Iterators and generators seem to be in for sure, and after reading your 'What's New in Python 2.2' document I understand what they do, so they aren't a problem. Ditto the other changes mentioned in your document. But I'm not sure about the Type/Class unification stuff - I don't know if it's going in, and I don't understand it, either! So here, I could use some help. Cheers, Simon Brunning TriSystems Ltd. sbrunning@trisystems.co.uk ----------------------------------------------------------------------- The information in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution, or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. TriSystems Ltd. cannot accept liability for statements made which are clearly the senders own. From mal@lemburg.com Fri Aug 3 10:13:56 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 03 Aug 2001 11:13:56 +0200 Subject: [Python-Dev] Python API version & optional features References: <200108030624.f736O2B00890@mira.informatik.hu-berlin.de> Message-ID: <3B6A6B54.3BA008C9@lemburg.com> "Martin v. Loewis" wrote: > > > Shouldn't Python automatically Do The Right Thing in that case? That > > would mean wrapping the UCS4 calls in a conversion layer, which > > isn't hard. I mean, it's just a question of adding or taking away > > zeroes, right? :) > > Given the smiley at the end of your message, I don't know whether you > meant your point seriously or not. > > I believe it is impossible for Python to do what you seem to think as > the "Right Thing". E.g. modules that use PyUnicode_AS_UNICODE will > behave differently when sizeof(Py_UNICODE) changes. Since this is a > macro, you cannot change their behavior at run-time; they have > assumption on the size of Py_UNICODE compiled-in. I've already checked in a patch which applies API name mangling to the Unicode APIs. The linker will then generate error messages which Python reports as ImportError in case there should be a mismatch in the sizeof(Py_UNICODE). The only thing which we require is that a Unicode aware extension use at least one of the mangled APIs -- shouldn't be much of a requirement, though :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From jack@oratrix.nl Fri Aug 3 11:44:51 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri, 03 Aug 2001 12:44:51 +0200 Subject: [Python-Dev] test_quopri is iso-latin-1 centric Message-ID: <20010803104451.88356303181@snelboot.oratrix.nl> I think test_quopri is too latin-1 centric. Alternatively, I'm mistaken, please enlighten me (is quopri supposed to convert to/from latin1, in stead of to/from local convention, as I expect?). What happens is the following. The test_quopri.py source contains lots of special characters, for instance the spanish-upsidedown-exclamation (to mention the first one). It converts this with quopri and expects to see =A1. This all works fine if your Python source is interpreted in latin-1, but it fails for other encoding. For instance, on my Mac, Python source is in MacRoman encoding. CVS knows all about this, so it happily converts the latin-1-upsidedown-exclam to a macroman-upsidedown-exclam, and if I look at the source code I see the same glyph as I see on Unix. However, the MacRoman encoding for this character is =D1, so test_quopri fails. I'm surprised the test doesn't fail on Windows as well, or do Windows pythonistas generally work with source in latin1? -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From loewis@informatik.hu-berlin.de Fri Aug 3 12:43:01 2001 From: loewis@informatik.hu-berlin.de (Martin von Loewis) Date: Fri, 3 Aug 2001 13:43:01 +0200 Subject: [Python-Dev] test_quopri is iso-latin-1 centric Message-ID: <200108031143.f73Bh1708106@artagnan.informatik.hu-berlin.de> > I think test_quopri is too latin-1 centric. Strictly speaking, there is nothing latin-1 centric in test_quopri.py whatsoever. > For instance, on my Mac, Python source is in MacRoman encoding. CVS > knows all about this, so it happily converts the > latin-1-upsidedown-exclam to a macroman-upsidedown-exclam, and if I > look at the source code I see the same glyph as I see on Unix. This is the problem. Python source code is not in Latin-1; bytes inside strings and comments are "as-is". So the CVS "binary" mode would come closer as to how python files should be treated, although you'd still would want to convert line-endings. > I'm surprised the test doesn't fail on Windows as well, or do > Windows pythonistas generally work with source in latin1? Most of these people probably use code page 1252, which is identical to latin-1 except in the range 0x80 to 0x9f. For test_quopri.py, the best thing would be to replace the characters outside range(128) to \x escaped ones, to avoid the problem with Mac CVS (which really is the problem here - if you unpack Python from the source distribution, the test should pass fine on your system). Regards, Martin From just@letterror.com Fri Aug 3 12:49:12 2001 From: just@letterror.com (Just van Rossum) Date: Fri, 3 Aug 2001 13:49:12 +0200 Subject: [Python-Dev] test_quopri is iso-latin-1 centric In-Reply-To: <200108031143.f73Bh1708106@artagnan.informatik.hu-berlin.de> Message-ID: <20010803134915-r01010700-a5cd514b-0910-010c@10.0.0.9> Martin von Loewis wrote: > For test_quopri.py, the best thing would be to replace the characters > outside range(128) to \x escaped ones, to avoid the problem with Mac > CVS (which really is the problem here - if you unpack Python from the > source distribution, the test should pass fine on your system). It doesn't matter what the problem is: the Python sources that are supposed to be portable should be 7-bit clean. Just From guido@zope.com Fri Aug 3 13:33:40 2001 From: guido@zope.com (Guido van Rossum) Date: Fri, 03 Aug 2001 08:33:40 -0400 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: Your message of "Thu, 02 Aug 2001 15:01:09 PDT." <20010802150109.B2033@netthink.co.uk> References: <20010802150109.B2033@netthink.co.uk> Message-ID: <200108031233.IAA01557@cj20424-a.reston1.va.home.com> > I also strongly agree. CGI's one application that draws a lot of people > into programming, and many of them for the first time - you get a lot of > web designers and the like wanting to go a bit further, and it'd be > nice to get them into the good kind of snake (Python) instead of the > bad kind of snake. (ASP) OK. CGI is important. But Ping's tracebackhook still shouldn't be the default. I'm willing to entertain a new command line option for it though, so that CGI scripts can use #!/usr/bin/env python -C as their first line. --Guido van Rossum (home page: http://www.python.org/~guido/) From jack@oratrix.nl Fri Aug 3 13:38:57 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri, 03 Aug 2001 14:38:57 +0200 Subject: [Python-Dev] test_quopri is iso-latin-1 centric In-Reply-To: Message by Martin von Loewis , Fri, 3 Aug 2001 13:43:01 +0200 , <200108031143.f73Bh1708106@artagnan.informatik.hu-berlin.de> Message-ID: <20010803123857.99202303181@snelboot.oratrix.nl> > This is the problem. Python source code is not in Latin-1; bytes > inside strings and comments are "as-is". So the CVS "binary" mode > would come closer as to how python files should be treated, although > you'd still would want to convert line-endings. Is this an "official" policy? As such I disagree with it. Either all Python source should be 7-bit clean or the source should be converted to the local convention, even for strings and comments. But if it is an official policy I'd like to hear it (who wants to channel Guido here?), because there's heaps of MacPython stuff that'll have to be converted back (I did the forward conversion a couple of months ago). As a point-in-case for local convention, you wouldn't want to be know as Martin von Lupsidedownquestionmarkwis by the chinese MacPython community or the Polish PythonWin users, would you? -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From guido@zope.com Fri Aug 3 13:47:43 2001 From: guido@zope.com (Guido van Rossum) Date: Fri, 03 Aug 2001 08:47:43 -0400 Subject: [Python-Dev] test_quopri is iso-latin-1 centric In-Reply-To: Your message of "Fri, 03 Aug 2001 13:43:01 +0200." <200108031143.f73Bh1708106@artagnan.informatik.hu-berlin.de> References: <200108031143.f73Bh1708106@artagnan.informatik.hu-berlin.de> Message-ID: <200108031247.IAA01719@cj20424-a.reston1.va.home.com> > > I think test_quopri is too latin-1 centric. > > Strictly speaking, there is nothing latin-1 centric in test_quopri.py > whatsoever. > > > For instance, on my Mac, Python source is in MacRoman encoding. CVS > > knows all about this, so it happily converts the > > latin-1-upsidedown-exclam to a macroman-upsidedown-exclam, and if I > > look at the source code I see the same glyph as I see on Unix. > > This is the problem. Python source code is not in Latin-1; bytes > inside strings and comments are "as-is". So the CVS "binary" mode > would come closer as to how python files should be treated, although > you'd still would want to convert line-endings. > > > I'm surprised the test doesn't fail on Windows as well, or do > > Windows pythonistas generally work with source in latin1? > > Most of these people probably use code page 1252, which is identical > to latin-1 except in the range 0x80 to 0x9f. > > For test_quopri.py, the best thing would be to replace the characters > outside range(128) to \x escaped ones, to avoid the problem with Mac > CVS (which really is the problem here - if you unpack Python from the > source distribution, the test should pass fine on your system). Well, exactly because Python source code is not in Latin-1, the test_quopri.py file is wrong IMO, because it uses non-ASCII characters. So please go ahead and change it into hex constants... (This doesn't occur on Windows because, I'm assuming, the Windows CVS client isn't as "smart" as the Mac CVS client.) --Guido van Rossum (home page: http://www.python.org/~guido/) From akuchlin@mems-exchange.org Fri Aug 3 13:48:23 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 3 Aug 2001 08:48:23 -0400 Subject: [Python-Dev] RE: Outdated Misc/cheatsheet In-Reply-To: <31575A892FF6D1118F5800600846864D78BF67@intrepid>; from SBrunning@trisystems.co.uk on Fri, Aug 03, 2001 at 08:53:39AM +0100 References: <31575A892FF6D1118F5800600846864D78BF67@intrepid> Message-ID: <20010803084823.A30133@ute.cnri.reston.va.us> On Fri, Aug 03, 2001 at 08:53:39AM +0100, Simon Brunning wrote: >Thanks, Andy, but I'm on the Python 2.1 version - it's no more than a week >away. Honest. Great! I'll wait for it, then. >But I'm not sure about the Type/Class unification stuff - I don't know if >it's going in, and I don't understand it, either! So here, I could use some >help. Guido wrote up a description that's at http://www.mirror.ac.uk/sites/ftp.python.org/pub/www.python.org/2.2/descrintro.html . I do plan to write a section on it, now that it seems closer to inevitable, but I don't think it affects beginner or intermediate-level programmers much. Mostly the changes will affect wizards who want to write ExtensionClass-like things. And thanks! --amk From guido@zope.com Fri Aug 3 13:49:11 2001 From: guido@zope.com (Guido van Rossum) Date: Fri, 03 Aug 2001 08:49:11 -0400 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: Your message of "Fri, 03 Aug 2001 08:33:40 EDT." <200108031233.IAA01557@cj20424-a.reston1.va.home.com> References: <20010802150109.B2033@netthink.co.uk> <200108031233.IAA01557@cj20424-a.reston1.va.home.com> Message-ID: <200108031249.IAA01751@cj20424-a.reston1.va.home.com> > I'm willing to entertain a new command line option for > it though, so that CGI scripts can use > > #!/usr/bin/env python -C > > as their first line. But it's fine to require making a call to cgi.whatever() too, and that's certainly a lot less work! --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@zope.com Fri Aug 3 13:50:35 2001 From: guido@zope.com (Guido van Rossum) Date: Fri, 03 Aug 2001 08:50:35 -0400 Subject: [Python-Dev] test_quopri is iso-latin-1 centric In-Reply-To: Your message of "Fri, 03 Aug 2001 14:38:57 +0200." <20010803123857.99202303181@snelboot.oratrix.nl> References: <20010803123857.99202303181@snelboot.oratrix.nl> Message-ID: <200108031250.IAA01771@cj20424-a.reston1.va.home.com> > > This is the problem. Python source code is not in Latin-1; bytes > > inside strings and comments are "as-is". So the CVS "binary" mode > > would come closer as to how python files should be treated, although > > you'd still would want to convert line-endings. > > Is this an "official" policy? As such I disagree with it. Either all > Python source should be 7-bit clean or the source should be > converted to the local convention, even for strings and > comments. But if it is an official policy I'd like to hear it (who > wants to channel Guido here?), because there's heaps of MacPython > stuff that'll have to be converted back (I did the forward > conversion a couple of months ago). Until we've picked a way to indicate the source encoding, it should definitely be 7-bit clean. > As a point-in-case for local convention, you wouldn't want to be > know as Martin von Lupsidedownquestionmarkwis by the chinese > MacPython community or the Polish PythonWin users, would you? ROFL :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@zope.com Fri Aug 3 13:54:17 2001 From: guido@zope.com (Guido van Rossum) Date: Fri, 03 Aug 2001 08:54:17 -0400 Subject: [Python-Dev] WebDAV in python 2.2? In-Reply-To: Your message of "Thu, 02 Aug 2001 22:18:44 EDT." References: Message-ID: <200108031254.IAA01818@cj20424-a.reston1.va.home.com> > GvR also wondered if Python 2.2 should support DAV ("That's another > open protocol that Python could easily support out of the box.") and > /F agreed, but Greg Stein, the DAV expert and champion on python-dev, > remained silent. Greg, I now know where you were. Since you're back, any comments? --Guido van Rossum (home page: http://www.python.org/~guido/) From gward@python.net Fri Aug 3 02:49:44 2001 From: gward@python.net (Greg Ward) Date: Thu, 2 Aug 2001 21:49:44 -0400 Subject: [Python-Dev] test results In-Reply-To: <15209.24217.765601.582838@beluga.mojam.com>; from skip@pobox.com on Thu, Aug 02, 2001 at 09:07:21AM -0500 References: <15209.24217.765601.582838@beluga.mojam.com> Message-ID: <20010802214944.A8810@gerg.ca> On 02 August 2001, Skip Montanaro said: > The linuxaudiodev test never seems to work for me (though I do hear an > agonizing wail (is that supposed to be Homer Simpson?) from my laptop's > speakers when it's run). The result is always: I am pleased-as-punch to report that test_linuxaudiodev *finally* works for me, both in 2.2a1 and the CVS as of this afternoon (after Tim's big merge). Also, the rest of the test suite seems to have passed just fine for me with the latest CVS; this is a Progeny Debian 1.0 system (pretty close to Debian 2.2 stable). ("Seems to have passed" because there's some aborted work from a year or so ago where I was trying to fix the lexer's handling of line termination. Gave up in despair -- brain too small -- but I still have non-working tokenizer tests sitting in my CVS working dir. Sigh.) Greg -- Greg Ward - programmer-at-large gward@python.net http://starship.python.net/~gward/ Always look on the bright side of life. From nas@python.ca Fri Aug 3 14:53:16 2001 From: nas@python.ca (Neil Schemenauer) Date: Fri, 3 Aug 2001 06:53:16 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <20010802103006.H1667@netthink.co.uk>; from simon@netthink.co.uk on Thu, Aug 02, 2001 at 10:30:06AM -0700 References: <20010802103006.H1667@netthink.co.uk> Message-ID: <20010803065316.A19638@glacier.fnational.com> Simon Cozens wrote: > On Wed, Aug 01, 2001 at 12:42:42AM -0400, Tim Peters wrote: > > case BINARY_DIVIDE: > > case? Wowsers. Hey, Ruby does that too. We use function pointers, > FWIW. People (including me) have tried using threaded code (http://www.complang.tuwien.ac.at/forth/threaded-code.html) and have not measured any significant advantage. I suspect the opcodes to much longer to evaluate than doing the dispatch. > Oh, and here's our divide, for comparison: > > PP(pp_divide) > { > dSP; dATARGET; tryAMAGICbin(div,opASSIGN); > { > dPOPPOPnnrl; > NV value; > if (right == 0.0) > DIE(aTHX_ "Illegal division by zero"); > value = left / right; > PUSHn( value ); > RETURN; > } > } Does every Perl opcode do a function call? Neil From nas@python.ca Fri Aug 3 14:57:36 2001 From: nas@python.ca (Neil Schemenauer) Date: Fri, 3 Aug 2001 06:57:36 -0700 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: <20010803082137.K626@xs4all.nl>; from thomas@xs4all.net on Fri, Aug 03, 2001 at 08:21:37AM +0200 References: <20010803082137.K626@xs4all.nl> Message-ID: <20010803065736.B19638@glacier.fnational.com> Thomas Wouters wrote: > I'm sorry, Ping, but I have to disagree with you there... You might not want > people to edit scripts, but I'm not sure howmany people want the > functionality of their CGI script(s) significantly changed without editing > their scripts. I know I wouldn't [...] I wouldn't either. Doing an import and/or a function is not too hard. I'm -1 on magically enabling Ping's traceback thing (cool as it is). Neil From thomas@xs4all.net Fri Aug 3 15:17:04 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 3 Aug 2001 16:17:04 +0200 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <20010802103006.H1667@netthink.co.uk> References: <20010802103006.H1667@netthink.co.uk> Message-ID: <20010803161704.M626@xs4all.nl> On Thu, Aug 02, 2001 at 10:30:06AM -0700, Simon Cozens wrote: > On Wed, Aug 01, 2001 at 12:42:42AM -0400, Tim Peters wrote: > > case BINARY_DIVIDE: > case? Wowsers. Hey, Ruby does that too. We use function pointers, > FWIW. Well, we use *both* :) But function-pointers alone aren't enough, are they ? Or are the bytecodes themselves the function pointers ? *shudder*. I guess you do opcodes as indices into an array-of-function-pointers ? There was talk about doing that for Python too, but I believe it ended up being not significantly faster. Some people have seen bigger speedups just reorganizing the case's in the Big Switch, but those same changes slowed things down on other platforms. > Oh, and here's our divide, for comparison: > In slightly more pseudo code: > PP(pp_divide) > { > get_the_stack_pointer; > find_somewhere_to_put_result; > check_if_overloaded; > { > NV right = SvNV(POP); > NV left = SvNV(POP); > NV value; > if (right == 0.0) > DIE(aTHX_ "Illegal division by zero"); > value = left / right; > PUSHn( value ); > RETURN; > } > } Right... Almost the same, except that the check_if_overloaded is a bigger check for Python, and more often true (I suspect.) And, as was said before, assignment isn't a value-filling operation, so instead of 'value = left / right' we have 'PyObject *value = new_object(left / right)' (pseudocode, again) and just push value onto the stack (to be stored into a variable by a subsequent opcode, or just kept in the stack for the running expression.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From just@letterror.com Fri Aug 3 15:44:47 2001 From: just@letterror.com (Just van Rossum) Date: Fri, 3 Aug 2001 16:44:47 +0200 Subject: [Python-Dev] (Idea) subclassable string: path object? Message-ID: <20010803164453-r01010700-1c882179-0910-010c@10.0.0.11> Every once in a while I wished for an path object to manipulate file system paths. Things like os.path.join(a, b, c, os.path.splitext(os.path.basename(p))[0] + ".ext") quickly get frustrating (so of course I never write them like that ;-). I thought of implementing a path object several times, but always stopped when I realized (for the Nth time ;-) that you'd then have to do something like file = open(p.tostring()) whenever you want to *use* your pat. That doesn't help at all. But: since strings are now subclassable (there are, aren't they?) this should no longer be a problem! Would it be a worthwile project to design and implement a path object for the standard library? Just From skip@pobox.com (Skip Montanaro) Fri Aug 3 16:00:26 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 3 Aug 2001 10:00:26 -0500 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: <20010803082137.K626@xs4all.nl> References: <20010803082137.K626@xs4all.nl> Message-ID: <15210.48266.673906.863831@beluga.mojam.com> >> I don't want people to have to edit every single script. Thomas> I'm sorry, Ping, but I have to disagree with you there... You Thomas> might not want people to edit scripts, but I'm not sure howmany Thomas> people want the functionality of their CGI script(s) Thomas> significantly changed without editing their scripts. I have to agree. Perhaps just as significant, what would happen to a CGI script that already attempts to do something to direct error output to a file or mail message the webmaster will see? Most of my users wouldn't know a traceback (simpler or otherwise) from a hole in the ground. I think I have perhaps two users who would understand that I was trying to create a more readable report. Skip From jack@oratrix.nl Fri Aug 3 16:00:09 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri, 03 Aug 2001 17:00:09 +0200 Subject: [Python-Dev] (Idea) subclassable string: path object? In-Reply-To: Message by Just van Rossum , Fri, 3 Aug 2001 16:44:47 +0200 , <20010803164453-r01010700-1c882179-0910-010c@10.0.0.11> Message-ID: <20010803150014.BF9D7162E14@oratrix.oratrix.nl> One problem I see is that what you'd really like is to overload + and split and such on path objects. But this creates a problem if you then pass this path object to something that expects old-fashioned strings: if it wants to manipulate that path it will use string operations, which suddenly have different semantics... Recently, Just van Rossum said: > Every once in a while I wished for an path object to manipulate file system > paths. Things like > os.path.join(a, b, c, os.path.splitext(os.path.basename(p))[0] + ".ext") > quickly get frustrating (so of course I never write them like that ;-). > > I thought of implementing a path object several times, but always stopped whe > n I > realized (for the Nth time ;-) that you'd then have to do something like > file = open(p.tostring()) > whenever you want to *use* your pat. That doesn't help at all. > > But: since strings are now subclassable (there are, aren't they?) this should > no > longer be a problem! > > Would it be a worthwile project to design and implement a path object for the > standard library? > > Just > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From just@letterror.com Fri Aug 3 16:11:09 2001 From: just@letterror.com (Just van Rossum) Date: Fri, 3 Aug 2001 17:11:09 +0200 Subject: [Python-Dev] (Idea) subclassable string: path object? In-Reply-To: <20010803150014.BF9D7162E14@oratrix.oratrix.nl> Message-ID: <20010803171113-r01010700-2e14948d-0910-010c@10.0.0.11> Jack Jansen wrote: > One problem I see is that what you'd really like is to overload + and > split and such on path objects. Yup. Although it's unclear to me what would be the best behavior for +, because it could mean several things: 1) p = Path(a_directory) + filename # os.path.join(a_directory, filename) 2) p = Path(a_file) + ".ext" # a_file + ".ext" 1) is very useful, but then 2) wouldn't work... So it's not all straightforward. > But this creates a problem if you then > pass this path object to something that expects old-fashioned strings: > if it wants to manipulate that path it will use string operations, > which suddenly have different semantics... Yes, that's important to consider, too. On the other hand: the os.path.* functions could be made aware of the path object, similar to string.py, except it should fall back to the existing code when a string gets passed. Just From Samuele Pedroni Fri Aug 3 16:30:48 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Fri, 3 Aug 2001 17:30:48 +0200 (MET DST) Subject: [Python-Dev] Parrot -- should life imitate satire? Message-ID: <200108031530.RAA14515@core.inf.ethz.ch> [Dan Sugalski] > >A question: when you say variable you mean variable (perl sense of that) > >or object. It has already been pointed out but it's really confusing > >from the point of view of python terminology. Will perl6 have only > >variables which contain objects, truly references to objects like Python > >or ...? > > Well, it sort of looks like: > > name--->variable--->value > Ok, I got your explanation. I see that perl6 will still have a rather convoluted model and quiet different from Python... > I think the variable here would correspond to a Python object Possible but not in a natural way. > I think it's the potential existence of references (or pointers, or > whatever you want to call them) that necessitate the three-level split. Makes perfectly sense. > sounds like the things I'm calling 'name' and 'variable' are much more > tightly bound in Python. No, a name/variable is just slot in a namespace/ symbol table, that contains a pointer to a value/object which contains data and also points to the type-specific vtable. Quiet different. Do your values come with a vtable that then is "copied" in the variable? Your vtable are for things like scalar, vs array vs. hash. Do they possibly change when the value change (e.g. string to integer for a scalar var)? > the way Python would do things, in which case Python variable vtables might > just have an "overwrite myself with the incoming info" function, rather > than allow it to be overridden) No, unless you have a generic Python PMC which mirrors/ or uses a vtable that goes together with the value (in the sense of your scheme). Otherwise I think python assigment would be more some flavor of reference (perl sense) assignment. If you map Python built-in data object to PMC then assignment is a just a matter of namespace manipulation. For the momement I don't really see a natural way of mapping python semantics over your three-level scheme, maybe is just the lack of details. Especially I see a lot of problem if you want a mapping that can also allow interoperability beetween Python and Perl code. > Yup. I am profoundly tempted, and I think it will go in, to have a "stable > code" switch that can be given to the optimizer which guarantees that, once > the compilation phase is done and we finally get to run for real, code will > not change. And then apply all the advanced optimization techniques at that > point. > I don't really get this. In python the kind of dynamic changes you can do are reloading code, changing the base classes of a class, adding/substituting methods in a class or instance. When an application really need one of this techniques with your philosophy it will be treated as 2nd-class citizen. What I was referring were de-optimization techniques (yes they are a mess to deal with for the compiler writer). Reading about native compilation of dynamic languages I have been conviced that in general performance is a matter of implemention and it better to keep it transparent to the user, having constructs that should be avoided not for design reasons but because the compiler uses some tricks and not others is a bad thing, for a dynamic language IMHO. And in any case you still have the problem with polymorphic calls and to deal with them - in case you want to speed them up - you still have to make asssumptions at runtime and possibly take them back. > > >For the rest is just speed vs. memory, customization is a clear example > >for that. > > Sure. The one nice thing that both perl and python have going for them is > the ultimate presence of all the source (or a high-enough level > representation of it to feed to the optimizer) at run time. We can tell > with more surety what side-effects various methods and functions in > external code libraries will have, and we can get more aggressive (maybe) > because of it. That is probably not directly true for polimorphic call sites. But of course you can type-profile the program at runtime. > Unlike, say, C, where passing a pointer to a routine outside > what the compiler can see will hobble the optimizer a whole lot in many > cases... > regards, Samuele Pedroni. From cgw@alum.mit.edu Fri Aug 3 17:13:40 2001 From: cgw@alum.mit.edu (Charles G Waldman) Date: Fri, 3 Aug 2001 11:13:40 -0500 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: References: Message-ID: <15210.52660.887708.675013@transamoeba.dyndns.org> On Fri, 3 Aug 2001 at 16:17:04 +0200, Thomas Wouters wrote: > Some people have seen bigger speedups just > reorganizing the case's in the Big Switch, but those same changes slowed > things down on other platforms. Clearly, part of Python's ./configure should try all different permutations of the 107 "case" statements in the Big Switch, and find the arrangement that maximizes Pystones on a per-platform basis. just-kidding-ly y'rs, -C From barry@zope.com Fri Aug 3 17:39:50 2001 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 3 Aug 2001 12:39:50 -0400 Subject: [Python-Dev] cgitb.py for Python 2.2 References: <15206.8227.652539.471067@anthem.wooz.org> <20010731075258.A2757@ute.cnri.reston.va.us> Message-ID: <15210.54230.552520.602993@anthem.wooz.org> >>>>> "AK" == Andrew Kuchling writes: AK> +0 from me, though I also think it would be better in cgi.py AK> and not in site.py. It would also be useful if it could mail AK> tracebacks and return a non-committal but secure error message AK> to the browser; I'll contribute that as a patch if cgitb.py AK> goes in. (Or should that be cgi/tb.py? Hmm...) Whatever happened to PEP 222? :) From barry@zope.com Fri Aug 3 17:46:15 2001 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 3 Aug 2001 12:46:15 -0400 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 References: <20010803082137.K626@xs4all.nl> <15210.48266.673906.863831@beluga.mojam.com> Message-ID: <15210.54615.364193.422123@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> I have to agree. Perhaps just as significant, what would SM> happen to a CGI script that already attempts to do something SM> to direct error output to a file or mail message the webmaster SM> will see? That's my major concern about enabling by default too. I'd now have to have a way to /disable/ it explicitly, but of course, I'd have to wrap that in a try/except in case I'm using an older cgi.py. SM> Most of my users wouldn't know a traceback (simpler or SM> otherwise) from a hole in the ground. I think I have perhaps SM> two users who would understand that I was trying to create a SM> more readable report. I agree. cgitb.py is without a doubt Very Cool and should definitely be in the std library. But it's of use only to a handful of people, primarily the developers of cgi module-based scripts. +1 on additing it to std library +1 on turning it on only with an explicit action (although I could live with enabling via side-effect on import of cgitb.py, count me -0 for that). -Barry From guido@zope.com Fri Aug 3 17:50:23 2001 From: guido@zope.com (Guido van Rossum) Date: Fri, 03 Aug 2001 12:50:23 -0400 Subject: [Python-Dev] (Idea) subclassable string: path object? In-Reply-To: Your message of "Fri, 03 Aug 2001 16:44:47 +0200." <20010803164453-r01010700-1c882179-0910-010c@10.0.0.11> References: <20010803164453-r01010700-1c882179-0910-010c@10.0.0.11> Message-ID: <200108031650.MAA03181@cj20424-a.reston1.va.home.com> > Every once in a while I wished for an path object to manipulate file > system paths. Things like > os.path.join(a, b, c, os.path.splitext(os.path.basename(p))[0] + ".ext") > quickly get frustrating (so of course I never write them like that ;-). > > I thought of implementing a path object several times, but always > stopped when I realized (for the Nth time ;-) that you'd then have > to do something like > file = open(p.tostring()) > whenever you want to *use* your pat. That doesn't help at all. Two better solutions: (1) If the Path class defines a __str__() method, you can write open(p). (2) You could add an open method to the Path class, and write p.open(). > But: since strings are now subclassable (there are, aren't they?) > this should no longer be a problem! > > Would it be a worthwile project to design and implement a path > object for the standard library? I'm lukewarm on adding this to the std library. Java has such a thing and I never warmed up to it. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@zope.com Fri Aug 3 17:52:03 2001 From: guido@zope.com (Guido van Rossum) Date: Fri, 03 Aug 2001 12:52:03 -0400 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: Your message of "Fri, 03 Aug 2001 12:46:15 EDT." <15210.54615.364193.422123@anthem.wooz.org> References: <20010803082137.K626@xs4all.nl> <15210.48266.673906.863831@beluga.mojam.com> <15210.54615.364193.422123@anthem.wooz.org> Message-ID: <200108031652.MAA03215@cj20424-a.reston1.va.home.com> > +1 on additing it to std library > +1 on turning it on only with an explicit action (although I could Agreed. > live with enabling via side-effect on import of cgitb.py, count me > -0 for that). -1 on enabling it with "import cgitb". Modules with import side effects are evil (and trickier to introspect, as Ping should know :-). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@zope.com Fri Aug 3 18:11:36 2001 From: guido@zope.com (Guido van Rossum) Date: Fri, 03 Aug 2001 13:11:36 -0400 Subject: [Python-Dev] www.python.org is back! Message-ID: <200108031711.NAA03374@cj20424-a.reston1.va.home.com> The Python sites http://www.python.org and ftp://ftp.python.org are back up, thanks to the energetic help of Thomas Wouters and XS4ALL (the dutch ISP that's his employer). We're still fetching some missing files from a mirror, so please be patient if your favorite download isn't there yet. We're also still working on restoring the FAQ wizard and the 2.0 MoinMoin and restarting the Ultraseek engine (search.python.org) -- I hope this will be completed later today. Thanks again to XS4ALL (http://www.xs4all.nl/) and Thomas! --Guido van Rossum (home page: http://www.python.org/~guido/) From just@letterror.com Fri Aug 3 18:21:53 2001 From: just@letterror.com (Just van Rossum) Date: Fri, 3 Aug 2001 19:21:53 +0200 Subject: [Python-Dev] (Idea) subclassable string: path object? In-Reply-To: <200108031650.MAA03181@cj20424-a.reston1.va.home.com> Message-ID: <20010803192155-r01010700-eaaed0b3-0910-010c@10.0.0.11> Guido van Rossum wrote: > Two better solutions: > > (1) If the Path class defines a __str__() method, you can write open(p). Ah, I had no idea that works these days: excellent, much nicer! > (2) You could add an open method to the Path class, and write p.open(). Mwah. I quite like (1). Maybe I'll just write a Path class and use it for a while, then I can be more sure of it's usefulness (or unusefulness...). I think it'll be a lot like string methods, in the sense that if all you know is the string module string methods seems unneccesary and overkill, but once you start using them you notice how much better it is. I see a Path object as an attempt to OO-ify os.path. I'll report back if it works out well. Just From tim.one@home.com Fri Aug 3 18:38:33 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 3 Aug 2001 13:38:33 -0400 Subject: [Python-Dev] www.python.org is back! In-Reply-To: <200108031711.NAA03374@cj20424-a.reston1.va.home.com> Message-ID: > The Python sites http://www.python.org and ftp://ftp.python.org are > back up, thanks to the energetic help of Thomas Wouters and XS4ALL > (the dutch ISP that's his employer). Not only up, but my response time browsing python.org is dramatically faster going to the Netherlands than to CNRI down the street. Downloads are much zippier too. Sweet. wondering-why-we-had-to-twist-thomas's-arm-so-hard-ly y'rs - tim From DavidA@ActiveState.com Fri Aug 3 19:27:27 2001 From: DavidA@ActiveState.com (David Ascher) Date: Fri, 03 Aug 2001 11:27:27 -0700 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 References: <20010802150109.B2033@netthink.co.uk> <200108031233.IAA01557@cj20424-a.reston1.va.home.com> <200108031249.IAA01751@cj20424-a.reston1.va.home.com> Message-ID: <3B6AED0F.AC515C76@ActiveState.com> Guido van Rossum wrote: > > > I'm willing to entertain a new command line option for > > it though, so that CGI scripts can use > > > > #!/usr/bin/env python -C > > > > as their first line. > > But it's fine to require making a call to cgi.whatever() too, and > that's certainly a lot less work! Furthermore, I believe that it's not always easy to get web servers to respect command line options set in shebang lines, so my guess is that doing cgi.whatever() is going to work for more people (e.g. people who have CGI access but not webserver config access). --david From sdm7g@Virginia.EDU Fri Aug 3 19:19:24 2001 From: sdm7g@Virginia.EDU (Steven D. Majewski) Date: Fri, 3 Aug 2001 14:19:24 -0400 (EDT) Subject: [Python-Dev] www.python.org is back! In-Reply-To: Message-ID: On Fri, 3 Aug 2001, Tim Peters wrote: > > The Python sites http://www.python.org and ftp://ftp.python.org are > > back up, thanks to the energetic help of Thomas Wouters and XS4ALL > > (the dutch ISP that's his employer). > > Not only up, but my response time browsing python.org is dramatically faster > going to the Netherlands than to CNRI down the street. Downloads are much > zippier too. Sweet. Is the Netherlands spinward or anti-spinward from your location with respect to the earth's rotation? I think you are 'down spin' . How about from CNRI ? That might explain it: less friction on the electrons flowing spinward. Waiting for my Nobel Prize! -- Steve From fdrake@acm.org Fri Aug 3 19:18:01 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 3 Aug 2001 14:18:01 -0400 (EDT) Subject: [Python-Dev] www.python.org is back! In-Reply-To: References: <200108031711.NAA03374@cj20424-a.reston1.va.home.com> Message-ID: <15210.60121.118788.294847@cj42289-a.reston1.va.home.com> Tim Peters writes: > wondering-why-we-had-to-twist-thomas's-arm-so-hard-ly y'rs - tim heck-just-wondering-why-we-had-to-drive-so-many-nails-in-CNRI's- coffin-ly y'rs -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From gmcm@hypernet.com Fri Aug 3 20:07:51 2001 From: gmcm@hypernet.com (Gordon McMillan) Date: Fri, 3 Aug 2001 15:07:51 -0400 Subject: [Python-Dev] www.python.org is back! In-Reply-To: References: Message-ID: <3B6ABE47.2090.22720523@localhost> Steven D. Majewski wrote: > > On Fri, 3 Aug 2001, Tim Peters wrote: > > > > The Python sites http://www.python.org and > > > ftp://ftp.python.org are back up, thanks to the energetic > > > help of Thomas Wouters and XS4ALL (the dutch ISP that's his > > > employer). > > > > Not only up, but my response time browsing python.org is > > dramatically faster going to the Netherlands than to CNRI down > > the street. Downloads are much zippier too. Sweet. > > Is the Netherlands spinward or anti-spinward from your location > with respect to the earth's rotation? I think you are 'down > spin' . How about from CNRI ? That might explain it: less > friction on the electrons flowing spinward. > > Waiting for my Nobel Prize! That's spinward from you, so unless you go pick it up, you'll never get it. - Gordon From gstein@lyra.org Fri Aug 3 20:12:54 2001 From: gstein@lyra.org (Greg Stein) Date: Fri, 3 Aug 2001 12:12:54 -0700 Subject: [Python-Dev] Re: WebDAV in python 2.2? In-Reply-To: <200108031254.IAA01818@cj20424-a.reston1.va.home.com>; from guido@zope.com on Fri, Aug 03, 2001 at 08:54:17AM -0400 References: <200108031254.IAA01818@cj20424-a.reston1.va.home.com> Message-ID: <20010803121254.B29583@lyra.org> On Fri, Aug 03, 2001 at 08:54:17AM -0400, Guido van Rossum wrote: > > GvR also wondered if Python 2.2 should support DAV ("That's another > > open protocol that Python could easily support out of the box.") and > > /F agreed, but Greg Stein, the DAV expert and champion on python-dev, > > remained silent. > > Greg, I now know where you were. Since you're back, any comments? I saw that thread while I was deleting over 200,000 duplicate messages from my inbox :-( ... I just haven't got to that part of my "real" inbox yet. My intent was to write a short PEP because there would be a couple modules to add to the lib: 1) httpauth -- subclasses httplib.HTTPConnection to add authenticated connections. Initially using the Basic auth method, but it could also be expanded to Digest (and others) 2) davlib -- my existing module. needs a couple items of work: allow alternate/fallback XML parsers (it uses Expat + qp_xml right now), and use the authenticated HTTP connections. That is at least a start on what needs to happen. We can flesh it out with a PEP or by checking in the modules. (Moshe and I talked about httpauth a bit at the last Python Conference; I know he is interested in working on something like that) I'm not sure how we normally write tests for network-based modules, but it shouldn't be too hard to have them run against test.webdav.org. Doc should be pretty straight-forward. Cheers, -g -- Greg Stein, http://www.lyra.org/ From tim.one@home.com Fri Aug 3 21:32:45 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 3 Aug 2001 16:32:45 -0400 Subject: [Python-Dev] test_quopri is iso-latin-1 centric In-Reply-To: <20010803104451.88356303181@snelboot.oratrix.nl> Message-ID: IIRC, I checked in test_quopri.py without noticing this (although I did notice that it relied on significant trailing whitespace!), so I'll fix it as penance. Note that the C std says the result of writing a text-mode file is *defined* only if the characters written are in range(32, 127) (note: chr(127) isn't suppoted) plus tab and newline. Also that space characters before a newline may disappear, and that the last line written must terminate with a newline. Jack's bad experience is one example of why the std says all this stuff. iow-c-text-mode-is-useless-ly y'rs - tim From tim.one@home.com Fri Aug 3 21:51:43 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 3 Aug 2001 16:51:43 -0400 Subject: [Python-Dev] test_quopri is iso-latin-1 centric In-Reply-To: <20010803123857.99202303181@snelboot.oratrix.nl> Message-ID: [Martin von Lupsidedownquestionmarkwis] > This is the problem. Python source code is not in Latin-1; bytes; > inside strings and comments are "as-is". The latter is a platform-dependent accident -- the ref man says Python source is 7-bit ASCII, and that was the intent. That C's ctype functions later grew locale-dependent behavior accounts for why Python did too, but even according to the C standard you can't count on transporting C text-mode files across platforms unless they avoid that stuff. > So the CVS "binary" mode would come closer as to how python files > should be treated, although you'd still would want to convert line- > endings. [Jack] > Is this an "official" policy? My policy is that Python source in the *distribution* adheres to the restrictions of the C std for C text-mode files, until such time as Python changes to read and write them in C binary mode, or stops using C stdio altogether. And that's as official as I am . > As such I disagree with it. Either all Python source should be 7-bit > clean or the source should be converted to the local convention, even > for strings and comments. But if it is an official policy I'd > like to hear it (who wants to channel Guido here?), because > there's heaps of MacPython stuff that'll have to be converted back > (I did the forward conversion a couple of months ago). Sorry, I don't know what "forward" and "back" mean to you. From sdm7g@Virginia.EDU Fri Aug 3 22:21:29 2001 From: sdm7g@Virginia.EDU (Steven D. Majewski) Date: Fri, 3 Aug 2001 17:21:29 -0400 (EDT) Subject: [Python-Dev] RFC: generator combinator syntax In-Reply-To: Message-ID: FYI: In the thread on comp.lang.python with the subject: "Nested generators is the python equivalent of unix pipe cmds." we're hashing out some ideas for functions for "the standard generator library" and syntax for operators to combine iterators. This thread started after thinking about my comments on Eric's request-for-comments on a file processing framework. I'm not actually in favor of following unix pipeline syntax so closely as to use the __or__ operator -- I think it's confusing python syntax -- but by having 'iterator | function' return a modified iterator, you can build iterator pipeline chain commands. For example: Files('.') | isGif Files( '.' ) is a generator that recursively enumerates all the files starting from the current (unix) working directory. isGif = lambda s : s.lower().find( '.gif' ) >= 0 and the operator returns the iterator that returns all of the ".gif" files below the current working directory. I'm actually proposing "&" to be the operator to do the above. I think "|", despite the unix convention, would be better for alternation ( i.e. the generator that yields one from column A, one from column (generator) B, etc. ) -- Steve From paulp@ActiveState.com Sat Aug 4 00:07:51 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Fri, 03 Aug 2001 16:07:51 -0700 Subject: [Python-Dev] RFC: generator combinator syntax References: Message-ID: <3B6B2EC7.93F61652@ActiveState.com> "Steven D. Majewski" wrote: > > > Files('.') | isGif > > Files( '.' ) is a generator that recursively enumerates all the files > starting from the current (unix) working directory. > > isGif = lambda s : s.lower().find( '.gif' ) >= 0 I propose: [yield file for file in Files('.') if file.lower.find(.'gif')>=0] By reusing the yield keyword we can get rid of the need for the new "|" or "&" syntax. Any kind of combination that can be done with list comprehensions would be possible with generator comprehensions. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From sdm7g@Virginia.EDU Sat Aug 4 00:35:04 2001 From: sdm7g@Virginia.EDU (Steven D. Majewski) Date: Fri, 3 Aug 2001 19:35:04 -0400 (EDT) Subject: [Python-Dev] RFC: generator combinator syntax In-Reply-To: <3B6B2EC7.93F61652@ActiveState.com> Message-ID: On Fri, 3 Aug 2001, Paul Prescod wrote: > I propose: > > [yield file for file in Files('.') > if file.lower.find(.'gif')>=0] > > By reusing the yield keyword we can get rid of the need for the new "|" > or "&" syntax. > > Any kind of combination that can be done with list comprehensions would > be possible with generator comprehensions. I assume you intend that "[yield file for..." expression to evaluate to a generator, not a list. I like it. I'm not sure that I want to get rid of the operator syntax. I'm still getting used to it. Either works a lot better than using filter: I was complaining in an earlier thread about having to explicitly ask for the .next attrib. It just took a while to figure out how they are meant to be used. Generators & Iterators really seem to open up functional programming in Python. I'm trying to put together some more basic file processing functionals: for example, fileExt(ext) instead of isGif: def fileExt(ext): return lambda s: os.path.splitext( s )[-1].lower() == ext.lower() Then isGif becomes: isGif = fileExt( '.gif' ) and others like fileSize, owner, etc. def fileSize( test ): return lambda s: test( os.stat(s)[stat.ST_SIZE] ) and folding the Genops class into the File function, I can do things like: for file in Files() & isGif & fileSize( lambda s : s > 1024 ): process( file ) -- Steve From simon@netthink.co.uk Sat Aug 4 01:54:04 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Fri, 3 Aug 2001 17:54:04 -0700 Subject: [Python-Dev] Python API version & optional features In-Reply-To: <3B6A6B54.3BA008C9@lemburg.com> Message-ID: <20010803175404.C13102@netthink.co.uk> On Fri, Aug 03, 2001 at 11:13:56AM +0200, M.-A. Lemburg wrote: > > Given the smiley at the end of your message, I don't know whether you > > meant your point seriously or not. > > The only thing which we require is that a Unicode aware > extension use at least one of the mangled APIs -- shouldn't > be much of a requirement, though :-) My point was that given a Py_UCS2_Foo it should be easy to automatically produce a Py_UCS4_Foo (or vice versa) so that people who are using the "wrong" mangled API can still get what they expect. But I'm probably misunderstanding something. Simon From simon@netthink.co.uk Sat Aug 4 01:56:02 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Fri, 3 Aug 2001 17:56:02 -0700 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: <200108031233.IAA01557@cj20424-a.reston1.va.home.com> Message-ID: <20010803175602.D13102@netthink.co.uk> On Fri, Aug 03, 2001 at 08:33:40AM -0400, Guido van Rossum wrote: > But Ping's tracebackhook still shouldn't be the default. Yep, understood. > I'm willing to entertain a new command line option for > it though, so that CGI scripts can use > #!/usr/bin/env python -C > as their first line. How about making it the default iff certain CGI-oid enviroment variables are set? Or would that be way too magical? Simon From barry@zope.com Sat Aug 4 01:24:41 2001 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 3 Aug 2001 20:24:41 -0400 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 References: <200108031233.IAA01557@cj20424-a.reston1.va.home.com> <20010803175602.D13102@netthink.co.uk> Message-ID: <15211.16585.852075.358121@anthem.wooz.org> >>>>> "SC" == Simon Cozens writes: >> I'm willing to entertain a new command line option for it >> though, so that CGI scripts can use #!/usr/bin/env python -C as >> their first line. SC> How about making it the default iff certain CGI-oid enviroment SC> variables are set? Or would that be way too magical? I'm concerned about magically enabling it, because then my existing cgi scripts have to be modified to turn it off or they will/might break. And I'll have to conditionalize the code to turn them off in case my users have an older version of Python. explicit-is-better-than-implicit-ly y'rs, -Barry From aahz@rahul.net Sat Aug 4 02:38:46 2001 From: aahz@rahul.net (Aahz Maruch) Date: Fri, 3 Aug 2001 18:38:46 -0700 (PDT) Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: <15211.16585.852075.358121@anthem.wooz.org> from "Barry A. Warsaw" at Aug 03, 2001 08:24:41 PM Message-ID: <20010804013846.8B64B99C97@waltz.rahul.net> Barry A. Warsaw wrote: > > SC> How about making it the default iff certain CGI-oid enviroment > SC> variables are set? Or would that be way too magical? > > I'm concerned about magically enabling it, because then my existing > cgi scripts have to be modified to turn it off or they will/might > break. And I'll have to conditionalize the code to turn them off in > case my users have an older version of Python. > > explicit-is-better-than-implicit-ly y'rs, > -Barry OTOH, looking for a CGITB variable might work. On the gripping hand, anyone who wants to work it that way can do so with two lines of code. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From fdrake@acm.org Sat Aug 4 03:03:38 2001 From: fdrake@acm.org (Fred L. Drake) Date: Fri, 3 Aug 2001 22:03:38 -0400 (EDT) Subject: [Python-Dev] [development doc updates] Message-ID: <20010804020338.338B82879B@cj42289-a.reston1.va.home.com> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ A variety of updates, including the first installment for the distutils API docs (in "Distributing Python Modules"), and some contributed material on embedding Python (in "Extending and Embedding the Python Interpreter"). There were many other small improvements and bug fixes. From gstein@lyra.org Sat Aug 4 04:22:27 2001 From: gstein@lyra.org (Greg Stein) Date: Fri, 3 Aug 2001 20:22:27 -0700 Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?] Message-ID: <20010803202227.A31091@lyra.org> [ misaddressed... ] ----- Forwarded message from Guido van Rossum ----- From: Guido van Rossum Subject: Re: [Python-Dev] Re: WebDAV in python 2.2? To: Greg Stein Date: Fri, 03 Aug 2001 15:18:37 -0400 > My intent was to write a short PEP because there would be a couple modules > to add to the lib: > > 1) httpauth -- subclasses httplib.HTTPConnection to add authenticated > connections. Initially using the Basic auth method, but it could also be > expanded to Digest (and others) > > 2) davlib -- my existing module. needs a couple items of work: allow > alternate/fallback XML parsers (it uses Expat + qp_xml right now), and > use the authenticated HTTP connections. > > That is at least a start on what needs to happen. We can flesh it out with a > PEP or by checking in the modules. > (Moshe and I talked about httpauth a bit at the last Python Conference; I > know he is interested in working on something like that) Good! > I'm not sure how we normally write tests for network-based modules, but it > shouldn't be too hard to have them run against test.webdav.org. Can someone help Greg with his question about testing network modules? Generally the test suite doesn't assume internet access; network code is "tested" using a second thread and localhost. --Guido van Rossum (home page: http://www.python.org/~guido/) ----- End forwarded message ----- -- Greg Stein, http://www.lyra.org/ From tim.one@home.com Sat Aug 4 06:33:03 2001 From: tim.one@home.com (Tim Peters) Date: Sat, 4 Aug 2001 01:33:03 -0400 Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler Message-ID: Some Socket Wizard should worry about the attached. What's life like on 64-bit Linux (etc)? Perhaps more autoconf crap could fudge this. once-again-windows-shows-the-way-ly y'rs - tim Background info from an earlier Python-Help msg: > I work on a Solaris 8.0 (64 bit) system. Python is compiled with > Sun CC (Forte 6.1) compiler with threading support. A small program > telnetting to a computer often dumps core. -----Original Message----- From: python-help-admin@python.org Sent: Friday, August 03, 2001 5:06 AM To: help@python.org Subject: Re: [Python-Help] Python threads with suncc (Forte 6.1) compiler ... I've received some answers that this code running fine in the same environment. A collegaue of mine compiled Python 2.0 as I wrote yesterday, and later he debugged this error. Purify showed us stack read and write outside the stack in Modules/selectmodule.c in function: Modules/selectmodule.c . . static PyObject * select_select(PyObject *self, PyObject *args) { #ifdef MS_WINDOWS /* This would be an awful lot of stack space on Windows! */ pylist *rfd2obj, *wfd2obj, *efd2obj; #else pylist rfd2obj[FD_SETSIZE + 3]; pylist wfd2obj[FD_SETSIZE + 3]; pylist efd2obj[FD_SETSIZE + 3]; #endif . . . } In our environment FD_SETSIZE is 65536 as defined in sys/select.h (see below). The allocated stack space in select_select is 3*sizeof(rfd2obj)*(FD_SETSIZE+3). It is more than 3MB. My collegaue told me that the difference between the addresses of the same variable in two seperate threads is about 2MB. Lets suppose char *p1 = (char *)rfd2obj in thread N and char *p2 = (char *)rfd2obj in thread N + 1, abs(p1-p2) is about 2MB. The stack is overwritten between the seperate threads. He could solve the problem, as solved on Windows with allocating these variables on the heap. Do you have any ideas, how can it happen? /usr/include/sys/select.h: . . #ifndef FD_SETSIZE #ifdef _LP64 #define FD_SETSIZE 65536 #else #define FD_SETSIZE 1024 #endif /* _LP64 */ . . ... _______________________________________________ Python-Help maillist - Python-Help@python.org http://mail.python.org/mailman/listinfo/python-help From guido@zope.com Sat Aug 4 07:29:08 2001 From: guido@zope.com (Guido van Rossum) Date: Sat, 04 Aug 2001 02:29:08 -0400 Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler In-Reply-To: Your message of "Sat, 04 Aug 2001 01:33:03 EDT." References: Message-ID: <200108040629.CAA10668@cj20424-a.reston1.va.home.com> > Some Socket Wizard should worry about the attached. What's life like on > 64-bit Linux (etc)? Dunno, but I doubt 64-bit Linux would be so dumb as to declare FD_SETSIZE to be 2**16. > Perhaps more autoconf crap could fudge this. I think we can avoid this. > once-again-windows-shows-the-way-ly y'rs - tim Yes, in a very simple way: instead of testing for MS_WINDOWS, we could test for a preposterous value of FD_SETSIZE. For example, change all three occurrences of #ifdef MS_WINDOWS into #if FD_SETSIZE > 1024 Looking at the code, a better solution may be to always allocate this thing on the heap, and to let list2set allocate it. The actual number of items *used* is the length of the list of file descriptors + 1 (a sentinel). Even if FD_SETSIZE is not preposterous, the array size allocated is the theoretical maximum, which is always ways larger than the size needed. (I also fail to understand why 3 extra elements are allocated; the algorithm seems to need only one extra.) Yet another approach would (shrudder :) be to forget about arrays and use a Python dict, since that's the usage: the data structure stores the mapping between file descriptors (unique small ints) and objects in the argument lists (either Python ints or objects with a fileno() method). Volunteers? Not me! :) --Guido van Rossum (home page: http://www.python.org/~guido/) From ping@lfw.org Sat Aug 4 07:45:05 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Fri, 3 Aug 2001 23:45:05 -0700 (PDT) Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: <15211.16585.852075.358121@anthem.wooz.org> Message-ID: Hi all, I'm glad this is being discussed, and i understand that some of you are concerned about changing default behaviour, but, uh, it kind of irks me that the major concerns aren't that well based in truth. Thomas Wouters wrote: > You might not want people to edit scripts, but I'm not sure howmany > people want the functionality of their CGI script(s) significantly > changed without editing their scripts. Skip Montanaro wrote: > I have to agree. Perhaps just as significant, what would happen to a CGI > script that already attempts to do something to direct error output to a > file or mail message the webmaster will see? Nothing! It would still work. No functionality would change at all! That's the whole point. This feature only kicks in at the point where a program has ALREADY FAILED. By definition, there is nothing the program could have done at that point. It's about to die. Barry Warsaw wrote: > I'm concerned about magically enabling it, because then my existing > cgi scripts have to be modified to turn it off or they will/might > break. They *couldn't* break. I mean, they could only "break" in the case where they're already broken. There seems to be this general nervousness about making a change, but -- well, i don't really see what you all are going on about. It seems to me that if you think about it for a bit, you'll see why the change couldn't break anything. If anything is there (e.g. some sort of CGI-imitation environment) running the script and catching exceptions, then there's no issue. sys.excepthook is only a last resort. Setting os.environ['GATEWAY_INTERFACE'] after the program starts doesn't magically install the feature -- it's only enabled if the GATEWAY_INTERFACE variable is present when Python starts, which can only be the case if the caller of the Python binary wants the script to behave like a CGI script, and cgitb meets that contract. Skip Montanaro wrote: > Most of my users wouldn't know a traceback (simpler or otherwise) from a > hole in the ground. I think I have perhaps two users who would understand > that I was trying to create a more readable report. It is certainly a legitimate concern that you may want to keep your tracebacks secret. However, this seems to be just as good an argument that ordinary Python programs shouldn't print tracebacks on stderr. Oh, no -- what if a user sees them? Well, the user saw it because something went wrong. Of course you don't want a typical user to worry about tracebacks. If your program was correct, that would never happen. When you do get one, something truly exceptional has occurred. And yet we do display tracebacks on stderr nevertheless, because You Never Know An Error Is About To Happen Before It Happens. And when it does happen, gosh! am i ever grateful i got that traceback. -- ?!ng From guido@zope.com Sat Aug 4 08:32:04 2001 From: guido@zope.com (Guido van Rossum) Date: Sat, 04 Aug 2001 03:32:04 -0400 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: Your message of "Fri, 03 Aug 2001 23:45:05 PDT." References: Message-ID: <200108040732.DAA10879@cj20424-a.reston1.va.home.com> Give it up, Ping. cgitb writes the tb to stdout, not to stderr, and that's enough of a difference in behavior to require explicit enabling. It's not worth your energy to try and convince us. Let's have the cgitb module as an optional feature. Let's update various documentation to suggest this boilerplate at the top of CGI modules: import cgi import cgitb; cgitb.enable() Regarding the secrecy of tracebacks: normal tracebacks are seen by a user who has logged in to the system and who can look at the source code anyway. Web tracebacks can be seen as invitations to hackerz without any source access to look for weaknesses in the script. That's a very different thing from a security point of view! (Security by obscurity, maybe, but nevertheless better than hanging out your dirty laundry to dry in public. Enough hackerz know Python.) --Guido van Rossum (home page: http://www.python.org/~guido/) From m@moshez.org Sat Aug 4 08:55:44 2001 From: m@moshez.org (Moshe Zadka) Date: Sat, 04 Aug 2001 10:55:44 +0300 Subject: [Python-Dev] Re: WebDAV in python 2.2? In-Reply-To: <20010803121254.B29583@lyra.org> References: <20010803121254.B29583@lyra.org>, <200108031254.IAA01818@cj20424-a.reston1.va.home.com> Message-ID: On Fri, 3 Aug 2001 12:12:54 -0700, Greg Stein wrote: > That is at least a start on what needs to happen. We can flesh it out with a > PEP or by checking in the modules. > (Moshe and I talked about httpauth a bit at the last Python Conference; I > know he is interested in working on something like that) Let me just clarify what we talked about: today, the HTTPConnection already know how to respond to code 100. urllib2 has a very nifty mechanism of handling all kinds of errors. I have a feeling the responsiblity split is not correct, and I want to think about some kind of better responsibility split. I'll try and write something up soonish. -- Moshe Zadka - http://moshez.geek (if you're not cool, http://moshez.org is still working) From tim.one@home.com Sat Aug 4 10:01:07 2001 From: tim.one@home.com (Tim Peters) Date: Sat, 4 Aug 2001 05:01:07 -0400 Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler In-Reply-To: <200108040629.CAA10668@cj20424-a.reston1.va.home.com> Message-ID: [Guido] > Dunno, but I doubt 64-bit Linux would be so dumb as to declare > FD_SETSIZE to be 2**16. Quick search on google shows it's "the normal case" on that 64-bit Solaris, nothing special under Tru64, and probably allowed on some others. > ... > instead of testing for MS_WINDOWS, we could test for a preposterous > value of FD_SETSIZE. For example, change all three occurrences of > > #ifdef MS_WINDOWS > > into > > #if FD_SETSIZE > 1024 Sure. > Looking at the code, a better solution may be to always allocate this > thing on the heap, and to let list2set allocate it. The actual number > of items *used* is the length of the list of file descriptors + 1 (a > sentinel). Even if FD_SETSIZE is not preposterous, the array size > allocated is the theoretical maximum, which is always ways larger than > the size needed. I assume the current code is just trying to save 3 malloc/free pairs in the usual case. Since I'm not a select wizard, I don't know whether the speed hit would matter (OTOH, if there are few descriptors it's cheap regardless, and if there are many it's expensive regardless). > (I also fail to understand why 3 extra elements are allocated; the > algorithm seems to need only one extra.) I've stared uncomprehendingly at that too. Anyone? It discouraged me from becoming a select wizard a couple of years ago . > Yet another approach would (shrudder :) be to forget about arrays and > use a Python dict, since that's the usage: the data structure stores > the mapping between file descriptors (unique small ints) and objects > in the argument lists (either Python ints or objects with a fileno() > method). > > Volunteers? Not me! :) Ditto. Someone who stopped at the first suggestion could be finished already, though . BTW, do we actually need *any* auxiliary data structure here? Looks to me like all these giant pylist thingies could be tossed in favor of making shallow copies of the input ifdlist etc list arguments. The giant pylists don't *appear* to accomplish anything except cache the result of a PyObject_AsFileDescriptor() call per object. But if that's not an expensive call (doesn't look expensive to me), it's not worth all this trouble no matter how spelled. From gward@python.net Sat Aug 4 16:06:53 2001 From: gward@python.net (Greg Ward) Date: Sat, 4 Aug 2001 11:06:53 -0400 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: <200108031233.IAA01557@cj20424-a.reston1.va.home.com>; from guido@zope.com on Fri, Aug 03, 2001 at 08:33:40AM -0400 References: <20010802150109.B2033@netthink.co.uk> <200108031233.IAA01557@cj20424-a.reston1.va.home.com> Message-ID: <20010804110653.A910@gerg.ca> On 03 August 2001, Guido van Rossum said: > it though, so that CGI scripts can use > > #!/usr/bin/env python -C > > as their first line. Completely off-topic: the "/usr/bin/env" hack doesn't work with command-line arguments under Linux: $ cat t.py #!/usr/bin/env python -i print "hello" $ ./t.py /usr/bin/env: python -i: No such file or directory Grumble. Andrew and I tried to track this down once, and found suspicious-looking code in the kernel (ie., I don't think it's the fault of GNU env). Greg -- Greg Ward - Python bigot gward@python.net http://starship.python.net/~gward/ "What do you mean -- a European or an African swallow?" From pedroni@inf.ethz.ch Sat Aug 4 16:09:37 2001 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Sat, 4 Aug 2001 17:09:37 +0200 Subject: [Python-Dev] Zope porting to jython References: <20010802103006.H1667@netthink.co.uk> <20010803161704.M626@xs4all.nl> Message-ID: <004c01c11cf7$7b7eec60$b75821c0@newmexico> Hi, from some recent messages on the jython lists it seems that different people/groups are starting to work on recreating some of the necessary software components in order to port Zope or a create a similar compatible framework on top of Java/Jython. Here's a link to latest related thread: http://aspn.activestate.com/ASPN/Mail/Message/Jython-dev/712337 Jeremy has already chimed in, my personal position is to help them as all other jython users, fixing incompatibilities with CPython and answering to questions. Nothing special. I can imagine that one of the big problem could be restricted execution (if Zope uses that), admitted that any C code functionality can be rewritten in Java. If the PythonLabs have an "official" position/opinion on this and in particular on the last quite speculative message (from Joseph Grace) in the thread, they may post something. I can say something about technical aspects, but I don't feel that I should channel/or invent PythonLabs opinions. regards, Samuele Pedroni. From gward@python.net Sat Aug 4 16:18:47 2001 From: gward@python.net (Greg Ward) Date: Sat, 4 Aug 2001 11:18:47 -0400 Subject: [Python-Dev] Proposed new module for the Python library In-Reply-To: <20010802223632.A31724@thyrsus.com>; from esr@thyrsus.com on Thu, Aug 02, 2001 at 10:36:32PM -0400 References: <20010802223632.A31724@thyrsus.com> Message-ID: <20010804111847.B910@gerg.ca> On 02 August 2001, Eric S. Raymond said: > There is a common `compiler-like' pattern in Unix scripts which is useful > for translation utilities of all sorts. A program following this pattern > behaves as a filter when no argument files are specified on the command > line, but otherwise transforms each file individually into a corresponding > output file. Sounds like a useful tool to have around. I'm not sure the name 'ccframe' conjures up images of the above paragraph, but I'm not sure what name *does*. This is the hardest sort of software to name! > Design comments? Critiques? Code on request. > > I'm already considering throwing exceptions on open and rename errors > instead of complaining to stderr and returning an error status. That > would be more Pythonic, though slightly less convenient in the > commonest cases. How to handle failure to open an input file depends on circumstances: * sometimes, it is utter calamity, and raising an exception is the right thing to do * sometimes, it's a user-level error, and try: file = open(filename) except IOError, err: sys.exit("%s: %s" % (filename, err.strerror)) is the right thing to do * sometimes, it's just something the user ought to know about, and the "except" clause above can be turned into sys.stderr.write("warning: %s: %s" % (filename, err.strerror)) continue # next file, please * conceivably, it's something you can just ignore IOW, I think this should be an option for users of the framework. The default should be to throw an exception -- kaBOOM! For me, the middle two options would be most commonly used, but modules should not sys.exit() or sys.stderr.write() unless they are explicitly told to do so. Greg -- Greg Ward - nerd gward@python.net http://starship.python.net/~gward/ My opinions may have changed, but not the fact that I am right. From fdrake@acm.org Sat Aug 4 17:03:06 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Sat, 4 Aug 2001 12:03:06 -0400 (EDT) Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: <20010804110653.A910@gerg.ca> References: <20010802150109.B2033@netthink.co.uk> <200108031233.IAA01557@cj20424-a.reston1.va.home.com> <20010804110653.A910@gerg.ca> Message-ID: <15212.7354.720723.721035@cj42289-a.reston1.va.home.com> Greg Ward writes: > Grumble. Andrew and I tried to track this down once, and found > suspicious-looking code in the kernel (ie., I don't think it's the fault > of GNU env). I had always understood that the sh-bang line could only provide a single argument to begin with. I don't know what you found in the kernel sources, but I'd expect it to be implementing that policy. So /usr/bin/env gets one parameter from the sh-bang line, "python", and then another parameter added by the kernel: the name of the script. Sounds about right to me. Whether we like the constraint that there be only one arg from the sh-bang line doesn't really matter; lots of systems enforce this constraint, so to be portable we need to live with it. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From itojun@iijlab.net Sat Aug 4 14:21:57 2001 From: itojun@iijlab.net (Jun-ichiro itojun Hagino) Date: Sat, 04 Aug 2001 22:21:57 +0900 Subject: [Python-Dev] segmentation fault in tuple parsing Message-ID: <20010804132157.6F3F67BC@starfruit.itojun.org> it looks that there's some issue with tuple passing (PyArg_ParseTuple). the last argument of vgetargs1(), which is 0, is dereferenced in convertsimple(). sorry if this is due to my mistake in socketmodule IPv6 support. itojun (gdb) run Starting program: /usr/home/itojun/work/python/python/dist/src/python Python 2.2a1 (#12, Aug 4 2001, 10:17:23) [GCC egcs-2.91.66 19990314 (egcs-1.1.2 release)] on netbsd1 Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> socket.getnameinfo(('::1', 80, 0, 0), 0) Program received signal SIGSEGV, Segmentation fault. convertsimple (arg=0x80e43ac, p_format=0xbfbfcda4, p_va=0xbfbfcf80, msgbuf=0xbfbfce68 "\af\027H ŽÇ\021\b\b+\030H") at Python/getargs.c:501 501 *p = ival; (gdb) bt #0 convertsimple (arg=0x80e43ac, p_format=0xbfbfcda4, p_va=0xbfbfcf80, msgbuf=0xbfbfce68 "\af\027H ŽÇ\021\b\b+\030H") at Python/getargs.c:501 #1 0x806ab6c in convertitem (arg=0x80e43ac, p_format=0xbfbfcf74, p_va=0xbfbfcf80, levels=0xbfbfcde8, msgbuf=0xbfbfce68 "\af\027H ŽÇ\021\b\b+\030H") at Python/getargs.c:351 #2 0x8068a72 in vgetargs1 (args=0x813d18c, format=0x481980b0 "i", p_va=0xbfbfcf80, compat=0) at Python/getargs.c:197 #3 0x80686fd in PyArg_ParseTuple (args=0x813d18c, format=0x481980ac "si|ii") at Python/getargs.c:52 #4 0x48194319 in PySocket_getnameinfo (self=0x0, args=0x811c3cc) at /usr/home/itojun/work/python/python/dist/src/Modules/socketmodule.c:2430 #5 0x8059f35 in call_cfunction (func=0x81871cc, arg=0x811c3cc, kw=0x0) at Python/ceval.c:3072 #6 0x805744d in eval_frame (f=0x80e7c0c) at Python/ceval.c:1912 #7 0x8058461 in PyEval_EvalCodeEx (co=0x80fbdc0, globals=0x80f2d0c, locals=0x80f2d0c, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2517 #8 0x8059ea3 in PyEval_EvalCode (co=0x80fbdc0, globals=0x80f2d0c, locals=0x80f2d0c) at Python/ceval.c:490 #9 0x8073f48 in run_node (n=0x80f5c00, filename=0x80b4975 "", globals=0x80f2d0c, locals=0x80f2d0c, flags=0xbfbfd65c) at Python/pythonrun.c:1058 #10 0x8072a0e in PyRun_InteractiveOneFlags (fp=0x80dafd0, filename=0x80b4975 "", flags=0xbfbfd65c) at Python/pythonrun.c:578 #11 0x8073997 in PyRun_InteractiveLoopFlags (fp=0x80dafd0, filename=0x80b4975 "", flags=0xbfbfd65c) at Python/pythonrun.c:514 #12 0x8073802 in PyRun_AnyFileExFlags (fp=0x80dafd0, filename=0x80b4975 "", closeit=0, flags=0xbfbfd65c) at Python/pythonrun.c:477 #13 0x8051da3 in Py_Main (argc=1, argv=0xbfbfd6b8) at Modules/main.c:332 #14 0x8051742 in main (argc=1, argv=0xbfbfd6b8) at Modules/python.c:10 #15 0x8051579 in ___start () (gdb) frame 1 #1 0x806ab6c in convertitem (arg=0x80e43ac, p_format=0xbfbfcf74, p_va=0xbfbfcf80, levels=0xbfbfcde8, msgbuf=0xbfbfce68 "\af\027H ŽÇ\021\b\b+\030H") at Python/getargs.c:351 351 msg = convertsimple(arg, &format, p_va, msgbuf); (gdb) print p_va $1 = (va_list *) 0xbfbfcf80 (gdb) frame 2 #2 0x8068a72 in vgetargs1 (args=0x813d18c, format=0x481980b0 "i", p_va=0xbfbfcf80, compat=0) at Python/getargs.c:197 197 msg = convertitem(PyTuple_GET_ITEM(args, i), &format, p_va, (gdb) list 192 } 193 194 for (i = 0; i < len; i++) { 195 if (*format == '|') 196 format++; 197 msg = convertitem(PyTuple_GET_ITEM(args, i), &format, p_va, 198 levels, msgbuf); 199 if (msg) { 200 seterror(i+1, msg, levels, fname, message); 201 return 0; (gdb) print format $2 = 0x481980b0 "i" (gdb) frame 3 #3 0x80686fd in PyArg_ParseTuple (args=0x813d18c, format=0x481980ac "si|ii") at Python/getargs.c:52 52 retval = vgetargs1(args, format, &va, 0); (gdb) list 47 { 48 int retval; 49 va_list va; 50 51 va_start(va, format); 52 retval = vgetargs1(args, format, &va, 0); 53 va_end(va); 54 return retval; 55 } 56 From itojun@iijlab.net Sat Aug 4 17:20:05 2001 From: itojun@iijlab.net (Jun-ichiro itojun Hagino) Date: Sun, 05 Aug 2001 01:20:05 +0900 Subject: [Python-Dev] IPv6 docs Message-ID: <20010804162005.E1D3B7BB@starfruit.itojun.org> Here's a diff for adding documentation for IPv6 funcs in socket module. I have no TeX on my laptop these days, so not sure if it compiles clean. also if someone could go through my broken english, it would be highly appreciated. itojun Index: Doc/lib/libftplib.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libftplib.tex,v retrieving revision 1.32 diff -u -r1.32 libftplib.tex --- Doc/lib/libftplib.tex 2001/07/06 19:28:48 1.32 +++ Doc/lib/libftplib.tex 2001/08/04 16:18:22 @@ -191,9 +191,9 @@ \begin{methoddesc}{transfercmd}{cmd\optional{, rest}} Initiate a transfer over the data connection. If the transfer is -active, send a \samp{PORT} command and the transfer command specified +active, send a \samp{EPRT} or \samp{PORT} command and the transfer command specified by \var{cmd}, and accept the connection. If the server is passive, -send a \samp{PASV} command, connect to it, and start the transfer +send a \samp{EPSV} or \samp{PASV} command, connect to it, and start the transfer command. Either way, return the socket for the connection. If optional \var{rest} is given, a \samp{REST} command is Index: Doc/lib/libsocket.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libsocket.tex,v retrieving revision 1.51 diff -u -r1.51 libsocket.tex --- Doc/lib/libsocket.tex 2001/01/10 19:34:52 1.51 +++ Doc/lib/libsocket.tex 2001/08/04 16:18:24 @@ -19,6 +19,8 @@ of information on the details of socket semantics. For \UNIX, refer to the manual pages; for Windows, see the WinSock (or Winsock 2) specification. +For IPv6-ready APIs, readers may want to refer to RFC2553 titled +\cite{Basic Socket Interface Extensions for IPv6}. The Python interface is a straightforward transliteration of the \UNIX{} system call and library interface for sockets to Python's @@ -30,20 +32,43 @@ receive operations is automatic, and buffer length is implicit on send operations. -Socket addresses are represented as a single string for the -\constant{AF_UNIX} address family and as a pair -\code{(\var{host}, \var{port})} for the \constant{AF_INET} address +Socket addresses are represented as follows. +A single string is used for the +\constant{AF_UNIX} address family. +A pair +\code{(\var{host}, \var{port})} is used for the \constant{AF_INET} address family, where \var{host} is a string representing either a hostname in Internet domain notation like -\code{'daring.cwi.nl'} or an IP address like \code{'100.50.200.5'}, -and \var{port} is an integral port number. Other address families are +\code{'daring.cwi.nl'} or an IPv4 address like \code{'100.50.200.5'}, +and \var{port} is an integral port number. +For \constant{AF_INET6} address family, a four-tuple \code{(\var{host}, +\var{port}, \var{flowinfo}, \var{scopeid})} is used, where +\var{flowinfo} and \var{scopeid} represents \code{sin6_flowinfo} +and \code{sin6_scope_id} member in \constant{struct sockaddr_in6} +in C. +For \module{socket} module methods, +\var{flowinfo} and \var{scopeid} can be omitted just for backward compatibility. +Note, however, omission of \var{scopeid} can cause problems in manipulating +scoped IPv6 addresses. +Other address families are currently not supported. The address format required by a particular socket object is automatically selected based on the address family specified when the socket object was created. -For IP addresses, two special forms are accepted instead of a host +For IPv4 addresses, two special forms are accepted instead of a host address: the empty string represents \constant{INADDR_ANY}, and the string \code{''} represents \constant{INADDR_BROADCAST}. +The behavior is not available for IPv6 for backward compatibility, +therefore, you may want to avoid these if you intend to support IPv6 with +your Python programs. + +If you use a hostname in \var{host} portion of IPv4/v6 socket address, +the program may present a nondeterministic behavior, +as Python uses the first address returned from the DNS resolution. +The socket address will be resolved differently into an actual +IPv4/v6 address, depending on the results from DNS resolution and/or +the host configuration. +For deterministic behavior use a numeric address in \var{host} portion. All errors raise exceptions. The normal exceptions for invalid argument types and out-of-memory conditions can be raised; errors @@ -57,7 +82,7 @@ \begin{excdesc}{error} -This exception is raised for socket- or address-related errors. +This exception is raised for socket-related errors. The accompanying value is either a string telling what went wrong or a pair \code{(\var{errno}, \var{string})} representing an error returned by a system @@ -66,8 +91,30 @@ names for the error codes defined by the underlying operating system. \end{excdesc} +\begin{excdesc}{herror} +This exception is raised for address-related errors, for functions that uses +\var{h_errno} in C API. +It includes \function{gethostbyname} and \function{gethostbyaddr}. +The accompanying value is either a string telling what went wrong or a +pair \code{(\var{h_errno}, \var{string})} +representing an error returned by a library call. +\var{string} represents the description of \var{h_errno}, as returned +by \cfunction{hstrerror} C API. +\end{excdesc} + +\begin{excdesc}{gaierror} +This exception is raised for address-related errors, for \function{getaddrinfo} +and \function{getnameinfo}. +The accompanying value is either a string telling what went wrong or a +pair \code{(\var{error}, \var{string})} +representing an error returned by a library call. +\var{string} represents the description of \var{error}, as returned +by \cfunction{gai_strerror} C API. +\end{excdesc} + \begin{datadesc}{AF_UNIX} \dataline{AF_INET} +\dataline{AF_INET6} These constants represent the address (and protocol) families, used for the first argument to \function{socket()}. If the \constant{AF_UNIX} constant is not defined then this protocol is @@ -93,6 +140,10 @@ \dataline{IPPORT_*} \dataline{INADDR_*} \dataline{IP_*} +\dataline{IPV6_*} +\dataline{EAI_*} +\dataline{AI_*} +\dataline{NI_*} Many constants of these forms, documented in the \UNIX{} documentation on sockets and/or the IP protocol, are also defined in the socket module. They are generally used in arguments to the \method{setsockopt()} and @@ -101,6 +152,30 @@ for a few symbols, default values are provided. \end{datadesc} +\begin{funcdesc}{getaddrinfo}{host, port\optional{, family, socktype, proto, flags}} +Resolves \var{host}/\var{port} argument, +into a sequence of 5-tuples that contain all the necessary argument for +the sockets manipulation. +\var{host} is a domain name, +an string representation of IPv4/v6 address or \code{None}. +\var{port} is a string service name (like \code{http}), +a numeric port number or \code{None}. +The rest of the arguments are optional and must be numeric if specified. +For \var{host} and \var{port}, by passing either an empty string or \code{None}, +you can pass \code{NULL} to the C API. +The \function{getaddrinfo()} function returns a list of the following 5-tuples: +\code{(\var{family}, \var{socktype}, \var{proto}, \var{canonname}, \var{sockaddr})}. +\var{family}, \var{socktype}, \var{proto} are all integer and are meant to +be passed to the \function{socket()} function. +\var{canonname} is a string representing the canonical name of the \var{host}. +It can be a numeric IPv4/v6 address when \code{AI_CANONNAME} is specified +against numeric \var{host}. +\var{sockaddr} is a tuple describing a socket address, as described above. +See \code{Lib/httplib.py} and other library files +for a typical usage of the function. +\versionadded{X.0} +\end{funcdesc} + \begin{funcdesc}{getfqdn}{\optional{name}} Return a fully qualified domain name for \var{name}. If \var{name} is omitted or empty, it is interpreted as the local @@ -113,26 +188,33 @@ \end{funcdesc} \begin{funcdesc}{gethostbyname}{hostname} -Translate a host name to IP address format. The IP address is +Translate a host name to IPv4 address format. The IPv4 address is returned as a string, e.g., \code{'100.50.200.5'}. If the host name -is an IP address itself it is returned unchanged. See +is an IPv4 address itself it is returned unchanged. See \function{gethostbyname_ex()} for a more complete interface. +\function{gethostbyname()} does not support IPv6 name resolution, and +\function{getaddrinfo()} should be used instead for IPv4/v6 dual stack support. \end{funcdesc} \begin{funcdesc}{gethostbyname_ex}{hostname} -Translate a host name to IP address format, extended interface. +Translate a host name to IPv4 address format, extended interface. Return a triple \code{(hostname, aliaslist, ipaddrlist)} where \code{hostname} is the primary host name responding to the given \var{ip_address}, \code{aliaslist} is a (possibly empty) list of alternative host names for the same address, and \code{ipaddrlist} is -a list of IP addresses for the same interface on the same +a list of IPv4 addresses for the same interface on the same host (often but not always a single address). +\function{gethostbyname_ex()} does not support IPv6 name resolution, and +\function{getaddrinfo()} should be used instead for IPv4/v6 dual stack support. \end{funcdesc} \begin{funcdesc}{gethostname}{} Return a string containing the hostname of the machine where -the Python interpreter is currently executing. If you want to know the -current machine's IP address, use \code{gethostbyname(gethostname())}. +the Python interpreter is currently executing. +If you want to know the current machine's IP address, you may want to use +\code{gethostbyname(gethostname())}. +This operation assumes that there is a valid address-to-host mapping for +the host, and the assumption does not always hold. Note: \function{gethostname()} doesn't always return the fully qualified domain name; use \code{gethostbyaddr(gethostname())} (see below). @@ -143,10 +225,19 @@ \var{ipaddrlist})} where \var{hostname} is the primary host name responding to the given \var{ip_address}, \var{aliaslist} is a (possibly empty) list of alternative host names for the same address, -and \var{ipaddrlist} is a list of IP addresses for the same interface +and \var{ipaddrlist} is a list of IPv4/v6 addresses for the same interface on the same host (most likely containing only a single address). To find the fully qualified domain name, use the function \function{getfqdn()}. +\function{gethostbyaddr} supports both IPv4 and IPv6. +\end{funcdesc} + +\begin{funcdesc}{getnameinfo}{sockaddr, flags} +Translate a socket address \var{sockaddr} into a 2-tuple +\code{(\var{host}, \var{port})}. +Depending on the settings of \var{flags}, the result can contain +a fully-qualified domain name or numeric address representation in \var{host}. +Similarly, \var{port} can contain a string port name or a numeric port number. \end{funcdesc} \begin{funcdesc}{getprotobyname}{protocolname} @@ -166,7 +257,7 @@ \begin{funcdesc}{socket}{family, type\optional{, proto}} Create a new socket using the given address family, socket type and -protocol number. The address family should be \constant{AF_INET} or +protocol number. The address family should be \constant{AF_INET}, \constant{AF_INET6} or \constant{AF_UNIX}. The socket type should be \constant{SOCK_STREAM}, \constant{SOCK_DGRAM} or perhaps one of the other \samp{SOCK_} constants. The protocol number is usually zero and may be omitted in that case. @@ -209,7 +300,7 @@ \end{funcdesc} \begin{funcdesc}{inet_aton}{ip_string} -Convert an IP address from dotted-quad string format +Convert an IPv4 address from dotted-quad string format (e.g.\ '123.45.67.89') to 32-bit packed binary format, as a string four characters in length. @@ -217,14 +308,17 @@ and needs objects of type \ctype{struct in_addr}, which is the C type for the 32-bit packed binary this function returns. -If the IP address string passed to this function is invalid, +If the IPv4 address string passed to this function is invalid, \exception{socket.error} will be raised. Note that exactly what is valid depends on the underlying C implementation of \cfunction{inet_aton()}. + +\function{inet_aton} does not support IPv6, and +\function{getnameinfo()} should be used instead for IPv4/v6 dual stack support. \end{funcdesc} \begin{funcdesc}{inet_ntoa}{packed_ip} -Convert a 32-bit packed IP address (a string four characters in +Convert a 32-bit packed IPv4 address (a string four characters in length) to its standard dotted-quad string representation (e.g. '123.45.67.89'). @@ -234,6 +328,9 @@ If the string passed to this function is not exactly 4 bytes in length, \exception{socket.error} will be raised. + +\function{inet_ntoa} does not support IPv6, and +\function{getnameinfo()} should be used instead for IPv4/v6 dual stack support. \end{funcdesc} \begin{datadesc}{SocketType} @@ -306,14 +403,14 @@ \begin{methoddesc}[socket]{getpeername}{} Return the remote address to which the socket is connected. This is -useful to find out the port number of a remote IP socket, for instance. +useful to find out the port number of a remote IPv4/v6 socket, for instance. (The format of the address returned depends on the address family --- see above.) On some systems this function is not supported. \end{methoddesc} \begin{methoddesc}[socket]{getsockname}{} Return the socket's own address. This is useful to find out the port -number of an IP socket, for instance. +number of an IPv4/v6 socket, for instance. (The format of the address returned depends on the address family --- see above.) \end{methoddesc} @@ -413,7 +510,7 @@ \subsection{Example \label{socket-example}} -Here are two minimal example programs using the TCP/IP protocol:\ a +Here are four minimal example programs using the TCP/IP protocol:\ a server that echoes all data that it receives back (servicing only one client), and a client using it. Note that a server must perform the sequence \function{socket()}, \method{bind()}, \method{listen()}, @@ -424,6 +521,8 @@ socket it is listening on but on the new socket returned by \method{accept()}. +The first two examples support IPv4 only. + \begin{verbatim} # Echo server program import socket @@ -450,6 +549,82 @@ PORT = 50007 # The same port as used by the server s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((HOST, PORT)) +s.send('Hello, world') +data = s.recv(1024) +s.close() +print 'Received', `data` +\end{verbatim} + +The next two examples are identical to the above two, but support both +IPv4 and IPv6. +The server side will listen to the first address family available +(it should listen to both instead). +On most of IPv6-ready systems, IPv6 will take precedence +and the server may not accept IPv4 traffic. +The client side will try to connect to the all addresses returned as a result +of the name resolution, and sends traffic to the first one connected +successfully. + +\begin{verbatim} +# Echo server program +import socket +import sys + +HOST = '' # Symbolic name meaning the local host +PORT = 50007 # Arbitrary non-privileged port +s = None +for res in socket.getaddrinfo('', PORT, socket.AF_UNSPEC, socket.SOCK_STREAM, 0, socket.AI_PASSIVE): + af, socktype, proto, canonname, sa = res + try: + s = socket.socket(af, socktype, proto) + except socket.error, msg: + s = None + continue + try: + s.bind(sa) + s.listen(1) + except socket.error, msg: + s.close() + s = None + continue + break +if s is None: + print 'could not open socket' + sys.exit(1) +conn, addr = s.accept() +print 'Connected by', addr +while 1: + data = conn.recv(1024) + if not data: break + conn.send(data) +conn.close() +\end{verbatim} + +\begin{verbatim} +# Echo client program +import socket +import sys + +HOST = 'daring.cwi.nl' # The remote host +PORT = 50007 # The same port as used by the server +s = None +for res in socket.getaddrinfo(HOST, PORT, socket.AF_UNSPEC, socket.SOCK_STREAM): + af, socktype, proto, canonname, sa = res + try: + s = socket.socket(af, socktype, proto) + except socket.error, msg: + s = None + continue + try: + s.connect(sa) + except socket.error, msg: + s.close() + s = None + continue + break +if s is None: + print 'could not open socket' + sys.exit(1) s.send('Hello, world') data = s.recv(1024) s.close() From itojun@iijlab.net Sat Aug 4 17:20:38 2001 From: itojun@iijlab.net (Jun-ichiro itojun Hagino) Date: Sun, 05 Aug 2001 01:20:38 +0900 Subject: [Python-Dev] socketmodule fix for IPv6 (mostly additional symbols) Message-ID: <20010804162038.144F47BB@starfruit.itojun.org> Index: Modules/socketmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/socketmodule.c,v retrieving revision 1.160 diff -u -r1.160 socketmodule.c --- Modules/socketmodule.c 2001/08/03 10:02:29 1.160 +++ Modules/socketmodule.c 2001/08/04 16:18:28 @@ -2433,6 +2433,7 @@ PyOS_snprintf(pbuf, sizeof(pbuf), "%d", port); memset(&hints, 0, sizeof(hints)); hints.ai_family = PF_UNSPEC; + hints.ai_socktype = SOCK_DGRAM; /* make numeric port happy */ error = getaddrinfo(hostp, pbuf, &hints, &res); if (error) { PyGAI_Err(error); @@ -3135,6 +3136,9 @@ #else insint(d, "IPPROTO_IP", 0); #endif +#ifdef IPPROTO_HOPOPTS + insint(d, "IPPROTO_HOPOPTS", IPPROTO_HOPOPTS); +#endif #ifdef IPPROTO_ICMP insint(d, "IPPROTO_ICMP", IPPROTO_ICMP); #else @@ -3146,6 +3150,12 @@ #ifdef IPPROTO_GGP insint(d, "IPPROTO_GGP", IPPROTO_GGP); #endif +#ifdef IPPROTO_IPV4 + insint(d, "IPPROTO_IPV4", IPPROTO_IPV4); +#endif +#ifdef IPPROTO_IPIP + insint(d, "IPPROTO_IPIP", IPPROTO_IPIP); +#endif #ifdef IPPROTO_TCP insint(d, "IPPROTO_TCP", IPPROTO_TCP); #else @@ -3174,12 +3184,54 @@ #ifdef IPPROTO_TP insint(d, "IPPROTO_TP", IPPROTO_TP); #endif +#ifdef IPPROTO_IPV6 + insint(d, "IPPROTO_IPV6", IPPROTO_IPV6); +#endif +#ifdef IPPROTO_ROUTING + insint(d, "IPPROTO_ROUTING", IPPROTO_ROUTING); +#endif +#ifdef IPPROTO_FRAGMENT + insint(d, "IPPROTO_FRAGMENT", IPPROTO_FRAGMENT); +#endif +#ifdef IPPROTO_RSVP + insint(d, "IPPROTO_RSVP", IPPROTO_RSVP); +#endif +#ifdef IPPROTO_GRE + insint(d, "IPPROTO_GRE", IPPROTO_GRE); +#endif +#ifdef IPPROTO_ESP + insint(d, "IPPROTO_ESP", IPPROTO_ESP); +#endif +#ifdef IPPROTO_AH + insint(d, "IPPROTO_AH", IPPROTO_AH); +#endif +#ifdef IPPROTO_MOBILE + insint(d, "IPPROTO_MOBILE", IPPROTO_MOBILE); +#endif +#ifdef IPPROTO_ICMPV6 + insint(d, "IPPROTO_ICMPV6", IPPROTO_ICMPV6); +#endif +#ifdef IPPROTO_NONE + insint(d, "IPPROTO_NONE", IPPROTO_NONE); +#endif +#ifdef IPPROTO_DSTOPTS + insint(d, "IPPROTO_DSTOPTS", IPPROTO_DSTOPTS); +#endif #ifdef IPPROTO_XTP insint(d, "IPPROTO_XTP", IPPROTO_XTP); #endif #ifdef IPPROTO_EON insint(d, "IPPROTO_EON", IPPROTO_EON); #endif +#ifdef IPPROTO_PIM + insint(d, "IPPROTO_PIM", IPPROTO_PIM); +#endif +#ifdef IPPROTO_IPCOMP + insint(d, "IPPROTO_IPCOMP", IPPROTO_IPCOMP); +#endif +#ifdef IPPROTO_VRRP + insint(d, "IPPROTO_VRRP", IPPROTO_VRRP); +#endif #ifdef IPPROTO_BIP insint(d, "IPPROTO_BIP", IPPROTO_BIP); #endif @@ -3242,7 +3294,7 @@ insint(d, "INADDR_NONE", 0xffffffff); #endif - /* IP [gs]etsockopt options */ + /* IPv4 [gs]etsockopt options */ #ifdef IP_OPTIONS insint(d, "IP_OPTIONS", IP_OPTIONS); #endif @@ -3290,6 +3342,26 @@ #endif #ifdef IP_MAX_MEMBERSHIPS insint(d, "IP_MAX_MEMBERSHIPS", IP_MAX_MEMBERSHIPS); +#endif + + /* IPv6 [gs]etsockopt options, defined in RFC2553 */ +#ifdef IPV6_JOIN_GROUP + insint(d, "IPV6_JOIN_GROUP", IPV6_JOIN_GROUP); +#endif +#ifdef IPV6_LEAVE_GROUP + insint(d, "IPV6_LEAVE_GROUP", IPV6_LEAVE_GROUP); +#endif +#ifdef IPV6_MULTICAST_HOPS + insint(d, "IPV6_MULTICAST_HOPS", IPV6_MULTICAST_HOPS); +#endif +#ifdef IPV6_MULTICAST_IF + insint(d, "IPV6_MULTICAST_IF", IPV6_MULTICAST_IF); +#endif +#ifdef IPV6_MULTICAST_LOOP + insint(d, "IPV6_MULTICAST_LOOP", IPV6_MULTICAST_LOOP); +#endif +#ifdef IPV6_UNICAST_HOPS + insint(d, "IPV6_UNICAST_HOPS", IPV6_UNICAST_HOPS); #endif /* TCP options */ From cgw@alum.mit.edu Sat Aug 4 18:02:04 2001 From: cgw@alum.mit.edu (Charles G Waldman) Date: Sat, 4 Aug 2001 12:02:04 -0500 Subject: [Python-Dev] [offtopic] /usr/bin/env vs linux In-Reply-To: References: Message-ID: <15212.10892.890406.103990@nyx.dyndns.org> Greg Ward wrote: > > Completely off-topic: the "/usr/bin/env" hack doesn't work with > command-line arguments under Linux: > > Grumble. Andrew and I tried to track this down once, and found > suspicious-looking code in the kernel (ie., I don't think it's the fault > of GNU env). I've noticed this as well. Do you have any pointers to the suspicious kernel code? Did you bring this up on the linux kernel development list? From guido@zope.com Sat Aug 4 18:44:51 2001 From: guido@zope.com (Guido van Rossum) Date: Sat, 04 Aug 2001 13:44:51 -0400 Subject: [Python-Dev] [offtopic] /usr/bin/env vs linux In-Reply-To: Your message of "Sat, 04 Aug 2001 12:02:04 CDT." <15212.10892.890406.103990@nyx.dyndns.org> References: <15212.10892.890406.103990@nyx.dyndns.org> Message-ID: <200108041744.NAA11809@cj20424-a.reston1.va.home.com> > Greg Ward wrote: > > > > Completely off-topic: the "/usr/bin/env" hack doesn't work with > > command-line arguments under Linux: > > > > Grumble. Andrew and I tried to track this down once, and found > > suspicious-looking code in the kernel (ie., I don't think it's the fault > > of GNU env). > > I've noticed this as well. Do you have any pointers to the suspicious > kernel code? Did you bring this up on the linux kernel development > list? This is age-old. The kernel only passes on one argument (and it has to be of limited length). In "/usr/bin/env python", "python" is that one argument. I guess there's a really good reason (like resource constraints) for this or it would've been fixed ages ago, wouldn't it? --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Sat Aug 4 19:07:21 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Sat, 4 Aug 2001 14:07:21 -0400 (EDT) Subject: [Python-Dev] IPv6 docs In-Reply-To: <20010804162005.E1D3B7BB@starfruit.itojun.org> References: <20010804162005.E1D3B7BB@starfruit.itojun.org> Message-ID: <15212.14809.33733.24175@cj42289-a.reston1.va.home.com> Jun-ichiro itojun Hagino writes: > Here's a diff for adding documentation for IPv6 funcs in > socket module. I have no TeX on my laptop these days, so not > sure if it compiles clean. also if someone could go through > my broken english, it would be highly appreciated. If you could post this as a separate patch on SourceForge and label it as documentation, I should be able to get to it this coming week. (If it just sits in my inbox, it may take a lot longer. ;) Thanks! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From m@moshez.org Sat Aug 4 19:06:51 2001 From: m@moshez.org (Moshe Zadka) Date: Sat, 04 Aug 2001 21:06:51 +0300 Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: <20010804110653.A910@gerg.ca> References: <20010804110653.A910@gerg.ca>, <20010802150109.B2033@netthink.co.uk> <200108031233.IAA01557@cj20424-a.reston1.va.home.com> Message-ID: On Sat, 4 Aug 2001, Greg Ward wrote: > Completely off-topic: the "/usr/bin/env" hack doesn't work with > command-line arguments under Linux: > > $ cat t.py > #!/usr/bin/env python -i > > print "hello" > > $ ./t.py > /usr/bin/env: python -i: No such file or directory That's what POSIX mandates, IIRC. Blame UNIX for using such an ugly hack for shebang lines. It's probably better to use #!/usr/bin/python and use binfmt_misc under linux with the correct echo > something/in/proc magic to alias it to the correct interpreter. -- Moshe Zadka - http://moshez.geek (if you're not cool, http://moshez.org is still working) From guido@zope.com Sat Aug 4 19:30:09 2001 From: guido@zope.com (Guido van Rossum) Date: Sat, 04 Aug 2001 14:30:09 -0400 Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler In-Reply-To: Your message of "Sat, 04 Aug 2001 05:01:07 EDT." References: Message-ID: <200108041830.OAA12163@cj20424-a.reston1.va.home.com> > BTW, do we actually need *any* auxiliary data structure here? Looks > to me like all these giant pylist thingies could be tossed in favor > of making shallow copies of the input ifdlist etc list arguments. > The giant pylists don't *appear* to accomplish anything except cache > the result of a PyObject_AsFileDescriptor() call per object. But if > that's not an expensive call (doesn't look expensive to me), it's > not worth all this trouble no matter how spelled. Yes, I think we do need this. PyObject_AsFileDescriptor() calls the fileno() method which could be a Python call. Maybe we could use a more sophisticated construct where we use a short array allocated on the stack for lists of fewer than 100 (say) files but use malloc for larger ones? --Guido van Rossum (home page: http://www.python.org/~guido/) From ping@lfw.org Sat Aug 4 19:37:52 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Sat, 4 Aug 2001 11:37:52 -0700 (PDT) Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: <200108010306.XAA18133@cj20424-a.reston1.va.home.com> Message-ID: On Tue, 31 Jul 2001, Guido van Rossum wrote: > You misunderstand. You proposed a few lines that would automatically > do this in site.py, which is always imported. I propose to add those > same lines to cgi.py, so that any code that imports cgi.py > *automatically* has he feature enabled. I assume that all CGI scripts > have an import of cgi. Sorry that i misunderstood. This sounds reasonable to me (though it seems your opinions may have changed since you wrote this). > I don't want Python to become a wannabe CGI language [...] Python > is a decent language for CGI, but Perl is the established standard > and then there's PHP which also has way more users than Python. This attitude surprised me a little and sounded somewhat sad. "Perl is the established standard"? Aw. I say if people want a better tool, they should use it. The fact that Perl has more users than Python doesn't make us give up on Python for system administration or text processing or database access, even though Perl can do all these things. The fact is Python's quite good. :) > I don't want to make choices that benefit CGI programmers to the > detriment of others (CGI is actually a pretty lame way of producing > active web content). I guess i have a hard time seeing the "detriment" part. What great detriment to others is being caused here, by a feature that only affects CGI to begin with? -- ?!ng From martin@loewis.home.cs.tu-berlin.de Sat Aug 4 21:52:12 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Sat, 4 Aug 2001 22:52:12 +0200 Subject: [Python-Dev] [offtopic] /usr/bin/env vs linux Message-ID: <200108042052.f74KqCn02631@mira.informatik.hu-berlin.de> > This is age-old. The kernel only passes on one argument (and it has > to be of limited length). In "/usr/bin/env python", "python" is > that one argument. I guess there's a really good reason (like > resource constraints) for this or it would've been fixed ages ago, > wouldn't it? It's in fs/binfmt_script.c, which I wrote about five years ago. The specific code is around /* * OK, we've parsed out the interpreter name and * (optional) argument. * Splice in (1) the interpreter's name for argv[0] * (2) (optional) argument to interpreter * (3) filename of shell script (replace argv[0]) * * This is done in reverse order, because of how the * user environment and arguments are stored. */ AFAICS, there is no good reason to limit this to a single argument, except for laziness. I copied this fragment from the earlier #! implementation by tytso, and did not question it. OTOH, the size constraint comes from BINPRM_BUF_SIZE, which is 128. The kernel only offers this much data initially to a binary format module for format detection, and the binfmt_script driver doesn't attempt to read any further data from disk (which it could, if it wanted to). It seems that Single Unix does not mandate any behaviour in this respect; #! is still but a common extension. On Solaris, #! is documented in exec(2) as #! pathname [arg ] so I guess Linux was just following SunOS once more here. Regards, Martin From ping@lfw.org Sat Aug 4 21:53:24 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Sat, 4 Aug 2001 13:53:24 -0700 (PDT) Subject: [Python-Dev] Re: cgitb.py for Python 2.2 In-Reply-To: <200108040732.DAA10879@cj20424-a.reston1.va.home.com> Message-ID: On Sat, 4 Aug 2001, Guido van Rossum wrote: > Give it up, Ping. cgitb writes the tb to stdout, not to stderr, and > that's enough of a difference in behavior to require explicit > enabling. It's not worth your energy to try and convince us. Okay. I defer to your sense of judgement. (My last message was composed before i read your message quoted above.) -- ?!ng From ping@lfw.org Sat Aug 4 22:11:38 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Sat, 4 Aug 2001 14:11:38 -0700 (PDT) Subject: [Python-Dev] Proposed new module for the Python library In-Reply-To: <20010804111847.B910@gerg.ca> Message-ID: On 02 August 2001, Eric S. Raymond said: > There is a common `compiler-like' pattern in Unix scripts which is useful > for translation utilities of all sorts. On Sat, 4 Aug 2001, Greg Ward wrote: > Sounds like a useful tool to have around. I'm not sure the name > 'ccframe' conjures up images of the above paragraph, but I'm not sure > what name *does*. This is the hardest sort of software to name! I too find the name fairly cryptic. "fileargs" perhaps? But it would be better to think about merging this and fileinput. -- ?!ng From martin@loewis.home.cs.tu-berlin.de Sat Aug 4 23:28:32 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Sun, 5 Aug 2001 00:28:32 +0200 Subject: [Python-Dev] IPv6 docs Message-ID: <200108042228.f74MSWo10569@mira.informatik.hu-berlin.de> > If you could post this as a separate patch on SourceForge and label > it as documentation, I should be able to get to it this coming week. I've applied itojun's patch, with slight modifications. Itojun, I have a few questions on this documentation, though: You write that socket.herror and socket.gaierror may both take either a string or a pair of error number/message. Is this correct, or was that a cut-n-paste error from socket.error? You also write that socket.gethostbyname may raise socket.herror. Is this true? It seems that you'll always get socket.gaierror instead. I've changed it to say that gethostbyname_ex may raise herrors. Anyway, thanks for the patch. Regards, Martin From martin@loewis.home.cs.tu-berlin.de Sat Aug 4 23:40:01 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Sun, 5 Aug 2001 00:40:01 +0200 Subject: [Python-Dev] socketmodule fix for IPv6 (mostly additional symbols) Message-ID: <200108042240.f74Me1i10657@mira.informatik.hu-berlin.de> Hi itojun, Thanks for your patch; I just applied it. Please don't send patches to python-dev, though. If you cannot use SourceForge for some reason, please send them to me instead. Note that SF really is the preferred channel for these things. Regards, Martin From gstein@lyra.org Sun Aug 5 01:35:21 2001 From: gstein@lyra.org (Greg Stein) Date: Sat, 4 Aug 2001 17:35:21 -0700 Subject: [Python-Dev] Re: WebDAV in python 2.2? In-Reply-To: ; from m@moshez.org on Sat, Aug 04, 2001 at 10:55:44AM +0300 References: <20010803121254.B29583@lyra.org>, <200108031254.IAA01818@cj20424-a.reston1.va.home.com> <20010803121254.B29583@lyra.org> Message-ID: <20010804173521.B31732@lyra.org> On Sat, Aug 04, 2001 at 10:55:44AM +0300, Moshe Zadka wrote: > On Fri, 3 Aug 2001 12:12:54 -0700, Greg Stein wrote: > > > That is at least a start on what needs to happen. We can flesh it out with a > > PEP or by checking in the modules. > > (Moshe and I talked about httpauth a bit at the last Python Conference; I > > know he is interested in working on something like that) > > Let me just clarify what we talked about: today, the HTTPConnection already > know how to respond to code 100. Euh... no, it doesn't. It *should*, but it doesn't today. To do it right, a client should be able to install a callback for when the 100-Continue is sent by the server. [ in fact, httplib's HTTPConnection and HTTPResponse should one day be reworked further to support a callback style model, passing the response object; that model allows for HTTP pipelining ] > urllib2 has a very nifty mechanism of handling > all kinds of errors. I have a feeling the responsiblity split is not correct, > and I want to think about some kind of better responsibility split. As I mentioned above, I'll be writing up a PEP discussing this stuff (since it is non-trivial, and I'm guessing there is more than just a couple people's input). Suggestions can go in there. I realized that it shouldn't be "httpauth" but something like "httpext" (extended stuff). We also want proxy support in there. Note that proxy support is connection-based. Authentication is per-request, but we can attach "defaults" for a connection. urllib(2) would simply use the richer HTTPConnection class, shedding some of its work. urllib is about handling a specific URL (of many varieties, not just HTTP), while the connection is about a connection to a server for multiple requests. We can bulk up the connection (IMO, via subclassing), and keep the rich facilities of urllib's "single" request basis. (yes, urllib caches and has richer error handling and stuff, but that is beside the point of beefing up the connection for *all* HTTP clients) Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Sun Aug 5 01:38:56 2001 From: gstein@lyra.org (Greg Stein) Date: Sat, 4 Aug 2001 17:38:56 -0700 Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?] In-Reply-To: <20010803202227.A31091@lyra.org>; from gstein@lyra.org on Fri, Aug 03, 2001 at 08:22:27PM -0700 References: <20010803202227.A31091@lyra.org> Message-ID: <20010804173856.C31732@lyra.org> > From: Guido van Rossum > Subject: Re: [Python-Dev] Re: WebDAV in python 2.2? > To: Greg Stein > Date: Fri, 03 Aug 2001 15:18:37 -0400 >... > > I'm not sure how we normally write tests for network-based modules, but it > > shouldn't be too hard to have them run against test.webdav.org. > > Can someone help Greg with his question about testing network modules? > Generally the test suite doesn't assume internet access; network code > is "tested" using a second thread and localhost. Quite doable, but we don't really have a DAV server that davlib.py could hit via localhost. I'm sure somebody could jury-rig a simple responder, though. Avoiding the bailing-wire-and-glue bit was the reason for suggesting test.webdav.org. Hmm. Some kind of subclass of BaseHTTPServer or something could probably just send back canned responses. Note that even httplib.py doesn't attempt a localhost connection for testing. Cheers, -g -- Greg Stein, http://www.lyra.org/ From guido@zope.com Sun Aug 5 03:19:57 2001 From: guido@zope.com (Guido van Rossum) Date: Sat, 04 Aug 2001 22:19:57 -0400 Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?] In-Reply-To: Your message of "Sat, 04 Aug 2001 17:38:56 PDT." <20010804173856.C31732@lyra.org> References: <20010803202227.A31091@lyra.org> <20010804173856.C31732@lyra.org> Message-ID: <200108050219.WAA12942@cj20424-a.reston1.va.home.com> > Quite doable, but we don't really have a DAV server that davlib.py could hit > via localhost. I'm sure somebody could jury-rig a simple responder, though. > Avoiding the bailing-wire-and-glue bit was the reason for suggesting > test.webdav.org. But we don't control how test.webdav.org will respond two (or ten) years from now. Plus there are firewalls, you don't want your modem to start dialing, etc., etc. > Hmm. Some kind of subclass of BaseHTTPServer or something could probably > just send back canned responses. Exactly. That's how SocketServer tests itself these days. > Note that even httplib.py doesn't attempt a localhost connection for > testing. The test in the test suite doesn't use the network at all. The test in the module is not used, exactly for this reason: it relies on accessibility of external resources. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Sun Aug 5 04:17:50 2001 From: tim.one@home.com (Tim Peters) Date: Sat, 4 Aug 2001 23:17:50 -0400 Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler In-Reply-To: <200108041830.OAA12163@cj20424-a.reston1.va.home.com> Message-ID: [Guido, on reworking select()] > Yes, I think we do need this. PyObject_AsFileDescriptor() calls the > fileno() method which could be a Python call. Ya, but for me to believe that *matters*, I have to believe people worried about high-performance select are passing objects with Python-implemented fileno methods to begin with. That's quite a stretch, isn't it? They have to pay for fileno() calls regardless, the question is only whether they pay it once or twice per object. > Maybe we could use a more sophisticated construct where we use a short > array allocated on the stack for lists of fewer than 100 (say) files > but use malloc for larger ones? Another idea: screw it . We have to allocate three lists anyway to hold the results. Allocate them at the start instead of the end, and then they can do double-duty, holding the cached fileno() results across the select() call. Fleshing it out a little, def list2set(inputlist, fdset): "Return (list of file descriptors, biggest fd seen); set fdset slots." n = len(inputlist) result = [None] * n if _MSC_VER: biggest = 0 else: biggest = -1 for i in range(n): obj = inputlist[i] fd = PyObject_AsFileDescriptor(obj) if fd == -1: return NULL, -1 result[i] = obj, fd FD_SET(fd, fdset) if not _MSC_VER: biggest = max(biggest, fd) return result, biggest # iolist is the result of an earlier list2set() call. It gets shrunk # to hold just the objects corresponding to the fds in fdset. def set2list(iolist, fdset): lastindex = 0 for obj, fd in iolist: if FD_ISSET(fd, fdset): iolist[lastindex] = obj lastindex += 1 if lastindex < len(iolist): del iolist[lastindex:] So it still does the perversely optimized caching, but no stack space required, no free() calls, only the 3 malloc() calls that are needed anyway, and up to 3 realloc() calls to shrink the output lists. It does create a bunch of 2-tuples, but they're on a free list and come cheap. From itojun@iijlab.net Sun Aug 5 05:57:26 2001 From: itojun@iijlab.net (Jun-ichiro itojun Hagino) Date: Sun, 05 Aug 2001 13:57:26 +0900 Subject: [Python-Dev] socketmodule fix for IPv6 (mostly additional symbols) In-Reply-To: martin's message of Sun, 05 Aug 2001 00:40:01 +0200. <200108042240.f74Me1i10657@mira.informatik.hu-berlin.de> Message-ID: <20010805045727.E78007BB@starfruit.itojun.org> >Thanks for your patch; I just applied it. Please don't send patches to >python-dev, though. If you cannot use SourceForge for some reason, >please send them to me instead. Note that SF really is the preferred >channel for these things. i see, next time i'll do so. thanks! itojun From itojun@iijlab.net Sun Aug 5 05:58:31 2001 From: itojun@iijlab.net (Jun-ichiro itojun Hagino) Date: Sun, 05 Aug 2001 13:58:31 +0900 Subject: [Python-Dev] IPv6 docs In-Reply-To: martin's message of Sun, 05 Aug 2001 00:28:32 +0200. <200108042228.f74MSWo10569@mira.informatik.hu-berlin.de> Message-ID: <20010805045831.9A7F87BB@starfruit.itojun.org> >You write that socket.herror and socket.gaierror may both take either >a string or a pair of error number/message. Is this correct, or was >that a cut-n-paste error from socket.error? yes, it is a cut-n-paste error. herror and gaierror is always a 2-tuple. >You also write that socket.gethostbyname may raise socket.herror. Is >this true? It seems that you'll always get socket.gaierror instead. >I've changed it to say that gethostbyname_ex may raise herrors. with C API gethostbyname() should return h_error. Python code internaly uses getnameinfo/getaddrinfo, so it returns socket.gaierror. you right. itojun From m@moshez.org Sun Aug 5 06:07:19 2001 From: m@moshez.org (Moshe Zadka) Date: Sun, 05 Aug 2001 08:07:19 +0300 Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?] In-Reply-To: <20010804173856.C31732@lyra.org> References: <20010804173856.C31732@lyra.org>, <20010803202227.A31091@lyra.org> Message-ID: On Sat, 4 Aug 2001 17:38:56 -0700, Greg Stein wrote: > Quite doable, but we don't really have a DAV server that davlib.py could hit > via localhost. http://webdav.de has one. > Note that even httplib.py doesn't attempt a localhost connection for > testing. But it should ;-) -- Moshe Zadka - http://moshez.geek (if you're not cool, http://moshez.org is still working) From rnd@onego.ru Sun Aug 5 08:54:14 2001 From: rnd@onego.ru (Roman Suzi) Date: Sun, 5 Aug 2001 11:54:14 +0400 (MSD) Subject: [Python-Dev] '*' instead of 'yield'? Message-ID: Hello! Maybe it will be nicier to have unary * instead of 'yield' for generators? Example I already posted to c.l.p: from __future__ import generators import calendar, string def calt(year, month): month_cal = calendar.monthcalendar(year, month) * """%s.%s""" % (month, year) * """""" for week in month_cal: * """\n""" for day in week: * """""" % (day or " ") * """""" * """
%s
""" print string.join(calt(2001, 8), "") (This will also enable yields from expressions) Sincerely yours, Roman Suzi -- _/ Russia _/ Karelia _/ Petrozavodsk _/ rnd@onego.ru _/ _/ Sunday, August 05, 2001 _/ Powered by Linux RedHat 6.2 _/ _/ "By all means, let's not confuse ourselves with the facts!" _/ From fdrake@acm.org Sun Aug 5 16:24:03 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Sun, 5 Aug 2001 11:24:03 -0400 (EDT) Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?] In-Reply-To: References: <20010804173856.C31732@lyra.org> <20010803202227.A31091@lyra.org> Message-ID: <15213.25875.726178.186594@cj42289-a.reston1.va.home.com> Moshe Zadka writes: > http://webdav.de has one. The pages there are inconsistent in their claims of status and version number, so it's not clear what the actual features are. I suspect that only the front page is up to date. Attempting to actually download the software causes an authentication dialog to pop up, so something is probably screwed up as well. The pages look slick, though. ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From mwh@python.net Sun Aug 5 19:11:18 2001 From: mwh@python.net (Michael Hudson) Date: 05 Aug 2001 14:11:18 -0400 Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler In-Reply-To: "Tim Peters"'s message of "Sat, 4 Aug 2001 23:17:50 -0400" References: Message-ID: <2mhevm4bsp.fsf@starship.python.net> "Tim Peters" writes: > [Guido, on reworking select()] > > Yes, I think we do need this. PyObject_AsFileDescriptor() calls the > > fileno() method which could be a Python call. > > Ya, but for me to believe that *matters*, I have to believe people worried > about high-performance select are passing objects with Python-implemented > fileno methods to begin with. That's quite a stretch, isn't it? They have > to pay for fileno() calls regardless, the question is only whether they pay > it once or twice per object. Posted admittedly without checking any facts: (1) Doesn't the Python fileno() method issue leave you open to ridiculous fileno() methods that mutate the list passed in to select()? (2) AIUI, if you really care about performance you should be using poll() anyway (tho' obviously this depends on your platform having poll() in the first place, and I don't have the faintest idea how widespread it is). Cheers, M. -- The ultimate laziness is not using Perl. That saves you so much work you wouldn't believe it if you had never tried it. -- Erik Naggum, comp.lang.lisp From tim.one@home.com Sun Aug 5 23:50:06 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 5 Aug 2001 18:50:06 -0400 Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler In-Reply-To: <2mhevm4bsp.fsf@starship.python.net> Message-ID: [Michael Hudson] > ... > (1) Doesn't the Python fileno() method issue leave you open to > ridiculous fileno() methods that mutate the list passed in to > select()? Yes, but that's par for the course, and selectmodule is suitably paranoid about it (unfortunately, this is *so* common in the implementation that it's rarely mentioned in comments -- so there's a real danger of "simplifying" seemingly overly-elaborate code into creating a hole). list2set() is unusual in letting PyList_GetItem catch it, though: int len = PyList_Size(list); ... for (i = 0; i < len; i++) { SOCKET v; /* any intervening fileno() calls could decr this refcnt */ if (!(o = PyList_GetItem(list, i))) return -1; Py_INCREF(o); It's more common to see for (i = 0; i < PyList_Size(list); i++) { i.e. to recompute the list length on each trip, and so silently ignore length-changing mutations. > (2) AIUI, New one on me! at-least-as-i-understand-it-ly y'rs - tim From gstein@lyra.org Sun Aug 5 23:58:17 2001 From: gstein@lyra.org (Greg Stein) Date: Sun, 5 Aug 2001 15:58:17 -0700 Subject: [Python-Dev] '*' instead of 'yield'? In-Reply-To: ; from rnd@onego.ru on Sun, Aug 05, 2001 at 11:54:14AM +0400 References: Message-ID: <20010805155817.B32658@lyra.org> On Sun, Aug 05, 2001 at 11:54:14AM +0400, Roman Suzi wrote: > Hello! > > Maybe it will be nicier to have unary * instead of 'yield' > for generators? This is Python, not Perl. Why use an obscure character instead of a keyword? It doesn't make sense to me... Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Mon Aug 6 00:00:11 2001 From: gstein@lyra.org (Greg Stein) Date: Sun, 5 Aug 2001 16:00:11 -0700 Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?] In-Reply-To: <15213.25875.726178.186594@cj42289-a.reston1.va.home.com>; from fdrake@acm.org on Sun, Aug 05, 2001 at 11:24:03AM -0400 References: <20010804173856.C31732@lyra.org> <20010803202227.A31091@lyra.org> <15213.25875.726178.186594@cj42289-a.reston1.va.home.com> Message-ID: <20010805160011.C32658@lyra.org> That is why I mentioned test.webdav.org :-) But no matter... we can have a loopback-based test. Cheers, -g On Sun, Aug 05, 2001 at 11:24:03AM -0400, Fred L. Drake, Jr. wrote: > > Moshe Zadka writes: > > http://webdav.de has one. > > The pages there are inconsistent in their claims of status and > version number, so it's not clear what the actual features are. I > suspect that only the front page is up to date. > Attempting to actually download the software causes an > authentication dialog to pop up, so something is probably screwed up > as well. > The pages look slick, though. ;-) > > > -Fred > > -- > Fred L. Drake, Jr. > PythonLabs at Zope Corporation > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev -- Greg Stein, http://www.lyra.org/ From tim.one@home.com Mon Aug 6 00:12:40 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 5 Aug 2001 19:12:40 -0400 Subject: [Python-Dev] '*' instead of 'yield'? In-Reply-To: <20010805155817.B32658@lyra.org> Message-ID: [Roman Suzi] > Maybe it will be nicier to have unary * instead of 'yield' > for generators? [Greg Stein] > This is Python, not Perl. Why use an obscure character instead of > a keyword? > It doesn't make sense to me... I doubt it would be warmly received in Perl-land, either. Roman, this will never fly unless you can talk Guido into also allowing "**" as a synonym for "return". I wish you luck . From thomas@xs4all.net Mon Aug 6 00:12:28 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 6 Aug 2001 01:12:28 +0200 Subject: [Python-Dev] '*' instead of 'yield'? In-Reply-To: References: Message-ID: <20010806011228.P626@xs4all.nl> On Sun, Aug 05, 2001 at 11:54:14AM +0400, Roman Suzi wrote: > Maybe it will be nicier to have unary * instead of 'yield' > for generators? Maybe not. Very certainly not for me, in any case :) > Example I already posted to c.l.p: > from __future__ import generators > import calendar, string > def calt(year, month): > month_cal = calendar.monthcalendar(year, month) > * """%s.%s""" % (month, year) > * """""" > for week in month_cal: > * """\n""" > for day in week: > * """""" % (day or " ") > * """""" > * """
%s
""" This looks so ugly and unreadable to me, it's just not funny. It makes '*' look like some kind of comment character. > (This will also enable yields from expressions) There is no reason for 'yield' not to be an expression. "not" and "and" and "lambda" and such are all expressions too, remember. But making 'yield' a statement is a very good choice, in my eyes. What should ((yield a) ** (yield b) and (yield c)) ^ (yield e) / yield f yield ? 'yield' is too similar to 'return' for it not to be a statement. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido@zope.com Mon Aug 6 01:30:36 2001 From: guido@zope.com (Guido van Rossum) Date: Sun, 05 Aug 2001 20:30:36 -0400 Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler In-Reply-To: Your message of "05 Aug 2001 14:11:18 EDT." <2mhevm4bsp.fsf@starship.python.net> References: <2mhevm4bsp.fsf@starship.python.net> Message-ID: <200108060030.UAA19421@cj20424-a.reston1.va.home.com> > "Tim Peters" writes: > > > [Guido, on reworking select()] > > > Yes, I think we do need this. PyObject_AsFileDescriptor() calls the > > > fileno() method which could be a Python call. > > > > Ya, but for me to believe that *matters*, I have to believe people worried > > about high-performance select are passing objects with Python-implemented > > fileno methods to begin with. That's quite a stretch, isn't it? They have > > to pay for fileno() calls regardless, the question is only whether they pay > > it once or twice per object. [And Tim posted an interesting alternative implementation] That looks like a good way to avoid the double fileno() calls. Drawback: the implementation you sketched seems to create a twople per file descriptor. That's not quite a malloc (twoples are cached), but still. If someone feels like implementing this, please go ahead! [MH] > Posted admittedly without checking any facts: > > (1) Doesn't the Python fileno() method issue leave you open to > ridiculous fileno() methods that mutate the list passed in to > select()? Yes, but the select code is immune to that -- it may get an exception but it correctly propagates those. > (2) AIUI, if you really care about performance you should be using > poll() anyway (tho' obviously this depends on your platform having > poll() in the first place, and I don't have the faintest idea how > widespread it is). Very widespread. I forget -- what's the reason for poll()'s claim to better performance? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@zope.com Mon Aug 6 01:38:17 2001 From: guido@zope.com (Guido van Rossum) Date: Sun, 05 Aug 2001 20:38:17 -0400 Subject: [Python-Dev] '*' instead of 'yield'? In-Reply-To: Your message of "Mon, 06 Aug 2001 01:12:28 +0200." <20010806011228.P626@xs4all.nl> References: <20010806011228.P626@xs4all.nl> Message-ID: <200108060038.UAA21789@cj20424-a.reston1.va.home.com> Of all the syntactic changes proposed on the newsgroup, by my estimate less than 0.1% ever makes it into the language. There's no need to worry that this will change, "print >>" notwithstanding. :) --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Mon Aug 6 01:46:44 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 5 Aug 2001 20:46:44 -0400 Subject: [Python-Dev] '*' instead of 'yield'? In-Reply-To: <20010806011228.P626@xs4all.nl> Message-ID: [Thomas Wouters] > ... > There is no reason for 'yield' not to be an expression. "not" and > "and" and "lambda" and such are all expressions too, remember. But > making 'yield' a statement is a very good choice, in my eyes. What > should > > ((yield a) ** (yield b) and (yield c)) ^ (yield e) / yield f > > yield ? 'yield' is too similar to 'return' for it not to be a statement. If we had been able to think of an "obviously correct" way for yield to return a value, we would have -- but we didn't, so we didn't . The temptation is that it's very handy for a coroutine transfer to "act like" a function call from a transferor's POV: this gives a way for *its* resumer to communicate info back to it. I first saw this in Icon, and it was a huge and obvious improvement over Simula 67, wherein coroutines could communicate only via stuffing values in shared globals. Since PEP 255's "Simple Generators" are only semi-coroutines, the need isn't acute, but could still be handy at times. Anyway, Neil implemented *lots* of ideas that didn't survive: we all agreed to slice out features left and right until the Simple in "Simple Generators" earned its name. They're still fancier than Icon's and CLU's generators (because explicitly resumable even in the absence of magical control context) all the same. I'm curious to see how people use the "extra" power in practice, and can't predict which kinds of extensions would be truly useful. Until real life makes that clear(er), I'm inclinded to resist any change; in the meantime, we haven't precluded turning yield into an operator someday, neither extending its syntax as a statement. right-now-it's-the-simplest-thing-that-could-possibly-work-ly y'rs - tim From tim.one@home.com Mon Aug 6 01:55:16 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 5 Aug 2001 20:55:16 -0400 Subject: [Python-Dev] '*' instead of 'yield'? In-Reply-To: <200108060038.UAA21789@cj20424-a.reston1.va.home.com> Message-ID: [Guido] > Of all the syntactic changes proposed on the newsgroup, by my estimate > less than 0.1% ever makes it into the language. Heh: that shows how little you still read the newsgroup <0.9 wink>. Note that PEP 255 recorded all of these variants suggested for "yield": return 3 and continue return and continue 3 return generating 3 continue return 3 return >> , 3 from generator return 3 return >> 3 return << 3 >> 3 << 3 I suppose I should add * 3 to the list now. if-you-accepted-0.1%-of-syntax-suggestions-it-would-have-stopped- looking-like-python-in-'94-ly y'rs - tim From skip@pobox.com (Skip Montanaro) Mon Aug 6 05:11:35 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Sun, 5 Aug 2001 23:11:35 -0500 Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?] In-Reply-To: <20010805160011.C32658@lyra.org> References: <20010804173856.C31732@lyra.org> <20010803202227.A31091@lyra.org> <15213.25875.726178.186594@cj42289-a.reston1.va.home.com> <20010805160011.C32658@lyra.org> Message-ID: <15214.6391.208655.549066@beluga.mojam.com> Greg> That is why I mentioned test.webdav.org :-) Greg> But no matter... we can have a loopback-based test. Tests are skipped because imports fail. Seems reasonable to me that some tests might be skipped if the network isn't up. It shouldn't be difficult to test if test.webdav.org is reachable and skip the davlib test if it's not. Skip From guido@zope.com Mon Aug 6 05:16:49 2001 From: guido@zope.com (Guido van Rossum) Date: Mon, 06 Aug 2001 00:16:49 -0400 Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?] In-Reply-To: Your message of "Sun, 05 Aug 2001 23:11:35 CDT." <15214.6391.208655.549066@beluga.mojam.com> References: <20010804173856.C31732@lyra.org> <20010803202227.A31091@lyra.org> <15213.25875.726178.186594@cj42289-a.reston1.va.home.com> <20010805160011.C32658@lyra.org> <15214.6391.208655.549066@beluga.mojam.com> Message-ID: <200108060416.AAA22597@cj20424-a.reston1.va.home.com> > Tests are skipped because imports fail. Seems reasonable to me that some > tests might be skipped if the network isn't up. It shouldn't be difficult > to test if test.webdav.org is reachable and skip the davlib test if it's > not. Sure, but only if we absolutely cannot test that module without reaching out over the net. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com (Skip Montanaro) Mon Aug 6 05:43:10 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Sun, 5 Aug 2001 23:43:10 -0500 Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?] In-Reply-To: <200108060416.AAA22597@cj20424-a.reston1.va.home.com> References: <20010804173856.C31732@lyra.org> <20010803202227.A31091@lyra.org> <15213.25875.726178.186594@cj42289-a.reston1.va.home.com> <20010805160011.C32658@lyra.org> <15214.6391.208655.549066@beluga.mojam.com> <200108060416.AAA22597@cj20424-a.reston1.va.home.com> Message-ID: <15214.8287.895.41269@beluga.mojam.com> >>>>> "Guido" == Guido van Rossum writes: >> Tests are skipped because imports fail. Seems reasonable to me that >> some tests might be skipped if the network isn't up. It shouldn't be >> difficult to test if test.webdav.org is reachable and skip the davlib >> test if it's not. Guido> Sure, but only if we absolutely cannot test that module without Guido> reaching out over the net. I thought that was the whole idea of this thread. There are apparently a couple fully functional dav servers out there. In the short-term, I think it would be better to write a test that relies on them than having to write a server - no matter how bare bones it is - just to perform some tests of a client-side library. who's-going-to-test-the-test-server?-ly y'rs, Skip From tim.one@home.com Mon Aug 6 06:17:15 2001 From: tim.one@home.com (Tim Peters) Date: Mon, 6 Aug 2001 01:17:15 -0400 Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler In-Reply-To: <200108060030.UAA19421@cj20424-a.reston1.va.home.com> Message-ID: [Guido, on Tim's select() thought-algorithm] > That looks like a good way to avoid the double fileno() calls. > Drawback: the implementation you sketched seems to create a twople per > file descriptor. That's not quite a malloc (twoples are cached), but > still. > > If someone feels like implementing this, please go ahead! Yes, it does create twoples (as I said in my original msg). That can also be avoided: allocate result lists initially each the same size as its corresponding input list. Store just fds in the slots. The twoples are now implicit, in parallel lists (reusing the input lists as-is to continue storing the objects). The sole reason this can't work now is that a fileno() call may reach back in and mutate one of the input lists -- that's the only thing creating explicit twoples "off to the side" is guarding against, and the only real reason I can see for why the current code is allocating giant pylist thingies and storing "redundant" pointers to the objects in them. So play the same trick list.sort plays: redirect the input lists' type pointers to the immutable list object for the duration of select(). It's really no feature that select() can return ill-defined results if a fileno() method mutates the list! I'd rather the language raise an exception if anyone *tries* to mutate an input list for the duration -- it's never a reasonable thing for a fileno() method to do. Then again, I was always the immutable-list trick's biggest fan . From mwh@python.net Mon Aug 6 10:18:33 2001 From: mwh@python.net (Michael Hudson) Date: 06 Aug 2001 05:18:33 -0400 Subject: [Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler In-Reply-To: Guido van Rossum's message of "Sun, 05 Aug 2001 20:30:36 -0400" References: <2mhevm4bsp.fsf@starship.python.net> <200108060030.UAA19421@cj20424-a.reston1.va.home.com> Message-ID: <2m4rrlwnpy.fsf@starship.python.net> Guido van Rossum writes: > > (2) AIUI, if you really care about performance you should be using > > poll() anyway (tho' obviously this depends on your platform having > > poll() in the first place, and I don't have the faintest idea how > > widespread it is). > > Very widespread. I forget -- what's the reason for poll()'s claim to > better performance? Appropriately enough, it avoids the need to much about with fd_sets for each syscall, which makes a difference when yo are dealing with huge numbers of sockets (again, I think). Cheers, M. -- I have long since given up dealing with people who hold idiotic opinions as if they had arrived at them through thinking about them. -- Erik Naggum, comp.lang.lisp From rnd@onego.ru Mon Aug 6 10:21:42 2001 From: rnd@onego.ru (Roman Suzi) Date: Mon, 6 Aug 2001 13:21:42 +0400 (MSD) Subject: [Python-Dev] '*' instead of 'yield'? In-Reply-To: Message-ID: On Sun, 5 Aug 2001, Tim Peters wrote: >[Guido] >> Of all the syntactic changes proposed on the newsgroup, by my estimate >> less than 0.1% ever makes it into the language. > >Heh: that shows how little you still read the newsgroup <0.9 wink>. Note >that PEP 255 recorded all of these variants suggested for "yield": > > return 3 and continue > return and continue 3 > return generating 3 > continue return 3 > return >> , 3 > from generator return 3 > return >> 3 > return << 3 > >> 3 > << 3 > >I suppose I should add > > * 3 '*' looks right to be yield: like a hole after something got thru paper ;-) I do not remember, but probably I also proposed: < 3 >to the list now. Or, yes, and next step is to use '?' in place of 'print' ;-) from __future__ import parrot # must allow for such things: = myfun(x, y, z): ? "how are you?" S = 0 4 i <- x..y,z: S += i ** S = mygen(x): 4 k <- 1..x: * k ? myfun(1, 3, 5) ? mygen(20) Another approach is lobby Python reserved words into Unicode. (Like APL did, IIRC). Then they could be represented by nice pictograms. I think, yhis thread is not constructive any more, as 90% of voices were AGAINST. Sincerely yours, Roman Suzi -- _/ Russia _/ Karelia _/ Petrozavodsk _/ rnd@onego.ru _/ _/ Monday, August 06, 2001 _/ Powered by Linux RedHat 6.2 _/ _/ "A pessimist is never disappointed." _/ From m@moshez.org Mon Aug 6 13:47:17 2001 From: m@moshez.org (Moshe Zadka) Date: Mon, 06 Aug 2001 15:47:17 +0300 Subject: [guido@zope.com: Re: [Python-Dev] Re: WebDAV in python 2.2?] In-Reply-To: <15214.6391.208655.549066@beluga.mojam.com> References: <15214.6391.208655.549066@beluga.mojam.com>, <20010804173856.C31732@lyra.org> <20010803202227.A31091@lyra.org> <15213.25875.726178.186594@cj42289-a.reston1.va.home.com> <20010805160011.C32658@lyra.org> Message-ID: On Sun, 5 Aug 2001 23:11:35 -0500, Skip Montanaro wrote: > Greg> That is why I mentioned test.webdav.org :-) > Greg> But no matter... we can have a loopback-based test. > > Tests are skipped because imports fail. Seems reasonable to me that some > tests might be skipped if the network isn't up. It shouldn't be difficult > to test if test.webdav.org is reachable and skip the davlib test if it's > not. The test for reachability might automatically dial-up for diald or its windows equivalent. -- Moshe Zadka - http://moshez.geek (if you're not cool, http://moshez.org is still working) From akuchlin@mems-exchange.org Mon Aug 6 18:40:46 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Mon, 06 Aug 2001 13:40:46 -0400 Subject: [Python-Dev] Cleaning up Misc/ directory Message-ID: There are a few files in the Misc/ directory that can probably be discarded: BLURB, BLURB.*, and HYPE : several duplicated descriptions of Python. I volunteer to edit them down to one BLURB file. editline-fix : is the problem and fix described still relevant? find_recursionlimit.py : unnecessary now that sys.getrecursionlimit() exists. fixfuncptrs.sh : seems to add casts to the type structure in a module RFD : the RFD for comp.lang.python. Can disappear now that the group has been created. --amk From skip@pobox.com (Skip Montanaro) Mon Aug 6 19:28:52 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 6 Aug 2001 13:28:52 -0500 Subject: [Python-Dev] Cleaning up Misc/ directory In-Reply-To: References: Message-ID: <15214.57828.64490.178664@beluga.mojam.com> amk> editline-fix : is the problem and fix described still relevant? I sort of doubt it. I'd delete it. Editline has changed in the last six years. I haven't linked Python with it in nearly that long. There are other command line completion libraries available as well. I just stumbled upon tecla: http://www.astro.caltech.edu/~mcs/tecla/index.html which seems to be actively maintained. Anybody tried it? It has an X11-style license and is a little over half the size of libreadline. Unlike editline, it appears it doesn't attempt to conform to readline's API in the areas the two libraries overlap. Doesn't look like it supports history files either. Skip From nas@python.ca Mon Aug 6 19:39:06 2001 From: nas@python.ca (Neil Schemenauer) Date: Mon, 6 Aug 2001 11:39:06 -0700 Subject: [Python-Dev] Cleaning up Misc/ directory In-Reply-To: ; from akuchlin@mems-exchange.org on Mon, Aug 06, 2001 at 01:40:46PM -0400 References: Message-ID: <20010806113906.A29653@glacier.fnational.com> Andrew Kuchling wrote: > find_recursionlimit.py : unnecessary now that sys.getrecursionlimit() > exists. Those two don't do the same thing. sys.getrecursionlimit() returns the current limit. find_recursionlimit.py tries to find out how high you can set it until you run out of stack space. Neil From guido@zope.com Mon Aug 6 19:40:17 2001 From: guido@zope.com (Guido van Rossum) Date: Mon, 06 Aug 2001 14:40:17 -0400 Subject: [Python-Dev] Cleaning up Misc/ directory In-Reply-To: Your message of "Mon, 06 Aug 2001 13:40:46 EDT." References: Message-ID: <200108061840.f76IeHO24574@odiug.digicool.com> > There are a few files in the Misc/ directory that can probably be > discarded: > > BLURB, BLURB.*, and HYPE : several duplicated descriptions of Python. > I volunteer to edit them down to one BLURB file. I would just delete them -- we have plenty of such material on the website. > editline-fix : is the problem and fix described still relevant? See Skip's response. Toss it. > find_recursionlimit.py : unnecessary now that sys.getrecursionlimit() > exists. Toss it. > fixfuncptrs.sh : seems to add casts to the type structure in a module Yeah, drop it. > RFD : the RFD for comp.lang.python. Can disappear now that the group > has been created. Hm, *that* one is cute as a historic document. You can also lose cheatsheet (a *very* old summary, there are much better ones around), comparisons (we have comparisons on the website now), faq2html.py (no longer used or relevant), and renumber.py (ditto -- we now have the FAQ wizard so we no longer maintain the FAQ in text form). --Guido van Rossum (home page: http://www.python.org/~guido/) From akuchlin@mems-exchange.org Mon Aug 6 20:32:13 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Mon, 6 Aug 2001 15:32:13 -0400 Subject: [Python-Dev] Cleaning up Misc/ directory In-Reply-To: <200108061840.f76IeHO24574@odiug.digicool.com>; from guido@zope.com on Mon, Aug 06, 2001 at 02:40:17PM -0400 References: <200108061840.f76IeHO24574@odiug.digicool.com> Message-ID: <20010806153212.G15095@ute.cnri.reston.va.us> On Mon, Aug 06, 2001 at 02:40:17PM -0400, Guido van Rossum wrote: >You can also lose cheatsheet ... All deleted, except for find_recursionlimit and RFD, and cheatsheet which has been updated to 2.0 (as previously discussed). --amk From klm@zope.com Mon Aug 6 23:04:00 2001 From: klm@zope.com (Ken Manheimer) Date: Mon, 6 Aug 2001 18:04:00 -0400 (EDT) Subject: [Python-Dev] '*' instead of 'yield'? In-Reply-To: Message-ID: Roman, that was a sensible way to end this particular thread - forgive me for gratuitously extending it a tiny bit, but i just can't resist... On Mon, 6 Aug 2001, Roman Suzi wrote: > [...] > Or, yes, and next step is to use '?' in place of 'print' ;-) > > from __future__ import parrot > > # must allow for such things: > > = myfun(x, y, z): > ? "how are you?" > S = 0 > 4 i <- x..y,z: > S += i > ** S > > = mygen(x): > 4 k <- 1..x: > * k > > ? myfun(1, 3, 5) > ? mygen(20) I don't feel this is an, um, balanced example if you leave alone the only non-expression in python that *is* signalled soley by punctuation, so: from __future__ import parrot comment must allow for such things: endcomment = myfun(x, y, z): ? "how are you?" S = 0 4 i <- x..y,z: S += i ** S ... :) and-voluntary-block-delimeter-close-would-be-"comment end endcomment"'ly y'rs, Ken Manheimer klm@digicool.com From gward@python.net Tue Aug 7 01:51:42 2001 From: gward@python.net (Greg Ward) Date: Mon, 6 Aug 2001 20:51:42 -0400 Subject: [Python-Dev] Interesting article: "Python Is Middleware" Message-ID: <20010806205142.A4265@gerg.ca> This was linked to from last week's Linux Weekly News, so you may have already seen it. If not, it's an interesting read: http://www.tundraware.com/Technology/Python-Is-Middleware/Python-Is-Middleware.html Note in particular the last paragraph: """ A Little Warning I worry about one thing and one thing only in the Python world. It is something I have witnessed in every new technology I've ever seen. Python is dangerously close to becoming a victim of Feeping Creaturism - not so much In Fact, but rather in this community's mindset of forever wanting to fiddle one more feature into the language. This will Kill Python commercially if it happens. Too many variations on the core language theme make deployment and management of real systems too expensive. I, for one, would like to see a date picked for a permanent moratorium on the language proper, after which, only bug fixes and new modules could be added. After that date, language changes would have to be part of some new language ("Grail"?) which would owe no allegiance to Python at all. """ Greg -- Greg Ward - programmer-at-large gward@python.net http://starship.python.net/~gward/ "Question authority!" "Oh yeah? Says who?" From guido@zope.com Tue Aug 7 03:10:51 2001 From: guido@zope.com (Guido van Rossum) Date: Mon, 06 Aug 2001 22:10:51 -0400 Subject: [Python-Dev] Interesting article: "Python Is Middleware" In-Reply-To: Your message of "Mon, 06 Aug 2001 20:51:42 EDT." <20010806205142.A4265@gerg.ca> References: <20010806205142.A4265@gerg.ca> Message-ID: <200108070210.WAA24345@cj20424-a.reston1.va.home.com> > This was linked to from last week's Linux Weekly News, so you may have > already seen it. If not, it's an interesting read: > > http://www.tundraware.com/Technology/Python-Is-Middleware/Python-Is-Middleware.html Indeed. > Note in particular the last paragraph: [snip] He's worrying too much, and under the influence of the recent division flame war. Understandable, but that causes me to weigh his remark *less* than when it had come out of the blue. --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Tue Aug 7 03:43:53 2001 From: nas@python.ca (Neil Schemenauer) Date: Mon, 6 Aug 2001 19:43:53 -0700 Subject: [Python-Dev] Intel's Open Runtime Platform (ORP) Message-ID: <20010806194353.A30524@glacier.fnational.com> I just saw this while updating my Debian machine: Package: orp Description: Java VM and JIT from Intel Research Lab. ORP stands for Open Runtime Platform, which is an Intel(TM) Open Source research platform for investigating dynamic compilation and memory management technologies. The basic ORP system incorporates a fast code generating JIT (Just-In-Time) as well as an optimizing JIT. It also includes several GC (Garbage Collection) algorithms, ranging from a simple mark-sweep algorithm to an advanced train algorithm. Its available at http://intel.com/research/mrl/orp/ and is released under a BSD-like license. Neil From tim.one@home.com Tue Aug 7 05:57:03 2001 From: tim.one@home.com (Tim Peters) Date: Tue, 7 Aug 2001 00:57:03 -0400 Subject: [Python-Dev] Interesting article: "Python Is Middleware" In-Reply-To: <200108070210.WAA24345@cj20424-a.reston1.va.home.com> Message-ID: [From Tim Daneliuk's article] > I worry about one thing and one thing only in the Python world. It > is something I have witnessed in every new technology I've ever seen. > Python is dangerously close to becoming a victim of Feeping Creaturism - > not so much In Fact, but rather in this community's mindset of forever > wanting to fiddle one more feature into the language. With the exception of Unicode, I'm hard pressed to think of any major change in Python that wasn't discussed for years and years, most back in '92 already (incl. augmented assignments, nested scopes, liberalizing the iteration protocol, the type/class split, unifying ints and longs, and even generators). I've noted before that we're fast running out of this kind of thing (meaning big features discussed for a long time). Perhaps a formal notion of interface, some flavor of optional static typing, and major tweaks to the numeric model, will still occur, but that's "about it" for semantic features that have been discussed forever. What else is on the horizon? Apart from those, even the Py-in-the-Sky PEPs are either both minor and 100% backward compatible ("Docstring Format", "Add a Built-in Set Type", "String Interpolation"), or likely dead on arrival (even if nobody is rude enough to say so ). > This will Kill Python commercially if it happens. I suppose we should take the hard lesson from Visual Basic there . > Too many variations on the core language theme make deployment and > management of real systems too expensive. Unfortunately for practical guidance, how many is "too many" can't be judged without first determining how expensive is "too expensive" -- or, I suspect, vice versa too. Commercial interests in my eyes are defined by tangible commitment of resource to achieve what they want; if they hired someone full-time to ensure Python compatibility across releases, I bet they could get it. jaw-flapping-won't-solve-it-ly y'rs - tim From thomas@xs4all.net Tue Aug 7 09:10:51 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 7 Aug 2001 10:10:51 +0200 Subject: [Python-Dev] Interesting article: "Python Is Middleware" In-Reply-To: References: Message-ID: <20010807101051.T626@xs4all.nl> On Tue, Aug 07, 2001 at 12:57:03AM -0400, Tim Peters wrote: > I've noted before that we're fast running out of this kind of thing (meaning > big features discussed for a long time). Perhaps a formal notion of > interface, some flavor of optional static typing, and major tweaks to the > numeric model, will still occur, but that's "about it" for semantic features > that have been discussed forever. Let's not forget 'free threading', which is another one of those 'generally benificial' features that will break code. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From PyChecker Tue Aug 7 13:20:58 2001 From: PyChecker (Neal Norwitz) Date: Tue, 07 Aug 2001 08:20:58 -0400 Subject: [Python-Dev] ANN: PyChecker v0.8 Message-ID: <3B6FDD2A.85FB5B12@metaslash.com> A new version of PyChecker is available for your hacking pleasure. PyChecker is a tool for finding common bugs in python source code. It finds problems that are typically caught by a compiler for less dynamic languages, like C and C++. Comments, criticisms, new ideas, and other feedback is welcome. Change Log from 0.7.5 to 0.8: * Add check for accessing list as list[1,2], should be slice 1:2 * Add -J/maxargs to warn when using too many arguments * Add -K/maxlocals to warn when using too many local variables * Add -D/intdivide to warn when using integer division * Add -O/objattrs to warn when using object.attribute that doesn't exist * Add -M/reimportself to warn when a module imports itself * Add -E/unusednames to provide a list of unused names to ignore (default is: [ '_', 'empty', 'unused' ]) * Major refactoring of warn.py -> utils, msgs, Warning, CodeChecks * Fix spurious warnings for No class attribute for dynamic classes * Fix spurious warnings when using objects from blacklisted modules (Instantiating object with arguments, but no constructor, etc) * Fix spurious warnings when using lambdas * Fix spurious warning (No global Y) when using: from X import Y (problem in python 1.5/1.6 only) In case you missed 0.7.5, here's the change Log from 0.7 to 0.7.5: * Suppress warnings on a per module/function/class/method basis with new suppressions = {} in .pycheckrc * Suppress warnings by setting __pychecker__ in source code * Change long argument behaviour --arg sets arg to true, --no-arg sets arg to false (also works for warning suppression) * Add -U/--reuseattr check if function/class/method names are reused * Add -T/--argsused check for unused method/function arguments * Add -G/--selfused ignore if self is unused in a method (requires --argsused) * Add -q/--stdlib to disable warnings from the standard library * Add -o/--override warning when a method has different signature than one being overridden in a base class * Add -F/--rcfile to generate a .pycheckrc file * Fix checking files in standard library before local file * Fix spurious warning when using from X import Y and imports in methods (Module (m) re-imported) * Fix spurious warning when doing: from X import Y, Z * Fix spurious warning when deriving from Exception() and instantiating object with multiple arguments * Fix method argument checks when calling base class methods * Fix error msg to base constructors (msg count was wrong) * Fix access to builtin module attributes (e.g., sys.exc_value) generating 'No attribute warnings' * Fix tests (forgot to add : after line number in expected results) PyChecker is available on Source Forge: Web page: http://pychecker.sourceforge.net/ Project page: http://sourceforge.net/projects/pychecker/ Neal -- pychecker@metaslash.com From guido@zope.com Tue Aug 7 13:37:59 2001 From: guido@zope.com (Guido van Rossum) Date: Tue, 07 Aug 2001 08:37:59 -0400 Subject: [Python-Dev] PyChecker v0.8 Message-ID: <200108071237.IAA27282@cj20424-a.reston1.va.home.com> I just saw another announcement of PyChecker. It's definitely growing to become a very useful (if chatty :-) tool! How about we attempt to fix at least all the real bugs it detects in the standard library, such as the typos in urllib2.py ... Please help! --Guido van Rossum (home page: http://www.python.org/~guido/) From arigo@ulb.ac.be Tue Aug 7 16:15:44 2001 From: arigo@ulb.ac.be (Armin Rigo) Date: Tue, 7 Aug 2001 17:15:44 +0200 (CEST) Subject: [Python-Dev] PEP 252: class methods Message-ID: Hello, PEP 252: as described, class methods have no equivalent in C++ or Java; I would just like to point out that the Pascal dialect "Delphi" of Borland has this feature (called "virtual class methods"), which I personnally used a lot in one of my applications. Armin From jack@oratrix.nl Tue Aug 7 16:34:04 2001 From: jack@oratrix.nl (Jack Jansen) Date: Tue, 07 Aug 2001 17:34:04 +0200 Subject: [Python-Dev] HAVE_HYPOT still needed? Message-ID: <20010807153404.8D33D303181@snelboot.oratrix.nl> As I was removing outdated #ifdefs my eye fell on HAVE_HYPOT, and I have this very vague recollection that some old 68K Macs were the only system where hypot() wasn't in the C library. Does anyone know this for sure? If so we could remove the whole hypot() stuff... -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From jeremy@zope.com Tue Aug 7 16:56:10 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Tue, 7 Aug 2001 11:56:10 -0400 (EDT) Subject: [Python-Dev] Zope porting to jython In-Reply-To: <004c01c11cf7$7b7eec60$b75821c0@newmexico> References: <20010802103006.H1667@netthink.co.uk> <20010803161704.M626@xs4all.nl> <004c01c11cf7$7b7eec60$b75821c0@newmexico> Message-ID: <15216.3994.244904.425445@slothrop.digicool.com> >>>>> "SP" == Samuele Pedroni writes: SP> my personal position is to help them as all other jython users, SP> fixing incompatibilities with CPython and answering to SP> questions. Nothing special. I can imagine that one of the big SP> problem could be restricted execution (if Zope uses that), SP> admitted that any C code functionality can be rewritten in Java. Zope doesn't use restricted execution. In Zope 2.4, the RestrictedPython project uses the compiler package to generate safe bytecode from Python source code. I presume something similar could be achieved using jythonc, but don't know how hard that would be. SP> If the PythonLabs have an "official" position/opinion on this SP> and in particular on the last quite speculative message (from SP> Joseph Grace) in the thread, they may post something. I can say SP> something about technical aspects, but I don't feel that I SP> should channel/or invent PythonLabs opinions. I don't think there's an official PythonLabs opinion on this in general. If we have one its this: The project(s) -- including the just announced phabric effort -- are worthwhile. We are happy to give advice and make changes to help the projects where we can. I expect the same holds true for the rest of Zope Corp. If there are genuine CPython/Jython incompatibilities that hinder the project, there's no reason not to fix them. The project may also serve as a vehicle to further explore what metaobject protocols should be exported by Jython. Jeremy From tim@zope.com Tue Aug 7 17:07:34 2001 From: tim@zope.com (Tim Peters) Date: Tue, 7 Aug 2001 12:07:34 -0400 Subject: [Python-Dev] HAVE_HYPOT still needed? In-Reply-To: <20010807153404.8D33D303181@snelboot.oratrix.nl> Message-ID: [Jack Jansen] > As I was removing outdated #ifdefs my eye fell on HAVE_HYPOT, and > I have this very vague recollection that some old 68K Macs were the > only system where hypot() wasn't in the C library. The only comments about this in the codebase and in NEWS suggest that's so. > Does anyone know this for sure? Perhaps God -- the rest of us run on a finite number of machines . > If so we could remove the whole hypot() stuff... +1, including nuking Python/hypot.c. hypot() is a std C library function, so any platform without it is plain busted; and an alpha release is the only time we dare test that hypothesis. From tim@zope.com Tue Aug 7 17:50:12 2001 From: tim@zope.com (Tim Peters) Date: Tue, 7 Aug 2001 12:50:12 -0400 Subject: [Python-Dev] HAVE_HYPOT still needed? In-Reply-To: Message-ID: > If so we could remove the whole hypot() stuff... Maybe not! I don't have a copy of the C89 std here, and web searches turn up conflicting claims. My favorite C89 web site: http://www-ccs.ucsd.edu/c/ does not list hypot among the C89 functions. It's definitely required in C99. To play it safe, I expect we need to keep our own hypot after all. Would still be nice to simplify this Mac-ish part of pyport.h: #ifndef HAVE_HYPOT extern double hypot(double, double); #ifdef MWERKS_BEFORE_PRO4 #define hypot we_dont_want_faulty_hypot_decl #endif #endif #ifndef HAVE_HYPOT #ifdef __MWERKS__ #undef hypot #endif #endif While we're at it, shame Barry into cleaning this up : #ifdef __NeXT__ #ifdef __sparc__ /* * This works around a bug in the NS/Sparc 3.3 pre-release * limits.h header file. * 10-Feb-1995 bwarsaw@cnri.reston.va.us */ #undef LONG_MIN #define LONG_MIN (-LONG_MAX-1) #endif #endif From paulp@ActiveState.com Tue Aug 7 18:33:21 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 07 Aug 2001 10:33:21 -0700 Subject: [Python-Dev] Interesting article: "Python Is Middleware" References: <20010807101051.T626@xs4all.nl> Message-ID: <3B702661.DB18CD36@ActiveState.com> Thomas Wouters wrote: > > On Tue, Aug 07, 2001 at 12:57:03AM -0400, Tim Peters wrote: > > > I've noted before that we're fast running out of this kind of thing (meaning > > big features discussed for a long time). Perhaps a formal notion of > > interface, some flavor of optional static typing, and major tweaks to the > > numeric model, will still occur, but that's "about it" for semantic features > > that have been discussed forever. > > Let's not forget 'free threading', which is another one of those 'generally > benificial' features that will break code. Is that true? I thought that you would have to ASK to use free threading in a particular program. I'm discussing threading models with our internal Perl guys and I'm curious about one thing: what prevents a person from running two non-communicating Python interpreters in two threads? What kind of data is shared between interpeters? -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From guido@python.org Tue Aug 7 18:46:00 2001 From: guido@python.org (Guido van Rossum) Date: Tue, 07 Aug 2001 13:46:00 -0400 Subject: [Python-Dev] Free threading In-Reply-To: Your message of "Tue, 07 Aug 2001 10:33:21 PDT." <3B702661.DB18CD36@ActiveState.com> References: <20010807101051.T626@xs4all.nl> <3B702661.DB18CD36@ActiveState.com> Message-ID: <200108071746.f77Hk0I31236@odiug.digicool.com> [Thomas] > > Let's not forget 'free threading', which is another one of those 'generally > > benificial' features that will break code. [Paul] > Is that true? I thought that you would have to ASK to use free threading > in a particular program. Dunno -- in any case I'm not even sure how free threading would break code (as long as it shouldn't be allowed to cause core dumps, basic data structures ought to be threadsafe under free threading). > I'm discussing threading models with our internal Perl guys and I'm > curious about one thing: what prevents a person from running two > non-communicating Python interpreters in two threads? What kind of data > is shared between interpeters? Nothing stops you from doing that -- the intention is that it's possible. Immutable objects (like strings and ints) and all built-in type objects (including extension types) are shared between all interpreters. Interned strings are shared. Exceptions are shared (there may actually be a bug related to initializing exceptions that I still have to track down). Extension modules that have global state may cause additional confusion. Operating system resources like file descriptors are shared. --Guido van Rossum (home page: http://www.python.org/~guido/) From paulp@ActiveState.com Tue Aug 7 21:17:02 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 07 Aug 2001 13:17:02 -0700 Subject: [Python-Dev] Free threading References: <20010807101051.T626@xs4all.nl> <3B702661.DB18CD36@ActiveState.com> <200108071746.f77Hk0I31236@odiug.digicool.com> Message-ID: <3B704CBE.88F7FC7A@ActiveState.com> Guido van Rossum wrote: > > ... > > Nothing stops you from doing that -- the intention is that it's > possible. I didn't ask the question properly. You can run multiple interpreters in multiple threads but they share a GIL, right? I'm really asking whether multiple interpreters can be as independent of each other as any other bit of carefuly written re-entrant C code. > ... Immutable objects (like strings and ints) and all built-in > type objects (including extension types) are shared between all > interpreters. Interned strings are shared. Exceptions are shared > (there may actually be a bug related to initializing exceptions that I > still have to track down). Extension modules that have global state > may cause additional confusion. Operating system resources like file > descriptors are shared. It might be interesting if there were a module that allowed the Python programmer to spawn new interpreters on new threads from within Python and a way to share primitive data structures between them. My understanding is that Perl and Tcl both use this "independent threads" model which scales really well on SMP but seems to me to be pretty brutal to program for. Nevertheless I can see how it would be useful where all you care about is SMP performance. http://search.cpan.org/doc/ABERGMAN/threads-shared-0.02/shared.pm -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From guido@python.org Tue Aug 7 22:17:41 2001 From: guido@python.org (Guido van Rossum) Date: Tue, 07 Aug 2001 17:17:41 -0400 Subject: [Python-Dev] Free threading In-Reply-To: Your message of "Tue, 07 Aug 2001 13:17:02 PDT." <3B704CBE.88F7FC7A@ActiveState.com> References: <20010807101051.T626@xs4all.nl> <3B702661.DB18CD36@ActiveState.com> <200108071746.f77Hk0I31236@odiug.digicool.com> <3B704CBE.88F7FC7A@ActiveState.com> Message-ID: <200108072117.RAA29094@cj20424-a.reston1.va.home.com> > > Nothing stops you from doing that -- the intention is that it's > > possible. > > I didn't ask the question properly. You can run multiple interpreters in > multiple threads but they share a GIL, right? I'm really asking whether > multiple interpreters can be as independent of each other as any other > bit of carefuly written re-entrant C code. The share the GIL to protect the other shared resources. Since immutable objects can be shared, there's no way to have completely independent interpreters. Anyway, I don't see the point: if you want two 100% independent interpreters, you can get that easily by forking two separate processes. That's what separate processes are *for*... > > ... Immutable objects (like strings and ints) and all built-in > > type objects (including extension types) are shared between all > > interpreters. Interned strings are shared. Exceptions are shared > > (there may actually be a bug related to initializing exceptions that I > > still have to track down). Extension modules that have global state > > may cause additional confusion. Operating system resources like file > > descriptors are shared. > > It might be interesting if there were a module that allowed the Python > programmer to spawn new interpreters on new threads from within Python > and a way to share primitive data structures between them. My > understanding is that Perl and Tcl both use this "independent threads" > model which scales really well on SMP but seems to me to be pretty > brutal to program for. Nevertheless I can see how it would be useful > where all you care about is SMP performance. > > http://search.cpan.org/doc/ABERGMAN/threads-shared-0.02/shared.pm I bet the level of sharing exposed here can be implemented easily on top of the mmap module. Essentially, no matter what you do on an SMP machine, it seems you have to minimize sharing because it slows things down too much. Every solution somehow ends up making private data (to a process/thread) the default and makes you request sharing explicitly. In Python you won't be able to share *variables*, but it should be easy enough to share objects via some memory-based RPC protocol. That sounds to me a quicker road to an efficient SMP solution than trying to get rid of the GIL. --Guido van Rossum (home page: http://www.python.org/~guido/) From dan@sidhe.org Tue Aug 7 22:44:59 2001 From: dan@sidhe.org (Dan Sugalski) Date: Tue, 07 Aug 2001 17:44:59 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <200108072135.XAA08800@core.inf.ethz.ch> Message-ID: <5.1.0.14.0.20010807174100.023c1c58@tuatha.sidhe.org> At 11:35 PM 8/7/2001 +0200, Samuele Pedroni wrote: >The intention of my last meassage was not to shut down discussion with a >pessimistic argument, I was just trying to pin down the potential problems, >maybe the tone was a bit bitter... Nope, not at all--I do intend to reply. (The tone was fine, BTW) The Real World just got in the way, as it has this annoying tendency to do. :) As a quick response (this is from memory, so if it makes no sense don't be surprised), the current Parrot design's adequate for heavy OO, but could be a lot better. I'm working on that, and as soon as I've a better under-the-hood design I'll drop a note and everyone can savage it as appropriate. (And the more the better, honestly) FWIW I've thrown the first partial draft of the Parrot assembly language spec out to the wolves. URL and details available to the interested. Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From jack@oratrix.nl Tue Aug 7 23:07:10 2001 From: jack@oratrix.nl (Jack Jansen) Date: Wed, 08 Aug 2001 00:07:10 +0200 Subject: [Python-Dev] Using patch mgr for peer review - sensible or chicken? Message-ID: <20010807220715.E68E714B630@oratrix.oratrix.nl> Folks, whenever I make a change to something essential, and something that I'm not sure I understand for the full 100%, I tend to not check the mods in directly but in stead put a patch into the patch manager. This has happened recently with a setup.py and a configure.in patch, for instance. My idea was that whoever felt responsible for the file in question would see the patch email come by, have a look, and say "fine". But whereas some patches get looked at quickly some can sit there for days (or in some cases even months). This is a nuisance to me, as by the time someone looks at the patch my development version of the file has evolved again. Moreover, I get the impression that I'm the only person with checkin permissions that does this, and that the rest of the developer community just checks stuff in and counts on it being (a) correct or (b) quickly fixed by someone else. Am I too chicken and should I just check things in if I'm reasonably convinced that they're okay? -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From fdrake@acm.org Tue Aug 7 23:06:40 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 7 Aug 2001 18:06:40 -0400 (EDT) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <5.1.0.14.0.20010807174100.023c1c58@tuatha.sidhe.org> References: <200108072135.XAA08800@core.inf.ethz.ch> <5.1.0.14.0.20010807174100.023c1c58@tuatha.sidhe.org> Message-ID: <15216.26224.856076.269695@cj42289-a.reston1.va.home.com> Dan Sugalski writes: > FWIW I've thrown the first partial draft of the Parrot assembly language > spec out to the wolves. URL and details available to the interested. I would be surprised if there were many people on python-dev who aren't interested. I don't know how many aren't in the throws of wrestling with "The Real World" though. Could you make the URL available on python-dev? Failing that, I'd like a change to read what you've written! Thanks! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Tue Aug 7 23:17:24 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 7 Aug 2001 18:17:24 -0400 (EDT) Subject: [Python-Dev] Using patch mgr for peer review - sensible or chicken? In-Reply-To: <20010807220715.E68E714B630@oratrix.oratrix.nl> References: <20010807220715.E68E714B630@oratrix.oratrix.nl> Message-ID: <15216.26868.443867.736337@cj42289-a.reston1.va.home.com> Jack Jansen writes: > But whereas some patches get looked at quickly some can sit there for > days (or in some cases even months). This is a nuisance to me, as by > the time someone looks at the patch my development version of the file > has evolved again. Moreover, I get the impression that I'm the only > person with checkin permissions that does this, and that the rest of You're not the only one to do this; the weakrefs implementation went through several versions of patches before being checked in (but it did receive comments and improvements from others). I've seen others do this as well. If you think a patch needs to be handled quicker, perhaps a note to python-dev? The SF patch manager helps us not lose patches, but it doesn't do anything to help us set priorities, or give us extra time to look at patches when we're just plain swamped. > Am I too chicken and should I just check things in if I'm reasonably > convinced that they're okay? If nobody responds to what you think is a reasonable patch, check it in and we'll scream if it wasn't. ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From jeremy@zope.com Tue Aug 7 23:23:01 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Tue, 7 Aug 2001 18:23:01 -0400 (EDT) Subject: [Python-Dev] interesting page of links on VM design Message-ID: <15216.27205.438072.842721@slothrop.digicool.com> http://www.ce.chalmers.se/~feldt/ruby/ideas/rubyvm/ Found this link by a circuitous route (David Simmons' post to language-dev, leading to www.smallscript.net which discusses QKS work, includes link to related topics). May be interesting, related to Parrot thread. Jeremy From jack@oratrix.nl Wed Aug 8 01:45:05 2001 From: jack@oratrix.nl (Jack Jansen) Date: Wed, 08 Aug 2001 02:45:05 +0200 Subject: [Python-Dev] Using patch mgr for peer review - sensible or chicken? In-Reply-To: Message by "Fred L. Drake, Jr." , Tue, 7 Aug 2001 18:17:24 -0400 (EDT) , <15216.26868.443867.736337@cj42289-a.reston1.va.home.com> Message-ID: <20010808004510.3BC8B14B630@oratrix.oratrix.nl> Recently, "Fred L. Drake, Jr." said: > If you think a patch needs to be handled quicker, perhaps a note to > python-dev? The SF patch manager helps us not lose patches, but it > doesn't do anything to help us set priorities, or give us extra time > to look at patches when we're just plain swamped. Ok, someone please look at patch 448261. It'll make the build process on MacOSX a lot more palatable. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From tim.one@home.com Wed Aug 8 01:48:48 2001 From: tim.one@home.com (Tim Peters) Date: Tue, 7 Aug 2001 20:48:48 -0400 Subject: [Python-Dev] Using patch mgr for peer review - sensible or chicken? In-Reply-To: <20010807220715.E68E714B630@oratrix.oratrix.nl> Message-ID: [Jack Jansen] > whenever I make a change to something essential, and something that > I'm not sure I understand for the full 100%, I tend to not check the > mods in directly but instead put a patch into the patch manager. Good! That's the way it's supposed to work. > This has happened recently with a setup.py and a configure.in patch, > for instance. My idea was that whoever felt responsible for the file > in question would see the patch email come by, have a look, and say > "fine". That part doesn't work: you should assign it to whoever you think is best qualified to judge the parts you're not sure about. It won't necessarily see action any quicker, but it feels good to have someone to blame . > But whereas some patches get looked at quickly some can sit there for > days (or in some cases even months). Yup! I imagine it's even worse for people we don't know well, who submit odd patches nobody knows what to do with. > This is a nuisance to me, as by the time someone looks at the patch > my development version of the file has evolved again. That went without saying, you know. > Moreover, I get the impression that I'm the only person with checkin > permissions that does this, No, you're not. "The rule" is that an expert in an area, with an uncontroversial patch (and if they can't judge controversy in advance, they're not expert in that area!), should just check something in. Any other case should go thru review. For example, poor Martin hasn't had much luck getting action from PythonLabs on some optimization patches, and I've had to redo a UTF-16 patch from scratch (because it no longer applied) in the 5 weeks I've been waiting for MAL to look at it (*told* you it felt good to have someone to blame ). So everyone gets burned by this. It's relatively rare for the Python-Dev'ers, though, because by and large most of us stick to areas where "I'm an expert, buzz off" applies. > and that the rest of the developer community just checks stuff in and > counts on it being (a) correct or (b) quickly fixed by someone else. I *prefer* that approach when the only question is whether it will break something on Windows. I build Python every day anyway, so applying a patch just to test Windows is just more work; it's easier to fix it myself than apply a patch, explain what needs to change in the patch, then undo the patch again. > Am I too chicken I forgot -- are you Dutch or German ? > and should I just check things in if I'm reasonably convinced that > they're okay? Depends on how sure you are. If not "very", first assign your patch to someone specific and give them a chance to disappoint you. From paulp@ActiveState.com Wed Aug 8 01:56:27 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 07 Aug 2001 17:56:27 -0700 Subject: [Python-Dev] Free threading References: <20010807101051.T626@xs4all.nl> <3B702661.DB18CD36@ActiveState.com> <200108071746.f77Hk0I31236@odiug.digicool.com> <3B704CBE.88F7FC7A@ActiveState.com> <200108072117.RAA29094@cj20424-a.reston1.va.home.com> Message-ID: <3B708E3A.618374A2@ActiveState.com> Guido van Rossum wrote: > >... > > Anyway, I don't see the point: if you want two 100% independent > interpreters, you can get that easily by forking two separate > processes. That's what separate processes are *for*... I am 95% in agreement with you and that's what I told the Perl guys. That's why I see their "ithreads" solution as a little weird. But the other 5% is when you are running in an already-threaded environment and it is only the Python stuff that you want to be separate (for performance). e.g. Apache, IIS, Sendmail, COM etc. Also, Windows doesn't have "fork" so spawning a process means actually running an executable from disk AFAIK. That's somewhat inefficient. But I'll repeat that in the big picture I agree with you. The combination of shared-data threads and independent-data processes is good enough for most real-world apps. >... > I bet the level of sharing exposed here can be implemented easily on > top of the mmap module. Please excuse my ignorance: how do you do locking in an mmap solution? >... > That sounds to me a quicker road to an efficient SMP solution than > trying to get rid of the GIL. That's why I mentioned it. For whatever reason, the GIL project seems to only be doable by Greg S. (or he's the only one with interest?) so I thought a new approach might catch someone's interest. Does your mild statement of approval indicate that you see the benefit of having independent in-process interpreters? Or are you still thinking about using real processes? If the former, it still seems that the interpreter itself needs a little work so that it is safely reentrant in the sense of any other C library. There especially needs to be a way to NOT share a GIL and NOT share any other mutable data. If you are talking about real processes, then "all" that's needed is an efficient RPC mechanism. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From guido@python.org Wed Aug 8 05:09:55 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 08 Aug 2001 00:09:55 -0400 Subject: [Python-Dev] Free threading In-Reply-To: Your message of "Tue, 07 Aug 2001 17:56:27 PDT." <3B708E3A.618374A2@ActiveState.com> References: <20010807101051.T626@xs4all.nl> <3B702661.DB18CD36@ActiveState.com> <200108071746.f77Hk0I31236@odiug.digicool.com> <3B704CBE.88F7FC7A@ActiveState.com> <200108072117.RAA29094@cj20424-a.reston1.va.home.com> <3B708E3A.618374A2@ActiveState.com> Message-ID: <200108080409.AAA29799@cj20424-a.reston1.va.home.com> [me] > > Anyway, I don't see the point: if you want two 100% independent > > interpreters, you can get that easily by forking two separate > > processes. That's what separate processes are *for*... [PP] > I am 95% in agreement with you and that's what I told the Perl guys. > That's why I see their "ithreads" solution as a little weird. But the > other 5% is when you are running in an already-threaded environment and > it is only the Python stuff that you want to be separate (for > performance). e.g. Apache, IIS, Sendmail, COM etc. Yeah, but it's just not worth rewriting all of Python to get rid of the GIL for *that*. There are plenty of other ways to deal with it, including pre-forked longer-running processes -- and remember, it's only necessary if you need more than one CPU just for running your Python code! (E.g. Zope, which has a perfectly good Zope-specific solution if you need multiple CPUs: ZEO.) > Also, Windows doesn't have "fork" so spawning a process means actually > running an executable from disk AFAIK. That's somewhat inefficient. But > I'll repeat that in the big picture I agree with you. The combination of > shared-data threads and independent-data processes is good enough for > most real-world apps. > >... > > I bet the level of sharing exposed here can be implemented easily on > > top of the mmap module. > > Please excuse my ignorance: how do you do locking in an mmap solution? Good question. I've never used mmap myself. :-) I know Unix shared memory has locks and semaphores; the mmap module apparently doesn't (possibly because Windows has a different philosophy there). I would probably hide mmap and all other gory details in a module whose interface is similar to that of Queue. It could limit the data type for items stored in the to strings, or it could use marshal or pickle (or even XML :-) for more flexibility. > >... > > That sounds to me a quicker road to an efficient SMP solution than > > trying to get rid of the GIL. > > That's why I mentioned it. For whatever reason, the GIL project seems to > only be doable by Greg S. (or he's the only one with interest?) so I > thought a new approach might catch someone's interest. IMO there's nothing about Greg here. He did it once, that's all. :-) > Does your mild statement of approval indicate that you see the benefit > of having independent in-process interpreters? My approval of what? I like the independence of in-process interpreters as far as it goes (and I would like to see the exception bug fixed) but I don't think we need something more independent. I think I was arguing that for full independence of the GIL you should fork a new process (Windows notwithstanding -- hey, it's not *that* bad there unless you are trying to fork several per second, in which case I would question your sanity anyway :-). > Or are you still thinking about using real processes? Yes. > If the former, it still seems that the > interpreter itself needs a little work so that it is safely reentrant in > the sense of any other C library. There especially needs to be a way to > NOT share a GIL and NOT share any other mutable data. That would still be very hard to fix, and I don't think it's worth it. > If you are talking about real processes, then "all" that's needed is an > efficient RPC mechanism. That's right, that's what my Queue-like idea above intended to be. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Wed Aug 8 05:31:48 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 8 Aug 2001 00:31:48 -0400 Subject: [Python-Dev] Free threading In-Reply-To: <200108072117.RAA29094@cj20424-a.reston1.va.home.com> Message-ID: [Guido] > ... > Anyway, I don't see the point: if you want two 100% independent > interpreters, you can get that easily by forking two separate > processes. That's what separate processes are *for*... So what is it that multiple interpreters within a single process *are* for? I'm comfortable with threads, and comfortable with processes, but multiple interpreters not-really-sharing yet not-really-isolated seem mostly an opportunity to track down subtle bugs . From skip@pobox.com (Skip Montanaro) Wed Aug 8 05:36:06 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Tue, 7 Aug 2001 23:36:06 -0500 Subject: [Python-Dev] use co_flags to identify instruction set Message-ID: <15216.49590.601115.957779@beluga.mojam.com> All this talk of common backends for Python, Perl, and Ruby goaded me into revisiting the Rattlesnake stuff I laid aside several years ago. Assuming I ever get anywhere with it, it would be nice if code objects could distinguish instruction sets based on a flag in the PyCodeObject struct. The co_flags field seems to have some room and be more-or-less the right place for this stuff. I propose that it be changed from signed to unsigned int and that three bits be reserved to identify an instruction set. Eight possible instruction sets might seem a bit much, but I'd rather have a little room for growth. If we count the current instruction set, the Rattlesnake (register) stuff I've been playing with, and Armin Rigo's Psyco VM as distinct instruction sets, we've already used three of the possible eight. I'm still fiddling with a 1.5.2 code base and am currently only using one bit in co_flags to distinguish the instruction set, but I do use it to indirect through a two-element array of function pointers and call the appropriate variant of eval_code2 (now eval_frame). Just to whet peoples' appetites a bit... I'm struggling to get conditional opcodes working at the moment, but have had pretty good success eliding unnecessary loads and stores in straight blocks of code. Given this trivial function: def f(a): b = a + 4 c = a + b return c The Rattlesnake optimizer can convert it from >> 0 LOAD_FAST 0 (a) 3 LOAD_CONST 1 (4) 6 BINARY_ADD 7 STORE_FAST 1 (b) 10 LOAD_FAST 0 (a) 13 LOAD_FAST 1 (b) 16 BINARY_ADD 17 STORE_FAST 2 (c) 20 LOAD_FAST 2 (c) 23 RETURN_VALUE to >> 0 (0336) LOAD_CONST_REG %r4, 4 3 (0077) BINARY_ADD_REG b, a, %r4 7 (0077) BINARY_ADD_REG c, a, b 11 (0075) RETURN_VALUE_REG c Needless to say, I expect it to run a bit faster than the original code. Rattlesnake takes advantage of a property of frame objects that Tim pointed out to me a long time ago, namely that the frame's locals and its temporary stack space are contiguous and can just be treated as a single register file. In fact, the code above can just as easily be written as >> 0 (0336) LOAD_CONST_REG %r3, 4 3 (0077) BINARY_ADD_REG %r1, %r0, %r3 7 (0077) BINARY_ADD_REG %r2, %r0, %r1 11 (0075) RETURN_VALUE_REG %r2 Skip From guido@python.org Wed Aug 8 05:46:45 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 08 Aug 2001 00:46:45 -0400 Subject: [Python-Dev] Free threading In-Reply-To: Your message of "Wed, 08 Aug 2001 00:31:48 EDT." References: Message-ID: <200108080446.AAA31230@cj20424-a.reston1.va.home.com> > [Guido] > > ... > > Anyway, I don't see the point: if you want two 100% independent > > interpreters, you can get that easily by forking two separate > > processes. That's what separate processes are *for*... > > So what is it that multiple interpreters within a single process *are* for? > I'm comfortable with threads, and comfortable with processes, but multiple > interpreters not-really-sharing yet not-really-isolated seem mostly an > opportunity to track down subtle bugs . It was always feature number one requested by folks coming from Tcl, where creating multiple interpreters (threaded or not) is an acceptable pastime for people of good manners. But it seems to be wasted on the average Pythoneer. In Python, there's one good use I can think of: an application may embed Python and occasionally want to run a good chunk of Python code. If it needs each chunk of Python code to run completely independent from each other chunk, creating a new interpreter for each chunk is a good way to avoid that changes to e.g. sys.path or tabnanny's globals made by one chunk affects the next chunk. The hosting application might want the execution to share memory with *it*, so these interpreters have to live in the same process. This works well if the next chunk is only started after the previous chunk is done; but then using Py_Initialize() and Py_Finalize() is enough (creating one interpreter at a time). If the chunks need to overlap in time, creating multiple interpreters is the right solution. But apparently this habit has fallen in disuse -- from looking at the code I believe there's a serious problem with exceptions, where initializing the second interpreter messes up the identities of the exception classes stored in global variables by the first interpreter, and nobody has ever complained about this. Or maybe they are still using Python 1.5.2, which may not have the same problem. Or maybe this is the reason why PyApache has a reptabion of instability. :( --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Wed Aug 8 07:21:05 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 8 Aug 2001 02:21:05 -0400 Subject: [Python-Dev] Windows build broken, Unix dweeb should fix Message-ID: First ....\Python\bltinmodule.c(595) : warning C4013: 'S_ISDIR' undefined; assuming extern returning int on the last line here: /* Test for existence or directory. */ if (!stat(filename, &s)) { if (S_ISDIR(s.st_mode)) and later it fails to link cuz S_ISDIR doesn't exist. Now bltinmodule.c doesn't even include , so I don't know why we'd *expect* the new code to work (although I know why *I* would expect it to work -- see below). getpath.c does import sys/stat.h, then hides this stuff 100 lines later: #ifndef S_ISREG #define S_ISREG(x) (((x) & S_IFMT) == S_IFREG) #endif #ifndef S_ISDIR #define S_ISDIR(x) (((x) & S_IFMT) == S_IFDIR) #endif import.c doesn't even trust S_IFMT to work: #if defined(PYCC_VACPP) /* VisualAge C/C++ Failed to Define MountType Field in sys/stat.h */ #define S_IFMT (S_IFDIR|S_IFCHR|S_IFREG) #endif #ifndef S_ISDIR #define S_ISDIR(mode) (((mode) & S_IFMT) == S_IFDIR) #endif Out-of-synch preprocessor-trick duplication like this is Major Evil, and I don't want to see some bastard child of those duplicated into bltinmodule.c too. Windows doesn't define S_ISDIR, so at least that macro is needed. The only fellow doing an OS/2 port I know of now is Andrew MacIntyre, and I happen to know that, as of Tuesday , he neither uses nor has access to a VisualAge C compiler -- so no telling whether the PYCC_VACPP silliness is still needed. I'd drop it! Who wants to move this crap into pyport.h? I would, but I don't want to break the Unix builds, or research 57 varieties of Unix spellings. pyport.h already #includes on Windows, so for Windows it suffices to add just the macros there (assuming these macros are worth the trouble of defining at all -- S_ISDIR() is only used once in each of these files, and I don't happen to find S_ISDIR(x) any clearer than its expansion). From paulp@ActiveState.com Wed Aug 8 07:29:20 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 07 Aug 2001 23:29:20 -0700 Subject: [Python-Dev] Free threading References: <20010807101051.T626@xs4all.nl> <3B702661.DB18CD36@ActiveState.com> <200108071746.f77Hk0I31236@odiug.digicool.com> <3B704CBE.88F7FC7A@ActiveState.com> <200108072117.RAA29094@cj20424-a.reston1.va.home.com> <3B708E3A.618374A2@ActiveState.com> <200108080409.AAA29799@cj20424-a.reston1.va.home.com> Message-ID: <3B70DC3F.659E53A4@ActiveState.com> Guido van Rossum wrote: > >... > > Yeah, but it's just not worth rewriting all of Python to get rid of > the GIL for *that*. I'm not campaigning against the GIL. That's a separate issue. I'm asking why all of the Python interpreters in a particular process have to share *anything*. My naive view is that avoiding process-wide data is just good practice in any library. Of course processes that WANT to share data must share a GIL until Python is free-threaded. For my own understanding I am curious, do "unrelated" Python interpreters share state: a) because that makes the implementation of the Python interpreter easier or b) because it makes it easier to write extending and embedding apps (don't have to pass around a context object)? c) that's just how it evolved. > ... There are plenty of other ways to deal with it, > including pre-forked longer-running processes -- and remember, it's > only necessary if you need more than one CPU just for running your > Python code! (E.g. Zope, which has a perfectly good Zope-specific > solution if you need multiple CPUs: ZEO.) Yes, you can always work around it. But any non-reentrant libary can be worked around. The only difference is that Python is a language so the library provides an inherent mechanism for working around the issue. I admit that in most cases this is a strong enough practical argument. >... > Good question. I've never used mmap myself. :-) I know Unix shared > memory has locks and semaphores; the mmap module apparently doesn't > (possibly because Windows has a different philosophy there). So do you know of any fast, portable mechanism for cross-process locking? That's the only thing between us and a nice queue-and-marshal based IPC mechanism! -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From loewis@informatik.hu-berlin.de Wed Aug 8 08:13:32 2001 From: loewis@informatik.hu-berlin.de (Martin von Loewis) Date: Wed, 8 Aug 2001 09:13:32 +0200 (MEST) Subject: [Python-Dev] PEP comments, bugs, and patches Message-ID: <200108080713.JAA21365@pandora.informatik.hu-berlin.de> I just found that PEP 1 is confusing in its instructions of how to submit feedback on a PEP. In one place, it says # A better strategy is to encourage public feedback directly to the # PEP author, who collects and integrates the comments back into the # PEP. which suggest that proposed amendments to a PEP should be sent to the author. Later, it says # Please use the SourceForge bug manager[6] if you want to report # problems with PEPs, or better yet, the SourceForge patch manager[2] # for submitting patches to PEPs. which suggest that proposed amendments should be sent as patches to SF. Now, which of these procedures is preferred? Does the second mean that every PEP author needs permission to modify the patch database? Should every such patch be assigned to the PEP editor (i.e. Barry)? In case you wonder, the specific patch in question is #448841. Regards, Martin From martin@strakt.com Wed Aug 8 08:14:04 2001 From: martin@strakt.com (Martin Sjögren) Date: Wed, 8 Aug 2001 09:14:04 +0200 Subject: [Python-Dev] BEGIN_ALLOW_THREADS In-Reply-To: <20010723142157.B16665@strakt.com> References: <20010723142157.B16665@strakt.com> Message-ID: <20010808091404.A11990@strakt.com> On Mon, Jul 23, 2001 at 02:21:57PM +0200, Martin Sj=F6gren wrote: > Is there a reason the Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS > don't allow an argument specifying what variable to save the state to? = I > needed this myself so I wrote the following: >=20 > #ifdef WITH_THREAD > # define MY_BEGIN_ALLOW_THREADS(st) \ > { st =3D PyEval_SaveThread(); } > # define MY_END_ALLOW_THREADS(st) \ > { PyEval_RestoreThread(st); st =3D NULL; } > #else > # define MY_BEGIN_ALLOW_THREADS(st) > # define MY_END_ALLOW_THREADS(st) { st =3D NULL; } > #endif >=20 > It works just fine but has one drawback: Whenever Py_BEGIN_ALLOW_THREAD= S > changes, I have to change my macros too. >=20 > Wouldn't it be reasonable to supply two sets of macros, one that allows > exactly this, and one that does what Py_BEGIN_ALLOW_THREADS currently > does. If you're wondering why I'm replying to my own age-old-mail, it is becaus= e I noticed the following snippet in python-announce-list: """ Martin Sj=F6gren wondered why the Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros don't take an argument specifying a variable to save the thread state to. He said he needed this, but didn't explain why. """ Looking back I see that, no, I did not motivate that one bit :) Here's the scoop: One of the functions (say foo()) in my extension module calls another function (real_foo()) which might block since it does creepy things like I/O. Being a nice fella I surrounded the call with Py_BEGIN_ALLOW_THREAD= S and Py_END_ALLOW_THREADS so that other threads could jump in and do stuff. The problem is that real_foo() might call a previously defined callback (written in C) which in turn has to call a Python callback (what good is an extension module if you have to write your callbacks in C?), but when = I try to PyEval_CallObject() it - KABOOM! Segfault somewhere in the interpreter (on PyFrame_New() or something like that). After a lot of thinking (and a lot of help from python-list) I figured ou= t that the state I push away using Py_BEGIN_ALLOW_THREADS must be retrieved again before the callback is called, or PyFrame_New() won't like me. Problem is that that state variable is declared in Py_BEGIN_ALLOW_THREADS and there's no way to reach it (and it also includes a naughty { with the corresponding } in the Py_END_ALLOW_THREADS so it won't work to use them anyway). My solution: I wrote the two macros above. This works for me, but obviously I have to update my macros when the "real" macros in the Python distribution change. Okay, that's a more thorough explanation, I hope everybody are happy :) Martin --=20 Martin Sj=F6gren martin@strakt.com ICQ : 41245059 Phone: +46 (0)31 405242 Cell: +46 (0)739 169191 GPG key: http://www.strakt.com/~martin/gpg.html From loewis@informatik.hu-berlin.de Wed Aug 8 08:23:29 2001 From: loewis@informatik.hu-berlin.de (Martin von Loewis) Date: Wed, 8 Aug 2001 09:23:29 +0200 (MEST) Subject: [Python-Dev] Windows build broken, Unix dweeb should fix Message-ID: <200108080723.JAA21739@pandora.informatik.hu-berlin.de> > Who wants to move this crap into pyport.h? I committed the execfile patch, so I'll also gonna do that part. So just expect more breakage later :-) Regards, Martin From mal@lemburg.com Wed Aug 8 09:00:37 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 08 Aug 2001 10:00:37 +0200 Subject: [Python-Dev] PEP comments, bugs, and patches References: <200108080713.JAA21365@pandora.informatik.hu-berlin.de> Message-ID: <3B70F1A5.59F81B84@lemburg.com> Martin von Loewis wrote: > > I just found that PEP 1 is confusing in its instructions of how to > submit feedback on a PEP. In one place, it says > > # A better strategy is to encourage public feedback directly to the > # PEP author, who collects and integrates the comments back into the > # PEP. > > which suggest that proposed amendments to a PEP should be sent to the > author. Later, it says > > # Please use the SourceForge bug manager[6] if you want to report > # problems with PEPs, or better yet, the SourceForge patch manager[2] > # for submitting patches to PEPs. > > which suggest that proposed amendments should be sent as patches to > SF. Now, which of these procedures is preferred? Does the second mean > that every PEP author needs permission to modify the patch database? > Should every such patch be assigned to the PEP editor (i.e. Barry)? > > In case you wonder, the specific patch in question is #448841. Just to clarify this a bit: I told Bob (the contributor) to upload the patch to SF so that I don't lose track of it. I wasn't aware that there actually is a PEP patch guideline somewhere -- for me the SF patch manager simply helps managing patch reviews, so I thought it makes good sense to tell people to upload patches for my PEPs to SF too. I don't think we need to make it a requirement though -- other PEP authors may have different tracking methods. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From mal@lemburg.com Wed Aug 8 10:08:50 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 08 Aug 2001 11:08:50 +0200 Subject: [Python-Dev] Free threading References: Message-ID: <3B7101A2.1D961A1@lemburg.com> Tim Peters wrote: > > [Guido] > > ... > > Anyway, I don't see the point: if you want two 100% independent > > interpreters, you can get that easily by forking two separate > > processes. That's what separate processes are *for*... > > So what is it that multiple interpreters within a single process *are* for? This kind of application often comes up in ASP engines where a new Python interpreter is started for each request. > I'm comfortable with threads, and comfortable with processes, but multiple > interpreters not-really-sharing yet not-really-isolated seem mostly an > opportunity to track down subtle bugs . mod_python et al. stumbled into these often enough ... it would be a Good Thing for these engines to have *all* global data of the interpreter on the heap rather than in static C globals. Extensions would have to use the same logic though, so implementing this won't be easy and will probably take a long time to get standardized. Not sure whether it's worth the pain... -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From mwh@python.net Wed Aug 8 10:13:37 2001 From: mwh@python.net (Michael Hudson) Date: 08 Aug 2001 05:13:37 -0400 Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12 In-Reply-To: Michael Hudson's message of "30 Jul 2001 14:16:26 -0400" References: <200107271949.PAA27171@cj20424-a.reston1.va.home.com> <2mn15pwg56.fsf@starship.python.net> <200107281357.JAA30859@cj20424-a.reston1.va.home.com> <2m3d7emged.fsf@starship.python.net> Message-ID: <2mvgjy29tq.fsf@starship.python.net> Michael Hudson writes: > Here's a fairly short pre-PEP on the issue. If I haven't made any > gross editorial blunders, can Barry give it a number and check the > sucker in? And here's a diff between what I posted and what I now have: *** micro-pep1 Wed Aug 8 05:00:36 2001 --- micro-pep Wed Aug 8 05:08:16 2001 *************** *** 1,13 **** PEP: XXXX Title: Supporting __future__ statements in simulated shells ! Version: $Version:$ Author: Michael Hudson Status: Draft Type: Standards Track Requires: 0236 Created: 30-Jul-2001 Python-Version: 2.2 ! Post-History: Abstract --- 1,13 ---- PEP: XXXX Title: Supporting __future__ statements in simulated shells ! Version: 2 Author: Michael Hudson Status: Draft Type: Standards Track Requires: 0236 Created: 30-Jul-2001 Python-Version: 2.2 ! Post-History: 30-Jul-2001 Abstract *************** *** 24,39 **** Specification I propose adding a fourth, optional, "flags" argument to the ! builtin "compile" function. If this argument is omitted, there ! will be no change in behaviour from that of Python 2.1. If it is present it is expected to be an integer, representing various possible compile time options as a bitfield. The bitfields will have the same values as the PyCF_* flags #defined ! in Include/pythonrun.h (at the time of writing there are only two ! - PyCF_NESTED_SCOPES and PyCF_GENERATORS). These are currently ! not exposed to Python, so I propose adding them to codeop.py ! (because it's already here, basically). XXX Should the supplied flags be or-ed with the flags of the calling frame, or do we override them? I'm for the former, --- 24,39 ---- Specification I propose adding a fourth, optional, "flags" argument to the ! builtin "compile" function. If this argument is omitted, ! there will be no change in behaviour from that of Python 2.1. If it is present it is expected to be an integer, representing various possible compile time options as a bitfield. The bitfields will have the same values as the PyCF_* flags #defined ! in Include/pythonrun.h (at the time of writing there are three - ! PyCF_NESTED_SCOPES, PyCF_GENERATORS and PyCF_DIVISION). These are ! currently not exposed to Python, so I propose adding them to ! codeop.py (because it's already here, basically). XXX Should the supplied flags be or-ed with the flags of the calling frame, or do we override them? I'm for the former, *************** *** 77,87 **** statement is added. Such events will hopefully be very rare, so such a burden is unlikely to cause significant pain. Implementation ! None yet; none of the above should be at all hard. If this draft ! is well received, I'll upload a patch to sf "soon" and point to it ! here. Copyright --- 77,107 ---- statement is added. Such events will hopefully be very rare, so such a burden is unlikely to cause significant pain. + Issues + + Paul Prescod has reasonably complained about the choice of a + bitfield as the fourth argument to compile(), claiming it is + obscure and unpythonic. + + There is also the thought of Jython compatibility; because Jython + has the ability to access any Java object without the PyObject + cruft needed in CPython, Jython already has a Python-visible + CompilerFlags object which has a boolean attribute + "compiler_flags", and will presumably have one fairly soon called + "generators". This would be doable in CPython, but it would + require more hacking of deep magical bits of the interpreter and + require bumping the PYTHON_API_VERSION (OTOH, the division patch + just went in, so there can't be a freeze on the C API just + yet...). + Implementation ! I've uploaded a preliminary implementation as: ! ! http://sourceforge.net/tracker/?func=detail&atid=305470&aid=449043&group_id=5470 ! ! I need to add docs and possibly implment a friendlier interface to ! compile(). Copyright Basically I want to know if I have licence to go mucking around in the PyEval_ interface and objects, doing things like making PyCompilerFlags a PyObject and changing it's fields. Cheers, M. -- 59. In English every word can be verbed. Would that it were so in our programming languages. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From thomas@xs4all.net Wed Aug 8 12:29:33 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 8 Aug 2001 13:29:33 +0200 Subject: [Python-Dev] PyChecker v0.8 In-Reply-To: <200108071237.IAA27282@cj20424-a.reston1.va.home.com> References: <200108071237.IAA27282@cj20424-a.reston1.va.home.com> Message-ID: <20010808132933.X626@xs4all.nl> On Tue, Aug 07, 2001 at 08:37:59AM -0400, Guido van Rossum wrote: > I just saw another announcement of PyChecker. It's definitely growing > to become a very useful (if chatty :-) tool! How about we attempt to > fix at least all the real bugs it detects in the standard library, > such as the typos in urllib2.py ... Please help! Well, I'm not sure how you're running it, but Python 2.2 is breaking PyChecker :-) First off, I get a ton of these: /home/thomas/python/python-CVS/dist/src/Lib/copy.py:303: Object (l) has no attribute (append) where (from the code) 'l' is obviously a list. The problem is this: _BUILTINS_ATTRS = { types.StringType : dir(''), types.TypeType : dir(type(0)), types.ListType : dir([]), [etc] I guess that should now be def compat_dir(o): return dir(o) or dir(type(o)) # Or perhaps just a switch on python version _BUILTINS_ATTRS = { types.StringType : compat_dir(''), types.TypeType : compat_dir(type(0)), [etc] Looking at the error output, I suspect there's some more incompatibilities in there, including one that causes pychecker to stop processing a couple of modules. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From gmcm@hypernet.com Wed Aug 8 14:45:07 2001 From: gmcm@hypernet.com (Gordon McMillan) Date: Wed, 8 Aug 2001 09:45:07 -0400 Subject: [Python-Dev] Free threading In-Reply-To: <200108080409.AAA29799@cj20424-a.reston1.va.home.com> References: Your message of "Tue, 07 Aug 2001 17:56:27 PDT." <3B708E3A.618374A2@ActiveState.com> Message-ID: <3B710A23.4832.3BFB3AA@localhost> [Paul] > > Please excuse my ignorance: how do you do locking in an mmap > > solution? [Guido] > Good question. I've never used mmap myself. :-) I know Unix > shared memory has locks and semaphores; the mmap module > apparently doesn't (possibly because Windows has a different > philosophy there). Right, but Windows has named events and semaphores (cross process), so as long as your definition of "cross- platform" is Windows / *nix, you're in good shape . - Gordon From gmcm@hypernet.com Wed Aug 8 14:45:07 2001 From: gmcm@hypernet.com (Gordon McMillan) Date: Wed, 8 Aug 2001 09:45:07 -0400 Subject: [Python-Dev] Free threading In-Reply-To: <200108080446.AAA31230@cj20424-a.reston1.va.home.com> References: Your message of "Wed, 08 Aug 2001 00:31:48 EDT." Message-ID: <3B710A23.22358.3BFB205@localhost> [Guido] Py_Initialize ... Py_Finalize, Py_Initialize ... > But apparently this habit has fallen in disuse -- from looking at > the code I believe there's a serious problem with exceptions, > where initializing the second interpreter messes up the > identities of the exception classes stored in global variables by > the first interpreter, and nobody has ever complained about this. Last week on c.l.py some embedder complained about this. On the second interpreter, he got an exception saying "this constructor takes no args", but the line in question was a function (taking no args). - Gordon From guido@python.org Wed Aug 8 15:36:39 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 08 Aug 2001 10:36:39 -0400 Subject: [Python-Dev] Free threading In-Reply-To: Your message of "Tue, 07 Aug 2001 23:29:20 PDT." <3B70DC3F.659E53A4@ActiveState.com> References: <20010807101051.T626@xs4all.nl> <3B702661.DB18CD36@ActiveState.com> <200108071746.f77Hk0I31236@odiug.digicool.com> <3B704CBE.88F7FC7A@ActiveState.com> <200108072117.RAA29094@cj20424-a.reston1.va.home.com> <3B708E3A.618374A2@ActiveState.com> <200108080409.AAA29799@cj20424-a.reston1.va.home.com> <3B70DC3F.659E53A4@ActiveState.com> Message-ID: <200108081436.KAA32493@cj20424-a.reston1.va.home.com> > I'm asking why all of the Python interpreters in a particular process > have to share *anything*. My naive view is that avoiding process-wide > data is just good practice in any library. Of course processes that WANT > to share data must share a GIL until Python is free-threaded. They share anything that's implemented as a global resource. This includes the caches and free lists for small strings and ints, the 'intern' database, etc. > For my own understanding I am curious, do "unrelated" Python > interpreters share state: > > a) because that makes the implementation of the Python interpreter > easier or > b) because it makes it easier to write extending and embedding apps > (don't have to pass around a context object)? > c) that's just how it evolved. All of the above. Otherwise, *every* API would have to carry an "interp" argument around (as does Tcl/Tk). > > ... There are plenty of other ways to deal with it, > > including pre-forked longer-running processes -- and remember, it's > > only necessary if you need more than one CPU just for running your > > Python code! (E.g. Zope, which has a perfectly good Zope-specific > > solution if you need multiple CPUs: ZEO.) > > Yes, you can always work around it. But any non-reentrant libary can be > worked around. The only difference is that Python is a language so the > library provides an inherent mechanism for working around the issue. I > admit that in most cases this is a strong enough practical argument. Sorry, I don't understand what you mean. What does Python being a language buy you? > >... > > Good question. I've never used mmap myself. :-) I know Unix shared > > memory has locks and semaphores; the mmap module apparently doesn't > > (possibly because Windows has a different philosophy there). > > So do you know of any fast, portable mechanism for cross-process > locking? That's the only thing between us and a nice queue-and-marshal > based IPC mechanism! Sorry, don't look at me. On Unix, there's something referred to as "System V IPC". For Windows, look at the various Win32 API calls to create locks -- I believe you can name them and the names are global. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Aug 8 15:40:11 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 08 Aug 2001 10:40:11 -0400 Subject: [Python-Dev] BEGIN_ALLOW_THREADS In-Reply-To: Your message of "Wed, 08 Aug 2001 09:14:04 +0200." <20010808091404.A11990@strakt.com> References: <20010723142157.B16665@strakt.com> <20010808091404.A11990@strakt.com> Message-ID: <200108081440.KAA32514@cj20424-a.reston1.va.home.com> > One of the functions (say foo()) in my extension module calls another > function (real_foo()) which might block since it does creepy things like > I/O. Being a nice fella I surrounded the call with Py_BEGIN_ALLOW_THREADS > and Py_END_ALLOW_THREADS so that other threads could jump in and do stuff. > > The problem is that real_foo() might call a previously defined callback > (written in C) which in turn has to call a Python callback (what good is > an extension module if you have to write your callbacks in C?), but when I > try to PyEval_CallObject() it - KABOOM! Segfault somewhere in the > interpreter (on PyFrame_New() or something like that). > > After a lot of thinking (and a lot of help from python-list) I figured out > that the state I push away using Py_BEGIN_ALLOW_THREADS must be retrieved > again before the callback is called, or PyFrame_New() won't like me. > Problem is that that state variable is declared in Py_BEGIN_ALLOW_THREADS > and there's no way to reach it (and it also includes a naughty { with the > corresponding } in the Py_END_ALLOW_THREADS so it won't work to use them > anyway). > > My solution: I wrote the two macros above. This works for me, but > obviously I have to update my macros when the "real" macros in the Python > distribution change. Instead of asking us to add more macros, I recommend that you use the lower-level C API that is used by the macros. There's a bunch of ways to save and restore the state. The macros are just shortcuts (and a handy way to make all the code disappear when threads don't exist). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Aug 8 15:47:37 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 08 Aug 2001 10:47:37 -0400 Subject: [Python-Dev] PEP comments, bugs, and patches In-Reply-To: Your message of "Wed, 08 Aug 2001 09:13:32 +0200." <200108080713.JAA21365@pandora.informatik.hu-berlin.de> References: <200108080713.JAA21365@pandora.informatik.hu-berlin.de> Message-ID: <200108081447.KAA32556@cj20424-a.reston1.va.home.com> > I just found that PEP 1 is confusing in its instructions of how to > submit feedback on a PEP. In one place, it says > > # A better strategy is to encourage public feedback directly to the > # PEP author, who collects and integrates the comments back into the > # PEP. > > which suggest that proposed amendments to a PEP should be sent to the > author. Later, it says > > # Please use the SourceForge bug manager[6] if you want to report > # problems with PEPs, or better yet, the SourceForge patch manager[2] > # for submitting patches to PEPs. > > which suggest that proposed amendments should be sent as patches to > SF. Now, which of these procedures is preferred? Does the second mean > that every PEP author needs permission to modify the patch database? > Should every such patch be assigned to the PEP editor (i.e. Barry)? What's best may depend on the stage a PEP is in. In the initial stages of a PEP, when the author is actively working on it, it may be more efficient to write the author than go through the (somewhat heavier) SF mechanisms. But later, when editing is less frantic, I see the advantages of the SF tracker. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Aug 8 16:04:03 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 08 Aug 2001 11:04:03 -0400 Subject: [Python-Dev] Free threading In-Reply-To: Your message of "Wed, 08 Aug 2001 09:45:07 EDT." <3B710A23.22358.3BFB205@localhost> References: Your message of "Wed, 08 Aug 2001 00:31:48 EDT." <3B710A23.22358.3BFB205@localhost> Message-ID: <200108081504.LAA32734@cj20424-a.reston1.va.home.com> > Last week on c.l.py some embedder complained about this. > On the second interpreter, he got an exception saying "this > constructor takes no args", but the line in question was a > function (taking no args). And nobody filed a bug report?!?!?! OK, I'm filing one now. http://sourceforge.net/tracker/index.php?func=detail&aid=449151&group_id=5470&atid=105470 --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Aug 8 16:31:29 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 08 Aug 2001 11:31:29 -0400 Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12 In-Reply-To: Your message of "08 Aug 2001 05:13:37 EDT." <2mvgjy29tq.fsf@starship.python.net> References: <200107271949.PAA27171@cj20424-a.reston1.va.home.com> <2mn15pwg56.fsf@starship.python.net> <200107281357.JAA30859@cj20424-a.reston1.va.home.com> <2m3d7emged.fsf@starship.python.net> <2mvgjy29tq.fsf@starship.python.net> Message-ID: <200108081531.LAA00619@cj20424-a.reston1.va.home.com> Hi Michael, Barry's on vacation, so I've done the honors. It's checked in as PEP 264. I like this a lot -- I hope we can add this to 2.2a2 (which was due today but is postponed for at least a week). I think the integral flags to compile() are fine, and 'or' is indeed the right thing. After reading the PEP, I understand the refactoring that I complained about in a comment on the patch. Good job! --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@loewis.home.cs.tu-berlin.de Wed Aug 8 16:55:30 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Wed, 8 Aug 2001 17:55:30 +0200 Subject: [Python-Dev] Free threading Message-ID: <200108081555.f78FtUx01194@mira.informatik.hu-berlin.de> > All of the above. Otherwise, *every* API would have to carry an > "interp" argument around (as does Tcl/Tk). Not necessarily. The interp argument could be stored in thread-local storage. Of course, using TLS opens another can of worms. Regards, Martin From Samuele Pedroni Wed Aug 8 17:21:23 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Wed, 8 Aug 2001 18:21:23 +0200 (MET DST) Subject: [Python-Dev] use co_flags to identify instruction set Message-ID: <200108081621.SAA03060@core.inf.ethz.ch> > All this talk of common backends for Python, Perl, and Ruby goaded me into > revisiting the Rattlesnake stuff I laid aside several years ago. Assuming I > ever get anywhere with it, it would be nice if code objects could > distinguish instruction sets based on a flag in the PyCodeObject struct. > The co_flags field seems to have some room and be more-or-less the right > place for this stuff. I'm wondering whether this is a good idea, there are some tools (at least inspect) in the std lib that depend on code objects to just contain the old good pvm bytecode, PyChecker also needs that (but we could have a flag to avoid generating strange code for its use...) As marginal note: for this reason they fail miserably under Jython, but that's clearly Jython fault. My point is that the above examples are perfectly meaningful and legitimate, and it is a good tradition and practice (Self, Smalltalk) even if you're seeking for speed, to present and accept code in a uniform manner, trough just one bytecode fortmat. Yes this way you waste memory, but ir is just the typical trade-off flexibility and speed vs. memory, if you have few memory you will simply disable your optimizer. A different point is to define and always use a new bytecode format. Or to banish code that rely on bytecode format for introspection ... I imagine that at some point someone will regret that. We should consider that AFAIK Psyco don't really produce in one shot a new stream of bytecode and on the other hand uses and keeps around a fairly complicated graph structure together with the chunks of bytecode (that's why it can probably be applied only to hot spots). I wonder if that fits in the old PyCodeObject model anyway. regards, Samuele Pedroni. From dan@sidhe.org Wed Aug 8 17:29:51 2001 From: dan@sidhe.org (Dan Sugalski) Date: Wed, 08 Aug 2001 12:29:51 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <15216.26224.856076.269695@cj42289-a.reston1.va.home.com> References: <5.1.0.14.0.20010807174100.023c1c58@tuatha.sidhe.org> <200108072135.XAA08800@core.inf.ethz.ch> <5.1.0.14.0.20010807174100.023c1c58@tuatha.sidhe.org> Message-ID: <5.1.0.14.0.20010808122734.01c04648@tuatha.sidhe.org> At 06:06 PM 8/7/2001 -0400, Fred L. Drake, Jr. wrote: >Dan Sugalski writes: > > FWIW I've thrown the first partial draft of the Parrot assembly language > > spec out to the wolves. URL and details available to the interested. > > I would be surprised if there were many people on python-dev who >aren't interested. I don't know how many aren't in the throws of >wrestling with "The Real World" though. Okay, the docs in progress live in: http://www.sidhe.org/~dan/perl in this case we're looking specifically at the perl_assembly.html file. (The original is the .txt file, which is really in POD format, not plain text, but close enough) Some of the other docs that are in there might be of some interest. If you can't get in, drop me a note--my ISP has threatened inbound port 80 blocking courtesy of the CodeRed nonsense. Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From tim.one@home.com Wed Aug 8 19:13:56 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 8 Aug 2001 14:13:56 -0400 Subject: [Python-Dev] FW: if-else with '|' operator - beginner's question/problem Message-ID: What a fascinating newbie confusion! It surprised and tickled me. FYI, in Icon this kind of thing "just works". e1 | e2 | ... | en is "a generator" there too, and acts like the Python def g(): for x in e1: yield x for x in e2: yield x ... for x in en: yield en Then the expression y1 == y2 generates the cross-product of the y1 and y2 sequences, succeeding in the end iff some pair of generated values compares equal. No, I'm not suggesting Python do that too . It's more in the way of noting that no matter *how* odd a question looks to you, some peoples' brains are so wired that it's hard for them to view it otherwise. finally-understanding-vi-users-ly y'rs - tim -----Original Message----- From: python-list-admin@python.org Sent: Wednesday, August 08, 2001 1:26 PM To: python-list@python.org Subject: if-else with '|' operator - beginner's question/problem Hi there, I wonder if someone could possibly tell me what is wrong with the following statement. I'm extremely embarrased to ask but here goes... >>> if (fname[1] == 'a'|'e'|'i'|'o'|'u'): vowel='true' Traceback (innermost last): File "", line 1, in ? if (fname[1] == 'a'|'e'|'i'|'o'|'u'): TypeError: bad operand type(s) for | --- fname[1] gives 'e' Thanks very much in advance! Lee -- http://mail.python.org/mailman/listinfo/python-list From tim.one@home.com Wed Aug 8 20:11:22 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 8 Aug 2001 15:11:22 -0400 Subject: [Python-Dev] PEP 264 (was RE: [Python-checkins] CVS: python/nondist/peps pep-0264.txt,NONE,1.1 pep-0000.txt,1.113,1.114) In-Reply-To: Message-ID: [Guido, checking in Michael Hudson's "Future statements in simulated shells". PEP ] > ... > Paul Prescod has reasonably complained about the choice of a > bitfield as the fourth argument to compile(), claiming it is > obscure and unpythonic. I've got a happy idea! __future__.py is a real std module, and e.g. __future__.nested_scopes __future__.generators __future__.division are honest-to-gosh Python objects, of class __future__._Feature. That's the perfect place to store feature-specific magic flag bits. So, e.g., I'm picturing compile(s1, s2, s3, __future__.generators.getflag()) and flags = 0 for fname in list_of_feature_names: flags |= getattr(__future__, fname).getflag() compile(s1, s2, s3, flags) etc. For that matter, list-of-feature-strings -> flag_bits could be added as a utility function to __future__.py. It's Python -- there's nothing you can't do with this in 5 seconds . From skip@pobox.com (Skip Montanaro) Wed Aug 8 21:29:55 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 8 Aug 2001 15:29:55 -0500 Subject: [Python-Dev] "not an acceptable base type"? Message-ID: <15217.41283.463143.160922@beluga.mojam.com> Are strings not yet subclassable or do I just need to "cvs up"? >>> class string(str): ... pass ... Traceback (most recent call last): File "", line 1, in ? TypeError: type 'str' is not an acceptable base type >>> "".__class__ == str 1 Thx, Skip From guido@python.org Wed Aug 8 21:33:53 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 08 Aug 2001 16:33:53 -0400 Subject: [Python-Dev] "not an acceptable base type"? In-Reply-To: Your message of "Wed, 08 Aug 2001 15:29:55 CDT." <15217.41283.463143.160922@beluga.mojam.com> References: <15217.41283.463143.160922@beluga.mojam.com> Message-ID: <200108082033.QAA05704@cj20424-a.reston1.va.home.com> > Are strings not yet subclassable or do I just need to "cvs up"? Be patient. We'll do this, but it's not yet done. For now, you can only subclass type, object, list, and dictionary. --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@loewis.home.cs.tu-berlin.de Wed Aug 8 21:34:47 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Wed, 8 Aug 2001 22:34:47 +0200 Subject: [Python-Dev] Any objections to adding threading.Timer? Message-ID: <200108082034.f78KYlB16985@mira.informatik.hu-berlin.de> In patch https://sourceforge.net/tracker/index.php?func=detail&aid=428326&group_id=5470&atid=305470 a new threading.Timer class is proposed. Since the patch was submitted two months ago, nobody seemed to care enough to comment on it. I like it, and I'll commit the patch unless I hear otherwise. Regards, Martin From skip@pobox.com (Skip Montanaro) Wed Aug 8 21:43:31 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 8 Aug 2001 15:43:31 -0500 Subject: [Python-Dev] "not an acceptable base type"? In-Reply-To: <200108082033.QAA05704@cj20424-a.reston1.va.home.com> References: <15217.41283.463143.160922@beluga.mojam.com> <200108082033.QAA05704@cj20424-a.reston1.va.home.com> Message-ID: <15217.42099.307124.333865@beluga.mojam.com> >> Are strings not yet subclassable or do I just need to "cvs up"? Guido> Be patient. We'll do this, but it's not yet done. For now, you Guido> can only subclass type, object, list, and dictionary. Ah, thanks. S From jeremy@zope.com Wed Aug 8 21:49:02 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 8 Aug 2001 16:49:02 -0400 (EDT) Subject: [Python-Dev] Any objections to adding threading.Timer? In-Reply-To: <200108082034.f78KYlB16985@mira.informatik.hu-berlin.de> References: <200108082034.f78KYlB16985@mira.informatik.hu-berlin.de> Message-ID: <15217.42430.308688.74304@slothrop.digicool.com> The idea sounded good in principle, but I didn't have time to look at it. I'll be happy with whatever you decide. Jeremy From mwh@python.net Wed Aug 8 22:54:50 2001 From: mwh@python.net (Michael Hudson) Date: 08 Aug 2001 17:54:50 -0400 Subject: [Python-Dev] PEP 264 (was RE: [Python-checkins] CVS: python/nondist/peps pep-0264.txt,NONE,1.1 pep-0000.txt,1.113,1.114) In-Reply-To: "Tim Peters"'s message of "Wed, 8 Aug 2001 15:11:22 -0400" References: Message-ID: <2m8zgudxp1.fsf@starship.python.net> Instant response, without that tedious-prone-to-problem-spotting thinking: nice idea! "Tim Peters" writes: > [Guido, checking in > Michael Hudson's "Future statements in simulated shells". > PEP > ] > > ... > > Paul Prescod has reasonably complained about the choice of a > > bitfield as the fourth argument to compile(), claiming it is > > obscure and unpythonic. > > I've got a happy idea! __future__.py is a real std module, and e.g. > > __future__.nested_scopes > __future__.generators > __future__.division > > are honest-to-gosh Python objects, of class __future__._Feature. That's the > perfect place to store feature-specific magic flag bits. So, e.g., I'm > picturing > > compile(s1, s2, s3, __future__.generators.getflag()) Or just __future__.generators.compile_flag presumably? > and > > flags = 0 > for fname in list_of_feature_names: > flags |= getattr(__future__, fname).getflag() > compile(s1, s2, s3, flags) > > etc. For that matter, list-of-feature-strings -> flag_bits could be > added as a utility function to __future__.py. It's Python -- > there's nothing you can't do with this in 5 seconds . "list_of_feature_names" might as well be spelt "__future__.__all__", I'd have thought. There's still the icky issue of PyEval_MergeCompilerFlags to consider. The fact that each future statement has two bits assigned to in - the PyCF_* ones in pythonrun.h and the CO_* ones in compile.h that allow __builtin__.compile to inherit the behaviour of surrounding code - seems nasty. We could just mandate that CO_FOO == PyCF_FOO << 12 Or add another flag to code objects. Or do something else. The possibility of fiddling with __future__.py had completely escaped my mind - so thanks for pointing me at that! Cheers, M. -- If you don't use emacs, you're a pathetic, mewling, masochistic weakling and I can't be bothered to convert you. -- Ron Echeverri From aahz@rahul.net Wed Aug 8 23:23:17 2001 From: aahz@rahul.net (Aahz Maruch) Date: Wed, 8 Aug 2001 15:23:17 -0700 (PDT) Subject: [Python-Dev] Free threading In-Reply-To: <200108081555.f78FtUx01194@mira.informatik.hu-berlin.de> from "Martin v. Loewis" at Aug 08, 2001 05:55:30 PM Message-ID: <20010808222318.1D66699C81@waltz.rahul.net> Martin v. Loewis wrote: >Guido: >> >> All of the above. Otherwise, *every* API would have to carry an >> "interp" argument around (as does Tcl/Tk). > > Not necessarily. The interp argument could be stored in thread-local > storage. Of course, using TLS opens another can of worms. We need thread-local storage, anyway. When I'm a little farther into my work on Decimal, I'm going to take a hard look at this because context needs to be global on a per-thread basis... -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From guido@python.org Wed Aug 8 23:30:54 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 08 Aug 2001 18:30:54 -0400 Subject: [Python-Dev] Free threading In-Reply-To: Your message of "Wed, 08 Aug 2001 15:23:17 PDT." <20010808222318.1D66699C81@waltz.rahul.net> References: <20010808222318.1D66699C81@waltz.rahul.net> Message-ID: <200108082230.SAA09334@cj20424-a.reston1.va.home.com> > We need thread-local storage, anyway. When I'm a little farther into my > work on Decimal, I'm going to take a hard look at this because context > needs to be global on a per-thread basis... (Thick French accent:) "We already got one!" In the C API, use PyThreadState_GetDict(): it returns a per-thread dictionary. The convention is that you pick a key unique to your extension module (e.g. your full extension module name) and then store your per-thread data under this key. In Python, you can store instance variables on the current thread object -- use threading.currentThread() to access it. --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Thu Aug 9 00:15:03 2001 From: nas@python.ca (Neil Schemenauer) Date: Wed, 8 Aug 2001 16:15:03 -0700 Subject: [Python-Dev] "not an acceptable base type"? In-Reply-To: <200108082033.QAA05704@cj20424-a.reston1.va.home.com>; from guido@python.org on Wed, Aug 08, 2001 at 04:33:53PM -0400 References: <15217.41283.463143.160922@beluga.mojam.com> <200108082033.QAA05704@cj20424-a.reston1.va.home.com> Message-ID: <20010808161503.B2974@glacier.fnational.com> Guido van Rossum wrote: > > Are strings not yet subclassable or do I just need to "cvs up"? > > Be patient. We'll do this, but it's not yet done. For now, you can > only subclass type, object, list, and dictionary. Is there something tricky about subclassing strings (and ints, floats, ...)? I've considered spending some time trying to make other types subclassable. If each python-dev'er takes one it shouldn't take too long. :-) Neil From guido@python.org Thu Aug 9 00:40:03 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 08 Aug 2001 19:40:03 -0400 Subject: [Python-Dev] "not an acceptable base type"? In-Reply-To: Your message of "Wed, 08 Aug 2001 16:15:03 PDT." <20010808161503.B2974@glacier.fnational.com> References: <15217.41283.463143.160922@beluga.mojam.com> <200108082033.QAA05704@cj20424-a.reston1.va.home.com> <20010808161503.B2974@glacier.fnational.com> Message-ID: <200108082340.TAA12043@cj20424-a.reston1.va.home.com> > Is there something tricky about subclassing strings (and ints, floats, > ...)? I've considered spending some time trying to make other types > subclassable. If each python-dev'er takes one it shouldn't take too > long. :-) The tricky thing is that you have to do most of the initialization in tp_new (not tp_init), and yet you must allow a subclass to override tp_alloc and tp_free. Immutable types and types with their own free list or caching definitely are trickier than the ones I did so far... --Guido van Rossum (home page: http://www.python.org/~guido/) From simon@netthink.co.uk Thu Aug 9 03:44:32 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Wed, 8 Aug 2001 19:44:32 -0700 Subject: [Python-Dev] Parrot -- let's move this thread. In-Reply-To: <15216.26224.856076.269695@cj42289-a.reston1.va.home.com> Message-ID: <20010808194432.B782@netthink.co.uk> On Tue, Aug 07, 2001 at 06:06:40PM -0400, Fred L. Drake, Jr. wrote: > I would be surprised if there were many people on python-dev who > aren't interested. On the other hand, python-dev is for the development of Python, and (for now at least, cackle cackle) Parrot isn't Python. Can we move this thread to language-dev@netthink.co.uk, a generic list for the implementors of various scripting languages. This is beneficial for several reasons; primarily because it allows python-dev to get back to work, but also because we have a lot of very Interesting People on language-dev who can help scrutinise the ideas we're discussing. Announcement of language-dev: http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2001-07/msg00030.html See also http://aspn.activestate.com/ASPN/Mail/Browse/Threaded/language-dev Simon From DavidA@ActiveState.com Thu Aug 9 01:20:29 2001 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 08 Aug 2001 17:20:29 -0700 Subject: [Python-Dev] Parrot -- let's move this thread. References: <20010808194432.B782@netthink.co.uk> Message-ID: <3B71D74D.BCF20B86@ActiveState.com> Simon Cozens wrote: > implementors of various scripting languages. This is beneficial for > several reasons; primarily because it allows python-dev to get back > to work, but also because we have a lot of very Interesting People > on language-dev who can help scrutinise the ideas we're discussing. I especially found the post by David Simmons fascinating: From DavidA@ActiveState.com Thu Aug 9 01:21:53 2001 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 08 Aug 2001 17:21:53 -0700 Subject: [Python-Dev] Parrot -- let's move this thread. References: <20010808194432.B782@netthink.co.uk> Message-ID: <3B71D7A1.2FDF099A@ActiveState.com> Sigh -- hit the wrong key. I meant to say: I especially found the post by David Simmons interesting: http://aspn.activestate.com/ASPN/Mail/Message/language-dev/722505 --david From paulp@ActiveState.com Thu Aug 9 02:11:41 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 08 Aug 2001 18:11:41 -0700 Subject: [Python-Dev] Common runtime sociology Message-ID: <3B71E34D.7A2DD001@ActiveState.com> It is my gut feeling that the only way to get to a common runtime (if it is possible at all) is to take the development out of the hands of any language group and put it in a trusted third party's hands. If any language group feels like its concerns are going to get second-class attention, they'll go back and focus on their own implementation. So that's just my feeling...if the goal of some people is really to have the different languages share implementation then the technical issues are secondary to the project organization and leadership. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From dan@sidhe.org Thu Aug 9 02:57:50 2001 From: dan@sidhe.org (Dan Sugalski) Date: Wed, 08 Aug 2001 21:57:50 -0400 Subject: [Python-Dev] Common runtime sociology In-Reply-To: <3B71E34D.7A2DD001@ActiveState.com> Message-ID: <5.1.0.14.0.20010808212559.0223fb60@tuatha.sidhe.org> At 06:11 PM 8/8/2001 -0700, Paul Prescod wrote: >It is my gut feeling that the only way to get to a common runtime (if it >is possible at all) is to take the development out of the hands of any >language group and put it in a trusted third party's hands. If any >language group feels like its concerns are going to get second-class >attention, they'll go back and focus on their own implementation. I think it's possible for things to be done otherwise, but it puts a heavy burden on the implementation group to be actively engaged with the language groups and be as impartial as can be managed. You almost need to take the implementation group that starts this and pry them loose to be an entity in and of themselves, separate from the original language group. (FWIW, this is part of my Master Plan. Dunno if it's part of Simon's) It's certainly the more difficult way to go than to start with an impartial third party. (Of course, then there's the problem of *finding* an impartial third party for something like this) For the record, if something like this does go through, I'm all for it and'll bow out of the way. (Any resemblance to dropped hot rocks is purely coincidental ;) As for the common runtime, that's by far the least of the problems. I mean, we all manage with C, of all things, running on hardware that's not in the least forgiving of dynamic languages. We'd be hard pressed to find a worse environment to run on short of dragging out one of the old LISP or Forth systems... Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From skip@pobox.com (Skip Montanaro) Thu Aug 9 03:41:04 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 8 Aug 2001 21:41:04 -0500 Subject: [Python-Dev] opcode dispatch - switch vs. function pointers Message-ID: <15217.63552.143782.572894@beluga.mojam.com> Popped over to the language-dev archive at ActiveState and came across this interesting post about opcode dispatch: http://aspn.activestate.com/ASPN/Mail/Message/language-dev/724204 I think I'm going to experiment a little with dispatch via function pointers in the Rattlesnake (once I get something that more-or-less works). Skip From tim.one@home.com Thu Aug 9 04:08:56 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 8 Aug 2001 23:08:56 -0400 Subject: [Python-Dev] Common runtime sociology In-Reply-To: <3B71E34D.7A2DD001@ActiveState.com> Message-ID: [Paul Prescod] > It is my gut feeling that the only way to get to a common runtime (if it > is possible at all) is to take the development out of the hands of any > language group and put it in a trusted third party's hands. Na. There are two ways it can happen: 1. A well-funded project with this as one of its primary goals. 2. Any sort of project, funded or not, whose lead(s) is(are) obsessed with making it happen. #1 may be close to MS .NET's approach, proving in part that the "common" in "common runtime" is still a matter of manipulable definition . Put in many hands, it's the Industry Consortia model, famous for producing large publicity/result ratios. If it's just an afterthought, or a "nice to have", or even a "it's one of the last things we'll give up -- but, ya, in the end it's expendable", it really has slim chance of surviving the difficulties. #2 is the best shot. Alas, few people can *decide* to become an insane visionary and make it stick . craziness-is-a-gift-from-the-gods-ly y'rs - tim From esr@thyrsus.com Thu Aug 9 04:21:47 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Wed, 8 Aug 2001 23:21:47 -0400 Subject: [Python-Dev] Common runtime sociology In-Reply-To: ; from tim.one@home.com on Wed, Aug 08, 2001 at 11:08:56PM -0400 References: <3B71E34D.7A2DD001@ActiveState.com> Message-ID: <20010808232147.A16038@thyrsus.com> Tim Peters : > #2 is the best shot. Alas, few people can *decide* to become an insane > visionary and make it stick . I've managed it. And Simon Cozens is staying over at my place tonight (he's quietly hacking on the daybed downstairs, with our cat for company). I think I'll show him this... Yes. Simon Cozens and I officially send you you a hearty: BWAHAHAHAHAHAHA! -- Eric S. Raymond According to the National Crime Survey administered by the Bureau of the Census and the National Institute of Justice, it was found that only 12 percent of those who use a gun to resist assault are injured, as are 17 percent of those who use a gun to resist robbery. These percentages are 27 and 25 percent, respectively, if they passively comply with the felon's demands. Three times as many were injured if they used other means of resistance. -- G. Kleck, "Policy Lessons from Recent Gun Control Research," Law and Contemporary Problems 49, no. 1. (Winter 1986.): 35-62. From tim.one@home.com Thu Aug 9 04:47:37 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 8 Aug 2001 23:47:37 -0400 Subject: [Python-Dev] Common runtime sociology In-Reply-To: <20010808232147.A16038@thyrsus.com> Message-ID: [TIm] > #2 is the best shot. Alas, few people can *decide* to become an insane > visionary and make it stick . [Eric S. Raymond] > I've managed it. Damn straight, Eric! That's one of the reasons I love you. Born a Volvo salesman, you spit in God's eye and decided to foment revolution instead. And God is very pleased by this too, I hear (although His eye still burns, and He does hold grudges). > And Simon Cozens is staying over at my place tonight (he's quietly > hacking on the daybed downstairs, with our cat for company). > I think I'll show him this... > > Yes. Simon Cozens and I officially send you you a hearty: > > BWAHAHAHAHAHAHA! I'll believe Simon has Crossed Over too if you can't find the cat Thursday morning . From esr@thyrsus.com Thu Aug 9 04:59:40 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Wed, 8 Aug 2001 23:59:40 -0400 Subject: [Python-Dev] Common runtime sociology In-Reply-To: ; from tim.one@home.com on Wed, Aug 08, 2001 at 11:47:37PM -0400 References: <20010808232147.A16038@thyrsus.com> Message-ID: <20010808235940.A16661@thyrsus.com> Tim Peters : > I'll believe Simon has Crossed Over too if you can't find the cat Thursday > morning . Hey! Hey! Simon, being a gentleman of taste, *likes* cats -- and not for breakfast either. My wife Cathy observes, correctly, that if we can't find the cat in the morning it will probably be because she has stowed away in his luggage. (The cat stowed away, I mean. Not Cathy. Cathy concedes that Simon is cute, but avers that he lacks my rugged masculine charm .) -- Eric S. Raymond Non-cooperation with evil is as much a duty as cooperation with good. -- Mohandas Gandhi From tim.one@home.com Thu Aug 9 05:31:30 2001 From: tim.one@home.com (Tim Peters) Date: Thu, 9 Aug 2001 00:31:30 -0400 Subject: [Python-Dev] Free threading In-Reply-To: <200108080446.AAA31230@cj20424-a.reston1.va.home.com> Message-ID: [Tim] >> So what is it that multiple interpreters within a single process > *are* for? [Guidp] > It was always feature number one requested by folks coming from Tcl, > where creating multiple interpreters (threaded or not) is an > acceptable pastime for people of good manners. But it seems to be > wasted on the average Pythoneer. Well, I'm not a Tcl guy, but Tcl has historically had no threads at all, yet did have an event model where the *illusion* of multiple threads could still be put to good use. I wouldn't fall over dead with surprise if that's all there were to it: Extreme Fear of Threads . I believe I've seen three requests more common from Tcl land, least to most common: 1. The Tcl event model indeed. 2. Tcl's more robust (than Python's) x-platform exec implementation (similar to Python os.system(), but with a platform-independent notation for pipes and redirection and envar substitution, and returning the exit status correctly even under command.com-based Windows flavors (where os.system() always returns 0)). I looked at this once -- there are several thousand lines of exec support code in Tcl (which shouldn't be surprising, since they're basically supplying one of the hardest parts of a x-platform shell!). 3. Variants of Tcl's uplevel, i.e. gimmicks for fiddling locals "up the stack". Maybe it's not called exactly "uplevel" in Tcl, but it's plain sick however it's spelled . > In Python, there's one good use I can think of: an application may > embed Python and occasionally want to run a good chunk of Python code. > If it needs each chunk of Python code to run completely independent > from each other chunk, creating a new interpreter for each chunk is a > good way to avoid that changes to e.g. sys.path or tabnanny's globals > made by one chunk affects the next chunk. The hosting application > might want the execution to share memory with *it*, so these > interpreters have to live in the same process. > > This works well if the next chunk is only started after the previous > chunk is done; but then using Py_Initialize() and Py_Finalize() is > enough (creating one interpreter at a time). If the chunks need to > overlap in time, creating multiple interpreters is the right solution. That's cool. There isn't much code supporting multiple interpreters, so there's little to gain by getting rid of it now. I just wish it were *used* -- and that it had a more compelling use case . From aahz@rahul.net Thu Aug 9 07:13:27 2001 From: aahz@rahul.net (Aahz Maruch) Date: Wed, 8 Aug 2001 23:13:27 -0700 (PDT) Subject: [Python-Dev] Free threading In-Reply-To: <200108082230.SAA09334@cj20424-a.reston1.va.home.com> from "Guido van Rossum" at Aug 08, 2001 06:30:54 PM Message-ID: <20010809061327.24EC299C81@waltz.rahul.net> Guido van Rossum wrote: > Aahz: >> >> We need thread-local storage, anyway. When I'm a little farther into my >> work on Decimal, I'm going to take a hard look at this because context >> needs to be global on a per-thread basis... > > In the C API, use PyThreadState_GetDict(): it returns a per-thread > dictionary. The convention is that you pick a key unique to your > extension module (e.g. your full extension module name) and then store > your per-thread data under this key. > > In Python, you can store instance variables on the current thread > object -- use threading.currentThread() to access it. It's not quite that simple, particularly if one is writing Python code that permits use of the thread module instead of requiring threading. But what I was really referring to was the annoyance of having every method call pull in the thread-local information. Seems to me that there ought to be some way of making it happen magically. Another issue is the problem of inheriting thread-local values when creating a new thread. That almost certainly requires a callback inside the thread creation machinery. Because I haven't thought much about this, there are probably other issues I don't know about.... -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From martin@strakt.com Thu Aug 9 07:28:56 2001 From: martin@strakt.com (Martin Sjögren) Date: Thu, 9 Aug 2001 08:28:56 +0200 Subject: [Python-Dev] BEGIN_ALLOW_THREADS In-Reply-To: <200108081440.KAA32514@cj20424-a.reston1.va.home.com> References: <20010723142157.B16665@strakt.com> <20010808091404.A11990@strakt.com> <200108081440.KAA32514@cj20424-a.reston1.va.home.com> Message-ID: <20010809082856.A16148@strakt.com> On Wed, Aug 08, 2001 at 10:40:11AM -0400, Guido van Rossum wrote: [snip] > Instead of asking us to add more macros, I recommend that you use the > lower-level C API that is used by the macros. There's a bunch of ways > to save and restore the state. The macros are just shortcuts (and a > handy way to make all the code disappear when threads don't exist). Well, that's exactly what I'm doing, isn't it? Martin --=20 Martin Sj=F6gren martin@strakt.com ICQ : 41245059 Phone: +46 (0)31 405242 Cell: +46 (0)739 169191 GPG key: http://www.strakt.com/~martin/gpg.html From thomas@xs4all.net Thu Aug 9 09:28:12 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 9 Aug 2001 10:28:12 +0200 Subject: [Python-Dev] Common runtime sociology In-Reply-To: <20010808235940.A16661@thyrsus.com> References: <20010808232147.A16038@thyrsus.com> <20010808235940.A16661@thyrsus.com> Message-ID: <20010809102812.D874@xs4all.nl> On Wed, Aug 08, 2001 at 11:59:40PM -0400, Eric S. Raymond wrote: > Simon, being a gentleman of taste, *likes* cats Ah, but more importantly, does the cat like Simon ? That's how you recognize *real* gentlemen. Especially if they sleep on the couch; if they sleep on the couch, they're Bad. If they lie on the couch as an extra pillow, providing bodyheat and attention-on-demand to the cat, they're acceptable. If they pet the cat throughout the night, and feed her in the morning... Cat heaven! Sleepers'-hell-but-watch-the-cat-not-care-ly y'rs ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From klm@zope.com Thu Aug 9 16:15:03 2001 From: klm@zope.com (Ken Manheimer) Date: Thu, 9 Aug 2001 11:15:03 -0400 (EDT) Subject: [Python-Dev] Free threading (and tcl upvar) In-Reply-To: Message-ID: On Thu, 9 Aug 2001, Tim Peters wrote: > 2. Tcl's more robust (than Python's) x-platform exec implementation > (similar to Python os.system(), but with a platform-independent > notation for pipes and redirection and envar substitution, and > returning the exit status correctly even under command.com-based > Windows flavors (where os.system() always returns 0)). I looked > at this once -- there are several thousand lines of exec support > code in Tcl (which shouldn't be surprising, since they're basically > supplying one of the hardest parts of a x-platform shell!). I recall you reporting that. It *would* be nice to have cross-platform pipes and exec, though i understand maintaining such non-portable code is too high a price to pay. I bet we went over this then, but i can't help but wonder whether an approach like tkinter, stealing the tcl code, would be a manageable approach? Mostly idle speculation, since i wouldn't have the time nor expertise to implement such a thing myself. > 3. Variants of Tcl's uplevel, i.e. gimmicks for fiddling locals > "up the stack". Maybe it's not called exactly "uplevel" in Tcl, > but it's plain sick however it's spelled . I think it's 'upvar'. I probably remember because it had something to do with my python roots. Pardon me while i geeze a bit... Back at nist i was looking for a scripting language for systems support type stuff, bounced off perl, and tried using tcl for a trial task, to see if i could use it. I made a dns nested-domain traverser, so i could automate a table-driven network service for all of nist's subdomains (producing an exports file for a nist-wide NFS service). I did it, but was not a happy camper - it was seriously painful to have to manage recursion by, essentially, managing the stack with upvar, &c. Mike mclay happened to drop by around then, and i whined to him about my tcl abomination um script - and he said, "have you looked at python?" I did, and, to make a long story short, lived happily ever after.-) 'upvar' *is* sick. Thanks, mike. Ken klm@zope.com [In a usenet news group, long long ago:] > [Someone]: > > >Programming language preferences are largely a matter of taste, an area > >where reasonable people will forever disagree. I would like to think > >that such disagreements could be predicated upon an understanding that > >most of the issues involved are not decidable by logical proofs. > > This is the 101 philosophy-student phenomena: "I can't explain it with > words". > > - marcus@ee.pdx.edu (Marcus Daniels) From simon@netthink.co.uk Thu Aug 9 19:12:30 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Thu, 9 Aug 2001 11:12:30 -0700 Subject: [Python-Dev] Common runtime sociology In-Reply-To: <5.1.0.14.0.20010808212559.0223fb60@tuatha.sidhe.org> Message-ID: <20010809111229.D1266@netthink.co.uk> On Wed, Aug 08, 2001 at 09:57:50PM -0400, Dan Sugalski wrote: > I think it's possible for things to be done otherwise, but it puts a heavy > burden on the implementation group to be actively engaged with the language > groups and be as impartial as can be managed. I think technically, it's very easy; Python people might understand the parallel that their interpeter per se isn't too concerned with the behaviour of individual types - all the type-specific behaviour is defined in the various vtables for the types. (Object/*.[ch]) It is, you know, possible to add Object/PerlVar.c to have an object behave like a Perl variable. And since you can specialise the hell out of what you do in Object/PerlVar.c while the main loop of the interpreter is very generic, you can make Perl types run as fast as Python ones or better. (The more I look at Python's VM, the more it seems to have many of the ideas that we want in it...) So *in* *a* *very* *restricted* sense*, I agree with Eric's proposal that we should look at using the Python interpreter as a base for Parrot - in design, if not in actual code. (Especially since we'll want to change it to a register model.) There are a few things I've identified that we should change from the way Python does stuff, (notably that we should make types mutable, and we should try and see if we can remove the "Type object" layer of abstraction, since that indirection slows us down) but the core of what you've got looks really, really useful. Now I've put myself on the line, it's time for Dan to come over and correct me. ("Ouch!") :) > You almost need to take the implementation group that starts this and > pry them loose to be an entity in and of themselves, separate from the > original language group. (FWIW, this is part of my Master Plan. Dunno > if it's part of Simon's) I was going to suggest a separate Parrot project, but thought that might be going too far. Glad we're on the same wavelength. Simon From guido@python.org Thu Aug 9 16:22:45 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 09 Aug 2001 11:22:45 -0400 Subject: [Python-Dev] Free threading (and tcl upvar) In-Reply-To: Your message of "Thu, 09 Aug 2001 11:15:03 EDT." References: Message-ID: <200108091522.f79FMja02243@odiug.digicool.com> > I recall you reporting that. It *would* be nice to have cross-platform > pipes and exec, though i understand maintaining such non-portable code is > too high a price to pay. I bet we went over this then, but i can't help > but wonder whether an approach like tkinter, stealing the tcl code, would > be a manageable approach? Mostly idle speculation, since i wouldn't have > the time nor expertise to implement such a thing myself. Tcl is a glue language that encourages you to glue external processes together. This is much less emphasized in Python, so I doubt that it will be as useful. I don't think it's worth the maintenance nightmare. Copying the Tcl code doesn't mean we have no maintenance -- the Tcl code undoubtedly uses lots of Tcl runtime convenience routines, which we would have to adapt to the Python runtime environment. > 'upvar' *is* sick. That's why we have sys._getframe(). ;-) --Guido van Rossum (home page: http://www.python.org/~guido/) From klm@zope.com Thu Aug 9 16:43:32 2001 From: klm@zope.com (Ken Manheimer) Date: Thu, 9 Aug 2001 11:43:32 -0400 (EDT) Subject: [Python-Dev] Free threading (and tcl upvar) In-Reply-To: <200108091522.f79FMja02243@odiug.digicool.com> Message-ID: On Thu, 9 Aug 2001, Guido van Rossum wrote: > > 'upvar' *is* sick. > > That's why we have sys._getframe(). ;-) We _have_ it, but most of the time we don't _need_ it - thank you. ! (And now that i'm finally using python 2, i could probably reimplement my dns walker similar to the way i did w/tcl, without having to resort to the 'raise' gambit to get a stack frame. Woo woo!-) Ken klm@zope.com "But i digress." From barry@zope.com Thu Aug 9 16:44:47 2001 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 9 Aug 2001 11:44:47 -0400 Subject: [Python-Dev] Free threading References: <200108080446.AAA31230@cj20424-a.reston1.va.home.com> Message-ID: <15218.45039.403666.534610@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> It was always feature number one requested by folks coming GvR> from Tcl, where creating multiple interpreters (threaded or GvR> not) is an acceptable pastime for people of good manners. GvR> But it seems to be wasted on the average Pythoneer. GvR> In Python, there's one good use I can think of: an GvR> application may embed Python and occasionally want to run a GvR> good chunk of Python code. If it needs each chunk of Python GvR> code to run completely independent from each other chunk, GvR> creating a new interpreter for each chunk is a good way to GvR> avoid that changes to e.g. sys.path or tabnanny's globals GvR> made by one chunk affects the next chunk. funny! in a previous life i wrote a large app that embedded tcl for its configuration language. i did exactly as guido describes for exactly that reason! in fact, in tcl, you could capture a complete snapshot of the interpreter in tcl source, replay that into a fresh interpreter and be back exactly where you started. a cheapo (but effective) "save-game" like mechanism. haven't yet had the need to do the same in python. GvR> But apparently this habit has fallen in disuse -- from GvR> looking at the code I believe there's a serious problem with GvR> exceptions, where initializing the second interpreter messes GvR> up the identities of the exception classes stored in global GvR> variables by the first interpreter, and nobody has ever GvR> complained about this. Or maybe they are still using Python GvR> 1.5.2, which may not have the same problem. Or maybe this is GvR> the reason why PyApache has a reptabion of instability. :( i vaguely remember looking at this, and (i thought) fixing it, but that memory's off-line through the weekend. :) -barry From jeremy@zope.com Thu Aug 9 17:26:05 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 9 Aug 2001 12:26:05 -0400 (EDT) Subject: [Python-Dev] Common runtime sociology In-Reply-To: <20010809111229.D1266@netthink.co.uk> References: <5.1.0.14.0.20010808212559.0223fb60@tuatha.sidhe.org> <20010809111229.D1266@netthink.co.uk> Message-ID: <15218.47517.833096.75695@slothrop.digicool.com> >>>>> "SC" == Simon Cozens writes: SC> (The more I look at Python's VM, the more it seems to have many SC> of the ideas that we want in it...) You're well on your way, Simon. Once you realized that you don't need Object/PerlVar.c, you will have reached enlightenment. I don't think the cat will even entertain the notion of running away with you before then. Jeremy From skip@pobox.com (Skip Montanaro) Thu Aug 9 17:28:36 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 9 Aug 2001 11:28:36 -0500 Subject: [Python-Dev] optimizing non-local object access Message-ID: <15218.47668.406742.453009@beluga.mojam.com> I had a thought last night about optimizing access to globals in other modules (e.g. "math.sin" or "string.uppercase"). I think it could actually work to speed up all non-local object access. The nice thing about module globals is that for the most part they are "almost constant". Once a module is imported, it's rare that its global name bindings actually change. I think it might suffice to create a notification system that allows an executing frame to register its interest in changes to a module's name bindings. Let me make this idea concrete with an example. Suppose I have the following code: import math def sinseq(n): l = [] for i in xrange(n): l.append(math.sin(i)) return l "math.sin" requires a global lookup and an attribute access each pass through the loop. Let's add an entry to the fastlocals array for it and an entry to the local names called "math.sin". The byte code compiler generates a LOAD_FAST instruction anywhere math.sin is accessed, and at the first point where it should be "live" (i.e., at the beginning of the function) it simply inserts a TRACK_OBJECT instruction with that slot in fastlocals as one argument and the "math.sin" slot in the local names as the other. When "math.sin" goes out of scope (i.e., at the end of the function), it inserts an UNTRACK_OBJECT instruction with "math.sin" as its argument. The same can be done for "l.append" and xrange. The bytecode for the above function would look something like: TRACK_OBJECT math.sin TRACK_OBJECT xrange >> 0 BUILD_LIST 0 ('\000', '\000') 3 STORE_FAST 1 (l) TRACK_OBJECT l.append 6 SETUP_LOOP 44 (to 53) 9 LOAD_FAST xrange 12 LOAD_FAST 0 (n) 15 CALL_FUNCTION 1 ('\001', '\000') 18 LOAD_CONST 1 (0) 21 FOR_LOOP 28 (to 52) 24 STORE_FAST 2 (i) 27 LOAD_FAST l.append LOAD_FAST math.sin 39 LOAD_FAST 2 (i) 42 CALL_FUNCTION 1 ('\001', '\000') 45 CALL_FUNCTION 1 ('\001', '\000') 48 POP_TOP 49 JUMP_ABSOLUTE 21 ('\025', '\000') >> 52 POP_BLOCK >> 53 LOAD_FAST 1 (l) UNTRACK_OBJECT l.append UNTRACK_OBJECT xrange UNTRACK_OBJECT math.sin 56 RETURN_VALUE TRACK_OBJECT and UNTRACK_OBJECT are responsible for registering and unregistering interest in an object. TRACK_OBJECT also caches the corresponding object in the fastlocals array. I think most or all of the work can be handled by PyObject_SetAttr and/or PyDict_SetItem. It seems to me that its greatest benefit would be to short circuit access to global variables in other modules. For the most part, these don't change once an import is completed, so you'd effectively be converting access to these objects into local variables, but you could use it to track all non-local variables or attributes. stepping-behind-the-egg-proof-screen-ly y'rs, Skip From aahz@rahul.net Thu Aug 9 17:33:27 2001 From: aahz@rahul.net (Aahz Maruch) Date: Thu, 9 Aug 2001 09:33:27 -0700 (PDT) Subject: [Python-Dev] Any objections to adding threading.Timer? In-Reply-To: <200108082034.f78KYlB16985@mira.informatik.hu-berlin.de> from "Martin v. Loewis" at Aug 08, 2001 10:34:47 PM Message-ID: <20010809163328.6691299C86@waltz.rahul.net> Martin v. Loewis wrote: > > In patch > > https://sourceforge.net/tracker/index.php?func=detail&aid=428326&group_id=5470&atid=305470 > > a new threading.Timer class is proposed. Since the patch was submitted > two months ago, nobody seemed to care enough to comment on it. I like > it, and I'll commit the patch unless I hear otherwise. I'm not objecting, precisely, but I have an allergic reaction to the way timed Event() works, even though there's really no other way to work it. I think that encouraging people to use it is a Bad Idea. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From paulp@ActiveState.com Thu Aug 9 17:40:38 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 09 Aug 2001 09:40:38 -0700 Subject: [Python-Dev] Common runtime sociology References: <5.1.0.14.0.20010808212559.0223fb60@tuatha.sidhe.org> <20010809111229.D1266@netthink.co.uk> <15218.47517.833096.75695@slothrop.digicool.com> Message-ID: <3B72BD06.687DC1CA@ActiveState.com> Jeremy Hylton wrote: > > >>>>> "SC" == Simon Cozens writes: > > SC> (The more I look at Python's VM, the more it seems to have many > SC> of the ideas that we want in it...) > > You're well on your way, Simon. Once you realized that you don't need > Object/PerlVar.c, you will have reached enlightenment. ;-) Object/PerlVar.c is like the teddy bear you take with you to camp to remind you of home. Or maybe its like the picture of your girlfriend you take to remind you that you already HAVE a girlfriend. <0.5 wink> -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From jeremy@zope.com Thu Aug 9 17:50:47 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 9 Aug 2001 12:50:47 -0400 (EDT) Subject: [Python-Dev] optimizing non-local object access In-Reply-To: <15218.47668.406742.453009@beluga.mojam.com> References: <15218.47668.406742.453009@beluga.mojam.com> Message-ID: <15218.48999.964618.234723@slothrop.digicool.com> I've been thinking about the same issues in my spare time. I certainly think the interpreter could be made faster by assuming that most bindings are static most of the time. Python has many places where bindings can change unexpectedly, but most of the time they don't change. My worry about your approach is that the track-object opcodes could add a lot of expense for objects only used once or twice. If the function uses math.sin inside a loop, it's an obvious win. If it uses it only once, it's not so clear. The other approach I had been thinking about, which may have made it into my pre-PEP post a few months ago, is to store module globals in a table that can be accessed like fast locals. Any reference to another module would use the fast globals table. For references to global names in another module, the calling module could lookup the offset of that name in the defining module's fast globals. To be more concrete: The math module would store the sin name in slot X. The first time the foobar module used math.sin it would lookup the slot of sin in the math table. The foobar module would store a pointer to math's fast globals and the index of the sin slot. Then math.sin would be accessed via a single opcode that used the stored information. This approach doesn't extend to instances in an obvious way, but I'd be happy dealing only with modules for a start. There's a fair amount of Zope code that does things like this: def hotspot_func(x, _tt=types.TupleType): ... I'd like to get rid of the abuse of default args to make access to TupleType fast. I have been thinking some about how to make a similar strategy work for instances. I think we could apply some lessons from the Self compiler and recompile generic bytecodes ("load the 'foo' attribute") into more specific bytecodes ("load slot 12"). But this is more speculation than anything else. Jeremy From Samuele Pedroni Thu Aug 9 18:39:33 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Thu, 9 Aug 2001 19:39:33 +0200 (MET DST) Subject: [Python-Dev] optimizing non-local object access Message-ID: <200108091739.TAA10088@core.inf.ethz.ch> [Jeremy Hylton] > I've been thinking about the same issues in my spare time. I > certainly think the interpreter could be made faster by assuming that > most bindings are static most of the time. Python has many places > where bindings can change unexpectedly, but most of the time they > don't change. > > My worry about your approach is that the track-object opcodes could > add a lot of expense for objects only used once or twice. If the > function uses math.sin inside a loop, it's an obvious win. If it uses > it only once, it's not so clear. I agree, and I wonder whether it is not going to explode (in term of effort and resources) down a call graph. > The other approach I had been thinking about, which may have made it > into my pre-PEP post a few months ago, is to store module globals in a > table that can be accessed like fast locals. Any reference to another > module would use the fast globals table. For references to global > names in another module, the calling module could lookup the offset of > that name in the defining module's fast globals. > I rembemer I liked it. > I have been thinking some about how to make a similar strategy work > for instances. I think we could apply some lessons from the Self > compiler and recompile generic bytecodes ("load the 'foo' attribute") > into more specific bytecodes ("load slot 12"). But this is more > speculation than anything else. Some considerations about this: Python vs. Smalltalk vs. Self Smalltalk classes have rigid sets of slots, single inheritance and all slot are private (protected?), so I imagine it is easy to number them. In Self the set of slots of an object is also fixed (there are specific reflective methods to modify the set of slots) and also which are mutable etc. OTOH Self has dynamic inheritance (something like the mutable __bases__ in Python). To deal with all this it uses customization (different version of a single method are lazily compiled against the various concrete structural variations of objects that inherit it) and assume that things found up in the inheritance tree are constant but guards itself about their changes Python is even less rigid than Self and there is not an explicit notion of the set of slots that goes with an object. So to achieve some improvement in slot access speed may be necessary to: - set more rigid rules on slots (but then that's not Python anymore) - find a way to detect some conservative approximation of the set of slots of a concrete class - use customization (consumes memory) so you can exploit a fixed layout for the approximated slots - use plain dictionary lookup for the slots missed that way Not impossible, not trivial and surely a thing that will pollute code clarity, OTOH whith this kind of optimizations in place, even just at the interp level, effective native compilation is not that far ... regards, Samuele Pedroni. From jeremy@zope.com Thu Aug 9 18:43:31 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 9 Aug 2001 13:43:31 -0400 (EDT) Subject: [Python-Dev] optimizing non-local object access In-Reply-To: <200108091739.TAA10088@core.inf.ethz.ch> References: <200108091739.TAA10088@core.inf.ethz.ch> Message-ID: <15218.52163.546008.296225@slothrop.digicool.com> I've made some space for this project in my Wiki: http://www.zope.org/Members/jeremy/CurrentAndFutureProjects/FastGlobals It's got the text of the PEP that I posted and a pointer to Guido's original message on the subject. Feel free to contribute more info if you're interested. (The permissions may be wrong at the moment. Let me know if you run into trouble.) Jeremy From jeremy@zope.com Thu Aug 9 18:49:29 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 9 Aug 2001 13:49:29 -0400 (EDT) Subject: [Python-Dev] optimizing non-local object access In-Reply-To: <200108091739.TAA10088@core.inf.ethz.ch> References: <200108091739.TAA10088@core.inf.ethz.ch> Message-ID: <15218.52521.776800.5873@slothrop.digicool.com> >>>>> "SP" == Samuele Pedroni writes: SP> Python is even less rigid than Self and there is not an explicit SP> notion of the set of slots that goes with an object. So to I agree that this is the key hurdle. SP> achieve some improvement in slot access speed may be necessary SP> to: SP> - set more rigid rules on slots (but then that's not Python SP> anymore) Not allowed :-). SP> - find a way to detect some conservative approximation of the SP> set of slots of a concrete class I think this can be done with better compiler support. SP> - use customization (consumes memory) so you can exploit a fixed SP> layout for the approximated slots SP> - use plain dictionary lookup for the slots missed that way Right. SP> Not impossible, not trivial and surely a thing that will pollute SP> code clarity, OTOH whith this kind of optimizations in place, SP> even just at the interp level, effective native compilation is SP> not that far ... It is a lot of work for instances, so I've been considering it separately from the module globals issue. But I think it is possible. Short of native-code, I wonder if we can extend the Python VM with opcodes that take advantage of slots. I'm not sure how closely this idea is related to Pysco. Imagine we didn't special case ints in BINARY_ADD. If you're doing mostly string addition, it's just slowing you down anyway :-). Instead, a function that expected to see ints would check the types of the relevant args. def f(x): return 2*x would become something like: def f(x): if isinstance(x, int): return 2 * x_as_int else: return 2 * x where the int branch could use opcodes that knew that x was an integer and could call its __add__ method directly. Jeremy From skip@pobox.com (Skip Montanaro) Thu Aug 9 18:58:37 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 9 Aug 2001 12:58:37 -0500 Subject: [Python-Dev] optimizing non-local object access In-Reply-To: <15218.48999.964618.234723@slothrop.digicool.com> References: <15218.47668.406742.453009@beluga.mojam.com> <15218.48999.964618.234723@slothrop.digicool.com> Message-ID: <15218.53069.713724.34519@beluga.mojam.com> Jeremy> My worry about your approach is that the track-object opcodes Jeremy> could add a lot of expense for objects only used once or twice. Jeremy> If the function uses math.sin inside a loop, it's an obvious Jeremy> win. If it uses it only once, it's not so clear. Even if math.sin is used just once you swap a LOAD_GLOBAL/LOAD_ATTR pair for a TRACK_OBJECT/LOAD_FAST/UNTRACK_OBJECT trio, so the hit you take shouldn't be terrible. (My assumption is that the register/unregister cost is fairly low and the actual notification/update code will almost never be executed.) You break even in total instructions executed with two accesses and win after that. In addition, this might be a strategy left for an optimization pass that would only make the change if the LOAD_ATTR and/or LOAD_GLOBAL instructions are executed in a loop. Jeremy> To be more concrete: The math module would store the sin name in Jeremy> slot X. The first time the foobar module used math.sin it would Jeremy> lookup the slot of sin in the math table. The foobar module Jeremy> would store a pointer to math's fast globals and the index of Jeremy> the sin slot. Then math.sin would be accessed via a single Jeremy> opcode that used the stored information. Unfortunately, the code that uses math.sin can't know that math is a module. It might be an instance with a sin attribute. Even worse, because of Python's dynamic nature, what the name "math" is bound to can change. You can't assume it will always be bound to a module object, even if it is the first time you set things up. I think you have to work with names and name bindings. I don't think you can make assumptions about what the names are bound to. The handwaving bit in my post was there because I am not familiar enough with the various possibilities for name rebinding. Does it all boil down to PyDict_SetItem or PyObject_SetAttr as I suspect? Are those functions too low-level, that is, have the names been forgetten completely at that point? If so, perhaps STORE_GLOBAL and STORE_ATTR would have to be modified to use PyDict_SetItemString and PyObject_SetAttrString instead. Skip From skip@pobox.com (Skip Montanaro) Thu Aug 9 19:05:28 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 9 Aug 2001 13:05:28 -0500 Subject: [Python-Dev] optimizing non-local object access In-Reply-To: <15218.52521.776800.5873@slothrop.digicool.com> References: <200108091739.TAA10088@core.inf.ethz.ch> <15218.52521.776800.5873@slothrop.digicool.com> Message-ID: <15218.53480.204468.854961@beluga.mojam.com> Jeremy> Imagine we didn't special case ints in BINARY_ADD. If you're Jeremy> doing mostly string addition, it's just slowing you down anyway Jeremy> :-). Instead, a function that expected to see ints would check Jeremy> the types of the relevant args. Jeremy> def f(x): Jeremy> return 2*x Jeremy> would become something like: Jeremy> def f(x): Jeremy> if isinstance(x, int): Jeremy> return 2 * x_as_int Jeremy> else: Jeremy> return 2 * x Jeremy> where the int branch could use opcodes that knew that x was an Jeremy> integer and could call its __add__ method directly. Yeah, but all you've really done is extract the current inline test for int arguments into a few more PyVM instructions. You've sacrificed a fairly fast special case test for more passes around the interpreter loop. I'm not sure how this can be a win in the current virtual machine. Skip From jeremy@zope.com Thu Aug 9 19:15:24 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 9 Aug 2001 14:15:24 -0400 (EDT) Subject: [Python-Dev] optimizing non-local object access In-Reply-To: <15218.53069.713724.34519@beluga.mojam.com> References: <15218.47668.406742.453009@beluga.mojam.com> <15218.48999.964618.234723@slothrop.digicool.com> <15218.53069.713724.34519@beluga.mojam.com> Message-ID: <15218.54076.512057.301420@slothrop.digicool.com> >>>>> "SM" == Skip Montanaro writes: Jeremy> My worry about your approach is that the track-object Jeremy> opcodes could add a lot of expense for objects only used Jeremy> once or twice. If the function uses math.sin inside a loop, Jeremy> it's an obvious win. If it uses it only once, it's not so Jeremy> clear. SM> Even if math.sin is used just once you swap a SM> LOAD_GLOBAL/LOAD_ATTR pair for a SM> TRACK_OBJECT/LOAD_FAST/UNTRACK_OBJECT trio, so the hit you take SM> shouldn't be terrible. (My assumption is that the SM> register/unregister cost is fairly low and the actual SM> notification/update code will almost never be executed.) You SM> break even in total instructions executed with two accesses and SM> win after that. I'm assuming some kind of memory allocation is necessary to accomodate an aribtrary number of handlers. If you need to resize an array that holds the pointers to the tracking callbacks, it could get expensive. I also wonder if you have to pay for tracking changes whenever the name is rebound in the module, or just when you need to use the name again. SM> In addition, this might be a strategy left for SM> an optimization pass that would only make the change if the SM> LOAD_ATTR and/or LOAD_GLOBAL instructions are executed in a SM> loop. Good point. Jeremy> To be more concrete: The math module would store the sin Jeremy> name in slot X. The first time the foobar module used Jeremy> math.sin it would lookup the slot of sin in the math table. Jeremy> The foobar module would store a pointer to math's fast Jeremy> globals and the index of the sin slot. Then math.sin would Jeremy> be accessed via a single opcode that used the stored Jeremy> information. SM> Unfortunately, the code that uses math.sin can't know that math SM> is a module. It might be an instance with a sin attribute. SM> Even worse, because of Python's dynamic nature, what the name SM> "math" is bound to can change. You can't assume it will always SM> be bound to a module object, even if it is the first time you SM> set things up. I think you have to work with names and name SM> bindings. I don't think you can make assumptions about what the SM> names are bound to. No assumptions necessary. The compiler only emits the new opcodes for names bound by import or attributes thereof. If the module name ('math') is rebound, the interpreter is responsible for reseting all of the other bindings that depend on it ('math.sin'). If the object 'math', isn't a module (even though the compiler guessed it would be), the opcodes fall back to the old implementation. The first time math.sin is used, we do the following: - check if the math.sin fast globals entry is initialized (it won't be, but it could be marked uninitialized or "don't use") - check that math is indeed a module - lookup the sin slot in math - record the slot in the fast globals table On future uses, the first step above will discover a valid binding. If the name math is rebound, the interpreter marks as the fast globlas refering to it as unitinitalized. One advantage of this approach is that the work is shared across all code in a module. If many functions use math.sin, the first one initializes the table and all the rest use it. SM> The handwaving bit in my post was there because I am not SM> familiar enough with the various possibilities for name SM> rebinding. Does it all boil down to PyDict_SetItem or SM> PyObject_SetAttr as I suspect? Are those functions too SM> low-level, that is, have the names been forgetten completely at SM> that point? If so, perhaps STORE_GLOBAL and STORE_ATTR would SM> have to be modified to use PyDict_SetItemString and SM> PyObject_SetAttrString instead. I think we hook in at the tp_getattr(o) level. Module objects can detect rebindings there and do whatever bookkeeping is necessary to keep references to its name consistent. I think this is the right approach for either technique we're discussing. Jeremy From jeremy@zope.com Thu Aug 9 19:22:02 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 9 Aug 2001 14:22:02 -0400 (EDT) Subject: [Python-Dev] optimizing non-local object access In-Reply-To: <15218.53480.204468.854961@beluga.mojam.com> References: <200108091739.TAA10088@core.inf.ethz.ch> <15218.52521.776800.5873@slothrop.digicool.com> <15218.53480.204468.854961@beluga.mojam.com> Message-ID: <15218.54474.374553.543595@slothrop.digicool.com> I haven't thought through whether it's a win for integers. I expect it would be a win for instances, particularly when several methods are called on the instances. My assumption is that the cost of an extra type dispatch pass through the interpreter loop is offset by the ability to quickly resolve attribute names. This is at best half-baked, so I don't expect to be too convincing. I still need to convince myself. Jeremy From skip@pobox.com (Skip Montanaro) Thu Aug 9 19:44:37 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 9 Aug 2001 13:44:37 -0500 Subject: [Python-Dev] optimizing non-local object access In-Reply-To: <15218.54076.512057.301420@slothrop.digicool.com> References: <15218.47668.406742.453009@beluga.mojam.com> <15218.48999.964618.234723@slothrop.digicool.com> <15218.53069.713724.34519@beluga.mojam.com> <15218.54076.512057.301420@slothrop.digicool.com> Message-ID: <15218.55829.400113.482371@beluga.mojam.com> Jeremy, Perhaps we need a bake-off to try both approaches. I think I can localize my changes fairly well, but it will take me a little time to come up-to-speed on changes to ceval.c since the 1.5.2 timeframe. (I will have to do that to migrate Rattlesnake to the current codebase and to work on some notes about PyVM that might eventually turn into a PyVM reference manual.) wish-i-could-get-paid-for-this-ly y'rs, Skip From Samuele Pedroni Thu Aug 9 19:50:33 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Thu, 9 Aug 2001 20:50:33 +0200 (MET DST) Subject: [Python-Dev] optimizing non-local object access Message-ID: <200108091850.UAA12818@core.inf.ethz.ch> > SP> - find a way to detect some conservative approximation of the > SP> set of slots of a concrete class > > I think this can be done with better compiler support. > The simplest thing would be to detect all 'attr' that appear in the body of the methods of a class statements in 'self.attr' form. Class construction at run-time will take the union of this together with the information computed for base classes Why at runtime? It avoids the trouble of poor-man-to-full-fledged type-inference for the compiler. It would be reasonably effective and at most would require the programmer to put a self.myFavoriteAttr = None in __init__. > SP> - use customization (consumes memory) so you can exploit a fixed > SP> layout for the approximated slots > SP> - use plain dictionary lookup for the slots missed that way > > Right Clearly this should be played two-hands with method dispatch. > > SP> Not impossible, not trivial and surely a thing that will pollute > SP> code clarity, OTOH whith this kind of optimizations in place, > SP> even just at the interp level, effective native compilation is > SP> not that far ... > > It is a lot of work for instances, so I've been considering it > separately from the module globals issue. But I think it is > possible. Short of native-code, I wonder if we can extend the Python > VM with opcodes that take advantage of slots. It is indeed a good idea, but the problem is that customising without native compilation can have a very poor compilation-time vs. runtime speedup ratio. OTOH we could customize only hot-spots. But I think is worth trying if we ever get at it. > I'm not sure how > closely this idea is related to Pysco. AFAIK it would be complementary, Armin Rigo pointed out that in principle Psyco can layout objects at will, but I remarked that is way of dealing with code comsumes a lot of memory so it is workable only if applied only to hot spots, and then we pay for conversion when non-Pysco code calls Psyco code. Psyco is doing some sort of customization (a generalisation of that), a layout for instances that can be exploited in term of speed under customization is a good thing for Psyco too IMHO. > Imagine we didn't special case ints in BINARY_ADD. If you're doing > mostly string addition, it's just slowing you down anyway :-). > Instead, a function that expected to see ints would check the types of > the relevant args. > > def f(x): > return 2*x > > would become something like: > > def f(x): > if isinstance(x, int): > return 2 * x_as_int > else: > return 2 * x > > where the int branch could use opcodes that knew that x was an integer > and could call its __add__ method directly. AFAIK this is somehow what Psyco does, I have not checked the code but my understanding is that it goes a hard way to do this kind of thing specifically when they can give you something. Customization would mean the following: class A: def __init__(self, a=2): self.a = a def double(self): self.a*=2 class B: def __init__(self, b=3): self.b = b def inc(self): self.b+=1 class C(A,B): def __init__(self,a,b): A.__init__(a) B.__init__(b) def mul(self): return a*b under customization if both instances of A and C are used and in both cases double is used there will be two versions of double to choose from, one that expect self to be of concrete class A and one that expect it to be of concrete class C. Both these versions can be constructed knowing the exact place of the slot a in the objects. It works clearly best with good pure single dispatch oo code ;). In any case both a more static layout for slots and some way to add hooks that count how many times a code object is invoked, would be a good start point for a lot of possible nice experiments. regards, Samuele. From fdrake@acm.org Thu Aug 9 19:50:00 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 9 Aug 2001 14:50:00 -0400 (EDT) Subject: [Python-Dev] optimizing non-local object access In-Reply-To: <15218.55829.400113.482371@beluga.mojam.com> References: <15218.47668.406742.453009@beluga.mojam.com> <15218.48999.964618.234723@slothrop.digicool.com> <15218.53069.713724.34519@beluga.mojam.com> <15218.54076.512057.301420@slothrop.digicool.com> <15218.55829.400113.482371@beluga.mojam.com> Message-ID: <15218.56152.976997.480495@cj42289-a.reston1.va.home.com> Skip Montanaro writes: > to do that to migrate Rattlesnake to the current codebase and to work on > some notes about PyVM that might eventually turn into a PyVM reference > manual.) Just let me know when you're ready to check this in! ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From jeremy@zope.com Thu Aug 9 19:55:28 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 9 Aug 2001 14:55:28 -0400 (EDT) Subject: [Python-Dev] optimizing non-local object access In-Reply-To: <15218.55829.400113.482371@beluga.mojam.com> References: <15218.47668.406742.453009@beluga.mojam.com> <15218.48999.964618.234723@slothrop.digicool.com> <15218.53069.713724.34519@beluga.mojam.com> <15218.54076.512057.301420@slothrop.digicool.com> <15218.55829.400113.482371@beluga.mojam.com> Message-ID: <15218.56480.865209.746007@slothrop.digicool.com> A bake-off sounds like a great idea. I probably won't get to this until after 2.2 -- and wouldn't be able to get it into 2.2 at this late date even if I had the time. Jeremy From aahz@rahul.net Thu Aug 9 21:00:13 2001 From: aahz@rahul.net (Aahz Maruch) Date: Thu, 9 Aug 2001 13:00:13 -0700 (PDT) Subject: [Python-Dev] Common runtime sociology In-Reply-To: <20010808232147.A16038@thyrsus.com> from "Eric S. Raymond" at Aug 08, 2001 11:21:47 PM Message-ID: <20010809200014.2188399C8E@waltz.rahul.net> Eric S. Raymond wrote: > Tim Peters : >> >> #2 is the best shot. Alas, few people can *decide* to become an insane >> visionary and make it stick . > > I've managed it. I've heard enough comments from other people to only agree heartily on the "insane" part. ;-) -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From jepler@inetnebr.com Thu Aug 9 21:05:51 2001 From: jepler@inetnebr.com (Jeff Epler) Date: Thu, 9 Aug 2001 15:05:51 -0500 Subject: [Python-Dev] optimizing non-local object access In-Reply-To: <15218.53480.204468.854961@beluga.mojam.com>; from skip@pobox.com on Thu, Aug 09, 2001 at 01:05:28PM -0500 References: <200108091739.TAA10088@core.inf.ethz.ch> <15218.52521.776800.5873@slothrop.digicool.com> <15218.53480.204468.854961@beluga.mojam.com> Message-ID: <20010809150549.F9751@inetnebr.com> On Thu, Aug 09, 2001 at 01:05:28PM -0500, Skip Montanaro wrote: > Yeah, but all you've really done is extract the current inline test for int > arguments into a few more PyVM instructions. You've sacrificed a fairly > fast special case test for more passes around the interpreter loop. I'm not > sure how this can be a win in the current virtual machine. The real gain comes when you can hoist the type test out of the loop (though that means proving the type condition is an invariant of the loop!) For instance, in the following, the type of x is invariant over the loop, and the test can be moved entirely outside the loop. Now, the loop itself can operate on ints, and hopefully benefit from it (in the most extreme case, the loop can become about 2 CPU (not VM) instructions per line, given even a fairly rudimentary JIT. Add a bit more to catch the overflow in the 'x*3+1' expression (or even the 'r+1' expression), whether to raise ValueError or silently become a long integer and bail back to the slow version. def f(x): """ The famous "hailstone" problem, assuming my memory is right --- nobody seems to know how to predict #iterations or highest intermediate value of x from the initial value, or has even proven that the algorithm completes for all values of x """ r = 0 while x > 1: r = r + 1 if x%2 == 0: x=x/2 # Truncating integer division else: x=x*3+1 return r def f(x): if isinstance(x, int): """run fully native code with x and r known to be platform ints""" else: """run python bytecode""" The "hard" parts, as I see them, are detecting/deducing these type invariants, even given hints from the programmer about the types of parameters (a la 'f = optimize(f, (int,))'), and having a way to "bail" to the original bytecode version. You could have if isinstance(x, int): try: """ fully native version """ except ValueError: pass """ run python bytecode if there was no optimized version, or this x happened to overflow platform int """ but that would not work if the code makes calls that have side-effects. (mutate objects, produce output, ...) Transmeta has patented their methods to do this, in their dynamic binary translating CPUs (not that their approaches are likely to have much relevance for the Python VM). Jeff From skip@pobox.com (Skip Montanaro) Thu Aug 9 21:15:24 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 9 Aug 2001 15:15:24 -0500 Subject: [Python-Dev] optimizing non-local object access In-Reply-To: <20010809150549.F9751@inetnebr.com> References: <200108091739.TAA10088@core.inf.ethz.ch> <15218.52521.776800.5873@slothrop.digicool.com> <15218.53480.204468.854961@beluga.mojam.com> <20010809150549.F9751@inetnebr.com> Message-ID: <15218.61276.654235.714590@beluga.mojam.com> Jeff> The real gain comes when you can hoist the type test out of the Jeff> loop (though that means proving the type condition is an invariant Jeff> of the loop!) Even more difficult if the programmer can subclass ints, yes?. Today, if I see x=x/2 and I know x is an int, I can assume x/2 is also (or a float once integer division returns floats). Once ints can be subclassed, a devious programmer could return something other than an int (say, a float, a rational, or the string "x/2"). Skip From neal@metaslash.com Thu Aug 9 21:15:53 2001 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 09 Aug 2001 16:15:53 -0400 Subject: [Python-Dev] Direction of PyChecker Message-ID: <3B72EF79.7BE8F6C1@metaslash.com> I'd like to solicit feedback on what everyone thinks about PyChecker. Specifically, where should it go? What should it include or not include? What should the default warnings be? I've seen some threads on c.l.p that checker might address. Do you think that there should be something added to the python FAQ/doc about checker? I'd be willing to write something. I'm close to exhausting my list of warnings/checks, let me know if there are additional warnings/checks you can think of. Thanks for your input, Neal PS I sent a patch to python-dev against 2.2a1, but it's 45k. So it's awating moderator approval. Sorry if this isn't the right venue for patches. From jepler@inetnebr.com Thu Aug 9 21:17:45 2001 From: jepler@inetnebr.com (Jeff Epler) Date: Thu, 9 Aug 2001 15:17:45 -0500 Subject: [Python-Dev] optimizing non-local object access In-Reply-To: <15218.61276.654235.714590@beluga.mojam.com>; from skip@pobox.com on Thu, Aug 09, 2001 at 03:15:24PM -0500 References: <200108091739.TAA10088@core.inf.ethz.ch> <15218.52521.776800.5873@slothrop.digicool.com> <15218.53480.204468.854961@beluga.mojam.com> <20010809150549.F9751@inetnebr.com> <15218.61276.654235.714590@beluga.mojam.com> Message-ID: <20010809151743.G9751@inetnebr.com> On Thu, Aug 09, 2001 at 03:15:24PM -0500, Skip Montanaro wrote: > > Jeff> The real gain comes when you can hoist the type test out of the > Jeff> loop (though that means proving the type condition is an invariant > Jeff> of the loop!) > > Even more difficult if the programmer can subclass ints, yes?. Today, if I > see x=x/2 and I know x is an int, I can assume x/2 is also (or a float once > integer division returns floats). Once ints can be subclassed, a devious > programmer could return something other than an int (say, a float, a > rational, or the string "x/2"). Then the test becomes 'x.__class__ is int', not 'isinstance(x, int)'. Jeff From martin@loewis.home.cs.tu-berlin.de Thu Aug 9 21:12:39 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Thu, 9 Aug 2001 22:12:39 +0200 Subject: [Python-Dev] optimizing non-local object access Message-ID: <200108092012.f79KCdv00872@mira.informatik.hu-berlin.de> > I think we hook in at the tp_getattr(o) level. Module objects can > detect rebindings there and do whatever bookkeeping is necessary to > keep references to its name consistent. I think this is the right > approach for either technique we're discussing. Would that catch changes like math.__dict__['sin'] = math.cos ?? Regards, Martin From martin@loewis.home.cs.tu-berlin.de Thu Aug 9 21:21:15 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Thu, 9 Aug 2001 22:21:15 +0200 Subject: [Python-Dev] Any objections to adding threading.Timer? In-Reply-To: <20010809163328.6691299C86@waltz.rahul.net> (aahz@rahul.net) References: <20010809163328.6691299C86@waltz.rahul.net> Message-ID: <200108092021.f79KLFI00918@mira.informatik.hu-berlin.de> > I'm not objecting, precisely, but I have an allergic reaction to the way > timed Event() works, even though there's really no other way to work it. > I think that encouraging people to use it is a Bad Idea. Can you elaborate in more detail what your concern about the timed Event() is? This one itself looks fine; if anything, _Condition.wait looks disturbing. Could you also kindly put your comments into http://sourceforge.net/tracker/?group_id=5470&atid=305470&func=detail&aid=428326 There are a number of procedural objections with regard to this patch (license, missing documentation); if you can add significant technical objections, that might be a reason to reject it - or to find a better alternative. Regards, Martin From fdrake@acm.org Thu Aug 9 21:28:28 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 9 Aug 2001 16:28:28 -0400 (EDT) Subject: [Python-Dev] optimizing non-local object access In-Reply-To: <15218.61276.654235.714590@beluga.mojam.com> References: <200108091739.TAA10088@core.inf.ethz.ch> <15218.52521.776800.5873@slothrop.digicool.com> <15218.53480.204468.854961@beluga.mojam.com> <20010809150549.F9751@inetnebr.com> <15218.61276.654235.714590@beluga.mojam.com> Message-ID: <15218.62060.653961.301524@cj42289-a.reston1.va.home.com> Skip Montanaro writes: > see x=x/2 and I know x is an int, I can assume x/2 is also (or a float once > integer division returns floats). Once ints can be subclassed, a devious When division can return a float, it isn't integer division any more. ;-) We can always assume int//int is an integer, though. But only if we really have integers; subclasses are different. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From skip@pobox.com (Skip Montanaro) Thu Aug 9 21:34:46 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 9 Aug 2001 15:34:46 -0500 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: <3B72EF79.7BE8F6C1@metaslash.com> References: <3B72EF79.7BE8F6C1@metaslash.com> Message-ID: <15218.62438.445125.366624@beluga.mojam.com> Neal> PS I sent a patch to python-dev against 2.2a1, but it's 45k. So Neal> it's awating moderator approval. Sorry if this isn't the right Neal> venue for patches. Neal, What's the patch cover? Adding PyChecker to the core or something else? Skip From neal@metaslash.com Thu Aug 9 20:33:52 2001 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 09 Aug 2001 15:33:52 -0400 Subject: [Python-Dev] 2.2a1 patch from PyChecker Message-ID: <3B72E5A0.DC2E8C4E@metaslash.com> This is a multi-part message in MIME format. --------------1899DB72A2CEC0DFFD96771A Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Attached is a patch to address some problems found by PyChecker. Mostly unused imports/locals, etc. There were a few methods that were removed because they were duplicated (ConfigParser.py:has_option, gzip.py:writelines, locale.py:set_locale). I deleted the first occurrence, but I'm not sure that was always right. I have changed many unused locals to _ (e.g., for _ in range(3):). This gets PyChecker to shut up about being unused. It also defaults to ignoring unused local variables if their name is 'empty' or 'unused'. There are many more unused warnings. Does anybody want me to change the variables to empty, unused, _, or something different? Should the default unused names be expanded? Note: I could not run under Python 2.2a1 because dir([]) does not return the methods. Here are some other warnings I didn't fix because I wasn't sure what should be done: audiodev.py:214: No global (BUFFERSIZE) found (this is old, should BUFFERSIZE be defined, a better exception raised, and/or deprecation warning?) cgi.py:820: Overriden method (values) doesn't match signature in class (cgi.FormContentDict) ihooks.py:491: Overriden method (reload) doesn't match signature in class (ihooks.BasicModuleImporter) imaplib.py:1026: No global (j) found ntpath.py:411: No global (WindowsError) found (I ran on unix, not sure if this is ok on windows.) pdb.py:138: Overriden method (default) doesn't match signature in class (cmd.Cmd) profile.py:179: No global (operator) found -- Neal --------------1899DB72A2CEC0DFFD96771A Content-Type: text/plain; charset=us-ascii; name="py-2.2a1-patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="py-2.2a1-patch" diff -C 5 Lib.orig/ConfigParser.py Lib/ConfigParser.py *** Lib.orig/ConfigParser.py Sat Jul 14 03:47:34 2001 --- Lib/ConfigParser.py Mon Aug 6 20:26:14 2001 *************** *** 83,93 **** write(fp) write the configuration state in .ini format """ - import sys import string import re __all__ = ["NoSectionError","DuplicateSectionError","NoOptionError", "InterpolationError","InterpolationDepthError","ParsingError", --- 83,92 ---- *************** *** 149,174 **** % (section, option, rawval)) self.option = option self.section = section class ParsingError(Error): ! def __init__(self, filename): ! Error.__init__(self, 'File contains parsing errors: %s' % filename) self.filename = filename self.errors = [] def append(self, lineno, line): self.errors.append((lineno, line)) self._msg = self._msg + '\n\t[line %2d]: %s' % (lineno, line) class MissingSectionHeaderError(ParsingError): def __init__(self, filename, lineno, line): ! Error.__init__( ! self, 'File contains no section headers.\nfile: %s, line: %d\n%s' % (filename, lineno, line)) - self.filename = filename self.lineno = lineno self.line = line --- 148,174 ---- % (section, option, rawval)) self.option = option self.section = section class ParsingError(Error): ! def __init__(self, filename, msg=None): ! if msg is None: ! msg = 'File contains parsing errors: %s' % filename ! Error.__init__(self, msg) self.filename = filename self.errors = [] def append(self, lineno, line): self.errors.append((lineno, line)) self._msg = self._msg + '\n\t[line %2d]: %s' % (lineno, line) class MissingSectionHeaderError(ParsingError): def __init__(self, filename, lineno, line): ! ParsingError.__init__( ! self, filename, 'File contains no section headers.\nfile: %s, line: %d\n%s' % (filename, lineno, line)) self.lineno = lineno self.line = line *************** *** 213,226 **** raise NoSectionError(section) opts.update(self.__defaults) if opts.has_key('__name__'): del opts['__name__'] return opts.keys() - - def has_option(self, section, option): - """Return whether the given section has the given option.""" - return option in self.options(section) def read(self, filenames): """Read and parse a filename or a list of filenames. Files that cannot be opened are silently ignored; this is --- 213,222 ---- diff -C 5 Lib.orig/Cookie.py Lib/Cookie.py *** Lib.orig/Cookie.py Sat Jul 7 18:55:27 2001 --- Lib/Cookie.py Mon Aug 6 20:26:31 2001 *************** *** 213,223 **** # |----helps out font-lock # # Import our required modules # ! import string, sys from UserDict import UserDict try: from cPickle import dumps, loads except ImportError: --- 213,223 ---- # |----helps out font-lock # # Import our required modules # ! import string from UserDict import UserDict try: from cPickle import dumps, loads except ImportError: diff -C 5 Lib.orig/MimeWriter.py Lib/MimeWriter.py *** Lib.orig/MimeWriter.py Fri Feb 9 04:34:36 2001 --- Lib/MimeWriter.py Mon Aug 6 20:27:58 2001 *************** *** 103,122 **** self._headers = [] def startbody(self, ctype, plist=[], prefix=1): for name, value in plist: ctype = ctype + ';\n %s=\"%s\"' % (name, value) ! self.addheader("Content-Type", ctype, prefix=prefix) self.flushheaders() self._fp.write("\n") return self._fp def startmultipartbody(self, subtype, boundary=None, plist=[], prefix=1): self._boundary = boundary or mimetools.choose_boundary() return self.startbody("multipart/" + subtype, [("boundary", self._boundary)] + plist, ! prefix=prefix) def nextpart(self): self._fp.write("\n--" + self._boundary + "\n") return self.__class__(self._fp) --- 103,122 ---- self._headers = [] def startbody(self, ctype, plist=[], prefix=1): for name, value in plist: ctype = ctype + ';\n %s=\"%s\"' % (name, value) ! self.addheader("Content-Type", ctype, prefix) self.flushheaders() self._fp.write("\n") return self._fp def startmultipartbody(self, subtype, boundary=None, plist=[], prefix=1): self._boundary = boundary or mimetools.choose_boundary() return self.startbody("multipart/" + subtype, [("boundary", self._boundary)] + plist, ! prefix) def nextpart(self): self._fp.write("\n--" + self._boundary + "\n") return self.__class__(self._fp) diff -C 5 Lib.orig/StringIO.py Lib/StringIO.py *** Lib.orig/StringIO.py Fri Feb 9 08:37:37 2001 --- Lib/StringIO.py Mon Aug 6 20:30:17 2001 *************** *** 182,192 **** print 'File length =', length f.seek(len(lines[0])) f.write(lines[1]) f.seek(0) print 'First line =', `f.readline()` - here = f.tell() line = f.readline() print 'Second line =', `line` f.seek(-len(line), 1) line2 = f.read(len(line)) if line != line2: --- 182,191 ---- diff -C 5 Lib.orig/asyncore.py Lib/asyncore.py *** Lib.orig/asyncore.py Sat Jul 7 18:55:27 2001 --- Lib/asyncore.py Mon Aug 6 20:31:22 2001 *************** *** 151,161 **** map=socket_map # timeout is in milliseconds timeout = int(timeout*1000) pollster = select.poll() if map: - l = [] for fd, obj in map.items(): flags = 0 if obj.readable(): flags = select.POLLIN if obj.writable(): --- 151,160 ---- diff -C 5 Lib.orig/bdb.py Lib/bdb.py *** Lib.orig/bdb.py Sat Jul 7 18:55:27 2001 --- Lib/bdb.py Mon Aug 6 20:43:40 2001 *************** *** 216,226 **** if not self.breaks.has_key(filename): self.breaks[filename] = [] list = self.breaks[filename] if not lineno in list: list.append(lineno) ! bp = Breakpoint(filename, lineno, temporary, cond) def clear_break(self, filename, lineno): filename = self.canonic(filename) if not self.breaks.has_key(filename): return 'There are no breakpoints in %s' % filename --- 216,226 ---- if not self.breaks.has_key(filename): self.breaks[filename] = [] list = self.breaks[filename] if not lineno in list: list.append(lineno) ! _ = Breakpoint(filename, lineno, temporary, cond) def clear_break(self, filename, lineno): filename = self.canonic(filename) if not self.breaks.has_key(filename): return 'There are no breakpoints in %s' % filename diff -C 5 Lib.orig/cgi.py Lib/cgi.py *** Lib.orig/cgi.py Sat Jul 7 18:55:27 2001 --- Lib/cgi.py Mon Aug 6 20:47:55 2001 *************** *** 846,856 **** Write minimal HTTP headers and dump all information provided to the script in HTML form. """ - import traceback print "Content-type: text/html" print sys.stderr = sys.stdout try: form = FieldStorage() # Replace with other classes to test those --- 846,855 ---- diff -C 5 Lib.orig/codecs.py Lib/codecs.py *** Lib.orig/codecs.py Fri Jul 6 03:01:30 2001 --- Lib/codecs.py Mon Aug 6 20:48:48 2001 *************** *** 5,15 **** (c) Copyright CNRI, All Rights Reserved. NO WARRANTY. """#" ! import struct, types, __builtin__ ### Registry and builtin stateless codec functions try: from _codecs import * --- 5,15 ---- (c) Copyright CNRI, All Rights Reserved. NO WARRANTY. """#" ! import struct, __builtin__ ### Registry and builtin stateless codec functions try: from _codecs import * *************** *** 215,225 **** data = read(size) i = 0 while 1: try: object, decodedbytes = decode(data, self.errors) ! except ValueError, why: # This method is slow but should work under pretty much # all conditions; at most 10 tries are made i = i + 1 newdata = read(1) if not newdata or i > 10: --- 215,225 ---- data = read(size) i = 0 while 1: try: object, decodedbytes = decode(data, self.errors) ! except ValueError: # This method is slow but should work under pretty much # all conditions; at most 10 tries are made i = i + 1 newdata = read(1) if not newdata or i > 10: diff -C 5 Lib.orig/codeop.py Lib/codeop.py *** Lib.orig/codeop.py Sat Apr 7 20:43:13 2001 --- Lib/codeop.py Mon Aug 6 20:49:33 2001 *************** *** 50,65 **** if line and line[0] != '#': break # Leave it alone else: source = "pass" # Replace it with a 'pass' statement ! err = err1 = err2 = None code = code1 = code2 = None try: code = compile(source, filename, symbol) ! except SyntaxError, err: pass try: code1 = compile(source + "\n", filename, symbol) except SyntaxError, err1: --- 50,65 ---- if line and line[0] != '#': break # Leave it alone else: source = "pass" # Replace it with a 'pass' statement ! err1 = err2 = None code = code1 = code2 = None try: code = compile(source, filename, symbol) ! except SyntaxError: pass try: code1 = compile(source + "\n", filename, symbol) except SyntaxError, err1: diff -C 5 Lib.orig/fileinput.py Lib/fileinput.py *** Lib.orig/fileinput.py Sat Jan 20 18:34:12 2001 --- Lib/fileinput.py Mon Aug 6 20:53:16 2001 *************** *** 287,297 **** backup = 0 opts, args = getopt.getopt(sys.argv[1:], "ib:") for o, a in opts: if o == '-i': inplace = 1 if o == '-b': backup = a ! for line in input(args, inplace=inplace, backup=backup): if line[-1:] == '\n': line = line[:-1] if line[-1:] == '\r': line = line[:-1] print "%d: %s[%d]%s %s" % (lineno(), filename(), filelineno(), isfirstline() and "*" or "", line) print "%d: %s[%d]" % (lineno(), filename(), filelineno()) --- 287,297 ---- backup = 0 opts, args = getopt.getopt(sys.argv[1:], "ib:") for o, a in opts: if o == '-i': inplace = 1 if o == '-b': backup = a ! for line in input(args, inplace, backup): if line[-1:] == '\n': line = line[:-1] if line[-1:] == '\r': line = line[:-1] print "%d: %s[%d]%s %s" % (lineno(), filename(), filelineno(), isfirstline() and "*" or "", line) print "%d: %s[%d]" % (lineno(), filename(), filelineno()) diff -C 5 Lib.orig/formatter.py Lib/formatter.py *** Lib.orig/formatter.py Sat Jul 7 18:55:27 2001 --- Lib/formatter.py Mon Aug 6 20:53:57 2001 *************** *** 299,311 **** def send_literal_data(self, data): pass class AbstractWriter(NullWriter): - def __init__(self): - pass - def new_alignment(self, align): print "new_alignment(%s)" % `align` def new_font(self, font): print "new_font(%s)" % `font` --- 299,308 ---- diff -C 5 Lib.orig/ftplib.py Lib/ftplib.py *** Lib.orig/ftplib.py Mon Apr 9 00:31:50 2001 --- Lib/ftplib.py Mon Aug 6 20:54:20 2001 *************** *** 34,44 **** # Modified by Siebren to support docstrings and PASV. # import os import sys - import string # Import SOCKS module if it exists, else standard socket module socket try: import SOCKS; socket = SOCKS; del SOCKS # import SOCKS as socket from socket import getfqdn; socket.getfqdn = getfqdn; del getfqdn --- 34,43 ---- diff -C 5 Lib.orig/getpass.py Lib/getpass.py *** Lib.orig/getpass.py Tue Feb 27 16:23:31 2001 --- Lib/getpass.py Mon Aug 6 20:56:40 2001 *************** *** 24,34 **** try: fd = sys.stdin.fileno() except: return default_getpass(prompt) - getpass = default_getpass old = termios.tcgetattr(fd) # a copy to save new = old[:] new[3] = new[3] & ~termios.ECHO # 3 == 'lflags' try: --- 24,33 ---- *************** *** 68,78 **** def _raw_input(prompt=""): # A raw_input() replacement that doesn't save the string in the # GNU readline history. - import sys prompt = str(prompt) if prompt: sys.stdout.write(prompt) line = sys.stdin.readline() if not line: --- 67,76 ---- diff -C 5 Lib.orig/gettext.py Lib/gettext.py *** Lib.orig/gettext.py Tue Jan 23 10:35:04 2001 --- Lib/gettext.py Mon Aug 6 20:57:07 2001 *************** *** 156,166 **** msgcount &= MASK masteridx &= MASK transidx &= MASK # Now put all messages from the .mo file buffer into the catalog # dictionary. ! for i in xrange(0, msgcount): mlen, moff = unpack(ii, buf[masteridx:masteridx+8]) moff &= MASK mend = moff + (mlen & MASK) tlen, toff = unpack(ii, buf[transidx:transidx+8]) toff &= MASK --- 156,166 ---- msgcount &= MASK masteridx &= MASK transidx &= MASK # Now put all messages from the .mo file buffer into the catalog # dictionary. ! for _ in xrange(0, msgcount): mlen, moff = unpack(ii, buf[masteridx:masteridx+8]) moff &= MASK mend = moff + (mlen & MASK) tlen, toff = unpack(ii, buf[transidx:transidx+8]) toff &= MASK diff -C 5 Lib.orig/gopherlib.py Lib/gopherlib.py *** Lib.orig/gopherlib.py Fri Feb 9 05:10:02 2001 --- Lib/gopherlib.py Mon Aug 6 20:57:23 2001 *************** *** 167,177 **** import getopt opts, args = getopt.getopt(sys.argv[1:], '') selector = DEF_SELECTOR type = selector[0] host = DEF_HOST - port = DEF_PORT if args: host = args[0] args = args[1:] if args: type = args[0] --- 167,176 ---- diff -C 5 Lib.orig/gzip.py Lib/gzip.py *** Lib.orig/gzip.py Tue Mar 20 10:51:14 2001 --- Lib/gzip.py Mon Aug 6 21:00:00 2001 *************** *** 137,149 **** if len(data) > 0: self.size = self.size + len(data) self.crc = zlib.crc32(data, self.crc) self.fileobj.write( self.compress.compress(data) ) - def writelines(self,lines): - self.write(" ".join(lines)) - def read(self, size=-1): if self.extrasize <= 0 and self.fileobj is None: return '' readsize = 1024 --- 137,146 ---- *************** *** 271,281 **** return 0 def readline(self, size=-1): if size < 0: size = sys.maxint bufs = [] - orig_size = size readsize = min(100, size) # Read from the file in small chunks while 1: if size == 0: return "".join(bufs) # Return resulting line --- 268,277 ---- *************** *** 318,328 **** def _test(): # Act like gzip; with -d, act like gunzip. # The input file is not deleted, however, nor are any other gzip # options or features supported. - import sys args = sys.argv[1:] decompress = args and args[0] == "-d" if decompress: args = args[1:] if not args: --- 314,323 ---- diff -C 5 Lib.orig/httplib.py Lib/httplib.py *** Lib.orig/httplib.py Sat Jul 7 18:55:27 2001 --- Lib/httplib.py Mon Aug 6 21:01:20 2001 *************** *** 588,598 **** msgbuf = [] while 1: try: buf = self.__ssl.read() ! except socket.sslerror, msg: break if buf == '': break msgbuf.append(buf) return StringIO("".join(msgbuf)) --- 588,598 ---- msgbuf = [] while 1: try: buf = self.__ssl.read() ! except socket.sslerror: break if buf == '': break msgbuf.append(buf) return StringIO("".join(msgbuf)) diff -C 5 Lib.orig/ihooks.py Lib/ihooks.py *** Lib.orig/ihooks.py Fri Feb 9 05:17:30 2001 --- Lib/ihooks.py Mon Aug 6 21:03:51 2001 *************** *** 489,503 **** return m def reload(self, module): name = module.__name__ if '.' not in name: ! return self.import_it(name, name, None, force_load=1) i = name.rfind('.') pname = name[:i] parent = self.modules[pname] ! return self.import_it(name[i+1:], name, parent, force_load=1) default_importer = None current_importer = None --- 489,503 ---- return m def reload(self, module): name = module.__name__ if '.' not in name: ! return self.import_it(name, name, None, 1) i = name.rfind('.') pname = name[:i] parent = self.modules[pname] ! return self.import_it(name[i+1:], name, parent, 1) default_importer = None current_importer = None diff -C 5 Lib.orig/imaplib.py Lib/imaplib.py *** Lib.orig/imaplib.py Sat Jul 7 18:55:27 2001 --- Lib/imaplib.py Mon Aug 6 21:05:16 2001 *************** *** 778,788 **** # Instead, send me details of the unexpected response and # I'll update the code in `_get_response()'. try: self._get_response() ! except self.abort, val: if __debug__: if self.debug >= 1: print_log() raise --- 778,788 ---- # Instead, send me details of the unexpected response and # I'll update the code in `_get_response()'. try: self._get_response() ! except self.abort: if __debug__: if self.debug >= 1: print_log() raise *************** *** 1041,1051 **** if __name__ == '__main__': ! import getopt, getpass, sys try: optlist, args = getopt.getopt(sys.argv[1:], 'd:') except getopt.error, val: pass --- 1041,1051 ---- if __name__ == '__main__': ! import getopt, getpass try: optlist, args = getopt.getopt(sys.argv[1:], 'd:') except getopt.error, val: pass diff -C 5 Lib.orig/locale.py Lib/locale.py *** Lib.orig/locale.py Mon Apr 16 12:04:10 2001 --- Lib/locale.py Mon Aug 6 21:07:58 2001 *************** *** 63,80 **** 'mon_thousands_sep': '', 'frac_digits': 127, 'mon_decimal_point': '', 'int_frac_digits': 127} - def setlocale(category, value=None): - """ setlocale(integer,string=None) -> string. - Activates/queries locale processing. - """ - if value is not None and value != 'C': - raise Error, '_locale emulation only supports "C" locale' - return 'C' - def strcoll(a,b): """ strcoll(string,string) -> int. Compares two strings according to the locale. """ return cmp(a,b) --- 63,72 ---- *************** *** 266,278 **** code = normalize(localename) if '.' in code: return code.split('.')[:2] elif code == 'C': return None, None ! else: ! raise ValueError, 'unknown locale: %s' % localename ! return l def _build_localename(localetuple): """ Builds a locale code from the given tuple (language code, encoding). --- 258,268 ---- code = normalize(localename) if '.' in code: return code.split('.')[:2] elif code == 'C': return None, None ! raise ValueError, 'unknown locale: %s' % localename def _build_localename(localetuple): """ Builds a locale code from the given tuple (language code, encoding). diff -C 5 Lib.orig/mailbox.py Lib/mailbox.py *** Lib.orig/mailbox.py Sat Jul 7 18:55:27 2001 --- Lib/mailbox.py Mon Aug 6 21:08:39 2001 *************** *** 256,268 **** self.fp.seek(pos) return def _test(): - import time import sys - import os args = sys.argv[1:] if not args: for key in 'MAILDIR', 'MAIL', 'LOGNAME', 'USER': if os.environ.has_key(key): --- 256,266 ---- diff -C 5 Lib.orig/multifile.py Lib/multifile.py *** Lib.orig/multifile.py Sun Mar 11 21:56:15 2001 --- Lib/multifile.py Mon Aug 6 21:11:47 2001 *************** *** 25,36 **** it normally attempts in order to make seeks relative to the beginning of the current file part. This may be useful when using MultiFile with a non- seekable stream object. """ - import sys - __all__ = ["MultiFile","Error"] class Error(Exception): pass --- 25,34 ---- diff -C 5 Lib.orig/nturl2path.py Lib/nturl2path.py *** Lib.orig/nturl2path.py Sat Feb 17 22:30:53 2001 --- Lib/nturl2path.py Mon Aug 6 21:14:48 2001 *************** *** 40,50 **** becomes ///C|/foo/bar/spam.foo """ ! import string, urllib if not ':' in p: # No drive specifier, just convert slashes and quote the name if p[:2] == '\\\\': # path is something like \\host\path\on\remote\host # convert this to ////host/path/on/remote/host --- 40,50 ---- becomes ///C|/foo/bar/spam.foo """ ! import urllib if not ':' in p: # No drive specifier, just convert slashes and quote the name if p[:2] == '\\\\': # path is something like \\host\path\on\remote\host # convert this to ////host/path/on/remote/host diff -C 5 Lib.orig/pdb.py Lib/pdb.py *** Lib.orig/pdb.py Fri Feb 9 18:28:07 2001 --- Lib/pdb.py Mon Aug 6 21:15:22 2001 *************** *** 214,224 **** else: filename = f arg = arg[colon+1:].lstrip() try: lineno = int(arg) ! except ValueError, msg: print '*** Bad lineno:', arg return else: # no colon; can be lineno or function try: --- 214,224 ---- else: filename = f arg = arg[colon+1:].lstrip() try: lineno = int(arg) ! except ValueError: print '*** Bad lineno:', arg return else: # no colon; can be lineno or function try: diff -C 5 Lib.orig/pipes.py Lib/pipes.py *** Lib.orig/pipes.py Sat Jul 7 18:55:28 2001 --- Lib/pipes.py Mon Aug 6 21:16:37 2001 *************** *** 57,67 **** For an example, see the function test() at the end of the file. """ # ' - import sys import re import os import tempfile import string --- 57,66 ---- diff -C 5 Lib.orig/posixfile.py Lib/posixfile.py *** Lib.orig/posixfile.py Sat Jul 7 18:55:28 2001 --- Lib/posixfile.py Thu Aug 9 15:25:13 2001 *************** *** 105,115 **** posix.dup2(self._file_.fileno(), fd) return posix.fdopen(fd, self._file_.mode) def flags(self, *which): ! import fcntl if which: if len(which) > 1: raise TypeError, 'Too many arguments' which = which[0] --- 105,115 ---- posix.dup2(self._file_.fileno(), fd) return posix.fdopen(fd, self._file_.mode) def flags(self, *which): ! import fcntl, os if which: if len(which) > 1: raise TypeError, 'Too many arguments' which = which[0] *************** *** 142,152 **** if os.O_NDELAY & l_flags: which = which + 'n' if os.O_SYNC & l_flags: which = which + 's' return which def lock(self, how, *args): ! import struct, fcntl if 'w' in how: l_type = fcntl.F_WRLCK elif 'r' in how: l_type = fcntl.F_RDLCK elif 'u' in how: l_type = fcntl.F_UNLCK else: raise TypeError, 'no type of lock specified' --- 142,152 ---- if os.O_NDELAY & l_flags: which = which + 'n' if os.O_SYNC & l_flags: which = which + 's' return which def lock(self, how, *args): ! import struct, fcntl, os if 'w' in how: l_type = fcntl.F_WRLCK elif 'r' in how: l_type = fcntl.F_RDLCK elif 'u' in how: l_type = fcntl.F_UNLCK else: raise TypeError, 'no type of lock specified' diff -C 5 Lib.orig/profile.py Lib/profile.py *** Lib.orig/profile.py Sat Jul 7 18:55:28 2001 --- Lib/profile.py Mon Aug 6 21:18:22 2001 *************** *** 143,153 **** self.cur = None self.cmd = "" if not timer: if os.name == 'mac': - import MacOS self.timer = MacOS.GetTicks self.dispatcher = self.trace_dispatch_mac self.get_time = _get_time_mac elif hasattr(time, 'clock'): self.timer = self.get_time = time.clock --- 143,152 ---- diff -C 5 Lib.orig/pstats.py Lib/pstats.py *** Lib.orig/pstats.py Sat Jul 7 18:55:28 2001 --- Lib/pstats.py Mon Aug 6 21:19:33 2001 *************** *** 183,193 **** def get_sort_arg_defs(self): """Expand all abbreviations that are unique.""" if not self.sort_arg_dict: self.sort_arg_dict = dict = {} - std_list = dict.keys() bad_list = {} for word in self.sort_arg_dict_default.keys(): fragment = word while fragment: if not fragment: --- 183,192 ---- diff -C 5 Lib.orig/pyclbr.py Lib/pyclbr.py *** Lib.orig/pyclbr.py Sun Feb 11 21:00:42 2001 --- Lib/pyclbr.py Mon Aug 6 21:20:20 2001 *************** *** 51,61 **** It can't locate the parent. It probably needs to have the same hairy logic that the import locator already does. (This logic exists coded in Python in the freeze package.) """ - import os import sys import imp import re import string --- 51,60 ---- *************** *** 201,211 **** f.close() _modules[module] = dict return dict _modules[module] = dict - imports = [] classstack = [] # stack of (class, indent) pairs src = f.read() f.close() # To avoid having to stop the regexp at each newline, instead --- 200,209 ---- diff -C 5 Lib.orig/random.py Lib/random.py *** Lib.orig/random.py Thu Feb 15 18:56:39 2001 --- Lib/random.py Mon Aug 6 21:20:48 2001 *************** *** 582,592 **** sum = 0.0 sqsum = 0.0 smallest = 1e10 largest = -1e10 t0 = time.time() ! for i in range(n): x = eval(code) sum = sum + x sqsum = sqsum + x*x smallest = min(x, smallest) largest = max(x, largest) --- 582,592 ---- sum = 0.0 sqsum = 0.0 smallest = 1e10 largest = -1e10 t0 = time.time() ! for _ in range(n): x = eval(code) sum = sum + x sqsum = sqsum + x*x smallest = min(x, smallest) largest = max(x, largest) *************** *** 623,633 **** s = getstate() jumpahead(N) r1 = random() # now do it the slow way setstate(s) ! for i in range(N): random() r2 = random() if r1 != r2: raise ValueError("jumpahead test failed " + `(N, r1, r2)`) --- 623,633 ---- s = getstate() jumpahead(N) r1 = random() # now do it the slow way setstate(s) ! for _ in range(N): random() r2 = random() if r1 != r2: raise ValueError("jumpahead test failed " + `(N, r1, r2)`) diff -C 5 Lib.orig/rexec.py Lib/rexec.py *** Lib.orig/rexec.py Sat Jul 7 18:55:28 2001 --- Lib/rexec.py Mon Aug 6 21:22:08 2001 *************** *** 366,376 **** tr = None return ty, va, tr def test(): ! import sys, getopt, traceback opts, args = getopt.getopt(sys.argv[1:], 'vt:') verbose = 0 trusted = [] for o, a in opts: if o == '-v': --- 366,376 ---- tr = None return ty, va, tr def test(): ! import getopt, traceback opts, args = getopt.getopt(sys.argv[1:], 'vt:') verbose = 0 trusted = [] for o, a in opts: if o == '-v': *************** *** 387,397 **** r.modules['sys'].path.insert(0, "") fp = sys.stdin if args and args[0] != '-': try: fp = open(args[0]) ! except IOError, msg: print "%s: can't open file %s" % (sys.argv[0], `args[0]`) return 1 if fp.isatty(): print "*** RESTRICTED *** Python", sys.version print sys.copyright --- 387,397 ---- r.modules['sys'].path.insert(0, "") fp = sys.stdin if args and args[0] != '-': try: fp = open(args[0]) ! except IOError: print "%s: can't open file %s" % (sys.argv[0], `args[0]`) return 1 if fp.isatty(): print "*** RESTRICTED *** Python", sys.version print sys.copyright diff -C 5 Lib.orig/rfc822.py Lib/rfc822.py *** Lib.orig/rfc822.py Tue Jul 17 00:19:05 2001 --- Lib/rfc822.py Mon Aug 6 21:22:48 2001 *************** *** 631,641 **** elif self.field[self.pos] == '@': self.pos = self.pos + 1 expectroute = 1 elif self.field[self.pos] == ':': self.pos = self.pos + 1 - expectaddrspec = 1 else: adlist = self.getaddrspec() self.pos = self.pos + 1 break self.gotonext() --- 631,640 ---- diff -C 5 Lib.orig/robotparser.py Lib/robotparser.py *** Lib.orig/robotparser.py Thu Feb 15 18:56:39 2001 --- Lib/robotparser.py Mon Aug 6 21:22:59 2001 *************** *** 229,239 **** print "ok (%s)" % ac print def _test(): global debug - import sys rp = RobotFileParser() debug = 1 # robots.txt that exists, gotten to by redirection rp.set_url('http://www.musi-cal.com/robots.txt') --- 229,238 ---- diff -C 5 Lib.orig/sched.py Lib/sched.py *** Lib.orig/sched.py Thu Feb 15 17:15:13 2001 --- Lib/sched.py Mon Aug 6 21:23:12 2001 *************** *** 100,106 **** now = self.timefunc() if now < time: self.delayfunc(time - now) else: del q[0] ! void = apply(action, argument) self.delayfunc(0) # Let other threads run --- 100,106 ---- now = self.timefunc() if now < time: self.delayfunc(time - now) else: del q[0] ! _ = apply(action, argument) self.delayfunc(0) # Let other threads run diff -C 5 Lib.orig/sgmllib.py Lib/sgmllib.py *** Lib.orig/sgmllib.py Mon Jul 16 17:39:41 2001 --- Lib/sgmllib.py Mon Aug 6 21:23:43 2001 *************** *** 8,18 **** # and CDATA (character data -- only end tags are special). RCDATA is # not supported at all. import re - import string __all__ = ["SGMLParser"] # Regular expressions used for parsing --- 8,17 ---- diff -C 5 Lib.orig/smtpd.py Lib/smtpd.py *** Lib.orig/smtpd.py Sun Apr 15 09:06:04 2001 --- Lib/smtpd.py Mon Aug 6 21:25:08 2001 *************** *** 370,380 **** class MailmanProxy(PureProxy): def process_message(self, peer, mailfrom, rcpttos, data): from cStringIO import StringIO - import paths from Mailman import Utils from Mailman import Message from Mailman import MailList # If the message is to a Mailman mailing list, then we'll invoke the # Mailman script directly, without going through the real smtpd. --- 370,379 ---- diff -C 5 Lib.orig/smtplib.py Lib/smtplib.py *** Lib.orig/smtplib.py Thu Feb 15 17:15:13 2001 --- Lib/smtplib.py Mon Aug 6 21:25:37 2001 *************** *** 518,528 **** # Test the sendmail method, which tests most of the others. # Note: This always sends to localhost. if __name__ == '__main__': ! import sys, rfc822 def prompt(prompt): sys.stdout.write(prompt + ": ") return sys.stdin.readline().strip() --- 518,528 ---- # Test the sendmail method, which tests most of the others. # Note: This always sends to localhost. if __name__ == '__main__': ! import sys def prompt(prompt): sys.stdout.write(prompt + ": ") return sys.stdin.readline().strip() diff -C 5 Lib.orig/sunau.py Lib/sunau.py *** Lib.orig/sunau.py Sun Jan 14 20:36:40 2001 --- Lib/sunau.py Mon Aug 6 21:26:36 2001 *************** *** 131,141 **** class Error(Exception): pass def _read_u32(file): x = 0L ! for i in range(4): byte = file.read(1) if byte == '': raise EOFError x = x*256 + ord(byte) return x --- 131,141 ---- class Error(Exception): pass def _read_u32(file): x = 0L ! for _ in range(4): byte = file.read(1) if byte == '': raise EOFError x = x*256 + ord(byte) return x diff -C 5 Lib.orig/threading.py Lib/threading.py *** Lib.orig/threading.py Mon Apr 2 16:15:57 2001 --- Lib/threading.py Mon Aug 6 21:30:23 2001 *************** *** 542,553 **** # Self-test code def _test(): - import random - class BoundedQueue(_Verbose): def __init__(self, limit): _Verbose.__init__(self) self.mon = RLock() --- 542,551 ---- *************** *** 605,616 **** def run(self): while self.count > 0: item = self.queue.get() print item self.count = self.count - 1 - - import time NP = 3 QL = 4 NI = 5 --- 603,612 ---- diff -C 5 Lib.orig/toaiff.py Lib/toaiff.py *** Lib.orig/toaiff.py Wed Feb 28 23:27:19 2001 --- Lib/toaiff.py Mon Aug 6 21:31:07 2001 *************** *** 84,94 **** fname = filename try: ftype = sndhdr.whathdr(fname) if ftype: ftype = ftype[0] # All we're interested in ! except IOError: if type(msg) == type(()) and len(msg) == 2 and \ type(msg[0]) == type(0) and type(msg[1]) == type(''): msg = msg[1] if type(msg) != type(''): msg = `msg` --- 84,94 ---- fname = filename try: ftype = sndhdr.whathdr(fname) if ftype: ftype = ftype[0] # All we're interested in ! except IOError, msg: if type(msg) == type(()) and len(msg) == 2 and \ type(msg[0]) == type(0) and type(msg[1]) == type(''): msg = msg[1] if type(msg) != type(''): msg = `msg` diff -C 5 Lib.orig/token.py Lib/token.py *** Lib.orig/token.py Fri Feb 9 19:20:16 2001 --- Lib/token.py Mon Aug 6 21:31:28 2001 *************** *** 80,90 **** return x == ENDMARKER def main(): import re - import string import sys args = sys.argv[1:] inFileName = args and args[0] or "Include/token.h" outFileName = "Lib/token.py" if len(args) > 1: --- 80,89 ---- diff -C 5 Lib.orig/urllib2.py Lib/urllib2.py *** Lib.orig/urllib2.py Sat Jul 7 18:55:28 2001 --- Lib/urllib2.py Mon Aug 6 21:36:33 2001 *************** *** 86,96 **** # ftp errors aren't handled cleanly # gopher can return a socket.error # check digest against correct (i.e. non-apache) implementation import socket - import UserDict import httplib import re import base64 import types import urlparse --- 86,95 ---- *************** *** 435,445 **** newurl = headers['location'] elif headers.has_key('uri'): newurl = headers['uri'] else: return ! nil = fp.read() fp.close() newurl = urlparse.urljoin(req.get_full_url(), newurl) # XXX Probably want to forget about the state of the current --- 434,444 ---- newurl = headers['location'] elif headers.has_key('uri'): newurl = headers['uri'] else: return ! _ = fp.read() fp.close() newurl = urlparse.urljoin(req.get_full_url(), newurl) # XXX Probably want to forget about the state of the current *************** *** 449,459 **** new.error_302_dict = {} if hasattr(req, 'error_302_dict'): if len(req.error_302_dict)>10 or \ req.error_302_dict.has_key(newurl): raise HTTPError(req.get_full_url(), code, ! self.inf_msg + msg, headers) new.error_302_dict.update(req.error_302_dict) new.error_302_dict[newurl] = newurl return self.parent.open(new) http_error_301 = http_error_302 --- 448,458 ---- new.error_302_dict = {} if hasattr(req, 'error_302_dict'): if len(req.error_302_dict)>10 or \ req.error_302_dict.has_key(newurl): raise HTTPError(req.get_full_url(), code, ! self.inf_msg + msg, headers, None) new.error_302_dict.update(req.error_302_dict) new.error_302_dict[newurl] = newurl return self.parent.open(new) http_error_301 = http_error_302 *************** *** 658,668 **** class AbstractDigestAuthHandler: def __init__(self, passwd=None): if passwd is None: ! passwd = HTTPPassowrdMgr() self.passwd = passwd self.add_password = self.passwd.add_password self.__current_realm = None def http_error_auth_reqed(self, authreq, host, req, headers): --- 657,667 ---- class AbstractDigestAuthHandler: def __init__(self, passwd=None): if passwd is None: ! passwd = HTTPPasswordMgr() self.passwd = passwd self.add_password = self.passwd.add_password self.__current_realm = None def http_error_auth_reqed(self, authreq, host, req, headers): diff -C 5 Lib.orig/urlparse.py Lib/urlparse.py *** Lib.orig/urlparse.py Wed Feb 28 23:27:19 2001 --- Lib/urlparse.py Mon Aug 6 21:37:47 2001 *************** *** 53,63 **** cached = _parse_cache.get(key, None) if cached: return cached if len(_parse_cache) >= MAX_CACHE_SIZE: # avoid runaway growth clear_cache() ! netloc = path = params = query = fragment = '' i = url.find(':') if i > 0: if url[:i] == 'http': # optimize the common case scheme = url[:i].lower() url = url[i+1:] --- 53,63 ---- cached = _parse_cache.get(key, None) if cached: return cached if len(_parse_cache) >= MAX_CACHE_SIZE: # avoid runaway growth clear_cache() ! netloc = params = query = fragment = '' i = url.find(':') if i > 0: if url[:i] == 'http': # optimize the common case scheme = url[:i].lower() url = url[i+1:] diff -C 5 Lib.orig/warnings.py Lib/warnings.py *** Lib.orig/warnings.py Wed Feb 28 23:27:19 2001 --- Lib/warnings.py Mon Aug 6 21:38:04 2001 *************** *** 223,233 **** for item in filters: print item hello = "hello world" warn(hello); warn(hello); warn(hello); warn(hello) warn(hello, UserWarning) warn(hello, DeprecationWarning) ! for i in range(3): warn(hello) filterwarnings("error", "", Warning, "", 0) try: warn(hello) except Exception, msg: --- 223,233 ---- for item in filters: print item hello = "hello world" warn(hello); warn(hello); warn(hello); warn(hello) warn(hello, UserWarning) warn(hello, DeprecationWarning) ! for _ in range(3): warn(hello) filterwarnings("error", "", Warning, "", 0) try: warn(hello) except Exception, msg: diff -C 5 Lib.orig/webbrowser.py Lib/webbrowser.py *** Lib.orig/webbrowser.py Thu Apr 12 18:07:27 2001 --- Lib/webbrowser.py Mon Aug 6 21:39:15 2001 *************** *** 72,82 **** controller = copy.copy(controller) controller.name = browser controller.basename = os.path.basename(browser) register(browser, None, controller) return [None, controller] ! ret # # Everything after this point initializes _browsers and _tryorder, # then disappears. Some class definitions and instances remain # live through these globals, but only the minimum set needed to --- 72,82 ---- controller = copy.copy(controller) controller.name = browser controller.basename = os.path.basename(browser) register(browser, None, controller) return [None, controller] ! return [None, None] # # Everything after this point initializes _browsers and _tryorder, # then disappears. Some class definitions and instances remain # live through these globals, but only the minimum set needed to diff -C 5 Lib.orig/xdrlib.py Lib/xdrlib.py *** Lib.orig/xdrlib.py Wed Feb 28 23:27:19 2001 --- Lib/xdrlib.py Mon Aug 6 21:40:21 2001 *************** *** 212,222 **** list.append(item) return list def unpack_farray(self, n, unpack_item): list = [] ! for i in range(n): list.append(unpack_item()) return list def unpack_array(self, unpack_item): n = self.unpack_uint() --- 212,222 ---- list.append(item) return list def unpack_farray(self, n, unpack_item): list = [] ! for _ in range(n): list.append(unpack_item()) return list def unpack_array(self, unpack_item): n = self.unpack_uint() --------------1899DB72A2CEC0DFFD96771A-- From neal@metaslash.com Thu Aug 9 21:46:58 2001 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 09 Aug 2001 16:46:58 -0400 Subject: [Python-Dev] Direction of PyChecker References: <3B72EF79.7BE8F6C1@metaslash.com> <15218.62438.445125.366624@beluga.mojam.com> Message-ID: <3B72F6C2.72E2DF48@metaslash.com> Skip Montanaro wrote: > > Neal> PS I sent a patch to python-dev against 2.2a1, but it's 45k. So > Neal> it's awating moderator approval. Sorry if this isn't the right > Neal> venue for patches. > What's the patch cover? Adding PyChecker to the core or something else? The patch is for the standard modules in Python-2.2a1/Lib. I didn't run PyChecker on any subdirs under Lib. I should do that too. Neal From guido@python.org Thu Aug 9 21:52:26 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 09 Aug 2001 16:52:26 -0400 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: Your message of "Thu, 09 Aug 2001 16:15:53 EDT." <3B72EF79.7BE8F6C1@metaslash.com> References: <3B72EF79.7BE8F6C1@metaslash.com> Message-ID: <200108092052.f79KqQg13802@odiug.digicool.com> > I'd like to solicit feedback on what everyone thinks about PyChecker. I love it when it works! > Specifically, where should it go? What should it include or not include? > What should the default warnings be? You'll never satisfy everyone. Personally, I write too much code without docstrings, so I find the complaints about missing docstrings annoying. I guess my preference would be to have the default mode ony complain about things that have a high probability of being *coding* errors -- undefined names and the like. I forget if you're checking string formatting conformance yet -- that would also be useful. > I've seen some threads on c.l.p that checker might address. Unfortunately the typical newbie doesn't have access to pychecker yet... > Do you think that there should be something added to the python > FAQ/doc about checker? I'd be willing to write something. Sure! You can add a FAQ entry yourself -- the password is "Spam". > I'm close to exhausting my list of warnings/checks, let me know > if there are additional warnings/checks you can think of. Python version compatibilities would be an interesting area to explore. I would like PyChecker to use a fundamentally different way of scanning modules. I believe that it currently imports the module and then scans the module's __dict__ for classes and functions, and analyses their bytecode. That's fragile, dangerous (if you don't know what a module does you may not want to import it), and sometimes causes spurious errors, like when I have this at the global level: import foo foo.bar() and no other references to the module foo anywhere in the code... You get an error about an unused foo module. > Thanks for your input, > Neal > > PS I sent a patch to python-dev against 2.2a1, but it's 45k. So > it's awating moderator approval. Sorry if this isn't the right > venue for patches. I've approved it, but in general, we'd all prefer it if you submitted patches to the SourceForge patch manager -- that's what it's there for. If you want to discuss it here, post a URL. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Aug 9 22:20:20 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 09 Aug 2001 17:20:20 -0400 Subject: [Python-Dev] 2.2a1 patch from PyChecker In-Reply-To: Your message of "Thu, 09 Aug 2001 15:33:52 EDT." <3B72E5A0.DC2E8C4E@metaslash.com> References: <3B72E5A0.DC2E8C4E@metaslash.com> Message-ID: <200108092120.f79LKNo13875@odiug.digicool.com> > Attached is a patch to address some problems found by PyChecker. > Mostly unused imports/locals, etc. Thanks! > There were a few methods that were removed because they were duplicated > (ConfigParser.py:has_option, gzip.py:writelines, locale.py:set_locale). > I deleted the first occurrence, but I'm not sure that was always right. It'll take us some time to review all the patches. > I have changed many unused locals to _ (e.g., for _ in range(3):). > This gets PyChecker to shut up about being unused. It also defaults > to ignoring unused local variables if their name is 'empty' or 'unused'. Argh! I don't like using '_' for this, because '_' is already a special built-in in interactive mode. I don't even like the fact that PyChecker warns about unused loop control variables at all -- I see nothing wrong with code like for i in range(10): print "X" and I would rather not have to write it differently specifically to shut up PyChecker. (In general, I find it annoying when a tool like this requires one to change one's perfectly fine coding style because of a blind spot in the tool. :-) I'd be happy if there was an option "complain if a loop control variable is not used" but it was off by default. I > There are many more unused warnings. Does anybody want me to change > the variables to empty, unused, _, or something different? No. > Should the default unused names be expanded? I tend to use "dummy" when I need a dummy variable. But even in tuple unpacks, I prefer to give all variables meaningful names even if I don't use all. To me, this: firstname, lastname, address, phone = get_employee() is more readable than this: firstname, lastname, dummy, dummy = get_employee() > Note: I could not run under Python 2.2a1 because dir([]) does not > return the methods. But you can treat it as an instance, and get the methods by looking at [].__class__.__dict__. Read PEP 252 -- it explains how the introspection interface will be rationalized. > Here are some other warnings I didn't fix because I wasn't sure what > should be done: > > audiodev.py:214: No global (BUFFERSIZE) found > (this is old, should BUFFERSIZE be defined, a better exception > raised, and/or deprecation warning?) I still haven't figured that one out myself. :-( > cgi.py:820: Overriden method (values) doesn't match signature in > class (cgi.FormContentDict) I'm afraid that one will have to remain -- it exists for b/w compatibility (with 6 year old code :-). > ihooks.py:491: Overriden method (reload) doesn't match signature in > class (ihooks.BasicModuleImporter) This is intentional. > imaplib.py:1026: No global (j) found Huh? There's no 'j' on that line. > ntpath.py:411: No global (WindowsError) found > (I ran on unix, not sure if this is ok on windows.) Yes, this exception is unique to Windows. > pdb.py:138: Overriden method (default) doesn't match signature in > class (cmd.Cmd) This one I don't understand. BTW, it would be nice if you could also mention the file and line number of the other reference. > profile.py:179: No global (operator) found That looks like a bug in profile.py to me: it is clearly hoping for "import operator". :-) But in general: thanks for PyCecker, Neil, and thanks for the patch set! --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Thu Aug 9 22:53:15 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 9 Aug 2001 17:53:15 -0400 (EDT) Subject: [Python-Dev] Re: optimizing non-local object access In-Reply-To: <200108092012.f79KCdv00872@mira.informatik.hu-berlin.de> References: <200108092012.f79KCdv00872@mira.informatik.hu-berlin.de> Message-ID: <15219.1611.162859.963171@slothrop.digicool.com> It would because __dict__ wouldn't be a dictionary; it would be a dlict (dictionary-list hybrid). This is the one compatibility issue that has come up. Code that expects module's to have a real dictionary will break -- both in Python where it is probably quite rare and at the C level where some API calls require real dicts. The __dict__ object will implement the mapping protocol. It just won't be a dict. Jeremy From jeremy@zope.com Thu Aug 9 23:00:11 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 9 Aug 2001 18:00:11 -0400 (EDT) Subject: [Python-Dev] Direction of PyChecker In-Reply-To: <200108092052.f79KqQg13802@odiug.digicool.com> References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> Message-ID: <15219.2027.131396.703309@slothrop.digicool.com> >>>>> "GvR" == Guido van Rossum writes: >> Specifically, where should it go? What should it include or not >> include? What should the default warnings be? GvR> You'll never satisfy everyone. Personally, I write too much GvR> code without docstrings, so I find the complaints about missing GvR> docstrings annoying. I guess my preference would be to have GvR> the default mode ony complain about things that have a high GvR> probability of being *coding* errors -- undefined names and the GvR> like. I forget if you're checking string formatting GvR> conformance yet -- that would also be useful. It would be much more helpful if the default setting identified real bugs rather than coding style issues. The doc string one is probably the most glaring. Add a -Wall option that turns on everything to satisfy the pedants. By way of example, I used pychecker on urllib2 yesterday. It reported 78 bugs of which 3 were real. Based on that feedback, I wouldn't use the tool again, but would let other people sift through the noise and report the bugs back to me. Jeremy From guido@python.org Thu Aug 9 23:03:59 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 09 Aug 2001 18:03:59 -0400 Subject: [Python-Dev] Re: optimizing non-local object access In-Reply-To: Your message of "Thu, 09 Aug 2001 17:53:15 EDT." <15219.1611.162859.963171@slothrop.digicool.com> References: <200108092012.f79KCdv00872@mira.informatik.hu-berlin.de> <15219.1611.162859.963171@slothrop.digicool.com> Message-ID: <200108092204.f79M41S14253@odiug.digicool.com> [Jeremy] > It would because __dict__ wouldn't be a dictionary; it would be a > dlict (dictionary-list hybrid). This is the one compatibility issue > that has come up. Code that expects module's to have a real > dictionary will break -- both in Python where it is probably quite > rare and at the C level where some API calls require real dicts. The > __dict__ object will implement the mapping protocol. It just won't be > a dict. In 2.2, if you don't use __getitem__ for access-by-number, you could subclass dictionary and add a separate set of APIs to access the variables by number (and to assign numbers to them in the first place). Then it would still behave like a real dictionary enough to be usable for the PyDict_* C API functions. --Guido van Rossum (home page: http://www.python.org/~guido/) From akuchlin@mems-exchange.org Thu Aug 9 23:09:03 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Thu, 9 Aug 2001 18:09:03 -0400 Subject: [Python-Dev] 2.2a1 patch from PyChecker In-Reply-To: <200108092120.f79LKNo13875@odiug.digicool.com>; from guido@python.org on Thu, Aug 09, 2001 at 05:20:20PM -0400 References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108092120.f79LKNo13875@odiug.digicool.com> Message-ID: <20010809180903.A31343@ute.cnri.reston.va.us> On Thu, Aug 09, 2001 at 05:20:20PM -0400, Guido van Rossum wrote: >> cgi.py:820: Overriden method (values) doesn't match signature in >> class (cgi.FormContentDict) > >I'm afraid that one will have to remain -- it exists for b/w >compatibility (with 6 year old code :-). That reminds me: should we put warnings in cgi.py for instantiations of FormContent and other old classes, so we can delete them in Python 2.3? One problem: if a class doesn't have an __init__ method, subclasses that have their own __init__ methods won't call it. Maybe this could be done by adding 'self.warn()' to all the methods of an outdated class, and defining a warn() method that raises a warning. >> imaplib.py:1026: No global (j) found >Huh? There's no 'j' on that line. Bug in the 2.1 version; the CVS code has been changed. --amk From jeremy@zope.com Thu Aug 9 23:13:51 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 9 Aug 2001 18:13:51 -0400 (EDT) Subject: [Python-Dev] Re: optimizing non-local object access In-Reply-To: <200108092204.f79M41S14253@odiug.digicool.com> References: <200108092012.f79KCdv00872@mira.informatik.hu-berlin.de> <15219.1611.162859.963171@slothrop.digicool.com> <200108092204.f79M41S14253@odiug.digicool.com> Message-ID: <15219.2847.865125.543986@slothrop.digicool.com> Have I mentioned that I love this type-class unification work :-) Jeremy From mwh@python.net Thu Aug 9 23:36:15 2001 From: mwh@python.net (Michael Hudson) Date: 09 Aug 2001 18:36:15 -0400 Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12 In-Reply-To: Guido van Rossum's message of "Wed, 08 Aug 2001 11:31:29 -0400" References: <200107271949.PAA27171@cj20424-a.reston1.va.home.com> <2mn15pwg56.fsf@starship.python.net> <200107281357.JAA30859@cj20424-a.reston1.va.home.com> <2m3d7emged.fsf@starship.python.net> <2mvgjy29tq.fsf@starship.python.net> <200108081531.LAA00619@cj20424-a.reston1.va.home.com> Message-ID: <2mg0b0uahs.fsf@starship.python.net> Guido van Rossum writes: > Hi Michael, > > Barry's on vacation, so I've done the honors. It's checked in as PEP > 264. Cool, thanks. > I like this a lot -- I hope we can add this to 2.2a2 (which was due > today but is postponed for at least a week). Well, I'm off sunseeking in a week; I'd hope it can get it done by then. > I think the integral flags to compile() are fine, Noted. > and 'or' is indeed the right thing. Hmm. While I agree that 'or'ing is the best thing to do by default, it would be nice if there was some way of requesting a blank starting point (this is the other unresolved issue in PEP 236, after all). Maybe it's not worth it. > After reading the PEP, I understand the refactoring that I > complained about in a comment on the patch. OK. > Good job! It's not done yet (and I thought this was a dead simple change!). I've uploaded a new patch, which takes Tim's suggestion of using the _Feature objects and runs with it. I'd appreciate it if you and Tim could cast an eye over it - if you think the apprach is sound then I'll update my PEP and draft some changes to 236. Cheers, M. -- Presumably pronging in the wrong place zogs it. -- Aldabra Stoddart, ucam.chat From fdrake@acm.org Thu Aug 9 23:58:07 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 9 Aug 2001 18:58:07 -0400 (EDT) Subject: [Python-Dev] Re: optimizing non-local object access In-Reply-To: <15219.1611.162859.963171@slothrop.digicool.com> References: <200108092012.f79KCdv00872@mira.informatik.hu-berlin.de> <15219.1611.162859.963171@slothrop.digicool.com> Message-ID: <15219.5503.655419.896502@cj42289-a.reston1.va.home.com> Jeremy Hylton writes: > that has come up. Code that expects module's to have a real > dictionary will break -- both in Python where it is probably quite > rare and at the C level where some API calls require real dicts. The > __dict__ object will implement the mapping protocol. It just won't be > a dict. This would certainly break many of the module initialization functions. Sounds like everyone should consider using PyModule_Add*() there. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Fri Aug 10 01:34:56 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 09 Aug 2001 20:34:56 -0400 Subject: [Python-Dev] 2.2a1 patch from PyChecker In-Reply-To: Your message of "Thu, 09 Aug 2001 18:09:03 EDT." <20010809180903.A31343@ute.cnri.reston.va.us> References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108092120.f79LKNo13875@odiug.digicool.com> <20010809180903.A31343@ute.cnri.reston.va.us> Message-ID: <200108100034.UAA15749@cj20424-a.reston1.va.home.com> > >> cgi.py:820: Overriden method (values) doesn't match signature in > >> class (cgi.FormContentDict) > > > >I'm afraid that one will have to remain -- it exists for b/w > >compatibility (with 6 year old code :-). > > That reminds me: should we put warnings in cgi.py for instantiations > of FormContent and other old classes, so we can delete them in Python > 2.3? Sure, but if I believe what folks on c.l.py say about the habits of the average CGI coder, they'll probably have to remain until python 3.0 -- few will bother to check the warnings. :-) / :-( > One problem: if a class doesn't have an __init__ method, > subclasses that have their own __init__ methods won't call it. Maybe > this could be done by adding 'self.warn()' to all the methods of an > outdated class, and defining a warn() method that raises a warning. Good idea! --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Aug 10 01:38:28 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 09 Aug 2001 20:38:28 -0400 Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12 In-Reply-To: Your message of "09 Aug 2001 18:36:15 EDT." <2mg0b0uahs.fsf@starship.python.net> References: <200107271949.PAA27171@cj20424-a.reston1.va.home.com> <2mn15pwg56.fsf@starship.python.net> <200107281357.JAA30859@cj20424-a.reston1.va.home.com> <2m3d7emged.fsf@starship.python.net> <2mvgjy29tq.fsf@starship.python.net> <200108081531.LAA00619@cj20424-a.reston1.va.home.com> <2mg0b0uahs.fsf@starship.python.net> Message-ID: <200108100038.UAA15791@cj20424-a.reston1.va.home.com> > I've uploaded a new patch, which takes Tim's suggestion of using the > _Feature objects and runs with it. I'd appreciate it if you and Tim > could cast an eye over it - if you think the apprach is sound then > I'll update my PEP and draft some changes to 236. I hope Tim can do this -- I need to take a break from chores. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Fri Aug 10 02:47:28 2001 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 9 Aug 2001 21:47:28 -0400 Subject: [Python-Dev] Free threading (and tcl upvar) References: <200108091522.f79FMja02243@odiug.digicool.com> Message-ID: <15219.15664.450890.783449@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> That's why we have sys._getframe(). ;-) btw, when you need it, sys._getframe() is a godsend. thank the genius who came up with that one . i've been using it in mailman in my _() wrapper (for marking translatable strings), and because you have to use %()s named interpolation instead of %s positional interpolation, using _getframe() to "get at" the locals one frame up makes the call site very clean and compact, with (imo) no loss of clarity or feeling of "too much magic". i'm planning on writing a paper for ipcx on this stuff. -barry From jeremy@zope.com Fri Aug 10 03:04:11 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 9 Aug 2001 22:04:11 -0400 (EDT) Subject: [Python-Dev] Free threading (and tcl upvar) In-Reply-To: <15219.15664.450890.783449@anthem.wooz.org> References: <200108091522.f79FMja02243@odiug.digicool.com> <15219.15664.450890.783449@anthem.wooz.org> Message-ID: <15219.16667.375672.59499@slothrop.digicool.com> >>>>> "BAW" == Barry A Warsaw writes: >>>>> "GvR" == Guido van Rossum writes: GvR> That's why we have sys._getframe(). ;-) BAW> btw, when you need it, sys._getframe() is a godsend. thank the BAW> genius who came up with that one . It takes all kinds, doesn't it? For every genius who comes up with something like sys._getframe(), we have some fool who comes up with something like "print >>". Jeremy From martin@loewis.home.cs.tu-berlin.de Fri Aug 10 07:00:46 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Fri, 10 Aug 2001 08:00:46 +0200 Subject: [Python-Dev] optimizing non-local object access Message-ID: <200108100600.f7A60kD01107@mira.informatik.hu-berlin.de> [Martin] > Would that catch changes like [accessing __dict__] [Jeremy] > It would because __dict__ wouldn't be a dictionary [Guido] > In 2.2, if you don't use __getitem__ for access-by-number, you could > subclass dictionary and add a separate set of APIs to access the > variables by number (and to assign numbers to them in the first > place). Then it would still behave like a real dictionary enough to > be usable for the PyDict_* C API functions. But then, changes that happen through PyDict_SetItem would not be tracked for the dlict objects, would they (assuming that the dlict objects keep an additional array per dictionary which needs to be updated)? Regards, Martin From tim.one@home.com Fri Aug 10 07:10:25 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 10 Aug 2001 02:10:25 -0400 Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12 In-Reply-To: <2mg0b0uahs.fsf@starship.python.net> Message-ID: [Michael Hudson] > Hmm. While I agree that 'or'ing is the best thing to do by default, > it would be nice if there was some way of requesting a blank starting > point (this is the other unresolved issue in PEP 236, after all). > Maybe it's not worth it. Well, there's no reason at all to believe that whatever future statements IDLE and doctest (for examples) happen to use in their own implementations are also appropriate for the user-code they're simulating. So the problem isn't solved in full unless that connection can be broken (is that hard? offhand it *sounds* like it just needs another yes/no argument). OTOH, IDLE and doctest (for examples) can easily enough be written to use no future-stmts at all of their own, so that code compiled from them gets a blank starting point. Whether that remains easy down the road depends on how silly we get in introducing stupid future stmts <0.9 wink>. From tim.one@home.com Fri Aug 10 07:34:04 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 10 Aug 2001 02:34:04 -0400 Subject: [Python-Dev] BEGIN_ALLOW_THREADS In-Reply-To: <20010809082856.A16148@strakt.com> Message-ID: [Guido] > Instead of asking us to add more macros, I recommend that you use the > lower-level C API that is used by the macros. There's a bunch of ways > to save and restore the state. The macros are just shortcuts (and a > handy way to make all the code disappear when threads don't exist). [Martin Sjögren] > Well, that's exactly what I'm doing, isn't it? You are using macros, yes . What Guido is telling you is that the set of macros we already have is adequate for everything we do, so it would just bloat our code base with stuff we never ever test if we were add additional stuff we never use. He's also telling you that you can write your macros in such a way that you don't have to change them across Python releases, and it was *that* complaint you seemed keenest about. That one you can solve on your own. From mwh@python.net Fri Aug 10 09:17:13 2001 From: mwh@python.net (Michael Hudson) Date: 10 Aug 2001 04:17:13 -0400 Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12 In-Reply-To: "Tim Peters"'s message of "Fri, 10 Aug 2001 02:10:25 -0400" References: Message-ID: <2m8zgsfhx2.fsf@starship.python.net> "Tim Peters" writes: > [Michael Hudson] > > Hmm. While I agree that 'or'ing is the best thing to do by default, > > it would be nice if there was some way of requesting a blank starting > > point (this is the other unresolved issue in PEP 236, after all). > > Maybe it's not worth it. > > Well, there's no reason at all to believe that whatever future statements > IDLE and doctest (for examples) happen to use in their own implementations > are also appropriate for the user-code they're simulating. So the problem > isn't solved in full unless that connection can be broken (is that hard? > offhand it *sounds* like it just needs another yes/no argument). It's a matter of interface, really. It's certainly not at all technically hard. Maybe: compile(text, filename, symbol[, flags[, dont_inherit]]) I worry that saying "you don't inherit behaviour from surrounding code as soon as you pass a flag" might get really, really confusing at times. > OTOH, IDLE and doctest (for examples) can easily enough be written > to use no future-stmts at all of their own, so that code compiled > from them gets a blank starting point. Whether that remains easy > down the road depends on how silly we get in introducing stupid > future stmts <0.9 wink>. This is indeed the issue. we-need-a-tritfield-ly y'rs M. -- Important data should not be entrusted to Pinstripe, as it may eat it and make loud belching noises. -- from the announcement of the beta of "Pinstripe" aka. Redhat 7.0 From jack@oratrix.nl Fri Aug 10 10:23:38 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri, 10 Aug 2001 11:23:38 +0200 Subject: [Python-Dev] struct initializer in gcmodule.c - Allowed? Message-ID: <20010810092338.CF134303181@snelboot.oratrix.nl> A recent change to gcmodule.c (function gc_referents_for()) gave it a structure initializer for an auto variable: struct referents refs = {objs, 0}; My compiler chokes on this, and my memory of the C standard also says this is not allowed. But then, my memory may have been superseded by more recent C standards... -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From guido@python.org Fri Aug 10 14:49:12 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 10 Aug 2001 09:49:12 -0400 Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12 In-Reply-To: Your message of "10 Aug 2001 04:17:13 EDT." <2m8zgsfhx2.fsf@starship.python.net> References: <2m8zgsfhx2.fsf@starship.python.net> Message-ID: <200108101349.JAA16834@cj20424-a.reston1.va.home.com> > It's a matter of interface, really. It's certainly not at all > technically hard. Maybe: > > compile(text, filename, symbol[, flags[, dont_inherit]]) Occam sez: let's add the dont_inherit argument when we have found a real use for it. My bigger worry about this interface is that the flags accepted should be carefully checked to be from the small set related to future statements. It would be harmful if the user could set flags like CO_OPTIMIZED, CO_GENERATOR, or CO_VARARGS this way! --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Fri Aug 10 15:04:04 2001 From: nas@python.ca (Neil Schemenauer) Date: Fri, 10 Aug 2001 07:04:04 -0700 Subject: [Python-Dev] struct initializer in gcmodule.c - Allowed? In-Reply-To: <20010810092338.CF134303181@snelboot.oratrix.nl>; from jack@oratrix.nl on Fri, Aug 10, 2001 at 11:23:38AM +0200 References: <20010810092338.CF134303181@snelboot.oratrix.nl> Message-ID: <20010810070404.A6727@glacier.fnational.com> Jack Jansen wrote: > A recent change to gcmodule.c (function gc_referents_for()) gave it a > structure initializer for an auto variable: > struct referents refs = {objs, 0}; > > My compiler chokes on this, and my memory of the C standard also says this is > not allowed. But then, my memory may have been superseded by more recent C > standards... I'm working on fixing this. That structure is not really needed so I'm removing it. Neil From mwh@python.net Fri Aug 10 15:08:50 2001 From: mwh@python.net (Michael Hudson) Date: 10 Aug 2001 10:08:50 -0400 Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12 In-Reply-To: Guido van Rossum's message of "Fri, 10 Aug 2001 09:49:12 -0400" References: <2m8zgsfhx2.fsf@starship.python.net> <200108101349.JAA16834@cj20424-a.reston1.va.home.com> Message-ID: <2mwv4c2ej1.fsf@starship.python.net> Guido van Rossum writes: > > It's a matter of interface, really. It's certainly not at all > > technically hard. Maybe: > > > > compile(text, filename, symbol[, flags[, dont_inherit]]) > > Occam sez: let's add the dont_inherit argument when we have found a > real use for it. Fair enough. > My bigger worry about this interface is that the flags accepted should > be carefully checked to be from the small set related to future > statements. It would be harmful if the user could set flags like > CO_OPTIMIZED, CO_GENERATOR, or CO_VARARGS this way! Wrong set of flags! There are two complementary sets of flags here: (1) The PyCF_* ones, defined in Include/pythonrun.h (2) The CO_* ones defined in Include/compile.h The proposed fourth argument to compile() should be a combination of the former set. I only use the latter to tell whether a __future__ statement was used in the text compiled (which is a bit horrible, but no better way springs to mind). There might be value in checking the flags passed to compile() anyway, but I can't see it being dangerous. Cheers, M. -- You owe The Oracle a TV with an 'intelligence' control - I've tried 'brightness' but that didn't work. -- Internet Oracularity #1192-01 From guido@python.org Fri Aug 10 15:19:58 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 10 Aug 2001 10:19:58 -0400 Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12 In-Reply-To: Your message of "10 Aug 2001 10:08:50 EDT." <2mwv4c2ej1.fsf@starship.python.net> References: <2m8zgsfhx2.fsf@starship.python.net> <200108101349.JAA16834@cj20424-a.reston1.va.home.com> <2mwv4c2ej1.fsf@starship.python.net> Message-ID: <200108101419.KAA17114@cj20424-a.reston1.va.home.com> > > My bigger worry about this interface is that the flags accepted should > > be carefully checked to be from the small set related to future > > statements. It would be harmful if the user could set flags like > > CO_OPTIMIZED, CO_GENERATOR, or CO_VARARGS this way! > > Wrong set of flags! > > There are two complementary sets of flags here: > > (1) The PyCF_* ones, defined in Include/pythonrun.h > (2) The CO_* ones defined in Include/compile.h > > The proposed fourth argument to compile() should be a combination of > the former set. > > I only use the latter to tell whether a __future__ statement was used > in the text compiled (which is a bit horrible, but no better way > springs to mind). > > There might be value in checking the flags passed to compile() anyway, > but I can't see it being dangerous. Ah, good. Makes sense. (I think I may have been confused by your original patch, which AFAICR *did* manipulate the co_flags field of the code object, which contains the CO_* flags. --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Fri Aug 10 16:05:08 2001 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 10 Aug 2001 11:05:08 -0400 Subject: [Python-Dev] Direction of PyChecker References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> Message-ID: <3B73F824.82A53539@metaslash.com> Guido van Rossum wrote: > You'll never satisfy everyone. Personally, I write too much code > without docstrings, so I find the complaints about missing docstrings > annoying. I guess my preference would be to have the default mode ony > complain about things that have a high probability of being *coding* > errors -- undefined names and the like. I forget if you're checking > string formatting conformance yet -- that would also be useful. I know I can't satisfy everyone. I'm not even sure who PyChecker should be geared to by default--newbie, intermediate, or advanced user. A bunch of the warnings annoy me too. :-) ISTM that it probably should be geared to newbies by default. Since, they are the ones that would likely benefit the most. It's also easier for more advanced users to change defaults. I don't really want to do turn on all the warnings, even though I've tended towards that. I'm trying to figure out what would be best. There is a single option -e/--errors that will disable all the doc string, style, and code complexity warnings. I think there's about 40 config options. So it can be configured, but would take a little work. There is checks for format strings. It should be pretty good now. > Unfortunately the typical newbie doesn't have access to pychecker > yet... True, but that can change. :-) > Sure! You can add a FAQ entry yourself -- the password is "Spam". Added 4.101. Pretty short, let me know if you have comments. > Python version compatibilities would be an interesting area to > explore. Just starting to do that. There is a check for integer division now. I don't really know of version incompatibilities, other than new features such as methods on strings, list comprehension, etc. Right now string methods are checked based on the interpreter running. So if you run PyChecker in 1.5.2 and do ''.strip(), it should complain. If you run PyChecker in 2.0+ and do ''.strip(), it should NOT complain. > I would like PyChecker to use a fundamentally different way of > scanning modules. I believe that it currently imports the module and > then scans the module's __dict__ for classes and functions, and > analyses their bytecode. That's fragile, dangerous (if you don't know > what a module does you may not want to import it), and sometimes > causes spurious errors, like when I have this at the global level: Yes, this is one of the worst problems I know of. I plan on changing this. Neal From Samuele Pedroni Fri Aug 10 16:16:07 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Fri, 10 Aug 2001 17:16:07 +0200 (MET DST) Subject: [Python-Dev] Direction of PyChecker Message-ID: <200108101516.RAA15086@core.inf.ethz.ch> Hi. [Neal Norwitz] > > > I would like PyChecker to use a fundamentally different way of > > scanning modules. I believe that it currently imports the module and > > then scans the module's __dict__ for classes and functions, and > > analyses their bytecode. That's fragile, dangerous (if you don't know > > what a module does you may not want to import it), and sometimes > > causes spurious errors, like when I have this at the global level: > > Yes, this is one of the worst problems I know of. I plan on changing this. > Let me know what are your plans about that, will you stick with some bytecode based approach? I'm consindering the possibility of adding a special mode to jython (based on hooks and on porting the CPython tools/compiler) in which you can retrieve the PVM bytecode of code objects, then there would be still to make PyChecker java classes aware, but it could run under jython. (Don't know when I will get at doing this) It seems a nice tool, I would like the jython users to be able to use it too. regards, Samuele Pedroni. From guido@python.org Fri Aug 10 16:15:14 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 10 Aug 2001 11:15:14 -0400 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: Your message of "Fri, 10 Aug 2001 11:05:08 EDT." <3B73F824.82A53539@metaslash.com> References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <3B73F824.82A53539@metaslash.com> Message-ID: <200108101515.LAA18907@cj20424-a.reston1.va.home.com> > > I would like PyChecker to use a fundamentally different way of > > scanning modules. I believe that it currently imports the module and > > then scans the module's __dict__ for classes and functions, and > > analyses their bytecode. That's fragile, dangerous (if you don't know > > what a module does you may not want to import it), and sometimes > > causes spurious errors, like when I have this at the global level: > > Yes, this is one of the worst problems I know of. I plan on changing this. If you don't want to give up scanning the bytecode (if that's what you do), you could use compile() on the source code to get the bytecode for the entire module. --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Fri Aug 10 16:17:56 2001 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 10 Aug 2001 11:17:56 -0400 Subject: [Python-Dev] 2.2a1 patch from PyChecker References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108092120.f79LKNo13875@odiug.digicool.com> Message-ID: <3B73FB24.369B5EFD@metaslash.com> Guido van Rossum wrote: > > I have changed many unused locals to _ (e.g., for _ in range(3):). > > This gets PyChecker to shut up about being unused. It also defaults > > to ignoring unused local variables if their name is 'empty' or 'unused'. > > Argh! I don't like using '_' for this, because '_' is already a > special built-in in interactive mode. I don't even like the fact that > PyChecker warns about unused loop control variables at all -- I see > nothing wrong with code like > > for i in range(10): print "X" > > and I would rather not have to write it differently specifically to > shut up PyChecker. (In general, I find it annoying when a tool like > this requires one to change one's perfectly fine coding style because > of a blind spot in the tool. :-) Agreed. I try to warn about things that people may care about. > I'd be happy if there was an option "complain if a loop control > variable is not used" but it was off by default. I looked at adding this check, but decided against it initially. I'll think about adding it. > > Should the default unused names be expanded? > > I tend to use "dummy" when I need a dummy variable. But even in tuple I added 'dummy' as another variable name to ignore if unused. > unpacks, I prefer to give all variables meaningful names even if I > don't use all. To me, this: > > firstname, lastname, address, phone = get_employee() > > is more readable than this: > > firstname, lastname, dummy, dummy = get_employee() I agree, in fact, the first line with real names, does not warn by default. There is an option to warn if address and phone are unused, -t/--tuple, but it must be manually enabled. > > Note: I could not run under Python 2.2a1 because dir([]) does not > > return the methods. > > But you can treat it as an instance, and get the methods by looking at > [].__class__.__dict__. Read PEP 252 -- it explains how the > introspection interface will be rationalized. Ok, I'll take a look. > > imaplib.py:1026: No global (j) found > > Huh? There's no 'j' on that line. I have: 1025: l = map(lambda x:'%s: "%s"' % (x[0], x[1][0] and '" "'.join(x[1]) or '' ), l) 1026: _mesg('untagged responses dump:%s%s' % (t, j(l, t))) ^^^^^^^ > > pdb.py:138: Overriden method (default) doesn't match signature in > > class (cmd.Cmd) > > This one I don't understand. BTW, it would be nice if you could also > mention the file and line number of the other reference. Good idea. > But in general: thanks for PyCecker, Neil, and thanks for the patch > set! Thanks for the feedback and feel free to make suggestions. PyChecker is still an experiment at this point, but it seems that many people have found it useful. My goal is to make it as useful as possible and help promote the use of python (or eliminate bogus complaints about python, like type checking, etc). Neal From guido@python.org Fri Aug 10 16:19:15 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 10 Aug 2001 11:19:15 -0400 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: Your message of "Fri, 10 Aug 2001 17:16:07 +0200." <200108101516.RAA15086@core.inf.ethz.ch> References: <200108101516.RAA15086@core.inf.ethz.ch> Message-ID: <200108101519.LAA18940@cj20424-a.reston1.va.home.com> > I'm consindering the possibility of adding a special mode to jython > (based on hooks and on porting the CPython tools/compiler) in which > you can retrieve the PVM bytecode of code objects, then there would > be still to make PyChecker java classes aware, but it could run > under jython. (Don't know when I will get at doing this) But is it really a good idea to codify PVM bytecode as the way to do introspection? I'd much rather have a standard library API that parses a module and gives you a list of variables, functions, classes, etc., and lets you drill down into the symbol table of each of those. (I realize more is needed, you need to be able to inspect every statement too -- that can be done by providing access to an AST.) --Guido van Rossum (home page: http://www.python.org/~guido/) From cgw@alum.mit.edu Fri Aug 10 16:21:58 2001 From: cgw@alum.mit.edu (Charles G Waldman) Date: Fri, 10 Aug 2001 10:21:58 -0500 Subject: [Python-Dev] ++x oddnes Message-ID: <15219.64534.411625.193775@nyx.dyndns.org> Sorry if this is old news. I don't remember anybody reporting this, but I don't follow c.l.py as closely as I used to. I've been writing a lot of C++ lately and my mind is stuck in limbo somewhere between C++ and Python. I inadvertently wrote the following: x = 0 while x < 10: # do something ++x Interestingly, this runs, it just never terminates! Looks like somehow ++x is being parsed as "+0 + x". Shouldn't this construct be flagged as a syntax error? This happens with both 2.1 and 2.2 From guido@python.org Fri Aug 10 16:23:37 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 10 Aug 2001 11:23:37 -0400 Subject: [Python-Dev] ++x oddnes In-Reply-To: Your message of "Fri, 10 Aug 2001 10:21:58 CDT." <15219.64534.411625.193775@nyx.dyndns.org> References: <15219.64534.411625.193775@nyx.dyndns.org> Message-ID: <200108101523.LAA19012@cj20424-a.reston1.va.home.com> > Sorry if this is old news. I don't remember anybody reporting this, > but I don't follow c.l.py as closely as I used to. It's new to me. :-) > I've been writing a lot of C++ lately and my mind is stuck in limbo > somewhere between C++ and Python. I inadvertently wrote the following: > > x = 0 > while x < 10: > # do something > ++x > > Interestingly, this runs, it just never terminates! Looks like > somehow ++x is being parsed as "+0 + x". Shouldn't this construct > be flagged as a syntax error? > > This happens with both 2.1 and 2.2 IMO not worth it changing the language. Train yourself to write x++ instead. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Fri Aug 10 16:38:48 2001 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 10 Aug 2001 11:38:48 -0400 Subject: [Python-Dev] Direction of PyChecker References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <15219.2027.131396.703309@slothrop.digicool.com> Message-ID: <3B740008.2CF200C2@metaslash.com> Jeremy Hylton wrote: > It would be much more helpful if the default setting identified real > bugs rather than coding style issues. The doc string one is probably > the most glaring. Add a -Wall option that turns on everything to > satisfy the pedants. This is generally true for more advanced users, but not necessarily for newbies. There is the -e/--errors which eliminates many warnings. > By way of example, I used pychecker on urllib2 yesterday. It reported > 78 bugs of which 3 were real. Based on that feedback, I wouldn't use > the tool again, but would let other people sift through the noise and > report the bugs back to me. I ran checker w/--errors on urllib2. There were some warnings which should have been excluded--I'll have to fix those. But of the 12 warnings produced there seems to be one real bug, which you may have caught: urllib2.py:1054: No global (OpenerDirectory) found I didn't investigate the other warnings closely. It's not always easy to know if there should really be a warning or not. For example, an unused variable could be a real bug, even though often it is not. No class attribute warnings are likely bugs, but could depend a lot on how one codes. I don't know if there is a good middle ground. That's part of the reason that you can specify a .pycheckrc file, rather than keep entering command line arguments. Also, there should probably be some more general classes of command line args, like -e/--errors. Suggestions? Neal From neal@metaslash.com Fri Aug 10 16:43:52 2001 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 10 Aug 2001 11:43:52 -0400 Subject: [Python-Dev] Direction of PyChecker References: <200108101516.RAA15086@core.inf.ethz.ch> <200108101519.LAA18940@cj20424-a.reston1.va.home.com> Message-ID: <3B740138.98E75C5F@metaslash.com> Guido van Rossum wrote: > > [Samuele Pedroni] > > I'm consindering the possibility of adding a special mode to jython > > (based on hooks and on porting the CPython tools/compiler) in which > > you can retrieve the PVM bytecode of code objects, then there would > > be still to make PyChecker java classes aware, but it could run > > under jython. (Don't know when I will get at doing this) > > But is it really a good idea to codify PVM bytecode as the way to do > introspection? I'd much rather have a standard library API that > parses a module and gives you a list of variables, functions, classes, > etc., and lets you drill down into the symbol table of each of those. > (I realize more is needed, you need to be able to inspect every > statement too -- that can be done by providing access to an AST.) While it would make my life easier to modify jython, I tend to agree with Guido. Ideally, checker would use ASTs. I really want checker to work with jython, but it's too much work to do right now. The reason why I chose to use the PVM bytecode was because it was faster and easier to get something. I started trying to use ASTs but it would have taken a lot longer to get anywhere. I would like to support pychecker under jython, just not sure of the best or fastest way to get there. Neal From thomas@xs4all.net Fri Aug 10 16:54:14 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 10 Aug 2001 17:54:14 +0200 Subject: [Python-Dev] ++x oddnes In-Reply-To: <15219.64534.411625.193775@nyx.dyndns.org> References: <15219.64534.411625.193775@nyx.dyndns.org> Message-ID: <20010810175414.E874@xs4all.nl> On Fri, Aug 10, 2001 at 10:21:58AM -0500, Charles G Waldman wrote: > ++x > Interestingly, this runs, it just never terminates! Looks like > somehow ++x is being parsed as "+0 + x". Actually, it's being parsed as '+(+x)': 6 LOAD_FAST 0 (x) 9 UNARY_POSITIVE 10 UNARY_POSITIVE The same goes for '--x'. If it were parsed as '+0 + x', 'x++' would also be parsed as 'x + +0'. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From arigo@ulb.ac.be Fri Aug 10 16:54:57 2001 From: arigo@ulb.ac.be (Armin Rigo) Date: Fri, 10 Aug 2001 17:54:57 +0200 (CEST) Subject: [Python-Dev] Psyco update Message-ID: Hello everybody, Just a quick update on the Python Specializing Compiler. I started to work on a quite different prototype one month ago. This is all very experimental, so I will delay the technical explanations for some more time. The motivations are as follows: the previous implementation loosed WAY too much time in compilation proper: around 0.2 seconds for a 10-15 lines function! I am not sure that writing Psyco in very optimized C would have been enough to be of much help in anything but the most particular cases. Because of this I am re-thinking the way it could work. While I did not abandon the base ideas, I switched to a slightly different approach which, I think, could produce slightly less efficient machine code but incredibly more quickly. This work is based on research articles describing techniques to produce code at run-time [XXX references]. With the execution of only 4 to 6 processor instructions, they output one instruction of the dynamic code. Compare it with a typical C compiler, which takes thousands times more clock cycles to run than the number of instructions it emits! Or course, I do not question that it outputs more optimized code than what can be done with 4 to 6 instructions; however, this has the great advantage that even if some dynamically produced code is only used a few times, even only once, the overhead of having produced it in the first place is very low -- maybe even lower than the time that would have been required to load it from a disk file if it had been precompiled, and in the worst cases similar to the current Python VM interpreter. A bientot, Armin. From nas@python.ca Fri Aug 10 17:01:11 2001 From: nas@python.ca (Neil Schemenauer) Date: Fri, 10 Aug 2001 09:01:11 -0700 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: <3B740138.98E75C5F@metaslash.com>; from neal@metaslash.com on Fri, Aug 10, 2001 at 11:43:52AM -0400 References: <200108101516.RAA15086@core.inf.ethz.ch> <200108101519.LAA18940@cj20424-a.reston1.va.home.com> <3B740138.98E75C5F@metaslash.com> Message-ID: <20010810090111.A7248@glacier.fnational.com> Neal Norwitz wrote: > The reason why I chose to use the PVM bytecode was because it was > faster and easier to get something. I started trying to use ASTs > but it would have taken a lot longer to get anywhere. Couldn't you use Tools/compiler to generate a nice AST? Neil From DavidA@ActiveState.com Fri Aug 10 18:01:05 2001 From: DavidA@ActiveState.com (David Ascher) Date: Fri, 10 Aug 2001 10:01:05 -0700 Subject: [Python-Dev] ++x oddnes References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> Message-ID: <3B741351.C7C2E2B8@ActiveState.com> Guido van Rossum wrote: > IMO not worth it changing the language. Train yourself to write x++ > instead. :-) Sounds like an excellent one for PyChecker =) -- David Ascher From jeremy@zope.com Fri Aug 10 19:09:28 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Fri, 10 Aug 2001 14:09:28 -0400 (EDT) Subject: [Python-Dev] Direction of PyChecker In-Reply-To: <3B740008.2CF200C2@metaslash.com> References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <15219.2027.131396.703309@slothrop.digicool.com> <3B740008.2CF200C2@metaslash.com> Message-ID: <15220.9048.366116.587916@slothrop.digicool.com> >>>>> "NN" == Neal Norwitz writes: NN> Jeremy Hylton wrote: >> It would be much more helpful if the default setting identified >> real bugs rather than coding style issues. The doc string one is >> probably the most glaring. Add a -Wall option that turns on >> everything to satisfy the pedants. NN> This is generally true for more advanced users, but not NN> necessarily for newbies. There is the -e/--errors which NN> eliminates many warnings. Do you mean that it is better for newbies if the tool emits many warnings though they may be spurious? I would think the reverse: A newbie is going to have a lot more trouble deciding that the bug reported by PyChecker is, in fact, not a bug at all. They haven't developed the expertise or confidence to disagree with the tool. In part, it's a question of first impressions. If the user sees a very high number of false positives the first time they see the tool, they're not going to see it as very useful. >> By way of example, I used pychecker on urllib2 yesterday. It >> reported 78 bugs of which 3 were real. Based on that feedback, I >> wouldn't use the tool again, but would let other people sift >> through the noise and report the bugs back to me. NN> I ran checker w/--errors on urllib2. There were some warnings NN> which should have been excluded--I'll have to fix those. But of NN> the 12 warnings produced there seems to be one real bug, which NN> you may have caught: I don't know what I'm doing differently today, but I don't see any of the warnings about classes without doc strings and the like. Could something have changed very recently? Was I imagining the output I saw the other day? NN> urllib2.py:1054: No global (OpenerDirectory) found Yes. This was one of the typos it found. NN> It's not always easy to know if there should really be a warning NN> or not. For example, an unused variable could be a real bug, NN> even though often it is not. No class attribute warnings are NN> likely bugs, but could depend a lot on how one codes. There seem to be a lot of spurious complaints related to class hierarchies. I see a bunch of complaints that a sublcass doesn't have an __init__() -- but subclasses don't need to have an __init__ and often shouldn't. There are also complaints about attribute used by an abstract base class but only defined in the subclass. Jeremy From Samuele Pedroni Fri Aug 10 19:21:19 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Fri, 10 Aug 2001 20:21:19 +0200 (MET DST) Subject: [Python-Dev] Direction of PyChecker Message-ID: <200108101821.UAA28722@core.inf.ethz.ch> [GvR] > But is it really a good idea to codify PVM bytecode as the way to do > introspection? No (but I don't know - for example - what is the tradition under Smalltalk impls) but it seems that people like to do it that way (inspect also uses bytecode in some places but maybe there is OK, because is starting from low-level to give an high-level API). Doesn't pydoc also use imports? On the other hand the new nested scope rules don't make life easy for someone using ASTs. At least Jeremy and I should come out with a really interoperable symtable module. At the moment such a beast does not exist under jython and the CPython part is under-specified. I imagine that a problem is the very late runtime semantics of python, it is very clear, but then just executing a module you are sure of what are the most likely values of some stuff. > I'd much rather have a standard library API that > parses a module and gives you a list of variables, functions, classes, > etc., and lets you drill down into the symbol table of each of those. > (I realize more is needed, you need to be able to inspect every > statement too -- that can be done by providing access to an AST.) > > --Guido van Rossum (home page: http://www.python.org/~guido/) regards, Samuele Pedroni. From neal@metaslash.com Fri Aug 10 20:15:57 2001 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 10 Aug 2001 15:15:57 -0400 Subject: [Python-Dev] Direction of PyChecker References: <200108101516.RAA15086@core.inf.ethz.ch> <200108101519.LAA18940@cj20424-a.reston1.va.home.com> <3B740138.98E75C5F@metaslash.com> <20010810090111.A7248@glacier.fnational.com> Message-ID: <3B7432ED.12A117C1@metaslash.com> Neil Schemenauer wrote: > > Neal Norwitz wrote: > > The reason why I chose to use the PVM bytecode was because it was > > faster and easier to get something. I started trying to use ASTs > > but it would have taken a lot longer to get anywhere. > > Couldn't you use Tools/compiler to generate a nice AST? I probably could. Recently, I looked at the tool in 2.2. But I didn't know about the compiler when I started. I looked at pylint (from 1999) and the ASTs that were generated from parser, etc. But it was going to require too much work at the time. And I was just trying to figure out if static analysis was even possible. I've considered the benefits/drawbacks of converting checker to use ASTs. The drawback is how much time it will take to convert, so I haven't done anything....yet. Neal From neal@metaslash.com Fri Aug 10 20:29:45 2001 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 10 Aug 2001 15:29:45 -0400 Subject: [Python-Dev] Direction of PyChecker References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <15219.2027.131396.703309@slothrop.digicool.com> <3B740008.2CF200C2@metaslash.com> <15220.9048.366116.587916@slothrop.digicool.com> Message-ID: <3B743629.E1220DBB@metaslash.com> Jeremy Hylton wrote: > > >>>>> "NN" == Neal Norwitz writes: > > NN> Jeremy Hylton wrote: > >> It would be much more helpful if the default setting identified > >> real bugs rather than coding style issues. The doc string one is > >> probably the most glaring. Add a -Wall option that turns on > >> everything to satisfy the pedants. > > NN> This is generally true for more advanced users, but not > NN> necessarily for newbies. There is the -e/--errors which > NN> eliminates many warnings. > > Do you mean that it is better for newbies if the tool emits many > warnings though they may be spurious? I would think the reverse: A > newbie is going to have a lot more trouble deciding that the bug > reported by PyChecker is, in fact, not a bug at all. They haven't > developed the expertise or confidence to disagree with the tool. You've got a very good point here. I'm not conviced, but I'm very close. So keep working. :-) I agree that the doc strings should not be warned about by default. That will be changed. As I said before, I think that the modules/classes in the standard library should have them. And docstrings are definitely doc, not code. > In part, it's a question of first impressions. If the user sees a > very high number of false positives the first time they see the tool, > they're not going to see it as very useful. I agree. > >> By way of example, I used pychecker on urllib2 yesterday. It > >> reported 78 bugs of which 3 were real. Based on that feedback, I > >> wouldn't use the tool again, but would let other people sift > >> through the noise and report the bugs back to me. > > NN> I ran checker w/--errors on urllib2. There were some warnings > NN> which should have been excluded--I'll have to fix those. But of > NN> the 12 warnings produced there seems to be one real bug, which > NN> you may have caught: > > I don't know what I'm doing differently today, but I don't see any of > the warnings about classes without doc strings and the like. Could > something have changed very recently? Was I imagining the output I > saw the other day? I didn't change it yet! :-) The default (currently) is to warn about missing docstrings for modules and classes, not functions or methods. This will change shortly (missing docstrings will not generate a warning by default). I also added some config options, so now the # of warnings on urllib2 dropped when using --errors. > NN> It's not always easy to know if there should really be a warning > NN> or not. For example, an unused variable could be a real bug, > NN> even though often it is not. No class attribute warnings are > NN> likely bugs, but could depend a lot on how one codes. > > There seem to be a lot of spurious complaints related to class > hierarchies. I see a bunch of complaints that a sublcass doesn't have > an __init__() -- but subclasses don't need to have an __init__ and > often shouldn't. Lacking an __init__() should be off by default. > There are also complaints about attribute used by an > abstract base class but only defined in the subclass. This is true. While python doesn't require setting, I think this feature can be dangerous. It seems better to init the attr to None. In urllib2, there are 6 warnings about 2 different attributes. I can provide a config option to turn that off too. Neal From guido@python.org Fri Aug 10 20:45:19 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 10 Aug 2001 15:45:19 -0400 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: Your message of "Fri, 10 Aug 2001 15:29:45 EDT." <3B743629.E1220DBB@metaslash.com> References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <15219.2027.131396.703309@slothrop.digicool.com> <3B740008.2CF200C2@metaslash.com> <15220.9048.366116.587916@slothrop.digicool.com> <3B743629.E1220DBB@metaslash.com> Message-ID: <200108101945.PAA24012@cj20424-a.reston1.va.home.com> > As I said before, I think that the modules/classes > in the standard library should have them. You can help by submitting some patches (one module at a time please :-). > And docstrings are definitely > doc, not code. Agreed, but I don;'t understand why you said this. > > There are also complaints about attribute used by an > > abstract base class but only defined in the subclass. > > This is true. While python doesn't require setting, I think this feature > can be dangerous. It seems better to init the attr to None. Agreed. The initialization to None can have a comment explaining what the subclass should do. Ditto for methods: in an abstract base class, the methods should be defined (so the signature is known) but raise NotImplementedError. > In urllib2, there are 6 warnings about 2 different attributes. How come PyChecker didn't find the other typo there? --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Fri Aug 10 20:56:25 2001 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 10 Aug 2001 15:56:25 -0400 Subject: [Python-Dev] Direction of PyChecker References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <15219.2027.131396.703309@slothrop.digicool.com> <3B740008.2CF200C2@metaslash.com> <15220.9048.366116.587916@slothrop.digicool.com> <3B743629.E1220DBB@metaslash.com> <200108101945.PAA24012@cj20424-a.reston1.va.home.com> Message-ID: <3B743C69.2BC9B090@metaslash.com> Guido van Rossum wrote: > > > As I said before, I think that the modules/classes > > in the standard library should have them. > > You can help by submitting some patches (one module at a time please :-). I'll do my best. And this time, I'll post to SF, with link here. :-) > > And docstrings are definitely > > doc, not code. > > Agreed, but I don;'t understand why you said this. I was just pointing out that docstrings should not be warned about and that they won't be in the future (about 5 minutes :-). > > > There are also complaints about attribute used by an > > > abstract base class but only defined in the subclass. > > > > This is true. While python doesn't require setting, I think this feature > > can be dangerous. It seems better to init the attr to None. > > Agreed. The initialization to None can have a comment explaining what > the subclass should do. > > Ditto for methods: in an abstract base class, the methods should be > defined (so the signature is known) but raise NotImplementedError. I will try to create patches for a lot of these warnings as well then. I think there are already comments, so I will just init to None. > > In urllib2, there are 6 warnings about 2 different attributes. > > How come PyChecker didn't find the other typo there? Not sure. What was broken? Was it in 2.2a1 or cvs? (I ran against 2.2a1.) Neal From nas@python.ca Fri Aug 10 21:21:05 2001 From: nas@python.ca (Neil Schemenauer) Date: Fri, 10 Aug 2001 13:21:05 -0700 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: <3B7432ED.12A117C1@metaslash.com>; from neal@metaslash.com on Fri, Aug 10, 2001 at 03:15:57PM -0400 References: <200108101516.RAA15086@core.inf.ethz.ch> <200108101519.LAA18940@cj20424-a.reston1.va.home.com> <3B740138.98E75C5F@metaslash.com> <20010810090111.A7248@glacier.fnational.com> <3B7432ED.12A117C1@metaslash.com> Message-ID: <20010810132104.B7903@glacier.fnational.com> Neal Norwitz wrote: > I looked at pylint (from 1999) and the ASTs that were generated > from parser, etc. But it was going to require too much work > at the time. The tree generated by Tools/compiler is much nicer than the one generated by parser. Neil From jeremy@zope.com Fri Aug 10 21:52:06 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Fri, 10 Aug 2001 16:52:06 -0400 (EDT) Subject: [Python-Dev] future and present Message-ID: <15220.18806.202695.349346@slothrop.digicool.com> If a script has from __future__ import nested_scopes and is run by Python 2.2, should the interpreter complain? The PEP says "the compiler knows which features names have been defined, and raises a compile-time error if a future_statement contains a feature not known to it." I think I'd interpret this as: nested_scopes is known to 2.2 (and redundant since the feature is now mandatory), therefore the compiler does nothing. Jeremy From guido@python.org Fri Aug 10 22:01:11 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 10 Aug 2001 17:01:11 -0400 Subject: [Python-Dev] future and present In-Reply-To: Your message of "Fri, 10 Aug 2001 16:52:06 EDT." <15220.18806.202695.349346@slothrop.digicool.com> References: <15220.18806.202695.349346@slothrop.digicool.com> Message-ID: <200108102101.RAA27718@cj20424-a.reston1.va.home.com> > If a script has > from __future__ import nested_scopes > and is run by Python 2.2, should the interpreter complain? Of course not! The whole point of the future statement is that it should be harmless once you're *in* that future (unless the future feature ends up retracted -- think of it as an alternate universe, a side effect of time machine misuse :-). > The PEP says "the compiler knows which features names have been > defined, and raises a compile-time error if a future_statement > contains a feature not known to it." I think I'd interpret this as: > nested_scopes is known to 2.2 (and redundant since the feature is > now mandatory), therefore the compiler does nothing. Correct. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Fri Aug 10 22:18:14 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 10 Aug 2001 17:18:14 -0400 Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12 In-Reply-To: <2m8zgsfhx2.fsf@starship.python.net> Message-ID: Man, I'm having a hard time following this! Perhaps people are talking passed each other? [Michael Hudson] > It's a matter of interface, really. It's certainly not at all > technically hard. Maybe: > > compile(text, filename, symbol[, flags[, dont_inherit]]) Sounds good -- let's do it. > I worry that saying "you don't inherit behaviour from surrounding code > as soon as you pass a flag" might get really, really confusing at > times. I would too, but did someone suggest that? Not me. If someone did, they're confusing issues to no good end. [Guido] > Occam sez: let's add the dont_inherit argument when we have found a > real use for it. It's trivial to add, impossible to work around its absence if necessary, and is an inherent part of the problem space (as Michael said, the future-PEP anticipated the problem). It seems as simple as possible without leaving problems unaddressed. [Michael] > There are two complementary sets of flags here: > > (1) The PyCF_* ones, defined in Include/pythonrun.h > (2) The CO_* ones defined in Include/compile.h > > The proposed fourth argument to compile() should be a combination of > the former set. > > I only use the latter to tell whether a __future__ statement was used > in the text compiled (which is a bit horrible, but no better way > springs to mind). If there were a better way, we would have used it instead of the goofy CO_* flags everywhere else too . > There might be value in checking the flags passed to compile() anyway, > but I can't see it being dangerous. It's so easy to make this bulletproof that there's no point discussing whether it *should* be made bulletproof. I'll see to it that it's bulletproof in the end, so if it's a hangup don't worry about it now. [Guido] > (I think I may have been confused by your original patch, which AFAICR *did* > manipulate the co_flags field of the code object, which contains the CO_* flags. Right, it did, and the newer scheme is safer. From DavidA@ActiveState.com Fri Aug 10 22:31:29 2001 From: DavidA@ActiveState.com (David Ascher) Date: Fri, 10 Aug 2001 14:31:29 -0700 Subject: [Python-Dev] future and present References: <15220.18806.202695.349346@slothrop.digicool.com> <200108102101.RAA27718@cj20424-a.reston1.va.home.com> Message-ID: <3B7452B1.CC57F1@ActiveState.com> Guido van Rossum wrote: > > > If a script has > > from __future__ import nested_scopes > > and is run by Python 2.2, should the interpreter complain? > > Of course not! The whole point of the future statement is that it > should be harmless once you're *in* that future (unless the future > feature ends up retracted -- think of it as an alternate universe, a > side effect of time machine misuse :-). Although there could be a warning (off by default) in 2.3 or whenever to let users know that the code is no longer needed. People tend to cut & paste and reuse code from others, without always knowing why. Noops should be culled from code as a general rule =). --david From guido@python.org Fri Aug 10 22:31:22 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 10 Aug 2001 17:31:22 -0400 Subject: [Python-Dev] Changing the Division Operator -- PEP 238, rev 1.12 In-Reply-To: Your message of "Fri, 10 Aug 2001 17:18:14 EDT." References: Message-ID: <200108102131.RAA28517@cj20424-a.reston1.va.home.com> OK, dont_inherit is good. --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Fri Aug 10 22:33:39 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 10 Aug 2001 17:33:39 -0400 (EDT) Subject: [Python-Dev] Direction of PyChecker In-Reply-To: <20010810132104.B7903@glacier.fnational.com> References: <200108101516.RAA15086@core.inf.ethz.ch> <200108101519.LAA18940@cj20424-a.reston1.va.home.com> <3B740138.98E75C5F@metaslash.com> <20010810090111.A7248@glacier.fnational.com> <3B7432ED.12A117C1@metaslash.com> <20010810132104.B7903@glacier.fnational.com> Message-ID: <15220.21299.541723.697757@cj42289-a.reston1.va.home.com> Neil Schemenauer writes: > The tree generated by Tools/compiler is much nicer than the one > generated by parser. An excellent reason for the compiler package to be documented! Jeremy? -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Fri Aug 10 22:35:14 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 10 Aug 2001 17:35:14 -0400 Subject: [Python-Dev] future and present In-Reply-To: Your message of "Fri, 10 Aug 2001 14:31:29 PDT." <3B7452B1.CC57F1@ActiveState.com> References: <15220.18806.202695.349346@slothrop.digicool.com> <200108102101.RAA27718@cj20424-a.reston1.va.home.com> <3B7452B1.CC57F1@ActiveState.com> Message-ID: <200108102135.RAA28904@cj20424-a.reston1.va.home.com> > Although there could be a warning (off by default) in 2.3 or whenever to > let users know that the code is no longer needed. People tend to cut & > paste and reuse code from others, without always knowing why. Noops > should be culled from code as a general rule =). Agreed. This could also be a PyChecker feature. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Fri Aug 10 22:43:10 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Fri, 10 Aug 2001 17:43:10 -0400 (EDT) Subject: [Python-Dev] Direction of PyChecker In-Reply-To: <15220.21299.541723.697757@cj42289-a.reston1.va.home.com> References: <200108101516.RAA15086@core.inf.ethz.ch> <200108101519.LAA18940@cj20424-a.reston1.va.home.com> <3B740138.98E75C5F@metaslash.com> <20010810090111.A7248@glacier.fnational.com> <3B7432ED.12A117C1@metaslash.com> <20010810132104.B7903@glacier.fnational.com> <15220.21299.541723.697757@cj42289-a.reston1.va.home.com> Message-ID: <15220.21870.462055.990576@slothrop.digicool.com> I'll work on it next week. Really! I will. Jeremy From fdrake@acm.org Fri Aug 10 22:47:12 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 10 Aug 2001 17:47:12 -0400 (EDT) Subject: [Python-Dev] Direction of PyChecker In-Reply-To: <15220.21870.462055.990576@slothrop.digicool.com> References: <200108101516.RAA15086@core.inf.ethz.ch> <200108101519.LAA18940@cj20424-a.reston1.va.home.com> <3B740138.98E75C5F@metaslash.com> <20010810090111.A7248@glacier.fnational.com> <3B7432ED.12A117C1@metaslash.com> <20010810132104.B7903@glacier.fnational.com> <15220.21299.541723.697757@cj42289-a.reston1.va.home.com> <15220.21870.462055.990576@slothrop.digicool.com> Message-ID: <15220.22112.452257.158074@cj42289-a.reston1.va.home.com> Jeremy Hylton writes: > I'll work on it next week. Really! I will. I'm sure you'll have it done by the end of Monday, so you don't have to deal with my staring at you accusingly all day Tuesday. ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From tim.one@home.com Fri Aug 10 22:57:48 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 10 Aug 2001 17:57:48 -0400 Subject: [Python-Dev] future and present In-Reply-To: <15220.18806.202695.349346@slothrop.digicool.com> Message-ID: [Jeremy] > If a script has > from __future__ import nested_scopes > and is run by Python 2.2, should the interpreter complain? No: Else MandatoryRelease records when the feature became part of the language; in releases at or after that, modules no longer need from __future__ import FeatureName to use the feature in question, but may continue to use such imports. [David Ascher] > Although there could be a warning (off by default) in 2.3 or whenever > to let users know that the code is no longer needed. People tend to > cut & paste and reuse code from others, without always knowing why. > Noops should be culled from code as a general rule =). The syntax of future statments was designed so that a trival sed script can find them reliably. Heck, even ActivePerl could find them . They have to begin in the first column, and have to start with "from __future__". I expect that many people will *want* to keep obsolete future stmts in forever, as a protection against trying to run code under earlier versions of the interpreter -- the semantics were designed so that such back-porting always complains (if to a pre-future version of Python, because the "from __future__ import ..." stmt raises a runtime ImportError due to the absence of module __future__.py (which has a reserved name); or if to a post-future pre-feature version, a compile-time error). People insanely concerned about this can query the MandatoryRelease fields of the future-features they import, raising an exception of their choosing if the MandatoryRelease is <= sys.version_info; e.g., from __future__ import division assert division.getMandatoryRelease() > sys.version_info That will complain the first time they run it under a Python where new division is "the rule". From tim.one@home.com Sat Aug 11 00:00:41 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 10 Aug 2001 19:00:41 -0400 Subject: [Python-Dev] Somebody stopped running tests? Message-ID: Over the last 12 hours, 4 tests started failing on Windows: 4 tests failed: test_descr test_future test_generators test_scope Would someone on Linux please confirm or deny it's not unique to Windows? I don't want to spend a lot of time tracking these down if the responsible parties see them on their boxes too. From tim.one@home.com Sat Aug 11 00:17:38 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 10 Aug 2001 19:17:38 -0400 Subject: [Python-Dev] RE: Somebody stopped running tests? In-Reply-To: Message-ID: [Tim said] > Over the last 12 hours, 4 tests started failing on Windows: > > 4 tests failed: test_descr test_future test_generators test_scope > > Would someone on Linux please confirm or deny it's not unique to > Windows? I don't want to spend a lot of time tracking these down > if the responsible parties see them on their boxes too. But only in release mode. In a debug build, these pass. This strongly suggests a change "somewhere" that accidentally relies on an uninitialized value. Under the release build, I get oodles of bogus warnings related to scope, like ../lib/test/test_scope.py:401: SyntaxWarning: local name 'x' in 'f1' shadows use of 'x' as global in nested scope 'f2' so best guess is it's Jeremy's recent future-flag fiddling patches. From aahz@rahul.net Sat Aug 11 00:18:09 2001 From: aahz@rahul.net (Aahz Maruch) Date: Fri, 10 Aug 2001 16:18:09 -0700 (PDT) Subject: [Python-Dev] Any objections to adding threading.Timer? In-Reply-To: <200108092021.f79KLFI00918@mira.informatik.hu-berlin.de> from "Martin v. Loewis" at Aug 09, 2001 10:21:15 PM Message-ID: <20010810231810.3FDD499C82@waltz.rahul.net> [sorry for the delay, network connectivity problems] Martin v. Loewis wrote: > Aahz: >> >> I'm not objecting, precisely, but I have an allergic reaction to the way >> timed Event() works, even though there's really no other way to work it. >> I think that encouraging people to use it is a Bad Idea. > > Can you elaborate in more detail what your concern about the timed > Event() is? This one itself looks fine; if anything, _Condition.wait > looks disturbing. Could you also kindly put your comments into > > http://sourceforge.net/tracker/?group_id=5470&atid=305470&func=detail&aid=428326 Hrm. I'll put my comments there if you ask, but my objection is more general than this specific feature. As you note, I'm actually objecting to the whole Condition.wait() feature. The main problem with this kind of timeout is the way the granularity works; there was a recent post to c.l.py where someone was complaining that it took a full second to respond to the Event() getting set. Fred: I think I did send a note to python-docs, but maybe I should file an SF doc bug? At the same time, when you really do want timeout functionality, there isn't any other way to do it in a cross-platform way. I think most people claim they want a timeout when they really shouldn't, so I'd like to make the feature less accessible. > There are a number of procedural objections with regard to this patch > (license, missing documentation); if you can add significant technical > objections, that might be a reason to reject it - or to find a better > alternative. Well, I can't quite call this a "significant" technical objection, given that this is the only good cross-platform scheme in a threaded environment. I'm just whining because I don't like it anyway. ;-) -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From pedroni@inf.ethz.ch Sat Aug 11 00:44:39 2001 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Sat, 11 Aug 2001 01:44:39 +0200 Subject: [Python-Dev] Direction of PyChecker References: <200108101516.RAA15086@core.inf.ethz.ch> <200108101519.LAA18940@cj20424-a.reston1.va.home.com> <3B740138.98E75C5F@metaslash.com> Message-ID: <00b601c121f6$6d35d340$8a73fea9@newmexico> > Guido van Rossum wrote: > > > > [Samuele Pedroni] > > > I'm consindering the possibility of adding a special mode to jython > > > (based on hooks and on porting the CPython tools/compiler) in which > > > you can retrieve the PVM bytecode of code objects, then there would > > > be still to make PyChecker java classes aware, but it could run > > > under jython. (Don't know when I will get at doing this) > > > > But is it really a good idea to codify PVM bytecode as the way to do > > introspection? I'd much rather have a standard library API that > > parses a module and gives you a list of variables, functions, classes, > > etc., and lets you drill down into the symbol table of each of those. > > (I realize more is needed, you need to be able to inspect every > > statement too -- that can be done by providing access to an AST.) > > While it would make my life easier to modify jython, I tend to agree > with Guido. Ideally, checker would use ASTs. I really want checker > to work with jython, but it's too much work to do right now. > > The reason why I chose to use the PVM bytecode was because it was > faster and easier to get something. I started trying to use ASTs > but it would have taken a lot longer to get anywhere. > > I would like to support pychecker under jython, just not sure of > the best or fastest way to get there. There have been a misunderestanding and don't like at all to play the role of the ominous sinner, ignorant of language design. I was just proposing a hack, not codifying anything at all, vs. what at the moment is just discussing about The Right Thing, aka wishful thinking. regards, Samuele Pedroni. From tim_one@email.msn.com Sat Aug 11 02:10:22 2001 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 10 Aug 2001 21:10:22 -0400 Subject: [Python-Dev] RE: Somebody stopped running tests? In-Reply-To: Message-ID: > Over the last 12 hours, 4 tests started failing on Windows: > > 4 tests failed: test_descr test_future test_generators test_scope > ... > But only in release mode. In a debug build, these pass. This strongly > suggests a change "somewhere" that accidentally relies on an > uninitialized value. And so it was. "Fixed" in current CVS, but Jeremy needs to fix it a right way. hacking-a-hack-that-wasn't-fully-unhacked-ly y'rs - tim From barry@zope.com Sat Aug 11 04:08:53 2001 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 10 Aug 2001 23:08:53 -0400 Subject: [Python-Dev] Free threading (and tcl upvar) References: <200108091522.f79FMja02243@odiug.digicool.com> <15219.15664.450890.783449@anthem.wooz.org> <15219.16667.375672.59499@slothrop.digicool.com> Message-ID: <15220.41413.209107.815162@anthem.wooz.org> >>>>> "JH" == Jeremy Hylton writes: JH> It takes all kinds, doesn't it? For every genius who comes up JH> with something like sys._getframe(), we have some fool who JH> comes up with something like "print >>". there is indeed a fine line between genius and insanity. From guido@python.org Sat Aug 11 05:06:14 2001 From: guido@python.org (Guido van Rossum) Date: Sat, 11 Aug 2001 00:06:14 -0400 Subject: [Python-Dev] Any objections to adding threading.Timer? In-Reply-To: Your message of "Fri, 10 Aug 2001 16:18:09 PDT." <20010810231810.3FDD499C82@waltz.rahul.net> References: <20010810231810.3FDD499C82@waltz.rahul.net> Message-ID: <200108110406.AAA04467@cj20424-a.reston1.va.home.com> > As you note, I'm actually objecting to the whole Condition.wait() > feature. The main problem with this kind of timeout is the way the > granularity works; there was a recent post to c.l.py where someone > was complaining that it took a full second to respond to the Event() > getting set. Hm, maybe we should think about adding some low-level thing somewhere so that wait(timeout=...) can be implemented more efficiently? I agree that it's ugly now -- busy-waiting with a sleep time doubling until it reaches 1 second... --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@rahul.net Sat Aug 11 05:42:36 2001 From: aahz@rahul.net (Aahz Maruch) Date: Fri, 10 Aug 2001 21:42:36 -0700 (PDT) Subject: [Python-Dev] Any objections to adding threading.Timer? In-Reply-To: <200108110406.AAA04467@cj20424-a.reston1.va.home.com> from "Guido van Rossum" at Aug 11, 2001 12:06:14 AM Message-ID: <20010811044236.9F2BA99C82@waltz.rahul.net> Guido van Rossum wrote: >Aahz: >> >> As you note, I'm actually objecting to the whole Condition.wait() >> feature. The main problem with this kind of timeout is the way the >> granularity works; there was a recent post to c.l.py where someone >> was complaining that it took a full second to respond to the Event() >> getting set. > > Hm, maybe we should think about adding some low-level thing somewhere > so that wait(timeout=...) can be implemented more efficiently? I > agree that it's ugly now -- busy-waiting with a sleep time doubling > until it reaches 1 second... Unfortunately, I don't think this is possible unless we add some platform-specific code to thread.c, and you'll need to ask Uncle Timmy for that. To the extent that I'm an expert on threading, I'm only an expert on Python threads. Unless someone is aware of some brilliantly simple solution, I think we need to live with the ugliness for now, but I'd prefer to discourage further efforts in the ugly direction. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From martin@loewis.home.cs.tu-berlin.de Sat Aug 11 08:43:05 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Sat, 11 Aug 2001 09:43:05 +0200 Subject: [Python-Dev] future and present Message-ID: <200108110743.f7B7h5702904@mira.informatik.hu-berlin.de> > Although there could be a warning (off by default) in 2.3 or > whenever to let users know that the code is no longer needed. > People tend to cut & paste and reuse code from others, without > always knowing why. Noops should be culled from code as a general > rule =). Actually, Tim promised to write a tool to remove unneeded future imports, when the feature was first discussed. Maybe he did not actually promise to write such a tool, but only explained how easy it would be to write, given that it is introspective :-) Regards, Martin From martin@loewis.home.cs.tu-berlin.de Sat Aug 11 09:06:35 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Sat, 11 Aug 2001 10:06:35 +0200 Subject: [Python-Dev] Any objections to adding threading.Timer? In-Reply-To: <20010810231810.3FDD499C82@waltz.rahul.net> (aahz@rahul.net) References: <20010810231810.3FDD499C82@waltz.rahul.net> Message-ID: <200108110806.f7B86Zp03107@mira.informatik.hu-berlin.de> > Hrm. I'll put my comments there if you ask, but my objection is more > general than this specific feature. As you note, I'm actually objecting > to the whole Condition.wait() feature. The main problem with this kind > of timeout is the way the granularity works; there was a recent post to > c.l.py where someone was complaining that it took a full second to > respond to the Event() getting set. Please do. Correct me if I'm wrong, but the timeout of the Timer won't be precise; that clearly is undesirable. It also seems that this would be easy to correct: If endtime-_time() < delay, you have to decrease delay. What then is left is that cancellation of the timer may take up to a second, that should be documented. I think it is possible to do better than that on some systems; maybe somebody feels inclined to improve the code on, say, UNIX. Regards, Martin From just@letterror.com Sat Aug 11 09:17:31 2001 From: just@letterror.com (Just van Rossum) Date: Sat, 11 Aug 2001 10:17:31 +0200 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: <200108092052.f79KqQg13802@odiug.digicool.com> Message-ID: <20010811101734-r01010700-bede4223-0910-010c@10.0.0.23> A couple of days ago, Guido wrote: > I would like PyChecker to use a fundamentally different way of > scanning modules. I believe that it currently imports the module and > then scans the module's __dict__ for classes and functions, and > analyses their bytecode. That's fragile, dangerous (if you don't know > what a module does you may not want to import it), [ ... ] [Off topic] The same seems to be true for pydoc in http-serving mode. It just imports any .py on sys.path, when doco is requested for it. Although pydoc is *really* cool, this aspect scares me a bit... (I guess if the port pydoc is listening to is visible from the outside world, it better run as a user with very limited permissions, right?) (Hm, that makes me think that it may be a good practice to name scripts that are not supposed to be imported so that they *can't* be imported?) Just From guido@python.org Sat Aug 11 13:52:16 2001 From: guido@python.org (Guido van Rossum) Date: Sat, 11 Aug 2001 08:52:16 -0400 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: Your message of "Sat, 11 Aug 2001 10:17:31 +0200." <20010811101734-r01010700-bede4223-0910-010c@10.0.0.23> References: <20010811101734-r01010700-bede4223-0910-010c@10.0.0.23> Message-ID: <200108111252.IAA05424@cj20424-a.reston1.va.home.com> > [Off topic] The same seems to be true for pydoc in http-serving mode. > It just imports any .py on sys.path, when doco is requested for it. > Although pydoc is *really* cool, this aspect scares me a bit... > (I guess if the port pydoc is listening to is visible from the outside > world, it better run as a user with very limited permissions, right?) Yes, this bothers me greatly. I hope that someone will fix this. > (Hm, that makes me think that it may be a good practice to name scripts > that are not supposed to be imported so that they *can't* be imported?) Indeedy! (Or make sure they're noy on sys.path. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Sat Aug 11 18:29:40 2001 From: tim.one@home.com (Tim Peters) Date: Sat, 11 Aug 2001 13:29:40 -0400 Subject: [Python-Dev] Any objections to adding threading.Timer? In-Reply-To: <20010811044236.9F2BA99C82@waltz.rahul.net> Message-ID: [Aahz] > Unfortunately, I don't think this is possible unless we add some > platform-specific code to thread.c, and you'll need to ask Uncle Timmy > for that. To the extent that I'm an expert on threading, I'm only an > expert on Python threads. AFAIK there are no reliable x-platform timeout facilities. Even on Windows, the current scheme sucks in (at least) three ways: 1. time.time() updates only 18.2 times per second on Windows, so 55ms is the finest granularity possible via any time.time()-based approach. 2. time.sleep() on Windows truncates the argument to an integer number of milliseconds (due to limitations of the Win32 Sleep), so any delay value < 0.001 acts like Sleep(0), i.e. returns immediately if no other threads are waiting (that's how Win32 Sleep(0) works). 3. As Glyph so visibly discovered on c.l.py recently, time.time() can "go backwards", and especially on laptops with Intel SpeedStep CPUs. All that said, I'd still change the timeout loop from endtime = _time() + timeout delay = 0.000001 # 1 usec while 1: gotit = waiter.acquire(0) if gotit or _time() >= endtime: break _sleep(delay) if delay < 1.0: delay = delay * 2.0 to endtime = _time() + timeout delay = 0.001 # 1 ms while 1: gotit = waiter.acquire(0) if gotit: break remaining = endtime - _time() if remaining < 0: break delay = min(delay * 2, remaining, .05) _sleep(delay) From tim.one@home.com Sat Aug 11 18:39:39 2001 From: tim.one@home.com (Tim Peters) Date: Sat, 11 Aug 2001 13:39:39 -0400 Subject: [Python-Dev] future and present In-Reply-To: <200108110743.f7B7h5702904@mira.informatik.hu-berlin.de> Message-ID: [Martin v. Loewis] > Actually, Tim promised to write a tool to remove unneeded future > imports, when the feature was first discussed. Maybe he did not > actually promise to write such a tool, but only explained how easy it > would be to write, given that it is introspective :-) Given that David is the only person to date to ask for this, and not even because he personally wants it, I now view this as a value-added proposition for ActiveState to sell . it-really-is-a-finger-exercise-ly y'rs - tim From paulp@ActiveState.com Sat Aug 11 18:49:41 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Sat, 11 Aug 2001 10:49:41 -0700 Subject: [Python-Dev] xmlrpclib Server Message-ID: <3B757035.6894450E@ActiveState.com> What would people think about renaming xmlrpclib.Server to xmlrpclib.RPCProxy or ServerProxy or something like that? I find it confusing that the *client* software creates a Server object. I would expect xmlrpclib.Server() to create a true XML-RPC Server. I like the way that SOAPPy works: * SOAPProxy creates a client-side proxy object for a server * SOAPServer creates a SOAP server. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From martin@loewis.home.cs.tu-berlin.de Sat Aug 11 19:46:19 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Sat, 11 Aug 2001 20:46:19 +0200 Subject: [Python-Dev] xmlrpclib Server Message-ID: <200108111846.f7BIkJZ04391@mira.informatik.hu-berlin.de> > What would people think about renaming xmlrpclib.Server to > xmlrpclib.RPCProxy or ServerProxy or something like that? The time machine at work? It is already called ServerProxy. Regards, Martin From paulp@ActiveState.com Sat Aug 11 21:16:04 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Sat, 11 Aug 2001 13:16:04 -0700 Subject: [Python-Dev] xmlrpclib Server References: <200108111846.f7BIkJZ04391@mira.informatik.hu-berlin.de> Message-ID: <3B759284.463FBE9A@ActiveState.com> "Martin v. Loewis" wrote: > > > What would people think about renaming xmlrpclib.Server to > > xmlrpclib.RPCProxy or ServerProxy or something like that? > > The time machine at work? It is already called ServerProxy. You are right! Doing a diff of the version I have hanging around and the CVS version, I see: < class ServerProxy: < < Server = ServerProxy --- > class Server: But the TeX documentation still says "Server". Would anyone object to changing the doc to make ServerProxy official? -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From neal@metaslash.com Sat Aug 11 21:17:39 2001 From: neal@metaslash.com (Neal Norwitz) Date: Sat, 11 Aug 2001 16:17:39 -0400 Subject: [Python-Dev] More bugs in 2.2a1 Message-ID: <3B7592E3.892D1995@metaslash.com> Here's more warnings from pychecker. There are a lot from plat-irix6 & plat-unixware7, not sure if those are real problems or not. Neal -- curses: has_key.py:176: Module (_curses) re-imported textpad.py:72: No global (ERR) found distutils: bcppcompiler.py:359: No global (UnknownFileError) found dist.py:583: Variable (klass) used before being set fancy_getopt.py:277: No global (value) found file_util.py:218: No global (errno) found distutils.command: bdist_rpm.py:170: No global (DistutilsOptionsError) found bdist_wininst.py:217: Module (base64) re-imported config.py:277: No global (exe) found install.py:389: No global (DistutilsPlatformError) found encodings: charmap.py:28: No global (strict) found lib-tk: Tix.py:81: self is not first method argument Tix.py:331: No global (value) found plat-irix6: FILE.py:32: No global (__major) found FILE.py:34: No global (__minor) found FILE.py:50: No global (splhi) found FILE.py:52: No global (getsr) found FILE.py:52: No global (issplhi) found FILE.py:66: No global (mutex_unlock) found FILE.py:91: No global (spinlock_init) found FILE.py:99: No global (PZERO) found FILE.py:99: No global (mutex_lock) found FILE.py:101: No global (mutex_trylock) found FILE.py:105: No global (sv_signal) found FILE.py:107: No global (sv_broadcast) found FILE.py:109: No global (sv_destroy) found FILE.py:113: No global (mrunlock) found FILE.py:115: No global (cmrpromote) found FILE.py:117: No global (mrdemote) found FILE.py:119: No global (mrfree) found FILE.py:123: No global (ismrlocked) found plat-unixware7: IN.py:75: No global (IN6_ADDR_EQUAL_L) found IN.py:77: No global (IN6_ADDR_COPY_L) found IN.py:162: No global (REMQUE) found IN.py:440: No global (CYCLES) found IN.py:440: No global (CYCLES_BETWEEN) found IN.py:482: No global (_ATOMIC_INT_READ) found IN.py:490: No global (DISABLE) found IN.py:494: No global (ENABLE) found IN.py:522: No global (LOCK) found IN.py:524: No global (TRYLOCK) found IN.py:526: No global (LOCK_SH) found IN.py:528: No global (RW_RDLOCK) found IN.py:530: No global (RW_WRLOCK) found IN.py:534: No global (B_FALSE) found IN.py:540: No global (crholdn) found IN.py:542: No global (crfreen) found IN.py:546: No global (sizeof) found IN.py:546: No global (uint) found IN.py:597: No global (B_TRUE) found IN.py:601: No global (TICKS) found IN.py:601: No global (TICKS_BETWEEN) found IN.py:617: No global (engine) found IN.py:638: No global (STRM_PUTCNT) found IN.py:638: No global (l) found IN.py:718: No global (caddr_t) found IN.py:832: No global (canput) found IN.py:834: No global (canputnext) found xml.sax: __init__.py:78: Module (sys) re-imported From mal@lemburg.com Sat Aug 11 15:36:01 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 11 Aug 2001 16:36:01 +0200 Subject: [Python-Dev] ++x oddnes References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> Message-ID: <3B7542D1.3E38E579@lemburg.com> David Ascher wrote: > > Guido van Rossum wrote: > > > IMO not worth it changing the language. Train yourself to write x++ > > instead. :-) > > Sounds like an excellent one for PyChecker =) Why ? ++x can be put to some real use: I have a counter type which actually uses ++x to increment the counter. Note that the interpreter sees ++x as +(+x), that is the "+" is interpreted as unary + -- perfectly legal Python if you ask me. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From paulp@ActiveState.com Sun Aug 12 00:52:49 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Sat, 11 Aug 2001 16:52:49 -0700 Subject: [Python-Dev] ++x oddnes References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com> Message-ID: <3B75C551.DFDF915C@ActiveState.com> "M.-A. Lemburg" wrote: > >.. > > Why ? ++x can be put to some real use: I have a counter > type which actually uses ++x to increment the counter. Do you implement a matching "x++"? What's wrong with x.inc()? > Note that the interpreter sees ++x as +(+x), that is the "+" is > interpreted as unary + -- perfectly legal Python if you ask me. Legal but perverse. Your users will expect +++x and x++ to be equally valid and have their C semantics. Plus you are using a Python construct that means one thing to mean something totally different just because it means the totally different thing in other languages. In Python that syntax would usually be side-effect free (and useless). -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From DavidA@ActiveState.com Sun Aug 12 01:11:09 2001 From: DavidA@ActiveState.com (David Ascher) Date: Sat, 11 Aug 2001 17:11:09 -0700 Subject: [Python-Dev] ++x oddnes References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com> Message-ID: <3B75C99D.303E52EC@ActiveState.com> "M.-A. Lemburg" wrote: > > David Ascher wrote: > > > > Guido van Rossum wrote: > > > > > IMO not worth it changing the language. Train yourself to write x++ > > > instead. :-) > > > > Sounds like an excellent one for PyChecker =) > > Why ? ++x can be put to some real use: I have a counter > type which actually uses ++x to increment the counter. The point of linters is to find things that are likely to be errors. I claim that ++foo is likely to be an error resulting from the corruption of young minds by C++ more often than deliberate and appropriate. But you could turn it off when you used PyChecker =). --david From tim.one@home.com Sun Aug 12 02:24:15 2001 From: tim.one@home.com (Tim Peters) Date: Sat, 11 Aug 2001 21:24:15 -0400 Subject: [Python-Dev] New regrtest.py gimmick Message-ID: Since checkin email isn't working, I'm sending this to Python-Dev. More than once I've been burned by tests getting skipped on Windows when they shouldn't have been. Indeed, for at least one release we didn't ship the XML libraries on Windows because I had no idea the associated tests weren't *supposed* to get skipped. regrtest.py now prints this for me (note the last line): ... 138 tests OK. 25 tests skipped: test_al test_cd test_cl test_commands test_crypt test_dbm test_dl test_fcntl test_fork1 test_gdbm test_gl test_grp test_imgfile test_largefile test_linuxaudiodev test_mhlib test_nis test_openpty test_poll test_pty test_pwd test_signal test_socketserver test_sunaudiodev test_timing Those skips are all expected on win32. For you it will print something like: ... MMM tests OK. NN tests skipped: XXX YYY ... Ask someone to teach regrtest.py about which tests are expected to get skipped on [sys.platform]. You do that by changing the new _ExpectedSkips.__init__ near the bottom of regrtest.py. If sys.platform isn't enough to determine the set of skipped tests, then sys.platform isn't particularly useful <0.9 wink>. From m.favas@per.dem.csiro.au Sun Aug 12 02:42:02 2001 From: m.favas@per.dem.csiro.au (Mark Favas) Date: Sun, 12 Aug 2001 09:42:02 +0800 Subject: [Python-Dev] New regrtest.py gimmick Message-ID: <3B75DEEA.7B12E66A@per.dem.csiro.au> [Tim tests for expected platform-specific skips] ... just a slight change needed for the new gimmick to be happy on other platforms Instead of: e = _ExpectedSkips() if e.isvalid(): surprise = _Set(skipped) - e.getexpected() plat = sys.platform #<------------------------------------- if surprise: print count(len(surprise), "skip"), \ "unexpected on", plat + ":", \ " ".join(surprise.tolist()) else: print "Those skips are all expected on", plat + "." else: print "Ask someone to teach regrtest.py about which tests are" print "expected to get skipped on", plat + "." try: e = _ExpectedSkips() plat = sys.platform #<+++++++++++++++++++++++++++++++++++++++++ if e.isvalid(): surprise = _Set(skipped) - e.getexpected() if surprise: print count(len(surprise), "skip"), \ "unexpected on", plat + ":", \ " ".join(surprise.tolist()) else: print "Those skips are all expected on", plat + "." else: print "Ask someone to teach regrtest.py about which tests are" print "expected to get skipped on", plat + "." -- Mark Favas - m.favas@per.dem.csiro.au CSIRO, Private Bag No 5, Wembley, Western Australia 6913, AUSTRALIA From tim.one@home.com Sun Aug 12 03:02:34 2001 From: tim.one@home.com (Tim Peters) Date: Sat, 11 Aug 2001 22:02:34 -0400 Subject: [Python-Dev] RE: New regrtest.py gimmick In-Reply-To: <3B75DEEA.7B12E66A@per.dem.csiro.au> Message-ID: ]Mark Favas] > ... just a slight change needed for the new gimmick to be happy on other > platforms What other platforms? If you were running on Windows like you're supposed to, this wouldn't have bothered you . the-suggested-change-has-been-gratefully-checked-in-ly y'rs - tim From tim.one@home.com Sun Aug 12 05:05:37 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 12 Aug 2001 00:05:37 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <5.1.0.14.0.20010801020813.022a56b8@24.8.96.48> Message-ID: [Dan Sugalski, on Wednesday, August 01, 2001 2:20 AM] > ... > Ouch, I'd bet that hurts. Has anyone timed the difference between > making lots of getc calls and making a few larger reads and managing > the buffers internally? I can see it going either way, and another > data point would be useful to have. There's been lots of this in Python-Dev, like in this thread: http://aspn.activestate.com/ASPN/Mail/Message/600485 I'll quote the high-order bit: My line-at-a-time test case used (rounding to nearest whole integers) 30 seconds in Python and 6 in Perl. The result of testing many changes to Python's implementation was that the excess 24 seconds broke down like so: 17 spent inside internal MS threadsafe getc() lock/unlock routines 5 uncertain, but evidence suggests much of it due to MS malloc/realloc (Perl does its own memory mgmt) 2 for not copying directly out of the platform FILE* implementation struct in a highly optimized loop (like Perl does) My last checkin to fileobject.c reclaimed 17 seconds on Win98SE while remaining threadsafe, via a combination of locking per line instead of per character, and invoking realloc much less often (only for lines exceeding 200 chars). Note that thread overhead is overwhelmingly the biggest hangup. Python has two threadsafe input tricks now: 1. On platforms that have flockfile(), funlockfile(), and getc_unlocked(), the last is used in a loop bracketed by the first two. 2. At least on Windows, which doesn't have those, we use the platform fgets() in an excruciating way, tricking it into letting us read lines with embedded null bytes. Oddly enough, in the timing reports I saw, approach #1 was never faster than approach #2, and on at least one platform (Tru64, IIRC) was slower. Of course fgets() is a primitive in std C because they *wanted* to make it possible for vendors to optimize it (in the ways Perl does), but it appears very few vendors do optimize it. On Windows it's the same old getc()-in-a-loop, but they lock/unlock the stream only once per fgets call (using internal stream functions that aren't exposed). The "2 seconds for not copying directly ... like Perl does" I reported above came from hacking together a thread-unsafe line input routine that used the same FILE* tricks Perl uses. That is, thread-unsafe getc-in-a-loop was 2 seconds slower than using thread-unsafe FILE* tricks. That's significant in absolute terms, but was lost in the noise compared to the other stuff we were fighting. From guido@python.org Sun Aug 12 05:11:56 2001 From: guido@python.org (Guido van Rossum) Date: Sun, 12 Aug 2001 00:11:56 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Your message of "Sun, 12 Aug 2001 00:05:37 EDT." References: Message-ID: <200108120411.AAA08638@cj20424-a.reston1.va.home.com> I don't have context for this thread anymore, so perhaps this is already what's being discussed: Perhaps a less ambitious (than Parrot) but still very useful project would be a portable stdio-like library that has the features dynamic languages need? This could be based upon a few low-level calls like read() and write() on Unix. These tend to exist in most platforms C libraries (even on Windows) because they're so darned useful. For full "portability" it should be possible to implement these yourself if the platform doesn't have them (or if they are not fast enough!). --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Sun Aug 12 05:23:42 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 12 Aug 2001 00:23:42 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <20010802103006.H1667@netthink.co.uk> Message-ID: [Simon Cozens on Thursday, August 02, 2001 1:30 PM] > ... > Oh, and here's our divide, for comparison: > > PP(pp_divide) > { > dSP; dATARGET; tryAMAGICbin(div,opASSIGN); > { > dPOPPOPnnrl; > NV value; > if (right == 0.0) > DIE(aTHX_ "Illegal division by zero"); > value = left / right; > PUSHn( value ); > RETURN; > } > } > > See, this is why we need a new interpreter. :) Yes, I've tried to read the Perl source before . What does "/" mean here? You're using native C arithmetic, or is this a C++ overload? Even simple arithmetic is long-winded in Python, because, for example, the int operations check for overflow, while the float operations allow for catching IEEE-754 exceptions (or SIGFPE in general) and translating them into Python-level exceptions. We also go thru piles and piles of code coercing among different numeric types. If you were to step thru the Python line x = 3.0 / 4 in a debugger, it's a race on Windows between your finger getting numb and the OS crashing ... From tim.one@home.com Sun Aug 12 09:49:40 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 12 Aug 2001 04:49:40 -0400 Subject: [Python-Dev] future and present In-Reply-To: Message-ID: There's a new Tools/scripts/cleanfuture.py to delete (or just remove the appropriate parts of) obsolete future statements. If you want any changes, make 'em yourself. In particular, some Unixoid will probably want to change it to fiddle file permissions. Note that this serves as a nice example of why generators are truly useful -- it doesn't use generators itself, but uses the .next() method of the generator interface to tokenize.py. This lets it parse files in a natural way without building godawful state machines by hand, yet runs very fast because future stmts can only appear near the start of a file (and so tokenize.py only has to work until the driver determines no more future stmts can appear). From mwh@python.net Sun Aug 12 10:22:40 2001 From: mwh@python.net (Michael Hudson) Date: 12 Aug 2001 05:22:40 -0400 Subject: [Python-Dev] Re: [Python-checkins] testing In-Reply-To: Guido van Rossum's message of "Sat, 11 Aug 2001 21:56:44 -0400" References: <200108120156.VAA06262@cj20424-a.reston1.va.home.com> Message-ID: <2mu1zd8wf3.fsf@starship.python.net> Guido van Rossum writes: > warning: somehow the python checkin mail is dropped on the floor. And bugs email, and a message I just sent myself from sf.net's web form all seem to be disappearing too. Mail sent to @users.sourceforge.net still seems to work, so at least *something* still works over there... Cheers, M. From martin@loewis.home.cs.tu-berlin.de Sun Aug 12 10:30:06 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Sun, 12 Aug 2001 11:30:06 +0200 Subject: [Python-Dev] CVS build breakage: snprintf finds its way into socketmodule.c In-Reply-To: (message from Alex Coventry on 24 Jul 2001 15:04:26 -0400) References: <200107240638.f6O6cPn03340@mira.informatik.hu-berlin.de> Message-ID: <200108120930.f7C9U6902163@mira.informatik.hu-berlin.de> > In PySocket_getaddrinfo, would it make sense to increase the allocation > of pbuf from 10 characters to, say, 30 characters, in case > > sprintf(pbuf, "%ld", PyInt_AsLong(pobj)); > > gets run on a 64-bit machine? It isn't strictly necessary, since the port number should never be that large; it doesn't hurt to increase the size, either, so I did that. Thanks, Martin From mal@lemburg.com Sun Aug 12 11:47:38 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Sun, 12 Aug 2001 12:47:38 +0200 Subject: [Python-Dev] ++x oddness References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com> <3B75C551.DFDF915C@ActiveState.com> Message-ID: <3B765ECA.1E806DD7@lemburg.com> Paul Prescod wrote: > > "M.-A. Lemburg" wrote: > > > >.. > > > > Why ? ++x can be put to some real use: I have a counter > > type which actually uses ++x to increment the counter. > > Do you implement a matching "x++"? What's wrong with x.inc()? x++ does not work. x.inc() would work too, but it involves a method lookup and function call, which "++x" doesn't. > > Note that the interpreter sees ++x as +(+x), that is the "+" is > > interpreted as unary + -- perfectly legal Python if you ask me. > > Legal but perverse. Your users will expect +++x and x++ to be equally > valid and have their C semantics. Plus you are using a Python construct > that means one thing to mean something totally different just because it > means the totally different thing in other languages. In Python that > syntax would usually be side-effect free (and useless). Well, it is not necessarily side-effect free, since Python does call a method slot on the underlying object and that slot can change of course change state. Nevermind, the syntax is only an experiment -- I just wanted to point out that "++x" is legal in Python (you can place any number of + before the object, BTW). Some examples: >>> +++++1 1 >>> ------2 2 >>> --2 2 >>> -2 -2 >>> ---2 -2 >>> ----2 2 -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From guido@python.org Sun Aug 12 15:53:29 2001 From: guido@python.org (Guido van Rossum) Date: Sun, 12 Aug 2001 10:53:29 -0400 Subject: [Python-Dev] CVS commit permission for Andrew MacIntyre Message-ID: <200108121453.KAA15888@cj20424-a.reston1.va.home.com> The SF patch manager has two large patch sets for OS/2 from Andrew I MacIntyre. By and large they are benign. Rather than applying these myself, I suggest giving Andrew checkin permission. Does anybody have an objection? --Guido van Rossum (home page: http://www.python.org/~guido/) From simon@netthink.co.uk Sun Aug 12 20:25:39 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Sun, 12 Aug 2001 20:25:39 +0100 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <200108120411.AAA08638@cj20424-a.reston1.va.home.com>; from guido@python.org on Sun, Aug 12, 2001 at 12:11:56AM -0400 References: <200108120411.AAA08638@cj20424-a.reston1.va.home.com> Message-ID: <20010812202539.C1231@deep-dark-truthful-mirror> On Sun, Aug 12, 2001 at 12:11:56AM -0400, Guido van Rossum wrote: > Perhaps a less ambitious (than Parrot) but still very useful project > would be a portable stdio-like library that has the features dynamic > languages need? I hope you do not think that I'm blowing Perl's trumpet again, but I see one of my roles here as sharing knowledge about things that Python has that Perl needs and vice versa. Perl 5.7.x has such a library, and it's my expectation (another sideways glance at Dan) that if we did Parrot, it would include this library. PerlIO attempts to be a very portable stdio (it's running on everything that Perl runs on, AFAIK) and it supports "line disciplines" in the sfio sense: you can stuff processing modules between the file and the filehandle. This allows us to do anything between altering buffering semantics, and decompressing and decrypting files on the fly. It'll need to be somewhat rewritten for Perl 6, at which point I'm hoping it'll be done in such a way that it'll make it easier to extract from the Perl mess around it and be made available to other projects. -- Putting heated bricks close to the news.admin.net-abuse.* groups. -- Megahal (trained on asr), 1998-11-06 From tim.one@home.com Sun Aug 12 20:42:44 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 12 Aug 2001 15:42:44 -0400 Subject: [Python-Dev] CVS commit permission for Andrew MacIntyre In-Reply-To: <200108121453.KAA15888@cj20424-a.reston1.va.home.com> Message-ID: [Guido van Rossum] > The SF patch manager has two large patch sets for OS/2 from Andrew I > MacIntyre. By and large they are benign. Rather than applying these > myself, I suggest giving Andrew checkin permission. Does anybody have > an objection? No objection here. BTW, I asked Andrew to upload his OS/2 patches -- but I had no idea they'd be so large ! From thomas@xs4all.net Sun Aug 12 21:25:47 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Sun, 12 Aug 2001 22:25:47 +0200 Subject: [Python-Dev] CVS commit permission for Andrew MacIntyre In-Reply-To: <200108121453.KAA15888@cj20424-a.reston1.va.home.com> References: <200108121453.KAA15888@cj20424-a.reston1.va.home.com> Message-ID: <20010812222547.I874@xs4all.nl> On Sun, Aug 12, 2001 at 10:53:29AM -0400, Guido van Rossum wrote: > The SF patch manager has two large patch sets for OS/2 from Andrew I > MacIntyre. By and large they are benign. Rather than applying these > myself, I suggest giving Andrew checkin permission. Does anybody have > an objection? None, though I'd suggest subscribing him to python-dev too, for the reasons I gave when I found out Just had checkin privs but wasn't on py-dev. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido@python.org Sun Aug 12 21:15:05 2001 From: guido@python.org (Guido van Rossum) Date: Sun, 12 Aug 2001 16:15:05 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Your message of "Sun, 12 Aug 2001 20:25:39 BST." <20010812202539.C1231@deep-dark-truthful-mirror> References: <200108120411.AAA08638@cj20424-a.reston1.va.home.com> <20010812202539.C1231@deep-dark-truthful-mirror> Message-ID: <200108122015.QAA16295@cj20424-a.reston1.va.home.com> > Perl 5.7.x has such a library, and it's my expectation (another sideways > glance at Dan) that if we did Parrot, it would include this library. > PerlIO attempts to be a very portable stdio (it's running on everything > that Perl runs on, AFAIK) and it supports "line disciplines" in the sfio > sense: you can stuff processing modules between the file and the > filehandle. This allows us to do anything between altering buffering > semantics, and decompressing and decrypting files on the fly. Great! > It'll need to be somewhat rewritten for Perl 6, at which point I'm > hoping it'll be done in such a way that it'll make it easier to extract > from the Perl mess around it and be made available to other projects. I hope that even if the Parrot goal is to support all languages, PerlIO can be made a separate and independent project, not relying on Perl internals. (And I'd change the name, but maybe that's asking too much. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From dan@sidhe.org Sun Aug 12 21:10:38 2001 From: dan@sidhe.org (Dan Sugalski) Date: Sun, 12 Aug 2001 16:10:38 -0400 (EDT) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <200108122015.QAA16295@cj20424-a.reston1.va.home.com> Message-ID: On Sun, 12 Aug 2001, Guido van Rossum wrote: > > Perl 5.7.x has such a library, and it's my expectation (another sideways > > glance at Dan) that if we did Parrot, it would include this library. > > PerlIO attempts to be a very portable stdio (it's running on everything > > that Perl runs on, AFAIK) and it supports "line disciplines" in the sfio > > sense: you can stuff processing modules between the file and the > > filehandle. This allows us to do anything between altering buffering > > semantics, and decompressing and decrypting files on the fly. > > Great! Two of the other design requirements are good async I/O support and good event handling support. (So there's something to handle the completed async I/Os) Rumors that we're just filing the serial numbers off the VMS I/O system are, of course, wildly exaggerated. ;-) > > It'll need to be somewhat rewritten for Perl 6, at which point I'm > > hoping it'll be done in such a way that it'll make it easier to extract > > from the Perl mess around it and be made available to other projects. > > I hope that even if the Parrot goal is to support all languages, > PerlIO can be made a separate and independent project, not relying on > Perl internals. (And I'd change the name, but maybe that's asking too > much. :-) Nah, a name change would certainly be in order--we'd have to call it the ParrotIO system. :) Making the core bits modular is one of the goals, for no other reason than to make it easy to have someone responsible for its maintenance and development that doesn't need to be intimately familiar with the rest of the interpreter system. And so it'll still function properly when we compile down to native code and don't actually *have* an interpreter handy. It ought to be an interesting project. Suggestions for features are, of course, always welcome. Dan From jack@oratrix.nl Sun Aug 12 22:31:31 2001 From: jack@oratrix.nl (Jack Jansen) Date: Sun, 12 Aug 2001 23:31:31 +0200 Subject: [Python-Dev] Any NeXT users out there? Message-ID: <20010812213136.8DA5C162E06@oratrix.oratrix.nl> Are there any NeXT Python users left out there? The reason I'm asking is that we're changing the --with-next-framework support beyond all recognition to make it work on Mac OS X. I am probably breaking what was there previously, BUT I'm pretty convinced that that didn't work anyway as it stood. At least, according to all rumour Mac OS X frameworks are very very similar to NeXT frameworks, and the support that was in configure.in and Makefile.pre.in was really no more than building a .dylib. There was no support to then install this dylib (but that could be/have been in a separate script), and moreover Python would never have been able to find the .dylib (again, as far as I can tell with a few weeks of OSX experience and no NeXT experience). -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From jack@oratrix.nl Sun Aug 12 23:17:19 2001 From: jack@oratrix.nl (Jack Jansen) Date: Mon, 13 Aug 2001 00:17:19 +0200 Subject: [Python-Dev] Feedback wanted on --with-next-framework patch Message-ID: <20010812221724.35589162E06@oratrix.oratrix.nl> I've put a patch to configure.in, Makefile.pre.in and Modules/getpath.c into the patch manager, number 450351 (will pasting URLs here work? The url is https://sourceforge.net/tracker/index.php?func=detail&aid=450350&group_id=5470&atid=305470 I've assigned it to Martin because he's been helpful with configure stuff in the past, but as we've added quite a few targets to the Makefile (used for framework builds only) I can't be sure we didn't break things on other platforms. "we", by the way, is myself and Tony Lownds, who did a large part of the work. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From martin@loewis.home.cs.tu-berlin.de Sun Aug 12 23:34:48 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Mon, 13 Aug 2001 00:34:48 +0200 Subject: [Python-Dev] Any NeXT users out there? Message-ID: <200108122234.f7CMYml01363@mira.informatik.hu-berlin.de> > The reason I'm asking is that we're changing the > --with-next-framework support beyond all recognition to make it > work on Mac OS X. I am probably breaking what was there previously, > BUT I'm pretty convinced that that didn't work anyway as it stood. Do you really need to call the option --with-next-framework? First, I think it should be an --enable option, since you are not offering to use or not to use some external tool or library. Furthermore, I find it very confusing to have to invoke incantations involving NeXT when compiling software for Mac OS. That Steve Jobs is head of the company is *not* enough reason :-) Finally, if you are going to break what it currently does, you may just as well remove the current code. I don't know whether NeXT(Step?) should be supported anymore, but I definitely know that MacOS should not be the vehicle to keep the NeXT code forever. I also know that system specific code simpler if you know what system it is intended to work on. Out of curiosity: Just what is a framework? (Should that be Jack, what is a framework ?-) Regards, Martin From paulp@ActiveState.com Mon Aug 13 00:41:51 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Sun, 12 Aug 2001 16:41:51 -0700 Subject: [Python-Dev] Parrot -- should life imitate satire? References: Message-ID: <3B77143F.F9158ED1@ActiveState.com> Dan Sugalski wrote: > >... > > Nah, a name change would certainly be in order--we'd have to call it the > ParrotIO system. :) Making the core bits modular is one of the goals, for > no other reason than to make it easy to have someone responsible for its > maintenance and development that doesn't need to be intimately familiar > with the rest of the interpreter system. And so it'll still function > properly when we compile down to native code and don't actually *have* an > interpreter handy. Would it make sense to poll both communities for people interested in helping with this effort and maybe making it a standalone project (perhaps on SourceForge?)? In fact, there are many bits of the C library that Python people would like to replace or improve but we don't have the resources alone. Maybe if all of the scripting language groups got together we would be able to do it. I'm not talking about the larger project of sharing bytecodes and so forth, but I could see that we probably have very similar needs with respect to files, memory allocation, networking, event loops, process handling and maybe threads. And the other scripting languages are probably not too far off either. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From esr@thyrsus.com Mon Aug 13 00:47:33 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Sun, 12 Aug 2001 19:47:33 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <3B77143F.F9158ED1@ActiveState.com>; from paulp@ActiveState.com on Sun, Aug 12, 2001 at 04:41:51PM -0700 References: <3B77143F.F9158ED1@ActiveState.com> Message-ID: <20010812194733.A8000@thyrsus.com> Paul Prescod : > Would it make sense to poll both communities for people interested in > helping with this effort and maybe making it a standalone project > (perhaps on SourceForge?)? > > In fact, there are many bits of the C library that Python people would > like to replace or improve but we don't have the resources alone. Maybe > if all of the scripting language groups got together we would be able to > do it. I'm not talking about the larger project of sharing bytecodes and > so forth, but I could see that we probably have very similar needs with > respect to files, memory allocation, networking, event loops, process > handling and maybe threads. And the other scripting languages are > probably not too far off either. I think this is a good idea in itself. And would also be a good pilot project for the Parrot unification. -- Eric S. Raymond The direct use of physical force is so poor a solution to the problem of limited resources that it is commonly employed only by small children and great nations. -- David Friedman From sterling@designmultimedia.com Mon Aug 13 12:49:23 2001 From: sterling@designmultimedia.com (Sterling Hughes) Date: Mon, 13 Aug 2001 07:49:23 -0400 (EDT) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <3B77143F.F9158ED1@ActiveState.com> Message-ID: On Sun, 12 Aug 2001, Paul Prescod wrote: > Dan Sugalski wrote: > > > >... > > > > Nah, a name change would certainly be in order--we'd have to call it the > > ParrotIO system. :) Making the core bits modular is one of the goals, for > > no other reason than to make it easy to have someone responsible for its > > maintenance and development that doesn't need to be intimately familiar > > with the rest of the interpreter system. And so it'll still function > > properly when we compile down to native code and don't actually *have* an > > interpreter handy. > > Would it make sense to poll both communities for people interested in > helping with this effort and maybe making it a standalone project > (perhaps on SourceForge?)? > > In fact, there are many bits of the C library that Python people would > like to replace or improve but we don't have the resources alone. Maybe > if all of the scripting language groups got together we would be able to > do it. I'm not talking about the larger project of sharing bytecodes and > so forth, but I could see that we probably have very similar needs with > respect to files, memory allocation, networking, event loops, process > handling and maybe threads. And the other scripting languages are > probably not too far off either. > I know that there are some PHP developers who are also thinking along the same lines -- myself for one. :) Also, another area this might reach is into the extension space, we're all developing the same extension for each of the different languages, it would be great to find a way to pool the resources of all the extension developers to create one extension together (ie, a singular XSLT extension, used in Perl, Python, PHP, etc.) -Sterling From simon@netthink.co.uk Mon Aug 13 01:06:33 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Mon, 13 Aug 2001 01:06:33 +0100 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <3B77143F.F9158ED1@ActiveState.com>; from paulp@ActiveState.com on Sun, Aug 12, 2001 at 04:41:51PM -0700 References: <3B77143F.F9158ED1@ActiveState.com> Message-ID: <20010813010633.A6537@deep-dark-truthful-mirror> On Sun, Aug 12, 2001 at 04:41:51PM -0700, Paul Prescod wrote: > Would it make sense to poll both communities for people interested in > helping with this effort and maybe making it a standalone project > (perhaps on SourceForge?)? Yes. But you'll have to talk to Nick () if you want to play with the current code, since it's his baby. > In fact, there are many bits of the C library that Python people would > like to replace or improve but we don't have the resources alone. *nod*. Perl reimplements far too much of C for my liking. (aton, sprintf, malloc [1] and friends.) I keep telling 'em to use glib. [1] ... but then any C program over a given size reimplements malloc ... -- The best book on programming for the layman is "Alice in Wonderland"; but that's because it's the best book on anything for the layman. - Alan Perlis From paulp@ActiveState.com Mon Aug 13 02:43:35 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Sun, 12 Aug 2001 18:43:35 -0700 Subject: [Python-Dev] Free threading References: Message-ID: <3B7730C7.FE3983A2@ActiveState.com> Tim Peters wrote: > >... > > Well, I'm not a Tcl guy, but Tcl has historically had no threads at all, yet > did have an event model where the *illusion* of multiple threads could still > be put to good use. I wouldn't fall over dead with surprise if that's all > there were to it: Extreme Fear of Threads . No, Tcl has real threads now. Tcl and Perl both share a model that the Perl guys call "Ithreads". (for interpreter threads or independent threads, I guess) where each interpreter is firewalled from the other unless you ask to share information. Tcl's threads were driven by the AOL guys' need to get massive scalability. >... > That's cool. There isn't much code supporting multiple interpreters, so > there's little to gain by getting rid of it now. I just wish it were > *used* -- and that it had a more compelling use case . Why do people use mod_python, fcgi, mod_snake, mod_php and all of those other things instead of pure CGI? The usual claim is that they dislike the cost of forking a process and loading the interpreter code. Now imagine a user with a 20 processor machine. She isn't going to be happy with the price of forking processes and using IPC for information sharing either. On the other hand, she isn't going to be happy with a shared GIL. Free threading helps, but if sharing data has a performance cost (e.g. by requiring reference count operations to be locked, or requiring mutexes on dictionaries) then you might not want to pay that cost either. The Perl guys convinced me of that much. The most popular "embedded scripting languages" (PHP and VBScript/ASP) use this totally independent thread model. As far as I know, neither has a concept of sharing information between threads. To a PHP programmer, that's what SQL and browser cookies are for. :-) -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From swm@linuxworld.com.au Mon Aug 13 03:04:32 2001 From: swm@linuxworld.com.au (Gavin Sherry) Date: Mon, 13 Aug 2001 12:04:32 +1000 (EST) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Message-ID: On Mon, 13 Aug 2001, Sterling Hughes wrote: > On Sun, 12 Aug 2001, Paul Prescod wrote: > > > Dan Sugalski wrote: > > > > > >... > > > > > > Nah, a name change would certainly be in order--we'd have to call it the > > > ParrotIO system. :) Making the core bits modular is one of the goals, for > > > no other reason than to make it easy to have someone responsible for its > > > maintenance and development that doesn't need to be intimately familiar > > > with the rest of the interpreter system. And so it'll still function > > > properly when we compile down to native code and don't actually *have* an > > > interpreter handy. > > > > Would it make sense to poll both communities for people interested in > > helping with this effort and maybe making it a standalone project > > (perhaps on SourceForge?)? > > > I know that there are some PHP developers who are also thinking > along the same lines -- myself for one. :) > > Also, another area this might reach is into the extension space, > we're all developing the same extension for each of the different > languages, it would be great to find a way to pool the resources of all > the extension developers to create one extension together (ie, a singular > XSLT extension, used in Perl, Python, PHP, etc.) By introducing another layer of abstraction we will most probably decrease the performance of all languages[1]. That is, since the universal interface to, given your example, the Sablotron XSLT library is the API and Perl, Python and PHP have internals different enough that there would have to be a certain amount of glue and translation work between the structures of the extension layer and the internals of the interpretter. Building extensions to these languages involves a lot of hack work I agree. The reason for this is that it is merely connecting the wires between an API and the language of interest. Anything more than that and you'll be damaging the performance of Perl, Python, PHP... [1] The situation where you could perhaps increase performance is the converse of what is being suggested. Presumably, if it is in the interest of language developers to have a unified extension layer, then the point of different which language developers want to maintain with with other languages is the syntax/grammar of the language. As I see it, along this line of thinking it makes sense to look at the feasibility of a language system (internals and extension layer) which handles plugable scripting engines. As such, the majority of work which goes into language development can be concentrated while allowing, to the users, automony in their choice of language. > > -Sterling > > Gavin From rasmus@php.net Mon Aug 13 03:30:54 2001 From: rasmus@php.net (Rasmus Lerdorf) Date: Sun, 12 Aug 2001 19:30:54 -0700 (PDT) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Message-ID: > By introducing another layer of abstraction we will most probably decrease > the performance of all languages[1]. That is, since the universal > interface to, given your example, the Sablotron XSLT library is the API > and Perl, Python and PHP have internals different enough that there would > have to be a certain amount of glue and translation work between the > structures of the extension layer and the internals of the interpretter. It should still be possible to come up with some guidelines for writing a single extension that supports multiple scripting languages. Especially for something like Sablotron which isn't that complex. You feed it some XML and some XSL and it spits back a result. Most of the work in writing the extension is figuring out how the user-visible API should look and then translating that API along with its function/method calls and argument type mangling to whatever the thing that is being glued is expecting. Note that I am not talking about runtime binary compatibility here. I am talking about source compatibility where potentially big chunks of code would be very different across the different languages. In an approach like this I don't see a performance issue. It is more of an education issue actually. And with a bit of effort I bet we could come up with enough common ground, or at least some nifty macro tricks, where people wouldn't feel overwhelmed by the task of supporting multiple scripting languages when they sat down to write some new backend library. Perhaps it would be a good experiment to take some really simple thing and write Perl, Python, PHP, Ruby, and Tcl extensions for it and compare notes. See where the overlap is, see where the biggest divergence is, and figure out if some of that divergence could be eliminated with some clever tricks. > [1] The situation where you could perhaps increase performance is the > converse of what is being suggested. Presumably, if it is in the interest > of language developers to have a unified extension layer, then the point > of different which language developers want to maintain with with other > languages is the syntax/grammar of the language. > > As I see it, along this line of thinking it makes sense to look at the > feasibility of a language system (internals and extension layer) which > handles plugable scripting engines. As such, the majority of work which > goes into language development can be concentrated while allowing, to the > users, automony in their choice of language. Sure, but now you are back on the holy grail and CLR stuff again. Baby steps... -Rasmus From skip@pobox.com (Skip Montanaro) Sat Aug 11 03:43:44 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 10 Aug 2001 21:43:44 -0500 Subject: [Python-Dev] Silly pystones question Message-ID: <15220.39904.966751.603159@beluga.mojam.com> In pystones.Proc0, most of the variables declared global are never assigned. Was that just slavish adherence to the style of code from which it was adapted or was the code changed at some point in the past, making the global declarations superfluous? Seems to me that the only time the global keyword should be used is when an assignment to the variable occurs in the code and you want to make sure you're modifying the global name. This is obviously a silly piece of code in which to question this technique, but I was looking at it to gauge the effects of possible global object access optimization. My first assumption was that a variable declared global wouldn't be subject to any optimization, but now I realize I'd have to look for variables that were operands of LOAD_GLOBAL instructions but not STORE_GLOBAL instructions. Skip From otaylor@redhat.com Mon Aug 13 04:05:43 2001 From: otaylor@redhat.com (Owen Taylor) Date: 12 Aug 2001 23:05:43 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Rasmus Lerdorf's message of "Sun, 12 Aug 2001 19:30:54 -0700 (PDT)" References: Message-ID: Rasmus Lerdorf writes: > > By introducing another layer of abstraction we will most probably decrease > > the performance of all languages[1]. That is, since the universal > > interface to, given your example, the Sablotron XSLT library is the API > > and Perl, Python and PHP have internals different enough that there would > > have to be a certain amount of glue and translation work between the > > structures of the extension layer and the internals of the interpretter. > > It should still be possible to come up with some guidelines for writing a > single extension that supports multiple scripting languages. Especially > for something like Sablotron which isn't that complex. You feed it some > XML and some XSL and it spits back a result. Most of the work in writing > the extension is figuring out how the user-visible API should look and > then translating that API along with its function/method calls and > argument type mangling to whatever the thing that is being glued is > expecting. > > Note that I am not talking about runtime binary compatibility here. I am > talking about source compatibility where potentially big chunks of code > would be very different across the different languages. In an approach > like this I don't see a performance issue. It is more of an education > issue actually. > > And with a bit of effort I bet we could come up with enough common ground, > or at least some nifty macro tricks, where people wouldn't feel > overwhelmed by the task of supporting multiple scripting languages when > they sat down to write some new backend library. I think knowledge of the rules for writing bindable libraries is much more useful than macro tricks. Writing down the rules would be a big project, though the set of rules isn't that big. A few important ones are: - User data stored with a structure should have an associated notifier. - Callback functions must take not just a function, but a function,user data, destroy notifier combination. - Data structures should be one of: * Immutable and refcounted. * Copy by value. * Copy by reference, reference counted, with destroy notification and a user data facility for maintaining persistant language <=> library associates. - Keep the C simple. Opaque structure with accessors is the ideal, structures initialized with magic macros one of the worse offendors. Etc. If you follow these rules, you get a a library where writing bindings for it is a brain-dead task. But still task. A library like GObject that standardizes memory management, and object-oriented structures helps quite a bit, but I'm certainly of the opinion that the eventual goal has to be - write a library using tools that produce an introspectable descriptions of its interfaces. - Language bindings are automatic and can be done without a compilation step. Regards, Owen From otaylor@redhat.com Mon Aug 13 04:10:02 2001 From: otaylor@redhat.com (Owen Taylor) Date: 12 Aug 2001 23:10:02 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Owen Taylor's message of "12 Aug 2001 23:05:43 -0400" References: Message-ID: Owen Taylor writes: > Rasmus Lerdorf writes: > > > > By introducing another layer of abstraction we will most probably decrease > > > the performance of all languages[1]. That is, since the universal > > > interface to, given your example, the Sablotron XSLT library is the API > > > and Perl, Python and PHP have internals different enough that there would > > > have to be a certain amount of glue and translation work between the > > > structures of the extension layer and the internals of the interpretter. > > > > It should still be possible to come up with some guidelines for writing a > > single extension that supports multiple scripting languages. Especially > > for something like Sablotron which isn't that complex. You feed it some > > XML and some XSL and it spits back a result. Most of the work in writing > > the extension is figuring out how the user-visible API should look and > > then translating that API along with its function/method calls and > > argument type mangling to whatever the thing that is being glued is > > expecting. > > > > Note that I am not talking about runtime binary compatibility here. I am > > talking about source compatibility where potentially big chunks of code > > would be very different across the different languages. In an approach > > like this I don't see a performance issue. It is more of an education > > issue actually. > > > > And with a bit of effort I bet we could come up with enough common ground, > > or at least some nifty macro tricks, where people wouldn't feel > > overwhelmed by the task of supporting multiple scripting languages when > > they sat down to write some new backend library. > > I think knowledge of the rules for writing bindable libraries is > much more useful than macro tricks. [...] > Etc. If you follow these rules, you get a a library where writing > bindings for it is a brain-dead task. But still task. A library like > GObject that standardizes memory management, and object-oriented > structures helps quite a bit, but I'm certainly of the opinion that > the eventual goal has to be > > - write a library using tools that produce an introspectable > descriptions of its interfaces. > > - Language bindings are automatic and can be done without > a compilation step. Oh yes, and the thing I forgot to mention: Write a library and the language bindings will come. The problem is getting a vast number of language bindings (the list of 38 language bindings for GTK+ at http://erik.bagfors.nu/gnome/languages.html is almost certainly not even complete). The problem is keeping language bindings complete and up-to-date. This is the biggest reason why automation is crucual. Regards, Owen From rasmus@php.net Mon Aug 13 04:17:13 2001 From: rasmus@php.net (Rasmus Lerdorf) Date: Sun, 12 Aug 2001 20:17:13 -0700 (PDT) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Message-ID: > - write a library using tools that produce an introspectable > descriptions of its interfaces. > > - Language bindings are automatic and can be done without > a compilation step. Of course, but this doesn't help much for all the legacy libraries out there that we all have to deal with. We all have the rather thankless task of having to figure out stuff like liboci8. It would be very nice if that work could be centralized into a single effort. But I agree, if such rules were written down and actually abided by in third-party libraries, our lives would be much easier. Perhaps something like swig is as far as we can go on the legacy libs though. -Rasmus From sterling@designmultimedia.com Mon Aug 13 16:21:30 2001 From: sterling@designmultimedia.com (Sterling Hughes) Date: Mon, 13 Aug 2001 11:21:30 -0400 (EDT) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Message-ID: On 12 Aug 2001, Owen Taylor wrote: > > Rasmus Lerdorf writes: > > > > By introducing another layer of abstraction we will most probably decrease > > > the performance of all languages[1]. That is, since the universal > > > interface to, given your example, the Sablotron XSLT library is the API > > > and Perl, Python and PHP have internals different enough that there would > > > have to be a certain amount of glue and translation work between the > > > structures of the extension layer and the internals of the interpretter. > > > > It should still be possible to come up with some guidelines for writing a > > single extension that supports multiple scripting languages. Especially > > for something like Sablotron which isn't that complex. You feed it some > > XML and some XSL and it spits back a result. Most of the work in writing > > the extension is figuring out how the user-visible API should look and > > then translating that API along with its function/method calls and > > argument type mangling to whatever the thing that is being glued is > > expecting. > > > > Note that I am not talking about runtime binary compatibility here. I am > > talking about source compatibility where potentially big chunks of code > > would be very different across the different languages. In an approach > > like this I don't see a performance issue. It is more of an education > > issue actually. > > > > And with a bit of effort I bet we could come up with enough common ground, > > or at least some nifty macro tricks, where people wouldn't feel > > overwhelmed by the task of supporting multiple scripting languages when > > they sat down to write some new backend library. > > I think knowledge of the rules for writing bindable libraries is > much more useful than macro tricks. > > Writing down the rules would be a big project, though the set > of rules isn't that big. A few important ones are: > > - User data stored with a structure should have an associated > notifier. > > - Callback functions must take not just a function, but > a function,user data, destroy notifier combination. > > - Data structures should be one of: > > * Immutable and refcounted. > * Copy by value. > * Copy by reference, reference counted, with destroy > notification and a user data facility for maintaining > persistant language <=> library associates. > > - Keep the C simple. Opaque structure with accessors is > the ideal, structures initialized with magic macros > one of the worse offendors. > Yes! Yes! Yes! Its surprising how many library authors seem to ignore this. > Etc. If you follow these rules, you get a a library where writing > bindings for it is a brain-dead task. But still task. A library like > GObject that standardizes memory management, and object-oriented > structures helps quite a bit, but I'm certainly of the opinion that > the eventual goal has to be > > - write a library using tools that produce an introspectable > descriptions of its interfaces. > > - Language bindings are automatic and can be done without > a compilation step. Not quite sure I understand you here -- write a library using tools that produce an introspectable descriptions of its interface??? context? Are you talking about a library that is written so that it can be analyzed by a program and then extension bindings for the individual language can be created? -Sterling From sterling@designmultimedia.com Mon Aug 13 16:40:23 2001 From: sterling@designmultimedia.com (Sterling Hughes) Date: Mon, 13 Aug 2001 11:40:23 -0400 (EDT) Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Message-ID: On Sun, 12 Aug 2001, Rasmus Lerdorf wrote: > > By introducing another layer of abstraction we will most probably decrease > > the performance of all languages[1]. That is, since the universal > > interface to, given your example, the Sablotron XSLT library is the API > > and Perl, Python and PHP have internals different enough that there would > > have to be a certain amount of glue and translation work between the > > structures of the extension layer and the internals of the interpretter. > > It should still be possible to come up with some guidelines for writing a > single extension that supports multiple scripting languages. Especially > for something like Sablotron which isn't that complex. You feed it some > XML and some XSL and it spits back a result. Most of the work in writing > the extension is figuring out how the user-visible API should look and > then translating that API along with its function/method calls and > argument type mangling to whatever the thing that is being glued is > expecting. > Well -- Sablotron is more of an annoyance in this regard than anything -- if you want to support SAX, Scheme, error handling, log handling and such, you need to register a million callbacks and then have each of these callbacks either do something useful, or call some user space function. puhhhh. The PHP-XSLT extension, all in all is 2525 lines of code, Makefiles, and explanations, 2525 more than I would've liked to write (http://cvs.php.net/cvs.php/php4/ext/xslt)... Using PHP-Sablotron as an example is something where having a couple of minds is better... The first version of the extension (http://cvs.php.net/cvs.php/php4/ext/sablot), left, uhhmm, something to be desired -- the api was a bit hard to understand, and it made code maintainability within the extension something close to hell. Perhaps if there were a couple more minds working on the extension, some of the mistakes of the original version might not have been present (perhaps if I had slept the weekend that I wrote it, that might have happened as well, who knows? :) Anyway, just counting now -- 4530 lines of "stuff" :) have been written on the PHP end of things for bring sablotron support in (actually, the newer XSLT extension will also support libxslt and Xalan, so some of its bloat for those extra libraries, but..) I imagine that on the Perl and Python ends at least 1,000 lines of code have been written on each side. Wouldn't it be spiffy if this only had to be done once? Hell, if there's a singular extension api, as you say, vendors may just write to that spec, and we don't have to worry about writing it at all! > Note that I am not talking about runtime binary compatibility here. I am > talking about source compatibility where potentially big chunks of code > would be very different across the different languages. In an approach > like this I don't see a performance issue. It is more of an education > issue actually. > > And with a bit of effort I bet we could come up with enough common ground, > or at least some nifty macro tricks, where people wouldn't feel > overwhelmed by the task of supporting multiple scripting languages when > they sat down to write some new backend library. > > Perhaps it would be a good experiment to take some really simple thing and > write Perl, Python, PHP, Ruby, and Tcl extensions for it and compare > notes. See where the overlap is, see where the biggest divergence is, and > figure out if some of that divergence could be eliminated with some clever > tricks. > I think this would be interesting at least -- I'll write the PHP extension (Hell, I already have, http://cvs.php.net/cvs.php/php4/ext/zip, it don't get easier than that :) -Sterling From tim.one@home.com Mon Aug 13 04:46:06 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 12 Aug 2001 23:46:06 -0400 Subject: [Python-Dev] Silly pystones question In-Reply-To: <15220.39904.966751.603159@beluga.mojam.com> Message-ID: [Skip Montanaro] > In pystones.Proc0, most of the variables declared global are > never assigned. Was that just slavish adherence to the style of code > from which it was adapted or was the code changed at some point in > the past, making the global declarations superfluous? You're a developer, Skip: it's time you learned how to use CVS : The evidence there says it's always been this way. > Seems to me that the only time the global keyword should be used is > when an assignment to the variable occurs in the code and you want > to make sure you're modifying the global name. Well, like any other declaration, it can also have documentation value; and sometimes you'll see every function in a module repeat the same global decl. "just for consistency" (I guess). > This is obviously a silly piece of code in which to question this > technique, but I was looking at it to gauge the effects of possible > global object access optimization. My first assumption was that a > variable declared global wouldn't be subject to any optimization, but > now I realize I'd have to look for variables that were operands of > LOAD_GLOBAL instructions but not STORE_GLOBAL instructions. Until The Rules change, any global (or builtin) is subject to rebinding at any time, even if there's no STORE_GLOBAL in sight. The "danger" across call sites is clear enough (a Python global is like a C extern), but in the presence of threads another thread can reach into the module and rebind its globals between any pair of byte codes (which C worms around via X3J11 refusing to say anthing at all about threads <0.9 wink>). From tim.one@home.com Mon Aug 13 05:26:05 2001 From: tim.one@home.com (Tim Peters) Date: Mon, 13 Aug 2001 00:26:05 -0400 Subject: [Python-Dev] Free threading In-Reply-To: <3B7730C7.FE3983A2@ActiveState.com> Message-ID: [Tim] > Well, I'm not a Tcl guy, but Tcl has historically had no threads at > all ... [Paul Prescod] > No, Tcl has real threads now. "Now", yes, not "historically". When Guido said that multiple interpreters were the #1 request from Tcl-land, I'm sure he was talking about pre-thread Tcl-land. > Tcl and Perl both share a model that the Perl guys call "Ithreads". > (for interpreter threads or independent threads, I guess) where each > interpreter is firewalled from the other unless you ask to share > information. Tcl's threads were driven by the AOL guys' need to get > massive scalability. You can't get massive scalability using OS threads, unless the OS is an oddball specifically designed for that. Worming around the bloat of Windows and Linux (etc) OS threads is Stackless Python's natural domain (simulated threads). The catch, of course, is they that don't run in parallel! Or maybe by "massive" you mean a few dozen? I worked in the "massively parallel supercomputer" biz, so I'm thinking more on the order of a few thousands of dozens. I bet you're not. > ... > Why do people use mod_python, fcgi, mod_snake, mod_php and all of those > other things instead of pure CGI? Beats me why people bother with web programming at all . > The usual claim is that they dislike the cost of forking a process > and loading the interpreter code. Now imagine a user with a 20 > processor machine. She isn't going to be happy with the price of > forking processes and using IPC for information sharing either. On > the other hand, she isn't going to be happy with a shared GIL. The cost of creating 20 processes is trivial "even on Windows" if you only need to do that once. Process-based (as opposed to thread-based) solutions are unnatural on Windows, though, and part of this argument seems cultural in related ways. > Free threading helps, but if sharing data has a performance cost (e.g. > by requiring reference count operations to be locked, or requiring > mutexes on dictionaries) then you might not want to pay that cost > either. The Perl guys convinced me of that much. IIRC, Greg's fabled free-threading version of Python took a speed hit of about a factor of 2 (for a program using only 1 thread, compared to that same program without the free-threading patches). > The most popular "embedded scripting languages" (PHP and VBScript/ASP) > use this totally independent thread model. As far as I know, neither has > a concept of sharing information between threads. To a PHP programmer, > that's what SQL and browser cookies are for. :-) Well, the idea that threads don't share information is foreign to every intense belief about the world *I've* ever been paid to adopt , and I'm not enough of a Windows geek to believe "threads are always the answer" even so. A pool of worker processes using OS-specific IPC as needed works great in my real-life experience, and if information sharing is rare, works especially great because it's not *fighting* the OS and C libraries tooth and nail. What of your hypothetical user earlier, who "isn't going to be happy with the price of ... using IPC for information sharing either"? That is, in what sense do isolated threads leave her happy about her information sharing needs? If she's happy to communicate via database transactions and queries, fear of IPC being too expensive wouldn't be rational. From otaylor@redhat.com Mon Aug 13 05:46:27 2001 From: otaylor@redhat.com (Owen Taylor) Date: 13 Aug 2001 00:46:27 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Sterling Hughes's message of "Mon, 13 Aug 2001 11:21:30 -0400 (EDT)" References: Message-ID: Sterling Hughes writes: > > Etc. If you follow these rules, you get a a library where writing > > bindings for it is a brain-dead task. But still task. A library like > > GObject that standardizes memory management, and object-oriented > > structures helps quite a bit, but I'm certainly of the opinion that > > the eventual goal has to be > > > > - write a library using tools that produce an introspectable > > descriptions of its interfaces. > > > > - Language bindings are automatic and can be done without > > a compilation step. > > > Not quite sure I understand you here -- write a library using tools > that produce an introspectable descriptions of its interface??? > context? > > Are you talking about a library that is written so that it can be > analyzed by a program and then extension bindings for the individual > language can be created? Sorry for the vagueness, just trying to be very generic. There are multiple ways of "automatically" getting information about the interfaces library/module, some being: - Generate the headers for the library from an IDL file, that can later be used (directly or via a typelib) for generating language bindings. (example XPCOM) - Produce interface information as part of the compilation process. (example .NET) - Parse header files to determine the interface. (examples: SWIG, most GTK+ bindings) Once you have that information, you can: - Compile an extension module for a particular language. - Generate bindings on the fly using dynamic invocation (XptCall, libffi, etc.) In that space of possiiblities, I tend to favor avoiding header parsing, and dynamic bindings rather than static bindings. Regards, Owen From jack@oratrix.nl Mon Aug 13 10:52:50 2001 From: jack@oratrix.nl (Jack Jansen) Date: Mon, 13 Aug 2001 11:52:50 +0200 Subject: [Python-Dev] Any NeXT users out there? In-Reply-To: Message by "Martin v. Loewis" , Mon, 13 Aug 2001 00:34:48 +0200 , <200108122234.f7CMYml01363@mira.informatik.hu-berlin.de> Message-ID: <20010813095250.7F568303181@snelboot.oratrix.nl> > Do you really need to call the option --with-next-framework? First, I > think it should be an --enable option, since you are not offering to > use or not to use some external tool or library. I'm unsure, I've never fully understood the intention behind --with and --enable. For instance, why is it "--enable-unicode" and "--with-threads"? I agree that the "next" bit should probably go, leaving "--with-framework" or "--enable-framework". > Furthermore, I find > it very confusing to have to invoke incantations involving NeXT when > compiling software for Mac OS. That Steve Jobs is head of the company > is *not* enough reason :-) The alternative is of course that we also provide a "--with-previous-framework". Any suggestions as to what it should do? :-) > Out of curiosity: Just what is a framework? (Should that be Jack, what > is a framework ?-) On a low level a framework is really little more than a directory containing a collection of directories and symlinks, with a bit of support in the compiler/linker (-framework option) and the dynamic loader (searching for frameworks over a search path). On a high level it is a very nifty way to package a shared library complete with all its supporting files. These supporting files range from things used at runtime (like the python Lib directory) to the C header files needed when you want to link against the library to the documentation. Everything is in a single directory, so install/uninstall is a breeze with no danger of missing things. Also, the structure of a framework allows you to have older, incompatible versions of the library (and support documents) in the framework as well, with these older versions only used by programs that were linked against the framework when that version was current. A framework is really an instance of a bundle. The other type of bundle is an Application Bundle, which is (you guessed it) a directory containing an application binary together with all of its support files. A very nifty feature is that you can put the frameworks that an application uses into the application bundle, thereby forestalling problems like Windows dll-hell. The application will even pick up newer, compatible, versions of the framework from the system directories if they happen to be there. The framework patch in sourceforge is the first step. The next step is to create a Python application bundle. We can then use a single trick in the Python main() program, where it will check to see whether it's being run from an application bundle and, if so, whether there's a magic file in the bundle (__main__.py comes to mind). If there is Python will run that file as the script. This will give Python users "freeze" functionality without using a compiler. (MacPython users have enjoyed this functionality for quite some time already, with BuildApplet and BuildApplication). And, of course, if we take the Python application bundle with the __main__.py in it and stuff the Python framework into its Frameworks subdirectory we have a complete standalone application that can be drag-drop installed without having to touch a C compiler, without having to install Python, and which won't even interfere if you happen to have an incompatible version of Python installed already. And you can uninstall it by dragging it to the trash without fear that you will wreck your existing Python installation in the process! Once all this is in place (1-2 weeks, I think) I would really like to try it with a Real World application. Zope is the first one that comes to mind, but there may be other options. I'd like to hear if someone wants to put some work into this. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From thomas@xs4all.net Mon Aug 13 11:09:10 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 13 Aug 2001 12:09:10 +0200 Subject: [Python-Dev] Any NeXT users out there? In-Reply-To: <20010813095250.7F568303181@snelboot.oratrix.nl> References: <200108122234.f7CMYml01363@mira.informatik.hu-berlin.de> <20010813095250.7F568303181@snelboot.oratrix.nl> Message-ID: <20010813120910.J874@xs4all.nl> On Mon, Aug 13, 2001 at 11:52:50AM +0200, Jack Jansen wrote: > > Do you really need to call the option --with-next-framework? First, I > > think it should be an --enable option, since you are not offering to > > use or not to use some external tool or library. > I'm unsure, I've never fully understood the intention behind --with and > --enable. For instance, why is it "--enable-unicode" and "--with-threads"? Like Martin explained earlier, '--with' is for options that require or might require an external package or library, and '--enable' is for enabling a feature wholy contained in The Software. So you have --enable-unicode(=type) and --with-threads(=external_threads_package). -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido@python.org Mon Aug 13 11:39:15 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 06:39:15 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: Your message of "Sun, 12 Aug 2001 19:47:33 EDT." <20010812194733.A8000@thyrsus.com> References: <3B77143F.F9158ED1@ActiveState.com> <20010812194733.A8000@thyrsus.com> Message-ID: <200108131039.GAA17322@cj20424-a.reston1.va.home.com> Can I make a request? Can we *not* crosspost between language-dev and python-dev? The whole point of language-dev is to keep the traffic *out* of the language-specific lists. Please trim your "to" lines. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Aug 13 11:44:00 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 06:44:00 -0400 Subject: [Python-Dev] Silly pystones question In-Reply-To: Your message of "Fri, 10 Aug 2001 21:43:44 CDT." <15220.39904.966751.603159@beluga.mojam.com> References: <15220.39904.966751.603159@beluga.mojam.com> Message-ID: <200108131044.GAA17367@cj20424-a.reston1.va.home.com> Whatever you do, *don't* succumb to the temptation of "fixing" pystone. It's a benchmark and unless it stays the same there's no way to compare to previous versions. --Guido van Rossum (home page: http://www.python.org/~guido/) From jack@oratrix.nl Mon Aug 13 13:42:30 2001 From: jack@oratrix.nl (Jack Jansen) Date: Mon, 13 Aug 2001 14:42:30 +0200 Subject: [Python-Dev] Any NeXT users out there? In-Reply-To: Message by Thomas Wouters , Mon, 13 Aug 2001 12:09:10 +0200 , <20010813120910.J874@xs4all.nl> Message-ID: <20010813124231.6C945303181@snelboot.oratrix.nl> > On Mon, Aug 13, 2001 at 11:52:50AM +0200, Jack Jansen wrote: > > > > Do you really need to call the option --with-next-framework? First, I > > > think it should be an --enable option, since you are not offering to > > > use or not to use some external tool or library. > > > I'm unsure, I've never fully understood the intention behind --with and > > --enable. For instance, why is it "--enable-unicode" and "--with-threads"? > > Like Martin explained earlier, '--with' is for options that require or might > require an external package or library, and '--enable' is for enabling a > feature wholy contained in The Software. So you have > --enable-unicode(=type) and --with-threads(=external_threads_package). Ah, thanks, that means that it should with "--with-framework", because there's external stuff involved. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From thomas@xs4all.net Mon Aug 13 14:24:13 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 13 Aug 2001 15:24:13 +0200 Subject: [Python-Dev] Any NeXT users out there? In-Reply-To: <20010813124231.6C945303181@snelboot.oratrix.nl> References: <20010813120910.J874@xs4all.nl> <20010813124231.6C945303181@snelboot.oratrix.nl> Message-ID: <20010813152413.N872@xs4all.nl> On Mon, Aug 13, 2001 at 02:42:30PM +0200, Jack Jansen wrote: > > Like Martin explained earlier, '--with' is for options that require or might > > require an external package or library, and '--enable' is for enabling a > > feature wholy contained in The Software. So you have > > --enable-unicode(=type) and --with-threads(=external_threads_package). > Ah, thanks, that means that it should with "--with-framework", because there's > external stuff involved. There's almost always external stuff involved :) the question is whether you need to be able to say '--with-framework=', or '--enable-framework='. Maybe the first is the case, but from what I've seen, it's more like the second. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido@python.org Mon Aug 13 14:40:06 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 09:40:06 -0400 Subject: [Python-Dev] yield without future statement? Message-ID: <200108131340.JAA17834@cj20424-a.reston1.va.home.com> On c.l.py, there's a small discussion going on the necessity of a future statement to enable the yield keyword. The general opinion seems to be that the future statement is redundant, since previous uses of yield will generally result in a syntax error -- and there aren't very many uses of yield in the first place. The other uses of future (nested scopes and division) were needed because these features cause *silent* failure -- but with yield, you'd have to work really really hard to cause a silent failure. Opinions? I'd love to get rid of this particular future statement. --Guido van Rossum (home page: http://www.python.org/~guido/) From jack@oratrix.nl Mon Aug 13 14:44:17 2001 From: jack@oratrix.nl (Jack Jansen) Date: Mon, 13 Aug 2001 15:44:17 +0200 Subject: [Python-Dev] Any NeXT users out there? In-Reply-To: Message by Thomas Wouters , Mon, 13 Aug 2001 15:24:13 +0200 , <20010813152413.N872@xs4all.nl> Message-ID: <20010813134417.99842303181@snelboot.oratrix.nl> > There's almost always external stuff involved :) the question is whether you > need to be able to say > '--with-framework=', > or '--enable-framework='. Maybe the first is the case, > but from what I've seen, it's more like the second. Hey, I'm easy! --enable-framework it is, so. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From skip@pobox.com (Skip Montanaro) Mon Aug 13 14:47:33 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 13 Aug 2001 08:47:33 -0500 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: <200108101515.LAA18907@cj20424-a.reston1.va.home.com> References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <3B73F824.82A53539@metaslash.com> <200108101515.LAA18907@cj20424-a.reston1.va.home.com> Message-ID: <15223.55925.912142.77948@beluga.mojam.com> >> [ importing to scan snipped ] >> >> Yes, this is one of the worst problems I know of. I plan on changing >> this. Guido> If you don't want to give up scanning the bytecode (if that's Guido> what you do), you could use compile() on the source code to get Guido> the bytecode for the entire module. I think we may need to start thinking about some standard interface to byte code scanning. (I keep thinking about Rattlesnake.) Perhaps there are standard tasks people are performing that could be abstracted into a standard module (for example, getting a list of global variables that are set by a function, or getting first set and first read points for particular variables). One complaint I saw voiced about the Rattlesnake idea was that code that scans byte code would break. Skip From guido@python.org Mon Aug 13 14:49:14 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 09:49:14 -0400 Subject: [Python-Dev] Any NeXT users out there? In-Reply-To: Your message of "Mon, 13 Aug 2001 15:24:13 +0200." <20010813152413.N872@xs4all.nl> References: <20010813120910.J874@xs4all.nl> <20010813124231.6C945303181@snelboot.oratrix.nl> <20010813152413.N872@xs4all.nl> Message-ID: <200108131349.JAA17882@cj20424-a.reston1.va.home.com> I've got to say, the distinction between --with and --enable, no matter how thoretically pure, in practice seems extremely fuzzy. Quoting Neil Schemenauer quoting the autoconf docs (http://mail.python.org/pipermail/python-dev/2001-January/011938.html): | If a software package has optional compile-time features, the | user can give `configure' command line options to specify | whether to compile them. The options have one of these forms: | | --enable-FEATURE[=ARG] | --disable-FEATURE | | Some packages require, or can optionally use, other software | packages which are already installed. The user can give | `configure' command line options to specify which such | external software to use. The options have one of these | forms: | | --with-package[=ARG] | --without-package Now, for example, should it be --with-ipv6 or --enable-ipv6? I would think --with, since this is clearly depending on optional external software. Yet, it is one of the few configure options that use --enable. I am seriously considering standardizing on --with, ignoring the autoconf docs. Given how vague the distinction is, I'm sure most users will find the distinction completely arbitrary even if we applied the rules correctly. They would be better served with a consistent rule they can remember, like "always use --with/--without". --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com (Skip Montanaro) Mon Aug 13 14:53:23 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 13 Aug 2001 08:53:23 -0500 Subject: [Python-Dev] ++x oddnes In-Reply-To: <3B7542D1.3E38E579@lemburg.com> References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com> Message-ID: <15223.56275.968603.294087@beluga.mojam.com> mal> David Ascher wrote: >> >> Guido van Rossum wrote: >> >> > IMO not worth it changing the language. Train yourself to write x++ >> > instead. :-) >> >> Sounds like an excellent one for PyChecker =) mal> Why ? ++x can be put to some real use: I have a counter type which mal> actually uses ++x to increment the counter. (I've been away for a few days, so perhaps this has been covered already. I'm still catching up.) How? ++x compiles to LOAD_FAST x UNARY_POSITIVE UNARY_POSITIVE I don't see any incrementing going on... Skip From guido@python.org Mon Aug 13 14:56:51 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 09:56:51 -0400 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: Your message of "Mon, 13 Aug 2001 08:47:33 CDT." <15223.55925.912142.77948@beluga.mojam.com> References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <3B73F824.82A53539@metaslash.com> <200108101515.LAA18907@cj20424-a.reston1.va.home.com> <15223.55925.912142.77948@beluga.mojam.com> Message-ID: <200108131356.JAA17936@cj20424-a.reston1.va.home.com> > I think we may need to start thinking about some standard interface to byte > code scanning. (I keep thinking about Rattlesnake.) Perhaps there are > standard tasks people are performing that could be abstracted into a > standard module (for example, getting a list of global variables that are > set by a function, or getting first set and first read points for particular > variables). One complaint I saw voiced about the Rattlesnake idea was that > code that scans byte code would break. Code that scans bytecode is well aware that it can break. People have often asked me about the stability of the bytecode, and I've always responded that it changes in every release, and is considered an internal implementation detail. A bytecode scanning API might be a good idea, but it might also end up being restrictive: it's hard to design such an API without making *some* assumptions about the general bytecode architecture, and a future implementation might violate those assumptions. Jython is a case in point: it doesn't support Python's bytecode at all -- and that's right, because it has no use for it. For tools like PyChecker, a higher level is more appropriate: give it an abstract syntax tree that can answer questions related to the symbol table, so that you can tell for example whether 'x = y' assigns a local to a global or vice versa -- and if a local is involved, in which function it is defined. For Rattlesnake, a bytecode accessor would of course still be needed -- but in its capacity as an implementation detail of CPython. --Guido van Rossum (home page: http://www.python.org/~guido/) From akuchlin@mems-exchange.org Mon Aug 13 15:01:17 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Mon, 13 Aug 2001 10:01:17 -0400 Subject: [Python-Dev] More bugs in 2.2a1 In-Reply-To: <3B7592E3.892D1995@metaslash.com>; from neal@metaslash.com on Sat, Aug 11, 2001 at 04:17:39PM -0400 References: <3B7592E3.892D1995@metaslash.com> Message-ID: <20010813100116.A791@ute.cnri.reston.va.us> On Sat, Aug 11, 2001 at 04:17:39PM -0400, Neal Norwitz wrote: >Here's more warnings from pychecker. There are a lot from plat-irix6 & >plat-unixware7, not sure if those are real problems or not. The plat-* problems all seem to be macros converted from C to Python. if a user ever calls them, they'll only get a NameError for their trouble, but since no one is likely to call them, this can probably just be ignored. I've checked in fixes for the actual bugs reported in the other modules; thanks! The only one I haven't done is the following one; there's definitely at least one bug here (item #2), but I don't know Tk well enough to fix it. >lib-tk: > Tix.py:81: self is not first method argument > Tix.py:331: No global (value) found --amk From guido@python.org Mon Aug 13 15:02:21 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 10:02:21 -0400 Subject: [Python-Dev] ++x oddnes In-Reply-To: Your message of "Mon, 13 Aug 2001 08:53:23 CDT." <15223.56275.968603.294087@beluga.mojam.com> References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com> <15223.56275.968603.294087@beluga.mojam.com> Message-ID: <200108131402.KAA17976@cj20424-a.reston1.va.home.com> > How? ++x compiles to > > LOAD_FAST x > UNARY_POSITIVE > UNARY_POSITIVE > > I don't see any incrementing going on... I'm guessing that MAL used a perverse implementation trick, where for a certain *mutable* counter type '+x' returns an object that contains a reference to x, and the unary plus operation on *that* object increments the counter it references. class C: def __init__(self): self.counter = 0 def inc(self): self.counter += 1 def __str__(self): return str(self.counter) def __pos__(self): return Ref(self) class Ref: def __init__(self, arg): self.referent = arg def __pos__(self): self.referent.inc() return self.referent a = C() print a # 0 print ++a # 1 print a # 1 Twisted, sick, call it what you want -- but please don't get used to this. The likelihood that I'll ever add a ++ operator is very small, but not zero. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From jack@oratrix.nl Mon Aug 13 15:06:43 2001 From: jack@oratrix.nl (Jack Jansen) Date: Mon, 13 Aug 2001 16:06:43 +0200 Subject: [Python-Dev] Any NeXT users out there? In-Reply-To: Message by Guido van Rossum , Mon, 13 Aug 2001 09:49:14 -0400 , <200108131349.JAA17882@cj20424-a.reston1.va.home.com> Message-ID: <20010813140643.9B5C1303181@snelboot.oratrix.nl> > I've got to say, the distinction between --with and --enable, no > matter how thoretically pure, in practice seems extremely fuzzy. ... Seeing that I'm in good company wrt my with/enable confusion I'll chime in one last time, there's a couple of cases where with is used that are plainly wrong. --with-suffix is probably the silliest one, as it is provided by autoconf itself. A distinction that I could understand is if 'with' options may require some external stuff (or may be required by some external stuff) and 'enable' were purely internal options. With that definition unicode, gc and malloc are the only 'enable' style options. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From guido@python.org Mon Aug 13 15:09:50 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 10:09:50 -0400 Subject: [Python-Dev] More bugs in 2.2a1 In-Reply-To: Your message of "Mon, 13 Aug 2001 10:01:17 EDT." <20010813100116.A791@ute.cnri.reston.va.us> References: <3B7592E3.892D1995@metaslash.com> <20010813100116.A791@ute.cnri.reston.va.us> Message-ID: <200108131409.KAA18039@cj20424-a.reston1.va.home.com> > I've checked in fixes for the actual bugs reported in the other > modules; thanks! Thanks, Andrew! (It really sucks that SF doesn't send checkin mail any more; I've complained but they are still asleep I assume.) > The only one I haven't done is the following one; > there's definitely at least one bug here (item #2), but I don't know > Tk well enough to fix it. > > >lib-tk: > > Tix.py:81: self is not first method argument According to line 78, Form.form is an alias for the config method, which definitely requires a self argument. I'll go ahead and fix it. > > Tix.py:331: No global (value) found Looks like a copy-and-paste error from __setitem__. I'll fix this too. --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik@pythonware.com Mon Aug 13 15:12:15 2001 From: fredrik@pythonware.com (Fredrik Lundh) Date: Mon, 13 Aug 2001 16:12:15 +0200 Subject: [Python-Dev] ++x oddnes References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com> <15223.56275.968603.294087@beluga.mojam.com> Message-ID: <062401c12401$f5b80690$0900a8c0@spiff> skip wrote: > How? ++x compiles to > > LOAD_FAST x > UNARY_POSITIVE > UNARY_POSITIVE > > I don't see any incrementing going on... try this: class StupidCounter: count = 0 def __pos__(self): self.count = self.count + 1 return 0 # ignore extra __pos__ calls def __int__(self): return self.count def __repr__(self): return repr(self.count) c = StupidCounter() print c ++c print c From skip@pobox.com (Skip Montanaro) Mon Aug 13 15:43:40 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 13 Aug 2001 09:43:40 -0500 Subject: [Python-Dev] CVS commit permission for Andrew MacIntyre In-Reply-To: <20010812222547.I874@xs4all.nl> References: <200108121453.KAA15888@cj20424-a.reston1.va.home.com> <20010812222547.I874@xs4all.nl> Message-ID: <15223.59292.919952.827402@beluga.mojam.com> Thomas> None, though I'd suggest subscribing him to python-dev too, for Thomas> the reasons I gave when I found out Just had checkin privs but Thomas> wasn't on py-dev. Perhaps a (one) requirement for checkin privileges should be a python-dev subscription? Skip From guido@python.org Mon Aug 13 15:57:34 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 10:57:34 -0400 Subject: [Python-Dev] CVS commit permission for Andrew MacIntyre In-Reply-To: Your message of "Mon, 13 Aug 2001 09:43:40 CDT." <15223.59292.919952.827402@beluga.mojam.com> References: <200108121453.KAA15888@cj20424-a.reston1.va.home.com> <20010812222547.I874@xs4all.nl> <15223.59292.919952.827402@beluga.mojam.com> Message-ID: <200108131457.f7DEvYQ07335@odiug.digicool.com> > Perhaps a (one) requirement for checkin privileges should be a python-dev > subscription? Sounds good to me. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com (Skip Montanaro) Mon Aug 13 15:59:34 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 13 Aug 2001 09:59:34 -0500 Subject: [Python-Dev] Silly pystones question In-Reply-To: References: <15220.39904.966751.603159@beluga.mojam.com> Message-ID: <15223.60246.928262.733591@beluga.mojam.com> Tim> You're a developer, Skip: it's time you learned how to use CVS Tim> : Tim> Thanks for the pointer to the CVS browsing stuff. In my own defense, I wrote that note while I was way offline in a rural cottage overlooking Lake Michigan, so I couldn't run my usual "cvs annotate". ;-) Tim> The evidence there says it's always been this way. Thanks. Tim> Until The Rules change, any global (or builtin) is subject to Tim> rebinding at any time, even if there's no STORE_GLOBAL in sight. Tim> The "danger" across call sites is clear enough (a Python global is Tim> like a C extern), but in the presence of threads another thread can Tim> reach into the module and rebind its globals between any pair of Tim> byte codes (which C worms around via X3J11 refusing to say anthing Tim> at all about threads <0.9 wink>). Yes, I understand this. I was thinking (initially, at least) of only applying such access optimizations in functions that themselves don't set the global in question. For example, in pystones, any references to the various Proc* functions would be candidates for such optimizations. This is only to make my life easier. The real thing here is that while I can "see" what I want with global tracking, I'm a bit at a loss how to implement it cleanly. I think what I need to do at this point is write a PEP and solicit some feedback on implementation possibilities. Skip From skip@pobox.com (Skip Montanaro) Mon Aug 13 16:02:56 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 13 Aug 2001 10:02:56 -0500 Subject: [Python-Dev] Silly pystones question In-Reply-To: <200108131044.GAA17367@cj20424-a.reston1.va.home.com> References: <15220.39904.966751.603159@beluga.mojam.com> <200108131044.GAA17367@cj20424-a.reston1.va.home.com> Message-ID: <15223.60448.58595.728950@beluga.mojam.com> Guido> Whatever you do, *don't* succumb to the temptation of "fixing" Guido> pystone. It's a benchmark and unless it stays the same there's Guido> no way to compare to previous versions. I had no intention of doing anything to it, I was just wondering... ;-) Skip From jeremy@zope.com Mon Aug 13 16:07:32 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 13 Aug 2001 11:07:32 -0400 (EDT) Subject: [Python-Dev] RE: Somebody stopped running tests? In-Reply-To: References: Message-ID: <15223.60724.456033.218059@slothrop.digicool.com> >>>>> "TP" == Tim Peters writes: >> Over the last 12 hours, 4 tests started failing on Windows: >> >> 4 tests failed: test_descr test_future test_generators test_scope >> ... But only in release mode. In a debug build, these pass. >> This strongly suggests a change "somewhere" that accidentally >> relies on an uninitialized value. TP> And so it was. "Fixed" in current CVS, but Jeremy needs to fix TP> it a right way. Fixed now -- thought I'd mention it in the absence of any checkin mail. I had started down the path of removing all the is-nested-scopes-enabled-or-not code, then decided to cleanup the handling for future statements first. I accidentally checked in one of the former changes with the latter checkin. Jeremy From skip@pobox.com (Skip Montanaro) Mon Aug 13 16:09:43 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 13 Aug 2001 10:09:43 -0500 Subject: [Python-Dev] ++x oddnes In-Reply-To: <062401c12401$f5b80690$0900a8c0@spiff> References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com> <15223.56275.968603.294087@beluga.mojam.com> <062401c12401$f5b80690$0900a8c0@spiff> Message-ID: <15223.60855.1990.594442@beluga.mojam.com> >> I don't see any incrementing going on... Fredrik> try this: Fredrik> class StupidCounter: Fredrik> count = 0 Fredrik> def __pos__(self): Fredrik> self.count = self.count + 1 Fredrik> return 0 # ignore extra __pos__ calls Fredrik> def __int__(self): Fredrik> return self.count Fredrik> def __repr__(self): Fredrik> return repr(self.count) Fredrik> c = StupidCounter() Fredrik> print c Fredrik> ++c Fredrik> print c Thanks, I understand now. Of course +c also increments the counter... S From esr@thyrsus.com Mon Aug 13 16:08:51 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 13 Aug 2001 11:08:51 -0400 Subject: [Python-Dev] yield without future statement? In-Reply-To: <200108131340.JAA17834@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Aug 13, 2001 at 09:40:06AM -0400 References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com> Message-ID: <20010813110851.B21008@thyrsus.com> Guido van Rossum : > On c.l.py, there's a small discussion going on the necessity of a > future statement to enable the yield keyword. The general opinion > seems to be that the future statement is redundant, since previous > uses of yield will generally result in a syntax error -- and there > aren't very many uses of yield in the first place. The other uses of > future (nested scopes and division) were needed because these features > cause *silent* failure -- but with yield, you'd have to work really > really hard to cause a silent failure. > > Opinions? I'd love to get rid of this particular future statement. I think you safely can. I buy this argument. -- Eric S. Raymond Non-cooperation with evil is as much a duty as cooperation with good. -- Mohandas Gandhi From jeremy@zope.com Mon Aug 13 16:23:04 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 13 Aug 2001 11:23:04 -0400 (EDT) Subject: [Python-Dev] Direction of PyChecker In-Reply-To: <200108131356.JAA17936@cj20424-a.reston1.va.home.com> References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <3B73F824.82A53539@metaslash.com> <200108101515.LAA18907@cj20424-a.reston1.va.home.com> <15223.55925.912142.77948@beluga.mojam.com> <200108131356.JAA17936@cj20424-a.reston1.va.home.com> Message-ID: <15223.61656.410876.775564@slothrop.digicool.com> >>>>> "GvR" == Guido van Rossum writes: GvR> For tools like PyChecker, a higher level is more appropriate: GvR> give it an abstract syntax tree that can answer questions GvR> related to the symbol table, so that you can tell for example GvR> whether 'x = y' assigns a local to a global or vice versa -- GvR> and if a local is involved, in which function it is defined. I'm not sure how much time I can make for it in the 2.2 release schedule, but I think a high-level source analysis module would be a valuable addition to the compiler package. We've got pychecker and pydoc as to drive the requirements. Do you think it's worth making time for in the 2.2 release schedule? Jeremy From guido@python.org Mon Aug 13 16:46:31 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 11:46:31 -0400 Subject: [Python-Dev] 2.2a1 patch from PyChecker In-Reply-To: Your message of "Thu, 09 Aug 2001 15:33:52 EDT." <3B72E5A0.DC2E8C4E@metaslash.com> References: <3B72E5A0.DC2E8C4E@metaslash.com> Message-ID: <200108131546.f7DFkVk07612@odiug.digicool.com> Hi Neal, Going through your patches, I notice a few areas where I disagree. - You want to change all occurrences of "except , :" where is unused. I don't -- often it's more regular to leave in. - You want to change all occurrences of "func(name=value)" where name is an argument without a default. I disagree that a default is necessary for an argument to be acceptable as a keyword argument. - You want to change "for i in " into "for _ in " when i is unused. I disagree (we've been through this). Also, I can't figure out why you want to delete the setlocale() method from the locale.py module. Apart from these, I'm applying many of these, and Andrew has already applied many more. So thanks again! One thing. Next time please use a recent CVS. Martin had already fixed many of the missing imports. (If others agree, we could give you checkin permission so you could check in non-controversial fixes yourself.) --Guido van Rossum (home page: http://www.python.org/~guido/) From mwh@python.net Mon Aug 13 16:43:43 2001 From: mwh@python.net (Michael Hudson) Date: 13 Aug 2001 11:43:43 -0400 Subject: [Python-Dev] Silly pystones question In-Reply-To: Skip Montanaro's message of "Mon, 13 Aug 2001 09:59:34 -0500" References: <15220.39904.966751.603159@beluga.mojam.com> <15223.60246.928262.733591@beluga.mojam.com> Message-ID: <2mwv48nexc.fsf@starship.python.net> Skip Montanaro writes: > The real thing here is that while I can "see" what I want with > global tracking, I'm a bit at a loss how to implement it cleanly. I > think what I need to do at this point is write a PEP and solicit > some feedback on implementation possibilities. Go for it! I'm sure there are plenty of folks here who would do their best to pick holes in and suggest improvements in any scheme you come up with. And the importance of having something concrete cannot possibly be overstated - I'm sure each one of us has enough handwavy ideas on this sort of topic for us all. I know I do. Cheers, M. -- Two things I learned for sure during a particularly intense acid trip in my own lost youth: (1) everything is a trivial special case of something else; and, (2) death is a bunch of blue spheres. -- Tim Peters, 1 May 1998 From guido@python.org Mon Aug 13 17:04:01 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 12:04:01 -0400 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: Your message of "Mon, 13 Aug 2001 11:23:04 EDT." <15223.61656.410876.775564@slothrop.digicool.com> References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <3B73F824.82A53539@metaslash.com> <200108101515.LAA18907@cj20424-a.reston1.va.home.com> <15223.55925.912142.77948@beluga.mojam.com> <200108131356.JAA17936@cj20424-a.reston1.va.home.com> <15223.61656.410876.775564@slothrop.digicool.com> Message-ID: <200108131604.f7DG41E08086@odiug.digicool.com> > GvR> For tools like PyChecker, a higher level is more appropriate: > GvR> give it an abstract syntax tree that can answer questions > GvR> related to the symbol table, so that you can tell for example > GvR> whether 'x = y' assigns a local to a global or vice versa -- > GvR> and if a local is involved, in which function it is defined. > > I'm not sure how much time I can make for it in the 2.2 release > schedule, but I think a high-level source analysis module would be a > valuable addition to the compiler package. We've got pychecker and > pydoc as to drive the requirements. Do you think it's worth making > time for in the 2.2 release schedule? > > Jeremy I think this falls in the "would be nice" category. I hope that non-PythonLabs folks might contribute something of this sort. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com (Skip Montanaro) Mon Aug 13 17:04:16 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 13 Aug 2001 11:04:16 -0500 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: <15223.61656.410876.775564@slothrop.digicool.com> References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <3B73F824.82A53539@metaslash.com> <200108101515.LAA18907@cj20424-a.reston1.va.home.com> <15223.55925.912142.77948@beluga.mojam.com> <200108131356.JAA17936@cj20424-a.reston1.va.home.com> <15223.61656.410876.775564@slothrop.digicool.com> Message-ID: <15223.64128.590894.72129@beluga.mojam.com> GvR> For tools like PyChecker, a higher level is more appropriate: give GvR> it an abstract syntax tree that can answer questions related to the GvR> symbol table, so that you can tell for example whether 'x = y' GvR> assigns a local to a global or vice versa -- and if a local is GvR> involved, in which function it is defined. Jeremy> I'm not sure how much time I can make for it in the 2.2 release Jeremy> schedule, but I think a high-level source analysis module would Jeremy> be a valuable addition to the compiler package. We've got Jeremy> pychecker and pydoc as to drive the requirements. Do you think Jeremy> it's worth making time for in the 2.2 release schedule? I don't think so. Designing features to suit the needs of just two tools may result in a module that's suited for them. Pydoc and pychecker work now. I suggest working on such a module for 2.3 instead, where you'll have a fair bit of time to properly analyze the requirements. Skip From martin@loewis.home.cs.tu-berlin.de Mon Aug 13 17:01:56 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Mon, 13 Aug 2001 18:01:56 +0200 Subject: [Python-Dev] Any NeXT users out there? In-Reply-To: <20010813095250.7F568303181@snelboot.oratrix.nl> (message from Jack Jansen on Mon, 13 Aug 2001 11:52:50 +0200) References: <20010813095250.7F568303181@snelboot.oratrix.nl> Message-ID: <200108131601.f7DG1uB01479@mira.informatik.hu-berlin.de> > I'm unsure, I've never fully understood the intention behind --with and > --enable. For instance, why is it "--enable-unicode" and "--with-threads"? I think the eventual choice is yours, given Guido's objections to gettext manual; my objections where primarily backed by this manual. > The alternative is of course that we also provide a > "--with-previous-framework". Any suggestions as to what it should do? :-) Based on your current patch, I think it is save to remove any claim that this does something useful on the Next. It may do, but that would be purely coincidental. > On a high level it is a very nifty way to package a shared library > complete with all its supporting files. These supporting files range > from things used at runtime (like the python Lib directory) to the C > header files needed when you want to link against the library to the > documentation. Ok. Then my question is: Why is it desirable to build libpython as a framework? You give some rationale below, but I fail to see the point. I'll pick what I think are your arguments to build a Python framework, and try to see what it gives to the end user - compared to the option of linking libpython into the Python interpreter. > Everything is in a single directory, so install/uninstall is a > breeze with no danger of missing things. In case of Python, "everything" is the header files, and the .py library, right? Couldn't you also get this by making Python an Application Bundle (which you intend to do anyway)? > Also, the structure of a framework allows you to have older, > incompatible versions of the library (and support documents) in the > framework as well, with these older versions only used by programs > that were linked against the framework when that version was > current. Again, why is that desirable? If you have multiple versions of Python installed, if each version has libpython linked statically, and comes with its own set of header files and .py files: there wouldn't be any older, incompatible versions of the library that you may need to defend against. > The framework patch in sourceforge is the first step. The next step > is to create a Python application bundle. We can then use a single > trick in the Python main() program, where it will check to see > whether it's being run from an application bundle and, if so, > whether there's a magic file in the bundle (__main__.py comes to > mind). If there is Python will run that file as the script. This > will give Python users "freeze" functionality without using a > compiler. For that second step, do you really need the first one? I agree that 'freezing' Python applications by just putting a Python installation into a subdirectory is cool (if I understand right how such a frozen application would work), however, I fail to see the role of the framework in here. You may wonder why I question that feature so much. To me, it is similar to building a shared libpython.so on Unix. I think this will cause many problems, and in no way it should be done just for a single platform (there were patches to do this for Linux, I believe, and such a patch to make it work specifically on UnixWare only even made it into 2.1). I think it is desirable that the feature "shared libpython" is implemented as uniformly as possible across systems. If Mac OS X wants to do it differently from everybody else, you need a very good reason. The only platform that already has a shared pythonXY.dll has a good reason: you cannot export symbols from the application to extension modules on Windows. Yet, this is also the only platform where the API version magic is meaningless, since extension modules break with every new Python version exactly because libpython is shared. Regards, Martin From m@moshez.org Mon Aug 13 17:12:19 2001 From: m@moshez.org (Moshe Zadka) Date: Mon, 13 Aug 2001 19:12:19 +0300 (IDT) Subject: [Python-Dev] ++x oddnes In-Reply-To: <062401c12401$f5b80690$0900a8c0@spiff> References: <062401c12401$f5b80690$0900a8c0@spiff>, <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com> <15223.56275.968603.294087@beluga.mojam.com> Message-ID: <20010813161219.F0B38392B@darjeeling> On Mon, 13 Aug 2001, "Fredrik Lundh" wrote: > try this: > > class StupidCounter: > count = 0 > def __pos__(self): > self.count = self.count + 1 > return 0 # ignore extra __pos__ calls > def __int__(self): > return self.count > def __repr__(self): > return repr(self.count) > > c = StupidCounter() > print c > ++c > print c Everybody is ignoring the obvious correct way to do it: class StupidCounter: count = 0 def __pos__(self): self.count = self.count+0.5 return self def __repr__(self): return str(int(self.count)) def __int__(self): return int(self.count) Let me just add, muhahahahahahahahahahahahaha!!!!!! -- The Official Moshe Zadka FAQ: http://moshez.geek The Official Moshe Zadka FAQ For Dummies: http://moshez.org Read the FAQ From jeremy@zope.com Mon Aug 13 17:24:02 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 13 Aug 2001 12:24:02 -0400 (EDT) Subject: [Python-Dev] yield without future statement? In-Reply-To: <200108131340.JAA17834@cj20424-a.reston1.va.home.com> References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com> Message-ID: <15223.65314.231423.190925@slothrop.digicool.com> >>>>> "GvR" == Guido van Rossum writes: GvR> On c.l.py, there's a small discussion going on the necessity of GvR> a future statement to enable the yield keyword. The general GvR> opinion seems to be that the future statement is redundant, GvR> since previous uses of yield will generally result in a syntax GvR> error -- and there aren't very many uses of yield in the first GvR> place. The other uses of future (nested scopes and division) GvR> were needed because these features cause *silent* failure -- GvR> but with yield, you'd have to work really really hard to cause GvR> a silent failure. I does seem almost comical that the introduction of yield as a keyword requires a future statement but the descr branch changes do not. Talk about silent failures -- dir() no longers work, method resolution for multiple inheritance changes, changes to the str names of types. Why don't these require a future statement? Jeremy From Samuele Pedroni Mon Aug 13 17:29:37 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Mon, 13 Aug 2001 18:29:37 +0200 (MET DST) Subject: [Python-Dev] yield without future statement? Message-ID: <200108131629.SAA06332@core.inf.ethz.ch> [Jeremy Hylton] > I does seem almost comical that the introduction of yield as a keyword > requires a future statement but the descr branch changes do not. Talk > about silent failures -- dir() no longers work, method resolution for > multiple inheritance changes, changes to the str names of types. Why > don't these require a future statement? > About dir, in Jython it already worked like it works now in CPython, so that's was just an implemenation detail :-). regards. From fredrik@pythonware.com Mon Aug 13 17:55:22 2001 From: fredrik@pythonware.com (Fredrik Lundh) Date: Mon, 13 Aug 2001 18:55:22 +0200 Subject: [Python-Dev] ++x oddnes References: <062401c12401$f5b80690$0900a8c0@spiff>, <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com> <15223.56275.968603.294087@beluga.mojam.com> <20010813161219.F0B38392B@darjeeling> Message-ID: <009c01c12418$bf3437d0$4ffa42d5@hagrid> moshe wrote: > Everybody is ignoring the obvious correct way to do it: > > class StupidCounter: > count = 0 > def __pos__(self): > self.count = self.count+0.5 > return self > def __repr__(self): return str(int(self.count)) > def __int__(self): return int(self.count) that was my first implementation (TooStupidCounter). but to avoid an endless float vs. integer thread, I decided to post the plain StupidCounter version instead. (SlightlyLessStupidCounter is same as StupidCounter, but returns self.count instead of 0) From mal@lemburg.com Mon Aug 13 19:36:47 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 13 Aug 2001 20:36:47 +0200 Subject: [Python-Dev] ++x oddness References: <15219.64534.411625.193775@nyx.dyndns.org> <200108101523.LAA19012@cj20424-a.reston1.va.home.com> <3B741351.C7C2E2B8@ActiveState.com> <3B7542D1.3E38E579@lemburg.com> <15223.56275.968603.294087@beluga.mojam.com> <200108131402.KAA17976@cj20424-a.reston1.va.home.com> Message-ID: <3B781E3F.5C433915@lemburg.com> Guido van Rossum wrote: > > > How? ++x compiles to > > > > LOAD_FAST x > > UNARY_POSITIVE > > UNARY_POSITIVE > > > > I don't see any incrementing going on... > > I'm guessing that MAL used a perverse implementation trick, where for > a certain *mutable* counter type '+x' returns an object that contains > a reference to x, and the unary plus operation on *that* object > increments the counter it references. Well, not quite that complicated: the type counts the number of calls to __pos__ and then increments the value in every second call. It was just an implementation joke and proof of concept thingie. The idea behind it was to be able to increment the counter (which is the most often used action on a counter after all) without having to go through the whole Python method lookup and call machinery. > Twisted, sick, call it what you want -- but please don't get used to > this. The likelihood that I'll ever add a ++ operator is very small, > but not zero. :-) Ah, but then you'll have to add it as post-fix operator: x++ (which is illegal today). The difference between ++x and x++ isn't needed in Python anyway, so this should work out nicely. Hmm, x++ could map directly to the tp_iternext slot on iterator objects... ;-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From neal@metaslash.com Mon Aug 13 20:04:44 2001 From: neal@metaslash.com (Neal Norwitz) Date: Mon, 13 Aug 2001 15:04:44 -0400 Subject: [Python-Dev] Direction of PyChecker References: <3B72EF79.7BE8F6C1@metaslash.com> <200108092052.f79KqQg13802@odiug.digicool.com> <3B73F824.82A53539@metaslash.com> <200108101515.LAA18907@cj20424-a.reston1.va.home.com> <15223.55925.912142.77948@beluga.mojam.com> <200108131356.JAA17936@cj20424-a.reston1.va.home.com> Message-ID: <3B7824CC.5452CDCE@metaslash.com> Guido van Rossum wrote: > For tools like PyChecker, a higher level is more appropriate: give it > an abstract syntax tree that can answer questions related to the > symbol table, so that you can tell for example whether 'x = y' assigns I agree ASTs are more appropriate, but I've been working on adding features. My plan is to move to ASTs eventually. Neal From neal@metaslash.com Mon Aug 13 20:18:28 2001 From: neal@metaslash.com (Neal Norwitz) Date: Mon, 13 Aug 2001 15:18:28 -0400 Subject: [Python-Dev] 2.2a1 patch from PyChecker References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108131546.f7DFkVk07612@odiug.digicool.com> Message-ID: <3B782804.26FB66BC@metaslash.com> Guido van Rossum wrote: > Going through your patches, I notice a few areas where I disagree. Ok. I won't make these changes in the future. > Also, I can't figure out why you want to delete the setlocale() method > from the locale.py module. In my version (2.2a1), there was was something like this: try: from x import * except ImportError: def setlocale(): pass def setlocale(): pass So it seems to me the setlocale in the except clause (which is the one that should have been deleted), will always be replaced with one outside try/except. Am I missing something or has the code changed? > Apart from these, I'm applying many of these, and Andrew has already > applied many more. So thanks again! Great. I will try to start working on the doc strings. > One thing. Next time please use a recent CVS. Ok. Neal From akuchlin@mems-exchange.org Mon Aug 13 20:22:39 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Mon, 13 Aug 2001 15:22:39 -0400 Subject: [Python-Dev] 2.2a1 patch from PyChecker In-Reply-To: <3B782804.26FB66BC@metaslash.com>; from neal@metaslash.com on Mon, Aug 13, 2001 at 03:18:28PM -0400 References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108131546.f7DFkVk07612@odiug.digicool.com> <3B782804.26FB66BC@metaslash.com> Message-ID: <20010813152239.B5318@ute.cnri.reston.va.us> On Mon, Aug 13, 2001 at 03:18:28PM -0400, Neal Norwitz wrote: >So it seems to me the setlocale in the except clause (which is the one >that should have been deleted), will always be replaced with one outside >try/except. Am I missing something or has the code changed? There's a sneaky bit where the code does '_setlocale = setlocale', and the second setlocale() definition uses _setlocale. So PyChecker is correct in reporting that setlocale() is being redefined, but this can't be removed without more extensive changes. I'd leave that decision up to the maintainer of locale.py. --amk From guido@python.org Mon Aug 13 21:05:10 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 16:05:10 -0400 Subject: [Python-Dev] 2.2a1 patch from PyChecker In-Reply-To: Your message of "Mon, 13 Aug 2001 15:18:28 EDT." <3B782804.26FB66BC@metaslash.com> References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108131546.f7DFkVk07612@odiug.digicool.com> <3B782804.26FB66BC@metaslash.com> Message-ID: <200108132005.f7DK5AE08644@odiug.digicool.com> > > Also, I can't figure out why you want to delete the setlocale() method > > from the locale.py module. > > In my version (2.2a1), there was was something like this: > > try: > from x import * > except ImportError: > def setlocale(): pass > > def setlocale(): pass > > So it seems to me the setlocale in the except clause (which is the one > that should have been deleted), will always be replaced with one outside > try/except. Am I missing something or has the code changed? You're right. I think that module is laid out a bit confusing. I think it's up to Martin von Loewis to make the cut -- it's basically his module. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Aug 13 21:47:55 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 16:47:55 -0400 Subject: [Python-Dev] yield without future statement? In-Reply-To: Your message of "Mon, 13 Aug 2001 12:24:02 EDT." <15223.65314.231423.190925@slothrop.digicool.com> References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com> <15223.65314.231423.190925@slothrop.digicool.com> Message-ID: <200108132047.f7DKltE08941@odiug.digicool.com> > I does seem almost comical that the introduction of yield as a keyword > requires a future statement but the descr branch changes do not. We discussed this extensively at a PythonLabs meeting today. Unless we get a real revolt about this from the users, we'll keep the future generators statement because that's the least work. > Talk about silent failures -- dir() no longers work, method > resolution for multiple inheritance changes, changes to the str > names of types. Why don't these require a future statement? The latest proposal is to let dir() return more rather than less: it will return the instance variable names *plus* all attributes defined by the class *and* its base classes. The MRO change is not incompatible: the MRO for classic classes is unchanged. You get the new MRO only when you explicitly request new classes (by inheriting from 'object' or specifying '__metaclass__ = type'). I don't know how I would make the name of the string object vary based on a future statement, and I doubt that this particular breakage will bother those affected much. In general, the unification introduces new or changed features only when they are explicitly requested. dir() and type("").__name__ are the rare exceptions. BTW, please read the introduction at http://www.python.org/2.2/descrintro.html --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas@xs4all.net Mon Aug 13 21:45:30 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 13 Aug 2001 22:45:30 +0200 Subject: [Python-Dev] 2.2a1 patch from PyChecker In-Reply-To: <200108131546.f7DFkVk07612@odiug.digicool.com> References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108131546.f7DFkVk07612@odiug.digicool.com> Message-ID: <20010813224530.L874@xs4all.nl> On Mon, Aug 13, 2001 at 11:46:31AM -0400, Guido van Rossum wrote: > (If others agree, we could give you checkin permission so you could check > in non-controversial fixes yourself.) If that meant PyChecker would be a standard part of the Python distribution, I would be more than all for it ;) But I would like to point out that most modules have their own maintainer, and they may have good reasons to code like they do, so if Neal does get commit privs, he shouldn't start fixing bugs on his own anyway (unless, of course, they're clearly wrong.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido@python.org Mon Aug 13 21:54:55 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 16:54:55 -0400 Subject: [Python-Dev] 2.2a1 patch from PyChecker In-Reply-To: Your message of "Mon, 13 Aug 2001 22:45:30 +0200." <20010813224530.L874@xs4all.nl> References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108131546.f7DFkVk07612@odiug.digicool.com> <20010813224530.L874@xs4all.nl> Message-ID: <200108132054.f7DKste09020@odiug.digicool.com> > > (If others agree, we could give you checkin permission so you could check > > in non-controversial fixes yourself.) > > If that meant PyChecker would be a standard part of the Python distribution, > I would be more than all for it ;) That's a separate step -- but definitely worth considering. For now, I expect that PyChecker's release cycle is so much faster than Python's that it's better to keep the two separate, but maybe a year from now PyChecker has grown up enough to do this. (We could put a PyChecker snapshot in the Tools directory around release time, but that's not much use if it keeps changing.) > But I would like to point out that most > modules have their own maintainer, and they may have good reasons to code > like they do, so if Neal does get commit privs, he shouldn't start fixing > bugs on his own anyway (unless, of course, they're clearly wrong.) I'm sure Neal will know his limits. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas@xs4all.net Mon Aug 13 21:50:43 2001 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 13 Aug 2001 22:50:43 +0200 Subject: [Python-Dev] yield without future statement? In-Reply-To: <200108131340.JAA17834@cj20424-a.reston1.va.home.com> References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com> Message-ID: <20010813225043.M874@xs4all.nl> On Mon, Aug 13, 2001 at 09:40:06AM -0400, Guido van Rossum wrote: > On c.l.py, there's a small discussion going on the necessity of a > future statement to enable the yield keyword. The general opinion > seems to be that the future statement is redundant, since previous > uses of yield will generally result in a syntax error -- and there > aren't very many uses of yield in the first place. The other uses of > future (nested scopes and division) were needed because these features > cause *silent* failure -- but with yield, you'd have to work really > really hard to cause a silent failure. I'm still against -- sorry! But it isn't just about silently breaking code, it's also about upgrade paths. I've said this before, but I don't feel comfortable upgrading our system-wide Python version, which thousands of users might use, without there being a clear warning that things will break. The user might not even notice that Python was upgraded, and not understand why the use of 'yield' as an identifier suddenly causes a syntax error. With a release where the use of yield generates a warning, that's their own fault. (And yes, CGI scripts dump their warnings in the Apache errorlog, but we have our own way of dealing with that :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From skip@pobox.com (Skip Montanaro) Mon Aug 13 23:08:52 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 13 Aug 2001 17:08:52 -0500 Subject: [Python-Dev] yield without future statement? In-Reply-To: <200108132047.f7DKltE08941@odiug.digicool.com> References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com> <15223.65314.231423.190925@slothrop.digicool.com> <200108132047.f7DKltE08941@odiug.digicool.com> Message-ID: <15224.20468.160433.254687@beluga.mojam.com> Guido> The latest proposal is to let dir() return more rather than less: Guido> it will return the instance variable names *plus* all attributes Guido> defined by the class *and* its base classes. While that may seem like a step in the right direction, it will still break things and not obviate the need for some sort of warning. Also, one might argue that for interactive use it might be less helpful than the current behavior for users to be overwhelmed with dir() output. I have these two functions available to me in interactive sessions: def dir(o=globals,showall=0): if not showall and hasattr(o, "__all__"): x = list(o.__all__) x.sort() return x from __builtin__ import dir return dir(o) def dirall(o, showall=0): attrs = dir(o, showall) if hasattr(o, "__bases__"): for b in o.__bases__: attrs.extend(dirall(b, showall)) if hasattr(o, "__class__") and o != o.__class__: attrs.extend(dirall(o.__class__, showall)) adict = {} for a in attrs: adict[a] = 1 attrs = adict.keys() attrs.sort() return attrs If I execute "dirall(gtk.GtkRadioButton())" I get a list with 175 entries. Granted, that's an extreme, but it suggests that in some cases the output of a recursive dir won't be all that helpful. What might be useful is to add a recursive flag to dir so that people can ask for everything. That way current behaviour would be preserved but users could get everything if they wanted to. Skip From ping@lfw.org Mon Aug 13 23:33:19 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Mon, 13 Aug 2001 15:33:19 -0700 (PDT) Subject: [Python-Dev] classmethod() and staticmethod() In-Reply-To: <200108132005.f7DK5AE08644@odiug.digicool.com> Message-ID: I'm very happy with the design for type-class unification -- it looks like an excellent move towards making Python more consistent and flexible. However, the mechanism for declaring class and static methods: class MyClass: def method(x, y): # part 1 ... method = staticmethod(method) # part 2 ...sticks out as awkward and strange to read. Kevin Smith and Paul Prescod commented earlier on this, and i agree with their opinion that a more direct declaration would be a better way to do this. I don't especially care what keyword or syntax is used to declare a static method, just that we require it to occur right where the method is being declared, and that it not be a "normal" expression (i.e. it must be invalid syntax in Python versions < 2.2). The two-part declaration above is unfortunate because: (a) A human reader doesn't immediately see what kind of method is being declared, and has to read ahead to figure out. (b) An automated documentation or syntax checking utility can't immediately see what kind of method is being declared, and has to scan ahead to figure out. Same thing for colourizing editors, IDEs that try to help you fix errors, etc. (c) It is possible for deceptive code to come about by accident. In much the same way that braces and indentation can get out of sync and confuse a reader, the appearance of parts 1 and 2 can get out of sync with the semantics of the code. (If someone forgets to write part 2, or inadvertently forgets to type in "self" as the first argument to an instance method, then part 1 will look like a static method even though it's an instance method. If someone writes part 2, but by habit forgets to omit "self" as the first argument, then part 1 will look like an instance method even though it's a static method. It would be much better to design this in the first place so that such mess-ups are not possible.) (d) The new declaration is syntactically correct in older Pythons. Someone unfamiliar with this new static-method mechanism will not be given very useful ideas about what is wrong. With an old version of Python, they'll get an error seeming to indicate that "staticmethod" is an undeclared function, and wonder what they need to import. I think it would be better for them to get a syntax error on the line where the method is declared, clearly indicating that a fundamentally new feature is being used and a new version of Python is required. -- ?!ng From greg@cosc.canterbury.ac.nz Tue Aug 14 00:26:55 2001 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 14 Aug 2001 11:26:55 +1200 (NZST) Subject: [Python-Dev] yield without future statement? In-Reply-To: <200108131340.JAA17834@cj20424-a.reston1.va.home.com> Message-ID: <200108132326.LAA02534@s454.cosc.canterbury.ac.nz> Guido: > The general opinion > seems to be that the future statement is redundant, since previous > uses of yield will generally result in a syntax error I thought the future statement was also to give users a release or two of warning before a semantics-changing feature becomes mandatory. Maybe I was wrong about the intention? I don't have particularly strong feelings either way in this case, but people more sensitive to code breakage might. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Tue Aug 14 01:18:32 2001 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 14 Aug 2001 12:18:32 +1200 (NZST) Subject: [Python-Dev] dir (was: yield without future statement?) In-Reply-To: <15224.20468.160433.254687@beluga.mojam.com> Message-ID: <200108140018.MAA02548@s454.cosc.canterbury.ac.nz> > If I execute "dirall(gtk.GtkRadioButton())" I get a list with 175 > entries. Seems to me that, for interactive use, it would be better to have a function that formats its output into categories, so you can see which are instance attributes and which are class attributes, and which superclass they come from. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From guido@python.org Tue Aug 14 02:58:03 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 21:58:03 -0400 Subject: [Python-Dev] classmethod() and staticmethod() In-Reply-To: Your message of "Mon, 13 Aug 2001 15:33:19 PDT." References: Message-ID: <200108140158.VAA18746@cj20424-a.reston1.va.home.com> > I'm very happy with the design for type-class unification -- it looks > like an excellent move towards making Python more consistent and flexible. Thanks! > However, the mechanism for declaring class and static methods: > > class MyClass: > def method(x, y): # part 1 > ... > > method = staticmethod(method) # part 2 > > ...sticks out as awkward and strange to read. Kevin Smith and Paul > Prescod commented earlier on this, and i agree with their opinion that > a more direct declaration would be a better way to do this. Understood, and agreed. It is my plan to eventually provide syntax for this. However, right now I need to concentrate on getting the underlying mechanisms to work. The unification is a lot of work, and I'd like to finish the run-time before I start thinking about syntactic sugar. I can't promise that I'll change the syntax before 2.2 is released -- there will be plenty of time after that, and there are plenty of more important things to do before I can make the release. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Aug 14 03:25:19 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 22:25:19 -0400 Subject: [Python-Dev] yield without future statement? In-Reply-To: Your message of "Mon, 13 Aug 2001 17:08:52 CDT." <15224.20468.160433.254687@beluga.mojam.com> References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com> <15223.65314.231423.190925@slothrop.digicool.com> <200108132047.f7DKltE08941@odiug.digicool.com> <15224.20468.160433.254687@beluga.mojam.com> Message-ID: <200108140225.WAA18816@cj20424-a.reston1.va.home.com> > Guido> The latest proposal is to let dir() return more rather than less: > Guido> it will return the instance variable names *plus* all attributes > Guido> defined by the class *and* its base classes. > > While that may seem like a step in the right direction, it will still break > things and not obviate the need for some sort of warning. Unclear. If you look it up, you'll find that dir() is documented very vaguely. It returns a list of strings, and the strings in that list are attributes of the object -- that's about it. If someone wrote code that relied on this returning a specific set of strings, they get what they deserve. Suppose this idea had started differently: as an improvement to dir(), to make it return all the attributes (or at least all the ones that are reasonably discoverable). I don't think anyone would complain. > Also, one might > argue that for interactive use it might be less helpful than the current > behavior for users to be overwhelmed with dir() output. I have these two > functions available to me in interactive sessions: > > def dir(o=globals,showall=0): > if not showall and hasattr(o, "__all__"): > x = list(o.__all__) > x.sort() > return x > from __builtin__ import dir > return dir(o) > > def dirall(o, showall=0): > attrs = dir(o, showall) > if hasattr(o, "__bases__"): > for b in o.__bases__: > attrs.extend(dirall(b, showall)) > if hasattr(o, "__class__") and o != o.__class__: > attrs.extend(dirall(o.__class__, showall)) > adict = {} > for a in attrs: > adict[a] = 1 > attrs = adict.keys() > attrs.sort() > return attrs > > If I execute "dirall(gtk.GtkRadioButton())" I get a list with 175 entries. > Granted, that's an extreme, but it suggests that in some cases the output of > a recursive dir won't be all that helpful. If interactive behavior changes, that can't be claimed to be "breaking old code". It doesn't take very long to get used to a different way of introspecting objects. > What might be useful is to add a recursive flag to dir so that people can > ask for everything. That way current behaviour would be preserved but users > could get everything if they wanted to. But "current behavior" is inconsistent. For some objects it includes instance variables and methods. For other objects it includes only instance variables. How useful is that? If an object has 175 methods, you need a class browser -- neither a recursive dir() nor a non-recursive dir() is very useful in that case. I would prefer to see help() improved to the point where it can be used for this purpose. I think I've heard just about every argument there is about dir(). In 2.1 and before, it is an ill-defined mess of a function. We can either try to give it a useful well-defined meaning, or deprecate it altogether. The only two useful well-defined meanings that I can see are either a sorted list of the keys of the argument's __dict__ (as implemented in Python 2.2a1), or a sorted list of *all* the discoverable attributes of the object. I actually prefer the first for myself, but this received lots of complaints about how returning [] for list instances would break code, so I am now proposing the second well-defined meaning. I find it hard to imagine how returning *more* names would still break code -- that code cannot have been very robust in the first place. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@rahul.net Tue Aug 14 03:31:11 2001 From: aahz@rahul.net (Aahz Maruch) Date: Mon, 13 Aug 2001 19:31:11 -0700 (PDT) Subject: [Python-Dev] 2.2a1: classmethod() and class attributes In-Reply-To: <200108132047.f7DKltE08941@odiug.digicool.com> from "Guido van Rossum" at Aug 13, 2001 04:47:55 PM Message-ID: <20010814023111.3273A99C80@waltz.rahul.net> Guido van Rossum wrote: > > BTW, please read the introduction at > http://www.python.org/2.2/descrintro.html Okay, I did that. It's not particularly convenient for me to try out 2.2a1, so I'm just going by what's written. One little hole that I don't see an answer to is what happens when you do this: class C(object): x = 0 def foo(cls): cls.x += 1 foo = classmethod(foo) C.foo() -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From barry@zope.com Tue Aug 14 03:49:05 2001 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 13 Aug 2001 22:49:05 -0400 Subject: [Python-Dev] 2.2a1 patch from PyChecker References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108092120.f79LKNo13875@odiug.digicool.com> <20010809180903.A31343@ute.cnri.reston.va.us> Message-ID: <15224.37281.601490.99294@anthem.wooz.org> >>>>> "AK" == Andrew Kuchling writes: AK> That reminds me: should we put warnings in cgi.py for AK> instantiations of FormContent and other old classes, so we can AK> delete them in Python 2.3? One problem: if a class doesn't AK> have an __init__ method, subclasses that have their own AK> __init__ methods won't call it. Maybe this could be done by AK> adding 'self.warn()' to all the methods of an outdated class, AK> and defining a warn() method that raises a warning. Which reminds me: do you think you'll one day resurrect PEP 222? -Barry From guido@python.org Tue Aug 14 04:59:29 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Aug 2001 23:59:29 -0400 Subject: [Python-Dev] 2.2a1: classmethod() and class attributes In-Reply-To: Your message of "Mon, 13 Aug 2001 19:31:11 PDT." <20010814023111.3273A99C80@waltz.rahul.net> References: <20010814023111.3273A99C80@waltz.rahul.net> Message-ID: <200108140359.XAA19714@cj20424-a.reston1.va.home.com> > One little hole that I don't see an answer to is what happens when > you do this: > > class C(object): > x = 0 > def foo(cls): > cls.x += 1 > foo = classmethod(foo) > > C.foo() As written, you get TypeError: can't set type attributes. If you add __dynamic__ = 1 to the class statement, it works as expected. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com (Skip Montanaro) Tue Aug 14 05:04:04 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 13 Aug 2001 23:04:04 -0500 Subject: [Python-Dev] yield without future statement? In-Reply-To: <200108140225.WAA18816@cj20424-a.reston1.va.home.com> References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com> <15223.65314.231423.190925@slothrop.digicool.com> <200108132047.f7DKltE08941@odiug.digicool.com> <15224.20468.160433.254687@beluga.mojam.com> <200108140225.WAA18816@cj20424-a.reston1.va.home.com> Message-ID: <15224.41780.829026.859816@beluga.mojam.com> Guido> Suppose this idea had started differently: as an improvement to Guido> dir(), to make it return all the attributes (or at least all the Guido> ones that are reasonably discoverable). I don't think anyone Guido> would complain. Yeah, but there have already been complaints about the change to what dir() returns for list instances in the context of the 2.2 class/type unification. I can only suppose there are going to be complaints about other changes (from other people) if dir() changes in other ways. That was the main point of my post. I probably should have avoided the issue of how verbose the output might be. most-people-don't-read-the-docs-ly y'rs, Skip From dgoodger@bigfoot.com Tue Aug 14 05:29:39 2001 From: dgoodger@bigfoot.com (David Goodger) Date: Tue, 14 Aug 2001 00:29:39 -0400 Subject: [Python-Dev] Re: [Python-checkins] testing Message-ID: I'm not sure if this is the problem, but here goes... I was unable to set up syncmail for my projects; the executable permissions wouldn't stick on CVSROOT/syncmail. I solved it by adding an explicit /usr/bin/python to CVSROOT/loginfo:: DEFAULT /usr/bin/python $CVSROOT/CVSROOT/syncmail ... syncmail no longer needs to be executable. Hope this helps. -- David Goodger dgoodger@bigfoot.com Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From dgoodger@bigfoot.com Tue Aug 14 05:44:52 2001 From: dgoodger@bigfoot.com (David Goodger) Date: Tue, 14 Aug 2001 00:44:52 -0400 Subject: [Python-Dev] AST mining (was Re: Direction of PyChecker) Message-ID: >From the PyChecker AST discussion, it seems we may have a common goal. For the Docstring Processing System, I am looking into gleaning information from the abstract syntax tree. From the working notes, under "Docstring Extractor": We need code that scans a parsed Python module, and returns an ordered tree containing the names, docstrings (including additional docstrings), and additional info (in parentheses below) of all of the following objects: - packages - modules - module attributes (+ values) - classes (+ inheritance) - class attributes (+ values) - instance attributes (+ values) - methods (+ formal parameters) - functions (+ formal parameters) In order to evaluate interpreted text cross-references, namespaces for each of the above will also be required. I'd be very interested in pooling efforts to make this easier. I know almost nothing about ASTs now, but that could change in a hurry :-). -- David Goodger dgoodger@bigfoot.com Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From guido@python.org Tue Aug 14 05:43:29 2001 From: guido@python.org (Guido van Rossum) Date: Tue, 14 Aug 2001 00:43:29 -0400 Subject: [Python-Dev] yield without future statement? In-Reply-To: Your message of "Mon, 13 Aug 2001 23:04:04 CDT." <15224.41780.829026.859816@beluga.mojam.com> References: <200108131340.JAA17834@cj20424-a.reston1.va.home.com> <15223.65314.231423.190925@slothrop.digicool.com> <200108132047.f7DKltE08941@odiug.digicool.com> <15224.20468.160433.254687@beluga.mojam.com> <200108140225.WAA18816@cj20424-a.reston1.va.home.com> <15224.41780.829026.859816@beluga.mojam.com> Message-ID: <200108140443.AAA19887@cj20424-a.reston1.va.home.com> > Guido> Suppose this idea had started differently: as an improvement to > Guido> dir(), to make it return all the attributes (or at least all the > Guido> ones that are reasonably discoverable). I don't think anyone > Guido> would complain. > > Yeah, but there have already been complaints about the change to what dir() > returns for list instances in the context of the 2.2 class/type unification. > I can only suppose there are going to be complaints about other changes > (from other people) if dir() changes in other ways. But the proposed change would revert the change that people complained about: it would return all the methods of a list, like before. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Tue Aug 14 06:26:19 2001 From: tim.one@home.com (Tim Peters) Date: Tue, 14 Aug 2001 01:26:19 -0400 Subject: [Python-Dev] AST mining (was Re: Direction of PyChecker) In-Reply-To: Message-ID: [David Goodger] > ... > For the Docstring Processing System, I am looking into gleaning > information from the abstract syntax tree. > ... > - packages > - modules > - module attributes (+ values) > - classes (+ inheritance) > - class attributes (+ values) > - instance attributes (+ values) > - methods (+ formal parameters) > - functions (+ formal parameters) > ... > I'd be very interested in pooling efforts to make this easier. I > know almost nothing about ASTs now, but that could change in a > hurry :-). Let me suggest you don't really want an AST -- you want an object model for Python source that answers the questions above directly. An AST may be an effective (under the covers) implementation technique to get such info, but if you don't want to wait for people to argue about "the right" AST and "the right" tree-based query language to make it better than completely useless , you can answer all the stuff above by building on tokenize.py now. BTW, exploiting generators in 2.2 can make a tokenize-based approach much more pleasant than before; see the newish Tools/scripts/cleanfuture.py (in CVS) for an example of how easily 1-token lookahead parsing can be coded now. From tim.one@home.com Tue Aug 14 06:36:49 2001 From: tim.one@home.com (Tim Peters) Date: Tue, 14 Aug 2001 01:36:49 -0400 Subject: [Python-Dev] yield without future statement? In-Reply-To: <15224.41780.829026.859816@beluga.mojam.com> Message-ID: [Skip Montanaro] > Yeah, but there have already been complaints about the change to > what dir() returns for list instances in the context of the 2.2 > class/type unification. I'm finding it impossible to keep up with all the discussions, but I believe I've seen every msg related to this one. I believe they were all wondering why dir([]) or dir({}) returned [] under 2.2a1 at *an interactive prompt*. Indeed, I beat everyone to that by asking Guido the same thing perhaps a month ago . > I can only suppose there are going to be complaints about other > changes (from other people) if dir() changes in other ways. Sure. And now that Guido has suggested a richer dir(), there will be complaints too if we leave dir() alone! The richer dir() attracted some real enthusiasm. But dir() simply has no well-defined purpose beyond dir(module), so IMO complaints about dir() are declarations of preference-- and nothing more than that --and so unavoidable. From skip@pobox.com (Skip Montanaro) Tue Aug 14 06:57:44 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Tue, 14 Aug 2001 00:57:44 -0500 Subject: [Python-Dev] Proposed PEP: Optimizing Global Variable and Attribute Access Message-ID: <15224.48600.621401.101492@beluga.mojam.com> Here's a first stab at a PEP for optimizing global variable and attribute access. Barry, when you get a chance, could you assign it a number and let me know about any PEP-related formatting mistakes I made? (It would be nice if PEP 1 referenced a piece of boilerplate people could use. While the PEP Style section lists the headers, I wasn't sure what the Version and Last-Modified headers were actually supposed to look like. Also, it wasn't clear what I was supposed to put in the Post-History: header.) Skip ------------------------------------------------------------------------------ PEP: Title: Optimizing Global Variable and Attribute Access Version: $Revision:$ Author: skip@pobox.com (Skip Montanaro) Status: Draft Type: Standards Track Python-Version: 2.3 Created: 13-Aug-2001 Post-History: Abstract The bindings for most global variables and attributes of other modules typically never change during the execution of a Python program, but because of Python's dynamic nature, code which accesses such global objects must run through a full lookup each time the object is needed. This PEP proposes a mechanism that allows code that accesses most global objects to treat them as local objects and places the burden of updating references on the code that changes the name bindings of such objects. Introduction Consider the workhorse function sre_compile._compile. It is the internal compilation function for the sre module. It consists almost entirely of a loop over the elements of the pattern being compiled, comparing opcodes with known constant values and appending tokens to an outpu list. Most of the comparisons are with constants imported from the sre_constants module. This means there are lots of LOAD_GLOBAL bytecodes in the compiled output of this module. Just by reading the code it's apparent that the author intended LITERAL, NOT_LITERAL, OPCODES and many other symbols to be constants. Still, each time they are involved in an expression, they must be looked up anew. Most global accesses are actually to objects that are "almost constants". This includes global variables in the current module as well as the attributes of other imported modules. Since they rarely change, it seems reasonable to place the burden of updating references to such objects on the code that changes the name bindings. If sre_constants.LITERAL is changed to refer to another object, perhaps it would be worthwhile for the code that modifies the sre_constants module dict to correct any active references to that object. By doing so, in many cases global variables and the attributes of many objects could be cached as local variables. If the bindings between the names given to the objects and the objects themselves changes rarely, the cost of keeping track of such objects should be low and the potential payoff fairly large. Proposed Change I propose that the Python virtual machine be modified to include TRACK_OBJECT and UNTRACK_OBJECT opcodes. TRACK_OBJECT would associate a global name or attribute of a global name with a slot in the local variable array and perform an initial lookup of the associated object to fill in the slot with a valid value. The association it creates would be noted by the code responsible for changing the name-to-object binding to cause the associated local variable to be updated. The UNTRACK_OBJECT opcode would delete any association between the name and the local variable slot. Rationale Global variables and attributes rarely change. For example, once a function imports the math module, the binding between the name "math" and the module it refers to aren't likely to change. Similarly, if the function that uses the math module refers to its "sin" attribute, it's unlikely to change. Still, every time the module wants to call the math.sin function, it must first execute a pair of instructions: LOAD_GLOBAL math LOAD_ATTR sin If the client module always assumed that math.sin was a local constant and it was the responsibility of "external forces" outside the function to keep the reference correct, we might have code like this: TRACK_OBJECT math.sin ... LOAD_FAST math.sin ... UNTRACK_OBJECT math.sin If the LOAD_FAST was in a loop the payoff in reduced global loads and attribute lookups could be significant. This technique could, in theory, be applied to any global variable access or attribute lookup. Consider this code: l = [] for i in range(10): l.append(math.sin(i)) return l Even though l is a local variable, you still pay the cost of loading l.append ten times in the loop. The compiler (or an optimizer) could recognize that both math.sin and l.append are being called in the loop and decide to generate the tracked local code, avoiding it for the builtin range() function because it's only called once during loop setup. According to a post to python-dev by Marc-Andre Lemburg [1], LOAD_GLOBAL opcodes account for over 7% of all instructions executed by the Python virtual machine. This can be a very expensive instruction, at least relative to a LOAD_FAST instruction, which is a simple array index and requires no extra function calls by the virtual machine. I believe many LOAD_GLOBAL instructions and LOAD_GLOBAL/ LOAD_ATTR pairs could be converted to LOAD_FAST instructions. Code that uses global variables heavily often resorts to various tricks to avoid global variable and attribute lookup. The aforementioned sre_compile._compile function caches the append method of the growing output list. Many people commonly abuse functions' default argument feature to cache global variable lookups. Both of these schemes are hackish and rarely address all the available opportunities for optimization. (For example, sre_compile._compile does not cache the two globals that it uses most frequently: the builtin len function and the global OPCODES array that it imports from sre_constants.py. Discussion Jeremy Hylton has an alternate proposal on the table [2]. His proposal seeks to create a hybrid dictionary/list object for use in global name lookups that would make global variable access look more like local variable access. While there is no C code available to examine, the Python implementation given in his proposal still appears to require dictionary key lookup. It doesn't appear that his proposal could speed local variable attribute lookup, which might be worthwhile in some situations. Backwards Compatibility I don't believe there will be any serious issues of backward compatibility. Obviously, Python bytecode that contains TRACK_OBJECT opcodes could not be executed by earlier versions of the interpreter, but breakage at the bytecode level is often assumed between versions. Implementation TBD. This is where I need help. I believe there should be either a central name/location registry or the code that modifies object attributes should be modified, but I'm not sure the best way to go about this. If you look at the code that implements the STORE_GLOBAL and STORE_ATTR opcodes, it seems likely that some changes will be required to PyDict_SetItem and PyObject_SetAttr or their String variants. Ideally, there'd be a fairly central place to localize these changes. If you begin considering tracking attributes of local variables you get into issues of modifying STORE_FAST as well, which could be a problem, since the name bindings for local variables are changed much more frequently. (I think an optimizer could avoid inserting the tracking code for the attributes for any local variables where the variable's name binding changes.) Performance I believe (though I have no code to prove it at this point), that implementing TRACK_OBJECT will generally not be much more expensive than a single LOAD_GLOBAL instruction or a LOAD_GLOBAL/LOAD_ATTR pair. An optimizer should be able to avoid converting LOAD_GLOBAL and LOAD_GLOBAL/LOAD_ATTR to the new scheme unless the object access occurred within a loop. Further down the line, a register-oriented replacement for the current Python virtual machine [3] could conceivably eliminate most of the LOAD_FAST instructions as well. The number of tracked objects should be relatively small. All active frames of all active threads could conceivably be tracking objects, but this seems small compared to the number of functions defined in a given application. References [1] http://mail.python.org/pipermail/python-dev/2000-July/007609.html [2] http://www.zope.org/Members/jeremy/CurrentAndFutureProjects/FastGlobalsPEP [3] http://www.musi-cal.com/~skip/python/rattlesnake20010813.tar.gz Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil End: From martin@loewis.home.cs.tu-berlin.de Tue Aug 14 09:05:01 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Tue, 14 Aug 2001 10:05:01 +0200 Subject: [Python-Dev] 2.2a1: classmethod() and class attributes Message-ID: <200108140805.f7E851j01484@mira.informatik.hu-berlin.de> > If you add __dynamic__ = 1 to the class statement, it works as > expected. Is that documented anywhere yet? I searched all three documents (the PEPs, and the intro), and could not find any mentioning of __dynamic__. Regards, Martin From gmcm@hypernet.com Tue Aug 14 13:11:13 2001 From: gmcm@hypernet.com (Gordon McMillan) Date: Tue, 14 Aug 2001 08:11:13 -0400 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: <15223.61656.410876.775564@slothrop.digicool.com> References: <200108131356.JAA17936@cj20424-a.reston1.va.home.com> Message-ID: <3B78DD21.31888.224FE356@localhost> Jeremy wrote: [byte code scanning -> AST analysis] > .... We've got > pychecker and pydoc as to drive the requirements. Don't forget freeze/modulefinder.py. - Gordon From jack@oratrix.nl Tue Aug 14 13:43:03 2001 From: jack@oratrix.nl (Jack Jansen) Date: Tue, 14 Aug 2001 14:43:03 +0200 Subject: [Python-Dev] Any NeXT users out there? In-Reply-To: Message by "Martin v. Loewis" , Mon, 13 Aug 2001 18:01:56 +0200 , <200108131601.f7DG1uB01479@mira.informatik.hu-berlin.de> Message-ID: <20010814124304.01511303181@snelboot.oratrix.nl> > > The alternative is of course that we also provide a > > "--with-previous-framework". Any suggestions as to what it should do? :-) > > Based on your current patch, I think it is save to remove any claim > that this does something useful on the Next. It may do, but that would > be purely coincidental. That was a joke:-) But seriously: if noone speaks up within the next couple of hours I will rip out NeXT support. I'll also add an option (--enable-toolbox-glue?) to rip out the glue code for MacPython/Carbon modules, thereby increasing the chances that this will all work on a bare Darwin without MacOSX. > > On a high level it is a very nifty way to package a shared library > > complete with all its supporting files. These supporting files range > > from things used at runtime (like the python Lib directory) to the C > > header files needed when you want to link against the library to the > > documentation. > > Ok. Then my question is: Why is it desirable to build libpython as a > framework? Sharing is the answer. When everything is finished an average Python user will have at least the interpreter application (the bundled drag-and-drop one) and the IDE installed. There may be auxiliary apps like the current MacPython's BuildApplet and BuildApplication. Eventually a slightly specialised interpreter that does the Cocoa/NIB magic on startup so that creating Cocoa apps in Python should be (almost?) as easy as creating them in ObjC or Java. They may have the commandline interpreter too, if they're one of the 5% of mac users who aren't scared sh*tless by a command line. They may have Zope. They may have any of a number of other apps that are yet to be created that are implemented in Python. > > Everything is in a single directory, so install/uninstall is a > > breeze with no danger of missing things. > > In case of Python, "everything" is the header files, and the .py > library, right? Yep. This is all you would need to either embed Python or extend it. So even for many developers there would be no reason to get a source distribution. > Couldn't you also get this by making Python an Application Bundle > (which you intend to do anyway)? I've looked at that, but I don't think you can get the sharing explained above by that. Also, the development tools all know about frameworks, and if we'd want to pick stuff up from an application bundle we would have to hand-craft things. > > Also, the structure of a framework allows you to have older, > > incompatible versions of the library (and support documents) in the > > framework as well, with these older versions only used by programs > > that were linked against the framework when that version was > > current. > > Again, why is that desirable? If you have multiple versions of Python > installed, if each version has libpython linked statically, and comes > with its own set of header files and .py files: there wouldn't be any > older, incompatible versions of the library that you may need to > defend against. You shouldn't really be thinking "multiple versions of Python" here but "multiple products which are implemented with different versions of Python". Think of the sort of things RealBasic can do, allowing end-users with a little programming experience create fullblown applications they can distribute. That's where I would like Python on OSX to go. And iff we get the ObjC stuff revived and make it work together with Cocoa then those end users can use Interface Builder to create their UI, so they main piece of functionality that Python misses when compared to RealBasic, the guibuilder, is provided for us by Apple. > You may wonder why I question that feature so much. To me, it is > similar to building a shared libpython.so on Unix. I think this will > cause many problems, and in no way it should be done just for a single > platform (there were patches to do this for Linux, I believe, and such > a patch to make it work specifically on UnixWare only even made it > into 2.1). I think it is desirable that the feature "shared libpython" > is implemented as uniformly as possible across systems. > > If Mac OS X wants to do it differently from everybody else, you need a > very good reason. This is probably the crux of your reasoning, and I think I disagree. For one, MacOSX (ok, OpenStep is where they got it from) is the first unix-based system that I'm aware of that seems to have a solution to the versioning problem. All unixes that I know simply stick a version number in the .so filename and leave it at that, and the only reason dll-hell hasn't hit unix as hard as Windows is because it's deployed a lot less, used by more knowledgeable people and Microsoft doesn't get to touch it. But the problem is essentially there. Now that MacOSX has a versioning system we should put it to good use. And maybe we'll like it so much that we can use it for all Pythons. Also, while I would like Python to be a model MacOSX citizen I'm also trying hard to make it as good a Python community citizen as it can be, so I'm putting a 100% compatible unix Python installation into the framework. Not only will this make pythonistas with a unix background feel at home, but it should also mean that there's no extra code needed in distutils and all the other stuff that knows what a Python installation looks like. > The only platform that already has a shared pythonXY.dll has a good > reason: you cannot export symbols from the application to extension > modules on Windows. Yet, this is also the only platform where the API > version magic is meaningless, since extension modules break with every > new Python version exactly because libpython is shared. The same is true for MacPython, and also true for Python on MacOSX. But whereas with MacPython extension modules also broke every single release (even if only because the underlying C library was a moving target) with MacOSX I think we can do a bit better. The only thing to work is is exactly how to put version numbers to good use, because extension modules linked against Python will be interested primarily in the API version number but applications using Python will be interested primarily in the 2.2-style version number, and I'm not sure yet how to work that out. Maybe prepending the API version number (so the full version would becomes something like "1011.2.2) will work. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From aahz@rahul.net Tue Aug 14 15:03:36 2001 From: aahz@rahul.net (Aahz Maruch) Date: Tue, 14 Aug 2001 07:03:36 -0700 (PDT) Subject: [Python-Dev] Proposed PEP: Optimizing Global Variable and Attribute Access In-Reply-To: <15224.48600.621401.101492@beluga.mojam.com> from "Skip Montanaro" at Aug 14, 2001 12:57:44 AM Message-ID: <20010814140336.C55D0E8C1@waltz.rahul.net> Skip Montanaro wrote: > > The number of tracked objects should be relatively small. All active > frames of all active threads could conceivably be tracking objects, but > this seems small compared to the number of functions defined in a given > application. Hmmm? That doesn't seem quite right to me. That's just a knee-jerk reaction, I don't have any real evidence. As a side note, I'm not sure whether some mention should be made that TRACK_OBJECT specifically does not work any differently from current Python when it comes to threads. That is, if it's a shared variable of any sort, you need a mutex if you're going to perform multiple operations on it. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From guido@python.org Tue Aug 14 15:13:28 2001 From: guido@python.org (Guido van Rossum) Date: Tue, 14 Aug 2001 10:13:28 -0400 Subject: [Python-Dev] Direction of PyChecker In-Reply-To: Your message of "Tue, 14 Aug 2001 08:11:13 EDT." <3B78DD21.31888.224FE356@localhost> References: <200108131356.JAA17936@cj20424-a.reston1.va.home.com> <3B78DD21.31888.224FE356@localhost> Message-ID: <200108141413.KAA20547@cj20424-a.reston1.va.home.com> > > .... We've got > > pychecker and pydoc as to drive the requirements. > > Don't forget freeze/modulefinder.py. And pyclbr. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com (Skip Montanaro) Tue Aug 14 15:48:41 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Tue, 14 Aug 2001 09:48:41 -0500 Subject: [Python-Dev] Proposed PEP: Optimizing Global Variable and Attribute Access In-Reply-To: <20010814140336.C55D0E8C1@waltz.rahul.net> References: <15224.48600.621401.101492@beluga.mojam.com> <20010814140336.C55D0E8C1@waltz.rahul.net> Message-ID: <15225.14921.675135.797646@beluga.mojam.com> Aahz> Skip Montanaro wrote: >> The number of tracked objects should be relatively small. All active >> frames of all active threads could conceivably be tracking objects, >> but this seems small compared to the number of functions defined in a >> given application. Aahz> Hmmm? That doesn't seem quite right to me. That's just a Aahz> knee-jerk reaction, I don't have any real evidence. Here's my rationale. Perhaps I've goofed somewhere. TRACK_OBJECT and UNTRACK_OBJECT opcodes will only be inserted in the bodies of functions. Functions are only considered active when there is an active (but possibly suspended) eval_frame call on the C stack that is executing that function's byte code. Suppose my module is def a(): print "a" def b(): print "b" a() def c(): print "c" b() At the point where a's print statement is executed, functions a and b are active. Function c is not. This reasoning applies on a per-thread basis. So the number of active functions by my definition is the number of times eval_frame appears on the call stack for all active threads. In most situations that will be far less (barring deep recursion) than the number of functions available to the application. All I guess I'm trying to communicate is that object tracking is a dynamic thing, not a static thing. Aahz> As a side note, I'm not sure whether some mention should be made Aahz> that TRACK_OBJECT specifically does not work any differently from Aahz> current Python when it comes to threads. That is, if it's a Aahz> shared variable of any sort, you need a mutex if you're going to Aahz> perform multiple operations on it. That's true. If a global object is shared among multiple threads, access to it must still be protected. I haven't added anything to the mix in that regard. I'll add a short section on threads. Hmm... What about this (dumb) code? l = [] lock = threading.Lock() ... def fill_l(): for i in range(1000): lock.acquire() l.append(math.sin(i)) lock.release() ... def consume_l(): while 1: lock.acquire() if l: elt = l.pop() lock.release() fiddle(elt) It's not clear from a static analysis of the code what the lock is protecting. (You can't tell at compile-time that threads are even involved can you?) Would or should it affect attempts to track "l.append" or "math.sin" in the fill_l function? If we annotate the code with mythical track_object and untrack_object builtins (I'm not proposing such functions (they can't be implemented with Python's call-by-value semantics), just illustrating where stuff would go in the bytecode), we get l = [] lock = threading.Lock() ... def fill_l(): track_object("l.append", append) track_object("math.sin", sin) for i in range(1000): lock.acquire() append(sin(i)) lock.release() untrack_object("math.sin", sin) untrack_object("l.append", append) ... def consume_l(): while 1: lock.acquire() if l: elt = l.pop() lock.release() fiddle(elt) Is that correct both with and without threads? Skip From akuchlin@mems-exchange.org Tue Aug 14 15:52:53 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Tue, 14 Aug 2001 10:52:53 -0400 Subject: [Python-Dev] 2.2a1 patch from PyChecker In-Reply-To: <15224.37281.601490.99294@anthem.wooz.org>; from barry@zope.com on Mon, Aug 13, 2001 at 10:49:05PM -0400 References: <3B72E5A0.DC2E8C4E@metaslash.com> <200108092120.f79LKNo13875@odiug.digicool.com> <20010809180903.A31343@ute.cnri.reston.va.us> <15224.37281.601490.99294@anthem.wooz.org> Message-ID: <20010814105253.B9746@ute.cnri.reston.va.us> On Mon, Aug 13, 2001 at 10:49:05PM -0400, Barry A. Warsaw wrote: >Which reminds me: do you think you'll one day resurrect PEP 222? I'll take another look at it. It seems impossible to converge on a list of advanced features to add (everyone wants something different), but it would be nice to clean out some of the cruft that's accumulated in cgi.py. --amk From neal@metaslash.com Tue Aug 14 16:13:32 2001 From: neal@metaslash.com (Neal Norwitz) Date: Tue, 14 Aug 2001 11:13:32 -0400 Subject: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker) References: Message-ID: <3B79401C.82D51FBA@metaslash.com> David Goodger wrote: > > >From the PyChecker AST discussion, it seems we may have a common goal. For It seems that way. > the Docstring Processing System, I am looking into gleaning information from > the abstract syntax tree. From the working notes, under "Docstring > Extractor": > > We need code that scans a parsed Python module, and returns an > ordered tree containing the names, docstrings (including > additional docstrings), and additional info (in parentheses below) > of all of the following objects: > - methods (+ formal parameters) > - functions (+ formal parameters) Would you also want default parameter values? PyChecker needs all of that, plus additional info. So there is a lot of overlap. > I'd be very interested in pooling efforts to make this easier. I know almost > nothing about ASTs now, but that could change in a hurry :-). Pooling efforts would be good. I also don't know anything about the ASTs/compiler, but am willing to work on it. Neal From jeremy@zope.com Tue Aug 14 16:29:01 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Tue, 14 Aug 2001 11:29:01 -0400 (EDT) Subject: [Python-Dev] AST mining (was Re: Direction of PyChecker) In-Reply-To: References: Message-ID: <15225.17341.690478.601359@slothrop.digicool.com> >>>>> "TP" == Tim Peters writes: TP> Let me suggest you don't really want an AST -- you want an TP> object model for Python source that answers the questions above TP> directly. Agreed. TP> directly. An AST may be an effective (under the covers) TP> implementation technique to get such info, but if you don't want TP> to wait for people to argue about "the right" AST and "the TP> right" tree-based query language to make it better than TP> completely useless , you can answer all the stuff above by TP> building on tokenize.py now. I wouldn't wait for people to argue about the right AST either. Use the one Greg and Bill came up with for p2c. It's in Tools/compiler in the Python distribution. It's much simpler than the parse tree produced by the parser module. And, as far as I know, no one is advocating a different AST. It doesn't have a query language, but neither does tokenize.py . Jeremy From guido@python.org Tue Aug 14 16:43:03 2001 From: guido@python.org (Guido van Rossum) Date: Tue, 14 Aug 2001 11:43:03 -0400 Subject: [Python-Dev] 2.2a1: classmethod() and class attributes In-Reply-To: Your message of "Tue, 14 Aug 2001 10:05:01 +0200." <200108140805.f7E851j01484@mira.informatik.hu-berlin.de> References: <200108140805.f7E851j01484@mira.informatik.hu-berlin.de> Message-ID: <200108141543.f7EFh3X10221@odiug.digicool.com> > > If you add __dynamic__ = 1 to the class statement, it works as > > expected. > > Is that documented anywhere yet? I searched all three documents (the > PEPs, and the intro), and could not find any mentioning of > __dynamic__. Not yet, sorry. Based on feedback, I may have to change the default to __dynamic__ = 1. --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Tue Aug 14 16:50:40 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Tue, 14 Aug 2001 11:50:40 -0400 (EDT) Subject: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker) In-Reply-To: <3B79401C.82D51FBA@metaslash.com> References: <3B79401C.82D51FBA@metaslash.com> Message-ID: <15225.18640.291156.850128@slothrop.digicool.com> >>>>> "NN" == Neal Norwitz writes: >> I'd be very interested in pooling efforts to make this easier. I >> know almost nothing about ASTs now, but that could change in a >> hurry :-). NN> Pooling efforts would be good. I also don't know anything about NN> the ASTs/compiler, but am willing to work on it. I'm on the hook for AST/compiler documentation, which I plan to work on this week. I will probably have more time for it later in the week than I will today or tomorrow. In the absence of documentation, here's a trivial example program that extracts some information about methods and attributes from a class and its methods. I wasn't exhaustive here. I'll get attributes assigned to by "self.x" in a method body and I'll get method definitions in the class body. I don't deal with obvious things like attributes defined at the class level. Jeremy from compiler import parseFile, walk, ast class Class: def __init__(self, name, bases): self.name = name self.bases = bases self.methods = {} self.attributes = {} def addMethod(self, meth): self.methods[meth.name] = meth def addInstanceAttr(self, name): self.attributes[name] = name def getMethodNames(self): return self.methods.keys() def getAttrNames(self): return self.attributes.keys() class Method: def __init__(self, name, args, defaults): self.name = name self.args = args self.defaults = defaults def getSelf(self): return self.args[0] class ClassExtractor: classes = [] def visitClass(self, node, klass=None, meth=None): c = Class(node.name, node.bases) self.visit(node.code, c) self.classes.append(c) def visitFunction(self, node, klass=None, meth=None): if klass is not None and meth is None: m = Method(node.name, node.argnames, node.defaults) klass.addMethod(m) self.visit(node.code, klass, m) else: self.visit(node.code) def visitAssAttr(self, node, klass=None, meth=None): if isinstance(node.expr, ast.Name) and meth is not None: if node.expr.name == meth.getSelf(): klass.addInstanceAttr(node.attrname) else: self.visit(node.expr) def main(py_files): extractor = ClassExtractor() for py in py_files: ast = parseFile(py) walk(ast, extractor) for klass in extractor.classes: print klass.name print klass.getMethodNames() print klass.getAttrNames() print if __name__ == "__main__": import sys main(sys.argv[1:]) From tony@lsl.co.uk Tue Aug 14 16:56:12 2001 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Tue, 14 Aug 2001 16:56:12 +0100 Subject: [Docstring-develop] RE: [Python-Dev] AST mining (was Re: Direction of PyChecker) In-Reply-To: <15225.17341.690478.601359@slothrop.digicool.com> Message-ID: <005301c124d9$a47e5350$f05aa8c0@lslp7o.int.lsl.co.uk> Jeremy Hylton wrote: > I wouldn't wait for people to argue about the right AST either. Use > the one Greg and Bill came up with for p2c. It's in Tools/compiler in > the Python distribution. It's much simpler than the parse tree > produced by the parser module. And, as far as I know, no one is > advocating a different AST. Indeed, it was the one I used for my stpy.py "testbed" earlier this year (this had code that trivially found docstrings for a module and its classes and functions/methods, and presented them (albeit not very prettily) in HTML as such. The code to use compiler was *very* simple, at this level. It even works under 1.5.2 (with, I think, two lines commented out). As I remember, it wasn't terribly fast, but who cares... It also lacked documentation, but that just means some more staring at code (and maybe the parser documentation) to work out what to expect. > It doesn't have a query language, but neither does tokenize.py . Hmm - for the purposes we're after, any "query" is purely theoretical, I think (i.e., in the sense that *everything* done on a datastructure might be considered a query). At least for the moment. Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ "How fleeting are all human passions compared with the massive continuity of ducks." - Dorothy L. Sayers, "Gaudy Night" My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From tim.one@home.com Tue Aug 14 16:58:20 2001 From: tim.one@home.com (Tim Peters) Date: Tue, 14 Aug 2001 11:58:20 -0400 Subject: [Python-Dev] AST mining (was Re: Direction of PyChecker) In-Reply-To: <15225.17341.690478.601359@slothrop.digicool.com> Message-ID: [Jeremy Hylton] > ... > I wouldn't wait for people to argue about the right AST either. Use > the one Greg and Bill came up with for p2c. It's in Tools/compiler in > the Python distribution. It's much simpler than the parse tree > produced by the parser module. And, as far as I know, no one is > advocating a different AST. > > It doesn't have a query language, but neither does tokenize.py . The likely problem I see is documentation: exactly what is this AST, and how does one use the compiler package? compiler's demo.py is exactly on-target for what people want to do here, and is very simple code, but how would someone coming into this cold come up with this code on their own? The import tricks don't help. I just tried to reverse-engineer demo.py and got stuck on the ast = compiler.parse(buf) line. Granted I've running a fever, but I'm not sure I would have thought to study __init__.py even if not. The relative feebleness of tokenize.py is an advantage in this respect: it doesn't do much, has some docs, and is all in one function, so it's straightforward to guess what it's doing. From aahz@rahul.net Tue Aug 14 17:16:10 2001 From: aahz@rahul.net (Aahz Maruch) Date: Tue, 14 Aug 2001 09:16:10 -0700 (PDT) Subject: [Python-Dev] Proposed PEP: Optimizing Global Variable and Attribute Access In-Reply-To: <15225.14921.675135.797646@beluga.mojam.com> from "Skip Montanaro" at Aug 14, 2001 09:48:41 AM Message-ID: <20010814161610.21ADDE8C1@waltz.rahul.net> [sorry, excessive quoting ahead] Skip Montanaro wrote: > > > Aahz> Skip Montanaro wrote: > > >> The number of tracked objects should be relatively small. All active > >> frames of all active threads could conceivably be tracking objects, > >> but this seems small compared to the number of functions defined in a > >> given application. > > Aahz> Hmmm? That doesn't seem quite right to me. That's just a > Aahz> knee-jerk reaction, I don't have any real evidence. > > Here's my rationale. Perhaps I've goofed somewhere. TRACK_OBJECT and > UNTRACK_OBJECT opcodes will only be inserted in the bodies of functions. > Functions are only considered active when there is an active (but possibly > suspended) eval_frame call on the C stack that is executing that function's > byte code. Suppose my module is > > def a(): > print "a" > > def b(): > print "b" > a() > > def c(): > print "c" > > b() > > At the point where a's print statement is executed, functions a and b are > active. Function c is not. > > This reasoning applies on a per-thread basis. So the number of active > functions by my definition is the number of times eval_frame appears on the > call stack for all active threads. In most situations that will be far less > (barring deep recursion) than the number of functions available to the > application. All I guess I'm trying to communicate is that object tracking > is a dynamic thing, not a static thing. That's true. My point was more that, in my knee-jerk opinion, if one uses, say, math.sin() at all in a module, it will be active in a large number of the callframes. Therefore, the number of tracked objects could easily get quite large. It's a relative thing, of course, and I still think your statistical analysis about cost/benefit is accurate. > Aahz> As a side note, I'm not sure whether some mention should be made > Aahz> that TRACK_OBJECT specifically does not work any differently from > Aahz> current Python when it comes to threads. That is, if it's a > Aahz> shared variable of any sort, you need a mutex if you're going to > Aahz> perform multiple operations on it. > > That's true. If a global object is shared among multiple threads, access to > it must still be protected. I haven't added anything to the mix in that > regard. I'll add a short section on threads. > > Hmm... What about this (dumb) code? > > l = [] > lock = threading.Lock() > ... > def fill_l(): > for i in range(1000): > lock.acquire() > l.append(math.sin(i)) > lock.release() > ... > def consume_l(): > while 1: > lock.acquire() > if l: > elt = l.pop() > lock.release() > fiddle(elt) > > It's not clear from a static analysis of the code what the lock is > protecting. (You can't tell at compile-time that threads are even involved > can you?) Would or should it affect attempts to track "l.append" or > "math.sin" in the fill_l function? Oooooo. Ouch. No, on third thought, it still doesn't matter. If someone is playing games with module globals and not using locks, that code is broken in current Python. We don't need your track() and untrack() functions, but that should be made clear in the PEP, I think. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From fdrake@acm.org Tue Aug 14 17:28:47 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 14 Aug 2001 12:28:47 -0400 (EDT) Subject: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker) In-Reply-To: <15225.18640.291156.850128@slothrop.digicool.com> References: <3B79401C.82D51FBA@metaslash.com> <15225.18640.291156.850128@slothrop.digicool.com> Message-ID: <15225.20927.158390.711237@cj42289-a.reston1.va.home.com> Jeremy Hylton writes: > I'm on the hook for AST/compiler documentation, which I plan to work > on this week. I will probably have more time for it later in the week > than I will today or tomorrow. So between standing over your shoulder to make sure this gets done, and watching over Barry's shoulder to make sure the smtpd module gets documented, I'm going to get a fair bit of exercise this week... well, I guess some people would consider that a good thing. ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From barry@zope.com Tue Aug 14 18:03:45 2001 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 14 Aug 2001 13:03:45 -0400 Subject: [Python-Dev] Python 2.2 Release Schedule Message-ID: <15225.23025.251404.251797@yyz.digicool.com> Here is PEP 251, the Python 2.2 release schedule, revised after our Pythonlabs meeting of yesterday. Please note the following highlights: - Final release is now scheduled for 19-Dec-2001 - We plan the second 2.2 alpha for 22-Aug-2001 - We plan to release a third alpha. As always, the PEP is available online at http://python.sourceforge.net/peps/pep-0251.html Enjoy, -Barry -------------------- snip snip -------------------- PEP: 251 Title: Python 2.2 Release Schedule Version: $Revision: 1.4 $ Author: guido@python.org (Guido van Rossum), barry@zope.com (Barry A. Warsaw) Status: Incomplete Type: Informational Created: 17-Apr-2001 Python-Version: 2.2 Post-History: 14-Aug-2001 Abstract This document describes the post-Python 2.1 development and release schedule. The schedule primarily concerns itself with PEP-sized items. Small bug fixes and changes will occur up until the first beta release. NOTE: the schedule below and the list of features under consideration are all subject to change! If the energy in the community changes or the feedback on a PEP is particularly positive or negative, this may affect decisions. Release Schedule Tentative future release dates. Note that we've slipped this compared to the schedule posted around the release of 2.2a1. 19-Dec-2001: 2.2 (final release) 12-Dec-2001: 2.2c1 (release candidate) 14-Nov-2001: 2.2b2 10-Oct-2001: 2.2b1 19-Sep-2001: 2.2a3 (new! a third alpha) 22-Aug-2001: 2.2a2 18-Jul-2001: 2.2a1 Release Manager Barry Warsaw will take over as the release manager. Guido and Barry will release 2.2a2 together, after that Barry will be responsible for releases. Release Mechanics We'd like to experiment with a new mechanism for releases: a week before every alpha, beta or other release, we'll fork off a branch which will become the release; changes to the branch will have to be approved before they can be checked in. This is how some other large projects (e.g. Mozilla) work, and we hope it will help reduce the number of bugs introduced in releases at the last minute. Planned features for 2.2 The following features are already checked in on the head revision (for a more detailed account, see Misc/NEWS): - iterators (PEP 234) - generators (PEP 255) - division (PEP 238) - unification of types and classes (PEP 252, PEP 253) Work on the class/type unification is still ongoing. The following features are under consideration: - unifying long ints and plain ints (PEP 237) There needs to be more discussion of each of these before we can decide. Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil End: From guido@python.org Tue Aug 14 19:42:42 2001 From: guido@python.org (Guido van Rossum) Date: Tue, 14 Aug 2001 14:42:42 -0400 Subject: [Python-Dev] Revised PEP 237 - Unifying Long Integers and Integers Message-ID: <200108141842.f7EIggo10895@odiug.digicool.com> Please comment on the completely revised PEP 237: http://python.sourceforge.net/peps/pep-0237.html I'm proposing an implementation plan that keeps int and long as separate types, but otherwise will totally remove the differences. I'm also proposing a transition plan that does one easy thing in 2.2 (change ops that raise OverflowError to return a long result instead), and puts off addressing the more severe incompatibilities to 2.3. This is mostly because I don't see how we can make time to design and implement the necessary future statements or other compatibility measures in time. --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Tue Aug 14 19:53:40 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Tue, 14 Aug 2001 14:53:40 -0400 (EDT) Subject: [Python-Dev] AST mining (was Re: Direction of PyChecker) In-Reply-To: References: <15225.17341.690478.601359@slothrop.digicool.com> Message-ID: <15225.29620.135166.848323@slothrop.digicool.com> >>>>> "TP" == Tim Peters writes: TP> [Jeremy Hylton] >> ... I wouldn't wait for people to argue about the right AST >> either. Use the one Greg and Bill came up with for p2c. It's in >> Tools/compiler in the Python distribution. It's much simpler >> than the parse tree produced by the parser module. And, as far >> as I know, no one is advocating a different AST. >> >> It doesn't have a query language, but neither does tokenize.py >> . TP> The likely problem I see is documentation: exactly what is this TP> AST, and how does one use the compiler package? As I already noted in this thread, the documentation problem will not survive the week. Jeremy From skip@pobox.com (Skip Montanaro) Tue Aug 14 20:03:40 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Tue, 14 Aug 2001 14:03:40 -0500 Subject: [Python-Dev] Re: Revised PEP 237 - Unifying Long Integers and Integers In-Reply-To: References: Message-ID: <15225.30220.37379.978290@beluga.mojam.com> Guido> I'm proposing an implementation plan that keeps int and long as Guido> separate types, but otherwise will totally remove the Guido> differences. ... Guido> Please mail comments to me or to python-dev@python.org. Looks reasonable to me. I don't generally program anywhere near the edges of current short integer arithmentic except when writing little demo functions like fib or fact. I wonder though, if some programmers might like the option of telling integer arithmentic raise OverflowError instead of coercing results to long ints. If you're expecting to be playing with numbers near zero and you start silently getting longs instead, it might be nice to see an exception instead of having your program wander off into the weeds (probably somewhat slowly). Just a thought. Skip From guido@python.org Tue Aug 14 20:13:40 2001 From: guido@python.org (Guido van Rossum) Date: Tue, 14 Aug 2001 15:13:40 -0400 Subject: [Python-Dev] Re: Revised PEP 237 - Unifying Long Integers and Integers In-Reply-To: Your message of "Tue, 14 Aug 2001 14:03:40 CDT." <15225.30220.37379.978290@beluga.mojam.com> References: <15225.30220.37379.978290@beluga.mojam.com> Message-ID: <200108141913.f7EJDeq11097@odiug.digicool.com> > Looks reasonable to me. I don't generally program anywhere near the > edges of current short integer arithmentic except when writing > little demo functions like fib or fact. I wonder though, if some > programmers might like the option of telling integer arithmentic > raise OverflowError instead of coercing results to long ints. You can do this by enabling the warning-on-overflow command line option (not yet specified) and then using -Werr to turn the warning into an error. --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@loewis.home.cs.tu-berlin.de Tue Aug 14 19:54:03 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Tue, 14 Aug 2001 20:54:03 +0200 Subject: [Python-Dev] Any NeXT users out there? In-Reply-To: <20010814124304.01511303181@snelboot.oratrix.nl> (message from Jack Jansen on Tue, 14 Aug 2001 14:43:03 +0200) References: <20010814124304.01511303181@snelboot.oratrix.nl> Message-ID: <200108141854.f7EIs3u02092@mira.informatik.hu-berlin.de> > Sharing is the answer. ... > This is probably the crux of your reasoning, and I think I > disagree. For one, MacOSX (ok, OpenStep is where they got it from) > is the first unix-based system that I'm aware of that seems to have > a solution to the versioning problem. I'm a bit sceptical to any claim that the software versioning problem is "solved"; experience tells me it isn't. If you allow me to make some half-serious predictions out of the blue, I'd say: - The API will change more often than you'ld like, so the newer version cannot replace the older one in a compatible way. - every potential application that could make use of the Python framework will come with its own copy of Python, just to reduce the hassles for the users of the application to get a separate copy of Python, and to allow them to completely uninstall everything by deleting a single directory. This, specifically, includes Zope, which by tradition always comes with its own Python installation (with extension modules and everything). - whenever the framework really is shared across applications, people will find that the application that did not bring the framework will break in subtle ways; authors of the application will stop using the framework in their next release. So the only application that really ever uses the framework will be the interpreter proper. That said, I see that your patch removes a lot of old next stuff, and replaces it with something that at least functions as designed, so I'm +0 on this patch. I may be wrong with my predictions, and the only way to prove me wrong is to use this patch, so just go ahead. Regards, Martin From mwh@python.net Tue Aug 14 22:35:15 2001 From: mwh@python.net (Michael Hudson) Date: 14 Aug 2001 17:35:15 -0400 Subject: [Python-Dev] PEP 264 In-Reply-To: Michael Hudson's message of "08 Aug 2001 17:54:50 -0400" References: <2m8zgudxp1.fsf@starship.python.net> Message-ID: <2mitfq8gvg.fsf_-_@starship.python.net> I *think* I'm done coding for implementing PEP 264; patch at: http://sourceforge.net/tracker/?func=detail&atid=305470&aid=449043&group_id=5470 I've also done a first draft of docs. The reason for mentioning this is I'm off on holiday tomorrow, so if you want changes before the proposed date for 2.2a2, you've got aobut 12 hours to tell me about it. Some tests might get attached to the patch later this evening. In related news, the material on nested scopes in the appendix of the refman should be moved out of the appendix. Someone who's not typing down a modem can do that. Cheers, M. -- The ultimate laziness is not using Perl. That saves you so much work you wouldn't believe it if you had never tried it. -- Erik Naggum, comp.lang.lisp From James_Althoff@i2.com Tue Aug 14 22:37:11 2001 From: James_Althoff@i2.com (James_Althoff@i2.com) Date: Tue, 14 Aug 2001 14:37:11 -0700 Subject: [Python-Dev] __class__ assignment in 2.2a1 Message-ID: The following seems like it could use a more helpful error message. Presumably it fails because you can only set the value of __class__ to that of another class object whose __dynamic__ attribute == 1? C:\>python Python 2.2a1 (#21, Jul 18 2001, 04:25:46) [MSC 32 bit (Intel)] on win32 Type "copyright", "credits" or "license" for more information. >>> class C(object): pass ... >>> class D: pass ... >>> class E: pass ... >>> d = D() >>> d.__class__ = E >>> d.__class__ = C Traceback (most recent call last): File "", line 1, in ? TypeError: __class__ must be set to a class >>> Jim From fdrake@acm.org Tue Aug 14 22:52:28 2001 From: fdrake@acm.org (Fred L. Drake) Date: Tue, 14 Aug 2001 17:52:28 -0400 (EDT) Subject: [Python-Dev] [development doc updates] Message-ID: <20010814215228.696DA28845@cj42289-a.reston1.va.home.com> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ A wide range of mostly small fixes and updates. Some new C API functions are documented. Examples are given for how to mark up tables in the "Documenting Python" guide. The language reference contains more information about support // and true division. From neal@metaslash.com Tue Aug 14 23:15:11 2001 From: neal@metaslash.com (Neal Norwitz) Date: Tue, 14 Aug 2001 18:15:11 -0400 Subject: [Python-Dev] Module docstrings patches Message-ID: <3B79A2EF.F5207F26@metaslash.com> I submitted the following to the patch manager at SF. 450979 Add module docstring - imputil 450980 Add module docstring - sre*, re 450981 Add module docstring - xmlrpc Should the pre module be marked as deprecated? There's still a comment in re that pre "will be removed in 2.0 final." Neal From James_Althoff@i2.com Tue Aug 14 23:23:22 2001 From: James_Althoff@i2.com (James_Althoff@i2.com) Date: Tue, 14 Aug 2001 15:23:22 -0700 Subject: [Python-Dev] __dynamic__ attribute Message-ID: Given that __dynamic__ will be part of Python in 2.2 . . . Is it worth considering the possibility of making it more granular (somehow controlling separately the modification of attribute values versus the addition or deletion of attributes, perhaps)? Currently it controls the changing of a class object in all ways (if set to 1 it makes the class object immutable). It seems to me there is a significant difference in general usage between: >>> class A(object): ... __dynamic__ = 1 ... count = 0 ... def incr(myclass): myclass.count += 1 ... incr = classmethod(incr) ... >>> A.count 0 >>> A.incr() >>> A.count 1 >>> which is a pretty common idiom in Java and C++ and other more dynamic things like adding/deleting attributes (data and methods) on the fly. Jim From James_Althoff@i2.com Tue Aug 14 23:36:52 2001 From: James_Althoff@i2.com (James_Althoff@i2.com) Date: Tue, 14 Aug 2001 15:36:52 -0700 Subject: [Python-Dev] re: __class__ assignment in 2.2a1 Message-ID: I guess the rule is that you can only assign __class__ for classic class instances and the new value has to be a classic class also (ignoring C-written metaclasses, etc.). So __dynamic__ doesn't come into play. Still seems like the error message is a bit confusing. >>>>>>>>>>>>>>>>>>>. The following seems like it could use a more helpful error message. Presumably it fails because you can only set the value of __class__ to that of another class object whose __dynamic__ attribute == 1? C:\>python Python 2.2a1 (#21, Jul 18 2001, 04:25:46) [MSC 32 bit (Intel)] on win32 Type "copyright", "credits" or "license" for more information. >>> class C(object): pass ... >>> class D: pass ... >>> class E: pass ... >>> d = D() >>> d.__class__ = E >>> d.__class__ = C Traceback (most recent call last): File "", line 1, in ? TypeError: __class__ must be set to a class >>> Jim From jeremy@zope.com Tue Aug 14 23:42:42 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Tue, 14 Aug 2001 18:42:42 -0400 (EDT) Subject: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker) In-Reply-To: <15225.20927.158390.711237@cj42289-a.reston1.va.home.com> References: <3B79401C.82D51FBA@metaslash.com> <15225.18640.291156.850128@slothrop.digicool.com> <15225.20927.158390.711237@cj42289-a.reston1.va.home.com> Message-ID: <15225.43362.623842.699456@slothrop.digicool.com> A first and quite incomplete draft of docs for the compiler package has been checked in Tools/compiler/doc. The HTML version is currently available from http://www.python.org/~jeremy/compiler/ Jeremy From barry@zope.com Tue Aug 14 23:48:57 2001 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 14 Aug 2001 18:48:57 -0400 Subject: [Python-Dev] PEP comments, bugs, and patches References: <200108080713.JAA21365@pandora.informatik.hu-berlin.de> Message-ID: <15225.43737.762160.158737@anthem.wooz.org> >>>>> "MvL" == Martin von Loewis writes: MvL> I just found that PEP 1 is confusing in its instructions of MvL> how to submit feedback on a PEP. I've just added some clarification about this to PEP 1. -Barry From barry@zope.com Wed Aug 15 01:08:56 2001 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 14 Aug 2001 20:08:56 -0400 Subject: [Python-Dev] Proposed PEP: Optimizing Global Variable and Attribute Access References: <15224.48600.621401.101492@beluga.mojam.com> Message-ID: <15225.48536.32259.429582@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> Here's a first stab at a PEP for optimizing global variable SM> and attribute access. Barry, when you get a chance, could you SM> assign it a number and let me know about any PEP-related SM> formatting mistakes I made? Done. It's PEP 266, but I had to shorten the title. It was too long even with the new limit <44 wink> (give 'em an inch... :). Other than that you did a fine job for a virgin PEP author! I made a few other edits, but mostly so I could piss on the hydrant. :) SM> (It would be nice if PEP 1 referenced a piece of boilerplate SM> people could use. While the PEP Style section lists the SM> headers, I wasn't sure what the Version and Last-Modified SM> headers were actually supposed to look like. Also, it wasn't SM> clear what I was supposed to put in the Post-History: header.) Both excellent suggestions. I've added PEP 9, which can be used as a template for both Informational and Standards Track PEPs. I've often found I wanted something I could just copy-and-edit to get started quickly on a new PEP. I've clarified a few things in PEP 1. Thanks! -Barry From skip@pobox.com (Skip Montanaro) Wed Aug 15 01:19:53 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Tue, 14 Aug 2001 19:19:53 -0500 Subject: [Python-Dev] Proposed PEP: Optimizing Global Variable and Attribute Access In-Reply-To: <15225.48536.32259.429582@anthem.wooz.org> References: <15224.48600.621401.101492@beluga.mojam.com> <15225.48536.32259.429582@anthem.wooz.org> Message-ID: <15225.49193.119814.964280@beluga.mojam.com> SM> Barry, when you get a chance, could you assign it a number and let SM> me know about any PEP-related formatting mistakes I made? BAW> Done. It's PEP 266, but I had to shorten the title. It was too BAW> long even with the new limit <44 wink> Hmmm... Wasn't aware there was a limit. I guess that's why you're the PEP wizard Barry! Thanks, Skip From gward@python.net Wed Aug 15 01:48:55 2001 From: gward@python.net (Greg Ward) Date: Tue, 14 Aug 2001 20:48:55 -0400 Subject: [Python-Dev] New regrtest.py gimmick In-Reply-To: ; from tim.one@home.com on Sat, Aug 11, 2001 at 09:24:15PM -0400 References: Message-ID: <20010814204855.A3097@gerg.ca> On 11 August 2001, Tim Peters said: > For you it will print something like: > > ... > MMM tests OK. > NN tests skipped: XXX YYY ... > Ask someone to teach regrtest.py about which tests are > expected to get skipped on [sys.platform]. > > > You do that by changing the new _ExpectedSkips.__init__ near the bottom of > regrtest.py. > > If sys.platform isn't enough to determine the set of skipped tests, then > sys.platform isn't particularly useful <0.9 wink>. Note that distutils.util.get_platform() is, for POSIX platforms at least, sys.platform on steroids: def get_platform (): """Return a string that identifies the current platform. This is used mainly to distinguish platform-specific build directories and platform-specific built distributions. Typically includes the OS name and version and the architecture (as supplied by 'os.uname()'), although the exact information included depends on the OS; eg. for IRIX the architecture isn't particularly important (IRIX only runs on SGI hardware), but for Linux the kernel version isn't particularly important. Examples of returned values: linux-i586 linux-alpha (?) solaris-2.6-sun4u irix-5.3 irix64-6.2 For non-POSIX platforms, currently just returns 'sys.platform'. """ But then, much of the Distutils is other parts of the standard library on steroids. (Sigh.) Greg -- Greg Ward - Linux geek gward@python.net http://starship.python.net/~gward/ A man without religion is like a fish without a bicycle. From gward@python.net Wed Aug 15 01:56:18 2001 From: gward@python.net (Greg Ward) Date: Tue, 14 Aug 2001 20:56:18 -0400 Subject: [Python-Dev] Parrot -- should life imitate satire? In-Reply-To: <3B77143F.F9158ED1@ActiveState.com>; from paulp@ActiveState.com on Sun, Aug 12, 2001 at 04:41:51PM -0700 References: <3B77143F.F9158ED1@ActiveState.com> Message-ID: <20010814205618.B3097@gerg.ca> On 12 August 2001, Paul Prescod said: > In fact, there are many bits of the C library that Python people would > like to replace or improve but we don't have the resources alone. Maybe > if all of the scripting language groups got together we would be able to > do it. I'm not talking about the larger project of sharing bytecodes and > so forth, but I could see that we probably have very similar needs with > respect to files, memory allocation, networking, event loops, process > handling and maybe threads. And the other scripting languages are > probably not too far off either. Isn't a lot of that addressed by glib, the C library used by GTK+ (and therefore Gnome) applications for all those things that C programs tend to do that aren't GUI-related? And speaking of I/O libraries, whatever happened to sfio? I remember struggling with it and PerlIO back around Perl 5.003/5.004 in a vain attempt to get FastCGI working. sfio seemed at the time like a good idea just didn't catch on, probably because stdio is "good enough" for most people. (But then, "most people" don't implement Perl or Python.) (I also remember that sfio's version number is the year of release -- so much for "release early, release often". It also had a rather idiosyncratic build system -- autoconf? what's that?) Greg -- Greg Ward - Unix bigot gward@python.net http://starship.python.net/~gward/ I'd rather have a bottle in front of me than have to have a frontal lobotomy. From dgoodger@bigfoot.com Wed Aug 15 02:48:44 2001 From: dgoodger@bigfoot.com (David Goodger) Date: Tue, 14 Aug 2001 21:48:44 -0400 Subject: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker) In-Reply-To: <3B79401C.82D51FBA@metaslash.com> Message-ID: Neal Norwitz wrote on 2001-08-14 11:13: > Would you also want default parameter values? Yes. > PyChecker needs all of that, plus additional info. So there is a lot > of overlap. I think the DPS will use a small subset of what PyChecker needs. The only extra information the DPS may need is attribute and additional docstrings, string literals in certain contexts that are not currently recognized as docstrings (see PEP 258). > Pooling efforts would be good. I also don't know anything about > the ASTs/compiler, but am willing to work on it. I look forward to it, and to reading Jeremy's docs. (Go, doc writer!) -- David Goodger dgoodger@bigfoot.com Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From aahz@rahul.net Wed Aug 15 03:15:18 2001 From: aahz@rahul.net (Aahz Maruch) Date: Tue, 14 Aug 2001 19:15:18 -0700 (PDT) Subject: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker) In-Reply-To: <15225.43362.623842.699456@slothrop.digicool.com> from "Jeremy Hylton" at Aug 14, 2001 06:42:42 PM Message-ID: <20010815021519.8371AE8C2@waltz.rahul.net> Jeremy Hylton wrote: > > A first and quite incomplete draft of docs for the compiler package > has been checked in Tools/compiler/doc. The HTML version is currently > available from > http://www.python.org/~jeremy/compiler/ You interested in comments on this incomplete draft? No? Well, here they are, anyway: http://www.python.org/~jeremy/compiler/node4.html What do you mean by "visitor instance"? Is that a fancy name for function (in which case, just say "function"), or is it something else? "Compile the file filename and generated filename.pyc" should be "generate" I think. http://www.python.org/~jeremy/compiler/node8.html getChildNodes() says "This method is like getChildNodes", which should be getChildren Hmmm... I guess that wasn't very much, in the end. I avoided making comments on areas that clearly were going to get more work. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From aahz@rahul.net Wed Aug 15 16:07:29 2001 From: aahz@rahul.net (Aahz Maruch) Date: Wed, 15 Aug 2001 08:07:29 -0700 (PDT) Subject: [Python-Dev] Re: 2.2a1: classmethod() and class attributes In-Reply-To: from "aahz" at Aug 13, 2001 07:31:11 PM Message-ID: <20010815150729.D2A2CE8CA@waltz.rahul.net> Aahz: > It's not particularly convenient for me to try out 2.2a1, so I'm just > going by what's written. One little hole that I don't see an answer to > is what happens when you do this: > > class C(object): > x = 0 > def foo(cls): > cls.x += 1 > foo = classmethod(foo) > > C.foo() Okay, after thinking about this a bit, I think that if the above code requires __dynamic__=1 to work, then the default for __dynamic__ should be changed. I don't find the arguments about changing __class__ to be particularly persuasive, but I think the above code *is* closely related to standard Python idioms that should work by default. +1 on changing __dynamic__ or at least enabling some kind of class variable mutability by default. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From Samuele Pedroni Wed Aug 15 16:17:58 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Wed, 15 Aug 2001 17:17:58 +0200 (MET DST) Subject: [Python-Dev] Re: 2.2a1: classmethod() and class attributes Message-ID: <200108151518.RAA15602@core.inf.ethz.ch> > Aahz: > > It's not particularly convenient for me to try out 2.2a1, so I'm just > > going by what's written. One little hole that I don't see an answer to > > is what happens when you do this: > > > > class C(object): > > x = 0 > > def foo(cls): > > cls.x += 1 > > foo = classmethod(foo) > > > > C.foo() > > Okay, after thinking about this a bit, I think that if the above code > requires __dynamic__=1 to work, then the default for __dynamic__ should > be changed. I don't find the arguments about changing __class__ to be > particularly persuasive, but I think the above code *is* closely related > to standard Python idioms that should work by default. > > +1 on changing __dynamic__ or at least enabling some kind of class > variable mutability by default. > -- My 2cts: I think this is also a matter of how much complex the language is perceived: Before 2.2: everything is dynamic and mutable (pretty general rule, a performance killer but that's another point ;) ) After: - this is mutable and that's not (a more complex picture) vs. - by default everything is mutable ... (advanced user) but you can tune that (for good reasons) A marginal note: I agree with James Althoff that some more granularity about dynamic behavior would be a good thing (but only if the newbie is protected from the burden) regards, Samuele Pedroni. From guido@python.org Wed Aug 15 16:25:53 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 15 Aug 2001 11:25:53 -0400 Subject: [Python-Dev] Re: 2.2a1: classmethod() and class attributes In-Reply-To: Your message of "Wed, 15 Aug 2001 08:07:29 PDT." <20010815150729.D2A2CE8CA@waltz.rahul.net> References: <20010815150729.D2A2CE8CA@waltz.rahul.net> Message-ID: <200108151525.LAA27011@cj20424-a.reston1.va.home.com> > +1 on changing __dynamic__ or at least enabling some kind of class > variable mutability by default. After converting the Tools/compiler package to the new class system, I tend to agree (I already said so on c.l.py in my last post there). Here's a problem, related to performance. The dispatch for special operations (like __iter__) from C uses a C function stored in a corresponding slot (e.g. tp_iter) in the type object. So when class X has an __iter__ method, there needs to be a slot wrapper (a C function that calls the __iter__ object) in the in the tp_iter slot in the type object representing X. For dynamic classes, I must assume that at any time someone can add a special method (like __iter__) to a class. It's hard to set up things so that the dispatch function is set in the type object at the moment C.__iter__ is assigned: it would require a class to keep track of all its subclasses, without keeping those subclasses alive. While I know I can do that using weak references, I don't like having to maintain all that administration. So at the moment, when a class is dynamic, I just stick all dispatch functions in the type object -- the dispatch functions will raise AttributeError when their corresponding method is not found. (This is the same approach used for classic classes, BTW.) This is slower than it should be -- the fully dynamic Tools/compiler package compiles itself about 25% slower this way. If I tweak it to use all static classes (not very hard), it runs at about the same speed as it does with classic classes. I imagine I could make it faster by using __slots__, but I don't know enough about the internals yet to be able to do that. My goal (before I'm happy with making __dynamic__=1 the default) is that dynamic classes should be at least as fast as classic classes. I haven't profiled it yet -- it's possible that there's a cheap hack possible by making more conservative assumptions about __getattr__ alone -- classic classes special-case __getattr__ too.) --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Wed Aug 15 16:30:01 2001 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 15 Aug 2001 11:30:01 -0400 Subject: [Python-Dev] PEP 9, Sample PEP Template Message-ID: <15226.38265.263478.150525@anthem.wooz.org> A number of people have suggested that there be a standard PEP template that makes it easier to get started with writing your own PEP. This is a good thing too, because hopefully it'll make the submissions I see more consistent with less initial editorial requirements (and fewer auto-rejects due to bad form ). To that end, I've written PEP 9, Sample PEP Template, which is available at the following URL, and attached below. http://python.sourceforge.net/peps/pep-0009.html Please take a look and let me know if any of the instructions in it are confusing. PEP 9 isn't a substitute for reading PEP 1, PEP Purpose and Guidelines, but it should be considered an adjunct. feeling-peppy-ly y'rs, -Barry -------------------- snip snip -------------------- PEP: 9 Title: Sample PEP Template Version: $Revision: 1.2 $ Last-Modified: $Date: 2001/08/15 15:28:57 $ Author: barry@zope.com (Barry A. Warsaw) Status: Active Type: Informational Created: 14-Aug-2001 Post-History: Abstract This PEP provides a boilerplate or sample template for creating your own PEPs. This should make it easy for you to conform your own PEPs to the style outlined in PEP 1, PEP Guidelines[1]. Rationale PEP submissions come in a wide variety of forms, not all adhering to the style guidelines set forth in PEP 1. Use this template, in conjunction with the prose guidelines in PEP 1, to ensure that your PEP submission won't get automatically rejected because of form. To use this template you must first decide whether your PEP is going to be Informational or Standards Track. Most PEPs are Standards Track because they propose a new feature for the Python language or standard library. When in doubt, read PEP 1 for details. Once you've decided which type of PEP yours is going to be, follow the directions in the appropriate section below. How to Use This Template for Standard Track PEPs - Make a copy of this file and perform the following edits. - replace the "PEP: 9" header with "PEP: XXX" since you don't yet have a PEP number assignment. - Change the Title: header to the title of your PEP. - Leave the Version: and Last-Modified: headers alone; we'll take care of those when we check your PEP into CVS. - Change the Author: header to include your email address and name. Be sure to follow the format carefully: your email address must appear first, and it should not be contained in angle brackets. Your full name should appear second and it should appear in parentheses. - If there is a mailing list for discussion of your new feature, add a Discussions-To: header right after the Author: header. You should not add a Discussions-To: header if the mailing list to be used is either python-list@python.org or python-dev@python.org, or if discussions should be sent to you directly. - Change the Status: header to "Draft". - Change the Type: header to "Standards Track". - If your feature depends on the acceptance of some other currently in-development PEP, add a Requires: header right after the Type: header. The value should be the PEP number of the PEP yours depends on. Don't add this header if your dependent feature is described in a Final PEP. - Change the Created: header to today's date. Be sure to follow the format carefully: it must be in dd-mmm-yyyy format, where the mmm is the 3 letter month abbreviation. - After the Created: header, add a Python-Version: header and set the value to the next planned version of Python, i.e. the one your new feature will hopefully make its first appearance in. Do not use an alpha or beta release designation here. Thus, if the last version of Python was 2.2 alpha 1 and you're hoping to get your new feature into Python 2.2, set the header to: Python-Version: 2.2 - Leave Post-History: alone for now; you'll add dates to this header after your PEP has been assigned a number and you've posted your PEP to python-list@python.org or python-dev@python.org. - Add a Replaces: header if your PEP describes a feature that will replace a feature described in some Final PEP. Usually you won't need to add this header. It's also unlikely that you'll ever add a Replaced-By: header. - Now write your abstract, rationale, and other content for your PEP, replacing all this gobbledygook with your own text. Be sure to adhere to the PEP 1 style guidelines, specifically on the prohibition of tab characters and the indentation requirements. - Update your References and Copyright section. Usually you'll place your PEP into the public domain, in which case just leave the Copyright section alone. Alternatively, you can use the Open Publication License[2], but public domain is still strongly preferred. - Leave the little Emacs turd at the end of this file alone, including the formfeed character (^L, or \f). - Send your PEP submission to the PEP editor, along with $100k in unmarked pennies. (Just kidding, I wanted to see if you were still awake. :) How to Use This Template for Informational PEPs - Make a copy of this file and perform the following edits. - replace the "PEP: 9" header with "PEP: XXX" since you don't yet have a PEP number assignment. - Change the Title: header to the title of your PEP. - Leave the Version: and Last-Modified: headers alone; we'll take care of those when we check your PEP into CVS. - Change the Author: header to include your email address and name. Be sure to follow the format carefully: your email address must appear first, and it should not be contained in angle brackets. Your full name should appear second and it should appear in parentheses. - If there is a mailing list for discussion of your new feature, add a Discussions-To: header right after the Author: header. You should not add a Discussions-To: header if the mailing list to be used is either python-list@python.org or python-dev@python.org, or if discussions should be sent to you directly. Most Information PEPs don't have a Discussions-To: header. - Change the Status: header to "Active". - Change the Type: header to "Informational". - Change the Created: header to today's date. Be sure to follow the format carefully: it must be in dd-mmm-yyyy format, where the mmm is the 3 letter month abbreviation. - Leave Post-History: alone for now; you'll add dates to this header after your PEP has been assigned a number and you've posted your PEP to python-list@python.org or python-dev@python.org. - Now write your abstract, rationale, and other content for your PEP, replacing all this gobbledygook with your own text. Be sure to adhere to the PEP 1 style guidelines, specifically on the prohibition of tab characters and the indentation requirements. - Update your References and Copyright section. Usually you'll place your PEP into the public domain, in which case just leave the Copyright section alone. Alternatively, you can use the Open Publication License[2], but public domain is still strongly preferred. - Leave the little Emacs turd at the end of this file alone, including the formfeed character (^L, or \f). - Send your PEP submission to the PEP editor, along with $100k in unmarked pennies. (Just kidding, I wanted to see if you were still awake. :) References [1] PEP 1, PEP Purpose and Guidelines http://www.python.org/peps/pep-0001.html [2] http://www.opencontent.org/openpub/ Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil End: From tony@lsl.co.uk Wed Aug 15 16:31:29 2001 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Wed, 15 Aug 2001 16:31:29 +0100 Subject: [Docstring-develop] Re: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker) In-Reply-To: <15225.18640.291156.850128@slothrop.digicool.com> Message-ID: <006d01c1259f$5ade9950$f05aa8c0@lslp7o.int.lsl.co.uk> Hmm. Is everyone on that list of recipients really interested? Anyway, I've spent a happy time (when I should have been earning money - oh well) expanding Jeremy's example, ast-mining.py, to do more things. I've done all of the easy stuff first (makes sense to me). Anyway, it now reports on a lot more stuff, and is probably already useful enough to be "plugged in" to initial DPS/reST usages. Rather than make this message even longer, I've put it at: http://www.tibsnjoan.co.uk/reST/ast-mining.py I've been testing it by running it on itself. Comments, as ever, welcome. Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ "How fleeting are all human passions compared with the massive continuity of ducks." - Dorothy L. Sayers, "Gaudy Night" My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From skip@pobox.com (Skip Montanaro) Wed Aug 15 17:00:52 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 15 Aug 2001 11:00:52 -0500 Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes? Message-ID: <15226.40116.68738.183684@beluga.mojam.com> In thinking about the implementation of TRACK_GLOBAL some more, I'm trying to figure out where object updating is going to take place. Is PyObject_GenericSetAttr the only way C-based objects should set attributes? If so, that would be a central place to provide this functionality. Otherwise, I fear that a lot of C code will have to be twiddled to provide this functionality. I worry especially about third-party extension modules. Accordingly, I took a look at http://www.python.org/doc/ext/dnt-type-methods.html Unfortunately, that section seems to be truncated (in the TeX source as well). I filed a bug report: http://sourceforge.net/tracker/index.php?func=detail&aid=451217&group_id=5470&atid=105470 Skip From jeremy@zope.com Wed Aug 15 17:06:25 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 15 Aug 2001 12:06:25 -0400 (EDT) Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes? In-Reply-To: <15226.40116.68738.183684@beluga.mojam.com> References: <15226.40116.68738.183684@beluga.mojam.com> Message-ID: <15226.40449.521721.265985@slothrop.digicool.com> >>>>> "SM" == Skip Montanaro writes: SM> In thinking about the implementation of TRACK_GLOBAL some more, SM> I'm trying to figure out where object updating is going to take SM> place. Is PyObject_GenericSetAttr the only way C-based objects SM> should set attributes? C code can manipulate __dict__, too, right? It's a pretty common idiom to get a module dictionary and use PyDict_SetItem(). I'm also curious about the new type-class unification stuff. Is it possible to modify a C object via something like a C struct and have that change be visible as a modified attrbute at the Python level? Jeremy From akuchlin@mems-exchange.org Wed Aug 15 17:49:42 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 15 Aug 2001 12:49:42 -0400 Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes? In-Reply-To: <15226.40449.521721.265985@slothrop.digicool.com>; from jeremy@zope.com on Wed, Aug 15, 2001 at 12:06:25PM -0400 References: <15226.40116.68738.183684@beluga.mojam.com> <15226.40449.521721.265985@slothrop.digicool.com> Message-ID: <20010815124942.D15765@ute.cnri.reston.va.us> On Wed, Aug 15, 2001 at 12:06:25PM -0400, Jeremy Hylton wrote: >C code can manipulate __dict__, too, right? It's a pretty common >idiom to get a module dictionary and use PyDict_SetItem(). But most commonly that's only done in the C extension's init*() function, right? --amk From akuchlin@mems-exchange.org Wed Aug 15 17:56:40 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 15 Aug 2001 12:56:40 -0400 Subject: [Python-Dev] New regrtest.py gimmick In-Reply-To: <20010814204855.A3097@gerg.ca>; from gward@python.net on Tue, Aug 14, 2001 at 08:48:55PM -0400 References: <20010814204855.A3097@gerg.ca> Message-ID: <20010815125640.E15765@ute.cnri.reston.va.us> On Tue, Aug 14, 2001 at 08:48:55PM -0400, Greg Ward wrote: >But then, much of the Distutils is other parts of the standard library >on steroids. (Sigh.) It would certainly be worth booting some bits of the Distutils up into the standard library: text_file, some of the functions in util.py, and perhaps fancy_getopt.py. --amk From skip@pobox.com (Skip Montanaro) Wed Aug 15 18:02:00 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 15 Aug 2001 12:02:00 -0500 Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes? In-Reply-To: <15226.40449.521721.265985@slothrop.digicool.com> References: <15226.40116.68738.183684@beluga.mojam.com> <15226.40449.521721.265985@slothrop.digicool.com> Message-ID: <15226.43784.663087.346750@beluga.mojam.com> SM> In thinking about the implementation of TRACK_GLOBAL some more, I'm SM> trying to figure out where object updating is going to take place. SM> Is PyObject_GenericSetAttr the only way C-based objects should set SM> attributes? Jeremy> C code can manipulate __dict__, too, right? It's a pretty Jeremy> common idiom to get a module dictionary and use Jeremy> PyDict_SetItem(). Yeah, I believe so. In theory, I think any code that could modify what LOAD_GLOBAL or LOAD_ATTR instructions might return would have to be changed to signal an update. Jeremy> I'm also curious about the new type-class unification stuff. Is Jeremy> it possible to modify a C object via something like a C struct Jeremy> and have that change be visible as a modified attrbute at the Jeremy> Python level? If a C extension defines a function that changes an exposed attribute I suspect it would have to be changed. I'll try and make this notification as easy as possible for the programmer (a single function call), but it would definitely require extension writers to be pretty careful in updating to a version of Python that contains this functionality. In the stuff I've seen so far this seems to be the most difficult problem to resolve. Skip From fdrake@acm.org Wed Aug 15 17:59:54 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 15 Aug 2001 12:59:54 -0400 (EDT) Subject: [Python-Dev] Re: AST mining (was Re: Direction of PyChecker) In-Reply-To: <20010815021519.8371AE8C2@waltz.rahul.net> References: <15225.43362.623842.699456@slothrop.digicool.com> <20010815021519.8371AE8C2@waltz.rahul.net> Message-ID: <15226.43658.208451.723960@cj42289-a.reston1.va.home.com> Aahz Maruch writes: > What do you mean by "visitor instance"? Is that a fancy name for > function (in which case, just say "function"), or is it something > else? I'll let Jeremy fix this. ;-) > "Compile the file filename and generated filename.pyc" should be > "generate" I think. Fixed in CVS. > http://www.python.org/~jeremy/compiler/node8.html > > getChildNodes() says "This method is like getChildNodes", which > should be getChildren Fixed in CVS. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Wed Aug 15 18:14:10 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 15 Aug 2001 13:14:10 -0400 Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes? In-Reply-To: Your message of "Wed, 15 Aug 2001 12:06:25 EDT." <15226.40449.521721.265985@slothrop.digicool.com> References: <15226.40116.68738.183684@beluga.mojam.com> <15226.40449.521721.265985@slothrop.digicool.com> Message-ID: <200108151714.NAA27351@cj20424-a.reston1.va.home.com> > C code can manipulate __dict__, too, right? It's a pretty common > idiom to get a module dictionary and use PyDict_SetItem(). Yes -- always using the PyModule_GetDict() API. > I'm also curious about the new type-class unification stuff. Is it > possible to modify a C object via something like a C struct and have > that change be visible as a modified attrbute at the Python level? Can you clarify this question? I feel I should be qualified to answer this but I don't understand the question... Maybe an example would help? --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com (Skip Montanaro) Wed Aug 15 18:37:10 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 15 Aug 2001 12:37:10 -0500 Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes? In-Reply-To: <200108151714.NAA27351@cj20424-a.reston1.va.home.com> References: <15226.40116.68738.183684@beluga.mojam.com> <15226.40449.521721.265985@slothrop.digicool.com> <200108151714.NAA27351@cj20424-a.reston1.va.home.com> Message-ID: <15226.45894.43208.544121@beluga.mojam.com> >> I'm also curious about the new type-class unification stuff. Is it >> possible to modify a C object via something like a C struct and have >> that change be visible as a modified attrbute at the Python level? Guido> Can you clarify this question? I feel I should be qualified to Guido> answer this but I don't understand the question... Maybe an Guido> example would help? I'll try to channel Jeremy. Even if I fail, I think the problem I describe is worth noting for PEP 0266. Suppose I have an extension module that declares a new type containing a "foo" slot in the C struct for the type. If I implement a method named "bar", it's obviously free to modify any of the contents in the struct in any way it sees fit. Now, suppose I also have a getattrfunc that will return the contents of the "foo" slot when asked to. There needs not be any setattrfunc or setattrofunc, yet the value in this slot has to be tracked if it might be accessed by a programmer. I suspect this is just one more place a C extension programmer can blow a gaping hole in his foot, but I worry that there's no obvious way to alert the programmer to such breakage if/when they upgrade to a version of Python that tracks globals. Seems like a source of very subtle bugs. Skip From guido@python.org Wed Aug 15 19:10:58 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 15 Aug 2001 14:10:58 -0400 Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes? In-Reply-To: Your message of "Wed, 15 Aug 2001 12:37:10 CDT." <15226.45894.43208.544121@beluga.mojam.com> References: <15226.40116.68738.183684@beluga.mojam.com> <15226.40449.521721.265985@slothrop.digicool.com> <200108151714.NAA27351@cj20424-a.reston1.va.home.com> <15226.45894.43208.544121@beluga.mojam.com> Message-ID: <200108151810.OAA30795@cj20424-a.reston1.va.home.com> [Jeremy] > >> I'm also curious about the new type-class unification stuff. Is it > >> possible to modify a C object via something like a C struct and have > >> that change be visible as a modified attrbute at the Python level? > > Guido> Can you clarify this question? I feel I should be qualified to > Guido> answer this but I don't understand the question... Maybe an > Guido> example would help? [Skip] > I'll try to channel Jeremy. Even if I fail, I think the problem I describe > is worth noting for PEP 0266. > > Suppose I have an extension module that declares a new type containing a > "foo" slot in the C struct for the type. If I implement a method named > "bar", it's obviously free to modify any of the contents in the struct in > any way it sees fit. Now, suppose I also have a getattrfunc that will > return the contents of the "foo" slot when asked to. There needs not be any > setattrfunc or setattrofunc, yet the value in this slot has to be tracked if > it might be accessed by a programmer. > > I suspect this is just one more place a C extension programmer can blow a > gaping hole in his foot, but I worry that there's no obvious way to alert > the programmer to such breakage if/when they upgrade to a version of Python > that tracks globals. Seems like a source of very subtle bugs. Under the new system, the recommended way to make "foo" an attribute of the object is to have an attribute descriptor named foo in the type's __dict__. This descriptor is most likely created when the type was initialized from the tp_members or tp_getset fields in the type object. (I've checked in an example to Modules/xxsubtype.c -- do a cvs update and look for "struct memberlist".) I suppose the globals-tracking mechanism could have enough smarts to look in the type's __dict__ for attribute descriptors; if it finds certain types of descriptors it should assume that these can be changed at any time. --Guido van Rossum (home page: http://www.python.org/~guido/) From MusicMaster" This newsletter is provided to you FREE! To unsubscribe, reply to this email with REMOVE in the Subject. August, 15, 2001 Today's Topics: (Full Stories, See Below) -MADONNA Cancels NJ Concert -Blues Traveler: New Album, New Tour -U2 Plans more US Tour Dates -311 To Headline Following Warped Tour -CONCERT TICKETS Anyone? MADONNA CANCELS NJ CONCERT: Madonna's Friday night (8/3) concert at the Continental Airlines Arena in East Rutherford, N.J., has been canceled, promoter Clear Channel Entertainment announced. According to the artist's management, she is suffering from laryngitis. The canceled show--which will not be rescheduled--was to have been Madonna's seventh sold-out performance in the New York area. The decision not to reschedule immediately drew an angry response from fans posting on her official website . "I think it's absolutely ridiculous that she didn't re-schedule the concert," one fan wrote. "I think it's safe to say that she alienated a lot of her fans for not being willing to move the show to a later date." All Ticketmaster phone and Internet orders will be automatically refunded and credited to the card holder within two weeks, according to a press release. Tickets purchased at Ticketmaster outlets and at the Continental Airlines Arena box office will be refunded at the point of purchase. (Ticketmaster is LiveDaily's parent company.) Madonna's tour is expected to resume in Boston on Aug. 7 and 8. BLUES TRAVELER: NEW ALBUM, NEW TOUR Blues Traveler returns to the tour circuit to promote its new album, "Bridge." The tour is set to begin August 18th. Blues Traveler has revealed the next batch of dates for its summer tour, which kicks off on Aug. 18 in Oneida, Wis. The band will perform in mostly secondary markets including Sedona, Ariz., Buffalo, N.Y., and Portsmouth, Va. The Pontiac, Mich., show on Aug. 31 is part of the free Labor Day festival Arts, Beats and Eats. On Aug. 25, Blues Traveler will perform on "Weekend Today" on NBC. More October tour dates will be announced soon, according to the band's spokesperson. The second leg of Blues Traveler's summer jaunt coincides with the release of "Back in the Day," the second single from its fifth studio album, "Bridge" (A&M Records/Interscope). GO Here For More Info: http://www.livedaily.citysearch.com/news/printable.html?id=3478 U2 Plans More US Tour Dates: U2, which wraps up a tour of Europe later this month, plans to return to North America later this year to play 25 additional concert dates, the Times of London reported this week. Though there have been persistent rumors that more North American dates will be added, no dates have been confirmed by the band or by concert organizers. The Times of London reported that U2's Elevation tour has grossed $142 million from worldwide ticket sales, and the that the new dates are expected to gross an additional $33 million. According to Pollstar, U2's March-June run through North America grossed $69 million. If estimates prove accurate--and if U2 does add the additional dates--U2's Elevation Tour could rank as the third-highest grossing North American tour of all time, behind 1994 outings by the Rolling Stones ($121 million) and Pink Floyd ($103.5 million), according to Pollstar. 311 To Headline Following Warped Tour As its stint on the Warped Tour winds down, 311 has revealed the dates for a late summer headlining tour. Warped ends on Sunday (8/12) in Pontiac, Mich., after which 311 heads to Europe for a handful of shows. The band returns to the United States in late August and begins its stateside tour on Sept. 8 in San Diego. The last date, according to its official website , is a Halloween show in Los Angeles. The group will then spend the early part of November in Japan. All of the dates are in support of 311's latest record, "From Chaos," 311's first for Volcano/Zomba. CONCERT TICKETS Anyone? We found a great place to get your next pair of tickets to your favorite event! We have dealt with them many times and they have really prooved a great source for tickets to concerts, sports, theater; All Nationwide. Check them out at www.tixx4u.com That's about it for now, see ya next time! From skip@pobox.com (Skip Montanaro) Wed Aug 15 19:24:41 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 15 Aug 2001 13:24:41 -0500 Subject: [Python-Dev] Just Updated! In-Reply-To: References: Message-ID: <15226.48745.552793.340591@beluga.mojam.com> >> This newsletter is provided to you FREE! >> To unsubscribe, reply to this email with REMOVE in the Subject. >> August, 15, 2001 >> Today's Topics: >> (Full Stories, See Below) >> -MADONNA Cancels NJ Concert >> -Blues Traveler: New Album, New Tour >> -U2 Plans more US Tour Dates >> -311 To Headline Following Warped Tour >> -CONCERT TICKETS Anyone? I wonder how these people knew to contact the Mojam and Musi-Cal webmaster at this address? Pretty smart, I'd say... ;-) -- Skip Montanaro (skip@pobox.com) http://www.mojam.com/ http://www.musi-cal.com/ From aahz@rahul.net Wed Aug 15 19:46:28 2001 From: aahz@rahul.net (Aahz Maruch) Date: Wed, 15 Aug 2001 11:46:28 -0700 (PDT) Subject: [Python-Dev] Just Updated! In-Reply-To: from "MusicMaster" at Aug 15, 2001 02:15:19 PM Message-ID: <20010815184628.9DE18E8C2@waltz.rahul.net> Well, now that Just has been updated, who's next? -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From fdrake@acm.org Wed Aug 15 19:47:38 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 15 Aug 2001 14:47:38 -0400 (EDT) Subject: [Python-Dev] Just Updated! In-Reply-To: <20010815184628.9DE18E8C2@waltz.rahul.net> References: <20010815184628.9DE18E8C2@waltz.rahul.net> Message-ID: <15226.50122.780207.399172@cj42289-a.reston1.va.home.com> Aahz Maruch writes: > Well, now that Just has been updated, who's next? Better check your physical security devices! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Wed Aug 15 20:41:36 2001 From: fdrake@acm.org (Fred L. Drake) Date: Wed, 15 Aug 2001 15:41:36 -0400 (EDT) Subject: [Python-Dev] [development doc updates] Message-ID: <20010815194136.8BEE328846@beowolf.digicool.com> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ More material on defining new types in C in "Extending & Embedding the Python Interpreter": http://python.sourceforge.net/devel-docs/ext/defining-new-types.html From mwh@python.net Wed Aug 15 21:55:46 2001 From: mwh@python.net (Michael Hudson) Date: 15 Aug 2001 16:55:46 -0400 Subject: [Python-Dev] PyObject_SetAttr - is it the only way types should set attributes? In-Reply-To: Skip Montanaro's message of "Wed, 15 Aug 2001 11:00:52 -0500" References: <15226.40116.68738.183684@beluga.mojam.com> Message-ID: <2m8zgl59gt.fsf@starship.python.net> Skip Montanaro writes: > I worry especially about third-party extension modules. > Accordingly, I took a look at > > http://www.python.org/doc/ext/dnt-type-methods.html > > Unfortunately, that section seems to be truncated (in the TeX source as > well). That would be because I ran out of steam at that point - a few months ago there were no docs on this at all! I'm glad someone's fleshed it out a bit more - something else I haven't gotten round to this summer. Cheers, M. -- My hat is lined with tinfoil for protection in the unlikely event that the droid gets his PowerPoint presentation working. -- Alan W. Frame, alt.sysadmin.recovery From tim.one@home.com Wed Aug 15 22:24:20 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 15 Aug 2001 17:24:20 -0400 Subject: [Python-Dev] PEP 264 In-Reply-To: <2mitfq8gvg.fsf_-_@starship.python.net> Message-ID: [Michael Hudson] > I *think* I'm done coding for implementing PEP 264; patch at: > > http://sourceforge.net/tracker/?func=detail&atid=305470&aid=449043 > &group_id=5470 > > I've also done a first draft of docs. > > The reason for mentioning this is I'm off on holiday tomorrow, so if > you want changes before the proposed date for 2.2a2, you've got aobut > 12 hours to tell me about it. Thanks, Michael! I was out sick yesterday, but intend to get on this tonight. You should enjoy your vacation! If it's a disaster, I'll probably villify you on python-dev, but will fix it anyway . > ... > In related news, the material on nested scopes in the appendix of the > refman should be moved out of the appendix. Someone who's not typing > down a modem can do that. I'm trusting that somebody else understood what that part meant. From jeremy@zope.com Wed Aug 15 23:03:09 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 15 Aug 2001 18:03:09 -0400 (EDT) Subject: [Python-Dev] PEP 264 In-Reply-To: References: <2mitfq8gvg.fsf_-_@starship.python.net> Message-ID: <15226.61853.662980.474359@slothrop.digicool.com> >>>>> "TP" == Tim Peters writes: >> ... In related news, the material on nested scopes in the >> appendix of the refman should be moved out of the appendix. >> Someone who's not typing down a modem can do that. TP> I'm trusting that somebody else understood what that part meant. They did, so you can ignore this message. I added the appendix, after first trying to revise the ref manual. (This was before nested scopes became optional for 2.1.) I'll try to dig up my original effort, regardless I think I am on the hook to fix the ref manual to explain scope rules clearly. Jeremy From paulp@ActiveState.com Tue Aug 14 07:50:28 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Mon, 13 Aug 2001 23:50:28 -0700 Subject: [Python-Dev] Free threading References: Message-ID: <3B78CA34.9A4E1765@ActiveState.com> Tim Peters wrote: > >... > > You can't get massive scalability using OS threads, unless the OS is an > oddball specifically designed for that. Worming around the bloat of Windows > and Linux (etc) OS threads is Stackless Python's natural domain (simulated > threads). The catch, of course, is they that don't run in parallel! Or > maybe by "massive" you mean a few dozen? I worked in the "massively > parallel supercomputer" biz, so I'm thinking more on the order of a few > thousands of dozens. I bet you're not. I am thinking about getting as many threads as there are processors to run truly in parallel. > > ... > > Why do people use mod_python, fcgi, mod_snake, mod_php and all of those > > other things instead of pure CGI? > > Beats me why people bother with web programming at all . Yeah, that describes alot of us in the Python world. I think that explains the emergence of PHP. <0.1 wink> Even Zope is really an OO database that made a few concessions to the fashionable object delivery mechanisms of the day. >... > IIRC, Greg's fabled free-threading version of Python took a speed hit of > about a factor of 2 (for a program using only 1 thread, compared to that > same program without the free-threading patches). The Perl guys considered this unacceptable and I can kind of see their point. You have two processors but you get roughly the same performance as one? >.. > Well, the idea that threads don't share information is foreign to every > intense belief about the world *I've* ever been paid to adopt , and > I'm not enough of a Windows geek to believe "threads are always the answer" > even so. A pool of worker processes using OS-specific IPC as needed works > great in my real-life experience, and if information sharing is rare, works > especially great because it's not *fighting* the OS and C libraries tooth > and nail. I really don't see this as fighting anything. There are thousands of C-coded libraries out there that can be used in multiple C threads without sharing information between them. Some of those libraries happen to be interpreters (e.g. PHP, VBScript) that are much more popular in embedded server applications than Python. Maybe just a coincidence but I think that there are a host of reasons and this may be one of them. > What of your hypothetical user earlier, who "isn't going to be happy with > the price of ... using IPC for information sharing either"? That is, in > what sense do isolated threads leave her happy about her information sharing > needs? You share information (if at all) through the threaded host. The host handles serialization of access to shared information. E.g. Apache wants to tell you about the cookies that go with a request. That shouldn't be copied from one process to another. It should be "just available." Sendmail wants to tell you about the message contents of the message you are filtering. Your SQL database with Python stored procedures has huge shared state but the Python procedures would never share state themselves. Your video game has a set of objects available representing physical objects. Python interpreters would never talk to each other about them directly -- they would always move the objects around the objectspace and the engine routes the information between the interpreters. There is no "IPC" between Python programs. All communication is between the host environment and the interpreter. If the host chooses to share the information with other interpreters then it is the host's job to manage serialization and it is the host's fault if performance lags. Anyhow, these servers exist and use threads for parallel processing. Either we play nicely with them as a good citizen or you don't. We bend over backwards to be a good embedded citizen on many other issues. We could argue that Apache 2 made a wrong turn in moving from processes towards threads but I think that's a little like announcing the falling of the tree in the wood. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From paulp@ActiveState.com Tue Aug 14 09:08:17 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Tue, 14 Aug 2001 01:08:17 -0700 Subject: [Python-Dev] More on Perl threads Message-ID: <3B78DC71.C1683936@ActiveState.com> I've learned a bit more about Perl Ithreads. There is a lot that isn't documented yet so I had to go to the developers. The idea is that there is a function called "share" that you use to share a value. Once you do so, the Perl interpreter builds a transparent proxy that ensures that it is handled in a thread-safe way. Thus you incur a performance price for only the variables you share. On the other hand, that price sounds much higher than the price of sharing things using the global interpreter lock (both in performance and coding complexity). Therefore it is not a good replacement for the GIL but perhaps a good alternative for those who need multiprocessor performance. According to Arthur Bergman: "One could say that we do "proxying" internally, but that is totally transparent to the perl programmer." sub foo { my $object = {}; bless($object, "Class"); share($object); return $object; } " Here's an interesting late-night idea of mine...could shareability be contagious? If you followed a reference from a shared object to an "ordinary" object in a different thread, the interpreter could build the proxy for you. Actually I think that this is how XPCOM proxies work. The more you work with stuff that came from another thread, the more your program slows down. But if you copy data when it makes sense, you get back to indpendent interpreter speeds until you need to share data again. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From dan@sidhe.org Wed Aug 15 23:52:47 2001 From: dan@sidhe.org (Dan Sugalski) Date: Wed, 15 Aug 2001 18:52:47 -0400 Subject: [Python-Dev] More on Perl threads In-Reply-To: <3B78DC71.C1683936@ActiveState.com> Message-ID: <5.1.0.14.0.20010815184654.02269300@tuatha.sidhe.org> At 01:08 AM 8/14/2001 -0700, Paul Prescod wrote: >The idea is that there is a function called "share" that you use to >share a value. Once you do so, the Perl interpreter builds a transparent >proxy that ensures that it is handled in a thread-safe way. Thus you >incur a performance price for only the variables you share. On the other >hand, that price sounds much higher than the price of sharing things >using the global interpreter lock (both in performance and coding >complexity). Therefore it is not a good replacement for the GIL but >perhaps a good alternative for those who need multiprocessor >performance. If your interpreters are otherwise disconnected, the proxy lock performance will be better than a GIL, if for no other reason then you won't have to get locks nearly so often. Other than that there's not much overhead--at most a dereference and a mutex get and release. That's not bad. It definitely makes a difference on SMP systems to be able to run multiple threads simultaneously. >Here's an interesting late-night idea of mine...could shareability be >contagious? If you followed a reference from a shared object to an >"ordinary" object in a different thread, the interpreter could build the >proxy for you. Yeah, but you then run the risk of the sharing contagion spreading pretty wildly. That risk is probably lower in Python than it is in Perl, though. (Store a reference to an interpreter's global symbol table into a shared variable and watch the sharing run rampant) Been there, done that, backed slowly away with my hands in full view. :) Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From dan@sidhe.org Wed Aug 15 23:59:47 2001 From: dan@sidhe.org (Dan Sugalski) Date: Wed, 15 Aug 2001 18:59:47 -0400 Subject: [Python-Dev] Free threading In-Reply-To: <3B78CA34.9A4E1765@ActiveState.com> References: Message-ID: <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org> At 11:50 PM 8/13/2001 -0700, Paul Prescod wrote: >Tim Peters wrote: > >... > > IIRC, Greg's fabled free-threading version of Python took a speed hit of > > about a factor of 2 (for a program using only 1 thread, compared to that > > same program without the free-threading patches). > >The Perl guys considered this unacceptable and I can kind of see their >point. You have two processors but you get roughly the same performance >as one? The perl speed hit's as much because of half-baked code as anything else. Changing the two instances of pthread_get_specific to pthread_get_specific_unchecked_np (which just bypasses the validation of the TSD key on Tru64 and VMS) sped up threaded perl by around 3-5% The original, pthread-like threading model in perl was never finished for a variety of reasons. Do use many of the things it exposed as object lessons, but speed isn't one of them. Or maybe it is. FWIW, here's a data point. When I took perl and wrapped all variable access in a thread-safe way (basically we got a lock before accessing a variable and let go when we were done) we took a 40% speed hit. Now, this was on OpenVMS where the mutexes are about 3-5x as expensive as on Tru64 on the same box, but it was a significant cost. I racked up a whole list of "Things to Not Do With Threads" when hacking the original perl thread model. (The first of which is "wedge them into an interpreter that wasn't written with threads in mind..." :) Battle scars are viewable on request. Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From skip@pobox.com (Skip Montanaro) Thu Aug 16 01:16:28 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 15 Aug 2001 19:16:28 -0500 Subject: [Python-Dev] PEP 266: Catching code that is not safe for tracking Message-ID: <15227.4316.969063.877269@beluga.mojam.com> While I was out for my semi-occasion-once-every-other-wednesday run this afternoon, I thought about the problem of identifying objects that are not safe for the tracking proposal in PEP 266. My worry is that an extension module that works in Python 2.2 will break in subtle ways in Python 2.3 if my tracking idea is enabled, because the module's code is not doing anything to notify the rest of the world when an attribute binding changes. I came up with the following possible solution. Add a new slot to type objects that the object's author must explicitly set to Py_TRACKING_SAFE (1?). It would default to Py_TRACKING_UNSAFE (0?). This value could be checked by _PyObject_TrackName and a warning raised if a program tries to track an unsafe object. Would that work? Skip From guido@python.org Thu Aug 16 07:46:58 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 16 Aug 2001 02:46:58 -0400 Subject: [Python-Dev] PEP 266: Catching code that is not safe for tracking In-Reply-To: Your message of "Wed, 15 Aug 2001 19:16:28 CDT." <15227.4316.969063.877269@beluga.mojam.com> References: <15227.4316.969063.877269@beluga.mojam.com> Message-ID: <200108160646.CAA08652@cj20424-a.reston1.va.home.com> > While I was out for my semi-occasion-once-every-other-wednesday run this > afternoon, I thought about the problem of identifying objects that are not > safe for the tracking proposal in PEP 266. My worry is that an extension > module that works in Python 2.2 will break in subtle ways in Python 2.3 if > my tracking idea is enabled, because the module's code is not doing anything > to notify the rest of the world when an attribute binding changes. > > I came up with the following possible solution. Add a new slot to type > objects that the object's author must explicitly set to Py_TRACKING_SAFE > (1?). It would default to Py_TRACKING_UNSAFE (0?). This value could be > checked by _PyObject_TrackName and a warning raised if a program tries to > track an unsafe object. > > Would that work? I have yet to find the time to read PEP 266, but I expect so -- this is how I solve many such concerns in 2.2 already (e.g. an extension type is only subclassable if it has the Py_TPFLAGS_BASETYPE flag, which is not included in Py_TPFLAGS_DEFAULT). --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com (Skip Montanaro) Thu Aug 16 08:05:54 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 16 Aug 2001 02:05:54 -0500 Subject: [Python-Dev] PEP 266: Catching code that is not safe for tracking In-Reply-To: <200108160646.CAA08652@cj20424-a.reston1.va.home.com> References: <15227.4316.969063.877269@beluga.mojam.com> <200108160646.CAA08652@cj20424-a.reston1.va.home.com> Message-ID: <15227.28882.297175.813711@beluga.mojam.com> >> I came up with the following possible solution. Add a new slot to >> type objects that the object's author must explicitly set to >> Py_TRACKING_SAFE (1?). It would default to Py_TRACKING_UNSAFE (0?). >> This value could be checked by _PyObject_TrackName and a warning >> raised if a program tries to track an unsafe object. >> >> Would that work? Guido> I have yet to find the time to read PEP 266, but I expect so -- this Guido> is how I solve many such concerns in 2.2 already (e.g. an extension Guido> type is only subclassable if it has the Py_TPFLAGS_BASETYPE flag, Guido> which is not included in Py_TPFLAGS_DEFAULT). Looks I should just add another flag to | into the tp_flags slot instead of allocating another int-sized slot. I hope to have some code for people to look at in the next few days. thank-god-ellen's-out-of-town-ly, y'rs, Skip From fredrik@pythonware.com Thu Aug 16 14:35:39 2001 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 16 Aug 2001 15:35:39 +0200 Subject: [Python-Dev] non-standard HTML tags in 2.1.1 documentation Message-ID: <00a101c12658$58131e30$0900a8c0@spiff> just grabbed the HTML versions of the 2.1.1 documentation, and found a couple of rather interesting HTML tags in there: can/should this be fixed? (to find the culprits, run tidy on the files, and look for Error lines. or use grep...) From fdrake@acm.org Thu Aug 16 14:52:07 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 16 Aug 2001 09:52:07 -0400 (EDT) Subject: [Python-Dev] non-standard HTML tags in 2.1.1 documentation In-Reply-To: <00a101c12658$58131e30$0900a8c0@spiff> References: <00a101c12658$58131e30$0900a8c0@spiff> Message-ID: <15227.53255.682873.653118@cj42289-a.reston1.va.home.com> Fredrik Lundh writes: > just grabbed the HTML versions of the 2.1.1 documentation, > and found a couple of rather interesting HTML tags in there: ... > can/should this be fixed? This should be fixed. Please file a documentation bug report on SF and assign it to me. Thanks! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From just@letterror.com Thu Aug 16 18:07:31 2001 From: just@letterror.com (Just van Rossum) Date: Thu, 16 Aug 2001 19:07:31 +0200 Subject: [Python-Dev] PyArg_ParseTuple format string: "h" vs. "H" Message-ID: <20010816190734-r01010700-898f3935-0910-010c@10.0.0.23> I was just bit by an older PyArg_ParseTuple format string behavior change: these days (since 2.1?) "h" barfs when you feed it an unsigned short value (even if it's within 16 bits). So ok, I updated my code to use "H" instead, only to find out hours later that "H" didn't exist yet in 1.5.2. I really need to be compatible with both so I'll have to work around it :-( Was this change really neccesary? Just From guido@python.org Thu Aug 16 18:55:52 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 16 Aug 2001 13:55:52 -0400 Subject: [Python-Dev] PyArg_ParseTuple format string: "h" vs. "H" In-Reply-To: Your message of "Thu, 16 Aug 2001 19:07:31 +0200." <20010816190734-r01010700-898f3935-0910-010c@10.0.0.23> References: <20010816190734-r01010700-898f3935-0910-010c@10.0.0.23> Message-ID: <200108161755.f7GHtqB17086@odiug.digicool.com> > I was just bit by an older PyArg_ParseTuple format string behavior > change: these days (since 2.1?) "h" barfs when you feed it an > unsigned short value (even if it's within 16 bits). So ok, I updated > my code to use "H" instead, only to find out hours later that "H" > didn't exist yet in 1.5.2. I really need to be compatible with both > so I'll have to work around it :-( > > Was this change really neccesary? Yes, the old behavior was indefensible. Jack complained about it too, but in the end agreed he'd update his code. If you really want compatibility without ifdefs and don't care about the range checks, you can use an i format and an int variable, and cast the int to (unsigned short) when using it. --Guido van Rossum (home page: http://www.python.org/~guido/) From just@letterror.com Thu Aug 16 19:27:48 2001 From: just@letterror.com (Just van Rossum) Date: Thu, 16 Aug 2001 20:27:48 +0200 Subject: [Python-Dev] PyArg_ParseTuple format string: "h" vs. "H" In-Reply-To: <200108161755.f7GHtqB17086@odiug.digicool.com> Message-ID: <20010816202751-r01010700-9187cf60-0910-010c@10.0.0.23> Guido van Rossum wrote: > Jack complained about it too, but in the end agreed he'd update his > code. I don't mind updating my code at all, it's writing special cases to be compatible with multiple Python versions that is, erm, yucky. > If you really want compatibility without ifdefs and don't care about > the range checks, you can use an i format and an int variable, and > cast the int to (unsigned short) when using it. That's how I did it. Just From jack@oratrix.nl Thu Aug 16 23:10:54 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri, 17 Aug 2001 00:10:54 +0200 Subject: [Python-Dev] Python version numbers and mac OS X In-Reply-To: Message by "Martin v. Loewis" , Tue, 14 Aug 2001 20:54:03 +0200 , <200108141854.f7EIs3u02092@mira.informatik.hu-berlin.de> Message-ID: <20010816221059.9F86F14B630@oratrix.oratrix.nl> There's a question of general interest at the very bottom of this message (that satrted out as a braindump to explain MaxOSX version numbers). Recently, "Martin v. Loewis" said: > > Sharing is the answer. > ... > > This is probably the crux of your reasoning, and I think I > > disagree. For one, MacOSX (ok, OpenStep is where they got it from) > > is the first unix-based system that I'm aware of that seems to have > > a solution to the versioning problem. > > I'm a bit sceptical to any claim that the software versioning problem > is "solved"; experience tells me it isn't. Agreed. But they've taken a number of steps that make me think that for Python they have the exact right solution. Here's how it works. They have two version numbers, the major and the minor. I say "number" here, but the major number is actually an arbitrary string and the minor number is a set of dotted numbers. The major number is expected to change everytime you make an incompatible change to your API (remove a routine, add a parameter to an existing routine, change a structure), exactly like the PYTHON_API_VERSION number. This number is encoded in the pathname of the dynamic library, which is something like /Library/Frameworks/Python.framework/Versions/1011/library. When you link you actually link against .../Versions/Current/library, and Current is a symlink to 1011. What is recorded in the binary is the real pathname, so at runtime a library with major incompatibility simply won't be found. And if you (the user) want to keep older versions around that's fine too. The minor version number is encoded in the library itself, and should change everytime a compatible change to the API is made (adding a routine, that sort of thing). Each library has an "oldest minor vesion number" and a "current minor version number". When you link the current number of the library is recorded in your application. When you run the number in your application should be in the range oldest..current of the library and you're fine. All that said there's still one thing that breaks for Python, and that is that it really has two distinct version numbers, the PYTHON_API_VERSION and the "real" version (2.2). Plugins are really only interested in the API version, but scripts that use Python (or programs that embed it) are really only interested in the real version number. Programs that both embed and extend Python, such as Zope, are really a pain because they're interested in both. Although this isn't as hard as it seems, because in increment of the API version by definition means an increment in the real version. I don't have a solution for this yet, mainly because the incrementing of the API version number hasn't always been completely fair to the last iota and tittle. For instance, the change of the list.append() semantics should really have incremented the API version number, but I don't think it did. If we could be sure that a newer Python core shared library could work together with an older Python Lib directory (as long as the API versions matched) and would provide at least the functionality of that older Python version we would be fine: use the API version number in the pathname and use the real version number in the lib/python2.2 and such. Hmm, count this as a request, I guess. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From fdrake@acm.org Thu Aug 16 23:09:36 2001 From: fdrake@acm.org (Fred L. Drake) Date: Thu, 16 Aug 2001 18:09:36 -0400 (EDT) Subject: [Python-Dev] [development doc updates] Message-ID: <20010816220936.4E38428845@cj42289-a.reston1.va.home.com> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ Improved documentation for the os.spawn*() functions. Fix some HTML generation problems. From tim.one@home.com Fri Aug 17 06:39:07 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 17 Aug 2001 01:39:07 -0400 Subject: [Python-Dev] Objections to PEP 264? Message-ID: If anyone has objections to Michael Hudson's PEP 264: Future statements in simulated shells http://python.sourceforge.net/peps/pep-0264.html raise them now. I like it (+1 -- it's an effective approach to two future-statement issues still unresolved). The implementation patch is here: http://sf.net/tracker/?func=detail&aid=449043&group_id=5470&atid=305470 Since Michael is on vaction, I'm going to fiddle it a bit before checking it in -- provided, of course, no killer objections are raised by you. From jack@oratrix.nl Fri Aug 17 10:20:14 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri, 17 Aug 2001 11:20:14 +0200 Subject: [Python-Dev] PyArg_ParseTuple format string: "h" vs. "H" In-Reply-To: Message by Guido van Rossum , Thu, 16 Aug 2001 13:55:52 -0400 , <200108161755.f7GHtqB17086@odiug.digicool.com> Message-ID: <20010817092014.EFA1B303181@snelboot.oratrix.nl> > > I was just bit by an older PyArg_ParseTuple format string behavior > > change: these days (since 2.1?) "h" barfs when you feed it an > > unsigned short value (even if it's within 16 bits). So ok, I updated > > my code to use "H" instead, only to find out hours later that "H" > > didn't exist yet in 1.5.2. I really need to be compatible with both > > so I'll have to work around it :-( > > > > Was this change really neccesary? > > Yes, the old behavior was indefensible. > > Jack complained about it too, but in the end agreed he'd update his > code. Well, "agreed" is a rather gentle description here. "Jack was dragged to his machine kicking and screaming where he spent months (literally) finding these nasty changes, and he still mutters and whimpers occasionally". I think that about 99% of the uses of the "h" format specifier were in code that I felt responsible for, and each of these had to be carefully inspected. I still find them, occasionally (there was another one last month). And don't even _dream_ of "fixing" the "l" format in the same way:-) -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From guido@python.org Fri Aug 17 12:00:03 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 17 Aug 2001 07:00:03 -0400 Subject: [Python-Dev] Objections to PEP 264? In-Reply-To: Your message of "Fri, 17 Aug 2001 01:39:07 EDT." References: Message-ID: <200108171100.HAA23046@cj20424-a.reston1.va.home.com> > If anyone has objections to Michael Hudson's PEP 264: > > Future statements in simulated shells > http://python.sourceforge.net/peps/pep-0264.html > > raise them now. I like it (+1 -- it's an effective approach to two > future-statement issues still unresolved). The implementation patch is > here: > > http://sf.net/tracker/?func=detail&aid=449043&group_id=5470&atid=305470 > > Since Michael is on vaction, I'm going to fiddle it a bit before checking it > in -- provided, of course, no killer objections are raised by you. +1 --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Aug 17 13:24:36 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 17 Aug 2001 08:24:36 -0400 Subject: [Python-Dev] Python version numbers and mac OS X In-Reply-To: Your message of "Fri, 17 Aug 2001 00:10:54 +0200." <20010816221059.9F86F14B630@oratrix.oratrix.nl> References: <20010816221059.9F86F14B630@oratrix.oratrix.nl> Message-ID: <200108171224.IAA24865@cj20424-a.reston1.va.home.com> I think the crux here lies in the definitions of compatible and incompatible changes. You give some examples of incompatible changes: > (remove a routine, add a parameter to an existing routine, change a > structure) and some examples of compatible changes: > (adding a routine, that sort of thing) Unfortunately, there's a vast area of possible changes that aren't really covered by these. An example from recent c.l.py discussions: in Python 2.2a1, changed dir(). I didn't change the signature (it still returns a sorted list of strings), but I changed what it returns in some cases: dir([]) used to return ['append', 'count', ...]; in 2.2a1 it returns []. That was deemed incompatible. Interesting enough, making it return a *longer* list of strings is deemed compatible! I'm sure it's easy to find examples in C as well. The moral of the story: a compatible version doesn't guarantee much beyond compatible linkage -- whether the application still works depends on a lot more than can be expressed by a minor version number. > I don't have a solution for this yet, mainly because the incrementing > of the API version number hasn't always been completely fair to the > last iota and tittle. For instance, the change of the list.append() > semantics should really have incremented the API version number, but I > don't think it did. Interesting point. For me, the API version number only describe the *C* API version, not the Python API. In this case, at least the combination (python version, C API version) did change. > If we could be sure that a newer Python core shared library could work > together with an older Python Lib directory (as long as the API > versions matched) and would provide at least the functionality of that > older Python version we would be fine: use the API version number in > the pathname and use the real version number in the lib/python2.2 and > such. While this could work, but it would be confusing: sys.version would be the version of the core shared library, and applications would expect to find the corresponding Lib modules. IOW, I don't think it's a useful feature to have. --Guido van Rossum (home page: http://www.python.org/~guido/) From jack@oratrix.nl Fri Aug 17 14:01:04 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri, 17 Aug 2001 15:01:04 +0200 Subject: [Python-Dev] Python version numbers and mac OS X In-Reply-To: Message by Guido van Rossum , Fri, 17 Aug 2001 08:24:36 -0400 , <200108171224.IAA24865@cj20424-a.reston1.va.home.com> Message-ID: <20010817130104.90E8C303181@snelboot.oratrix.nl> > > If we could be sure that a newer Python core shared library could work > > together with an older Python Lib directory (as long as the API > > versions matched) and would provide at least the functionality of that > > older Python version we would be fine: use the API version number in > > the pathname and use the real version number in the lib/python2.2 and > > such. > > While this could work, but it would be confusing: sys.version would be > the version of the core shared library, and applications would expect > to find the corresponding Lib modules. IOW, I don't think it's a > useful feature to have. Ah, bah! Count on Guido to come with a showstopper for a nifty idea:-( This means that we must consider the core shared lib and the lib/python2.2 directory to be very closely coupled. Which in turn means that the only reasonable thing to use as the version number in the framework pathname is the "2.2" style version number. Which means that although all the infrastructure is there we can't have backward compatible plugin modules. Which means Martin was right with his original comment. ... unless there is a way in which we could record a different pathname for the shared library depending on whether we are linking an extension or an application: the former would check that the API version matches, the latter that the 2.2-style version matches. Hmm, MacOSX also has special magic for plugins, I guess it's time to start looking into that. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From tim.one@home.com Fri Aug 17 20:17:51 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 17 Aug 2001 15:17:51 -0400 Subject: [Python-Dev] test_builtin failing for anyone else? Message-ID: I'm seeing this after Martin's large Unicode patch. However, I'm also working on a different patch that mucks with builtins, so I'd just like to know whether I'm the only one seeing this: C:\Code\python\PCbuild>python ../lib/test/test_builtin.py 4. Built-in functions test_b1 __import__ abs apply callable chr cmp coerce compile complex delattr dir divmod eval execfile filter float Traceback (most recent call last): File "../lib/test/test_builtin.py", line 9, in ? import test_b1 File "../lib/test\test_b1.py", line 259, in ? if float(unicode(" \u0663.\u0661\u0664 ")) != 3.14: ValueError: invalid literal for float(): \u0663.\u0661\u0664 C:\Code\python\PCbuild> Oops -- I'm also seeing a Unicode-related failure in test_format.py: ... u'abc %\\\u3000' % 1 works? ... no Unexpected exceptions.ValueError : "unsupported format character '\\' (0x5c) at index 5" That's it. Anyone else? From guido@python.org Fri Aug 17 20:20:18 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 17 Aug 2001 15:20:18 -0400 Subject: [Python-Dev] 2.2a2 release next Wednesday Message-ID: <200108171920.PAA10067@cj20424-a.reston1.va.home.com> Barry & I are planning to release Python 2.2a2 next Wednesday (August 19), according to the (updated) schedule in PEP 251. Tomorrow (Saturday, August 15) I plan to fork off a short-lived "release branch", from which the release will be done. Nobody but Barry & I should be checking things in on the release branch. We'll selectively merges the trunk into the branch if needed. We'll merge the branch back to the trunk after the release. Anybody who wants something to show up in 2.2a2 would be wise to check it in today or before dawn tomorrow. Please mail me if you need me to wait for something specific. PS Martin: I currently have two failing tests! - test_b1 crashes (in the "float" subtest): Traceback (most recent call last): File "../Lib/test/test_b1.py", line 259, in ? if float(unicode(" \u0663.\u0661\u0664 ")) != 3.14: ValueError: invalid literal for float(): \u0663.\u0661\u0664 - test_format.py fails: u'abc %\\\u3000' % 1 works? ... no Unexpected exceptions.ValueError : "unsupported format character '\\' (0x5c) at index 5" I expect these are casualties of the --disable-unicode checkins. --Guido van Rossum (home page: http://www.python.org/~guido/) From esr@thyrsus.com Fri Aug 17 20:46:29 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Fri, 17 Aug 2001 15:46:29 -0400 Subject: [Python-Dev] 2.2a2 release next Wednesday In-Reply-To: <200108171920.PAA10067@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Aug 17, 2001 at 03:20:18PM -0400 References: <200108171920.PAA10067@cj20424-a.reston1.va.home.com> Message-ID: <20010817154629.D1304@thyrsus.com> Guido van Rossum : > Anybody who wants something to show up in 2.2a2 would be wise to check > it in today or before dawn tomorrow. Please mail me if you need me to > wait for something specific. OK. Don't wait on this, but I'm going to try to find time to check in my stuff that provides compilerlike framework support for scripts. Code is tested, docs are written. The issue is packaging; I was going to make it a separate ccframe module, but I'm thinking Greg Ewing's suggestion that it should live in fileinput is on balance a good one. But that means I have to merge in the docs. I'll probably get to this tonight. -- Eric S. Raymond To make inexpensive guns impossible to get is to say that you're putting a money test on getting a gun. It's racism in its worst form. -- Roy Innis, president of the Congress of Racial Equality (CORE), 1988 From gstein@lyra.org Fri Aug 17 21:30:56 2001 From: gstein@lyra.org (Greg Stein) Date: Fri, 17 Aug 2001 13:30:56 -0700 Subject: [Python-Dev] Free threading In-Reply-To: <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org>; from dan@sidhe.org on Wed, Aug 15, 2001 at 06:59:47PM -0400 References: <3B78CA34.9A4E1765@ActiveState.com> <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org> Message-ID: <20010817133056.C1223@lyra.org> On Wed, Aug 15, 2001 at 06:59:47PM -0400, Dan Sugalski wrote: > At 11:50 PM 8/13/2001 -0700, Paul Prescod wrote: > >Tim Peters wrote: > > >... > > > IIRC, Greg's fabled free-threading version of Python took a speed hit of > > > about a factor of 2 (for a program using only 1 thread, compared to that > > > same program without the free-threading patches). Yah. That's about right. The largest problem was dealing with dictionaries. Since you never knew whether a specific dictionary was shared (between threads) or not, you always had to lock the access. And since all namespaces use dictionaries... > >The Perl guys considered this unacceptable and I can kind of see their > >point. You have two processors but you get roughly the same performance > >as one? No. For definitional purposes, let's say that for normal Python on one processor, you get 1 Python Speed Unit (PSU). With my free-threading patches, that uniprocessor would get around 0.6 PSU. Move to a multiprocessor with 2 CPUs, running a 2-thread program with no synchronization (e.g. each is simply doing their thing rather than potentially intefering with each other). Regular Python would get about 0.95 PSU because the GIL imposes some overhead (and you can't ever get more than 1 because of the GIL). With the free-threading, you would get about 1.2 PSU. On a three processor system, regular Python still gets 0.95 PSU. The free-threading goes up to maybe 1.6 PSU. We observed non-linear scaling with the processors under free threading. 2 processors was fine, but 3 or 4 didn't buy you much more than 2. The problem was lock contention. With that many things going, the contention around Python's internal structures simply killed further scaling performance. >... > I racked up a whole list of "Things to Not Do With Threads" when hacking > the original perl thread model. (The first of which is "wedge them into an > interpreter that wasn't written with threads in mind..." :) Battle scars > are viewable on request. I hear ya. Same here. But Python has since become ever worse re: free threading capability (more globals to arbitrate access to). Last time, I tried to optimize the memory associate with each list/dict. That slowed some things down. Atomic incr/decr wasn't really available under Linux (Win has InterlockedIncrement and friends), so the Linux incr/decr was a bit slower than it should it have been. There are a number of things that I'd do differently the next time around. Cheers, -g -- Greg Stein, http://www.lyra.org/ From guido@python.org Fri Aug 17 21:36:21 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 17 Aug 2001 16:36:21 -0400 Subject: [Python-Dev] Free threading In-Reply-To: Your message of "Fri, 17 Aug 2001 13:30:56 PDT." <20010817133056.C1223@lyra.org> References: <3B78CA34.9A4E1765@ActiveState.com> <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org> <20010817133056.C1223@lyra.org> Message-ID: <200108172036.QAA15090@cj20424-a.reston1.va.home.com> > Last time, I tried to optimize the memory associate with each > list/dict. That slowed some things down. Atomic incr/decr wasn't > really available under Linux (Win has InterlockedIncrement and > friends), so the Linux incr/decr was a bit slower than it should it > have been. This would be an opportunity for some highly platform-specific assembler, assuming we'll always have configure --{enable,disable}-free-threading. > There are a number of things that I'd do differently the next time around. When will that be? --Guido van Rossum (home page: http://www.python.org/~guido/) From gstein@lyra.org Fri Aug 17 21:43:49 2001 From: gstein@lyra.org (Greg Stein) Date: Fri, 17 Aug 2001 13:43:49 -0700 Subject: [Python-Dev] Free threading In-Reply-To: <20010809061327.24EC299C81@waltz.rahul.net>; from aahz@rahul.net on Wed, Aug 08, 2001 at 11:13:27PM -0700 References: <200108082230.SAA09334@cj20424-a.reston1.va.home.com> <20010809061327.24EC299C81@waltz.rahul.net> Message-ID: <20010817134349.D1223@lyra.org> On Wed, Aug 08, 2001 at 11:13:27PM -0700, Aahz Maruch wrote: > Guido van Rossum wrote: > > Aahz: > >> We need thread-local storage, anyway. When I'm a little farther into my > >> work on Decimal, I'm going to take a hard look at this because context > >> needs to be global on a per-thread basis... > > > > In the C API, use PyThreadState_GetDict(): it returns a per-thread > > dictionary. The convention is that you pick a key unique to your > > extension module (e.g. your full extension module name) and then store > > your per-thread data under this key. Chicken and egg. Note that in a free threading context, there is no such thing as _PyThreadState_Current. Thus, you can't get to that dict. Instead, you pick up your thread state pointer from TLS (or an equiv mechanism on other platforms). >... > But what I was really referring to was the annoyance of having every > method call pull in the thread-local information. Seems to me that > there ought to be some way of making it happen magically. Not much you can do here, except to start passing around the thread state as a parameter. (not the interp! that is derived from the thread state, but it can't be done the other way around) Note that I posted some of the steps (towards free threading) last year. Each of these are low cost, but will help set a framework for further threading work: http://mail.python.org/pipermail/python-dev/2000-April/003605.html Cheers, -g -- Greg Stein, http://www.lyra.org/ From loewis@informatik.hu-berlin.de Fri Aug 17 23:14:22 2001 From: loewis@informatik.hu-berlin.de (Martin von Loewis) Date: Sat, 18 Aug 2001 00:14:22 +0200 (MEST) Subject: [Python-Dev] test_builtin failing for anyone else? Message-ID: <200108172214.AAA05711@punica.informatik.hu-berlin.de> > I'm seeing this after Martin's large Unicode patch. ... > That's it. Anyone else? You might think I should have seen it too. I didn't, because I ran the testsuite for --enable-unicode without the patches to the tests, and re-ran the test suite then only for --disable-unicode (*) :-( Anyway, I just committed a patch that fixes it for me... Regards, Martin (*) That also produces a number of failures, but that is expected: some tests still absolutely require Unicode support, and some of the libraries do as well, mainly pickle. From skip@pobox.com (Skip Montanaro) Sat Aug 18 00:54:27 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 17 Aug 2001 18:54:27 -0500 Subject: [Python-Dev] tempnam/tmpnam_r Message-ID: <15229.44723.470234.595607@beluga.mojam.com> I get this from gcc: libpython2.2.a(posixmodule.o): In function `posix_tmpnam': /home/skip/src/python/head/dist/src/build/../Modules/posixmodule.c:4262: the use of `tmpnam_r' is dangerous, better use `mkstemp' libpython2.2.a(posixmodule.o): In function `posix_tempnam': /home/skip/src/python/head/dist/src/build/../Modules/posixmodule.c:4217: the use of `tempnam' is dangerous, better use `mkstemp' I realize this has come up before. Is it possible to preserve the semantics of posix_tmpnam and posix_tempnam while switching the implementation over to mkstemp where it's available? Alternatively, perhaps Python should produce a warning similar to what gcc produces the first time these functions are called. Skip From gstein@lyra.org Sat Aug 18 01:20:44 2001 From: gstein@lyra.org (Greg Stein) Date: Fri, 17 Aug 2001 17:20:44 -0700 Subject: [Python-Dev] Free threading In-Reply-To: <200108172036.QAA15090@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Aug 17, 2001 at 04:36:21PM -0400 References: <3B78CA34.9A4E1765@ActiveState.com> <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org> <20010817133056.C1223@lyra.org> <200108172036.QAA15090@cj20424-a.reston1.va.home.com> Message-ID: <20010817172044.G1223@lyra.org> On Fri, Aug 17, 2001 at 04:36:21PM -0400, Guido van Rossum wrote: > > Last time, I tried to optimize the memory associate with each > > list/dict. That slowed some things down. Atomic incr/decr wasn't > > really available under Linux (Win has InterlockedIncrement and > > friends), so the Linux incr/decr was a bit slower than it should it > > have been. > > This would be an opportunity for some highly platform-specific > assembler, Yah. > assuming we'll always have > configure --{enable,disable}-free-threading. Good assumption :-) As its been pointed out recently on this list, and in the past, free threading isn't the "end all" of threading. There are some tradeoffs involved. > > There are a number of things that I'd do differently the next time around. > > When will that be? I started my note with if/when :-) Between the Python http-based modules, ViewCVS, edna, Subversion, Apache, and mod_dav... I'm a bit booked up for a while :-) and don't have an "itch" to get free threading completed. The "baby steps" note that I posted last year was a hope to get others involved, who may have more immediate needs for this kind of functionality. Kind of a road map from my earlier experience. Cheers, -g -- Greg Stein, http://www.lyra.org/ From guido@python.org Sat Aug 18 02:28:24 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 17 Aug 2001 21:28:24 -0400 Subject: [Python-Dev] Free threading In-Reply-To: Your message of "Fri, 17 Aug 2001 17:20:44 PDT." <20010817172044.G1223@lyra.org> References: <3B78CA34.9A4E1765@ActiveState.com> <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org> <20010817133056.C1223@lyra.org> <200108172036.QAA15090@cj20424-a.reston1.va.home.com> <20010817172044.G1223@lyra.org> Message-ID: <200108180128.VAA19233@cj20424-a.reston1.va.home.com> > I started my note with if/when :-) Between the Python http-based modules, > ViewCVS, edna, Subversion, Apache, and mod_dav... I'm a bit booked up for a > while :-) and don't have an "itch" to get free threading completed. Understood. > The "baby steps" note that I posted last year was a hope to get others > involved, who may have more immediate needs for this kind of functionality. > Kind of a road map from my earlier experience. Thanks for pointing to those -- they are a good place to start indeed! Now, when *I* have some free time on my hands... (chuckle). --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com (Skip Montanaro) Sat Aug 18 03:02:20 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 17 Aug 2001 21:02:20 -0500 Subject: [Python-Dev] Free threading In-Reply-To: <20010817172044.G1223@lyra.org> References: <3B78CA34.9A4E1765@ActiveState.com> <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org> <20010817133056.C1223@lyra.org> <200108172036.QAA15090@cj20424-a.reston1.va.home.com> <20010817172044.G1223@lyra.org> Message-ID: <15229.52396.371840.484869@beluga.mojam.com> Greg> The "baby steps" note that I posted last year was a hope to get Greg> others involved, who may have more immediate needs for this kind Greg> of functionality. Kind of a road map from my earlier experience. Of course, I deleted that reference already, but I can dig it up if I need it. Not that I'm volunteering for this or anything, but perhaps there are some things that authors should be alerted so their code in the meantime doesn't make a future free threading Python that much more difficult to implement. This sort of information might profitably be added to some documentation now (ext/embed or c api manuals?). Skip From skip@pobox.com (Skip Montanaro) Sat Aug 18 03:16:14 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 17 Aug 2001 21:16:14 -0500 Subject: [Python-Dev] Py_FatalError... Message-ID: <15229.53230.429726.683520@beluga.mojam.com> Dumb question: Why is it Py_FatalError and not PyErr_FatalError? Skip From martin@loewis.home.cs.tu-berlin.de Sat Aug 18 08:09:20 2001 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Sat, 18 Aug 2001 09:09:20 +0200 Subject: [Python-Dev] tempnam/tmpnam_r Message-ID: <200108180709.f7I79KC01025@mira.informatik.hu-berlin.de> > I realize this has come up before. Is it possible to preserve the > semantics of posix_tmpnam and posix_tempnam while switching the > implementation over to mkstemp where it's available? No. If it was possible, the C library would have taken such an approach to fix it, instead of coming up with a way to complain. In case you wonder what is dangerous about these functions: they both return strings of filenames that where unique at the time this was tested. Typically, the application will then pass that string to open(). Now, a malicious application may wait for the moment when tempnam returns, and create the file. Then, the Python application will open the temporary file, which happens to be created already. Thus, the malicious application will be able to find out and modify the data that the Python application has put into the temporary file. The only true solution is to find a temporary name and open the file for exclusive access (O_EXCL); if that fails, pick another temporary name. mkstemp does that and returns both the open file handle, and the file name. Regards, Martin From skip@pobox.com (Skip Montanaro) Sat Aug 18 17:32:07 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Sat, 18 Aug 2001 11:32:07 -0500 Subject: [Python-Dev] tempnam/tmpnam_r In-Reply-To: <200108180709.f7I79KC01025@mira.informatik.hu-berlin.de> References: <200108180709.f7I79KC01025@mira.informatik.hu-berlin.de> Message-ID: <15230.39047.773294.802231@beluga.mojam.com> >> I realize this has come up before. Is it possible to preserve the >> semantics of posix_tmpnam and posix_tempnam while switching the >> implementation over to mkstemp where it's available? Martin> No. If it was possible, the C library would have taken such an Martin> approach to fix it, instead of coming up with a way to complain. Understood. Martin> In case you wonder what is dangerous about these functions: they Martin> both return strings of filenames that where unique at the time Martin> this was tested. Typically, the application will then pass that Martin> string to open(). Now, a malicious application may wait for the Martin> moment when tempnam returns, and create the file. Then, the Martin> Python application will open the temporary file, which happens Martin> to be created already. Thus, the malicious application will be Martin> able to find out and modify the data that the Python application Martin> has put into the temporary file. Given that these two functions have a race condition and that it can be exploited when used in C code, it seems to me that it would be even easier to exploit when used in Python code, since Python programs tend to run so much slower than C programs at the granularity of measurement we're talking about. If that's the case, then I think it makes a stronger argument for deprecating the existing tmpnam and tempnam functions in favor of mkstemp. I realize there are some issues to resolve. First, mkstemp returns a file descriptor and modifies the input filename template. The return value to Python code would have to be a tuple (filename, fd). Second, if we want to encourage people to use it in place of the builtin open function, it will have to return a normal Python file object instead of an integer file descriptor. Third, is mkstemp pretty universally available? Finally, I still think use of tmpnam or tempnam should emit warnings. Skip From guido@python.org Sat Aug 18 18:08:15 2001 From: guido@python.org (Guido van Rossum) Date: Sat, 18 Aug 2001 13:08:15 -0400 Subject: [Python-Dev] Py_FatalError... In-Reply-To: Your message of "Fri, 17 Aug 2001 21:16:14 CDT." <15229.53230.429726.683520@beluga.mojam.com> References: <15229.53230.429726.683520@beluga.mojam.com> Message-ID: <200108181708.NAA20608@cj20424-a.reston1.va.home.com> > Dumb question: Why is it Py_FatalError and not PyErr_FatalError? Because it has nothing to do with exceptions. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Sat Aug 18 18:11:58 2001 From: guido@python.org (Guido van Rossum) Date: Sat, 18 Aug 2001 13:11:58 -0400 Subject: [Python-Dev] tempnam/tmpnam_r In-Reply-To: Your message of "Sat, 18 Aug 2001 11:32:07 CDT." <15230.39047.773294.802231@beluga.mojam.com> References: <200108180709.f7I79KC01025@mira.informatik.hu-berlin.de> <15230.39047.773294.802231@beluga.mojam.com> Message-ID: <200108181711.NAA20953@cj20424-a.reston1.va.home.com> > Finally, I still think use of tmpnam or tempnam should emit warnings. Me too. Go ahead and check something in! --Guido van Rossum (home page: http://www.python.org/~guido/) From gstein@lyra.org Sat Aug 18 18:36:39 2001 From: gstein@lyra.org (Greg Stein) Date: Sat, 18 Aug 2001 10:36:39 -0700 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib socket.py,1.11,1.12 In-Reply-To: ; from gvanrossum@users.sourceforge.net on Fri, Aug 17, 2001 at 06:23:22PM -0700 References: Message-ID: <20010818103639.B8246@lyra.org> On Fri, Aug 17, 2001 at 06:23:22PM -0700, Guido van Rossum wrote: > Update of /cvsroot/python/python/dist/src/Lib > In directory usw-pr-cvs1:/tmp/cvs-serv16772 > > Modified Files: > socket.py > Log Message: > When the socket is closed, don't just assign 0 to self._sock. > This breaks software that excepts a socket.error but not an > AttributeError. >... > + class _closedsocket: > + > + def _bummer(self): > + raise error(9, 'Bad file descriptor') > + > + _s = "def %s(self, *args): self._bummer()\n\n" > + for _m in _socketmethods: > + exec _s % _m Isn't this last one easier written has: for _m in _socketmethods: vars()[_m] = _bummer And change _bummer to have a *args. Cheers, -g -- Greg Stein, http://www.lyra.org/ From skip@pobox.com (Skip Montanaro) Sat Aug 18 19:53:00 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Sat, 18 Aug 2001 13:53:00 -0500 Subject: [Python-Dev] tempnam/tmpnam_r In-Reply-To: <200108181711.NAA20953@cj20424-a.reston1.va.home.com> References: <200108180709.f7I79KC01025@mira.informatik.hu-berlin.de> <15230.39047.773294.802231@beluga.mojam.com> <200108181711.NAA20953@cj20424-a.reston1.va.home.com> Message-ID: <15230.47500.33766.834605@beluga.mojam.com> Skip> Finally, I still think use of tmpnam or tempnam should emit Skip> warnings. Guido> Me too. Go ahead and check something in! Done. Skip From guido@python.org Sat Aug 18 21:59:29 2001 From: guido@python.org (Guido van Rossum) Date: Sat, 18 Aug 2001 16:59:29 -0400 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib socket.py,1.11,1.12 In-Reply-To: Your message of "Sat, 18 Aug 2001 10:36:39 PDT." <20010818103639.B8246@lyra.org> References: <20010818103639.B8246@lyra.org> Message-ID: <200108182059.QAA23666@cj20424-a.reston1.va.home.com> > Isn't this last one easier written has: > > for _m in _socketmethods: > vars()[_m] = _bummer > > And change _bummer to have a *args. Good idea. For an even better idea, see my checkin. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Sat Aug 18 22:30:53 2001 From: guido@python.org (Guido van Rossum) Date: Sat, 18 Aug 2001 17:30:53 -0400 Subject: [Python-Dev] Python 2.2a2 - release branch created Message-ID: <200108182130.RAA25913@cj20424-a.reston1.va.home.com> I've created the Python 2.2a2 release branch. The CVS tag is 'r22a2-branch'. (There's also a tag 'r22a2-fork' -- that's the point on the trunk where the branch forks off.) So far the only change in the branch is the version number. Only the release managers (Barry and I) should make checkins to the branch. All others check in to the trunk. Please notify the release managers (a comment in the checkin message would suffice) if you think your checkin should be merged into the branch. The release is planned for Wednesday, August 19. After the release is done, the branch will be merged back into the trunk. --Guido van Rossum (home page: http://www.python.org/~guido/) From gball@cfa.harvard.edu Sat Aug 18 23:09:29 2001 From: gball@cfa.harvard.edu (Greg Ball) Date: Sat, 18 Aug 2001 18:09:29 -0400 (EDT) Subject: [Python-Dev] Objections to PEP 264? Message-ID: Tim's latest change to __future__.py has toasted the build process. Python itself builds fine, then is run to build the extensions. site imports distutils distutils imports re re imports sre sre imports copy_reg copy_reg imports types types imports __future__ __future__ imports new, uh oh, new doesn't yet exist... python then tries to keep going but with distutils non-functional, it doesn't stand a chance. If the new module is now considered mainstream, maybe it could be built statically? --Greg Ball From guido@python.org Sun Aug 19 06:30:11 2001 From: guido@python.org (Guido van Rossum) Date: Sun, 19 Aug 2001 01:30:11 -0400 Subject: [Python-Dev] Objections to PEP 264? In-Reply-To: Your message of "Sat, 18 Aug 2001 18:09:29 EDT." References: Message-ID: <200108190530.BAA00766@cj20424-a.reston1.va.home.com> > Tim's latest change to __future__.py has toasted the build process. Python > itself builds fine, then is run to build the extensions. > > site imports distutils > distutils imports re > re imports sre > sre imports copy_reg > copy_reg imports types > types imports __future__ > __future__ imports new, uh oh, new doesn't yet exist... > > python then tries to keep going but with distutils non-functional, > it doesn't stand a chance. > > If the new module is now considered mainstream, maybe it could be built > statically? Good catch! (This uncovered some problems in my setup -- I hadn't picked up the latest versions of a few modules. :-( ) I solved it a little differently, by adding hackery to __future__ that avoids the dependency. But maybe you're right and it should be built statically (it's in the main DLL on Windows). --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Sun Aug 19 17:22:54 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Sun, 19 Aug 2001 12:22:54 -0400 (EDT) Subject: [Python-Dev] Free threading In-Reply-To: <20010817172044.G1223@lyra.org> References: <3B78CA34.9A4E1765@ActiveState.com> <5.1.0.14.0.20010815185358.01ba1e68@tuatha.sidhe.org> <20010817133056.C1223@lyra.org> <200108172036.QAA15090@cj20424-a.reston1.va.home.com> <20010817172044.G1223@lyra.org> Message-ID: <15231.59358.703308.343183@cj42289-a.reston1.va.home.com> Greg Stein writes: > I started my note with if/when :-) Between the Python http-based modules, > ViewCVS, edna, Subversion, Apache, and mod_dav... I'm a bit booked up for a And don't forget the various build-process bugs & patches for Expat, either! ;-) For the patches, if you can tell me yea/nay, I can deal with the mechanics of committing them. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From skip@pobox.com (Skip Montanaro) Sun Aug 19 18:45:50 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Sun, 19 Aug 2001 12:45:50 -0500 Subject: [Python-Dev] Instruction count frequency server Message-ID: <15231.64334.71337.225588@beluga.mojam.com> Last night during the midst of a lengthy reply to a post by Alex Martelli, it occurred to me that it might be useful for people fiddling around with the Python VM to have access to a large body of instruction count information. Marc-Andre Lemburg posted a note a year or so ago that summarized dynamic instruction count frequencies gathered over 100 million PyVM instruction cycles. While looking at his table I noticed that he reported a very high frequency of LOAD_NAME instructions. I took a quick peek at static instruction counts in the core library and found that LOAD_NAME almost never appears in that particular chunk of Python bytecode. So, although MAL collected data over a large number of instruction cycles, it's not obvious that his programs were "typical". This morning I did something about that. I wrote a little XML-RPC server that allows people to post or retrieve instruction count frequencies. It's at manatee.mojam.com, port 7304. To get a quick synopsis of the methods the server implements, try this: dxp = xmlrpclib.Server("http://manatee.mojam.com:7304") print dxp.synopsis() More detail can be had with print dxp.usage() The usage response contains details about the individual methods, run-length encode/decode functions, and hints on how to send sys.getdxp() output to the server automatically whenever a program exits normally. Server response isn't terribly speedy, mostly because when sending or receiving instruction count data a fair number of bytes stream one way or the other, even though the data is run-length encoded and the server will gzip encode output when it can. Even when queried from the same machine as the server it takes about 5 seconds to return non-trivial output. Oh well, XML-RPC isn't perfect for everything. Hopefully the load won't be too heavy. Feedback is welcome. Skip From tim.one@home.com Sun Aug 19 20:34:02 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 19 Aug 2001 15:34:02 -0400 Subject: [Python-Dev] Objections to PEP 264? In-Reply-To: Message-ID: [Greg Ball] > Tim's latest change to __future__.py has toasted the build process. All right! I've been trying for *years* to screw the Linux build, and can finally retire -- payback is soooo sweet . eventually-python-won't-compile-on-anything-ly y'rs - tim From jack@oratrix.nl Sun Aug 19 21:36:30 2001 From: jack@oratrix.nl (Jack Jansen) Date: Sun, 19 Aug 2001 22:36:30 +0200 Subject: [Python-Dev] test_threaded_import hanging on MacOS and Irix Message-ID: <20010819203635.80A65162E14@oratrix.oratrix.nl> test_threaded_import occasionally hangs in MacPython from the current CVS tree (about 30% of the time) and I was going to look into this shortly, assuming it to be a Mac-specific error. But today suddenly it also hung on my Irix 6.5 Python, also uninterruptible with control-C. Is anyone else seeing this? Could this be a race that is machine-independent? I do happen to know that the SGI is rather aggressive with thread switching, and so is my MacPython pthread implementation... -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From tim.one@home.com Sun Aug 19 23:46:54 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 19 Aug 2001 18:46:54 -0400 Subject: [Python-Dev] FW: Case Sensitivity: Survey Results Message-ID: Wow, Sheila really worked on this! Check out her web site. -----Original Message----- From: python-list-admin@python.org [mailto:python-list-admin@python.org]On Behalf Of Sheila King Sent: Saturday, August 18, 2001 2:25 AM To: python-list@python.org Subject: Case Sensitivity: Survey Results In response to this message: http://mail.python.org/pipermail/python-list/2001-July/054788.html which continued the discussion about case sensitivity in the Python language, I announced my intention to conduct a survey of computer science teachers on this issue, here: http://mail.python.org/pipermail/python-list/2001-July/055020.html The survey is now complete and results are available. I received a total of 129 responses. For summaries and to read the actual responses, you can go here: http://www.thinkspot.net/sheila/computers/case/casesurvey.html My take on the whole thing, is this: The overwhelming majority of teachers surveyed on this topic felt it was a non-issue (well over 80% of them). They felt that students have either little or no difficulty due to case-sensitivity. Perhaps, if one were designing a language from scratch (which is no longer the case with Python), it would be reasonable to adopt case-insensitivity. It seems there is little benefit to be derived from case-sensitivity (although some respondents felt that precision in approach and thinking, richness in language and naming choices, plus compiler efficiency made sensitivity preferable). So, in the interest of generating fewer error messages, insensitivity may be preferable. However, given that this was felt to be an extremely minor issue, I see no reason to entertain the issue of changing an already existing programming language either from case-sensitive to case-insensitive or vice-versa. There would be little to be gained, and certainly it would not justify backward compatibility issues with already existing code. Time would be better spent developing better error messages, or more friendly programming environments for the beginner. As I said, that is my take on this issue, and I feel that it is supported by the survey results. More details on the results of the survey, including all responses to the survey, are online here: http://www.thinkspot.net/sheila/computers/case/casesurvey.html -- Sheila King http://www.thinkspot.net/sheila/ http://www.k12groups.org/ -- http://mail.python.org/mailman/listinfo/python-list From tim.one@home.com Mon Aug 20 00:11:21 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 19 Aug 2001 19:11:21 -0400 Subject: [Python-Dev] test_threaded_import hanging on MacOS and Irix In-Reply-To: <20010819203635.80A65162E14@oratrix.oratrix.nl> Message-ID: [Jack Jansen] > test_threaded_import occasionally hangs in MacPython from the current > CVS tree (about 30% of the time) and I was going to look into this > shortly, assuming it to be a Mac-specific error. How do you run it? Via regrtest in a clump regrtest by itself directly debug or release build? with or without purging all .pyc files first? with or without -v? Does varying any of the above change the symptoms? Does it do anything *other* than hang (for example, display any msg(s) to the screen)? > But today suddenly it also hung on my Irix 6.5 Python, also > uninterruptible with control-C. Nothing is interesting on Irix unless you compile Python without optimization there first (virtually every bug ever reported on SGI boxes has been cured by turning optimization off). > Is anyone else seeing this? Not me. > Could this be a race that is machine-independent? More like a race that's machine-dependent, yes? Since the point of the import lock is to avoid races, of course it's just as likely to cause them as to cure them . > I do happen to know that the SGI is rather aggressive with thread > switching, and so is my MacPython pthread implementation... Ditto Windows, and no problem there. From esr@thyrsus.com Mon Aug 20 00:54:49 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Sun, 19 Aug 2001 19:54:49 -0400 Subject: [Python-Dev] Apparent bug in 2.2a1 Message-ID: <20010819195449.A10264@thyrsus.com> # This trivial Python program fails. # # Python 2.2a1+ (#12, Aug 18 2001, 14:42:50) # [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-85)] on linux2 # # To really confuse yourself, set xml to 0 and watch it run without error. xml = 1 stem = "a" if xml: stem += "b", else: stem += "b" # Here is the error I see: # # Traceback (most recent call last): # File "foo.py", line 11, in ? # stem += "b", # TypeError: cannot add type "tuple" to string -- Eric S. Raymond The world is filled with violence. Because criminals carry guns, we decent law-abiding citizens should also have guns. Otherwise they will win and the decent people will lose. -- James Earl Jones From alex_c@MIT.EDU Mon Aug 20 01:00:50 2001 From: alex_c@MIT.EDU (Alex Coventry) Date: 19 Aug 2001 20:00:50 -0400 Subject: [Python-Dev] Apparent bug in 2.2a1 In-Reply-To: "Eric S. Raymond"'s message of "Sun, 19 Aug 2001 19:54:49 -0400" References: <20010819195449.A10264@thyrsus.com> Message-ID: The problem may be the comma following the "b". That makes the RHS a tuple. Alex. From esr@thyrsus.com Mon Aug 20 01:11:26 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Sun, 19 Aug 2001 20:11:26 -0400 Subject: [Python-Dev] Apparent bug in 2.2a1 In-Reply-To: ; from alex_c@MIT.EDU on Sun, Aug 19, 2001 at 08:00:50PM -0400 References: <20010819195449.A10264@thyrsus.com> Message-ID: <20010819201126.A10488@thyrsus.com> Alex Coventry : > The problem may be the comma following the "b". That makes the RHS a > tuple. My bad. Damn that trailing comma is easy to miss. -- Eric S. Raymond In every country and in every age, the priest has been hostile to liberty. He is always in alliance with the despot, abetting his abuses in return for protection to his own. -- Thomas Jefferson, 1814 From skip@pobox.com (Skip Montanaro) Mon Aug 20 01:32:20 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Sun, 19 Aug 2001 19:32:20 -0500 Subject: [Python-Dev] Apparent bug in 2.2a1 In-Reply-To: <20010819201126.A10488@thyrsus.com> References: <20010819195449.A10264@thyrsus.com> <20010819201126.A10488@thyrsus.com> Message-ID: <15232.23188.12163.512361@beluga.mojam.com> Eric> Damn that trailing comma is easy to miss. Which is why my tuples always snuggle up to parens. I almost wish paren-free tuple creation was deprecated. (You think the integer division stink was bad... ;-) Skip From pedroni@inf.ethz.ch Mon Aug 20 01:49:35 2001 From: pedroni@inf.ethz.ch (Samuele Pedroni) Date: Mon, 20 Aug 2001 02:49:35 +0200 Subject: [Python-Dev] Q: playing with 2.2a1 => strange behaviour, or misunderstanding References: <200108171920.PAA10067@cj20424-a.reston1.va.home.com> Message-ID: <000d01c12911$fca5ed40$8a73fea9@newmexico> I have tried this with 2.2a1: 1. I have not checked the related C code 2. I have not checked the bug report list >>> class C(object): pass ... >>> c=C() >>> c.a=3 >>> object.__getattr__(c,'a') 3 >>> object.__setattr__(c,'a',6) Traceback (most recent call last): File "", line 1, in ? TypeError: function takes exactly 2 arguments (3 given) Is that meaningful or a problem? I'm trying to understand type/class unification to consider possible jython issues. regards, Samuele Pedroni From guido@python.org Mon Aug 20 03:23:25 2001 From: guido@python.org (Guido van Rossum) Date: Sun, 19 Aug 2001 22:23:25 -0400 Subject: [Python-Dev] Apparent bug in 2.2a1 In-Reply-To: Your message of "Sun, 19 Aug 2001 19:32:20 CDT." <15232.23188.12163.512361@beluga.mojam.com> References: <20010819195449.A10264@thyrsus.com> <20010819201126.A10488@thyrsus.com> <15232.23188.12163.512361@beluga.mojam.com> Message-ID: <200108200223.WAA10316@cj20424-a.reston1.va.home.com> > Eric> Damn that trailing comma is easy to miss. > > Which is why my tuples always snuggle up to parens. I almost wish > paren-free tuple creation was deprecated. (You think the integer division > stink was bad... ;-) Idea for PyChecker? --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Aug 20 14:11:15 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 20 Aug 2001 09:11:15 -0400 Subject: [Python-Dev] Q: playing with 2.2a1 => strange behaviour, or misunderstanding In-Reply-To: Your message of "Mon, 20 Aug 2001 02:49:35 +0200." <000d01c12911$fca5ed40$8a73fea9@newmexico> References: <200108171920.PAA10067@cj20424-a.reston1.va.home.com> <000d01c12911$fca5ed40$8a73fea9@newmexico> Message-ID: <200108201311.JAA17884@cj20424-a.reston1.va.home.com> > I have tried this with 2.2a1: > 1. I have not checked the related C code > 2. I have not checked the bug report list > > >>> class C(object): pass > ... > >>> c=C() > >>> c.a=3 > >>> object.__getattr__(c,'a') > 3 > >>> object.__setattr__(c,'a',6) > Traceback (most recent call last): > File "", line 1, in ? > TypeError: function takes exactly 2 arguments (3 given) > > Is that meaningful or a problem? > I'm trying to understand type/class unification to consider possible jython > issues. > > regards, Samuele Pedroni This was a 2.2a1 specific problem: in that version, the object class didn't define a setattr dispatch slot, and object.__setattr__ ended up returning a *bound* method of the object *class*. In 2.2a2 this won't be a problem -- the object now defines __setattr__. --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Mon Aug 20 17:18:26 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 20 Aug 2001 12:18:26 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib compilerlike.py,NONE,1.1 In-Reply-To: References: Message-ID: <15233.14418.396030.108069@slothrop.digicool.com> >>>>> "ESR" == Eric S Raymond writes: ESR> --- NEW FILE: compilerlike.py --- ESR> """ compilerlike -- framework code for building compiler-like ESR> programs. ESR> There is a common `compiler-like' pattern in Unix scripts which ESR> is useful for translation utilities of all sorts. A program ESR> following this pattern behaves as a filter when no argument ESR> files are specified on the command line, but otherwise ESR> transforms each file individually into a corresponding output ESR> file. I think the name of this module is confusing as best. It doesn't have anything to do with compilation -- and the pattern you describe doesn't apply to compilers I'm familiar with (e.g. gcc). This library may be useful, but it needs a better name and short description to help people understand its use. I think there was a suggestion to integrate this code with fileinput, which seems to make sense. fileinput seems to describe what this module does -- deal with files as input. Jeremy From pf@artcom-gmbh.de Mon Aug 20 18:28:21 2001 From: pf@artcom-gmbh.de (Peter Funk) Date: Mon, 20 Aug 2001 19:28:21 +0200 (CEST) Subject: Choosing a new name for compilerlike.py (was: [Python-Dev] Re: [Python-checkins] CVS ...) In-Reply-To: <15233.14418.396030.108069@slothrop.digicool.com> from Jeremy Hylton at "Aug 20, 2001 12:18:26 pm" Message-ID: Hi, > >>>>> "ESR" == Eric S Raymond writes: > > ESR> --- NEW FILE: compilerlike.py --- > ESR> """ compilerlike -- framework code for building compiler-like > ESR> programs. > > ESR> There is a common `compiler-like' pattern in Unix scripts which > ESR> is useful for translation utilities of all sorts. A program > ESR> following this pattern behaves as a filter when no argument > ESR> files are specified on the command line, but otherwise > ESR> transforms each file individually into a corresponding output > ESR> file. Jeremy Hylton wrote: > I think the name of this module is confusing as best. I agree with Jeremy. > It doesn't have > anything to do with compilation -- and the pattern you describe > doesn't apply to compilers I'm familiar with (e.g. gcc). This library > may be useful, but it needs a better name and short description to > help people understand its use. > > I think there was a suggestion to integrate this code with fileinput, > which seems to make sense. fileinput seems to describe what this > module does -- deal with files as input. Yes, this makes some sense. On the other hand the main purpose of this module seems to be to get Unix convention to use a hyphen as alias-name for 'stdin' and 'stdout' factored out into a standard module. What about calling this 'FilterFrameWork.py'? I had quick glance into this module and I think this needs some more thoughts. Using filecmp.cmp each time to find out, whether the provided callback function has done something to the data looks costly and is not very generic. Much of the framework functionality has to do with command line argument processing, which is often coupled with option processing (fancygetopt jumps into my mind here) and in many similar scripts if have seen or written myself often these commands provide the (additional) possibility to specify either input or output or both using option arguments. For example: foobar --input=baz.in --output=baz.out or foobar -i baz.in -o baz.out So IMO this module is not yet ready for the Python standard library. Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen, Germany) From tim@zope.com Mon Aug 20 20:50:39 2001 From: tim@zope.com (Tim Peters) Date: Mon, 20 Aug 2001 15:50:39 -0400 Subject: [Python-Dev] RE: [Python-checkins] CVS: python/dist/src/Lib __future__.py,1.9.2.1,1.9.2.2 In-Reply-To: Message-ID: [Guido] > ... > I've discovered another reason not to *require* the new module: new is > not allowed in restricted mode. Without this patch, this means that > any import from __future__ would fail, and in turn e.g. "import types" > would fail. > > I'm beginning to wonder if sticking the CO_ constants in __future__ > was such a good idea... Sure it was. The problem has nothing to do with __future__, it has to do with a failure of introspection, i.e. that the CO_xxx flags weren't exposed anywhere. That also causes by-hand duplication of the magic little integers in pyassem.py. We wouldn't have any problems if I had picked some module other than "new" to host them (well, any other module that happens always to be available, unlike "new"); but "new" is the module they fit best. I could move 'em to, e.g., sys. From guido@python.org Mon Aug 20 22:53:24 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 20 Aug 2001 17:53:24 -0400 Subject: [Python-Dev] RE: [Python-checkins] CVS: python/dist/src/Lib __future__.py,1.9.2.1,1.9.2.2 In-Reply-To: Your message of "Mon, 20 Aug 2001 15:50:39 EDT." References: Message-ID: <200108202153.RAA18839@cj20424-a.reston1.va.home.com> > [Guido] > > ... > > I've discovered another reason not to *require* the new module: new is > > not allowed in restricted mode. Without this patch, this means that > > any import from __future__ would fail, and in turn e.g. "import types" > > would fail. > > > > I'm beginning to wonder if sticking the CO_ constants in __future__ > > was such a good idea... > > Sure it was. The problem has nothing to do with __future__, it has to do > with a failure of introspection, i.e. that the CO_xxx flags weren't exposed > anywhere. That also causes by-hand duplication of the magic little integers > in pyassem.py. We wouldn't have any problems if I had picked some module > other than "new" to host them (well, any other module that happens always to > be available, unlike "new"); but "new" is the module they fit best. I could > move 'em to, e.g., sys. Sys would make more sense, for sure. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com (Skip Montanaro) Mon Aug 20 23:42:21 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 20 Aug 2001 17:42:21 -0500 Subject: [Python-Dev] Improving bsddb detection in setup.py Message-ID: <15233.37453.222314.927103@beluga.mojam.com> Folks, Back about the time 2.1.1 was in the throes of final release, I submitted a patch to setup.py. Thomas didn't want to apply it that close to the release. It's still sitting there: http://sourceforge.net/tracker/?func=detail&aid=440725&group_id=5470&atid=105470 I'd like to see this or something like it get into the distribution. Sleepycat keeps changing file formats, so there are now three incompatible versions of libdb out there. It's a pain in the tush making the switch. I know there's another bsddb module floating around out there as well. If that turns out to be backward compatible with the existing bsddb module, that would be okay too. I think you're always going to have to do a fancy dance in setup.py because of the proliferation of incompatible versions of libdb. Skip From fdrake@acm.org Mon Aug 20 23:41:02 2001 From: fdrake@acm.org (Fred L. Drake) Date: Mon, 20 Aug 2001 18:41:02 -0400 (EDT) Subject: [Python-Dev] [development doc updates] Message-ID: <20010820224102.B942E28846@beowolf.digicool.com> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ Final documentation for Python 2.2 alpha 2. From barry@zope.com Mon Aug 20 23:54:28 2001 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 20 Aug 2001 18:54:28 -0400 Subject: [Python-Dev] Improving bsddb detection in setup.py References: <15233.37453.222314.927103@beluga.mojam.com> Message-ID: <15233.38180.702486.900448@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> Back about the time 2.1.1 was in the throes of final release, SM> I submitted a patch to setup.py. Thomas didn't want to apply SM> it that close to the release. It's still sitting there: SM> SM> http://sourceforge.net/tracker/?func=detail&aid=440725&group_id=5470&atid=105470 SM> I'd like to see this or something like it get into the SM> distribution. Sleepycat keeps changing file formats, so there SM> are now three incompatible versions of libdb out there. It's SM> a pain in the tush making the switch. SM> I know there's another bsddb module floating around out there SM> as well. If that turns out to be backward compatible with the SM> existing bsddb module, that would be okay too. I think you're SM> always going to have to do a fancy dance in setup.py because SM> of the proliferation of incompatible versions of libdb. Robin Dunn & Andrew Kuchling's PyBSDDB3 pybsddb.sf.net It claims to be a backwards compatible drop in for the default bsddb module, supporting back to 1.85. It's been On My List of things to look at for a while now, but I just haven't had time. I've used PyBSDDB3 for some Zope/ZODB work and it seems quite stable. I'd like it to be the default for Python 2.2 if the backwards compatibility works out acceptably. -Barry From gstein@lyra.org Tue Aug 21 02:53:23 2001 From: gstein@lyra.org (Greg Stein) Date: Mon, 20 Aug 2001 18:53:23 -0700 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0268.txt,NONE,1.1 pep-0000.txt,1.128,1.129 pep-0267.txt,1.1,1.2 In-Reply-To: ; from bwarsaw@users.sourceforge.net on Mon, Aug 20, 2001 at 05:02:28PM -0700 References: Message-ID: <20010820185322.J11073@lyra.org> I saw no 267.... A while back, when I asked about adding PEPs, the answer was, "yes, you can add them yourself if you want; just pick off the next number." So I don't want to hear about a "spank" from you :-) And per my checkin message: the rest of the content arrives tonite... Cheers, -g On Mon, Aug 20, 2001 at 05:02:28PM -0700, Barry Warsaw wrote: > Update of /cvsroot/python/python/nondist/peps > In directory usw-pr-cvs1:/tmp/cvs-serv28647 > > Modified Files: > pep-0000.txt pep-0267.txt > Added Files: > pep-0268.txt > Log Message: > Greg Stein was a bad boy! <268 spank> > > Jeremy Hylton's "Optimized Access to Module Namespaces" PEP gets 267. > > We'll let Greg slide this time, so his (albeit incomplete other than a > Rationale) PEP gets 268. It must get filled in soon though or it'll > get moved to the "Empty" category. :( >... -- Greg Stein, http://www.lyra.org/ From guido@python.org Tue Aug 21 03:35:19 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 20 Aug 2001 22:35:19 -0400 Subject: [Python-Dev] Improving bsddb detection in setup.py In-Reply-To: Your message of "Mon, 20 Aug 2001 17:42:21 CDT." <15233.37453.222314.927103@beluga.mojam.com> References: <15233.37453.222314.927103@beluga.mojam.com> Message-ID: <200108210235.WAA19541@cj20424-a.reston1.va.home.com> > Folks, > > Back about the time 2.1.1 was in the throes of final release, I submitted a > patch to setup.py. Thomas didn't want to apply it that close to the > release. It's still sitting there: > > http://sourceforge.net/tracker/?func=detail&aid=440725&group_id=5470&atid=105470 > > I'd like to see this or something like it get into the distribution. > Sleepycat keeps changing file formats, so there are now three incompatible > versions of libdb out there. It's a pain in the tush making the switch. > > I know there's another bsddb module floating around out there as well. If > that turns out to be backward compatible with the existing bsddb module, > that would be okay too. I think you're always going to have to do a fancy > dance in setup.py because of the proliferation of incompatible versions of > libdb. > > Skip Skip, now would be the perfect time to check it in. We'll try it in the alpha. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Aug 21 03:38:52 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 20 Aug 2001 22:38:52 -0400 Subject: [Python-Dev] Improving bsddb detection in setup.py In-Reply-To: Your message of "Mon, 20 Aug 2001 18:54:28 EDT." <15233.38180.702486.900448@anthem.wooz.org> References: <15233.37453.222314.927103@beluga.mojam.com> <15233.38180.702486.900448@anthem.wooz.org> Message-ID: <200108210238.WAA19589@cj20424-a.reston1.va.home.com> > Robin Dunn & Andrew Kuchling's PyBSDDB3 > > pybsddb.sf.net > > It claims to be a backwards compatible drop in for the default bsddb > module, supporting back to 1.85. It's been On My List of things to > look at for a while now, but I just haven't had time. I've used > PyBSDDB3 for some Zope/ZODB work and it seems quite stable. I'd like > it to be the default for Python 2.2 if the backwards compatibility > works out acceptably. But that requires that you have BSDDB 3, right? Because of its licensing, not everybody has it. So I think it's still a good idea to have Skip's patch as well. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Tue Aug 21 03:44:25 2001 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 20 Aug 2001 22:44:25 -0400 Subject: [Python-Dev] Improving bsddb detection in setup.py References: <15233.37453.222314.927103@beluga.mojam.com> <15233.38180.702486.900448@anthem.wooz.org> <200108210238.WAA19589@cj20424-a.reston1.va.home.com> Message-ID: <15233.51977.331659.112484@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> But that requires that you have BSDDB 3, right? Because of GvR> its licensing, not everybody has it. So I think it's still a GvR> good idea to have Skip's patch as well. No, PyBSDDB claims to be BSD 1.85 compatible. I haven't tried that yet, but I think it will work even if you have the free, old version around. -Barry From guido@python.org Tue Aug 21 03:46:06 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 20 Aug 2001 22:46:06 -0400 Subject: [Python-Dev] Improving bsddb detection in setup.py In-Reply-To: Your message of "Mon, 20 Aug 2001 22:44:25 EDT." <15233.51977.331659.112484@anthem.wooz.org> References: <15233.37453.222314.927103@beluga.mojam.com> <15233.38180.702486.900448@anthem.wooz.org> <200108210238.WAA19589@cj20424-a.reston1.va.home.com> <15233.51977.331659.112484@anthem.wooz.org> Message-ID: <200108210246.WAA19717@cj20424-a.reston1.va.home.com> > GvR> But that requires that you have BSDDB 3, right? Because of > GvR> its licensing, not everybody has it. So I think it's still a > GvR> good idea to have Skip's patch as well. > > No, PyBSDDB claims to be BSD 1.85 compatible. I haven't tried that > yet, but I think it will work even if you have the free, old version > around. So it's *both* compatible with BSD 1.95 *and* with the old bsddb module. Then what does it buy you? --Guido van Rossum (home page: http://www.python.org/~guido/) From grant.griffin@iowegian.com Tue Aug 21 04:05:52 2001 From: grant.griffin@iowegian.com (Grant Griffin) Date: Mon, 20 Aug 2001 22:05:52 -0500 Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value Message-ID: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com> PEP: 265 Title: Sorting Dictionaries by Value Version: $Revision: 0.1 $ Author: g2@iowegian.com (Grant Griffin) Status: Draft Type: Standards Track Created: 8-Aug-2001 Python-Version: 2.2 Post-History: Abstract This PEP suggests a "sort by value" operation for dictionaries. The primary benefit would be in terms of "batteries included" support for a common Python idiom which, in its current form, is both difficult for beginners to understand and cumbersome for all to implement. Motivation A common use of dictionaries is to count occurrences by setting the value of d[key] to "1" on its first occurrence, then increment the value on each subsequent occurrence. This can be done several different ways, but the "get" method is the most succinct: d[key] = d.get(key, 0) + 1 Once all occurrences have been counted, a common use of the resulting dictionary is to print the occurrences in occurrence-sorted order, often with the largest value first. This leads to a need to sort a dictionary's items by value. The canonical method of doing so in Python is to first use d.items() to get a list of the dictionary's items, then invert the ordering of each item's tuple from (key, value) into (value, key), then sort the list; since Python sorts the list based on the first item of the tuple, the list of (inverted) items is therefore sorted by value. If desired, the list can then be reversed, and the (value, key) tuples can be re- inverted back to (key, value). (However, in my experience, the inverted tuple ordering is fine for most purposes, e.g. printing out the list.) For example, given an occurrence count of: >>> d = {'a':2, 'b':23, 'c':5, 'd':17, 'e':1} we might do: >>> items = d.items() >>> items = [(v, k) for (k, v) in items] >>> items.sort() >>> items.reverse() # so largest is first >>> items = [(k, v) for (v, k) in items] resulting in: >>> items [('b', 23), ('d', 17), ('c', 5), ('a', 2), ('e', 1)] which shows the list in by-value order, largest first. (In this case, 'b' was found to have the most occurrences.) This works fine, but is "hard to use" in two aspects. First, although this idiom is known to veteran Pythoneers, it is not at all obvious to newbies--either in terms of its algorithm (inverting the ordering of item tuples) or its implementation (using list comprehensions--which are difficult for beginners.) Second, it requires having to repeatedly type a lot of "gunge", resulting in both tedium and mistakes. We therefore would rather Python provide a method of sorting dictionaries by value which would be both easy for newbies to understand (or, better yet, not to _have to_ understand) and easier for all to use. Rationale As Tim Peters has pointed out, this sort of thing brings on the problem of trying to be all things to all people. Therefore, we will limit its scope to try to hit "the sweet spot". Unusual cases (e.g. sorting via a custom comparison function) can, of course, be handled "manually" using present methods. Here are some simple possibilities: The "items" method of dictionaries can be augmented with new parameters having default values that provide for full backwards-compatibility: 1) items(sort_by_values=0, reversed=0) or maybe just: 2) items(sort_by_values=0) since reversing a list is easy enough. Alternatively, items() could simply let us control the (key, value) order: 3) items(values_first=0) Again, this is fully backwards-compatible. It does less "work" than the others, but it at least eases the most complicated/tricky part of the sort-by-value problem: inverting the order of item tuples. Using this is very simple: items = d.items(1) items.sort() items.reverse() # (if desired) The primary drawback of the preceding three approaches is the additional overhead for the parameter-less "items()" case, due to having to process default parameters. (However, if one assumes that items() gets used primarily for creating sort-by-value lists, this is not really a drawback in practice.) Alternatively, we might add a new dictionary method which somehow embodies "sorting". This approach offers two advantages. First, it avoids adding overhead to the items() method. Second, it is perhaps more accessible to newbies: when they go looking for a method for sorting dictionaries, they hopefully run into this one, and they will not have to understand the finer points of tuple inversion and list sorting to achieve sort-by-value. To allow the four basic possibilities of sorting by key/value and in forward/reverse order, we could add this method: 4) sorted_items(by_value=0, reversed=0) I believe the most common case would actually be "by_value=1, reversed=1", but the defaults values given here might lead to fewer surprises by users: sorted_items() would be the same as items() followed by sort(). Finally (as a last resort), we could use: 5) items_sorted_by_value(reversed=0) Implementation The proposed dictionary methods would necessarily be implemented in C. Presumably, the implementation would be fairly simple since it involves just adding a few calls to Python's existing machinery. Concerns Aside from the run-time overhead already addressed in possibilities 1- 3, concerns with this proposal probably will fall into the categories of "feature bloat" and/or "code bloat". However, I believe that several of the suggestions made here will result in quite minimal bloat, resulting in a good tradeoff between bloat and "value added". Tim Peters has noted that implementing this in C might not be significantly faster than implementing it in Python today. However, the major benefits intended here are "accessibility" and "ease of use", not "speed". Therefore, as long as it is not noticeably slower (in the case of plain "items()", speed need not be a consideration. References A related thread called "counting occurrences" appeared on comp.lang.python in August, 2001. This included examples of approaches to systematizing the sort-by-value problem by implementing it as reusable Python functions and classes. Copyright This document has been placed in the public domain. -------------------------------------------------------------------- Also...this just in from Tim Peters: Here's one to add to the PEP as an alternative: dict.sort(sort_specifier) where sort_specifier is one of these strings: 'kv->kv' 'kv->vk' 'kv->k' 'kv->v' 'vk->kv' 'vk->vk' 'vk->k' 'vk->v' The first 4 sort by keys first, the second 4 by values first. The string to the right of '->' says whether you want a list of (key, value) pairs, (value, key) pairs, just keys, or just values. So to sort by values and return only the keys (corresponding to increasing order of values), dict.sort('vk->k') There are two other sort_specifier possibilities: 'k' 'v' dict.sort('k') is the same as ksort(dict): def ksort(dict): x=dict.keys() x.sort() return x and similarly for dict.sort('v') returning dict.values() sorted. Reversals are the user's problem. Your comments and suggestions are welcome and appreciated. I think the two big questions for the community here are: 1) Is this useful enough to justify the feature/code bloat? (Tim has suggested that this might be a hard sell to Guido, so if you like it in whatever form, let's hear from you!) 2) Of all the many possibilities, which do you find the most useful and Pythonic? thanks-much-ly y'rs, =g2 -- _____________________________________________________________________ Grant R. Griffin g2@dspguru.com Publisher of dspGuru http://www.dspguru.com Iowegian International Corporation http://www.iowegian.com From barry@zope.com Tue Aug 21 04:58:20 2001 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 20 Aug 2001 23:58:20 -0400 Subject: [Python-Dev] Improving bsddb detection in setup.py References: <15233.37453.222314.927103@beluga.mojam.com> <15233.38180.702486.900448@anthem.wooz.org> <200108210238.WAA19589@cj20424-a.reston1.va.home.com> <15233.51977.331659.112484@anthem.wooz.org> <200108210246.WAA19717@cj20424-a.reston1.va.home.com> Message-ID: <15233.56412.2158.771422@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> So it's *both* compatible with BSD 1.95 *and* with the old GvR> bsddb module. Then what does it buy you? It will link against the newer libraries too, if you have them. -Barry From skip@pobox.com (Skip Montanaro) Tue Aug 21 05:27:36 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 20 Aug 2001 23:27:36 -0500 Subject: [Python-Dev] Improving bsddb detection in setup.py In-Reply-To: <200108210235.WAA19541@cj20424-a.reston1.va.home.com> References: <15233.37453.222314.927103@beluga.mojam.com> <200108210235.WAA19541@cj20424-a.reston1.va.home.com> Message-ID: <15233.58168.610817.101621@beluga.mojam.com> >> Back about the time 2.1.1 was in the throes of final release, I >> submitted a patch to setup.py. Thomas didn't want to apply it that >> close to the release. It's still sitting there: >> >> http://sourceforge.net/tracker/?func=detail&aid=440725&group_id=5470&atid=105470 ... Guido> Skip, now would be the perfect time to check it in. We'll try it Guido> in the alpha. He takes a deep breath, closes his eyes *very* tight, and pushes the magic pink button on the console that says "Push only in case of emergency"... Strange, nothing bad happened. At any rate, I just checked in setup.py 1.51 with the patch I had previously submitted and assigned that patch to myself. I'm fairly certain it will need some tweaking, but is almost assuredly better than 1.50. Skip From skip@pobox.com (Skip Montanaro) Tue Aug 21 05:35:45 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 20 Aug 2001 23:35:45 -0500 Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value In-Reply-To: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com> References: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com> Message-ID: <15233.58657.726456.31064@beluga.mojam.com> Grant> ... since Python sorts the list based on the first item of the Grant> tuple, the list of (inverted) items is therefore sorted by value. Well, to be picky, it uses all the elements of the tuple in the comparison, it's just that the highest priority item is in the first slot. ;-) Skip From greg@electricrain.com Tue Aug 21 05:36:16 2001 From: greg@electricrain.com (Gregory P. Smith) Date: Mon, 20 Aug 2001 21:36:16 -0700 Subject: [pybsddb] Re: [Python-Dev] Improving bsddb detection in setup.py In-Reply-To: <200108210246.WAA19717@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Aug 20, 2001 at 10:46:06PM -0400 References: <15233.37453.222314.927103@beluga.mojam.com> <15233.38180.702486.900448@anthem.wooz.org> <200108210238.WAA19589@cj20424-a.reston1.va.home.com> <15233.51977.331659.112484@anthem.wooz.org> <200108210246.WAA19717@cj20424-a.reston1.va.home.com> Message-ID: <20010820213616.A25688@zot.electricrain.com> On Mon, Aug 20, 2001 at 10:46:06PM -0400, Guido van Rossum wrote: > > GvR> But that requires that you have BSDDB 3, right? Because of > > GvR> its licensing, not everybody has it. So I think it's still a > > GvR> good idea to have Skip's patch as well. > > > > No, PyBSDDB claims to be BSD 1.85 compatible. I haven't tried that > > yet, but I think it will work even if you have the free, old version > > around. > > So it's *both* compatible with BSD 1.95 *and* with the old bsddb > module. Then what does it buy you? pybsddb only links with BerkeleyDB 3.x. It supports a bsddb module compatible interface to make upgrading apps that use the old bsddb module smoother. If you want to use the free old version you still need the old bsddb module. If you want to use the great new features (transactions, threading, cursors, etc.) available in BerkeleyDB 3 you need pybsddb. I'm not certain it makes sense to include pybsddb with python as it generally needs updating for every 3.x release of BerkeleyDB since seepycat keeps improving the library interface in minor ways. Including a mention of pybsddb and a hyperlink to the sourceforge project in the bsddb module documentation "would be a good thing" (tm). -- Gregory P. Smith gnupg/pgp: http://electricrain.com/greg/keys/ C379 1F92 3703 52C9 87C4 BE58 6CDA DB87 105D 9163 From barry@zope.com Tue Aug 21 05:54:23 2001 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 21 Aug 2001 00:54:23 -0400 Subject: [pybsddb] Re: [Python-Dev] Improving bsddb detection in setup.py References: <15233.37453.222314.927103@beluga.mojam.com> <15233.38180.702486.900448@anthem.wooz.org> <200108210238.WAA19589@cj20424-a.reston1.va.home.com> <15233.51977.331659.112484@anthem.wooz.org> <200108210246.WAA19717@cj20424-a.reston1.va.home.com> <20010820213616.A25688@zot.electricrain.com> Message-ID: <15233.59775.192458.503056@anthem.wooz.org> >>>>> "GPS" == Gregory P Smith writes: GPS> pybsddb only links with BerkeleyDB 3.x. It supports a bsddb GPS> module compatible interface to make upgrading apps that use GPS> the old bsddb module smoother. Okay, darn. Forget that idea. -Barry From pf@artcom-gmbh.de Tue Aug 21 07:43:00 2001 From: pf@artcom-gmbh.de (Peter Funk) Date: Tue, 21 Aug 2001 08:43:00 +0200 (CEST) Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value In-Reply-To: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com> from Grant Griffin at "Aug 20, 2001 10:05:52 pm" Message-ID: Grant Griffin: > PEP: 265 > Title: Sorting Dictionaries by Value [...] > 1) Is this useful enough to justify the feature/code bloat? (Tim has > suggested that this might be a hard sell to Guido, so if you like it in > whatever form, let's hear from you!) I think, this is feature bloat. IMHO using dictionaries as counters for something is only one very special application of this generic basic data structure. A standard library module containing a class derived from dictionary should be able to fullfill this 'batteries included' wish. The documentation of the builtin type dictionary could contain pointers to useful derived classes, so that newbies seeking such a feature have a better chance to find it. Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen, Germany) From content-management@high-tech-communcations.com Tue Aug 21 08:11:06 2001 From: content-management@high-tech-communcations.com (Victor Black) Date: Tue, 21 Aug 2001 00:11:06 -0700 Subject: [Python-Dev] New web utility Message-ID: <200108210711.f7L7B6J02297@mail.high-tech-communications.com>

I noticed your email address on a list serve related to technology and web development.  With your permission, we
would like to send you information regarding new web tools and utilities based on your interests.  Please click the
following link and opt-in to our product updates and e-newsletter, click here: http://216.133.228.90/cm/

Cordially,

Victor Black
High-Tech-Communications.com

If you would like to be removed from our database, please click here: http://216.133.228.90/cm/remove.cgi

 

From jack@oratrix.nl Tue Aug 21 15:56:32 2001 From: jack@oratrix.nl (Jack Jansen) Date: Tue, 21 Aug 2001 16:56:32 +0200 Subject: [Python-Dev] Putting a builtin module into a package Message-ID: <20010821145632.7CC15303181@snelboot.oratrix.nl> Is there a way to put a builtin module into a package, or should I really not want to do this? It seems that if in the module init() routine I call Py_InitModule("Carbon.Res") and if I put the module into _PyImport_Inittab as "Carbon.Res" things sort-of work, at first glance. Or should I rename the module to an underscore name and simply put a wrapper module in the package? -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From skip@pobox.com (Skip Montanaro) Tue Aug 21 16:58:34 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Tue, 21 Aug 2001 10:58:34 -0500 Subject: [Python-Dev] Putting a builtin module into a package In-Reply-To: <20010821145632.7CC15303181@snelboot.oratrix.nl> References: <20010821145632.7CC15303181@snelboot.oratrix.nl> Message-ID: <15234.34090.225332.420431@beluga.mojam.com> Jack> Or should I rename the module to an underscore name and simply put Jack> a wrapper module in the package? This seems the safest (and the sanest) to me. I realize you're thinking mostly about MacOS, but putting object files inside packages that are otherwise Python code makes the whole package platform-dependent. In general, it would prevent system administrators from installing the Lib/site-packages directory tree on a shared filesystem like NFS. In your package __init__.py file you might want to check for the existence of any object modules and raise ImportError as early as possible. Skip From DavidA@ActiveState.com Tue Aug 21 20:01:18 2001 From: DavidA@ActiveState.com (David Ascher) Date: Tue, 21 Aug 2001 12:01:18 -0700 Subject: [Python-Dev] copy, len and the like as 'object' methods? Message-ID: <3B82AFFE.652A19F5@ActiveState.com> I just came back from teaching Python on a cruise ship attended by mostly Perl folks. It was an interesting experience teaching Python with Randal in the audience =). One issue that came up is some of the lack of uniformity on what things are statements, methods and built-in functions. In the long-term version of the type/class unification work, things like int() become class constructors, which makes beautiful sense. The fact that int() calls __int__ methods fits nicely with type conversion mechanisms. However, there are a few things which still seem oddballish: copy.copy(), copy.deepcopy(), len() These basically call magic methods of their arguments (whether tp_slots or __methods__), and many languages implement them strictly as object methods. str() and repr() are a little weird -- I'm not sure which one will gain 'class constructor' status when the type/class unification work is done -- from the definition I'd say repr() should win, but the name is quite unfortunate given its new role... Guido, thoughts? Summary: Should copy, deepcopy and len be added as object methods? And if yes, how? Not all objects are copyable or measurable. Interfaces seem the right way to do this, but interfaces aren't in the plans so far that I know... What about a stringification method? --david From gstein@lyra.org Tue Aug 21 21:40:36 2001 From: gstein@lyra.org (Greg Stein) Date: Tue, 21 Aug 2001 13:40:36 -0700 Subject: [Python-Dev] PEP 268 - Extended HTTP functionality and WebDAV Message-ID: <20010821134036.J13229@lyra.org> I've completed my first draft of PEP 268 and am seeking feedback. http://python.sourceforge.net/peps/pep-0268.html My next step is to begin implementing the 'httpx' module and rejiggering the 'davlib' module. This work will occur in the /nondist/sandbox/ area (probably under a Lib subdir there). This work is intended to go into Python 2.2. Possibly appearing in time for 2.2a3, but 2.2b1 at the latest. Cheers, -g -- Greg Stein, http://www.lyra.org/ From jack@oratrix.nl Tue Aug 21 22:41:31 2001 From: jack@oratrix.nl (Jack Jansen) Date: Tue, 21 Aug 2001 23:41:31 +0200 Subject: [Python-Dev] Re: [Pythonmac-SIG] Java-Python Extension In-Reply-To: Message by Jim Harrison , Tue, 21 Aug 2001 16:27:00 -0400 , Message-ID: <20010821214136.3C982162E06@oratrix.oratrix.nl> Python-dev-ils: this just came in on the Pythonmac-SIG. Did anyone have a look at this already? Recently, Jim Harrison said: > Here's something that might have implications for cross-platform GUI > development in Python (something recently discussed on the list)... > > "JPE is a seamless, complete, and efficient integration of Java and standard > Python (C Python). JPE provides Java access to Python's native extension, > and Java's Swing to Python developers." > > http://sourceforge.net/projects/jpe/ The mind boggles... This person (at least: there seems to be only one person in the developer group right now) has apparently hung a complete Python interpreter inside a JVM and written all the glue code. I have no time to look into this further, but from a quick inspection of the CVS tree it looks as though it it reasonably complete. It definitely looks a lot better than many of the vaporware 1-person projects you can find on sourceforge... -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From barry@scottb.demon.co.uk Wed Aug 22 00:21:53 2001 From: barry@scottb.demon.co.uk (Barry Scott) Date: Wed, 22 Aug 2001 00:21:53 +0100 Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value In-Reply-To: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com> Message-ID: <000201c12a98$0fe70b00$060210ac@private> Seems to me that the problem is the sort method of lists is lacking not that dictionaries need some special smarts. I'd look to improve sort for some common cases where sort and lambda are deemed to hard. (Is using lambda a definition of hard?) If lambda is used you would solve the problem with: >>> items = d.items() >>> items.sort( lambda a, b: cmp( b[1], a[1] ) ) What if you interpret the parameter of sort: * - do as today * callable - do as today * int - sort by index into sequence * string - sort by this field of object If you had the improved sort you could: >>> items = d.items() >>> items.sort( 1 ) >>> items.reverse() BArry > For example, given an occurrence count of: > > >>> d = {'a':2, 'b':23, 'c':5, 'd':17, 'e':1} > > we might do: > > >>> items = d.items() > >>> items = [(v, k) for (k, v) in items] > >>> items.sort() > >>> items.reverse() # so largest is first > >>> items = [(k, v) for (v, k) in items] > > resulting in: > > >>> items > [('b', 23), ('d', 17), ('c', 5), ('a', 2), ('e', 1)] > From guido@python.org Wed Aug 22 01:43:33 2001 From: guido@python.org (Guido van Rossum) Date: Tue, 21 Aug 2001 20:43:33 -0400 Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value In-Reply-To: Your message of "Wed, 22 Aug 2001 00:21:53 BST." <000201c12a98$0fe70b00$060210ac@private> References: <000201c12a98$0fe70b00$060210ac@private> Message-ID: <200108220043.UAA24429@cj20424-a.reston1.va.home.com> > Seems to me that the problem is the sort method of lists is lacking > not that dictionaries need some special smarts. Hear, hear. > I'd look to improve sort for some common cases where sort and lambda > are deemed to hard. (Is using lambda a definition of hard?) > > If lambda is used you would solve the problem with: > > >>> items = d.items() > >>> items.sort( lambda a, b: cmp( b[1], a[1] ) ) > > What if you interpret the parameter of sort: > > * - do as today > * callable - do as today > * int - sort by index into sequence > * string - sort by this field of object Or better still, list.sort() wouldn't need to be changed at all if there was a standard library module to create compison routines, e.g. from __future__ import nested_scopes def cmp_by_index(i): return lambda a, b: cmp((a[i], a), (b[i], b)) # etc. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Wed Aug 22 01:48:15 2001 From: guido@python.org (Guido van Rossum) Date: Tue, 21 Aug 2001 20:48:15 -0400 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: Your message of "Tue, 21 Aug 2001 12:01:18 PDT." <3B82AFFE.652A19F5@ActiveState.com> References: <3B82AFFE.652A19F5@ActiveState.com> Message-ID: <200108220048.UAA24452@cj20424-a.reston1.va.home.com> > I just came back from teaching Python on a cruise ship attended by > mostly Perl folks. It was an interesting experience teaching Python > with Randal in the audience =). Tell us more... > One issue that came up is some of the lack of uniformity on what things > are statements, methods and built-in functions. In the long-term > version of the type/class unification work, things like int() become > class constructors, which makes beautiful sense. The fact that int() > calls __int__ methods fits nicely with type conversion mechanisms. Except that __int__ is poorly defined -- sometimes it is used as a precision-losing number cast (e.g. int(3.5) returns 3), sometimes it is an exact conversion (e.g. int("3")). > However, there are a few things which still seem oddballish: > > copy.copy(), copy.deepcopy(), len() > > These basically call magic methods of their arguments (whether tp_slots > or __methods__), and many languages implement them strictly as object > methods. Yes, it's an old Python choice. OO zealots don't like it much, but it has the advantage that you don't have to introduce methods right away. Too late to change. > str() and repr() are a little weird -- I'm not sure which one will gain > 'class constructor' status when the type/class unification work is done > -- from the definition I'd say repr() should win, but the name is quite > unfortunate given its new role... Guido, thoughts? str() has won -- in part because of its name, in part because it is the "copy constructor", returning a string input unchanged. > Summary: Should copy, deepcopy and len be added as object methods? And > if yes, how? Not all objects are copyable or measurable. Interfaces > seem the right way to do this, but interfaces aren't in the plans so far > that I know... > > What about a stringification method? I'd say leave it alone (we're getting enough complaints about "gratuitous" language changes as it is, and the changes we've committed to have very good reasons). --Guido van Rossum (home page: http://www.python.org/~guido/) From grant.griffin@iowegian.com Wed Aug 22 14:32:57 2001 From: grant.griffin@iowegian.com (Grant Griffin) Date: Wed, 22 Aug 2001 08:32:57 -0500 Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value In-Reply-To: References: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com> Message-ID: <4.3.2.7.2.20010822081617.00dcab00@pop.kcnet.com> At 08:43 AM 8/21/2001 +0200, Peter Funk wrote: >Grant Griffin: > > PEP: 265 > > Title: Sorting Dictionaries by Value >[...] > > 1) Is this useful enough to justify the feature/code bloat? (Tim has > > suggested that this might be a hard sell to Guido, so if you like it in > > whatever form, let's hear from you!) > >I think, this is feature bloat. IMHO using dictionaries as counters >for something is only one very special application of this generic >basic data structure. > >A standard library module containing a class derived from >dictionary should be able to fullfill this 'batteries included' wish. >The documentation of the builtin type dictionary could contain >pointers to useful derived classes, so that newbies seeking such >a feature have a better chance to find it. That's not a bad idea, and if that's how this turns out, I will think it had turned out well. However, the problem I see with a class-based approach is that this is such a small thing that I'm not even sure how to write a class for it that isn't more grandiose than it deserves. (Tim's suggestion might help here by expanding the scope of the idea .) In comparison, the simplest of my proposals--the one I'm favoring at the moment--is just to add a boolean parameter to items: items(values_first=0) This basically avoids the need for the tuple swap: >>> items = [(v, k) for (k, v) in items] and it's somewhat faster in the sort-by-values case, because the tuples are built in the right order rather than being built in the wrong order, then swapped. So, in terms of feature bloat, we have one new parameter. And in terms of code bloat, we have what I estimate to be 4-15 lines of C (depending on whether you optimize on speed or size.) I guess my sense is that the bloat is small enough to be worth it. thanks, =g2 From guido@python.org Wed Aug 22 15:15:37 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 22 Aug 2001 10:15:37 -0400 Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value In-Reply-To: Your message of "Wed, 22 Aug 2001 08:32:57 CDT." <4.3.2.7.2.20010822081617.00dcab00@pop.kcnet.com> References: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com> <4.3.2.7.2.20010822081617.00dcab00@pop.kcnet.com> Message-ID: <200108221415.KAA27226@cj20424-a.reston1.va.home.com> > However, the problem I see with a class-based approach is that this > is such a small thing that I'm not even sure how to write a class > for it that isn't more grandiose than it deserves. Believe me, adding a new method to a heavily-used built-in object is much more grandiose than adding a library module. In addition to "15 lines of code" we need to update the documentation, add a few lines to the test suite, add code to Jython mimicking the feature; 30 books will be a little more out of date, and so on. --Guido van Rossum (home page: http://www.python.org/~guido/) From DavidA@ActiveState.com Wed Aug 22 17:27:09 2001 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 22 Aug 2001 09:27:09 -0700 Subject: [Python-Dev] copy, len and the like as 'object' methods? References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> Message-ID: <3B83DD5D.3260BC9C@ActiveState.com> Guido van Rossum wrote: > > > I just came back from teaching Python on a cruise ship attended by > > mostly Perl folks. It was an interesting experience teaching Python > > with Randal in the audience =). > > Tell us more... Not much to tell. Randal was polite in my class and didn't interrupt too much =). There were some interesting discussions between Paul Prescod, Cameron Laird, Damian Conway, Mark-Jason Dominus, Randal and myself about Perl 6, which is looking more and more like Python from what I can tell. The most interesting bit of knowledge I learned is that the $, % and @ characters have a name -- they are sigils. And their use is going to change in Perl 6 to be consistent across contexts, as they'll be bound with the variable. Soon I may be able to say that "If Python hadn't been around, Perl 6 would be my language of choice". > Except that __int__ is poorly defined -- sometimes it is used as a > precision-losing number cast (e.g. int(3.5) returns 3), sometimes it > is an exact conversion (e.g. int("3")). Actually, that doesn't bother me -- the definition which works well is "convert yourself to an integer". The fact that the float object decided to consider that as a truncation is possibly a bug in the float object's decision, but __int__ is still 'convert to int, however you (the type) see fit'. > Yes, it's an old Python choice. OO zealots don't like it much, but it > has the advantage that you don't have to introduce methods right > away. Too late to change. I wasn't suggesting that we change it. Only that we add methods to the objects, much like we added string methods but kept the string module. > str() has won -- in part because of its name, in part because it is > the "copy constructor", returning a string input unchanged. Ah, subtle point I had missed. > I'd say leave it alone (we're getting enough complaints about > "gratuitous" language changes as it is, and the changes we've > committed to have very good reasons). Again, I wasn't considering getting rid of old builtins. We're going to be adding new methods to all of the types anyway (since they'll derive from object), so I was just suggesting that len or length be an alias for __len__. A minor issue at most. Cheers, --david From guido@python.org Wed Aug 22 17:40:45 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 22 Aug 2001 12:40:45 -0400 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: Your message of "Wed, 22 Aug 2001 09:27:09 PDT." <3B83DD5D.3260BC9C@ActiveState.com> References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> Message-ID: <200108221640.f7MGejt29123@odiug.digicool.com> > Again, I wasn't considering getting rid of old builtins. We're going to > be adding new methods to all of the types anyway (since they'll derive > from object), so I was just suggesting that len or length be an alias > for __len__. Hm, but this would violate TOOWTDI. --Guido van Rossum (home page: http://www.python.org/~guido/) From Samuele Pedroni Wed Aug 22 17:46:52 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Wed, 22 Aug 2001 18:46:52 +0200 (MET DST) Subject: [Python-Dev] PEP 252, PEP 253 and jython issues Message-ID: <200108221647.SAA28110@core.inf.ethz.ch> FYI I'm up to finish with the whole picture tour of PEP 252/PEP 253 wrt to pontential jython issues. My head has not exploded but I'm a bit worried: - on one hand for the amount of work that seems necessary on Jython side (but that's our problem) - on the other hand by some design tangle: how to mostly preserve the codebase, how to make the life for advanced users easy and natural, how to follow the spec. I hope both that I'm just not clever enough and I have overlooked things wrongly ... Here is an outline: high-probably non-issues: descr stuff up to backward compatibility with non doc/ supported stuff top elements of the type/metatype hiearchies. potential issues (design tangles): "layout/inheritance scheme for types/subclassed types": - subclassing of types (in java, in Jython) - multiple inheritance in the context of type subclassing - mro - potential mismatch between concrete subclassing (how Jython is coded and probably users hope to do subclassing) and "delegation" (how CPython is coded) - interdipendence between object behaviours in CPython, and Jython codebase and typical idioms - how super/dynamic binded behaviour should be invoked? from C/Python/Java tomorrow will follow a filled-in version. regards, Samuele Pedroni. From DavidA@ActiveState.com Wed Aug 22 17:55:54 2001 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 22 Aug 2001 09:55:54 -0700 Subject: [Python-Dev] copy, len and the like as 'object' methods? References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <200108221640.f7MGejt29123@odiug.digicool.com> Message-ID: <3B83E41A.DC0DE5D@ActiveState.com> Guido van Rossum wrote: > > > Again, I wasn't considering getting rid of old builtins. We're going to > > be adding new methods to all of the types anyway (since they'll derive > > from object), so I was just suggesting that len or length be an alias > > for __len__. > > Hm, but this would violate TOOWTDI. So do string methods =). --da From paulp@ActiveState.com Wed Aug 22 18:39:39 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 22 Aug 2001 09:39:39 -0800 Subject: [Python-Dev] copy, len and the like as 'object' methods? References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> Message-ID: <3B83EE5B.CEB38C48@ActiveState.com> David Ascher wrote: > >... > > Not much to tell. Randal was polite in my class and didn't interrupt > too much =). Randal was polite in general but there was a point where he interjected: "And you say *Perl* has a lot of special cases? Python seems to have just as many!" These "function syntaxed methods" are one example but there are others. Some of the others are convenient (e.g. special casing of non-tuples after %) and some are backwards compatibility issues. Also, every time we'd point to a feature of Python (OO syntax, exception handling, generators, *) that was clearly better than Perl, Randal claimed it was already slated to be fixed for Perl 6. We suggested he print t-shirts with that refrain. > ... There were some interesting discussions between Paul > Prescod, Cameron Laird, Damian Conway, Mark-Jason Dominus, Randal and > myself about Perl 6, which is looking more and more like Python from > what I can tell. My sense was "Second System Syndrome" but Damian Conway is confident that they don't have that problem. * from my point of view it seemed that we agreed that Python was better than Perl at almost everything but then I didn't sit in on Damian's "Data Munging" talk. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From jeremy@zope.com Wed Aug 22 18:52:06 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Wed, 22 Aug 2001 13:52:06 -0400 (EDT) Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: <3B83EE5B.CEB38C48@ActiveState.com> References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> Message-ID: <15235.61766.827878.699521@slothrop.digicool.com> >>>>> "PP" == Paul Prescod writes: PP> My sense was "Second System Syndrome" but Damian Conway is PP> confident that they don't have that problem. Isn't that one of the signs of the second system effect? <0.5 wink> Did he offer any reason for his confidence? Jeremy From James_Althoff@i2.com Wed Aug 22 19:11:51 2001 From: James_Althoff@i2.com (James_Althoff@i2.com) Date: Wed, 22 Aug 2001 11:11:51 -0700 Subject: [Python-Dev] copy, len and the like as 'object' methods? Message-ID: David Ascher wrote: >Again, I wasn't considering getting rid of old builtins. We're going to >be adding new methods to all of the types anyway (since they'll derive >from object), so I was just suggesting that len or length be an alias >for __len__. (While admittedly being an OO zealot ;-) I too would be pleased to see a len/length method. And I really enjoy using the new string methods. Thanks for adding them! :-) Jim From paulp@ActiveState.com Wed Aug 22 19:35:23 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 22 Aug 2001 10:35:23 -0800 Subject: [Python-Dev] copy, len and the like as 'object' methods? References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <200108221640.f7MGejt29123@odiug.digicool.com> Message-ID: <3B83FB6B.5AF81238@ActiveState.com> Guido van Rossum wrote: > > > Again, I wasn't considering getting rid of old builtins. We're going to > > be adding new methods to all of the types anyway (since they'll derive > > from object), so I was just suggesting that len or length be an alias > > for __len__. > > Hm, but this would violate TOOWTDI. I am not an OO zealot but I am a consistency zealot. Calling a function that just turns around and calls a method is somewhat confusing and increasingly out of step with all of the other languages that support OO. I would appreciate if you could put this on the long-term fixes list and then you could choose an appropriate time when the howls of pain would be subdued. It really isn't in the same class as the division change because it is easy to interpret old code in the old way. We could support the "preferred syntax" and the "legacy syntax" indefinitely and there would still only one RIGHT way to do it. (just like you could still throw strings as exceptions but you aren't supposed to...) -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From DavidA@ActiveState.com Wed Aug 22 19:44:36 2001 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 22 Aug 2001 11:44:36 -0700 Subject: [Python-Dev] copy, len and the like as 'object' methods? References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> Message-ID: <3B83FD94.515D871@ActiveState.com> Jeremy Hylton wrote: > > >>>>> "PP" == Paul Prescod writes: > > PP> My sense was "Second System Syndrome" but Damian Conway is > PP> confident that they don't have that problem. > > Isn't that one of the signs of the second system effect? <0.5 wink> > > Did he offer any reason for his confidence? I think the fundamental source of their confidence lay in Randal's answer when I challenged the claim that they would write translators which would automatically translate 95% of Perl5 code to Perl6: "If we're smart enough to write Perl5, we can do this". I think it glosses over a lot of important issues, but this isn't the forum to discuss it. Perl6 is being defined pretty much as the kitchen sink -- any cool feature from any cool language is going to be supported. Paul and I are skeptical about the ability to make that a coherent whole, but then again Perl doesn't really claim coherence, from what I can tell. Perl6 will have generators, coroutines but not continuations, higher-order functions, etc. etc. At the same time, they're trying to fix some of the issues which made Perl4/5 hard to learn, such as sigil mutability, etc. It's designed by committees (30 or so?), on the assumption that integration is a smaller issue than designing each 'feature' well. It's an interesting project to watch from the outside. There is also apparently a real split between people who are involved in Perl6 and those who aren't at all. I did have a very interesting chat w/ Damian Conway -- he said that Perl6 is already succeeding because it's revitalized the Perl5 community, and that there are lots of high-quality modules added to CPAN inspired by the Perl6 discussions. It also makes me a little jealous -- Perl manages to be a place where significant progress is made in the libraries as distinct from the core. In Pythonia, there is too much emphasis among the elite (us =) on adding features to the core as opposed to library modules, IMO. Related is the fact that much 'cutting-edge' work in Perl is written in Perl, while most of the cutting edge stuff in Pythonia is done in C. Some of that relates to the almost infinite mutability of the Perl engine by Perl itself, which is not a design goal of Python's -- we don't want 'just anyone' adding new keywords, for example. Still, there is room for exploration and prototyping there which we don't have such easy access to. --david From barry@zope.com Wed Aug 22 19:55:49 2001 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 22 Aug 2001 14:55:49 -0400 Subject: [Python-Dev] RELEASED: Python 2.2a2 is out! Message-ID: <15236.53.399240.678090@yyz.digicool.com> We've released Python 2.2a2, the second alpha for Python 2.2, for your enjoyment, education, and edification. Download it from: http://www.python.org/2.2/ Give it a good try, and report what breaks to the bug tracker: http://sourceforge.net/bugs/?group_id=5470 New features in this release include: - Tim Peters developed a brand new Windows installer using Wise 8.1, generously donated to us by Wise Solutions. - The floor division operator // has been added as outlined in PEP 238. The / operator still provides classic division (and will until Python 3.0) unless "from __future__ import division" is included, in which case the / operator will provide true division. - The type/class unification (PEP 252-253) was integrated into the trunk and is not so tentative any more (the exact specification of some features is still tentative). A lot of work has done on fixing bugs and adding robustness and features (performance still has to come a long way). - Lots of bug fixes, contributed patches, and other stuff. See the Misc/NEWS file in the distribution, or see the release notes on SourceForge: http://sourceforge.net/project/shownotes.php?release_id=49302 As usual, Andrew Kuchling is writing a gentle introduction to the most important changes (currently excluding type/class unification), titled "What's New in Python 2.2": http://www.amk.ca/python/2.2/ There is an introduction to the type/class unification at: http://www.python.org/2.2/descrintro.html Thanks to everybody who contributed to this release, including all the 2.2 alpha 1 testers! Enjoy! -Barry & Guido From paulp@ActiveState.com Wed Aug 22 19:56:33 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 22 Aug 2001 10:56:33 -0800 Subject: [Python-Dev] copy, len and the like as 'object' methods? References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> Message-ID: <3B840061.A55BA194@ActiveState.com> Jeremy Hylton wrote: > > >>>>> "PP" == Paul Prescod writes: > > PP> My sense was "Second System Syndrome" but Damian Conway is > PP> confident that they don't have that problem. > > Isn't that one of the signs of the second system effect? <0.5 wink> > > Did he offer any reason for his confidence? Nothing directly to that question (we ran out of time). But overall of the Perl guys said things like: a) we're got a bunch smart people working on it (Larry, Damian, Dan, etc.) b) everything we're doing has been done somewhere before so it isn't research (e.g. generators, proper exception handling, decent OO syntax, etc.) c) Perl 5 is a pretty complex system and people of similar caliber built that so why should Perl 6 be any different d) Perl 6's implementation will be so much cleaner than Perl 5's that integration of parts will be a lot easier. e) we've set up all of these working groups to parallelize development and that will help alot too. http://www.perl.com/pub/a/2000/09/perl6mail.html I think that they are way too optimistic, but there were a few things I learned that at least hinted in directions of sanity: Perl 5 support may be provided through some sort of dual interpreter model rather than trying to write a single interpreter that supports both. Dual interpreters are pretty well understood (pyperl, jpe, Perl.NET etc.) So "full backwards compatibility" may be possible using that trick. And Damian is trying to talk Larry out of a strict requirement that the Perl compiler be written in Perl. Also, as David just said, you have to remember that Python people have a much higher bar of coherence to jump over than the Perl guys. If they put a bunch of random stuff in the language in a way that didn't really feel right together -- then it would be just like Perl 5 only with more features. Whereas we spend a week debating how the 'yield' keyword should interact with the 'def' keyword. Damian also said that Perl 6 has to be faster than Perl 5 or nobody will use it. Neil Bauman asked him why Perl wasn't being written in Perl (Neil's opinion is that Perl is literally the greatest language ever created). Damian responded with a great quote: "Performance. If Performance wasn't an issue, I'd probably write Perl 6 in Python or something like it with wonderfully beautiful OO abstractions and it would run like a dog." -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From akuchlin@mems-exchange.org Wed Aug 22 20:07:46 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 22 Aug 2001 15:07:46 -0400 Subject: [Python-Dev] Library development In-Reply-To: <3B83FD94.515D871@ActiveState.com>; from DavidA@ActiveState.com on Wed, Aug 22, 2001 at 11:44:36AM -0700 References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> <3B83FD94.515D871@ActiveState.com> Message-ID: <20010822150746.A21476@ute.mems-exchange.org> On Wed, Aug 22, 2001 at 11:44:36AM -0700, David Ascher wrote: >which would automatically translate 95% of Perl5 code to Perl6: "If >we're smart enough to write Perl5, we can do this". I think it glosses "We let our code base deteroriate into an incomprehensible mess of code, and damn it, we can do it again!" >libraries as distinct from the core. In Pythonia, there is too much >emphasis among the elite (us =) on adding features to the core as >opposed to library modules, IMO. Related is the fact that much >'cutting-edge' work in Perl is written in Perl, while most of the >cutting edge stuff in Pythonia is done in C. Some of that relates to *Yes*. That's mostly why I've been drifting away from python-dev, too; the core is becoming more and more dull, the new features are more and more esoteric, and the more interesting action is in Python-based applications and libraries. As a bonus, if I work on a standalone package, I don't need to worry if a change is going to screw up the X thousands of Python users; only the much smaller pool of users of the package is at risk, and it's easier to decide to break things or provide backwards compatibility. >new keywords, for example. Still, there is room for exploration and >prototyping there which we don't have such easy access to. Tools/compiler lets you try out language modifications, though, so it's equally possible to do that with Python (witness PTL). We just don't do that very much, that's all. --amk From barry@scottb.demon.co.uk Wed Aug 22 20:19:40 2001 From: barry@scottb.demon.co.uk (Barry Scott) Date: Wed, 22 Aug 2001 20:19:40 +0100 Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value In-Reply-To: <200108221415.KAA27226@cj20424-a.reston1.va.home.com> Message-ID: <000701c12b3f$642ab8e0$060210ac@private> Of course you'd still have to update the docs to point the folks to the useful comparison functions to use with sort. Barry > -----Original Message----- > From: python-dev-admin@python.org [mailto:python-dev-admin@python.org]On > Behalf Of Guido van Rossum > Sent: 22 August 2001 15:16 > To: Grant Griffin > Cc: Peter Funk; python-dev@python.org > Subject: Re: [Python-Dev] PEP 265 - Sorting Dictionaries by Value > > > > However, the problem I see with a class-based approach is that this > > is such a small thing that I'm not even sure how to write a class > > for it that isn't more grandiose than it deserves. > > Believe me, adding a new method to a heavily-used built-in object is > much more grandiose than adding a library module. In addition to "15 > lines of code" we need to update the documentation, add a few lines to > the test suite, add code to Jython mimicking the feature; 30 books > will be a little more out of date, and so on. > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > From barry@scottb.demon.co.uk Wed Aug 22 20:25:05 2001 From: barry@scottb.demon.co.uk (Barry Scott) Date: Wed, 22 Aug 2001 20:25:05 +0100 Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value In-Reply-To: <200108220043.UAA24429@cj20424-a.reston1.va.home.com> Message-ID: <000801c12b40$25ae3640$060210ac@private> Oh I like that! > Or better still, list.sort() wouldn't need to be changed at all if > there was a standard library module to create compison routines, e.g. > > from __future__ import nested_scopes > > def cmp_by_index(i): > return lambda a, b: cmp((a[i], a), (b[i], b)) How should all the variations be handled? * uses a naming convention and create the functions one by one? cmp_by_index_accending - sort low to high at index n cmp_by_index_decending - sort high to low at index n cmp_by_field_accending - sort low to high at field n cmp_by_field_decending - sort high to low at field n ... case blind, case sensitive versions... * have a make_cmp function that takes a description and build the cmp function on demand? Would the description be a string with keywords in it? "index=4,accending field=5,case-blind,decending" sort by index 4 accending then by field=5 case blind comparison. Barry From guido@python.org Wed Aug 22 20:38:25 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 22 Aug 2001 15:38:25 -0400 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: Your message of "Wed, 22 Aug 2001 09:39:39 -0800." <3B83EE5B.CEB38C48@ActiveState.com> References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> Message-ID: <200108221938.f7MJcPj29382@odiug.digicool.com> > My sense was "Second System Syndrome" but Damian Conway is confident > that they don't have that problem. Yeah, right. It has second system syndrome written all over it. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Wed Aug 22 20:38:33 2001 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 22 Aug 2001 15:38:33 -0400 Subject: [Python-Dev] copy, len and the like as 'object' methods? References: Message-ID: <15236.2617.169057.464118@anthem.wooz.org> >>>>> "JA" == James Althoff writes: JA> (While admittedly being an OO zealot ;-) I too would be JA> pleased to see a len/length method. I have to admit I miss them too sometimes. ;) From Greg.Wilson@baltimore.com Wed Aug 22 20:36:21 2001 From: Greg.Wilson@baltimore.com (Greg Wilson) Date: Wed, 22 Aug 2001 15:36:21 -0400 Subject: [Python-Dev] Re: copy, len and the like as 'object' methods? Message-ID: <930BBCA4CEBBD411BE6500508BB3328F3A8ECC@nsamcanms1.ca.baltimore.com> FWIW, David's 'len()' methods would definitely make Python easier to teach --- just about every beginner types 'x.len()' instead of 'len(x)' over and over during their first two or three exercises. Greg ----------------------------------------------------------------------------------------------------------------- The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error or there are any problems please notify the originator immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Baltimore Technologies plc will not be liable for direct, special, indirect or consequential damages arising from alteration of the contents of this message by a third party or as a result of any virus being passed on. In addition, certain Marketing collateral may be added from time to time to promote Baltimore Technologies products, services, Global e-Security or appearance at trade shows and conferences. This footnote confirms that this email message has been swept by Baltimore MIMEsweeper for Content Security threats, including computer viruses. From guido@python.org Wed Aug 22 20:46:01 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 22 Aug 2001 15:46:01 -0400 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: Your message of "Wed, 22 Aug 2001 11:11:51 PDT." References: Message-ID: <200108221946.f7MJk1j29412@odiug.digicool.com> > David Ascher wrote: > >Again, I wasn't considering getting rid of old builtins. We're going to > >be adding new methods to all of the types anyway (since they'll derive > >from object), so I was just suggesting that len or length be an alias > >for __len__. James Althoff: > (While admittedly being an OO zealot ;-) I too would be pleased to see a > len/length method. And I really enjoy using the new string methods. > Thanks for adding them! :-) While it will admittedly take a while, the intention of string methods was to eventually phase out the string module. So TOOWTDI is only violated during the transition. If we decide to add a len() method, we'll also have to decide to (eventually) get rid of len(), or we'd violate TOOWTDI *permanently*. Personally, I don't think that the "problem" this change is addressing is worth asking everybody to change their code -- so we'd have live with len(x) and x.len() as viable ways to ask for x's length *forever*, violating TOOWTDI permanently. I don't like that. (You may disagree, but at least I have thought about it and I can explain my motivation. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From dan@sidhe.org Wed Aug 22 20:57:34 2001 From: dan@sidhe.org (Dan Sugalski) Date: Wed, 22 Aug 2001 15:57:34 -0400 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: <200108221938.f7MJcPj29382@odiug.digicool.com> References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> Message-ID: <5.1.0.14.0.20010822155714.08b95330@tuatha.sidhe.org> At 03:38 PM 8/22/2001 -0400, Guido van Rossum wrote: > > My sense was "Second System Syndrome" but Damian Conway is confident > > that they don't have that problem. > >Yeah, right. It has second system syndrome written all over it. :-) You underestimate the amount of black-hearted cynicism we can muster. ;-P Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From dan@sidhe.org Wed Aug 22 20:57:01 2001 From: dan@sidhe.org (Dan Sugalski) Date: Wed, 22 Aug 2001 15:57:01 -0400 Subject: [Python-Dev] Library development In-Reply-To: <20010822150746.A21476@ute.mems-exchange.org> References: <3B83FD94.515D871@ActiveState.com> <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> <3B83FD94.515D871@ActiveState.com> Message-ID: <5.1.0.14.0.20010822155219.08c3a4b0@tuatha.sidhe.org> At 03:07 PM 8/22/2001 -0400, Andrew Kuchling wrote: >On Wed, Aug 22, 2001 at 11:44:36AM -0700, David Ascher wrote: > >which would automatically translate 95% of Perl5 code to Perl6: "If > >we're smart enough to write Perl5, we can do this". I think it glosses > > "We let our code base deteroriate into an incomprehensible >mess of code, and damn it, we can do it again!" Now, now, let's not throw stones. :) >*Yes*. That's mostly why I've been drifting away from python-dev, >too; the core is becoming more and more dull, the new features are >more and more esoteric, and the more interesting action is in >Python-based applications and libraries. As a bonus, if I work on a >standalone package, I don't need to worry if a change is going to >screw up the X thousands of Python users; only the much smaller pool >of users of the package is at risk, and it's easier to decide to break >things or provide backwards compatibility. Is this a bad thing, though? C's not changed much in the past decade or two, but look at how much more canned functionality's available now than in, say, 1980. (Heck, add Perl, Python, and Ruby to the list of C's canned functionality if you like) Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From dan@sidhe.org Wed Aug 22 20:51:59 2001 From: dan@sidhe.org (Dan Sugalski) Date: Wed, 22 Aug 2001 15:51:59 -0400 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: <3B83FD94.515D871@ActiveState.com> References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> Message-ID: <5.1.0.14.0.20010822153946.08c02de8@tuatha.sidhe.org> At 11:44 AM 8/22/2001 -0700, David Ascher wrote: >Jeremy Hylton wrote: > > > > >>>>> "PP" == Paul Prescod writes: > > > > PP> My sense was "Second System Syndrome" but Damian Conway is > > PP> confident that they don't have that problem. > > > > Isn't that one of the signs of the second system effect? <0.5 wink> > > > > Did he offer any reason for his confidence? > >I think the fundamental source of their confidence lay in Randal's >answer when I challenged the claim that they would write translators >which would automatically translate 95% of Perl5 code to Perl6: "If >we're smart enough to write Perl5, we can do this". I think it glosses >over a lot of important issues, but this isn't the forum to discuss it. Randal's confidence is based in some part on the fact that he's watching other people do it, and it's a lot easier to be confident that way. :) I, on the other hand, am nervous as all hell, honestly. (OTOH I feel the same way about raising my kids) Perl 5 to perl 6 isn't that tough though (we do chunks of Python and Ruby to perl 5, after all), and we do have a full-featured perl 5 parser handy. It's as much a task of either reimplementing the perl 5 C-based parser in perl as part of the perl 6 parser front-end (Non-trivial, but not rocket science either) or build a perl 5 bytecode->perl 6 bytecode translator, which is significantly easier. Or just teach the perl 5 parser to spit out a syntax tree, which it pretty much does already. >Perl6 is being defined pretty much as the kitchen sink -- any cool >feature from any cool language is going to be supported. > >It's designed by committees (30 or so?), on the >assumption that integration is a smaller issue than designing each >'feature' well. Forgive the snippage, but the above two quotes are directly related. Perl 6 the language is *not* being designed by committees, though parts of it look that way from the outside. There's exactly one person responsible for the design of perl 6--Larry. Damian's feeding a lot of interesting ideas to him, but Larry's ultimately responsible for putting together a coherent whole. The committee setup at the beginning was designed to let the people who had an interest in a particular topic get together and flesh out a proposal or four for Larry, but we made no promises that the proposals would see the light of day unchanged in the final language design. The internals design is being handled by a small group of folks, about six, each of us with a different area of competence. We're working things out so we have some good walls between components so they can be implemented semi-separately and in parallel. Part of me would like to have the "design by committee" feel stand until we pull the rabbit out of the hat later, but that might convince people that committee design can actually work in large projects and, well, we have enough of that nonsense around as it is. >I did have a very interesting chat w/ Damian Conway -- he said that >Perl6 is already succeeding because it's revitalized the Perl5 >community, FWIW, the success does spread beyond perl. If nothing else, there's a fair amount of "Perl can do that? Well, then, so can *my* language!" feeling amongst folks who feel strongly about languages and such. (Can't imagine who that'd be... :) Perl's doing more, which is prompting folks to do more in other languages, as a tit for tat thing if nothing else. And most any time you add functionality, everybody wins. Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From skip@pobox.com (Skip Montanaro) Wed Aug 22 21:01:30 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 22 Aug 2001 15:01:30 -0500 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: <3B83FD94.515D871@ActiveState.com> References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> <3B83FD94.515D871@ActiveState.com> Message-ID: <15236.3994.507401.287832@beluga.mojam.com> David> It also makes me a little jealous -- Perl manages to be a place David> where significant progress is made in the libraries as distinct David> from the core. In Pythonia, there is too much emphasis among the David> elite (us =) on adding features to the core as opposed to library David> modules, IMO. Related is the fact that much 'cutting-edge' work David> in Perl is written in Perl, while most of the cutting edge stuff David> in Pythonia is done in C. I think there is a lot of emphasis in the Python community on getting the platform "right". That leads to long debates about the correct way to do division, for instance. Also, Python is more platform-independent than Perl, so more work is necessary in the core to make and keep it that way. In addition, check out "man perlguts". The first statement begins This document attempts to describe how to use the Perl API, ... No hedging is necessary in either the Python C API manual or the Python Extending and Embedding manual. In fact, in the Extending and Embedding manual begins It is quite easy to add new built-in modules to Python, if you know how to program in C. So the problem you describe is is clearly Guido fault. He made Python's internals too clean. If he would have obfuscated the internals more, more people would have been drawn to the libraries (or repelled by the internals). ;-) On the flip side, I also think there is a lot of cool stuff going on in the library sector. You don't hear about it as much in part because Python doesn't have an equivalent to CPAN. I'm not talking just about the cataloging part (Vaults of Parnassus), but the build part (distutils) and automatic mirroring/dependency checking/download part (for which Python has, as yet, nothing remotely like it). That makes the barrier to use of libraries (and thus use by a critical mass) not in the core that much higher. There are how many Python Server Pages like things? They probably all have some very nifty features, but tend to be used by rather small pockets of people, at least relative to the overall segment of the Python community doing webish stuff. I'm not entirely sure why that's the case, but I suspect it has to do in part with the barrier to discovering, downloading, and installing these packages. Skip From paulp@ActiveState.com Wed Aug 22 21:45:57 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 22 Aug 2001 13:45:57 -0700 Subject: [Python-Dev] More feedback from script scape Message-ID: <3B841A05.F292DD5B@ActiveState.com> Something else I heard at script scape is that for simple extensions (wrappers around C code) Perl has not only caught up to Python but surpassed it thanks to inline::C, inline::Python etc. As soon as you move to doing anything complicated with the interpreter internals, Python wins again, but it still annoys me that Perl has leapfrogged us in wrapping simplicity! -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From paulp@ActiveState.com Wed Aug 22 22:03:42 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 22 Aug 2001 14:03:42 -0700 Subject: [Python-Dev] Perl5->Perl6 References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> <5.1.0.14.0.20010822153946.08c02de8@tuatha.sidhe.org> Message-ID: <3B841E2E.C3B462C8@ActiveState.com> Dan Sugalski wrote: > >... > > Perl 5 to perl 6 isn't that tough though (we do chunks of Python and Ruby > to perl 5, after all), and we do have a full-featured perl 5 parser handy. You guys are really focused on the syntax, but to me, the difficult things to handle are subtle *semantic* changes. That's why I said in another message that a two-interpreter system might be the easiest route. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From dan@sidhe.org Wed Aug 22 22:17:12 2001 From: dan@sidhe.org (Dan Sugalski) Date: Wed, 22 Aug 2001 17:17:12 -0400 Subject: [Python-Dev] Perl5->Perl6 In-Reply-To: <3B841E2E.C3B462C8@ActiveState.com> References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> <5.1.0.14.0.20010822153946.08c02de8@tuatha.sidhe.org> Message-ID: <5.1.0.14.0.20010822170534.08c50170@tuatha.sidhe.org> At 02:03 PM 8/22/2001 -0700, Paul Prescod wrote: >Dan Sugalski wrote: > > > >... > > > > Perl 5 to perl 6 isn't that tough though (we do chunks of Python and Ruby > > to perl 5, after all), and we do have a full-featured perl 5 parser handy. > >You guys are really focused on the syntax, but to me, the difficult >things to handle are subtle *semantic* changes. That's why I said in >another message that a two-interpreter system might be the easiest >route. Well, as I've had some Python (and Java) folks happily point out, perl doesn't have any well-defined semantics anyway. :) Seriously, we've a reasonably comprehensive test suite now with perl 5, and a very large code base with expected behavior to draw on to test the conversion. Much of the perl 5->perl 6 transition is strictly syntactic. It's possible that some of perl 5's undefined edge behaviour won't translate, but that'll only be because Larry's decided not to make it translate. Anything else is a bug, and one to be squashed. (We're not losing turing-completeness--perfect execution of past behaviour isn't unattainable) This isn't, after all, the first, or second, or even third time we've done this. The version number's 6 for a reason... Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From gstein@lyra.org Wed Aug 22 23:29:42 2001 From: gstein@lyra.org (Greg Stein) Date: Wed, 22 Aug 2001 15:29:42 -0700 Subject: huge sig (was: Re: [Python-Dev] Re: copy, len and the like as 'object' methods?) In-Reply-To: <930BBCA4CEBBD411BE6500508BB3328F3A8ECC@nsamcanms1.ca.baltimore.com>; from Greg.Wilson@baltimore.com on Wed, Aug 22, 2001 at 03:36:21PM -0400 References: <930BBCA4CEBBD411BE6500508BB3328F3A8ECC@nsamcanms1.ca.baltimore.com> Message-ID: <20010822152941.E15670@lyra.org> Euh... that .sig is *way* bigger than your message. And it doesn't really apply to a public forum, now does it? :-) Cheers, -g On Wed, Aug 22, 2001 at 03:36:21PM -0400, Greg Wilson wrote: > FWIW, David's 'len()' methods would definitely make Python easier > to teach --- just about every beginner types 'x.len()' instead of > 'len(x)' over and over during their first two or three exercises. > > Greg > > > ----------------------------------------------------------------------------------------------------------------- > The information contained in this message is confidential and is intended > for the addressee(s) only. If you have received this message in error or > there are any problems please notify the originator immediately. The > unauthorized use, disclosure, copying or alteration of this message is > strictly forbidden. Baltimore Technologies plc will not be liable for direct, > special, indirect or consequential damages arising from alteration of the > contents of this message by a third party or as a result of any virus being > passed on. > > In addition, certain Marketing collateral may be added from time to time to > promote Baltimore Technologies products, services, Global e-Security or > appearance at trade shows and conferences. > > This footnote confirms that this email message has been swept by > Baltimore MIMEsweeper for Content Security threats, including > computer viruses. > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev -- Greg Stein, http://www.lyra.org/ From aahz@rahul.net Wed Aug 22 23:48:54 2001 From: aahz@rahul.net (Aahz Maruch) Date: Wed, 22 Aug 2001 15:48:54 -0700 (PDT) Subject: [Python-Dev] Deprecation of the string module In-Reply-To: <200108221946.f7MJk1j29412@odiug.digicool.com> from "Guido van Rossum" at Aug 22, 2001 03:46:01 PM Message-ID: <20010822224855.31B8FE8C7@waltz.rahul.net> Guido van Rossum wrote: > > While it will admittedly take a while, the intention of string methods > was to eventually phase out the string module. So TOOWTDI is only > violated during the transition. I'm glad to see that you're using the past tense with regard to your intentions for the string module. (I'm phrasing this as a joke, but I'm utterly serious.) -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine. From barry@zope.com Wed Aug 22 23:53:05 2001 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 22 Aug 2001 18:53:05 -0400 Subject: [Python-Dev] More feedback from script scape References: <3B841A05.F292DD5B@ActiveState.com> Message-ID: <15236.14289.311782.621168@anthem.wooz.org> >>>>> "PP" == Paul Prescod writes: PP> Something else I heard at script scape is that for simple PP> extensions (wrappers around C code) Perl has not only caught PP> up to Python but surpassed it thanks to inline::C, PP> inline::Python etc. As soon as you move to doing anything PP> complicated with the interpreter internals, Python wins again, PP> but it still annoys me that Perl has leapfrogged us in PP> wrapping simplicity! How about a PEP, Paul?! I've no clue what inline::C or inline::Python do, and no time to learn about it, but if you have a good grasp of them, try to think about how those ideas might translate to Python. A PEP on simplifying extension writing would be welcome. -Barry From paulp@ActiveState.com Wed Aug 22 23:53:53 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 22 Aug 2001 15:53:53 -0700 Subject: [Python-Dev] copy, len and the like as 'object' methods? References: <200108221946.f7MJk1j29412@odiug.digicool.com> Message-ID: <3B843801.BA1D8541@ActiveState.com> Guido van Rossum wrote: > >... > > While it will admittedly take a while, the intention of string methods > was to eventually phase out the string module. So TOOWTDI is only > violated during the transition. Why wouldn't a similar strategy work for length and string attributes? >... > Personally, I don't think that the "problem" this change is addressing > is worth asking everybody to change their code -- so we'd have live > with len(x) and x.len() as viable ways to ask for x's length > *forever*, violating TOOWTDI permanently. I don't like that. How was the string methods problem more serious than this one? I'm not trying to be difficult: I really don't perceive a difference. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From ping@lfw.org Thu Aug 23 00:05:23 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Wed, 22 Aug 2001 16:05:23 -0700 (PDT) Subject: [Python-Dev] Inline code in other languages In-Reply-To: <15236.14289.311782.621168@anthem.wooz.org> Message-ID: On Wed, 22 Aug 2001, Barry A. Warsaw wrote: > I've no clue what inline::C or inline::Python do, I went and checked this out after reading Paul's message. It seems the basic idea is that the Inline module will take a string containing C code, parse out the function declarations, generate the appropriate wrapper functions, compile an extension module to produce a .so file in a temporary directory, and dynamically load it on the spot. Functions with convertible argument and return types become subroutines callable from Perl. Quite slick. The end result is that you can say: use Inline C => 'void greet() { printf("Hello world!\n"); }'; greet; ...and have this work. This takes a long time the first time you do it, but the text of the inline code is MD5-hashed and the .so file is left lying around, so it's fast the next time around. It should be pretty feasible to do this for Python, i think, provided the sys module provides enough details about how Python was compiled and installed. -- ?!ng From paulp@ActiveState.com Thu Aug 23 00:10:58 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 22 Aug 2001 16:10:58 -0700 Subject: [Python-Dev] More feedback from script scape References: <3B841A05.F292DD5B@ActiveState.com> <15236.14289.311782.621168@anthem.wooz.org> Message-ID: <3B843C02.6EFEC933@ActiveState.com> "Barry A. Warsaw" wrote: > >... > > How about a PEP, Paul?! > > I've no clue what inline::C or inline::Python do, and no time to learn > about it, but if you have a good grasp of them, try to think about how > those ideas might translate to Python. A PEP on simplifying extension > writing would be welcome. I have only a superficial understanding which I can convey in this email. If you think my superficial understanding is worth PEP-ing, then I can do that. http://inline.perl.org/ Okay, remember in the old days when you did inline assembly in C code? You did that because assembly was a speedup over C and there were a few things you could do in assembly but not in C (e.g. talk directly to devices and registers by memory address). Fast forward to the present. Why do you use C extensions? 1. Speedup 2. Direct access to stuff that has a C API. So why not inline C in Python (or Perl)? use Inline C => <<'END_C'; void greet() { printf("Hello, world\n"); } END_C greet(); This does what you would expect. But what's going on under the hood? 1. Inline pseudo-parses the C code and looks for function and parameter declarations. 2. Inline generates the wrapper code that Python programmers usually either write by hand or get SWIG to do. 3. Inline generates a dynamic library in a special "cache" directory using MakeMaker, the system C compiler, etc. 4. Inline loads the library it just created as a Perl extension module. You (the user) invoke all of this stuff just by running the program. Inline doesn't recompile over and over again because it uses MD5 to recognize that the thing has already been compiled. Inline does have a strategy for dealing with modules that is supposed to be distributed instead of just used locally. It essentially compiles the code once and then you ask an external program to package it as a source or binary distribution. I suggest you skim the examples in this article: http://www.perl.com/pub/a/2001/02/inline.html You'll see: * calling win32 functions without using distutils or MakeMaker explicitly * writing a C program that behaves as if it were a script (no build step) * embedding Python in Perl and other weird stuff that will make you feel like your "head has just been wrapped around a brick". (quote from the article) -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From paulp@ActiveState.com Thu Aug 23 00:19:19 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 22 Aug 2001 16:19:19 -0700 Subject: [Python-Dev] Inline code in other languages References: Message-ID: <3B843DF7.31198022@ActiveState.com> Ka-Ping Yee wrote: > >... > > It should be pretty feasible to do this for Python, i think, provided > the sys module provides enough details about how Python was compiled > and installed. I agree! The fundamental advantage of inline is that you don't have to worry about refcounts and PyObjects but you also don't have to learn to use SWIG. (SWIG might be helpful under the covers as the wrapper generator technology...) It also removes the whole build step from projects that want to wrap C code. This makes C extension wrapping much more Pythonic. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From guido@python.org Thu Aug 23 00:17:33 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 22 Aug 2001 19:17:33 -0400 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: Your message of "Wed, 22 Aug 2001 15:53:53 PDT." <3B843801.BA1D8541@ActiveState.com> References: <200108221946.f7MJk1j29412@odiug.digicool.com> <3B843801.BA1D8541@ActiveState.com> Message-ID: <200108222317.TAA30578@cj20424-a.reston1.va.home.com> > > While it will admittedly take a while, the intention of string methods > > was to eventually phase out the string module. So TOOWTDI is only > > violated during the transition. > > Why wouldn't a similar strategy work for length and string attributes? It hardly worked for string methods (see Aahz's response). > >... > > Personally, I don't think that the "problem" this change is addressing > > is worth asking everybody to change their code -- so we'd have live > > with len(x) and x.len() as viable ways to ask for x's length > > *forever*, violating TOOWTDI permanently. I don't like that. > > How was the string methods problem more serious than this one? It was motivated by the advent of Unicode, which required us to think more OO. AFAICT, the len() method proposal is just an attempt to be more OO, and to avoid being made fun of by a bunch of Perlers. I don't care about #1, and it's naive to think that solving one issue will avoid #2. Any change like this has a *huge* cost to the community (just reread the division thread). > I'm not trying to be difficult: I really don't perceive a difference. I do -- see above. --Guido van Rossum (home page: http://www.python.org/~guido/) From James_Althoff@i2.com Thu Aug 23 00:52:40 2001 From: James_Althoff@i2.com (James_Althoff@i2.com) Date: Wed, 22 Aug 2001 16:52:40 -0700 Subject: [Python-Dev] copy, len and the like as 'object' methods? Message-ID: Guido wrote: >AFAICT, the len() method proposal is just an attempt to be more OO, >and to avoid being made fun of by a bunch of Perlers. I don't care >about #1, and it's naive to think that solving one issue will avoid >#2. I agree that it isn't so important to be swayed by #1 and #2 above. But there is a #3. I've introduced over fifty programmers (coming from Java) to Jython and a high percentage trip over this initially (momentarily, at least ;-). It doesn't take much to get past it, obviously, but it's a little nicer not to stumble to begin with. (my final 2 cents worth :-) Jim From paulp@ActiveState.com Thu Aug 23 00:54:21 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 22 Aug 2001 16:54:21 -0700 Subject: [Python-Dev] copy, len and the like as 'object' methods? References: <200108221946.f7MJk1j29412@odiug.digicool.com> <3B843801.BA1D8541@ActiveState.com> <200108222317.TAA30578@cj20424-a.reston1.va.home.com> Message-ID: <3B84462D.230FFF3F@ActiveState.com> Guido van Rossum wrote: > >... > > AFAICT, the len() method proposal is just an attempt to be more OO, > and to avoid being made fun of by a bunch of Perlers. I don't care > about #1, and it's naive to think that solving one issue will avoid > #2. OO is merely accidental. The real goal is to be more consistent. Python is a language that uses methods for polymorphism so consistency pushes towards methods. If Python were Scheme and had an OO appendage in the middle of its standard library I'd dislike that too. The only reason Perlers come into the picture is because they are pointing out that we claim to be consistent but in this case are not. And the only reason any of us care about consistency is because inconsistency has a cost in education and memorization...and according to Greg Wilson he's seen it first-hand in this case. (I personally had more experience with confusion over string functions instead of string methods) > Any change like this has a *huge* cost to the community (just reread > the division thread). Division is a special case. There is a big difference between re-purposing existing syntax and adding new methods to existing objects. If one is clearly the "new, better" way and the other is historical, then the books document the new way and most users eventually forget that the old way even exists. e.g. string exceptions, string.functions, UserDict (after type-class), regex, etc. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From ping@lfw.org Thu Aug 23 01:04:23 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Wed, 22 Aug 2001 17:04:23 -0700 (PDT) Subject: [Python-Dev] Re: copy, len and the like as 'object' methods? In-Reply-To: <3B84462D.230FFF3F@ActiveState.com> Message-ID: On Wed, 22 Aug 2001, Paul Prescod wrote: > OO is merely accidental. The real goal is to be more consistent. Python > is a language that uses methods for polymorphism so consistency pushes > towards methods. Hmm. And what of, say, abs(-3)? Should that be -3.abs()? Would you also argue for [1, 2, 3].repr() and "abc".hash()? Where do you draw the line? Maybe my habits are too ingrained at this point, but i kind of like the boundary between __methods__ and ordinary methods... it would somehow bug me to have all these methods like abs, len, hash crowding in on my ordinary-method namespace. -- ?!ng From barry@zope.com Thu Aug 23 01:52:25 2001 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 22 Aug 2001 20:52:25 -0400 Subject: [Python-Dev] Re: Inline code in other languages References: <15236.14289.311782.621168@anthem.wooz.org> Message-ID: <15236.21449.231509.725435@anthem.wooz.org> >>>>> "KY" == Ka-Ping Yee writes: KY> ...and have this work. This takes a long time the first time KY> you do it, but the text of the inline code is MD5-hashed and KY> the .so file is left lying around, so it's fast the next time KY> around. KY> It should be pretty feasible to do this for Python, i think, KY> provided the sys module provides enough details about how KY> Python was compiled and installed. Slick indeed! I don't see much that's beyond Python's grasp. The syntax of course would be more Pythonic, but you could probably put all this in a module, hook it up with distutils, and hack out something rough in a day or two. Tim will port it to Windows along with a frozen cygwin gcc compiler in a huge frozen binary. Viola! -Barry From barry@zope.com Thu Aug 23 01:55:25 2001 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 22 Aug 2001 20:55:25 -0400 Subject: [Python-Dev] More feedback from script scape References: <3B841A05.F292DD5B@ActiveState.com> <15236.14289.311782.621168@anthem.wooz.org> <3B843C02.6EFEC933@ActiveState.com> Message-ID: <15236.21629.394538.234568@anthem.wooz.org> >>>>> "PP" == Paul Prescod writes: PP> I have only a superficial understanding which I can convey in PP> this email. If you think my superficial understanding is worth PP> PEP-ing, then I can do that. >From what you and Ping described, I doubt it needs to be PEP'd. It could all be done in a module, without any impact on the language I'd imagine. -Barry From ping@lfw.org Thu Aug 23 02:02:46 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Wed, 22 Aug 2001 18:02:46 -0700 (PDT) Subject: [Python-Dev] Re: Inline code in other languages In-Reply-To: <15236.21449.231509.725435@anthem.wooz.org> Message-ID: > Slick indeed! I don't see much that's beyond Python's grasp. The > syntax of course would be more Pythonic, Perhaps: import inline inline.C(''' void greet(char* str) { printf("%s, %s, %s, lovely %s\n", str, str, str, str); } ''') from inline.C import greet greet('spam') ...? -- ?!ng From paulp@ActiveState.com Thu Aug 23 02:08:40 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 22 Aug 2001 18:08:40 -0700 Subject: [Python-Dev] Re: copy, len and the like as 'object' methods? References: Message-ID: <3B845798.7505DC91@ActiveState.com> Ka-Ping Yee wrote: > >... > > Hmm. And what of, say, abs(-3)? Should that be -3.abs()? The former is more in-line with traditional mathematical notation. I think that counts for something. And anyhow, abs is not fully polymorphic so it is a little bit of a different case. > Would you also argue for [1, 2, 3].repr() and "abc".hash()? Yes. But hashing and repring are done much less often than taking the length. So I wouldn't worry about them...in this round anyhow. Plus repr has a first-class syntax so the function is kind of redundant already! > Where do you draw the line? If a function takes a single argument that is a Python object and immediately dispatches to a magic method, and it is "important" enough to be a builtin, then it should probably just be a method. What is the point of hiding what is really going on -- a method call! > Maybe my habits are too ingrained > at this point, but i kind of like the boundary between __methods__ > and ordinary methods... it would somehow bug me to have all these > methods like abs, len, hash crowding in on my ordinary-method > namespace. I can appreciate that your tastes are different from mine...in fact I said something similar to David when he first suggested it to me but on reflection it seemed such a subtle thing when compared against the weirdness of the definition of len: def len(obj): return obj.__length__() It does a little bit more but not much! If you put that in a module you wrote you would consider it a little bit bizarre, useless and confusing. Plus it is another indirection which saps a tiny bit of performance for nothing. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From skip@pobox.com (Skip Montanaro) Thu Aug 23 02:13:43 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 22 Aug 2001 20:13:43 -0500 Subject: [Python-Dev] More feedback from script scape In-Reply-To: <15236.21629.394538.234568@anthem.wooz.org> References: <3B841A05.F292DD5B@ActiveState.com> <15236.14289.311782.621168@anthem.wooz.org> <3B843C02.6EFEC933@ActiveState.com> <15236.21629.394538.234568@anthem.wooz.org> Message-ID: <15236.22727.471154.470699@beluga.mojam.com> BAW> From what you and Ping described, I doubt it needs to be PEP'd. It BAW> could all be done in a module, without any impact on the language BAW> I'd imagine. Agreed. I believe it's the same with Perl's Inline module. In fact, I first encountered it about a year ago when trying to figure out how to write Mason components in Python. I seem to recall that the author(s) wrote Inline as a lark. Skip From barry@zope.com Thu Aug 23 02:20:51 2001 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 22 Aug 2001 21:20:51 -0400 Subject: [Python-Dev] Re: Inline code in other languages References: <15236.21449.231509.725435@anthem.wooz.org> Message-ID: <15236.23155.857942.461104@anthem.wooz.org> >>>>> "KY" == Ka-Ping Yee writes: >> Slick indeed! I don't see much that's beyond Python's grasp. >> The syntax of course would be more Pythonic, KY> Perhaps: | import inline | inline.C(''' | void greet(char* str) | { | printf("%s, %s, %s, lovely %s\n", str, str, str, str); | } | ''') | from inline.C import greet | greet('spam') KY> ...? Works for me! -Barry From DavidA@ActiveState.com Thu Aug 23 02:28:57 2001 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 22 Aug 2001 18:28:57 -0700 Subject: [Python-Dev] copy, len and the like as 'object' methods? References: <200108221946.f7MJk1j29412@odiug.digicool.com> <3B843801.BA1D8541@ActiveState.com> <200108222317.TAA30578@cj20424-a.reston1.va.home.com> Message-ID: <3B845C59.5DF822A4@ActiveState.com> Guido van Rossum wrote: > AFAICT, the len() method proposal is just an attempt to be more OO, > and to avoid being made fun of by a bunch of Perlers. I don't care > about #1, and it's naive to think that solving one issue will avoid > #2. Actually, I couldn't care less about #1 or #2. Paul and I hold our own against Perl folks every day =), and I'm not a fan of pure-OO'ness. I do try to make _learning_ Python as easy as possible, and inconsistencies such as this one do get in the way. People who learn the language ask things like "how do I make a copy of something?" and the answer to that is currently quite complicated. It just gets in the way, and it wastes synapses. My only point is that if the language were to be designed today, my guess is that those things would be methods (or interfaces :-). --david From paulp@ActiveState.com Thu Aug 23 02:43:21 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Wed, 22 Aug 2001 18:43:21 -0700 Subject: [Python-Dev] PyInline References: <3B841A05.F292DD5B@ActiveState.com> <15236.14289.311782.621168@anthem.wooz.org> <3B843C02.6EFEC933@ActiveState.com> <15236.21629.394538.234568@anthem.wooz.org> <15236.22727.471154.470699@beluga.mojam.com> Message-ID: <3B845FB9.2C165BD2@ActiveState.com> Skip Montanaro wrote: > > BAW> From what you and Ping described, I doubt it needs to be PEP'd. It > BAW> could all be done in a module, without any impact on the language > BAW> I'd imagine. > > Agreed. I believe it's the same with Perl's Inline module. In fact, I > first encountered it about a year ago when trying to figure out how to write > Mason components in Python. I seem to recall that the author(s) wrote > Inline as a lark. Yes but Inline has grown into an all-consuming obsession. He left his job at ActiveState to pursue it "full time". There is even an Inline t-shirt. Inline won an award for best new module at the Perl conference. Outside of Perl 6, it is probably the biggest new thing in the Perl world. It isn't as trivial as you think. It is essentially a superset of SWIG's Python-related functionality. Don't forget that an important part of it is trying to autodetect types. It quickly gets into figuring out structs and then classes and then templates and then madness ensues. Obviously you have to decide how much you are going to support and just require "compliant" wrappers for the rest. Using SWIG might allow you to offload a bunch of that effort for PyInline::C. Plus, there is Inline::C, Inline::Python, Inline::Fortran, Inline::ASM, Inline::Java, etc. http://search.cpan.org/search?mode=module&query=Inline%3a%3a Finally, you have to think hard about how the thing interacts with binary distributions for people without compilers. That includes a hefty chunk of Python's user base. This is much more important than the other two points. I don't want to discourage anyone thinking about tackling it but I also want to give you a reasonable sense of what needs to be implemented. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From beazley@cs.uchicago.edu Thu Aug 23 03:16:02 2001 From: beazley@cs.uchicago.edu (David Beazley) Date: Wed, 22 Aug 2001 21:16:02 -0500 (CDT) Subject: [Python-Dev] Inline and SWIG In-Reply-To: References: Message-ID: <15236.26466.84309.855339@gargoyle.cs.uchicago.edu> > > It isn't as trivial as you think. It is essentially a superset of SWIG's > Python-related functionality. Don't forget that an important part of it > is trying to autodetect types. It quickly gets into figuring out structs > and then classes and then templates and then madness ensues. Obviously > you have to decide how much you are going to support and just require > "compliant" wrappers for the rest. Using SWIG might allow you to offload > a bunch of that effort for PyInline::C. I've looked briefly at Inline and agree that it's a pretty nifty approach. However, usability aside, it's got a number of pretty major limitations once you start getting into structure wrapping, C++ classes, and other advanced types of extension wrapping. I think a common misconception about wrapper generation tools in general is that it's just a matter of hacking up a quick parser and throwing something together in a couple of days. The *real* problem IMHO is actually trying to figure out how to properly handle all of the facets of the C/C++ type system (pointers, references, qualifiers, classes, templates, typedef, inheritance, multiple inheritance, member pointers, pass by value, etc.) and how that's supposed to hook into Python. What's worse, if you get *ANY* part of the type system wrong, everything becomes a hellish nightmare later (and I can speak from personal experience on this :-). We've been working on an upcoming major new SWIG release for quite some time now and about 80% of that work has been related to redesigning the internal type system. Maybe this sort of thing is obvious to everyone else and I'm missing some trivial point, but I sure haven't found the C++ type system to be this totally trivial thing I could just hack up in a few days and stick in a code generator! In any case, I think the coolest thing about Inline is the ability to add small little bits of C code on-the-fly without having to restart the interpreter. That's a pretty neat trick that would probably be useful in the context of working with larger C extensions. I'm not sure I'd want to build a huge 400 function C extension module with that kind of approach, but something like that would definitely be useful for prototyping and adding code on the fly. I almost wonder if something like SWIG could be used behind the scenes to do code generation for something like that. Cheers, Dave From ping@lfw.org Thu Aug 23 03:25:49 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Wed, 22 Aug 2001 19:25:49 -0700 (PDT) Subject: [Python-Dev] Re: Inline and SWIG In-Reply-To: <15236.26466.84309.855339@gargoyle.cs.uchicago.edu> Message-ID: On Wed, 22 Aug 2001, David Beazley wrote: > I've looked briefly at Inline and agree that it's a pretty nifty > approach. However, usability aside, it's got a number of pretty major > limitations once you start getting into structure wrapping, C++ > classes, and other advanced types of extension wrapping. I think the audiences are different. SWIG is for taking an existing library of C or C++ code and making it available in Python. It seems to me that Inline is about making multiple languages available to a programmer while the programmer is writing new code. I agree with you that the first task is much, much harder. However, there is also a useful payoff from the second. -- ?!ng From beazley@cs.uchicago.edu Thu Aug 23 03:31:00 2001 From: beazley@cs.uchicago.edu (David Beazley) Date: Wed, 22 Aug 2001 21:31:00 -0500 (CDT) Subject: [Python-Dev] Re: Inline and SWIG In-Reply-To: References: <15236.26466.84309.855339@gargoyle.cs.uchicago.edu> Message-ID: <15236.27364.304751.836868@gargoyle.cs.uchicago.edu> Ka-Ping Yee writes: > I think the audiences are different. SWIG is for taking an existing > library of C or C++ code and making it available in Python. It seems > to me that Inline is about making multiple languages available to a > programmer while the programmer is writing new code. I agree with you > that the first task is much, much harder. However, there is also a > useful payoff from the second. Definitely! I saw a demo of Inline when I visited Activestate awhile back and was blown away ("You mean you can do that?!?"). Pretty neat stuff. Cheers, Dave From guido@python.org Thu Aug 23 03:29:35 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 22 Aug 2001 22:29:35 -0400 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: Your message of "Wed, 22 Aug 2001 16:54:21 PDT." <3B84462D.230FFF3F@ActiveState.com> References: <200108221946.f7MJk1j29412@odiug.digicool.com> <3B843801.BA1D8541@ActiveState.com> <200108222317.TAA30578@cj20424-a.reston1.va.home.com> <3B84462D.230FFF3F@ActiveState.com> Message-ID: <200108230229.WAA31882@cj20424-a.reston1.va.home.com> > OO is merely accidental. The real goal is to be more consistent. Python > is a language that uses methods for polymorphism so consistency pushes > towards methods. If Python were Scheme and had an OO appendage in the > middle of its standard library I'd dislike that too. > > The only reason Perlers come into the picture is because they are > pointing out that we claim to be consistent but in this case are not. > And the only reason any of us care about consistency is because > inconsistency has a cost in education and memorization...and according > to Greg Wilson he's seen it first-hand in this case. (I personally had > more experience with confusion over string functions instead of string > methods) > > > Any change like this has a *huge* cost to the community (just reread > > the division thread). > > Division is a special case. There is a big difference between > re-purposing existing syntax and adding new methods to existing objects. > > If one is clearly the "new, better" way and the other is historical, > then the books document the new way and most users eventually forget > that the old way even exists. e.g. string exceptions, string.functions, > UserDict (after type-class), regex, etc. I see nothing that you didn't say before. Nothing to sway me to set the wheels in motion to make this particular change. My position remains "it ain't broken, so don't fix it." --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Aug 23 03:33:02 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 22 Aug 2001 22:33:02 -0400 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: Your message of "Wed, 22 Aug 2001 18:28:57 PDT." <3B845C59.5DF822A4@ActiveState.com> References: <200108221946.f7MJk1j29412@odiug.digicool.com> <3B843801.BA1D8541@ActiveState.com> <200108222317.TAA30578@cj20424-a.reston1.va.home.com> <3B845C59.5DF822A4@ActiveState.com> Message-ID: <200108230233.WAA31918@cj20424-a.reston1.va.home.com> > Actually, I couldn't care less about #1 or #2. Paul and I hold our own > against Perl folks every day =), and I'm not a fan of pure-OO'ness. I > do try to make _learning_ Python as easy as possible, and > inconsistencies such as this one do get in the way. This one seems to be something that you stumble over once, and then get used to. > People who learn the language ask things like "how do I make a copy of > something?" and the answer to that is currently quite complicated. It > just gets in the way, and it wastes synapses. Maybe they are asking the wrong question. I think that copying isn't all that important in Python -- that's why it's in a module. > My only point is that if the language were to be designed today, my > guess is that those things would be methods (or interfaces :-). Sure. But they're not so broken as to warrant a redesign. There are more important battles. --Guido van Rossum (home page: http://www.python.org/~guido/) From grant.griffin@iowegian.com Thu Aug 23 03:47:56 2001 From: grant.griffin@iowegian.com (Grant Griffin) Date: Wed, 22 Aug 2001 21:47:56 -0500 Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value In-Reply-To: <200108221415.KAA27226@cj20424-a.reston1.va.home.com> References: <4.3.2.7.2.20010820215632.00cb69e0@pop.kcnet.com> <4.3.2.7.2.20010822081617.00dcab00@pop.kcnet.com> Message-ID: <4.3.2.7.2.20010822212254.00cbd220@pop.kcnet.com> At 10:15 AM 8/22/2001 -0400, Guido van Rossum wrote: > > However, the problem I see with a class-based approach is that this > > is such a small thing that I'm not even sure how to write a class > > for it that isn't more grandiose than it deserves. > >Believe me, adding a new method to a heavily-used built-in object is >much more grandiose than adding a library module. In addition to "15 >lines of code" we need to update the documentation, add a few lines to >the test suite, add code to Jython mimicking the feature; 30 books >will be a little more out of date, and so on. I guess I don't see how this one is different from any other feature change to Python in that regard: they all require code, tests, and docs. The only real logistical advantage I can see for implementing it as module is that the port to Jython would be virtually free. Aside from that, it seems like adding a new module (in any form) would cause much _more_ bloat overall in terms of SLOCs, tests and documentation. For example, a new library module would get a doc page of its own, but a single new Boolean parameter added to items() could be documented by a footnote on the "Mapping Types" page, e.g., "If the 'values_first' parameter is set to true, the item tuples returned are in (value, key) order rather than the default (key, value) order." The test suite for the new parameter could be just a line or two which would compare the values-first items of a test dictionary to the expected list. So, in total, my preferred suggestion, "items(values_first=0)", (which actually isn't a _new_ method!) would cause bloat on the order of: 15 lines of C (and Java) 1 sentence of documentation 3 lines of test code and best of all!: 0 __future__ statements And as for making 30 books a little more out of date...well...if Tim O'Reilly doesn't give me a big wet kiss for that, I'm gonna demand he refund the $54.95 I just paid for the Second Edition of "Programming Python" . =g2 From greg@cosc.canterbury.ac.nz Thu Aug 23 03:57:37 2001 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 23 Aug 2001 14:57:37 +1200 (NZST) Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value In-Reply-To: <4.3.2.7.2.20010822081617.00dcab00@pop.kcnet.com> Message-ID: <200108230257.OAA04095@s454.cosc.canterbury.ac.nz> Grant Griffin : > items(values_first=0) Or perhaps a separate method called smeti() :-) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Thu Aug 23 04:01:49 2001 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 23 Aug 2001 15:01:49 +1200 (NZST) Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value In-Reply-To: <000801c12b40$25ae3640$060210ac@private> Message-ID: <200108230301.PAA04098@s454.cosc.canterbury.ac.nz> Barry Scott : > How should all the variations be handled? def order(index = None, field = None, descending = 0): ... Examples: from sorting import order x.sort(order(index = 3)) x.sort(order(field = "customer_name", descending = 1)) The next step is to allow sort() itself to take the same keyword parameters directly: x.sort(index = 3) x.sort(field = "customer_name", descending = 1) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From DavidA@ActiveState.com Thu Aug 23 04:31:55 2001 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 22 Aug 2001 20:31:55 -0700 Subject: [Python-Dev] Re: Inline and SWIG References: <15236.26466.84309.855339@gargoyle.cs.uchicago.edu> <15236.27364.304751.836868@gargoyle.cs.uchicago.edu> Message-ID: <3B84792B.7DDF7C1B@ActiveState.com> Note that I know of at least two if not three different inline-like projects for Python. One is the compiler module in scipy (Eric Jones & Travis Oliphant): http://scipy.net/cgi-bin/viewcvs.cgi/~checkout~/scipy/compiler/doc/readme?rev=1.2&content-type=text/plain One is the stuff that Pat Miller from LLNL presented at OSCON in San Diego, which sounded similar in spirit (alas, I missed the talk, and don't know anything about its availability -- Pat?), and there was a third one which I heard mentioned but don't know any details about. Eric, do you remember what I'm talking about? I'd recommend that people track down the authors of those modules, as I do think there is a great opportunity there. (I take credit for inspiring Eric Jones by mentioning Inline:: to him in Long Beach =). --david ascher From tim.one@home.com Thu Aug 23 04:32:50 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 22 Aug 2001 23:32:50 -0400 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: <200108230233.WAA31918@cj20424-a.reston1.va.home.com> Message-ID: [David Ascher] > My only point is that if the language were to be designed today, my > guess is that those things would be methods (or interfaces :-). [Guido] > Sure. Say what?! What kind of stupid-ass language spells the len() function as if it were a method? That's carrying consistency to an irrational extreme -- trust your decade-old intuition on this one (which was challenged no later than 9 years ago on the same ground). > But they're not so broken as to warrant a redesign. There are > more important battles. Proof: every thread bloats in inverse proportion to its true importance. So there . incisively y'rs - tim From DavidA@ActiveState.com Thu Aug 23 04:45:07 2001 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 22 Aug 2001 20:45:07 -0700 Subject: [Python-Dev] copy, len and the like as 'object' methods? References: <200108221946.f7MJk1j29412@odiug.digicool.com> <3B843801.BA1D8541@ActiveState.com> <200108222317.TAA30578@cj20424-a.reston1.va.home.com> <3B845C59.5DF822A4@ActiveState.com> <200108230233.WAA31918@cj20424-a.reston1.va.home.com> Message-ID: <3B847C43.2082509E@ActiveState.com> Guido van Rossum wrote: > > > Actually, I couldn't care less about #1 or #2. Paul and I hold our own > > against Perl folks every day =), and I'm not a fan of pure-OO'ness. I > > do try to make _learning_ Python as easy as possible, and > > inconsistencies such as this one do get in the way. > > This one seems to be something that you stumble over once, and then > get used to. I don't disagree, but people have said the same thing about case-sensitivity, integer division, and Perl. =) > > People who learn the language ask things like "how do I make a copy of > > something?" and the answer to that is currently quite complicated. It > > just gets in the way, and it wastes synapses. > > Maybe they are asking the wrong question. I think that copying isn't > all that important in Python -- that's why it's in a module. I'm tired of this argument, so I'll quit. I'll just say that IMO students can't ask the wrong questions. The body of knowledge that they bring to the table is perfectly valid. They're trying to 'grok' a large complicated system, and the questions they ask reflect steps in understanding. FWIW, I only talk about copying after I've discussed references in depth, since that makes the argument you mention more convincing. I only half buy the argument myself, however, since the elegance of the presentation is always marred by having to explain such oddities, no matter how smoothly I finagle it. I see much of Python's recent evolution as getting rid of wrinkles in the system. I just pointed out a wrinkle, that's all. Note that I started with a question, and I'll accept that you don't want to change it. Next topic. --david From tim.one@home.com Thu Aug 23 05:06:28 2001 From: tim.one@home.com (Tim Peters) Date: Thu, 23 Aug 2001 00:06:28 -0400 Subject: [Python-Dev] Cuddly constants seek loving, stable home Message-ID: __future__.py now contains this abomination: try: import new as _new # for CO_xxx symbols except ImportError: # May happen during build class _new: CO_NESTED = 0x0010 CO_GENERATOR_ALLOWED = 0x1000 CO_FUTURE_DIVISION = 0x2000 "new" turned out to be a bad module to hold these constants, despite that that's where they belong (i.e., along with the function that creates new code objects), first because "new" wasn't always built "in time" on Unixy boxes, and second because "new" isn't available at all in restricted mode. So they can't live in "new", but duplicating their values by hand is also unbearable. "sys" is the only other module I see where they *might* fit, but they don't really belong there (indeed, trying to make a case that they do is a lot like arguing "len" should be a method ). So I propose a new builtin module, _sysconsts. The underscore is to announce its obscure internal nature. I propose only to expose the CO_xxx #defines at the start, but of course others could be added. Any objections? ain't-worth-a-pep-unless-there-are-ly y'rs - tim From guido@python.org Thu Aug 23 05:07:22 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 23 Aug 2001 00:07:22 -0400 Subject: [Python-Dev] Cuddly constants seek loving, stable home In-Reply-To: Your message of "Thu, 23 Aug 2001 00:06:28 EDT." References: Message-ID: <200108230407.AAA00339@cj20424-a.reston1.va.home.com> > So I propose a new builtin module, _sysconsts. The underscore is to > announce its obscure internal nature. I propose only to expose the CO_xxx > #defines at the start, but of course others could be added. +1 on the concept, +0 on the name. Maybe _sysconst (singular)? --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Thu Aug 23 05:17:45 2001 From: tim.one@home.com (Tim Peters) Date: Thu, 23 Aug 2001 00:17:45 -0400 Subject: [Python-Dev] Cuddly constants seek loving, stable home In-Reply-To: <200108230407.AAA00339@cj20424-a.reston1.va.home.com> Message-ID: [Tim, proposes a new _sysconst module] [Guido] > +1 on the concept, +0 on the name. Maybe _sysconst (singular)? Ya, that's what I said -- you must have suffered an email error . the-singular-is-certainly-easier-to-pronounce-ly y'rs - tim From esr@snark.thyrsus.com Thu Aug 23 05:12:56 2001 From: esr@snark.thyrsus.com (Eric S. Raymond) Date: Thu, 23 Aug 2001 00:12:56 -0400 Subject: [Python-Dev] Cuddly constants seek loving, stable home In-Reply-To: ; from tim.one@home.com on Thu, Aug 23, 2001 at 12:06:28AM -0400 References: Message-ID: <20010823001256.A3031@thyrsus.com> Tim Peters : > So I propose a new builtin module, _sysconsts. The underscore is to > announce its obscure internal nature. I propose only to expose the CO_xxx > #defines at the start, but of course others could be added. > > Any objections? +0. -- Eric S. Raymond I don't like the idea that the police department seems bent on keeping a pool of unarmed victims available for the predations of the criminal class. -- David Mohler, 1989, on being denied a carry permit in NYC From guido@python.org Thu Aug 23 05:40:39 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 23 Aug 2001 00:40:39 -0400 Subject: [Python-Dev] Near Final PEP 237 - Unifying Long Integers and Integers Message-ID: <200108230440.AAA00660@cj20424-a.reston1.va.home.com> Please comment on a new version of PEP 237: http://python.sourceforge.net/peps/pep-0237.html I've updated a few parts of the PEP for more accuracy and to reflect more implementation details. I've had very few negative responses to this PEP, and quite a few positive, so I'm assuming this is not controversial and we can follow the transition plan described in the PEP. Only phase A of the transition plan will be implemented in Python 2.2. I've implemented phase A completely, and tentatively checked it into CVS; unless a major showstopper is brought up I will release it with Python 2.2a3 (it didn't make it into 2.2a2). Please use the SF bug tracker to report bugs in the implementation. Please mail comments on the PEP to python-dev@python.org or to me. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Thu Aug 23 06:47:45 2001 From: tim.one@home.com (Tim Peters) Date: Thu, 23 Aug 2001 01:47:45 -0400 Subject: [Python-Dev] Near Final PEP 237 - Unifying Long Integers and Integers In-Reply-To: <200108230440.AAA00660@cj20424-a.reston1.va.home.com> Message-ID: [Guido] > ... > I've had very few negative responses to this PEP, and quite a few > positive, so I'm assuming this is not controversial and we can follow > the transition plan described in the PEP. Only phase A of the > transition plan will be implemented in Python 2.2. FYI, there have been no negatives on c.l.py either, and at least two positives -- but it's barely been discussed at all yet. I changed the subject line of the one thread that drifted into it to contain "PEP 237". +1 from me. From greg@cosc.canterbury.ac.nz Thu Aug 23 07:01:28 2001 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 23 Aug 2001 18:01:28 +1200 (NZST) Subject: [Python-Dev] Inline and SWIG In-Reply-To: <15236.26466.84309.855339@gargoyle.cs.uchicago.edu> Message-ID: <200108230601.SAA04304@s454.cosc.canterbury.ac.nz> David Beazley : > I've looked briefly at Inline and agree that it's a pretty nifty > approach. However, usability aside, it's got a number of pretty major > limitations once you start getting into structure wrapping, C++ > classes, and other advanced types of extension wrapping. Indeed, you've put your finger on what seems like the biggest problem with the current crop of semi-automatic extension generators. I don't think there is ever going to be a completely automatic solution to this, because, once you get beyond the basic types like ints, floats and strings, there is no unique mapping between Python types and C types. While the tool could provide some sort of default translation for arbitrary types, it's unlikely to be what you need to interface directly with some existing C library you're trying to wrap. So you're faced with writing a lot of tedious and error-prone code for messing with PyObjects and reference counts. I've had some ideas floating around in my head for a while concerning what to do about this. The key observation is that such code usually spends a lot more lines manipulating Python data than it does C data, which leads me to the conclusion that C is the wrong language to write it in. What's the right language? One that's good at manipulating Python objects. Which language is the best at doing that? Why, Python, of course. But not just Python, because we need to be able to manipulate C data structures as well. So we need a language that looks like Python at one end and C at the other... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From ping@lfw.org Thu Aug 23 09:37:51 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Thu, 23 Aug 2001 01:37:51 -0700 (PDT) Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home In-Reply-To: Message-ID: On Thu, 23 Aug 2001, Tim Peters wrote: > So I propose a new builtin module, _sysconsts. The underscore is to > announce its obscure internal nature. I propose only to expose the CO_xxx > #defines at the start, but of course others could be added. Just a suggestion, but how about sys.consts? Like os.path, as in sys.consts.CO_NESTED, sys.consts.CO_FUTURE_DIVISION, etc. So there's only one new object in sys, and it can contain the weird internal stuff. :) (Errr... the names are rather weird. CO_NESTED_SCOPES, CO_TRUE_DIVISION, and CO_GENERATORS would be better. If i had my druthers it would be CO_* constant FUTURE_* constant __future__ symbol CO_NESTED_SCOPES FUTURE_NESTED_SCOPES nested_scopes CO_GENERATORS FUTURE_GENERATORS generators CO_TRUE_DIVISION FUTURE_TRUE_DIVISION true_division instead. If CO_NESTED, why not CO_TRUE? If CO_GENERATORS_ALLOWED, why not CO_NESTED_SCOPES_ALLOWED?) Anyway. Hmm, as a side thought, perhaps keywords and tokens should go here too (sys.keywords, sys.tokens) instead of having to keep separate .py files in sync with the interpreter core. -- ?!ng From fredrik@pythonware.com Thu Aug 23 09:06:56 2001 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 23 Aug 2001 10:06:56 +0200 Subject: [Python-Dev] PEP 268 - Extended HTTP functionality and WebDAV References: <20010821134036.J13229@lyra.org> Message-ID: <00bf01c12bae$c03cd2c0$0900a8c0@spiff> greg wrote: > I've completed my first draft of PEP 268 and am seeking feedback. > > http://python.sourceforge.net/peps/pep-0268.html looks good to me. > My next step is to begin implementing the 'httpx' module and rejiggering the > 'davlib' module. This work will occur in the /nondist/sandbox/ area > (probably under a Lib subdir there). excellent! From fredrik@pythonware.com Thu Aug 23 09:17:51 2001 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 23 Aug 2001 10:17:51 +0200 Subject: [Python-Dev] Re: copy, len and the like as 'object' methods? References: <930BBCA4CEBBD411BE6500508BB3328F3A8ECC@nsamcanms1.ca.baltimore.com> Message-ID: <00c001c12bae$c053b620$0900a8c0@spiff> greg wilson wrote: > FWIW, David's 'len()' methods would definitely make Python easier > to teach --- just about every beginner types 'x.len()' instead of > 'len(x)' over and over during their first two or three exercises. my students don't -- so it's probably a bug in your training material, not in Python ;-) cheers /F From larsga@garshol.priv.no Thu Aug 23 09:41:14 2001 From: larsga@garshol.priv.no (Lars Marius Garshol) Date: 23 Aug 2001 10:41:14 +0200 Subject: [Python-Dev] Re: Inline code in other languages In-Reply-To: <15236.21449.231509.725435@anthem.wooz.org> References: <15236.14289.311782.621168@anthem.wooz.org> <15236.21449.231509.725435@anthem.wooz.org> Message-ID: * Barry A. Warsaw | | Slick indeed! I don't see much that's beyond Python's grasp. The | syntax of course would be more Pythonic, [...] Why not make it import C files, instead of putting them inline? I guess you very often want to reuse the C functions in several different places anyway, and you'd mostly want to have includes and stuff as well. This way you can also get rid of the MD5-ing and just use ordinary modification times to figure out what needs to be recompiled. You can't inline pieces smaller than functions anyway, and to me the mix between Python and C code looks very ugly. --Lars M. From Paul.Moore@atosorigin.com Thu Aug 23 10:12:45 2001 From: Paul.Moore@atosorigin.com (Moore, Paul) Date: Thu, 23 Aug 2001 10:12:45 +0100 Subject: [Python-Dev] Re: copy, len and the like as 'object' methods? Message-ID: <714DFA46B9BBD0119CD000805FC1F53B01B5AFA3@UKRUX002.rundc.uk.origin-it.com> > I can appreciate that your tastes are different from > mine...in fact I said something similar to David when he > first suggested it to me but on reflection it seemed such > a subtle thing when compared against the weirdness of the > definition of len: > > def len(obj): > return obj.__length__() > > It does a little bit more but not much! A strange thought, but with the advent of type/class unification and the 'object' base class, is it possible to inject new methods into the 'object' type at runtime, allowing: def len(self): return self.__length__() object.len = len This would seem quite slick (or is that "sick" :-), but it makes the language seem more mutable than might be healthy... (Or is 'object' not this mutable...? Must try 2.2a2 and see...) Paul. From simon@netthink.co.uk Thu Aug 23 10:52:41 2001 From: simon@netthink.co.uk (Simon Cozens) Date: Thu, 23 Aug 2001 10:52:41 +0100 Subject: [Python-Dev] Perl5->Perl6 In-Reply-To: <5.1.0.14.0.20010822170534.08c50170@tuatha.sidhe.org> References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> <5.1.0.14.0.20010822153946.08c02de8@tuatha.sidhe.org> <5.1.0.14.0.20010822170534.08c50170@tuatha.sidhe.org> Message-ID: <20010823105241.B8531@netthink.co.uk> On Wed, Aug 22, 2001 at 05:17:12PM -0400, Dan Sugalski wrote: > Seriously, we've a reasonably comprehensive test suite now with perl 5, and > a very large code base with expected behavior to draw on to test the > conversion. Much of the perl 5->perl 6 transition is strictly syntactic. Also don't forget that we have a prototype of Perl 6 which implements pretty much everything that Larry has specified in Apocalypses 1 and 2. It is a patch of nearly 200 lines to the current Perl 5 interpreter. We're not changing *that* drastically, folks. From gmcm@hypernet.com Thu Aug 23 11:37:04 2001 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 23 Aug 2001 06:37:04 -0400 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: <3B847C43.2082509E@ActiveState.com> Message-ID: <3B84A490.15494.5052EA06@localhost> [Guido] > > Maybe they are asking the wrong question. I think that copying > > isn't all that important in Python -- that's why it's in a > > module. [David] > I'm tired of this argument, so I'll quit. I'll just say that IMO > students can't ask the wrong questions. Not to prolong it, but Guido's comment made me realize that I haven't imported copy in years. Haven't written a clone() method in years, either, despite the fact that coming from Java and C++ that was one of my favorite methods. get-yourself-a-big-stick-and-teach-'em-Zen-ly y'rs - Gordon From skip@pobox.com (Skip Montanaro) Thu Aug 23 14:11:42 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 23 Aug 2001 08:11:42 -0500 Subject: [Python-Dev] Inline caveat Message-ID: <15237.270.649981.730739@beluga.mojam.com> There is, of course, a definite downside to Inline. Your code will get harder to read and maintain because you and whoever else has to maintain your code has to be familiar with all the languages you are inlining. Skip From thomas.heller@ion-tof.com Thu Aug 23 14:27:05 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 23 Aug 2001 15:27:05 +0200 Subject: [Python-Dev] Inline caveat References: <15237.270.649981.730739@beluga.mojam.com> Message-ID: <034801c12bd7$4dabc3f0$e000a8c0@thomasnotebook> > There is, of course, a definite downside to Inline. Your code will get > harder to read and maintain because you and whoever else has to maintain > your code has to be familiar with all the languages you are inlining. > > Skip This is more an argument against mixed language programming than against Inline. Inline makes it easier (a no-brainer) to locate the code which has to be changed, to determine what has to be recompiled, where (and how) it has to be installed, and so on. I've played a little bit with the idea and I certainly like it. Thomas From guido@python.org Thu Aug 23 14:24:52 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 23 Aug 2001 09:24:52 -0400 Subject: [Python-Dev] Re: copy, len and the like as 'object' methods? In-Reply-To: Your message of "Thu, 23 Aug 2001 10:12:45 BST." <714DFA46B9BBD0119CD000805FC1F53B01B5AFA3@UKRUX002.rundc.uk.origin-it.com> References: <714DFA46B9BBD0119CD000805FC1F53B01B5AFA3@UKRUX002.rundc.uk.origin-it.com> Message-ID: <200108231324.JAA01440@cj20424-a.reston1.va.home.com> > A strange thought, but with the advent of type/class unification and the > 'object' base class, is it possible to inject new methods into the 'object' > type at runtime, allowing: > > def len(self): > return self.__length__() > object.len = len No you can't. Built-in types are immutable. Two reasons: they're shared between multiple interpreters, and it would encourage modules to modify a built-in type to suit their own needs, which it might break other modules. --Guido van Rossum (home page: http://www.python.org/~guido/) From gstein@lyra.org Thu Aug 23 14:59:07 2001 From: gstein@lyra.org (Greg Stein) Date: Thu, 23 Aug 2001 06:59:07 -0700 Subject: [Python-Dev] PEP 265 - Sorting Dictionaries by Value In-Reply-To: <200108230301.PAA04098@s454.cosc.canterbury.ac.nz>; from greg@cosc.canterbury.ac.nz on Thu, Aug 23, 2001 at 03:01:49PM +1200 References: <000801c12b40$25ae3640$060210ac@private> <200108230301.PAA04098@s454.cosc.canterbury.ac.nz> Message-ID: <20010823065907.W16727@lyra.org> On Thu, Aug 23, 2001 at 03:01:49PM +1200, Greg Ewing wrote: > Barry Scott : > > > How should all the variations be handled? > > def order(index = None, field = None, descending = 0): Lose the last arg... call them ascending() and descending(). Sorting already implies an "order" so that name is redundant >... > from sorting import order > > x.sort(order(index = 3)) > x.sort(order(field = "customer_name", descending = 1)) from sorting import ascending, descending x.sort(ascending(index = 3)) x.sort(descending(field = 'customer_name')) > The next step is to allow sort() itself to take the same > keyword parameters directly: > > x.sort(index = 3) > x.sort(field = "customer_name", descending = 1) eek... no :-) Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Thu Aug 23 15:03:00 2001 From: gstein@lyra.org (Greg Stein) Date: Thu, 23 Aug 2001 07:03:00 -0700 Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home In-Reply-To: ; from ping@lfw.org on Thu, Aug 23, 2001 at 01:37:51AM -0700 References: Message-ID: <20010823070259.X16727@lyra.org> On Thu, Aug 23, 2001 at 01:37:51AM -0700, Ka-Ping Yee wrote: > On Thu, 23 Aug 2001, Tim Peters wrote: > > So I propose a new builtin module, _sysconsts. The underscore is to > > announce its obscure internal nature. I propose only to expose the CO_xxx > > #defines at the start, but of course others could be added. > > Just a suggestion, but how about sys.consts? Like os.path, as in > sys.consts.CO_NESTED, sys.consts.CO_FUTURE_DIVISION, etc. So there's > only one new object in sys, and it can contain the weird internal stuff. :) +1 -0 on a new module. Just make sys.consts some kind of attributed object. >... > Hmm, as a side thought, perhaps keywords and tokens should go here too > (sys.keywords, sys.tokens) instead of having to keep separate .py files > in sync with the interpreter core. +1 ... Good call. That would toss some of the synchronization problems with those files today. Cheers, -g -- Greg Stein, http://www.lyra.org/ From skip@pobox.com (Skip Montanaro) Thu Aug 23 15:19:20 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 23 Aug 2001 09:19:20 -0500 Subject: [Python-Dev] Inline caveat In-Reply-To: <034801c12bd7$4dabc3f0$e000a8c0@thomasnotebook> References: <15237.270.649981.730739@beluga.mojam.com> <034801c12bd7$4dabc3f0$e000a8c0@thomasnotebook> Message-ID: <15237.4328.899938.564006@beluga.mojam.com> >> There is, of course, a definite downside to Inline. Your code will >> get harder to read and maintain because you and whoever else has to >> maintain your code has to be familiar with all the languages you are >> inlining. Thomas> This is more an argument against mixed language programming Thomas> than against Inline. Certainly. Today, you have to keep the various bits of code written in different languages separate. Inline certainly lowers that barrier and makes it feasible to do some spot optimizations quickly. For example, several years ago I needed to be able to compute the distance between two lat/longs. I wrote: def distance(ll1, ll2): dlat = ll1[0] - ll2[0] longdist = (ll1[1] - ll2[1]) * math.fabs(math.cos(DEGTORAD*dlat)) return math.sqrt(MILESPERDEGLAT**2 * (dlat*dlat+longdist*longdist)) This turned out to be too slow and I wound up writing a C extension that exposed just that one function. An Inline module would have been the perfect solution to this problem (assuming it could figure out the shared library needed to be linked with -lm). Thomas> Inline makes it easier (a no-brainer) to locate the code which Thomas> has to be changed, to determine what has to be recompiled, where Thomas> (and how) it has to be installed, and so on. Yes, but my original assertion still stands. With C and Python in separate modules, I can tell people "pay no attention to the man behind the curtain" (that being the C code). With the C code right there in the midst of your Python code it's pretty hard to ignore the fact that the man is not behind the curtain and he's flashing you. ;-) Skip From barry@zope.com Thu Aug 23 15:29:53 2001 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 23 Aug 2001 10:29:53 -0400 Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home References: <20010823070259.X16727@lyra.org> Message-ID: <15237.4961.597225.931722@anthem.wooz.org> >>>>> "KY" == Ka-Ping Yee writes: KY> Just a suggestion, but how about sys.consts? Like os.path, as KY> in sys.consts.CO_NESTED, sys.consts.CO_FUTURE_DIVISION, etc. KY> So there's only one new object in sys, and it can contain the KY> weird internal stuff. :) +1. It's much better to qualify them in a subpackage or object than put them in sys's top level namespace. -Barry From guido@python.org Thu Aug 23 15:32:20 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 23 Aug 2001 10:32:20 -0400 Subject: [Python-Dev] Inline caveat In-Reply-To: Your message of "Thu, 23 Aug 2001 15:27:05 +0200." <034801c12bd7$4dabc3f0$e000a8c0@thomasnotebook> References: <15237.270.649981.730739@beluga.mojam.com> <034801c12bd7$4dabc3f0$e000a8c0@thomasnotebook> Message-ID: <200108231432.f7NEWKr05785@odiug.digicool.com> > > There is, of course, a definite downside to Inline. Your code will get > > harder to read and maintain because you and whoever else has to maintain > > your code has to be familiar with all the languages you are inlining. > > > > Skip > This is more an argument against mixed language programming > than against Inline. > Inline makes it easier (a no-brainer) to locate the code which > has to be changed, to determine what has to be recompiled, > where (and how) it has to be installed, and so on. > > I've played a little bit with the idea and I certainly > like it. > > Thomas If the alternative is writing an extension module, the inline approach sounds viable to me. I could also imagine auto-wrapping existing C functions based on parseable function signatures. I believe VB supports this. Dangerous, but powerful, and very useful in the right hands. It seems to me that the important idea here is to break away from building and distributing a separate extension module, which gets you in the edit-compile-link-test-run loop that Python tries to avoid. Rather than requiring an expert who knows how to download, build, install and use SWIG or how to write Python extensions, all the expertise is automated. Getting this to work on Windows for the average Windows user would be a big hassle, limiting the portability and therefore the usefulness. Of course, Jython already has all this, thanks to Java's introspection capabilities. I wonder if the new C++ standard makes enough run-time type information able to be able to pull of the same trick? --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Thu Aug 23 15:31:57 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 23 Aug 2001 10:31:57 -0400 (EDT) Subject: [Python-Dev] Inline caveat In-Reply-To: <15237.270.649981.730739@beluga.mojam.com> References: <15237.270.649981.730739@beluga.mojam.com> Message-ID: <15237.5085.898843.495797@cj42289-a.reston1.va.home.com> Skip Montanaro writes: > There is, of course, a definite downside to Inline. Your code will get > harder to read and maintain because you and whoever else has to maintain > your code has to be familiar with all the languages you are inlining. Inline::K, anyone? ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From barry@zope.com Thu Aug 23 15:36:54 2001 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 23 Aug 2001 10:36:54 -0400 Subject: [Python-Dev] Inline caveat References: <15237.270.649981.730739@beluga.mojam.com> Message-ID: <15237.5382.431639.149435@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> There is, of course, a definite downside to Inline. Your code SM> will get harder to read and maintain because you and whoever SM> else has to maintain your code has to be familiar with all the SM> languages you are inlining. It's also harder to maintain because it's harder to edit and debug. Maybe all them fancy IDEs you youngin's use can easily support editing mutiple different languages in the same file, but if so, let me know! The tools/editors/debuggers I'm familiar with don't deal with any of that very well, if at all. You have a similar problem trying to knit HTML and Python together, as is proven by the proliferation of such approaches (e.g. DTML, PTL, HTMLgen, etc., etc.). It's bearable for one-liners and small bits of intermingling, but anything large than a line or two gets pretty unweildy pretty fast. -Barry From guido@python.org Thu Aug 23 15:43:37 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 23 Aug 2001 10:43:37 -0400 Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home In-Reply-To: Your message of "Thu, 23 Aug 2001 10:29:53 EDT." <15237.4961.597225.931722@anthem.wooz.org> References: <20010823070259.X16727@lyra.org> <15237.4961.597225.931722@anthem.wooz.org> Message-ID: <200108231443.f7NEhb505947@odiug.digicool.com> > >>>>> "KY" == Ka-Ping Yee writes: > > KY> Just a suggestion, but how about sys.consts? Like os.path, as > KY> in sys.consts.CO_NESTED, sys.consts.CO_FUTURE_DIVISION, etc. > KY> So there's only one new object in sys, and it can contain the > KY> weird internal stuff. :) > > +1. It's much better to qualify them in a subpackage or object than > put them in sys's top level namespace. +0. It's the better solution but I bet it's more work to code it than _sysconst. I'm not volunteering, and I want Tim to have time to help with the type/class unification, which is behind schedule. --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Thu Aug 23 15:44:16 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 23 Aug 2001 10:44:16 -0400 (EDT) Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home In-Reply-To: <20010823070259.X16727@lyra.org> References: <20010823070259.X16727@lyra.org> Message-ID: <15237.5824.599253.83454@cj42289-a.reston1.va.home.com> Greg Stein writes: > +1 > > -0 on a new module. Just make sys.consts some kind of attributed object. Agreed. pyexpat just creates new module objects and fills them in for things like error codes. > +1 ... Good call. That would toss some of the synchronization problems with > those files today. Yep. The existing modules could import * from those new objects so that backward compatibility is retained. The advantage of the current approach, however, is that we don't end up with a bunch of initialization code that ends up sticking around as part of the process. Not a real problem for Unix or Windows, but it might be on embedded platforms like the Palm. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Thu Aug 23 16:03:20 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 23 Aug 2001 11:03:20 -0400 (EDT) Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home In-Reply-To: <200108231443.f7NEhb505947@odiug.digicool.com> References: <20010823070259.X16727@lyra.org> <15237.4961.597225.931722@anthem.wooz.org> <200108231443.f7NEhb505947@odiug.digicool.com> Message-ID: <15237.6968.594854.181777@cj42289-a.reston1.va.home.com> Guido van Rossum writes: > +0. It's the better solution but I bet it's more work to code it than > _sysconst. I'm not volunteering, and I want Tim to have time to help > with the type/class unification, which is behind schedule. One interesting thing about these modules is that the executable C code is no longer needed once the modules has been initialized (similar to the code object for a module; it creates things and then goes away). Perhaps there should be a C equivalent. When the shared-object importer loads a .so/.dll/.pyd, it looks for init and calls that, and then keeps the object loaded. Perhaps a second name could be checked for, build, which indicates that the initialized module has no dependencies on the C code, which which case the .so/.dll/.pyd gets unloaded after the build returns. Another possibility is to simply regenerate token.py and symbol.py whenever token.h and graminit.h change. That can be driven by make so they don't fall out of sync with the rest of the grammar. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From skip@pobox.com (Skip Montanaro) Thu Aug 23 16:41:49 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 23 Aug 2001 10:41:49 -0500 Subject: [Python-Dev] Inline caveat In-Reply-To: <15237.5085.898843.495797@cj42289-a.reston1.va.home.com> References: <15237.270.649981.730739@beluga.mojam.com> <15237.5085.898843.495797@cj42289-a.reston1.va.home.com> Message-ID: <15237.9277.750760.986482@beluga.mojam.com> >> There is, of course, a definite downside to Inline. Your code will >> get harder to read and maintain because you and whoever else has to >> maintain your code has to be familiar with all the languages you are >> inlining. Fred> Inline::K, anyone? ;-) I was actually thinking more about Inline::Intercal. ;-) Skip From thomas.heller@ion-tof.com Thu Aug 23 16:58:08 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 23 Aug 2001 17:58:08 +0200 Subject: [Python-Dev] Inline caveat References: <15237.270.649981.730739@beluga.mojam.com> <034801c12bd7$4dabc3f0$e000a8c0@thomasnotebook> <200108231432.f7NEWKr05785@odiug.digicool.com> Message-ID: <052201c12bec$675bf300$e000a8c0@thomasnotebook> [Guido, about Inline] > If the alternative is writing an extension module, the inline approach > sounds viable to me. > > I could also imagine auto-wrapping existing C functions based on > parseable function signatures. I believe VB supports this. > Dangerous, but powerful, and very useful in the right hands. Isn't this the calldll approach? > It seems to me that the important idea here is to break away from > building and distributing a separate extension module, which gets you > in the edit-compile-link-test-run loop that Python tries to avoid. > Rather than requiring an expert who knows how to download, build, > install and use SWIG or how to write Python extensions, all the > expertise is automated. > > Getting this to work on Windows for the average Windows user would be > a big hassle, limiting the portability and therefore the usefulness. My prototype writes a xxx.c file, and uses distutils to build an extension module of it. The only requirement is a working VC++ compiler. Thomas From cgw@alum.mit.edu Thu Aug 23 17:27:31 2001 From: cgw@alum.mit.edu (Charles G Waldman) Date: Thu, 23 Aug 2001 11:27:31 -0500 Subject: [Python-Dev] Re: Inline::caveat In-Reply-To: References: Message-ID: <15237.12019.841104.61849@nyx.dyndns.org> > It's also harder to maintain because it's harder to edit and debug. > > Maybe all them fancy IDEs you youngin's use can easily support editing > mutiple different languages in the same file, but if so, let me know! > The tools/editors/debuggers I'm familiar with don't deal with any of > that very well, if at all. I agree that debugging any Inline code is going to be difficult (it's already tricky enough debugging C modules generated by SWIG), but as far as editor support goes - the "fancy IDE" of choice (XEmacs, of course) actually can support such mixed-language editing, once you've loaded up the appropriate elisp and done some customization... Have you seen the "MMM" package? MMM stands for "Multiple Major Modes" which allows you to have, just like the name suggests, multiple major modes within one buffer. I have used this successfully for editing HTML with embedded JavaScript (although I hate to admit that I have written such junk!). You can get it at: http://members.tripod.com/gchen2/xemacs/ This package seems to be little-known. Maybe it should be added to the XEmacs package database? (I'll bring this up on xemacs-beta) FWIW, -C From barry@zope.com Thu Aug 23 18:33:29 2001 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 23 Aug 2001 13:33:29 -0400 Subject: [Python-Dev] Re: Inline::caveat References: <15237.12019.841104.61849@nyx.dyndns.org> Message-ID: <15237.15977.879565.852317@anthem.wooz.org> >>>>> "CGW" == Charles G Waldman writes: CGW> Have you seen the "MMM" package? MMM stands for "Multiple CGW> Major Modes" which allows you to have, just like the name CGW> suggests, multiple major modes within one buffer. I have CGW> used this successfully for editing HTML with embedded CGW> JavaScript (although I hate to admit that I have written such CGW> junk!). You can get it at: CGW> http://members.tripod.com/gchen2/xemacs/ Except that trying to download the actual mmm.el.gz file it puts me into a Tripod infloop. I can't actually get at the file. Can you send me a copy? CGW> This package seems to be little-known. Maybe it should be CGW> added to the XEmacs package database? (I'll bring this up on CGW> xemacs-beta) Indeed, it sounds cool. -Barry From thomas.heller@ion-tof.com Thu Aug 23 18:49:40 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 23 Aug 2001 19:49:40 +0200 Subject: [Python-Dev] Inline and SWIG References: <200108230601.SAA04304@s454.cosc.canterbury.ac.nz> Message-ID: <05c601c12bfb$fcd46c50$e000a8c0@thomasnotebook> From: "Greg Ewing" > David Beazley : > > > I've looked briefly at Inline and agree that it's a pretty nifty > > approach. However, usability aside, it's got a number of pretty major > > limitations once you start getting into structure wrapping, C++ > > classes, and other advanced types of extension wrapping. > > Indeed, you've put your finger on what seems like the > biggest problem with the current crop of semi-automatic > extension generators. > > I don't think there is ever going to be a completely > automatic solution to this, because, once you get beyond > the basic types like ints, floats and strings, there is > no unique mapping between Python types and C types. > > While the tool could provide some sort of default > translation for arbitrary types, it's unlikely to be > what you need to interface directly with some existing > C library you're trying to wrap. So you're faced with > writing a lot of tedious and error-prone code for messing > with PyObjects and reference counts. > > I've had some ideas floating around in my head for a > while concerning what to do about this. The key > observation is that such code usually spends a lot > more lines manipulating Python data than it does C > data, which leads me to the conclusion that C is the > wrong language to write it in. > > What's the right language? One that's good at manipulating > Python objects. Which language is the best at doing > that? Why, Python, of course. But not just Python, > because we need to be able to manipulate C data > structures as well. So we need a language that looks > like Python at one end and C at the other... I've written (well, it's not finished and maybe never will) an extension allowing to create or wrap and manipulate C structures and unions from Python. The basics are similar to an OO-ified struct module, but the format codes are extensible. It even handles the Microsoft TypeInfo structures, which are full of recursions. Makes a pretty good pair with a calldll-like extension module. Thomas From paulp@ActiveState.com Thu Aug 23 19:31:29 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 23 Aug 2001 11:31:29 -0700 Subject: [Python-Dev] Inline and SWIG References: <200108230601.SAA04304@s454.cosc.canterbury.ac.nz> Message-ID: <3B854C01.A20AE630@ActiveState.com> Greg Ewing wrote: > >... > > What's the right language? One that's good at manipulating > Python objects. Which language is the best at doing > that? Why, Python, of course. But not just Python, > because we need to be able to manipulate C data > structures as well. So we need a language that looks > like Python at one end and C at the other... You are describing inline. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From paulp@ActiveState.com Thu Aug 23 19:36:11 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 23 Aug 2001 11:36:11 -0700 Subject: [Python-Dev] Re: Inline code in other languages References: <15236.14289.311782.621168@anthem.wooz.org> <15236.21449.231509.725435@anthem.wooz.org> Message-ID: <3B854D1B.1DA8191C@ActiveState.com> Lars Marius Garshol wrote: > >... > > Why not make it import C files, instead of putting them inline? I > guess you very often want to reuse the C functions in several > different places anyway, and you'd mostly want to have includes and > stuff as well. If you want to reuse a C function then you put it in a seperate file and use #include to include it. Allowing inline code is a choice...the programmer can choose to modularize if it helps him or her. I think there is a reason PHP is so much more popular than DTML-like solutions (duck!). > This way you can also get rid of the MD5-ing and just use ordinary > modification times to figure out what needs to be recompiled. MD5-ing is more reliable anyhow. Some build systems (e.g. CONS, SCONS) are moving from mod times to build times anyhow. > You can't inline pieces smaller than functions anyway, and to me the > mix between Python and C code looks very ugly. Beauty is in the eye of the beholder....I think that forcing stuff into separate files for aesthetic or philosophical reasons is severe. Ultimately there is an important usability benefit in allowing inlining (and I'll refer again to the amazing popularity of PHP). -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From thomas.heller@ion-tof.com Thu Aug 23 19:48:36 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 23 Aug 2001 20:48:36 +0200 Subject: [Python-Dev] Customizing the binding of attributes Message-ID: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook> In Python 2.2, is it possible to customize the binding of class attributes, to emulate the normal behaviour of methods (function -> unbound method -> bound method)? I think I have read something about this, but I can't find it anymore. Thomas From guido@python.org Thu Aug 23 20:23:42 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 23 Aug 2001 15:23:42 -0400 Subject: [Python-Dev] Customizing the binding of attributes In-Reply-To: Your message of "Thu, 23 Aug 2001 20:48:36 +0200." <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook> References: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook> Message-ID: <200108231923.f7NJNg307631@odiug.digicool.com> > In Python 2.2, is it possible to customize the binding > of class attributes, to emulate the normal behaviour > of methods (function -> unbound method -> bound method)? > > I think I have read something about this, but I can't > find it anymore. I'm not sure what you are asking about, but let me give an example. You can write an auxiliary class that "describes" an instance attribute. Instances of this "descriptor" class are stored in a class dict. The descriptor class implements a method __get__ which is called in two situations: (a) when a class attribute is retrieved, .__get__(None, ) is called; this can return an unbound method (b) when an instance attribute is retrieved, .__get__(, References: <200108230601.SAA04304@s454.cosc.canterbury.ac.nz> <05c601c12bfb$fcd46c50$e000a8c0@thomasnotebook> Message-ID: <15237.22789.951079.581327@grendel.digicool.com> Thomas Heller writes: > Makes a pretty good pair with a calldll-like extension module. I guess calldll is very different from the dl module, which has documentation. Is anyone up to writing docs for calldll? I've CC'd to the Doc-SIG since there may be volunteers there ;-) -- please remove python-dev from doc-related followups. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From gward@python.net Thu Aug 23 21:17:45 2001 From: gward@python.net (Greg Ward) Date: Thu, 23 Aug 2001 16:17:45 -0400 Subject: [Python-Dev] Putting a builtin module into a package In-Reply-To: <15234.34090.225332.420431@beluga.mojam.com>; from skip@pobox.com on Tue, Aug 21, 2001 at 10:58:34AM -0500 References: <20010821145632.7CC15303181@snelboot.oratrix.nl> <15234.34090.225332.420431@beluga.mojam.com> Message-ID: <20010823161745.A2375@gerg.ca> On 21 August 2001, Skip Montanaro said: > > Jack> Or should I rename the module to an underscore name and simply put > Jack> a wrapper module in the package? > > This seems the safest (and the sanest) to me. I realize you're thinking > mostly about MacOS, but putting object files inside packages that are > otherwise Python code makes the whole package platform-dependent. I think Jack was talking about *built-in* extensions, not .so (or .pyd) files. Dynamically loaded extensions are indeed easy -- you just put 'em in the package directory, and it Just Works. Yes, this means that people maintaining a multi-architecture installation have to have prefix != external-prefix, but they already do: /usr/bin/python is platform-specific! Greg -- Greg Ward - Unix bigot gward@python.net http://starship.python.net/~gward/ I just read that 50% of the population has below median IQ! From paulp@ActiveState.com Thu Aug 23 22:12:32 2001 From: paulp@ActiveState.com (Paul Prescod) Date: Thu, 23 Aug 2001 14:12:32 -0700 Subject: [Python-Dev] Perl5->Perl6 References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> <5.1.0.14.0.20010822153946.08c02de8@tuatha.sidhe.org> <5.1.0.14.0.20010822170534.08c50170@tuatha.sidhe.org> <20010823105241.B8531@netthink.co.uk> Message-ID: <3B8571C0.4ABCEDC5@ActiveState.com> Simon Cozens wrote: > >... > > Also don't forget that we have a prototype of Perl 6 which implements pretty > much everything that Larry has specified in Apocalypses 1 and 2. > > It is a patch of nearly 200 lines to the current Perl 5 interpreter. > > We're not changing *that* drastically, folks. According to Damian there will be generators and a first-class class syntax and method getters/setters and metaclasses and optional static type declarations and ... Larry just hasn't got to the heavy part of the Camel book. :-) -- Take a recipe. Leave a recipe. Python Cookbook! http://www.ActiveState.com/pythoncookbook From Samuele Pedroni Thu Aug 23 22:56:42 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Thu, 23 Aug 2001 23:56:42 +0200 (MET DST) Subject: [Python-Dev] PEP 252, PEP 253 and jython issues Message-ID: <200108232156.XAA09481@core.inf.ethz.ch> Hi. OK, here is my major concern Given the following three types subclassed in Pyhon: class A(CSpecialList): ... class B(list): ... class C(A,B): ... where CSpecialList is a subtype of list written in C. Now the old mro would be: C A CSpecialList list object B list object that translate to the new mro C A CSpecialList B list object The question is it possible for CSpecialList for the code that define for example sq_ass_slice to use sq_slice code in a safe manner? (AFAIK listobject does something similar) And how can safely CSpecialList invoke a "super" behaviour, it is safe for it to refer to the behaviour offered by list. In principle given the mro e.g. a __getslice__ redefined in B should shadow such a behaviour? A natural way to design PEP252, and PEP253 for Jython would be start to start from the equation: object = org.python.core.PyObject then it is necessary to build a class that truly correspond to the type metatype, actually jython fakes using the metatype used for Java Classes for that. It would make sense to use Java subclassing to implement type subclassing, at least at layout level this does not clash with the multiple inheritance rule in best_base. And actually is also how Jython now works: for example PyList extends PySequence that extends PyObject. But than there is the issue of method resolution order: from the viewpoint of Python code we can implement anything, not that easy ... But the at the Java level, where we construct types, the codebase uses the normal single inheritance of java and the codebase is full of super.foo invocations and of methods that cross-call each other (potentially overriden) versions, and this happen also for behaviour that correspond to the slots/ __foo__ special methods of CPython. That's why the two questions are important? regards. From neilh@scintilla.org Thu Aug 23 23:15:53 2001 From: neilh@scintilla.org (Neil Hodgson) Date: Fri, 24 Aug 2001 08:15:53 +1000 Subject: [Python-Dev] Re: Inline::caveat References: <15237.12019.841104.61849@nyx.dyndns.org> Message-ID: <012b01c12c21$2e332050$0acc8490@neil> Charles G Waldman: > ... but as > far as editor support goes - the "fancy IDE" of choice (XEmacs, of > course) actually can support such mixed-language editing, once you've > loaded up the appropriate elisp and done some customization... How do you find the language boundaries to enable mixed language editing or debugging? If Python gets an inline module, I'd hope to see enough lexical clues so that an editor can work out where the C++ is so it can be treated as C++. Neil From greg@cosc.canterbury.ac.nz Thu Aug 23 23:55:00 2001 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 24 Aug 2001 10:55:00 +1200 (NZST) Subject: [Python-Dev] Inline and SWIG In-Reply-To: <3B854C01.A20AE630@ActiveState.com> Message-ID: <200108232255.KAA04393@s454.cosc.canterbury.ac.nz> Paul Prescod : > You are describing inline. No, I'm not describing Inline, that's the point. I have some fairly detailed ideas on what I am describing, but this message is too small to contain them... As a brief example, the Inline::C example posted earlier would look something like this: cdef printf(char *,...): extern def hello(s): printf("Hello, world, %s\n", s) But that's way too simple an example to really convey anything. More later... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From gstein@lyra.org Fri Aug 24 03:30:00 2001 From: gstein@lyra.org (Greg Stein) Date: Thu, 23 Aug 2001 19:30:00 -0700 Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home In-Reply-To: <200108231443.f7NEhb505947@odiug.digicool.com>; from guido@python.org on Thu, Aug 23, 2001 at 10:43:37AM -0400 References: <20010823070259.X16727@lyra.org> <15237.4961.597225.931722@anthem.wooz.org> <200108231443.f7NEhb505947@odiug.digicool.com> Message-ID: <20010823193000.N26054@lyra.org> On Thu, Aug 23, 2001 at 10:43:37AM -0400, Guido van Rossum wrote: > > >>>>> "KY" == Ka-Ping Yee writes: > > KY> Just a suggestion, but how about sys.consts? Like os.path, as > > KY> in sys.consts.CO_NESTED, sys.consts.CO_FUTURE_DIVISION, etc. > > KY> So there's only one new object in sys, and it can contain the > > KY> weird internal stuff. :) >... > +0. It's the better solution but I bet it's more work to code it than > _sysconst. I'm not volunteering, and I want Tim to have time to help > with the type/class unification, which is behind schedule. Hmm... that sounds a lot like trading off expediency for Rightness. If it is the better solution, then that should be done. If you don't want Tim to work on it, then that is a different problem. It could be done later, or a volunteer could step up to do it. It isn't exactly a time critical change, now is it? Cheers, -g -- Greg Stein, http://www.lyra.org/ From beazley@cs.uchicago.edu Fri Aug 24 03:26:09 2001 From: beazley@cs.uchicago.edu (David Beazley) Date: Thu, 23 Aug 2001 21:26:09 -0500 (CDT) Subject: [Python-Dev] Near Final PEP 237 - Unifying Long Integers and Integers In-Reply-To: References: Message-ID: <15237.47937.50866.571629@gargoyle.cs.uchicago.edu> Guido van Rossum writes: > Please comment on a new version of PEP 237: > > Please mail comments on the PEP to python-dev@python.org or to me. > The PEP looks good to me. Can I make a very small C API feature request? (assuming this hasn't been implemented already). Since "integers" can now have arbitrary precision and can represent large unsigned values, can you add three new format characters to PyArg_ParseTuple() and Py_BuildValue() for the C datatypes "unsigned int", "unsigned long", and "unsigned long long"? The "u" and "l" namespace is a little crowded (and I don't think you would want to break that). However, here's one idea: 'I' - unsigned int (consistent with H and B) 'p' - unsigned long ('p' is for positive) 'P' - unsigned long long Cheers, Dave From jeremy@zope.com Fri Aug 24 03:59:30 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 23 Aug 2001 22:59:30 -0400 (EDT) Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home In-Reply-To: <20010823193000.N26054@lyra.org> References: <20010823070259.X16727@lyra.org> <15237.4961.597225.931722@anthem.wooz.org> <200108231443.f7NEhb505947@odiug.digicool.com> <20010823193000.N26054@lyra.org> Message-ID: <15237.49938.566295.179733@slothrop.digicool.com> >>>>> "GS" == Greg Stein writes: GS> It isn't exactly a time critical change, now is it? Not any more time critical than documenting the new httplib API . speaking-as-an-expert-on-putting-off-documentation-ly y'rs, Jeremy From tim.one@home.com Fri Aug 24 04:31:57 2001 From: tim.one@home.com (Tim Peters) Date: Thu, 23 Aug 2001 23:31:57 -0400 Subject: [Python-Dev] Re: Cuddly constants seek loving, stable home In-Reply-To: <15237.49938.566295.179733@slothrop.digicool.com> Message-ID: Something that might change your views about Elegance here: calling these particular constants "cuddly" was intended to be ironic. If *any* program uses one of these, I can pretty much guarantee it will eventually break. They're not part of a public API. The two current uses (__future__ and pyassem) are part of the implementation of the core, and just happen to be coded in Python. For that reason I'm -1 on putting these particular constants in a location with an inviting name, or making their own names prettier, etc etc -- I'm even -1 on documenting them (outside of comments in the internal C header file where they're #define'd). Rather than argue about it, I'm going to revert the changes to new and __future__, backing off to defining them by hand in __future__.py. doesn't-stop-anyone-else-from-doing-the-right-thing-later-just-think- twice-about-how-right-it-is-ly y'rs - tim From eric@scipy.org Fri Aug 24 04:05:57 2001 From: eric@scipy.org (eric jones) Date: Thu, 23 Aug 2001 23:05:57 -0400 Subject: [Python-Dev] Re: Inline and SWIG References: <15236.26466.84309.855339@gargoyle.cs.uchicago.edu> <15236.27364.304751.836868@gargoyle.cs.uchicago.edu> <3B84792B.7DDF7C1B@ActiveState.com> Message-ID: <003701c12c49$b33bf3d0$c27ba8c0@ericlaptop> The scipy.compiler modules (www.scipy.org) is aimed at Numeric arrays and is slightly different than inlining C code in that it translates a Python Numeric expression to C++(blitz++ arrays), compiles the result to a function, and calls it automatically. It figures out the array types the local name space, and keeps up with the compiled module after the first compile so that they don't have to be compiled again. The entire process was greatly eased by Jeremy AST stuff, distutils, and blitz++. The most up-to-date description of it is here: http://www.scipy.org/site_content/tutorials/compiler_notes Much of this machinery is identical to what is needed for inline C, and I've tinkered with it a little. It looks like it won't be extremely hard (famous last words...) to handle inlining of numeric looping code. The limited focus on numerical stuff gets around many of the hairy issues involved with handling lists of arbitrary types, and got a usable tool out the door quickly. Luckily it also attacks the most expensive part of scientific codes and provides 2-10 speed-up over Numeric array expressions. I'd like to circle out from the numeric-only approach and tackle some of the more difficult typing issues after the numeric stuff all works. Any input on this thing is welcome. Patrick Miller's PyCOD is another compiler that takes a different approach to compiling (processing bytecodes). It's a little more ambitious than scipy.compiler because it handles more than just numeric expressions. I don't know what its release status is either. The other compiler project I know about is Psyco discussed here: http://aspn.activestate.com/ASPN/Mail/Message/609552 It sounds extremely cool and is the most ambitious of the three. Not sure how it affects inline C though. Seems more like this aims to make inline C obsolete before it even exists. > I'd recommend that people track down the authors of those modules, as I > do think there is a great opportunity there. (I take credit for > inspiring Eric Jones by mentioning Inline:: to him in Long Beach =). Heck, if you'll sign up to add support for array broadcasting, I'll name it scipy.da_compiler... eric ----- Original Message ----- From: "David Ascher" To: "David Beazley" Cc: "Ka-Ping Yee" ; ; ; ; Sent: Wednesday, August 22, 2001 11:31 PM Subject: Re: [Python-Dev] Re: Inline and SWIG > Note that I know of at least two if not three different inline-like > projects for Python. > > One is the compiler module in scipy (Eric Jones & Travis Oliphant): > > http://scipy.net/cgi-bin/viewcvs.cgi/~checkout~/scipy/compiler/doc/readme?re v=1.2&content-type=text/plain > > One is the stuff that Pat Miller from LLNL presented at OSCON in San > Diego, which sounded similar in spirit (alas, I missed the talk, and > don't know anything about its availability -- Pat?), and there was a > third one which I heard mentioned but don't know any details about. > Eric, do you remember what I'm talking about? > > I'd recommend that people track down the authors of those modules, as I > do think there is a great opportunity there. (I take credit for > inspiring Eric Jones by mentioning Inline:: to him in Long Beach =). > > --david ascher From thomas.heller@ion-tof.com Fri Aug 24 08:00:31 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 24 Aug 2001 09:00:31 +0200 Subject: [Python-Dev] Customizing the binding of attributes References: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook> <200108231923.f7NJNg307631@odiug.digicool.com> Message-ID: <09f401c12c6a$7717ab80$e000a8c0@thomasnotebook> From: "Guido van Rossum" > I'm not sure what you are asking about, but let me give an example. > You can write an auxiliary class that "describes" an instance > attribute. Instances of this "descriptor" class are stored in a class > dict. The descriptor class implements a method __get__ which is > called in two situations: > > (a) when a class attribute is retrieved, .__get__(None, ) > is called; this can return an unbound method > > (b) when an instance attribute is retrieved, .__get__(, is called; this can return a bound method > > Does this help? > Yes, thanks. Exactly what I meant. You only forgot to mention that the descriptor class must derive from object: is this intended? I needed this to write class X: method = myinstancemethod(func) instead of class X: pass X.method = new.instancemethod(func, None, X) where myinstancemethod is the descriptor class, and func is any callable object (maybe a function implemented in C). Two additional comments: - Shouldn't (my)instancemethod be a builtin? Similar to staticmethod and classmethod? - The following seems to be a bug (remember, I'm on Windows): C:\>c:\python22\python.exe Python 2.2a2 (#22, Aug 22 2001, 01:24:03) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> class myinstancemethod(object): ... def __init__(self, func): ... self.func = func ... def __get__(self, inst, klass=None): ... import new ... return new.instancemethod(self.func, inst, klass) ... >>> class X: ... meth = myinstancemethod(lambda *x: x) ... >>> X().meth of <__main__.X instance at 007C7F04>> >>> X().meth() (<__main__.X instance at 007E80E4>,) >>> X.meth Traceback (most recent call last): File "", line 1, in ? SystemError: NULL object passed to Py_BuildValue Thomas From guido@python.org Fri Aug 24 10:08:48 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 24 Aug 2001 05:08:48 -0400 Subject: [Python-Dev] Near Final PEP 237 - Unifying Long Integers and Integers In-Reply-To: Your message of "Thu, 23 Aug 2001 21:26:09 CDT." <15237.47937.50866.571629@gargoyle.cs.uchicago.edu> References: <15237.47937.50866.571629@gargoyle.cs.uchicago.edu> Message-ID: <200108240908.FAA03478@cj20424-a.reston1.va.home.com> > Since "integers" can now have arbitrary precision and can represent > large unsigned values, can you add three new format characters to > PyArg_ParseTuple() and Py_BuildValue() for the C datatypes "unsigned int", > "unsigned long", and "unsigned long long"? > > The "u" and "l" namespace is a little crowded (and I don't think you > would want to break that). However, here's one idea: > > 'I' - unsigned int (consistent with H and B) > 'p' - unsigned long ('p' is for positive) > 'P' - unsigned long long Good idea. I've added this to the SF bug tracker as a feature request. Hopefully someone will submit a patch too! --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Aug 24 11:03:34 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 24 Aug 2001 06:03:34 -0400 Subject: [Python-Dev] Customizing the binding of attributes In-Reply-To: Your message of "Fri, 24 Aug 2001 09:00:31 +0200." <09f401c12c6a$7717ab80$e000a8c0@thomasnotebook> References: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook> <200108231923.f7NJNg307631@odiug.digicool.com> <09f401c12c6a$7717ab80$e000a8c0@thomasnotebook> Message-ID: <200108241003.GAA04383@cj20424-a.reston1.va.home.com> > From: "Guido van Rossum" > > I'm not sure what you are asking about, but let me give an example. > > You can write an auxiliary class that "describes" an instance > > attribute. Instances of this "descriptor" class are stored in a class > > dict. The descriptor class implements a method __get__ which is > > called in two situations: > > > > (a) when a class attribute is retrieved, .__get__(None, ) > > is called; this can return an unbound method > > > > (b) when an instance attribute is retrieved, .__get__(, > is called; this can return a bound method > > > > Does this help? From: "Thomas Heller" > Yes, thanks. Exactly what I meant. You only forgot to mention that > the descriptor class must derive from object: is this intended? Yes, almost all new features require you to switch to new-style classes; the classic class implementation is left unchanged (as much as possible) for compatibility. In particular, classic classes can't define __get__ to the effect above. These descriptors semi-work as class attributes of classic classes, but the corresponding __set__ feature does not, because classic classes *always* store in __dict__ upon assignment; new-style classes first look for a descriptor implementing __set__ (really the C dispatch function tp_descr_set) and use that to override the default assignment. > I needed this to write > > class X: > method = myinstancemethod(func) > > instead of > > class X: > pass > X.method = new.instancemethod(func, None, X) > > where myinstancemethod is the descriptor class, and func > is any callable object (maybe a function implemented in C). > > Two additional comments: > > - Shouldn't (my)instancemethod be a builtin? Similar to > staticmethod and classmethod? Yes. I just checked this in. The new name is 'getset' unless you have a better idea. > - The following seems to be a bug (remember, I'm on Windows): > > C:\>c:\python22\python.exe > Python 2.2a2 (#22, Aug 22 2001, 01:24:03) [MSC 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> class myinstancemethod(object): > ... def __init__(self, func): > ... self.func = func > ... def __get__(self, inst, klass=None): > ... import new > ... return new.instancemethod(self.func, inst, klass) > ... > >>> class X: > ... meth = myinstancemethod(lambda *x: x) > ... > >>> X().meth > of <__main__.X instance at 007C7F04>> > >>> X().meth() > (<__main__.X instance at 007E80E4>,) > >>> X.meth > Traceback (most recent call last): > File "", line 1, in ? > SystemError: NULL object passed to Py_BuildValue Yup. Thanks! Try this patch: *** typeobject.c 2001/08/22 19:24:42 2.43 --- typeobject.c 2001/08/24 10:05:25 *************** *** 2806,2811 **** --- 2806,2815 ---- Py_INCREF(self); return self; } + if (obj == NULL) + obj = Py_None; + if (type == NULL) + type = Py_None; return PyObject_CallFunction(get, "OOO", self, obj, type); } --Guido van Rossum (home page: http://www.python.org/~guido/) From Samuele Pedroni Fri Aug 24 11:16:28 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Fri, 24 Aug 2001 12:16:28 +0200 (MET DST) Subject: [Python-Dev] jython issues and PEP 252, PEP 253 Message-ID: <200108241016.MAA07882@core.inf.ethz.ch> Hi. I'm feeling a bit annoying and I don't like that. I'm not in hurry for a answer for my questions on PEP253 and PEP252 but were them at least parseable, did they make sense? or are too vague, and underdetailed should I bring more elements in the picture? regards, Samuele Pedroni. From jack@oratrix.nl Fri Aug 24 11:48:02 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri, 24 Aug 2001 12:48:02 +0200 Subject: [Python-Dev] Inline and SWIG In-Reply-To: Message by David Beazley , Wed, 22 Aug 2001 21:16:02 -0500 (CDT) , <15236.26466.84309.855339@gargoyle.cs.uchicago.edu> Message-ID: <20010824104802.B28D0303181@snelboot.oratrix.nl> > The *real* problem IMHO is actually > trying to figure out how to properly handle all of the facets of the > C/C++ type system (pointers, references, qualifiers, classes, > templates, typedef, inheritance, multiple inheritance, member > pointers, pass by value, etc.) and how that's supposed to hook into > Python. And at this time I customarily put in my plug for bgen. Bgen is far ahead of SWIG in this respect, and completely customizable in Python. It will definitely not do all the things you specify, but it'll do a lot more than SWIG, -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From guido@python.org Fri Aug 24 13:54:10 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 24 Aug 2001 08:54:10 -0400 Subject: [Python-Dev] PEP 252, PEP 253 and jython issues In-Reply-To: Your message of "Thu, 23 Aug 2001 23:56:42 +0200." <200108232156.XAA09481@core.inf.ethz.ch> References: <200108232156.XAA09481@core.inf.ethz.ch> Message-ID: <200108241254.IAA05377@cj20424-a.reston1.va.home.com> > OK, here is my major concern > > Given the following three types subclassed in Pyhon: > class A(CSpecialList): ... > > class B(list): ... > > class C(A,B): ... > > where CSpecialList is a subtype of list written in C. > > Now the old mro would be: > > C A CSpecialList list object B list object > > that translate to the new mro > > C A CSpecialList B list object Correct. > The question is it possible for CSpecialList for the code that define > for example sq_ass_slice to use sq_slice code in a safe manner? > (AFAIK listobject does something similar) I'm not 100% sure I understand this question, but CSpecialList can certainly override sq_ass_slice. Then, if it wants to call self.__slice__ (in Python terms) it could call obj->ob_type->tp_as_sequence->sq_slice -- this would call C's obj->ob_type->tp_as_sequence->__slice__ if it had one. > And how can safely CSpecialList invoke a "super" behaviour, it is safe > for it to refer to the behaviour offered by list. In principle > given the mro e.g. a __getslice__ redefined in B should shadow > such a behaviour? I don't know if cooperative super calls are possible at the C level. I certainly didn't define an API for this. I didn't define a Python level API either, but I know how to o it and it can even be coded in Python. Hm, I guess that makes it a theoretical possibility at the C level. I don't expect this to be a common use pattern though (famous last words :-). > A natural way to design PEP252, and PEP253 for Jython would > be start to start from the equation: > > object = org.python.core.PyObject Yes. > then it is necessary to build a class that truly correspond to the type > metatype, actually jython fakes using the metatype used for Java Classes for > that. > > It would make sense to use Java subclassing to implement > type subclassing, at least at layout level this does not clash with > the multiple inheritance rule in best_base. So you would rule out multiple inheritance of types? Fine with me, but you may regret this a few yearsfrom now (when we all code multiple inheritance all the time :-). > And actually is also how Jython now works: for example > PyList extends PySequence that extends PyObject. And that's how it should be. > But than there is the issue of method resolution order: from the viewpoint > of Python code we can implement anything, not that easy ... > > But the at the Java level, where we construct types, the codebase uses the > normal single inheritance of java and the codebase is full of super.foo > invocations and of methods that cross-call each other (potentially overriden) > versions, and this happen also for behaviour that correspond to the slots/ > __foo__ special methods of CPython. > > That's why the two questions are important? I see. CPython doesn't have this problem (yet) because it doesn't use inheritance at the C level. I propose that you just try to live with this. A safe rule would be to require that all Python classes in a given inheritance graph should derive from the same C/Java class. I don't know if we should enforce this or just warn about it in the documentation. I don't want to force you to call a cooperative super method at the Java level -- it would be very slow, I suspect... --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Fri Aug 24 14:32:22 2001 From: nas@python.ca (Neil Schemenauer) Date: Fri, 24 Aug 2001 06:32:22 -0700 Subject: [Python-Dev] Customizing the binding of attributes In-Reply-To: <200108241003.GAA04383@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Aug 24, 2001 at 06:03:34AM -0400 References: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook> <200108231923.f7NJNg307631@odiug.digicool.com> <09f401c12c6a$7717ab80$e000a8c0@thomasnotebook> <200108241003.GAA04383@cj20424-a.reston1.va.home.com> Message-ID: <20010824063222.A12847@glacier.fnational.com> Guido van Rossum wrote: > In particular, classic classes can't define __get__ to the effect > above. Why __get__() and __set__() and not get() and set()? Neil From Samuele Pedroni Fri Aug 24 14:57:25 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Fri, 24 Aug 2001 15:57:25 +0200 (MET DST) Subject: [Python-Dev] Comments: PEP 252, PEP 253 and jython issues Message-ID: <200108241357.PAA16863@core.inf.ethz.ch> Thanks for the answers, I'm happy the questions were parseable, Here are some comments, I think they may be meanigful also for CPython > > The question is it possible for CSpecialList for the code that define > > for example sq_ass_slice to use sq_slice code in a safe manner? > > (AFAIK listobject does something similar) > > I'm not 100% sure I understand this question, but CSpecialList can > certainly override sq_ass_slice. Then, if it wants to call > self.__slice__ (in Python terms) it could call > obj->ob_type->tp_as_sequence->sq_slice -- this would call C's > obj->ob_type->tp_as_sequence->__slice__ if it had one. I overlooked this, I think we can implement that also in Jython. An important point: should that be the only way to use a Python level exposed and overridable method from C code, is that left to the programmer ? The problem is that with the new mro possibly independently C defined things get interleaved with Python level code, so I imagine that the effect should be a bit disciplined ... I saw that list_ass_slice calls list_slice directly, is that OK? > > And how can safely CSpecialList invoke a "super" behaviour, it is saf > > for it to refer to the behaviour offered by list. In principle > > given the mro e.g. a __getslice__ redefined in B should shadow > > such a behaviour? > > I don't know if cooperative super calls are possible at the C level. > I certainly didn't define an API for this. I didn't define a Python > level API either, but I know how to o it and it can even be coded in > Python. Hm, I guess that makes it a theoretical possibility at the C > level. > > I don't expect this to be a common use pattern though (famous last > words :-). But it is certainely a pattern that a writer of Jython types would expect to use. The problem is still that C/Java stuff with the new mro get interleaved with Python level behaviour. IMHO you would better discipline this and also the above point . I don't know if there is some GUI toolkit that implement inheritance using C and not some language enforcing OO like C++, etc. Other OO languages like Smalltalk, Self, or Java allow only to define primitives or single methods to add lowel-level functionality, they don't try to allow subtyping at that level, the previous scenario in Python where you could define just opaque types, and in particular your C code never/normally got interleaved with Python behaviour was on that line. At worst you could say that you allow subtyping but not code sharing, but then even a C method for an object should be very careful using another overridable method of the very same object. OK you could say: that's C level, everybody should care for herself, but with Java we cannot do the same. > > It would make sense to use Java subclassing to implement > > type subclassing, at least at layout level this does not clash with > > the multiple inheritance rule in best_base. > > So you would rule out multiple inheritance of types? Fine with me, > but you may regret this a few yearsfrom now (when we all code multiple > inheritance all the time :-). No it is ruled out at Java level, not Jython level. But it is not also ruled out in CPython at C level? > > But than there is the issue of method resolution order: from the viewpoint > > of Python code we can implement anything, not that easy ... > > > > But the at the Java level, where we construct types, the codebase uses the > > normal single inheritance of java and the codebase is full of super.foo > > invocations and of methods that cross-call each other (potentially overriden) > > versions, and this happen also for behaviour that correspond to the slots/ > > __foo__ special methods of CPython. > > > > That's why the two questions are important? > > I see. CPython doesn't have this problem (yet) because it doesn't use > inheritance at the C level. see above. > I propose that you just try to live with > this. Related e.g. to Zope porting projects there have been a lot of pressure on jython-dev regarding how to code new types and metaclasses, A Jython type writer expect to be able to use Java naturally (i.e. use inheritance), for that, so the situation isn't that easy, is not even easy with the current codebase, because given the new mro one can even put some Python behaviour between: PyList PythonLevelCode and PyObject :-( (*) > A safe rule would be to require that all Python classes in a > given inheritance graph should derive from the same C/Java class. I That would mean that all C/Java behaviour is always only at the top of the mro, right? I had the same idea because this would make my life easier. A problem: some of the Zope porters reported that there are classes that inherits from more than a single ExtensionClass in Zope :-( don't know if that is true. (**) > don't know if we should enforce this or just warn about it in the > documentation. It would make our (jython-dev) life easier and avoid some impredictable problem even with CPython and code sharing, but see previous note (**) > I don't want to force you to call a cooperative super > method at the Java level -- it would be very slow, I suspect... Yes ... very slow and we would have to use that also at the very core of the hierarchy, see (*) but the problem is more complicated, in Java given three classes C extends B extends A and a method m C.m overrides B.m overrides A.m there is not direct way (even using reflection) to apply B.m behaviour to a C object, unless the programmer has left some hook in C using super or in B ( a method B_m e.g.). ... complicated :( regards. From guido@python.org Fri Aug 24 15:04:43 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 24 Aug 2001 10:04:43 -0400 Subject: [Python-Dev] Customizing the binding of attributes In-Reply-To: Your message of "Fri, 24 Aug 2001 06:32:22 PDT." <20010824063222.A12847@glacier.fnational.com> References: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook> <200108231923.f7NJNg307631@odiug.digicool.com> <09f401c12c6a$7717ab80$e000a8c0@thomasnotebook> <200108241003.GAA04383@cj20424-a.reston1.va.home.com> <20010824063222.A12847@glacier.fnational.com> Message-ID: <200108241404.KAA05644@cj20424-a.reston1.va.home.com> > Guido van Rossum wrote: > > In particular, classic classes can't define __get__ to the effect > > above. > > Why __get__() and __set__() and not get() and set()? Because they're mapped to the dispatch slots tp_descr_get and tp_descr_set. (Note that PEP 252 has get() and set() in some places, like the C API section. This is a mistake.) --Guido van Rossum (home page: http://www.python.org/~guido/) From Samuele Pedroni Fri Aug 24 15:21:14 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Fri, 24 Aug 2001 16:21:14 +0200 (MET DST) Subject: [Python-Dev] Afterthoughts: PEP 252, PEP 253 and jython issues Message-ID: <200108241421.QAA18057@core.inf.ethz.ch> Yes, I'm thinking furiously. GvR > A safe rule would be to require that all Python classes in a GvR > given inheritance graph should derive from the same C/Java class. I > That would mean that all C/Java behaviour is always only at the top > of the mro, right? > > I had the same idea because this would make my life easier. > > A problem: some of the Zope porters reported that there are classes > that inherits from more than a single ExtensionClass in Zope :-( > don't know if that is true. (**) Oops, this is already a problem with the actual best_base rule, I don't know if my reinterpretation is OK, but doesn't it basically states that all C types subclassed by a new C level class should be in a subtyping relationship that means: with A isa list isa object B isa special_list isa list isa object C isa object class D(A,B,C): ... is OK but with Z isa Cbadboy isa object then class D(Z,A) isn't allowed. but if that is already *not* *a* *problem* then I think the rule could be not that bad ... regards. From guido@python.org Fri Aug 24 15:35:45 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 24 Aug 2001 10:35:45 -0400 Subject: [Python-Dev] Comments: PEP 252, PEP 253 and jython issues In-Reply-To: Your message of "Fri, 24 Aug 2001 15:57:25 +0200." <200108241357.PAA16863@core.inf.ethz.ch> References: <200108241357.PAA16863@core.inf.ethz.ch> Message-ID: <200108241435.KAA05784@cj20424-a.reston1.va.home.com> > Thanks for the answers, I'm happy the questions were parseable, > Here are some comments, I think they may be meanigful also for CPython > > > > The question is it possible for CSpecialList for the code that define > > > for example sq_ass_slice to use sq_slice code in a safe manner? > > > (AFAIK listobject does something similar) > > > > I'm not 100% sure I understand this question, but CSpecialList can > > certainly override sq_ass_slice. Then, if it wants to call > > self.__slice__ (in Python terms) it could call > > obj->ob_type->tp_as_sequence->sq_slice -- this would call C's > > obj->ob_type->tp_as_sequence->__slice__ if it had one. > I overlooked this, I think we can implement that also in Jython. > > An important point: should that be the only way to use a Python level > exposed and overridable method from C code, is that left > to the programmer ? I'm not sure I understand this question. Can you give an example? I don't expect that calling into Python from C will be very common. There are ways to call any Python object from C if you need to. > The problem is that with the new mro possibly independently C defined > things get interleaved with Python level code, > so I imagine that the effect should be a bit disciplined ... I don't expect this to happen regularly. if you count on this I think you're on really thin ice. > I saw that list_ass_slice calls list_slice directly, is that OK? It's certainly safe -- any subclass of list will share the same structure lay-out (this is enforced by the subclassing machinery) so the list_slice call will find the object properly inititialized. There are lots of places where the C code doesn't bother to look for overridden methods -- the PyDict_* API is a common example. I have no intention of fixing this. > > > And how can safely CSpecialList invoke a "super" behaviour, it is saf > > > for it to refer to the behaviour offered by list. In principle > > > given the mro e.g. a __getslice__ redefined in B should shadow > > > such a behaviour? > > > > I don't know if cooperative super calls are possible at the C level. > > I certainly didn't define an API for this. I didn't define a Python > > level API either, but I know how to o it and it can even be coded in > > Python. Hm, I guess that makes it a theoretical possibility at the C > > level. > > > > I don't expect this to be a common use pattern though (famous last > > words :-). > > But it is certainely a pattern that a writer of Jython types would expect > to use. The problem is still that C/Java stuff with the new mro > get interleaved with Python level behaviour. If you think this will be a problem in Jython, you may have to provide a fast efficient Java-level "cooperative super" operation. > IMHO you would better discipline this and also the above point . What do you mean by "discipline"? > I don't know if there is some GUI toolkit that implement inheritance > using C and not some language enforcing OO like C++, etc. Sorry, I don't understand the relevance of this? > Other OO languages like Smalltalk, Self, or Java allow only to define > primitives or single methods to add lowel-level functionality, > they don't try to allow subtyping at that level, the previous > scenario in Python where you could define just opaque types, > and in particular your C code never/normally got interleaved with Python > behaviour was on that line. Well, if you like that better, maybe I should remove the instructions for subclassing at the C level? :-) My intention is to make the maximum number of different paradigms usable in Python. You can do cooperative multiple inheritance, but only of all your classes are designed with cooperation in mind. If some class uses C-level (or Java-level) inheritance and is not written cooperatively, you have to be a little careful using it as a base class in a cooperatively written class. I don't want to enforce cooperative coding at the Python level either: the "BaseClass.method(self, args)" approach is still valid, but you have to beware of the consequences. > At worst you could say that you allow subtyping but not code sharing, > but then even a C method for an object should be very careful using > another overridable method of the very same object. I definitely have to support code sharing. I think subclassing at the C level is useful to create variations of built-in objects with additional properties, e.g. numbers with additional operations or formatting parameters, dictionaries with certain restrictions on the keys, etc. I don't think such subclasses should be used in complex multiple inheritance lattices. > OK you could say: that's C level, everybody should care for herself, > but with Java we cannot do the same. Why not? > > > It would make sense to use Java subclassing to implement > > > type subclassing, at least at layout level this does not clash with > > > the multiple inheritance rule in best_base. > > > > So you would rule out multiple inheritance of types? Fine with me, > > but you may regret this a few yearsfrom now (when we all code multiple > > inheritance all the time :-). > No it is ruled out at Java level, not Jython level. > But it is not also ruled out in CPython at C level? Mostly because I don't provide a way to create multiple base classes. That would not be hard to add though -- if there's demand. > > > But than there is the issue of method resolution order: from the > > > viewpoint of Python code we can implement anything, not that > > > easy ... > > > > > > But the at the Java level, where we construct types, the > > > codebase uses the normal single inheritance of java and the > > > codebase is full of super.foo invocations and of methods that > > > cross-call each other (potentially overriden) versions, and this > > > happen also for behaviour that correspond to the slots/ __foo__ > > > special methods of CPython. > > > > > > That's why the two questions are important? > > > > I see. CPython doesn't have this problem (yet) because it doesn't use > > inheritance at the C level. > see above. > > > I propose that you just try to live with this. > Related e.g. to Zope porting projects there have been a lot of > pressure on jython-dev regarding how to code new types and > metaclasses, A Jython type writer expect to be able to use Java > naturally (i.e. use inheritance), for that, so the situation isn't > that easy, is not even easy with the current codebase, because given > the new mro one can even put some Python behaviour between: > > PyList PythonLevelCode and PyObject :-( (*) Just sa no. :-) > > A safe rule would be to require that all Python classes in a > > given inheritance graph should derive from the same C/Java class. I > That would mean that all C/Java behaviour is always only at the top > of the mro, right? If you mean at the end, yes. > I had the same idea because this would make my life easier. Sounds good to me. > A problem: some of the Zope porters reported that there are classes > that inherits from more than a single ExtensionClass in Zope :-( > don't know if that is true. (**) Me neither. But the problem is probably shallow -- Zope mostly uses mixins of various sorts, so these classes probably don't override stuff -- they just define new methods. (I haven't looked at what it would take to replace ExtensionClass with the new metaclasses, but that's definitely on the agenda.) > > don't know if we should enforce this or just warn about it in the > > documentation. > It would make our (jython-dev) life easier and avoid some impredictable > problem even with CPython and code sharing, > but see previous note (**) So enforce it and see how it turns out in practice. > > I don't want to force you to call a cooperative super > > method at the Java level -- it would be very slow, I suspect... > Yes ... very slow and we would have to use that also at the very core > of the hierarchy, see (*) > but the problem is more complicated, in Java given three classes > > C extends B extends A > and a method m > C.m overrides B.m overrides A.m > > there is not direct way (even using reflection) > to apply B.m behaviour to a C object, unless > the programmer has left some hook in C using super > or in B ( a method B_m e.g.). I guess C.m has to call super.m to invoke B.m, right? Does this mean that the following can't be made to work? class C(list): def add_spam(self): return list.append(self, "spam") It's legal Python (although in general questionable style of course). > ... complicated :( :-( --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Aug 24 15:52:28 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 24 Aug 2001 10:52:28 -0400 Subject: [Python-Dev] Afterthoughts: PEP 252, PEP 253 and jython issues In-Reply-To: Your message of "Fri, 24 Aug 2001 16:21:14 +0200." <200108241421.QAA18057@core.inf.ethz.ch> References: <200108241421.QAA18057@core.inf.ethz.ch> Message-ID: <200108241452.KAA05961@cj20424-a.reston1.va.home.com> > Yes, I'm thinking furiously. See if I can keep up with you. :-) > GvR > A safe rule would be to require that all Python classes in a > GvR > given inheritance graph should derive from the same C/Java class. I > > > That would mean that all C/Java behaviour is always only at the top > > of the mro, right? > > > > I had the same idea because this would make my life easier. > > > > A problem: some of the Zope porters reported that there are classes > > that inherits from more than a single ExtensionClass in Zope :-( > > don't know if that is true. (**) > Oops, this is already a problem with the actual best_base rule, > I don't know if my reinterpretation is OK, but doesn't it > basically states that all C types subclassed by a new C level class > should be in a subtyping relationship that means: > > with > A isa list isa object > B isa special_list isa list isa object > C isa object > > class D(A,B,C): ... is OK > > but with > > Z isa Cbadboy isa object > > then class D(Z,A) isn't allowed. Correct. You can't multiply inherit from two types that have a different structure layout. The C code must be able to safely cast a PyObject* to a PyListObject* or a Cbadboy* and if those don't have the same structure it's not safe. The implementation restricts this pretty effectively I believe. (You can't multiply inherit from list and dictionary, but you can from list and xxsubtype.spamlist.) > but if that is already *not* *a* *problem* then I think the rule could > be not that bad ... What do you mean by "already *not* *a* *problem*"? I hope "it is already forbidden". And "the rule" refers back to my proposal quoted above with "GvR >", right? --Guido van Rossum (home page: http://www.python.org/~guido/) From Samuele Pedroni Fri Aug 24 16:50:05 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Fri, 24 Aug 2001 17:50:05 +0200 (MET DST) Subject: [Python-Dev] Comments2: PEP 252, PEP 253 and jython issues Message-ID: <200108241550.RAA21561@core.inf.ethz.ch> [GvR] > > > The problem is that with the new mro possibly independently C defined > > things get interleaved with Python level code, > > so I imagine that the effect should be a bit disciplined ... > > I don't expect this to happen regularly. if you count on this I think > you're on really thin ice. No I'm not counting on that, I'm worried about this scenario A subclasses object and redefines __getattr__ B subclasses list then class C(B,A): ... will have the following mro: C B list A object In Jython codebase hypothetically but possibly list could contains calls like: __getattr__(...) and super.__getattr__ now A is in between list and object: what is the right thing here? I don't expect or invite users to do that, but your rules allow A to end up in that position in the mro, your philosophy seems to be this is thin ice and so let it be ... > > I saw that list_ass_slice calls list_slice directly, is that OK? > > It's certainly safe -- any subclass of list will share the same > structure lay-out (this is enforced by the subclassing machinery) so > the list_slice call will find the object properly inititialized. > > There are lots of places where the C code doesn't bother to look for > overridden methods -- the PyDict_* API is a common example. I have no > intention of fixing this. Fine with that, but ... > > > Other OO languages like Smalltalk, Self, or Java allow only to define > > primitives or single methods to add lowel-level functionality, > > they don't try to allow subtyping at that level, the previous > > scenario in Python where you could define just opaque types, > > and in particular your C code never/normally got interleaved with Python > > behaviour was on that line. > > Well, if you like that better, maybe I should remove the instructions > for subclassing at the C level? :-) No, I'm just saying that you are moving from a world were you had docume ntable C black-boxes and clear Python semantics, to a world where (but you say that would be rare) you can have Python code whose behaviour depends on very fine-grained decision inside C code. On one side your rules make it very easy to interleave Python and C code in the mro, on the other hand I agree that will be rarely done by the wary user because it's just surprise-land, don't know about the unwary ... > > My intention is to make the maximum number of different paradigms > usable in Python. You can do cooperative multiple inheritance, but > only of all your classes are designed with cooperation in mind. If > some class uses C-level (or Java-level) inheritance and is not written > cooperatively, you have to be a little careful using it as a base > class in a cooperatively written class. I don't want to enforce > cooperative coding at the Python level either: the > "BaseClass.method(self, args)" approach is still valid, but you have > to beware of the consequences. Do you feel is that a good thing, especially if new-style classes become the default? It is not a rethoric question, Python is becoming a little single dispatch CLOS with a MOP, descriptors etc but CLOS has only next-method and strange beasts like method combinators and don't allow to interleave low-level coded behaviour in the class mro. > > > At worst you could say that you allow subtyping but not code sharing, > > but then even a C method for an object should be very careful using > > another overridable method of the very same object. > > I definitely have to support code sharing. I think subclassing at the > C level is useful to create variations of built-in objects with > additional properties, e.g. numbers with additional operations or > formatting parameters, dictionaries with certain restrictions on the > keys, etc. I don't think such subclasses should be used in complex > multiple inheritance lattices. Better to enforce that? > > > OK you could say: that's C level, everybody should care for herself, > > but with Java we cannot do the same. > > Why not? Because users would like writing Jython types to become easier, and be possible using Java OOP model, without surprises, or undocumented subtlities, any deviation is an hard sell because Java has already an OOP model. >> > > > It would make sense to use Java subclassing to implement > > > > type subclassing, at least at layout level this does not clash with > > > > the multiple inheritance rule in best_base. > > > > > > So you would rule out multiple inheritance of types? Fine with me, > > > but you may regret this a few yearsfrom now (when we all code multiple > > > inheritance all the time :-). > > > No it is ruled out at Java level, not Jython level. > > But it is not also ruled out in CPython at C level? > > Mostly because I don't provide a way to create multiple base classes. > That would not be hard to add though -- if there's demand. But this is a path that Jython could not follow. > > see above. > > > > > I propose that you just try to live with this. > > > Related e.g. to Zope porting projects there have been a lot of > > pressure on jython-dev regarding how to code new types and > > metaclasses, A Jython type writer expect to be able to use Java > > naturally (i.e. use inheritance), for that, so the situation isn't > > that easy, is not even easy with the current codebase, because given > > the new mro one can even put some Python behaviour between: > > > > PyList PythonLevelCode and PyObject :-( (*) > > Just sa no. :-) See above. > > > > A safe rule would be to require that all Python classes in a > > > given inheritance graph should derive from the same C/Java class. I > > > That would mean that all C/Java behaviour is always only at the top > > of the mro, right? > > If you mean at the end, yes. Of course. > > > I had the same idea because this would make my life easier. > > Sounds good to me. > > > A problem: some of the Zope porters reported that there are classes > > that inherits from more than a single ExtensionClass in Zope :-( > > don't know if that is true. (**) > > Me neither. But the problem is probably shallow -- Zope mostly uses > mixins of various sorts, so these classes probably don't override > stuff -- they just define new methods. (I haven't looked at what it > would take to replace ExtensionClass with the new metaclasses, but > that's definitely on the agenda.) > > > > don't know if we should enforce this or just warn about it in the > > > documentation. > > > It would make our (jython-dev) life easier and avoid some impredictable > > problem even with CPython and code sharing, > > but see previous note (**) > > So enforce it and see how it turns out in practice. It's probably the best solution, at least for Jython, and could even make sense for CPython (see above points) > > > I don't want to force you to call a cooperative super > > > method at the Java level -- it would be very slow, I suspect... > > > Yes ... very slow and we would have to use that also at the very core > > of the hierarchy, see (*) > > but the problem is more complicated, in Java given three classes > > > > C extends B extends A > > and a method m > > C.m overrides B.m overrides A.m > > > > there is not direct way (even using reflection) > > to apply B.m behaviour to a C object, unless > > the programmer has left some hook in C using super > > or in B ( a method B_m e.g.). > > I guess C.m has to call super.m to invoke B.m, right? Yup. > > Does this mean that the following can't be made to work? > > class C(list): > def add_spam(self): > return list.append(self, "spam") > > It's legal Python (although in general questionable style of course). Yes, it can be made to work (it already works for Java subclassing and type subclassing would be some flavor of that), but the point is that your Java stuff should be at the top of the mro. Because then you can insert some hooks using super. regards. From guido@python.org Fri Aug 24 18:09:37 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 24 Aug 2001 13:09:37 -0400 Subject: [Python-Dev] super() In-Reply-To: Your message of "Fri, 24 Aug 2001 18:55:38 +0200." <021001c12cbd$9a182800$e000a8c0@thomasnotebook> References: <021001c12cbd$9a182800$e000a8c0@thomasnotebook> Message-ID: <200108241709.NAA13060@cj20424-a.reston1.va.home.com> Thomas Heller wrote (in private email, responding to a checkin message): > From: "Guido van Rossum" > > Add 'super', another new object type with magical properties. > > > > super(type) -> unbound super object > > super(type, obj) -> bound super object; requires isinstance(obj, type) > > > > Typical use to call a cooperative superclass method: > > > > class C(B): > > def meth(self, arg): > > super(C, self).meth(arg); > > Shouldn't that be > class C(B): > def meth(self, arg): > super(B, self).meth(arg) > ^ > to call B.meth(self, arg)? > > Thomas No. Good question, though! You have to pass your *own* class so that this finds the right super-method when multiple inheritance involving a "diamond" diagram. You may want to read the section on Method resolution order in PEP 253 first. Look at this example: class A(object): def meth(self, a): return "A.meth(%r)" % a print A().meth(1) class B(A): def __init__(self): self.__super = super(B, self) def meth(self, a): return "B.meth(%r) -> " % a + self.__super.meth(a) print B().meth(2) class C(A): __dynamic__ = 1 def meth(self, a): return "C.meth(%r) -> " % a + self.__super.meth(a) C._C__super = super(C) print C().meth(3) class D(C, B): def meth(self, a): return "D.meth(%r) -> " % a + super(D, self).meth(a) print D().meth(4) # D.meth(4) -> C.meth(4) -> B.meth(4) -> A.meth(4) D has the following inheritance graph: A ^ ^ / \ / \ / \ / \ C B ^ ^ \ / \ / \ / \ / D When you have a C or B instance, C.meth's super references A. But when you have a D instance, C.meth's super references B! In other words, D.meth calls C.meth calls B.meth calls A.meth. This is why you need to pass self to super() -- it needs the __mro__ sequence of the instance. Remember that D.__mro__ == (D,C,B,A,object). C.meth calls super(C, D()).meth. This looks for C in D.__mro__, and then starts searching for meth at the next class -- finding B.meth. If you wonder *why* the MRO is designed this way, imagine that meth() does some sort of saving to disk. Each subclass saves its own stuff and then calls the save method of its base class. Without the behavior described above, D.save() would have to call C.save() and B.save(), but this would call A.save() twice! This is part of a pattern called cooperative class design, described in the book Putting Metaclasses to Work (for a reference, see PEP 253). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Aug 24 18:36:59 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 24 Aug 2001 13:36:59 -0400 Subject: [Python-Dev] Comments2: PEP 252, PEP 253 and jython issues In-Reply-To: Your message of "Fri, 24 Aug 2001 17:50:05 +0200." <200108241550.RAA21561@core.inf.ethz.ch> References: <200108241550.RAA21561@core.inf.ethz.ch> Message-ID: <200108241736.NAA14289@cj20424-a.reston1.va.home.com> [SP] > > > The problem is that with the new mro possibly independently C > > > defined things get interleaved with Python level code, so I > > > imagine that the effect should be a bit disciplined ... > > > [GvR] > > > > I don't expect this to happen regularly. if you count on this I > > think you're on really thin ice. [SP] > No I'm not counting on that, I'm worried about this scenario > > A subclasses object and redefines __getattr__ > B subclasses list > > then class C(B,A): ... > > will have the following mro: > > C B list A object > > In Jython codebase hypothetically but possibly list could contains > calls like: > > __getattr__(...) > and super.__getattr__ This is Java code, right? > now A is in between list and object: what is the right thing here? Just say that this is what happens. The MRO is inconsistent in this case. > I don't expect or invite users to do that, but your rules allow A to > end up in that position in the mro, your philosophy seems to be > this is thin ice and so let it be ... Exactly. > > > I saw that list_ass_slice calls list_slice directly, is that OK? > > > > It's certainly safe -- any subclass of list will share the same > > structure lay-out (this is enforced by the subclassing machinery) so > > the list_slice call will find the object properly inititialized. > > > > There are lots of places where the C code doesn't bother to look > > for overridden methods -- the PyDict_* API is a common example. I > > have no intention of fixing this. > Fine with that, but ... > > > > Other OO languages like Smalltalk, Self, or Java allow only to > > > define primitives or single methods to add lowel-level > > > functionality, they don't try to allow subtyping at that level, > > > the previous scenario in Python where you could define just > > > opaque types, and in particular your C code never/normally got > > > interleaved with Python > > > behaviour was on that line. > > > > Well, if you like that better, maybe I should remove the > > instructions for subclassing at the C level? :-) > No, I'm just saying that you are moving from a world were you had > documentable C black-boxes and clear Python semantics, to a world > where (but you say that would be rare) you can have Python code > whose behaviour depends on very fine-grained decision inside C code. I would recommend that C programmers not indulge in this. They should continue to write C types that can be treated as black boxes. The C-level single inheritance is best seen as code sharing. > On one side your rules make it very easy to interleave Python and C > code in the mro, on the other hand I agree that will be rarely done > by the wary user because it's just surprise-land, don't know about > the unwary ... On the third hand, if a C programmer *wants* to write a class with the proper cooperative super calling, they can do so, it's just a bit clumsy. Should I provide a C API to make this easier? I doubt that it will be used much. If there's demand, it can be added later. > > My intention is to make the maximum number of different paradigms > > usable in Python. You can do cooperative multiple inheritance, > > but only of all your classes are designed with cooperation in > > mind. If some class uses C-level (or Java-level) inheritance and > > is not written cooperatively, you have to be a little careful > > using it as a base class in a cooperatively written class. I > > don't want to enforce cooperative coding at the Python level > > either: the "BaseClass.method(self, args)" approach is still > > valid, but you have to beware of the consequences. > Do you feel is that a good thing, especially if new-style classes become > the default? Time will tell. I expect that some people will design vast cooperative class hierarchies. I expect that other people will stick to single inheritance or very simple-minded uses of multiple inheritance (for example, they could use the mix-in pattern that was about the only sensible use of multiple inheritance with classic classes). Anyway, I expect new classes won't become the default until Python 3.0. > It is not a rethoric question, Python is becoming a little single > dispatch CLOS with a MOP, descriptors etc but CLOS has only > next-method and strange beasts like method combinators and don't > allow to interleave low-level coded behaviour in the class mro. Nothing new: classic classes already allow you to cheat or mess up in various ways by not calling your superclass method. > > > At worst you could say that you allow subtyping but not code > > > sharing, but then even a C method for an object should be very > > > careful using another overridable method of the very same > > > object. > > > > I definitely have to support code sharing. I think subclassing at > > the C level is useful to create variations of built-in objects > > with additional properties, e.g. numbers with additional > > operations or formatting parameters, dictionaries with certain > > restrictions on the keys, etc. I don't think such subclasses > > should be used in complex multiple inheritance lattices. > Better to enforce that? Why? I can think of situations where you have a MRO like this: C list B object but where B doesn't override anything that list might use. In fact, this is the typical mix-in situation. I don't want to forbid this. > > > OK you could say: that's C level, everybody should care for > > > herself, but with Java we cannot do the same. > > > > Why not? > Because users would like writing Jython types to become easier, Do you mean writing Jython types in Java, or in Python? > and be possible using Java OOP model, without surprises, or > undocumented subtlities, any deviation is an hard sell because Java > has already an OOP model. Recommend strongly that they stick to single inheritance for types implemented in Java -- that should do the trick. > > > > > It would make sense to use Java subclassing to implement > > > > > type subclassing, at least at layout level this does not > > > > > clash with the multiple inheritance rule in best_base. > > > > > > > > So you would rule out multiple inheritance of types? Fine > > > > with me, but you may regret this a few yearsfrom now (when we > > > > all code multiple inheritance all the time :-). > > > > > No it is ruled out at Java level, not Jython level. > > > But it is not also ruled out in CPython at C level? > > > > Mostly because I don't provide a way to create multiple base > > classes. That would not be hard to add though -- if there's > > demand. > But this is a path that Jython could not follow. Correct -- another reason why it's not a good idea to encourage this. What you could do though, even in Jython, is simulate everything that goes on in a class statement: build your own dict, populate it with function objects, and then call type.__new__(name, bases, dict). Voila, instant class. :-) > > > see above. > > > > > > > I propose that you just try to live with this. > > > > > Related e.g. to Zope porting projects there have been a lot of > > > pressure on jython-dev regarding how to code new types and > > > metaclasses, A Jython type writer expect to be able to use Java > > > naturally (i.e. use inheritance), for that, so the situation > > > isn't that easy, is not even easy with the current codebase, > > > because given the new mro one can even put some Python behaviour > > > between: > > > > > > PyList PythonLevelCode and PyObject :-( (*) > > > > Just sa no. :-) > See above. Sorry, my context stack just overflowed. If this is still an issue, please start over. :-) [snip] > > > I had the same idea because this would make my life easier. > > > > Sounds good to me. > > > > > A problem: some of the Zope porters reported that there are > > > classes that inherits from more than a single ExtensionClass in > > > Zope :-( don't know if that is true. (**) > > > > Me neither. But the problem is probably shallow -- Zope mostly > > uses mixins of various sorts, so these classes probably don't > > override stuff -- they just define new methods. (I haven't looked > > at what it would take to replace ExtensionClass with the new > > metaclasses, but that's definitely on the agenda.) > > > > > > don't know if we should enforce this or just warn about it in > > > > the documentation. > > > > > It would make our (jython-dev) life easier and avoid some > > > impredictable problem even with CPython and code sharing, but > > > see previous note (**) > > > > So enforce it and see how it turns out in practice. > It's probably the best solution, at least for Jython, > and could even make sense for CPython (see above points) Good. We seem to agree. > > > > I don't want to force you to call a cooperative super method > > > > at the Java level -- it would be very slow, I suspect... > > > > > Yes ... very slow and we would have to use that also at the very > > > core of the hierarchy, see (*) but the problem is more > > > complicated, in Java given three classes > > > > > > C extends B extends A > > > and a method m > > > C.m overrides B.m overrides A.m > > > > > > there is not direct way (even using reflection) > > > to apply B.m behaviour to a C object, unless > > > the programmer has left some hook in C using super > > > or in B ( a method B_m e.g.). > > > > I guess C.m has to call super.m to invoke B.m, right? > Yup. > > Does this mean that the following can't be made to work? > > > > class C(list): > > def add_spam(self): > > return list.append(self, "spam") > > > > It's legal Python (although in general questionable style of course). > Yes, it can be made to work (it already works for Java subclassing > and type subclassing would be some flavor of that), but the point is > that your Java stuff should be at the top of the mro. Because then > you can insert some hooks using super. > > regards. I hope this discussion so far has been helpful -- I have to pack for my trip and will soon be off to San Francisco until next Wednesday. --Guido van Rossum (home page: http://www.python.org/~guido/) From mwh@python.net Fri Aug 24 19:09:53 2001 From: mwh@python.net (Michael Hudson) Date: 24 Aug 2001 14:09:53 -0400 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: Paul Prescod's message of "Wed, 22 Aug 2001 10:56:33 -0800" References: <3B82AFFE.652A19F5@ActiveState.com> <200108220048.UAA24452@cj20424-a.reston1.va.home.com> <3B83DD5D.3260BC9C@ActiveState.com> <3B83EE5B.CEB38C48@ActiveState.com> <15235.61766.827878.699521@slothrop.digicool.com> <3B840061.A55BA194@ActiveState.com> Message-ID: <2m7kvtcosu.fsf@starship.python.net> Paul Prescod writes: > Neil Bauman asked him why Perl wasn't being written in Perl (Neil's > opinion is that Perl is literally the greatest language ever > created). Damian responded with a great quote: "Performance. If > Performance wasn't an issue, I'd probably write Perl 6 in Python or > something like it with wonderfully beautiful OO abstractions and it > would run like a dog." Tell them to use ocaml. Cheers, M. -- You have run into the classic Dmachine problem: your machine has become occupied by a malevolent spirit. Replacing hardware or software will not fix this - you need an exorcist. -- Tim Bradshaw, comp.lang.lisp From mwh@python.net Fri Aug 24 19:11:53 2001 From: mwh@python.net (Michael Hudson) Date: 24 Aug 2001 14:11:53 -0400 Subject: [Python-Dev] Re: copy, len and the like as 'object' methods? In-Reply-To: Paul Prescod's message of "Wed, 22 Aug 2001 18:08:40 -0700" References: <3B845798.7505DC91@ActiveState.com> Message-ID: <2m4rqxcopi.fsf@starship.python.net> Paul Prescod writes: > > Would you also argue for [1, 2, 3].repr() and "abc".hash()? > > Yes. But hashing and repring are done much less often than taking the > length. Note that this is an argument for keeping len(o) over o.len() - at least you bump into it early. Reminds me of what Laura Creighton said about the results of her case-sensitivity survey. Cheers, M. -- First of all, email me your AOL password as a security measure. You may find that won't be able to connect to the 'net for a while. This is normal. The next thing to do is turn your computer upside down and shake it to reboot it. -- Darren Tucker, asr From mwh@python.net Fri Aug 24 19:17:58 2001 From: mwh@python.net (Michael Hudson) Date: 24 Aug 2001 14:17:58 -0400 Subject: [Python-Dev] PEP 264 In-Reply-To: "Tim Peters"'s message of "Wed, 15 Aug 2001 17:24:20 -0400" References: Message-ID: <2m1ym1cofd.fsf@starship.python.net> "Tim Peters" writes: > [Michael Hudson] > > I *think* I'm done coding for implementing PEP 264; patch at: > > > > http://sourceforge.net/tracker/?func=detail&atid=305470&aid=449043 > > &group_id=5470 > > > > I've also done a first draft of docs. > > > > The reason for mentioning this is I'm off on holiday tomorrow, so if > > you want changes before the proposed date for 2.2a2, you've got aobut > > 12 hours to tell me about it. > > Thanks, Michael! I was out sick yesterday, but intend to get on this > tonight. You should enjoy your vacation! If it's a disaster, I'll probably > villify you on python-dev, but will fix it anyway . Thank you in turn for fixing all my cock-ups! I'd just like to note in passing that I considered most of the places the CO_* constants ended up in the last week, and rejected them as too gross. My idea was that __future__.generators.compiler_flag was to be the *only* (Python) name for CO_GENERATORS_ALLOWED (for example). Are there problems with this (other than the fact that you have to make sure two sets of magic numbers match up - but you have to edit __future__.py when you add a new future feature anyway...). I should probably add some words to the PEP on this topic, anyway. Cheers, M. -- ... when all the programmes on all the channels actually were made by actors with cleft pallettes speaking lines by dyslexic writers filmed by blind cameramen instead of merely seeming like that, it somehow made the whole thing more worthwhile. -- HHGTG, Episode 11 From mwh@python.net Fri Aug 24 19:23:23 2001 From: mwh@python.net (Michael Hudson) Date: 24 Aug 2001 14:23:23 -0400 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Python bltinmodule.c,2.227,2.228 In-Reply-To: Tim Peters's message of "Fri, 17 Aug 2001 15:11:30 -0700" References: Message-ID: <2my9o9b9lw.fsf@starship.python.net> Tim Peters writes: > ! /* XXX Warn if (supplied_flags & PyCF_MASK_OBSOLETE) != 0? */ I think not. If, as I *think*, all code objects compiled in 2.2 have ob.co_flags & __future__.nested_scopes.compiler_flag non-zero, then, eg., idle will compile all code typed interactively (after the first line, anyway) with __future__.nested_scopes.compiler_flag in the supplied_flags argument to compile(). This could be wormed around, obviously, but I don't see the point. Cheers, M. From Samuele Pedroni Fri Aug 24 19:28:50 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Fri, 24 Aug 2001 20:28:50 +0200 (MET DST) Subject: [Python-Dev] nightmare: PEP 252, PEP 253 and jython issues Message-ID: <200108241829.UAA26901@core.inf.ethz.ch> [GvR] > > > now A is in between list and object: what is the right thing here? > > Just say that this is what happens. The MRO is inconsistent in this > case. > > > I don't expect or invite users to do that, but your rules allow A to > > end up in that position in the mro, your philosophy seems to be > > this is thin ice and so let it be ... > > Exactly. I wish I would not feel uneasy with that > > On the third hand, if a C programmer *wants* to write a class with the > proper cooperative super calling, they can do so, it's just a bit > clumsy. Should I provide a C API to make this easier? I doubt that > it will be used much. If there's demand, it can be added later. I hope there will not be. GvR> I don't think such subclasses GvR> should be used in complex multiple inheritance lattices. > > > Better to enforce that? > > Why? I can think of situations where you have a MRO like this: > > C list B object > > but where B doesn't override anything that list might use. In fact, > this is the typical mix-in situation. I don't want to forbid this. OK, I see. (*) GVR> So enforce it and see how it turns out in practice. > > > It's probably the best solution, at least for Jython, > > and could even make sense for CPython (see above points) > > Good. We seem to agree. Yup but I see your point (*) so probably I can't. [me] *bad-guy-java* > > > > Yes ... very slow and we would have to use that also at the very > > > > core of the hierarchy, see (*) but the problem is more > > > > complicated, in Java given three classes > > > > > > > > C extends B extends A > > > > and a method m > > > > C.m overrides B.m overrides A.m > > > > > > > > there is not direct way (even using reflection) > > > > to apply B.m behaviour to a C object, unless > > > > the programmer has left some hook in C using super > > > > or in B ( a method B_m e.g.). > > I hope this discussion so far has been helpful -- I have to pack for > my trip and will soon be off to San Francisco until next Wednesday. > Yes it was, thanks but I overlooked a central point, namely *bad-guy-java* I fear, because of it, we have to code all the built-in types this way, class SpamType { ... SpamTypes__foo__ { ... } ... __foo__ { } } Or find other slower ways, otherwise no descrs for them A nightmare :-( regards. From guido@python.org Fri Aug 24 19:44:05 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 24 Aug 2001 14:44:05 -0400 Subject: [Python-Dev] nightmare: PEP 252, PEP 253 and jython issues In-Reply-To: Your message of "Fri, 24 Aug 2001 20:28:50 +0200." <200108241829.UAA26901@core.inf.ethz.ch> References: <200108241829.UAA26901@core.inf.ethz.ch> Message-ID: <200108241844.OAA14685@cj20424-a.reston1.va.home.com> [snip] [SP] > *bad-guy-java* > > > > > Yes ... very slow and we would have to use that also at the very > > > > > core of the hierarchy, see (*) but the problem is more > > > > > complicated, in Java given three classes > > > > > > > > > > C extends B extends A > > > > > and a method m > > > > > C.m overrides B.m overrides A.m > > > > > > > > > > there is not direct way (even using reflection) > > > > > to apply B.m behaviour to a C object, unless > > > > > the programmer has left some hook in C using super > > > > > or in B ( a method B_m e.g.). > [snip] > I overlooked a central point, namely *bad-guy-java* > > I fear, because of it, we have to code all the built-in types this way, > > > class SpamType { > > ... SpamTypes__foo__ { ... > } > > ... __foo__ { } > > } > > Or find other slower ways, otherwise no descrs for them As long as the slowness only applies when someone invokes SpamType.__foo__(x), that's acceptable. --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas.heller@ion-tof.com Fri Aug 24 19:57:04 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 24 Aug 2001 20:57:04 +0200 Subject: [Python-Dev] Customizing the binding of attributes References: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook> <200108231923.f7NJNg307631@odiug.digicool.com> <09f401c12c6a$7717ab80$e000a8c0@thomasnotebook> <200108241003.GAA04383@cj20424-a.reston1.va.home.com> Message-ID: <03cf01c12cce$90822d20$e000a8c0@thomasnotebook> [I wrote] > > - Shouldn't (my)instancemethod be a builtin? Similar to > > staticmethod and classmethod? > > Yes. I just checked this in. The new name is 'getset' unless you > have a better idea. I wasn't talking about your getset object (which is already mentioned in the PEP), I was thinking of an 'instancemethod' object, which would accept any callable (for example a *function* implemented in an extension module), and convert it into an unbound or bound method when retrieved from a class or an instance. Something like class instancemethod(object): def __init__(self, callable): self.callable = callable def __get__(self, inst, klass=None): import new return new.instancemethod(self.callable, inst, klass) only faster ;-) Thomas From guido@python.org Fri Aug 24 20:49:42 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 24 Aug 2001 15:49:42 -0400 Subject: [Python-Dev] Customizing the binding of attributes In-Reply-To: Your message of "Fri, 24 Aug 2001 20:57:04 +0200." <03cf01c12cce$90822d20$e000a8c0@thomasnotebook> References: <064201c12c04$37dfa6e0$e000a8c0@thomasnotebook> <200108231923.f7NJNg307631@odiug.digicool.com> <09f401c12c6a$7717ab80$e000a8c0@thomasnotebook> <200108241003.GAA04383@cj20424-a.reston1.va.home.com> <03cf01c12cce$90822d20$e000a8c0@thomasnotebook> Message-ID: <200108241949.PAA16386@cj20424-a.reston1.va.home.com> > I wasn't talking about your getset object (which is already mentioned > in the PEP), I was thinking of an 'instancemethod' object, > which would accept any callable (for example a *function* > implemented in an extension module), and convert it into an unbound > or bound method when retrieved from a class or an instance. > Something like > > class instancemethod(object): > def __init__(self, callable): > self.callable = callable > > def __get__(self, inst, klass=None): > import new > return new.instancemethod(self.callable, inst, klass) > > only faster ;-) I believe the (unbound) instance method type itself has this functionality, except it has no working constructor. I will fix that (not today :-). --Guido van Rossum (home page: http://www.python.org/~guido/) From Samuele Pedroni Fri Aug 24 20:54:07 2001 From: Samuele Pedroni (Samuele Pedroni) Date: Fri, 24 Aug 2001 21:54:07 +0200 (MET DST) Subject: [Python-Dev] nightmare: PEP 252, PEP 253 and jython issues Message-ID: <200108241954.VAA29770@core.inf.ethz.ch> [GvR] > > [SP] > > *bad-guy-java* > > > > > > Yes ... very slow and we would have to use that also at the very > > > > > > core of the hierarchy, see (*) but the problem is more > > > > > > complicated, in Java given three classes > > > > > > > > > > > > C extends B extends A > > > > > > and a method m > > > > > > C.m overrides B.m overrides A.m > > > > > > > > > > > > there is not direct way (even using reflection) > > > > > > to apply B.m behaviour to a C object, unless > > > > > > the programmer has left some hook in C using super > > > > > > or in B ( a method B_m e.g.). > > > [snip] > > I overlooked a central point, namely *bad-guy-java* > > > > I fear, because of it, we have to code all the built-in types this way, > > > > > > class SpamType { > > > > ... SpamTypes__foo__ { ... > > } > > > > ... __foo__ { } > > > > } > > > > Or find other slower ways, otherwise no descrs for them > > As long as the slowness only applies when someone invokes > SpamType.__foo__(x), that's acceptable. Not that simple, *bad-guy-java* is really nasty given your design for PEP 252 and PEP 253, you should put some hook to overridable behaviour or completely give up with Java OOP ... but it's our problem, your design up to my mro concerns is clearly is sound. But Jython 2.2 will be a major challenge. Samuele. From tim.one@home.com Fri Aug 24 23:26:13 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 24 Aug 2001 18:26:13 -0400 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Python bltinmodule.c,2.227,2.228 In-Reply-To: <2my9o9b9lw.fsf@starship.python.net> Message-ID: > ! /* XXX Warn if (supplied_flags & PyCF_MASK_OBSOLETE) != 0? */ [Michael Hudson] > I think not. > > If, as I *think*, all code objects compiled in 2.2 have > > ob.co_flags & __future__.nested_scopes.compiler_flag > > non-zero, then, eg., idle will compile all code typed interactively > (after the first line, anyway) with > __future__.nested_scopes.compiler_flag in the supplied_flags argument > to compile(). This could be wormed around, obviously, but I don't see > the point. Your belief was true of 2.2a1 but not of 2.2a2. For 2.2a1, Guido simply turned on nested_scopes by default (and so every code object had CO_NESTED set). For 2.2a2, Jeremy returned from paternity leave and took away the *possibility* of turning nested scopes off. As a result, the *only* reference to CO_NESTED now in the C part of Python is in the #define of PyCF_MASK_OBSOLETE, and Python itself never sets CO_NESTED. That's part of what I meant when I said (in a different recent thread) that any piece of code using these flags is eventually going to get surprised: these implementation details are as internal and transient as they get. This isn't critical now, but I can't help but notice that we've got 8 bit positions assigned in the co_flags int. One is meaningless in 2.2a2 (CO_NESTED), and two more will eventually become meaningless (CO_GENERATOR_ALLOWED and CO_FUTURE_DIVISION). I'm not worried about running out of "real" co_flags bits, but so long as we're stuck *also* using co_flags to communicate-- from caller to dynamically-compiled callee --which still-optional future-features are in effect, we burn another bit position every time we dream up a future-feature with semantic content. So, but later rather than sooner, I expect we're going to have to recycle these temporary future-feature co_flags bits. When that happens, it may be prudent to *warn* about obsolete flag values for a release cycle (before recycling them), for the benefit of insane people who can't restrain themselves from playing with undocumented bits . From tim.one@home.com Fri Aug 24 23:41:05 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 24 Aug 2001 18:41:05 -0400 Subject: [Python-Dev] PEP 264 In-Reply-To: <2m1ym1cofd.fsf@starship.python.net> Message-ID: [Michael Hudson] > Thank you in turn for fixing all my cock-ups! There were few -- it turned out to take much longer than "it should have" for reasons I don't want to bother recounting now, but rest assured there was plenty of blame to go around. Not nearly so much blame as glory, though . > I'd just like to note in passing that I considered most of the places > the CO_* constants ended up in the last week, and rejected them as too > gross. My idea was that > > __future__.generators.compiler_flag > > was to be the *only* (Python) name for CO_GENERATORS_ALLOWED (for > example). Are there problems with this (other than the fact that you > have to make sure two sets of magic numbers match up - but you have to > edit __future__.py when you add a new future feature anyway...). The problem is that nobody is going to *remember* the subtleties. IIRC, almost every bug I tracked down was due to *someone* neglecting to fiddle all the places that needed to be fiddled when changing future-flags. So The Rule I implemented was two-fold: 1. If possible, remove the need to remember at all (e.g., that was done for PyEval_MergeCompilerFlags, which hadn't been updated to know about CO_FUTURE_DIVISION (a bug), but was still uselessly merging CO_NESTED (a waste)). 2. When not possible, guarantee that a dumb-ass global editor search will find all "conceptual" uses of a CO_xxx flag name. #2 is why __future__.py contains the names "CO_NESTED" etc. Since it's not possible to hide the need entirely, it's imperative to make the dumbest approach imaginable likely to succeed for the next time, when the novelty has warn off and nobody quite remembers anymore exactly what needs to be done. pyassem.py (in the Compiler package) does the same thing. > I should probably add some words to the PEP on this topic, anyway. I don't think it's needed; at this point the bugs have all been shamed into good behavior by the relentless application of simple good taste . From skip@pobox.com (Skip Montanaro) Fri Aug 24 23:52:08 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 24 Aug 2001 17:52:08 -0500 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Python bltinmodule.c,2.227,2.228 In-Reply-To: References: <2my9o9b9lw.fsf@starship.python.net> Message-ID: <15238.55960.530563.607085@cb40507-c.elmhst1.il.home.com> Tim> This isn't critical now, but I can't help but notice that we've got Tim> 8 bit positions assigned in the co_flags int.... Tim> So, but later rather than sooner, I expect we're going to have to Tim> recycle these temporary future-feature co_flags bits. I asked about reserving three bits to specify the virtual machine the codestring is meant to be executed on. ;-) Skip From nas@python.ca Sat Aug 25 02:05:58 2001 From: nas@python.ca (Neil Schemenauer) Date: Fri, 24 Aug 2001 18:05:58 -0700 Subject: [Python-Dev] shutting up tmpnam, tempnam, link warnings Message-ID: <20010824180558.A14251@glacier.fnational.com> If you're using a recent version of GNU libc you know what I mean. They are annoying as hell and there seems to be no way to disable them cleanly. Here's a hack: cd /lib perl -pe 's/gnu\.warning\.(tempnam|tmpnam|tmpnam_r)/gnu.wanting.\1/g' \ < libc-2.2.3.so > /tmp/libc-2.2.3.so cp -b /tmp/libc-2.2.3.so libc-2.2.3.so No warranty implied. :-) Neil From esr@thyrsus.com Sat Aug 25 00:51:56 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Fri, 24 Aug 2001 19:51:56 -0400 Subject: [Python-Dev] Negative index in insert() Message-ID: <20010824195156.A1787@thyrsus.com> Given >>> a = [1, 2, 3, 4] because >>> a[-2] -3 I expected that a.insert(-2, 0) would yield [1, 2, 0, 3, 4]. It was a rude shock to discover that >>> a [0, 1, 2, 3, 4] In fact I think this may be the nastiest surprise Python has handed me since I started using it. The reference manual says "same as s[i:i] = [x] if i >= 0" which of course doesn't cover the i < 0 case. David Beasley's reference says "Inserts x at index i" which sounds like the behavior I was expecting but didn't get. Is this a deliberate design choice, an oversight, or a plain bug? If it's a choice, it's damn poorly documented -- this deserves at least a footnote in the list methods table. If it's an oversight or bug, I volunteer to fix it. -- Eric S. Raymond "America is at that awkward stage. It's too late to work within the system, but too early to shoot the bastards." -- Claire Wolfe From Greg.Wilson@baltimore.com Sat Aug 25 16:05:00 2001 From: Greg.Wilson@baltimore.com (Greg Wilson) Date: Sat, 25 Aug 2001 11:05:00 -0400 Subject: [Python-Dev] %b formatting Message-ID: <930BBCA4CEBBD411BE6500508BB3328F3A8F0F@nsamcanms1.ca.baltimore.com> Patch 455076 adds "%b" (binary) formatting for integers and long integers to both strings and unicodes. As this is my first time hacking on the Python source, I'd be grateful for comments. Thanks, Greg p.s. apologies for the length of the signature --- it's another joy of corporate life :-( ----------------------------------------------------------------------------------------------------------------- The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error or there are any problems please notify the originator immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Baltimore Technologies plc will not be liable for direct, special, indirect or consequential damages arising from alteration of the contents of this message by a third party or as a result of any virus being passed on. In addition, certain Marketing collateral may be added from time to time to promote Baltimore Technologies products, services, Global e-Security or appearance at trade shows and conferences. This footnote confirms that this email message has been swept by Baltimore MIMEsweeper for Content Security threats, including computer viruses. From mwh@python.net Sat Aug 25 17:42:12 2001 From: mwh@python.net (Michael Hudson) Date: 25 Aug 2001 12:42:12 -0400 Subject: [Python-Dev] Re: copy, len and the like as 'object' methods? In-Reply-To: Michael Hudson's message of "24 Aug 2001 14:11:53 -0400" References: <3B845798.7505DC91@ActiveState.com> <2m4rqxcopi.fsf@starship.python.net> Message-ID: <2mr8u03xcr.fsf@starship.python.net> Michael Hudson writes: > Reminds me of what Laura Creighton said about the results of her > case-sensitivity survey. As Aahz has reminded me, it was Sheila King. -- languages shape the way we think, or don't. -- Erik Naggum, comp.lang.lisp From tim.one@home.com Sun Aug 26 05:08:18 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 26 Aug 2001 00:08:18 -0400 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Python bltinmodule.c,2.227,2.228 In-Reply-To: <15238.55960.530563.607085@cb40507-c.elmhst1.il.home.com> Message-ID: [Skip Montanaro] > I asked about reserving three bits to specify the virtual machine the > codestring is meant to be executed on. ;-) Not a problem -- CO_GENERATOR is the only "real" new co_flag added in years, in 10 years we've only defined 5 real co_flags total, and there are no PEPs on the table that would need another one. So you may as well ask for 27 VM bits . From tim.one@home.com Sun Aug 26 07:14:15 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 26 Aug 2001 02:14:15 -0400 Subject: [Python-Dev] Negative index in insert() In-Reply-To: <20010824195156.A1787@thyrsus.com> Message-ID: [Eric S. Raymond] > Given > > >>> a = [1, 2, 3, 4] > > because > > >>> a[-2] > -3 > > I expected that a.insert(-2, 0) would yield [1, 2, 0, 3, 4]. It was a > rude shock to discover that > > >>> a > [0, 1, 2, 3, 4] > > In fact I think this may be the nastiest surprise Python has > handed me since I started using it. I never noticed it, but agree it's strange. Did you open a feature request on SourceForge? Note that array objects (created by the array module) share this behavior. > The reference manual says "same as s[i:i] = [x] if i >= 0" which > of course doesn't cover the i < 0 case. Which formally means its behavior is undefined. > David Beasley's reference says "Inserts x at index i" which sounds like > the behavior I was expecting but didn't get. Ya, so file a bug report with David . > Is this a deliberate design choice, an oversight, or a plain bug? If > it's a choice, it's damn poorly documented -- this deserves at least a > footnote in the list methods table. If it's an oversight or bug, I > volunteer to fix it. Only Guido *may* be able to say for sure. Plausible: the behavior for i<0 was deliberate at the start, and when Python *grew* negative indices (it didn't always accept them), fear of incompatibility prevented changing .insert behavior (in the other cases where negative indices "grew a meaning", the old behavior was to raise an exception; but .insert accepted them silently). Digging thru CVS, the current behavior for i<0 has always been that way (since listobject.c rev 1.1); and the "if i >= 0" qualifier was added to libtypes.tex a bit over 6 years ago, with checkin comment correct description of list.insert() Hard call! I agree it should change, but if it breaks code the "but it was formally undefined" comeback doesn't really warm peoples' hearts. first-one-to-suggest-a-future-stmt-dies-ly y'rs - tim From gstein@lyra.org Sun Aug 26 07:17:35 2001 From: gstein@lyra.org (Greg Stein) Date: Sat, 25 Aug 2001 23:17:35 -0700 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects stringobject.c,2.124,2.125 In-Reply-To: ; from bwarsaw@users.sourceforge.net on Fri, Aug 24, 2001 at 11:32:08AM -0700 References: Message-ID: <20010825231735.B2111@lyra.org> On Fri, Aug 24, 2001 at 11:32:08AM -0700, Barry Warsaw wrote: >... > + PyObject * > + PyString_FromFormat(const char *format, ...) > + { > + va_list vargs; > + > + #ifdef HAVE_STDARG_PROTOTYPES > + va_start(vargs, format); > + #else > + va_start(vargs); > + #endif > + return PyString_FromFormatV(format, vargs); > + } For safety/propriety, you should call va_end() on the vargs variable. Cheers, -g -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Sun Aug 26 07:21:47 2001 From: gstein@lyra.org (Greg Stein) Date: Sat, 25 Aug 2001 23:21:47 -0700 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects stringobject.c,2.124,2.125 In-Reply-To: ; from bwarsaw@users.sourceforge.net on Fri, Aug 24, 2001 at 11:32:08AM -0700 References: Message-ID: <20010825232147.C2111@lyra.org> On Fri, Aug 24, 2001 at 11:32:08AM -0700, Barry Warsaw wrote: > Update of /cvsroot/python/python/dist/src/Objects > In directory usw-pr-cvs1:/tmp/cvs-serv13719/Objects > > Modified Files: > stringobject.c >... > + end: > + _PyString_Resize(&string, s - PyString_AsString(string)); You could use: s - PyString_AS_STRING(string) Cheers, -g -- Greg Stein, http://www.lyra.org/ From esr@thyrsus.com Sun Aug 26 07:30:40 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Sun, 26 Aug 2001 02:30:40 -0400 Subject: [Python-Dev] Negative index in insert() In-Reply-To: ; from tim.one@home.com on Sun, Aug 26, 2001 at 02:14:15AM -0400 References: <20010824195156.A1787@thyrsus.com> Message-ID: <20010826023040.A32589@thyrsus.com> Tim Peters : > I never noticed it, but agree it's strange. Did you open a feature request > on SourceForge? Feature request? Heck, if Guido green-lights this I'll fix it myself. I haven't looked at the code yet but I'll be deeply surprised if it takes me longer than half an hour to nail this bug. > Hard call! I agree it should change, but if it breaks code the "but it was > formally undefined" comeback doesn't really warm peoples' hearts. If we're going to "fix" division, we sure ought to fix this. I've grown so used to Python being a smooth, consistent tool that fits comfortably in my hand that I found this really quite shocking. -- Eric S. Raymond "Both oligarch and tyrant mistrust the people, and therefore deprive them of arms." --Aristotle From esr@snark.thyrsus.com Sun Aug 26 10:11:46 2001 From: esr@snark.thyrsus.com (Eric S. Raymond) Date: Sun, 26 Aug 2001 05:11:46 -0400 Subject: [Python-Dev] Generalizing "in" to pairs of sequences Message-ID: <200108260911.f7Q9BkD30038@snark.thyrsus.com> This seems to be my week to ask simple, stupid questions. Is there any good semantic or philosophical reason that these aren't legal? >>> "ab" in "cabcd" 1 >>> "xy" in "cabcd" 0 >>> (1, 2) in (0, 1, 2, 3) 1 >>> (9, 8) in (0, 1, 2, 3) 0 Proposed -- the following definition of "in" as a binary operation on sequence types: a is "in" b if a occurs as a subsequence of b. I'd like to see Python support this mainly to get rid of the following common but ugly idiom: if b.find(a) > -1: # Do one thing else: # Do another It's really easy to forget and miscode the guard "if b.find(a):". By contrast, see how much clearer this is: if a in b: # Do one thing else: # Do another Thoughts? -- Eric S. Raymond I don't like the idea that the police department seems bent on keeping a pool of unarmed victims available for the predations of the criminal class. -- David Mohler, 1989, on being denied a carry permit in NYC From ping@lfw.org Sun Aug 26 10:25:31 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Sun, 26 Aug 2001 02:25:31 -0700 (PDT) Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences In-Reply-To: <200108260911.f7Q9BkD30038@snark.thyrsus.com> Message-ID: On Sun, 26 Aug 2001, Eric S. Raymond wrote: > This seems to be my week to ask simple, stupid questions. > > Is there any good semantic or philosophical reason that these aren't legal? Yes. > >>> "ab" in "cabcd" > 1 > >>> "xy" in "cabcd" > 0 > >>> (1, 2) in (0, 1, 2, 3) > 1 > >>> (9, 8) in (0, 1, 2, 3) > 0 The current meaning of "in" is: Given a sequence b, a is "in" b if a is an element of b. Your "subsequence" interpretation would conflict with this meaning. Here's current behaviour: >>> "ab" in ("ab", "cd") 1 >>> "ab" in ("a", "b", "c", "d") 0 >>> (1, 2) in ((0, 1), (1, 2), (2, 3)) 1 >>> (1, 2) in (0, 1, 2, 3) 0 "in" cannot have both meanings. -- ?!ng "Computers are useless. They can only give you answers." -- Pablo Picasso From ping@lfw.org Sun Aug 26 10:39:55 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Sun, 26 Aug 2001 02:39:55 -0700 (PDT) Subject: [Python-Dev] Re: Negative index in insert() In-Reply-To: Message-ID: Eric S. Raymond wrote: > >>> a = [1, 2, 3, 4] [...] > I expected that a.insert(-2, 0) would yield [1, 2, 0, 3, 4]. It was a > rude shock to discover that > > >>> a > [0, 1, 2, 3, 4] If this is fixed, i think we should do a complete job of it and make sure exceptions are raised when the index is out of range. Notice (in current Python): >>> a = [1, 2, 3, 4] >>> a.insert(999, 0) >>> a [1, 2, 3, 4, 0] >>> a.insert(-999, 0) >>> a [0, 1, 2, 3, 4, 0] So what's happening is that Python is silently clipping the index to [0..n]. To bring this in line with indexing behaviour, it should not only interpret values in the range [-n..n], but also raise IndexError when the value is outside of [-n..n]. (Compare to pop() -- the behaviour i'm suggesting is exactly what pop() does, except that the range for pop() is [-n..n-1].) This has the potential for breaking code, and so the transition should probably be done with a minor-version's worth of warning, but i think if we're going to change it we should make it right. When this is done, i believe we'll be down to exactly two ways of handling sequence indices -- (a) x and y are silently clipped to [-n..n-1] *only* in an [x:y]-style slice (b) in all other cases, an IndexError is raised if the index is out of range and these two cases are easy to separate and explain, and all will be consistent in its interpretation of negative numbers. Such simplicity would be nice to have. -- ?!ng "Computers are useless. They can only give you answers." -- Pablo Picasso From ping@lfw.org Sun Aug 26 11:43:06 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Sun, 26 Aug 2001 03:43:06 -0700 (PDT) Subject: [Python-Dev] Re: Negative index in insert() Message-ID: On Sun, 26 Aug 2001, Ka-Ping Yee wrote: > (a) x and y are silently clipped to [-n..n-1] > *only* in an [x:y]-style slice Sorry -- of course i meant [-n..n] here. -- ?!ng From esr@thyrsus.com Sun Aug 26 11:54:37 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Sun, 26 Aug 2001 06:54:37 -0400 Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences In-Reply-To: ; from ping@lfw.org on Sun, Aug 26, 2001 at 02:25:31AM -0700 References: <200108260911.f7Q9BkD30038@snark.thyrsus.com> Message-ID: <20010826065437.A8269@thyrsus.com> Ka-Ping Yee : > Your "subsequence" interpretation would conflict with this meaning. > Here's current behaviour: > > >>> "ab" in ("ab", "cd") > 1 > >>> "ab" in ("a", "b", "c", "d") > 0 > >>> (1, 2) in ((0, 1), (1, 2), (2, 3)) > 1 > >>> (1, 2) in (0, 1, 2, 3) > 0 > > "in" cannot have both meanings. You're right. But since a string can't be an element of a string, the case I'm really interested in would still work. -- Eric S. Raymond "Both oligarch and tyrant mistrust the people, and therefore deprive them of arms." --Aristotle From ping@lfw.org Sun Aug 26 12:01:01 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Sun, 26 Aug 2001 04:01:01 -0700 (PDT) Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences In-Reply-To: <20010826065437.A8269@thyrsus.com> Message-ID: On Sun, 26 Aug 2001, Eric S. Raymond wrote: > You're right. But since a string can't be an element of a string, the > case I'm really interested in would still work. It's arguable. A character can indeed be an element of a string, and in Python characters are one-character strings. So this violates x in s <=> there exists i such that s[i] == x You could argue that strings should have special behaviour for "in" to support the common case of finding a substring; then you would have to argue on grounds of "practicality beats purity". I might support the argument on grounds of practicality beating purity, but i'd have to think carefully about violating the above definition. I can see that "substring in string" would be very convenient, but then if char in '0123456789': ... would suddenly have a very different meaning -- it would succeed for char = '12'. -- ?!ng From skip@pobox.com (Skip Montanaro) Sun Aug 26 17:36:52 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Sun, 26 Aug 2001 11:36:52 -0500 Subject: [Python-Dev] can anonymous bug reports/patches be squelched? Message-ID: <15241.9636.961594.597780@beluga.mojam.com> I noticed a bsddbmodule report today and checked it out. An anonymous submitter had reported a bug with bsddbmodule.c. Martin responded with "try --without-libdb" and asked the person to report whether that helped. Two weeks later, after no response from the submitter (how would he have seen it?), Jeremy closed the bug. That was the right thing to do under the circumstances, but I would if there's some way to not allow anonymous bug reports and patches. It seems almost impossible to actually get a response back to the submitter in such cases, so I don't see how you can have a useful dialog about the report/patch. Skip From fdrake@acm.org Sun Aug 26 17:43:23 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Sun, 26 Aug 2001 12:43:23 -0400 Subject: [Python-Dev] can anonymous bug reports/patches be squelched? In-Reply-To: <15241.9636.961594.597780@beluga.mojam.com> References: <15241.9636.961594.597780@beluga.mojam.com> Message-ID: <15241.10027.588676.942551@grendel.digicool.com> Skip Montanaro writes: > That was the right thing to do under the circumstances, but I would if > there's some way to not allow anonymous bug reports and patches. It seems > almost impossible to actually get a response back to the submitter in such > cases, so I don't see how you can have a useful dialog about the > report/patch. Anonymous means the submitter was not logged in, but appearantly they have an option to leave an email address (which is not shown, which is fine). I don't think there's a requirement for an email address, though, and I don't see any way for us to determine if they left one. If there were a way to *require* an email address, or at least determine if there was one, I'd be satisfied. Not allowing anonymous reports at all would not be very good; many of them have proved useful (it doesn't matter who reports a typo in the docs, just that it gets assigned to me). -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From mwh@python.net Sun Aug 26 19:41:39 2001 From: mwh@python.net (Michael Hudson) Date: 26 Aug 2001 14:41:39 -0400 Subject: [Python-Dev] PEP 264 In-Reply-To: "Tim Peters"'s message of "Fri, 24 Aug 2001 18:41:05 -0400" References: Message-ID: <2m4rqu4qak.fsf@starship.python.net> "Tim Peters" writes: > [Michael Hudson] > > I'd just like to note in passing that I considered most of the places > > the CO_* constants ended up in the last week, and rejected them as too > > gross. My idea was that > > > > __future__.generators.compiler_flag > > > > was to be the *only* (Python) name for CO_GENERATORS_ALLOWED (for > > example). Are there problems with this (other than the fact that you > > have to make sure two sets of magic numbers match up - but you have to > > edit __future__.py when you add a new future feature anyway...). > > The problem is that nobody is going to *remember* the subtleties. IIRC, > almost every bug I tracked down was due to *someone* neglecting to fiddle > all the places that needed to be fiddled when changing future-flags. So The > Rule I implemented was two-fold: [schnipp] OK, I see. In this spirit, wouldn't it be better to define the PyCF_MASK and PyCF_MASK_OBSOLETE flags in the same place as the CO_* flags they're made up of? Cheers, M. -- Richard Gabriel was wrong: worse is not better, lying is better. Languages and systems succeed in the marketplace to the extent that their proponents lie about what they can do. -- Tim Bradshaw, comp.lang.lisp From greg@cosc.canterbury.ac.nz Mon Aug 27 00:20:06 2001 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 27 Aug 2001 11:20:06 +1200 (NZST) Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: <2m7kvtcosu.fsf@starship.python.net> Message-ID: <200108262320.LAA04767@s454.cosc.canterbury.ac.nz> Michael Hudson : > Paul Prescod writes: > > Damian responded with a great quote: "...If > > Performance wasn't an issue, I'd probably write Perl 6 in Python or > > something like it with wonderfully beautiful OO abstractions and it > > would run like a dog." At least it would eliminate all the "why is Python slower than Perl" ng questions. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From skip@pobox.com (Skip Montanaro) Mon Aug 27 03:01:32 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Sun, 26 Aug 2001 21:01:32 -0500 Subject: [Python-Dev] Where are locals initialized? Message-ID: <15241.43516.529415.828023@beluga.mojam.com> I'm probably just too tired to be looking at this right now, but I'm having no luck figuring out just where local variables are initialized. In 1.5.2 eval_code2 is called with all the bits and pieces needed to create a frame and does so. In 2.1 eval_frame is called with an already initialized frame. Either way I expected to see a call to PyFrame_LocalsToFast in or around PyFrame_New, but the places where I see it don't seem like the logical place (to my misfiring brain anyway). Any pointers appreciated, Skip From barry@zope.com Mon Aug 27 04:07:30 2001 From: barry@zope.com (Barry A. Warsaw) Date: Sun, 26 Aug 2001 23:07:30 -0400 Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences References: <20010826065437.A8269@thyrsus.com> Message-ID: <15241.47474.461384.774856@anthem.wooz.org> ESR> You're right. But since a string can't be an element of a ESR> string, the case I'm really interested in would still work. KY> It's arguable. A character can indeed be an element of a KY> string, and in Python characters are one-character strings. KY> So this violates KY> x in s <=> there exists i such that s[i] == x KY> You could argue that strings should have special behaviour for KY> "in" to support the common case of finding a substring; then KY> you would have to argue on grounds of "practicality beats KY> purity". And against TOOWTDI: >>> 'cabcd'.find('ab') >= 0 1 >>> 'cabcd'.find('xx') >= 0 0 -Barry From barry@zope.com Mon Aug 27 04:08:33 2001 From: barry@zope.com (Barry A. Warsaw) Date: Sun, 26 Aug 2001 23:08:33 -0400 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects stringobject.c,2.124,2.125 References: <20010825231735.B2111@lyra.org> Message-ID: <15241.47537.351411.119384@anthem.wooz.org> >>>>> "GS" == Greg Stein writes: GS> For safety/propriety, you should call va_end() on the vargs GS> variable. Good catch! I meant to add that after noticing the original code didn't contain it, and after double checking with the Linux manpage for va_start. I touched so much other code I forgot to go back and add that call. Will do so now... GS> You could use: s - PyString_AS_STRING(string) Good call. The original code had the former, but you're right that this is better. Thanks, -Barry From ask@valueclick.com Mon Aug 27 10:51:27 2001 From: ask@valueclick.com (Ask Bjoern Hansen) Date: Mon, 27 Aug 2001 02:51:27 -0700 (PDT) Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: <15236.3994.507401.287832@beluga.mojam.com> Message-ID: On Wed, 22 Aug 2001, Skip Montanaro wrote: [...] > libraries (and thus use by a critical mass) not in the core that much > higher. There are how many Python Server Pages like things? #$&$&%. Don't worry; even if we only look at those on CPAN we have more templating systems in Perl than you would ever want to count. > They probably all have some very nifty features, but tend to be > used by rather small pockets of people, at least relative to the > overall segment of the Python community doing webish stuff. > I'm not entirely sure why that's the case, but I suspect it has > to do in part with the barrier to discovering, downloading, and > installing these packages. Nah, some things just have to be implemented and reimplemented and reimplemented and reimplemented in slightly different ways. :-) - ask -- ask bjoern hansen, http://ask.netcetera.dk/ !try; do(); more than a billion impressions per week, http://valueclick.com From alex_c@MIT.EDU Mon Aug 27 12:41:26 2001 From: alex_c@MIT.EDU (Alex Coventry) Date: 27 Aug 2001 07:41:26 -0400 Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences In-Reply-To: barry@zope.com's message of "Sun, 26 Aug 2001 23:07:30 -0400" References: <15241.47474.461384.774856@anthem.wooz.org> Message-ID: Sometimes I find myself wishing that strings had a method "substringp" that would do basically what Eric wants, like >>> 'cabcd'.substringp('ca') 1 >>> 'cabcd'.substringp('xx') 0 substringp would probably be the wrong name for it, though. Alex. From barry@zope.com Mon Aug 27 14:51:03 2001 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 27 Aug 2001 09:51:03 -0400 Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences References: <15241.47474.461384.774856@anthem.wooz.org> Message-ID: <15242.20551.521357.800605@anthem.wooz.org> >>>>> "AC" == Alex Coventry writes: AC> Sometimes I find myself wishing that strings had a method AC> "substringp" that would do basically what Eric wants, like >> 'cabcd'.substringp('ca') AC> 1 >> 'cabcd'.substringp('xx') AC> 0 AC> substringp would probably be the wrong name for it, though. Hmm, "hassubstring()"? not-implying-a-vote-on-adding-such-a-method-ly y'rs, -Barry From jeremy@zope.com Mon Aug 27 15:27:17 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 27 Aug 2001 10:27:17 -0400 (EDT) Subject: [Python-Dev] Where are locals initialized? In-Reply-To: <15241.43516.529415.828023@beluga.mojam.com> References: <15241.43516.529415.828023@beluga.mojam.com> Message-ID: <15242.22725.241556.803488@slothrop.digicool.com> It's in PyEval_EvalCodeEx(). Look for the SETLOCAL() macro. The code in lines 2305 to 2494 is all dealing with function arguments. Jeremy From nas@python.ca Mon Aug 27 17:23:49 2001 From: nas@python.ca (Neil Schemenauer) Date: Mon, 27 Aug 2001 09:23:49 -0700 Subject: [Python-Dev] python generator design bug? Message-ID: <20010827092349.A23538@glacier.arctrix.com> Eric Kidd thinks so: http://www.advogato.org/person/emk/ Here's an excerpt: [...] there's a subtle bug in the Python design. Consider tree traversal: def inorder(t): if (t.left != None): for (node in inorder(t.left)): yield node yield t if (t.right != None): for (node in inorder(t.right)): yield node If you study this carefully, you'll see that (unless the optimizer intervenes), Python has turned a perfectly good O(N) tree traversal into an O(N log N) traversal. Thoughts? Neil From guido@python.org Mon Aug 27 17:38:15 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 27 Aug 2001 12:38:15 -0400 Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences In-Reply-To: Your message of "Sun, 26 Aug 2001 04:01:01 PDT." References: Message-ID: <200108271638.MAA25363@cj20424-a.reston1.va.home.com> > On Sun, 26 Aug 2001, Eric S. Raymond wrote: > > You're right. But since a string can't be an element of a string, the > > case I'm really interested in would still work. > > It's arguable. A character can indeed be an element of a string, and > in Python characters are one-character strings. So this violates > > x in s <=> there exists i such that s[i] == x > > You could argue that strings should have special behaviour for "in" > to support the common case of finding a substring; then you would have > to argue on grounds of "practicality beats purity". > > I might support the argument on grounds of practicality beating purity, > but i'd have to think carefully about violating the above definition. > > I can see that "substring in string" would be very convenient, but then > > if char in '0123456789': ... > > would suddenly have a very different meaning -- it would succeed for > char = '12'. We *could* allow this for strings, but not for other sequences. For strings, type(s[0]) == type(s), and this avoids the ambiguity. The fact that currently ``noncharacter in string'' raises an exception makes it relatively painess to add semantics for it. But I'll have to think about why I ruled it out originall. I'll track down Eric here at LinuxWorld and discuss it with him, if we find time. --Guido van Rossum (home page: http://www.python.org/~guido/) From esr@thyrsus.com Mon Aug 27 19:01:34 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 27 Aug 2001 14:01:34 -0400 Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences In-Reply-To: <15242.20551.521357.800605@anthem.wooz.org>; from barry@zope.com on Mon, Aug 27, 2001 at 09:51:03AM -0400 References: <15241.47474.461384.774856@anthem.wooz.org> <15242.20551.521357.800605@anthem.wooz.org> Message-ID: <20010827140134.A7687@thyrsus.com> Barry A. Warsaw : > AC> substringp would probably be the wrong name for it, though. > > Hmm, "hassubstring()"? How about "contains"? E.g. if a.contains(b): # do stuff -- Eric S. Raymond (Those) who are trying to read the Second Amendment out of the Constitution by claiming it's not an individual right (are) courting disaster by encouraging others to use the same means to eliminate portions of the Constitution they don't like. -- Alan Dershowitz, Harvard Law School From esr@thyrsus.com Mon Aug 27 19:08:10 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 27 Aug 2001 14:08:10 -0400 Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences In-Reply-To: <200108271638.MAA25363@cj20424-a.reston1.va.home.com>; from guido@python.org on Mon, Aug 27, 2001 at 12:38:15PM -0400 References: <200108271638.MAA25363@cj20424-a.reston1.va.home.com> Message-ID: <20010827140810.B7687@thyrsus.com> Guido van Rossum : > We *could* allow this for strings, but not for other sequences. > For strings, type(s[0]) == type(s), and this avoids the ambiguity. > The fact that currently ``noncharacter in string'' raises an exception > makes it relatively painess to add semantics for it. But I'll have to > think about why I ruled it out originall. I'll track down Eric here > at LinuxWorld and discuss it with him, if we find time. Alas, due to a massive screwup by IDG that I only discovered two days ago, I won't be at LinuxWorld this time. Which bites; I was really looking forward to going. -- Eric S. Raymond "The bearing of arms is the essential medium through which the individual asserts both his social power and his participation in politics as a responsible moral being..." -- J.G.A. Pocock, describing the beliefs of the founders of the U.S. From James_Althoff@i2.com Mon Aug 27 19:17:25 2001 From: James_Althoff@i2.com (James_Althoff@i2.com) Date: Mon, 27 Aug 2001 11:17:25 -0700 Subject: [Python-Dev] copy, len and the like as 'object' methods? Message-ID: Tim Peters wrote: >Say what?! What kind of stupid-ass language spells the len() function as if >it were a method? That's carrying consistency to an irrational extreme -- Well, only those stupid-ass, irrational, and extremist languages that include classes and methods as an integral and uniform part of the language from the very beginning, I guess. The "stupid-ass, irrational, extremist" side is merely saying that mylist.append(x) mylist.length() mylist.extend(y) mylist.length() is more sensible (easier for new-to-python programmers to discover, more consistent, more mainstream) than mylist.append(x) len(mylist) mylist.extend(y) len(mylist) and that len(mylist) as syntactic sugar for mylist.__len__() is sugar that adds no value compared to just doing the straightforward thing of writing mylist.length() or mylist.len(). But you all clearly like spelling it len(mylist) and it's already there, so ... whatever. Jim From tim.one@home.com Mon Aug 27 20:03:00 2001 From: tim.one@home.com (Tim Peters) Date: Mon, 27 Aug 2001 15:03:00 -0400 Subject: [Python-Dev] python generator design bug? In-Reply-To: <20010827092349.A23538@glacier.arctrix.com> Message-ID: [Neil Schemenauer] > Eric Kidd thinks so: > > http://www.advogato.org/person/emk/ > > Here's an excerpt: > > [...] there's a subtle bug in the Python design. Consider tree > traversal: > > def inorder(t): > if (t.left != None): > for (node in inorder(t.left)): > yield node > yield t > if (t.right != None): > for (node in inorder(t.right)): > yield node > > If you study this carefully, you'll see that (unless the optimizer > intervenes), Python has turned a perfectly good O(N) tree traversal > into an O(N log N) traversal. > > Thoughts? Replied to this before on c.l.py (but to David Eppstein): http://aspn.activestate.com/ASPN/Mail/Message/624273 enumerate-a-btree-and-"log(n)"-is-1-ly y'rs - tim From tim.one@home.com Mon Aug 27 21:00:36 2001 From: tim.one@home.com (Tim Peters) Date: Mon, 27 Aug 2001 16:00:36 -0400 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: Message-ID: [Tim, giving his "" key a rest] > Say what?! What kind of stupid-ass language spells the len() function as > if it were a method? That's carrying consistency to an irrational > extreme -- [James_Althoff@i2.com] > Well, only those stupid-ass, irrational, and extremist languages that > include classes and methods as an integral and uniform part of > the language from the very beginning, I guess. I was more than half joking, but not entirely. > The "stupid-ass, irrational, extremist" side is merely saying that > > mylist.append(x) > mylist.length() > mylist.extend(y) > mylist.length() > > is more sensible (easier for new-to-python programmers to discover, > more consistent, more mainstream) Of course that's what they're saying, although "mainstream" is debatable: pure-OO languages aren't exactly mainstream, and Python isn't a pure-OO language. It doesn't follow that Python is mainstream, but neither does it follow that ".length()" isn't an eyesore . > But you all clearly like spelling it len(mylist) and it's already > there, so > ... whatever. I wouldn't want to write math.pi.sin() either -- functional notation for pure (referentially transparent; side-effect free) functions has a long and fine tradition, in and out of computer languages, and I *think* "get the length of some list", not "list, of all the things I may ask you for, in this case I want your length". It simply reads better to me as a function. Guido said he'd do it the other way had he to do it over, so I'll have to rest on "it's already there". From jack@oratrix.nl Mon Aug 27 21:09:28 2001 From: jack@oratrix.nl (Jack Jansen) Date: Mon, 27 Aug 2001 22:09:28 +0200 Subject: [Python-Dev] CVS tags, unix Python and MacPython Message-ID: <20010827200933.7A2FCB5E00@oratrix.oratrix.nl> Folks, I need some good ideas on how to organize tag names for MacPython releases. When the MacPython repository was still separate things were fairly simple: when I did a MacPython release what ended up in the source tree was simply the (correctly tagged) Mac subtree from that repository and whatever happened to be current in the main repository (usually a couple of weeks newer than the corresponding unix release), with one or two files modified. When the repositories were merged I switched over to using the same tags as the unix release used. This has sort-of worked for the 2.1.1 release, although with some grumblings of the people doing the unix 2.1.1 release. Things will get worse with the 2.2a2 release, as there are a couple of things that I definitely want in there that aren't finished yet, so it'll be at least another week until I'm done, and moreover I'll have to be making a few (minor) mods to files outside the Mac subtree. This problem will continue for as long as MacPython stays alive (unless everyone is willing to delay all distributions until I'm done with MacPython too:-). I can see 3 possible solutions, I'd like to hear what people think is the best course of action (and what the powers-that-be sanction): - I can move the "r22a2" tag for the specific files. I don't think I like this solution (despite its simplicity) because the files in question are used by the OSX unix build too. - I can add an "r22a2-mac" tag all over the tree. - I can add an "r22a2" tag to the Mac subtree only, check out the whole r22a2 tagged Python tree and manually check out those files for which I need newer versions. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.cwi.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From tim.one@home.com Mon Aug 27 23:10:53 2001 From: tim.one@home.com (Tim Peters) Date: Mon, 27 Aug 2001 18:10:53 -0400 Subject: [Python-Dev] PEP 264 In-Reply-To: <2m4rqu4qak.fsf@starship.python.net> Message-ID: [Michael Hudson] > OK, I see. In this spirit, wouldn't it be better to define the > PyCF_MASK and PyCF_MASK_OBSOLETE flags in the same place as the CO_* > flags they're made up of? -0. I don't see that it matters. Wouldn't object if someone else wanted to, but they appear to make most sense where they are, next to the decl of the PyCompilerFlags struct (the "CF" in PyCF_MASK refers to Compiler Flags, not Code Flags -- these bits are doing double-duty in 2.2a2). From guido@python.org Tue Aug 28 18:03:08 2001 From: guido@python.org (Guido van Rossum) Date: Tue, 28 Aug 2001 13:03:08 -0400 Subject: [Python-Dev] CVS tags, unix Python and MacPython In-Reply-To: Your message of "Mon, 27 Aug 2001 22:09:28 +0200." <20010827200933.7A2FCB5E00@oratrix.oratrix.nl> References: <20010827200933.7A2FCB5E00@oratrix.oratrix.nl> Message-ID: <200108281703.NAA01931@cj20424-a.reston1.va.home.com> > - I can move the "r22a2" tag for the specific files. I don't think I > like this solution (despite its simplicity) because the files in > question are used by the OSX unix build too. That would be bad. > - I can add an "r22a2-mac" tag all over the tree. This sounds good to me. > - I can add an "r22a2" tag to the Mac subtree only, check out the > whole r22a2 tagged Python tree and manually check out those files for > which I need newer versions. It's up to you. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@scottb.demon.co.uk Tue Aug 28 19:41:53 2001 From: barry@scottb.demon.co.uk (Barry Scott) Date: Tue, 28 Aug 2001 19:41:53 +0100 Subject: [Python-Dev] copy, len and the like as 'object' methods? In-Reply-To: Message-ID: <000601c12ff1$1b0571d0$060210ac@private> > Tim said: > I *think* "get the > length of some list", not "list, of all the things I may ask you for, in > this case I want your length". It simply reads better to me as a function. length of list and type len(list) read the file and type file.read() maybe thinking... list how long are you list.length() file I want to read you file.read() BArry From tim.one@home.com Tue Aug 28 20:01:47 2001 From: tim.one@home.com (Tim Peters) Date: Tue, 28 Aug 2001 15:01:47 -0400 Subject: [Python-Dev] FW: PyInline Released: Put C source code directly "inline" with your Python! Message-ID: FYI. I haven't checked it out. -----Original Message----- From: python-list-admin@python.org [mailto:python-list-admin@python.org]On Behalf Of Ken Simpson Sent: Tuesday, August 28, 2001 2:06 PM To: python-list@python.org Subject: PyInline Released: Put C source code directly "inline" with your Python! The PyInline module allows you to put source code from other programming languages directly "inline" in a Python script or module. The code is automatically compiled as needed and then loaded for immediate access from Python. In short, PyInline lets you do things like this: from PyInline import C import __main__ # Let's put some C code "inline" with our Python: x = C.Builder(code=""" #include void ja(char *str) { printf("Just another %s hacker\n", str); } """, targetmodule=__main__) # Add C methods to the __main__ module. x.build() ja("Inline") --- PyInline is the Python equivalent of Brian Ingerson's Inline module for Perl (http://inline.perl.org), which was awarded the "Best Module" award at this year's Perl Conference. More information about PyInline can be found at the PyInline home page on SourceForge: http://pyinline.sourceforge.net To discuss PyInline further, please join the pyinline-discuss mailing list at http://lists.sourceforge.net/lists/listinfo/pyinline-discuss -- http://mail.python.org/mailman/listinfo/python-list From barry@zope.com Tue Aug 28 21:52:30 2001 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 28 Aug 2001 16:52:30 -0400 Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences References: <15241.47474.461384.774856@anthem.wooz.org> <15242.20551.521357.800605@anthem.wooz.org> <20010827140134.A7687@thyrsus.com> Message-ID: <15244.1166.351735.428963@anthem.wooz.org> >>>>> "ESR" == Eric S Raymond writes: >> substringp would probably be the wrong name for it, though. >> Hmm, "hassubstring()"? ESR> How about "contains"? E.g. | if a.contains(b): | # do stuff Heh, and from there it's a small leap to a.__contains__(b), a.k.a. "b in a". Sneaky Eric, very sneaky. :) -Barry From esr@thyrsus.com Tue Aug 28 22:20:15 2001 From: esr@thyrsus.com (Eric S. Raymond) Date: Tue, 28 Aug 2001 17:20:15 -0400 Subject: [Python-Dev] Re: Generalizing "in" to pairs of sequences In-Reply-To: <15244.1166.351735.428963@anthem.wooz.org>; from barry@zope.com on Tue, Aug 28, 2001 at 04:52:30PM -0400 References: <15241.47474.461384.774856@anthem.wooz.org> <15242.20551.521357.800605@anthem.wooz.org> <20010827140134.A7687@thyrsus.com> <15244.1166.351735.428963@anthem.wooz.org> Message-ID: <20010828172015.C7227@thyrsus.com> Barry A. Warsaw : > ESR> How about "contains"? E.g. > > | if a.contains(b): > | # do stuff > > Heh, and from there it's a small leap to a.__contains__(b), a.k.a. > "b in a". > > Sneaky Eric, very sneaky. :) You know, that didn't consciously occur to me. Which only goes to show that my unconscious mind is even *more* evil...BWAHAHAHAHAA! -- Eric S. Raymond Society in every state is a blessing, but government even in its best state is but a necessary evil; in its worst state an intolerable one; for when we suffer, or are exposed to the same miseries *by a government*, which we might expect in a country *without government*, our calamities is heightened by reflecting that we furnish the means by which we suffer." -- Thomas Paine From m.favas@per.dem.csiro.au Wed Aug 29 01:59:29 2001 From: m.favas@per.dem.csiro.au (Mark Favas) Date: Wed, 29 Aug 2001 08:59:29 +0800 Subject: [Python-Dev] Change from "config.h" to "pyconfig.h" may be a ticking #include bomb Message-ID: <3B8C3E71.A9A4C0BA@per.dem.csiro.au> Some little while ago, Python 2.2's "configure" started to produce pyconfig.h rather than config.h (a good thing). "make install" installs this file into /usr/local/include/python2.2 where it can be used by extensions, etc. However, any config.h file produced and installed before the change to pyconfig.h remains in /usr/local/include/python2.2. I've also been building the XML extension. Only when building this extension on a brand new Python 2.2 platform did I find that XML's extension/expat/lib/xmlparse.c #include's rather than (probably not the only extension that does this). Old 2.2 installations found the old config.h file and compiled without error, although they should have been using pyconfig.h. To pick up this sort of bug, perhaps "make install" should attempt to remove config.h if it exists in the target include directory? -- Mark Favas - m.favas@per.dem.csiro.au CSIRO, Private Bag No 5, Wembley, Western Australia 6913, AUSTRALIA From jack@oratrix.nl Wed Aug 29 13:29:51 2001 From: jack@oratrix.nl (Jack Jansen) Date: Wed, 29 Aug 2001 14:29:51 +0200 Subject: [Python-Dev] test_threaded_import Message-ID: <20010829122951.A22DC303181@snelboot.oratrix.nl> I spent the last couple of days hunting for the bug in MacPython threaded import, but it turned out the bug is in the test harness. (I did find another serious bug in the process, so the time wasn't wasted, but still) The problem is that test_threaded_import cannot run inside an "import". It knows this, and so does regrtest, so they run the test after the import by calling test_threaded_import.test_main(), which does the actual test. All good and fine if you run regrtest as a main script from the commandline, but of course it breaks if you run autotest, or if you "import regrtest" in an interactive window, because in these cases the import in the main thread already holds the lock. I can simply add an "if not verbose", as in test_socketserver, but that isn't really the problem. Does anyone know a better solution? -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.cwi.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From tim.one@home.com Wed Aug 29 20:03:12 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 29 Aug 2001 15:03:12 -0400 Subject: [Python-Dev] test_threaded_import In-Reply-To: <20010829122951.A22DC303181@snelboot.oratrix.nl> Message-ID: [Jack Jansen] > I spent the last couple of days hunting for the bug in MacPython > threaded import, but it turned out the bug is in the test harness. > (I did find another serious bug in the process, so the time wasn't > wasted, but still) Hmm! You originally said this test hung "about 30% of the time", and didn't respond when I asked how you ran it. Given the way you are running it, it should have hung 100% of the time. So something here is still muddy. > The problem is that test_threaded_import cannot run inside an > "import". It knows this, and so does regrtest, so they run the test > after the import by calling test_threaded_import.test_main(), which > does the actual test. > > All good and fine if you run regrtest as a main script from the > commandline, Or if you run test_threaded_import directly (leaving regrtest out of it). > but of course it breaks if you run autotest, So it does! I've never done that. > or if you "import regrtest" in an interactive window, Are you sure about that? Works fine for me, and should work fine. > because in these cases the import in the main thread > already holds the lock. It shouldn't hurt anything to import regrtest -- it's not until regrtest.main() gets invoked that a problem can occur. > I can simply add an "if not verbose", as in test_socketserver, > but that isn't really the problem. Does anyone know a better > solution? I don't like that either -- test_socketserver almost never gets run on my box simply because it's an oddball. For the same reason I'm sure the test_exceptions part of test_math never gets run on anyone's box. Assuming you were suffering an illusion when saying "import regrest in an interactive window" doesn't work, I'd be content to special-case the snot out of autotest, like so: Index: test_threaded_import.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_threaded_import.py,v retrieving revision 1.2 diff -c -r1.2 test_threaded_import.py *** test_threaded_import.py 2001/05/22 18:28:25 1.2 --- test_threaded_import.py 2001/08/29 18:53:38 *************** *** 6,12 **** # randrange, and then Python hangs. import thread ! from test_support import verbose critical_section = thread.allocate_lock() done = thread.allocate_lock() --- 6,12 ---- # randrange, and then Python hangs. import thread ! from test_support import verbose, TestSkipped critical_section = thread.allocate_lock() done = thread.allocate_lock() *************** *** 32,37 **** --- 32,41 ---- def test_main(): # magic name! see above global N, done + import sys + for modname in sys.modules: + if modname.find('autotest') >= 0: + raise TestSkipped("can't run from autotest") done.acquire() for N in (20, 50) * 3: if verbose: From jack@oratrix.nl Wed Aug 29 21:16:43 2001 From: jack@oratrix.nl (Jack Jansen) Date: Wed, 29 Aug 2001 22:16:43 +0200 Subject: [Python-Dev] test_threaded_import In-Reply-To: Message by "Tim Peters" , Wed, 29 Aug 2001 15:03:12 -0400 , Message-ID: <20010829201648.9ED5115F546@oratrix.oratrix.nl> Recently, "Tim Peters" said: > Hmm! You originally said this test hung "about 30% of the time", and didn't > respond when I asked how you ran it. Given the way you are running it, it > should have hung 100% of the time. So something here is still muddy. Elementary, my dear Watson. I do a "make install", which runs the regression tests twice, succeeding both times. Then I do "python" and "import autotest", at which point it fails, thereby failing approximately 30% of the time. Of course, not being Sherlock Jansen, I didn't realise autotest!=regrtest for this:-) > > or if you "import regrtest" in an interactive window, > > Are you sure about that? Works fine for me, and should work fine. Sherlock Peters, you're absolutely right, of course:-) The regrtest ran afoul of the small stack space for threads in MacPython (for which I checked in a fix this afternoon). I didn't try it again when that bug was fixed. I'll apply your patch, which seems to fix the problem fine for the time being. Still, there's three issues that I think need to be solved at some point: - the fact that you (or some other thread) are holding the import lock is one of the few Python state bits that are not open to introspection. - There may be denial of service attacks possible with this if a 16 year old scriptkiddy can think of a way to exploit it in a restricted interpreter. - The whole idea that a process deadlocked on semaphore locks is completely uninterruptible also smells bad. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.cwi.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From tim.one@home.com Thu Aug 30 00:42:59 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 29 Aug 2001 19:42:59 -0400 Subject: [Python-Dev] test_threaded_import In-Reply-To: <20010829201648.9ED5115F546@oratrix.oratrix.nl> Message-ID: [Jack Jansen] > ... > Elementary, my dear Watson. I do a "make install", which runs the > regression tests twice, succeeding both times. Then I do "python" and > "import autotest", at which point it fails, thereby failing > approximately 30% of the time. Of course, not being Sherlock Jansen, I > didn't realise autotest!=regrtest for this:-) Ah, a signficant digits error. Had you said "approximately 33% of the time", or even "approximately 3e1%", I would have guessed instantly . > ... > I'll apply your patch, which seems to fix the problem fine for the > time being. Still, there's three issues that I think need to be solved > at some point: > - the fact that you (or some other thread) are holding the import lock > is one of the few Python state bits that are not open to > introspection. The patch below adds introspection, sufficient to avoid special-casing autotest. Guido? I'd upload the patch to SF instead, but, as you've (Jack) noted, that's a black hole <0.9 wink>. > - There may be denial of service attacks possible with this if a 16 > year old scriptkiddy can think of a way to exploit it in a > restricted interpreter. Someone else will have to worry about that; in a world where a 20-character regexp can take centuries to match, I don't take DOS from a general-purpose programming language seriously. > - The whole idea that a process deadlocked on semaphore locks is > completely uninterruptible also smells bad. That's threads for you: start a thread, risk a deadlock. There's no portable way to interrupt a thread waiting on a lock, and, e.g., no *safe* way at all on Windows short of killing the whole process. Index: Python/import.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/import.c,v retrieving revision 2.185 diff -c -r2.185 import.c *** Python/import.c 2001/08/13 23:05:44 2.185 --- Python/import.c 2001/08/29 23:25:28 *************** *** 138,143 **** --- 138,151 ---- static long import_lock_thread = -1; static int import_lock_level = 0; + static PyObject * + imp_lock_held(PyObject *self, PyObject *args) + { + if (!PyArg_ParseTuple(args, ":lock_held")) + return NULL; + return PyInt_FromLong(import_lock_thread != -1); + } + static void lock_import(void) { *************** *** 179,184 **** --- 187,200 ---- #define lock_import() #define unlock_import() + static PyObject * + imp_lock_held(PyObject *self, PyObject *args) + { + if (!PyArg_ParseTuple(args, ":lock_held")) + return NULL; + return PyInt_FromLong(0); + } + #endif /* Helper for sys */ *************** *** 2339,2350 **** --- 2355,2373 ---- The module name must include the full package name, if any.\ "; + static char doc_lock_held[] = "\ + lock_held() -> 0 or 1\n\ + Return 1 if the import lock is currently held.\n\ + On platforms without threads, return 0.\ + "; + static PyMethodDef imp_methods[] = { {"find_module", imp_find_module, 1, doc_find_module}, {"get_magic", imp_get_magic, 1, doc_get_magic}, {"get_suffixes", imp_get_suffixes, 1, doc_get_suffixes}, {"load_module", imp_load_module, 1, doc_load_module}, {"new_module", imp_new_module, 1, doc_new_module}, + {"lock_held", imp_lock_held, 1, doc_lock_held}, /* The rest are obsolete */ {"get_frozen_object", imp_get_frozen_object, 1}, {"init_builtin", imp_init_builtin, 1}, From guido@python.org Thu Aug 30 16:01:24 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 30 Aug 2001 11:01:24 -0400 Subject: [Python-Dev] canned response added Message-ID: <200108301501.f7UF1OQ18292@odiug.digicool.com> At Jack's suggestion, I've used a canned response "Missing upload" to the Bugs and Patches trackers at SourceForge. This should save us a bit of time dealing with this annoying SF bug. --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik@pythonware.com Thu Aug 30 18:15:07 2001 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 30 Aug 2001 19:15:07 +0200 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Modules _sre.c,2.63,2.64 References: Message-ID: <007601c13177$539e0a20$42fb42d5@hagrid> sjoerd wrote: > > Modified Files: > _sre.c > Log Message: > Removed unreachable return to silence SGI compiler. > > ! /* shouldn't end up here */ > ! return SRE_ERROR_ILLEGAL; > } > > --- 1141,1145 ---- > } > > ! /* can't end up here */ > } I hate stuff like this: that line was there to make sure *I* don't mess up when developing SRE, not to deal with potentially broken compilers or misfired electrons. isn't there any way to tell the SGI compiler to stop whining about this? From tim@zope.com Thu Aug 30 19:50:15 2001 From: tim@zope.com (Tim Peters) Date: Thu, 30 Aug 2001 14:50:15 -0400 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Modules _sre.c,2.63,2.64 In-Reply-To: <007601c13177$539e0a20$42fb42d5@hagrid> Message-ID: >> _sre.c >> Log Message: >> Removed unreachable return to silence SGI compiler. >> >> ! /* shouldn't end up here */ >> ! return SRE_ERROR_ILLEGAL; >> } >> >> --- 1141,1145 ---- >> } >> >> ! /* can't end up here */ >> } [Fredrik Lundh] > I hate stuff like this: that line was there to make sure *I* don't > mess up when developing SRE, not to deal with potentially broken > compilers or misfired electrons. > > isn't there any way to tell the SGI compiler to stop whining about > this? I've had better luck x-platform with, e.g., assert(!"can't get here"); /* !string is always false */ In non-debug builds, it vanishes so there's nothing for the compiler to gripe about; while in debug builds most compilers don't do enough flow analysis to realize there *is* something to gripe about. Overall, it would be more useful if SGI compilers in particular produced: C00666: Warning: -O may produce buggy code. after every executable line . From skip@pobox.com (Skip Montanaro) Fri Aug 31 06:51:35 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 31 Aug 2001 00:51:35 -0500 Subject: [Python-Dev] timeoutsocket.py Message-ID: <15247.9703.69434.567507@beluga.mojam.com> I just submitted a feature request to add Tim O'Malley's timeoutsocket.py module to the Python distribution. Please have a look at http://sourceforge.net/tracker/index.php?func=detail&aid=457114&group_id=5470&atid=105470 and comment if you like. Skip From sjoerd.mullender@oratrix.com Fri Aug 31 10:08:19 2001 From: sjoerd.mullender@oratrix.com (Sjoerd Mullender) Date: Fri, 31 Aug 2001 11:08:19 +0200 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Modules _sre.c,2.63,2.64 In-Reply-To: Your message of Thu, 30 Aug 2001 14:50:15 -0400. References: Message-ID: <20010831090820.9BFC7301382@bireme.oratrix.nl> Well, there is the possibility to add -woff 1110 to the compiler command line. However, I don't agree that this is whining and I wouldn't like to see any warnings suppressed like this. Warnings signal a potential bug, any so does this one. If the flow of control did reach this particular line, the compiler would also give a warning, namely that the function has both return value and return void (or words to that effect). I use the compiler in non-debug mode (-g option, no -O option), and I do get these warnings, so some compilers do do enough flow control in non-debug mode. On Thu, Aug 30 2001 "Tim Peters" wrote: > >> _sre.c > >> Log Message: > >> Removed unreachable return to silence SGI compiler. > >> > >> ! /* shouldn't end up here */ > >> ! return SRE_ERROR_ILLEGAL; > >> } > >> > >> --- 1141,1145 ---- > >> } > >> > >> ! /* can't end up here */ > >> } > > [Fredrik Lundh] > > I hate stuff like this: that line was there to make sure *I* don't > > mess up when developing SRE, not to deal with potentially broken > > compilers or misfired electrons. > > > > isn't there any way to tell the SGI compiler to stop whining about > > this? > > I've had better luck x-platform with, e.g., > > assert(!"can't get here"); /* !string is always false */ > > In non-debug builds, it vanishes so there's nothing for the compiler to > gripe about; while in debug builds most compilers don't do enough flow > analysis to realize there *is* something to gripe about. > > Overall, it would be more useful if SGI compilers in particular produced: > > C00666: Warning: -O may produce buggy code. > > after every executable line . > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > -- Sjoerd Mullender From gward@python.net Fri Aug 31 14:15:07 2001 From: gward@python.net (Greg Ward) Date: Fri, 31 Aug 2001 09:15:07 -0400 Subject: [Python-Dev] Re: Negative index in insert() In-Reply-To: ; from ping@lfw.org on Sun, Aug 26, 2001 at 02:39:55AM -0700 References: Message-ID: <20010831091507.A15968@gerg.ca> On 26 August 2001, Ka-Ping Yee said: > If this is fixed, i think we should do a complete job of it and > make sure exceptions are raised when the index is out of range. [...] > This has the potential for breaking code, and so the transition > should probably be done with a minor-version's worth of warning, > but i think if we're going to change it we should make it right. By "minor version" do you mean 2.2 -> 2.3, or 2.2 -> 2.2.1? I presume the former -- we're not allowed to make incompatible changes on point releases anymore! Of course, this implies a f***** statement. (I'm not going to say the word, Tim said the first one who does dies...) Greg -- Greg Ward - just another /P(erl|ython)/ hacker gward@python.net http://starship.python.net/~gward/ If you and a friend are being chased by a lion, it is not necessary to outrun the lion. It is only necessary to outrun your friend. From gward@python.net Fri Aug 31 14:35:19 2001 From: gward@python.net (Greg Ward) Date: Fri, 31 Aug 2001 09:35:19 -0400 Subject: [Python-Dev] PEP 250 Message-ID: <20010831093519.A16031@gerg.ca> Thomas -- have you been following the saga of patch #449054, part of the PEP 250 implementation. If I understand things correctly, this patch only partly implements PEP 250 -- the rest is waiting on a change to bdist_wininst. What's the status of this change? Going to make it in? Please see http://sourceforge.net/tracker/?func=detail&atid=305470&aid=449054&group_id=5470 if you haven't already... Greg -- Greg Ward - geek-at-large gward@python.net http://starship.python.net/~gward/ Just because you're paranoid doesn't mean they *aren't* out to get you. From akuchlin@mems-exchange.org Fri Aug 31 15:48:43 2001 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 31 Aug 2001 10:48:43 -0400 Subject: [Python-Dev] CVS: python/dist/src/Modules _sre.c,2.63,2.64 In-Reply-To: <20010831090820.9BFC7301382@bireme.oratrix.nl>; from sjoerd.mullender@oratrix.com on Fri, Aug 31, 2001 at 11:08:19AM +0200 References: <20010831090820.9BFC7301382@bireme.oratrix.nl> Message-ID: <20010831104842.D30221@ute.mems-exchange.org> On Fri, Aug 31, 2001 at 11:08:19AM +0200, Sjoerd Mullender wrote: >any so does this one. If the flow of control did reach this >particular line, the compiler would also give a warning, namely that >the function has both return value and return void (or words to that >effect). Your compiler might, but I don't know if Fredrik's will, and for the SRE code Fredrik's compiler is more important. Perhaps Tim's suggested assert() can fix things so that both of you are happy. --amk From thomas.heller@ion-tof.com Fri Aug 31 15:53:50 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 31 Aug 2001 16:53:50 +0200 Subject: [Python-Dev] PEP 250 References: <20010831093519.A16031@gerg.ca> Message-ID: <017a01c1322c$bf558fd0$e000a8c0@thomasnotebook> From: "Greg Ward" > Thomas -- > > have you been following the saga of patch #449054, part of the PEP 250 > implementation. If I understand things correctly, this patch only > partly implements PEP 250 -- the rest is waiting on a change to > bdist_wininst. What's the status of this change? Going to make it in? Sure, will do it before 2.2a3. That's Sep 19, right? Looking at the patch again, it seems sufficient to install into lib/site-packages if sys.version >= 2.2. No fiddling with loading python at runtime, importing distutils.sysconfig, and so on in bdist_wininst. Thomas From guido@python.org Fri Aug 31 20:28:25 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 31 Aug 2001 15:28:25 -0400 Subject: [Python-Dev] Would it make sense to issue an extra alpha next week? Message-ID: <200108311928.PAA21836@cj20424-a.reston1.va.home.com> When I was updating the NEWS file this morning, I realized that we've already added a truckload of nifty stuff in the 9 days since 2.2a2 was released: int overflows return longs, classic division warnings, subclassing builtins, super, getset, two fixes to float literals, the new GC API, and PyString_FromFormat[V]. I expect that I'll be concentrating on documentation for the type/class unification next. While writing the first piece of documentation, I realized that, sadly, several more advanced things aren't available in 2.2a2 yet. The next alpha is planned for Sept. 19, almost three weeks off still. Does anybody object against the release of an extra release, 2.2a3, around Sept. 5? We could do 2.2a4 on Sept. 19, or a week later if the schedule gets too crowded. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Fri Aug 31 20:29:02 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 31 Aug 2001 15:29:02 -0400 Subject: [Python-Dev] -Dwarn, long->double overflow (was RE: [Python-checkins] CVS: python/dist/src/Misc NEWS,1.219,1.220) In-Reply-To: Message-ID: [Guido] > + + A new command line option, -D, is added to control run-time > + warnings for the use of classic division. > ... > + Using -Dwarn issues a run-time warning about all uses of classic > + division for int, long, float and complex arguments. I'm unclear on why we warn about classic division when a float or complex is involved: C:\Code\python\PCbuild>python -Dwarn Python 2.2a2+ (#22, Aug 31 2001, 14:36:57) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> 2./3. __main__:1: DeprecationWarning: classic float division 0.66666666666666663 >>> Given that this is going to return the same result even in Python 3.0, what is it warning me about? That is, as an end user, I'm being warned about behavior I can't do anything about, and behavior that Python isn't going to change anyway. Different issue: I would like to add OverflowError when coercion of long to float yields a senseless result. PEP 238 mentions this as a possibility, and >>> from __future__ import division >>> x = 1L << 3000 >>> x/(2*x) -1.#IND >>> really sucks. For that matter, >>> float(1L << 3000) 1.#INF >>> has always sucked; future division just makes it suck harder . Any objections to raising OverflowError here? Note this is a bit painful for a bogus reason: PyFloat_AsDouble returns a double, and nothing in the codebase now checks it for a -1.0 error return (despite that it can already produce one). So half the effort would be repairing code currently ignoring the possibility of error. Third issue: I don't see a *good* reason for the future-division x/(2*x) above not to return 0.5. long_true_divide could be made smart enough to do that (more generally, to return a good float approximation whenever the true result is representable as a float). From guido@python.org Fri Aug 31 20:39:21 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 31 Aug 2001 15:39:21 -0400 Subject: [Python-Dev] Re: -Dwarn, long->double overflow (was RE: [Python-checkins] CVS: python/dist/src/Misc NEWS,1.219,1.220) In-Reply-To: Your message of "Fri, 31 Aug 2001 15:29:02 EDT." References: Message-ID: <200108311939.PAA21883@cj20424-a.reston1.va.home.com> > [Guido] > > + + A new command line option, -D, is added to control run-time > > + warnings for the use of classic division. > > ... > > + Using -Dwarn issues a run-time warning about all uses of classic > > + division for int, long, float and complex arguments. > > I'm unclear on why we warn about classic division when a float or complex is > involved: > > C:\Code\python\PCbuild>python -Dwarn > Python 2.2a2+ (#22, Aug 31 2001, 14:36:57) [MSC 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> 2./3. > __main__:1: DeprecationWarning: classic float division > 0.66666666666666663 > >>> > > Given that this is going to return the same result even in Python 3.0, what > is it warning me about? That is, as an end user, I'm being warned about > behavior I can't do anything about, and behavior that Python isn't going to > change anyway. Sorry, I should have explained the intention. This warning is intended for perusal by a tool (yet to create) that reads the warnings and can automatically fix your code by inserting the proper future division statements and/or replace / by //. The tool parses the source looking for / operators. For each / it sees if it has corresponding warnings, and if so, if the warnings are consistent (only int/long or only float/complex), it does the right thing. If all / operators in a module have at least one warning, it assumes that it has full coverage, and it adds a future division statement to the module. If there are some / operators for which it doesn't have any warnings, it asks the user to create a better test case. For this, it needs to know about float and complex divisions as well. > Different issue: I would like to add OverflowError when coercion of > long to float yields a senseless result. PEP 238 mentions this as a > possibility, and > > >>> from __future__ import division > >>> x = 1L << 3000 > >>> x/(2*x) > -1.#IND > >>> > > really sucks. For that matter, > > >>> float(1L << 3000) > 1.#INF > >>> > > has always sucked; future division just makes it suck harder . > > Any objections to raising OverflowError here? Sounds OK to me. > Note this is a bit painful for a bogus reason: PyFloat_AsDouble returns a > double, and nothing in the codebase now checks it for a -1.0 error return > (despite that it can already produce one). So half the effort would be > repairing code currently ignoring the possibility of error. > > > Third issue: I don't see a *good* reason for the future-division > > x/(2*x) > > above not to return 0.5. long_true_divide could be made smart enough to do > that (more generally, to return a good float approximation whenever the true > result is representable as a float). Sure, but I also don't see a good reason to make this a priority. It's a "don't care" corner of the language to me. --Guido van Rossum (home page: http://www.python.org/~guido/) From gward@python.net Fri Aug 31 21:33:29 2001 From: gward@python.net (Greg Ward) Date: Fri, 31 Aug 2001 16:33:29 -0400 Subject: [Python-Dev] PEP 250 In-Reply-To: <017a01c1322c$bf558fd0$e000a8c0@thomasnotebook>; from thomas.heller@ion-tof.com on Fri, Aug 31, 2001 at 04:53:50PM +0200 References: <20010831093519.A16031@gerg.ca> <017a01c1322c$bf558fd0$e000a8c0@thomasnotebook> Message-ID: <20010831163329.A16507@gerg.ca> On 31 August 2001, Thomas Heller said: > Sure, will do it before 2.2a3. That's Sep 19, right? The sooner the better -- this is a "priority 9" patch! > Looking at the patch again, it seems sufficient to install into > lib/site-packages if sys.version >= 2.2. No fiddling with loading python > at runtime, importing distutils.sysconfig, and so on in bdist_wininst. Sure, whatever. Up to your best judgment; I've never actually looked at the C++ code that's compiled into that big blob of binary data in bdist_wininst.py. The C++ probably wouldn't mean much more to me than the binary blob does. ;-) Greg -- Greg Ward - Unix geek gward@python.net http://starship.python.net/~gward/ My opinions may have changed, but not the fact that I am right. From tim.one@home.com Fri Aug 31 22:40:18 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 31 Aug 2001 17:40:18 -0400 Subject: [Python-Dev] RE: -Dwarn, long->double overflow (was RE: [Python-checkins] CVS: python/dist/src/Misc NEWS,1.219,1.220) In-Reply-To: <200108311939.PAA21883@cj20424-a.reston1.va.home.com> Message-ID: [Tim] >> I'm unclear on why we warn about classic division when a float >> or complex is involved: [Guido] > ... > This warning is intended for perusal by a tool (yet to create) that > reads the warnings and can automatically fix your code by inserting > the proper future division statements and/or replace / by //. Got it now. Thanks! Cool idea, BTW. >> I would like to add OverflowError when coercion of long to float >> yields a senseless result. > Sounds OK to me. Anyone else object? >> Third issue: I don't see a *good* reason for the future-division >> >> x/(2*x) >> >> above not to return 0.5 [when x is a large long]. > Sure, but I also don't see a good reason to make this a priority. > It's a "don't care" corner of the language to me. Then I'll leave it alone, but I can't understand this view (it's not like we can claim we're victims of the platform C or libm here -- nonsense results in such cases are on our heads, and an implementation that settles for total loss of information when 53 good bits are obviously possible is at best careless). From gstein@lyra.org Fri Aug 31 22:54:05 2001 From: gstein@lyra.org (Greg Stein) Date: Fri, 31 Aug 2001 14:54:05 -0700 Subject: [Python-Dev] Would it make sense to issue an extra alpha next week? In-Reply-To: <200108311928.PAA21836@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Aug 31, 2001 at 03:28:25PM -0400 References: <200108311928.PAA21836@cj20424-a.reston1.va.home.com> Message-ID: <20010831145405.F3654@lyra.org> +1 On Fri, Aug 31, 2001 at 03:28:25PM -0400, Guido van Rossum wrote: > When I was updating the NEWS file this morning, I realized that we've > already added a truckload of nifty stuff in the 9 days since 2.2a2 was > released: int overflows return longs, classic division warnings, > subclassing builtins, super, getset, two fixes to float literals, the > new GC API, and PyString_FromFormat[V]. > > I expect that I'll be concentrating on documentation for the > type/class unification next. While writing the first piece of > documentation, I realized that, sadly, several more advanced things > aren't available in 2.2a2 yet. > > The next alpha is planned for Sept. 19, almost three weeks off still. > > Does anybody object against the release of an extra release, 2.2a3, > around Sept. 5? We could do 2.2a4 on Sept. 19, or a week later if the > schedule gets too crowded. > > --Guido van Rossum (home page: http://www.python.org/~guido/) -- Greg Stein, http://www.lyra.org/